connectors

Creates, updates, deletes or gets a connector resource or lists connectors in a region

Overview

Name	connectors
Type	Resource
Description	Resource Type definition for AWS::KafkaConnect::Connector
Id	awscc.kafkaconnect.connectors

Fields

get (all properties)

Name	Datatype	Description
▶capacity	object	Information about the capacity allocated to the connector.
connector_arn	string	Amazon Resource Name for the created Connector.
connector_configuration	object	The configuration for the connector.
connector_description	string	A summary description of the connector.
connector_name	string	The name of the connector.
▶kafka_cluster	object	Details of how to connect to the Kafka cluster.
▶kafka_cluster_client_authentication	object	Details of the client authentication used by the Kafka cluster.
▶kafka_cluster_encryption_in_transit	object	Details of encryption in transit to the Kafka cluster.
kafka_connect_version	string	The version of Kafka Connect. It has to be compatible with both the Kafka cluster's version and the plugins.
▶log_delivery	object	Details of what logs are delivered and where they are delivered.
▶plugins	array	List of plugins to use with the connector.
service_execution_role_arn	string	The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon S3 objects and other external resources.
▶tags	array	A collection of tags associated with a resource
▶worker_configuration	object	The configuration of the workers, which are the processes that run the connector logic.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
connector_arn	string	Amazon Resource Name for the created Connector.
region	string	AWS region.

For more information, see AWS::KafkaConnect::Connector.

Methods

Name	Resource	Accessible by	Required Params
create_resource	connectors	INSERT	Capacity, ConnectorConfiguration, ConnectorName, KafkaConnectVersion, KafkaCluster, KafkaClusterClientAuthentication, KafkaClusterEncryptionInTransit, Plugins, ServiceExecutionRoleArn, region
delete_resource	connectors	DELETE	Identifier, region
update_resource	connectors	UPDATE	Identifier, PatchDocument, region
list_resources	connectors_list_only	SELECT	region
get_resource	connectors	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual connector.

SELECT
  region,
  capacity,
  connector_arn,
  connector_configuration,
  connector_description,
  connector_name,
  kafka_cluster,
  kafka_cluster_client_authentication,
  kafka_cluster_encryption_in_transit,
  kafka_connect_version,
  log_delivery,
  plugins,
  service_execution_role_arn,
  tags,
  worker_configuration
FROM awscc.kafkaconnect.connectors
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ connector_arn }}';

list (identifiers only)

Lists all connectors in a region.

SELECT
  region,
  connector_arn
FROM awscc.kafkaconnect.connectors_list_only
WHERE
  region = '{{ region }}';

INSERT example

Use the following StackQL query and manifest file to create a new connector resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.kafkaconnect.connectors (
  Capacity,
  ConnectorConfiguration,
  ConnectorName,
  KafkaCluster,
  KafkaClusterClientAuthentication,
  KafkaClusterEncryptionInTransit,
  KafkaConnectVersion,
  Plugins,
  ServiceExecutionRoleArn,
  region
)
SELECT
  '{{ capacity }}',
  '{{ connector_configuration }}',
  '{{ connector_name }}',
  '{{ kafka_cluster }}',
  '{{ kafka_cluster_client_authentication }}',
  '{{ kafka_cluster_encryption_in_transit }}',
  '{{ kafka_connect_version }}',
  '{{ plugins }}',
  '{{ service_execution_role_arn }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

All Properties

/*+ create */
INSERT INTO awscc.kafkaconnect.connectors (
  Capacity,
  ConnectorConfiguration,
  ConnectorDescription,
  ConnectorName,
  KafkaCluster,
  KafkaClusterClientAuthentication,
  KafkaClusterEncryptionInTransit,
  KafkaConnectVersion,
  LogDelivery,
  Plugins,
  ServiceExecutionRoleArn,
  Tags,
  WorkerConfiguration,
  region
)
SELECT
  '{{ capacity }}',
  '{{ connector_configuration }}',
  '{{ connector_description }}',
  '{{ connector_name }}',
  '{{ kafka_cluster }}',
  '{{ kafka_cluster_client_authentication }}',
  '{{ kafka_cluster_encryption_in_transit }}',
  '{{ kafka_connect_version }}',
  '{{ log_delivery }}',
  '{{ plugins }}',
  '{{ service_execution_role_arn }}',
  '{{ tags }}',
  '{{ worker_configuration }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Manifest

version: 1
name: stack name
description: stack description
providers:
- aws
globals:
- name: region
  value: '{{ vars.AWS_REGION }}'
resources:
- name: connector
  props:
    - name: capacity
      value:
        auto_scaling:
          max_worker_count: '{{ max_worker_count }}'
          min_worker_count: '{{ min_worker_count }}'
          scale_in_policy:
            cpu_utilization_percentage: '{{ cpu_utilization_percentage }}'
          scale_out_policy:
            cpu_utilization_percentage: '{{ cpu_utilization_percentage }}'
          mcu_count: '{{ mcu_count }}'
        provisioned_capacity:
          mcu_count: '{{ mcu_count }}'
          worker_count: '{{ worker_count }}'
    - name: connector_configuration
      value: {}
    - name: connector_description
      value: '{{ connector_description }}'
    - name: connector_name
      value: '{{ connector_name }}'
    - name: kafka_cluster
      value:
        apache_kafka_cluster:
          bootstrap_servers: '{{ bootstrap_servers }}'
          vpc:
            security_groups:
              - '{{ security_groups[0] }}'
            subnets:
              - '{{ subnets[0] }}'
    - name: kafka_cluster_client_authentication
      value:
        authentication_type: '{{ authentication_type }}'
    - name: kafka_cluster_encryption_in_transit
      value:
        encryption_type: '{{ encryption_type }}'
    - name: kafka_connect_version
      value: '{{ kafka_connect_version }}'
    - name: log_delivery
      value:
        worker_log_delivery:
          cloud_watch_logs:
            enabled: '{{ enabled }}'
            log_group: '{{ log_group }}'
          firehose:
            delivery_stream: '{{ delivery_stream }}'
            enabled: '{{ enabled }}'
          s3:
            bucket: '{{ bucket }}'
            enabled: '{{ enabled }}'
            prefix: '{{ prefix }}'
    - name: plugins
      value:
        - custom_plugin:
            name: '{{ name }}'
            description: '{{ description }}'
            content_type: '{{ content_type }}'
            location:
              s3_location:
                bucket_arn: '{{ bucket_arn }}'
                file_key: '{{ file_key }}'
                object_version: '{{ object_version }}'
            tags:
              - key: '{{ key }}'
                value: '{{ value }}'
    - name: service_execution_role_arn
      value: '{{ service_execution_role_arn }}'
    - name: tags
      value:
        - null
    - name: worker_configuration
      value:
        name: '{{ name }}'
        description: '{{ description }}'
        properties_file_content: '{{ properties_file_content }}'
        tags:
          - null

UPDATE example

Use the following StackQL query and manifest file to update a connector resource, using stack-deploy.

/*+ update */
UPDATE awscc.kafkaconnect.connectors
SET PatchDocument = string('{{ {
    "Capacity": capacity,
    "ConnectorConfiguration": connector_configuration,
    "Tags": tags
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ connector_arn }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

DELETE example

/*+ delete */
DELETE FROM awscc.kafkaconnect.connectors
WHERE
  Identifier = '{{ connector_arn }}' AND
  region = '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Additional Parameters

Mutable resources in the Cloud Control provider support additional optional parameters which can be supplied with INSERT, UPDATE, or DELETE operations. These include:

Parameter	Description
ClientToken	A unique identifier to ensure the idempotency of the resource request. This allows the provider to accurately distinguish between retries and new requests. A client token is valid for 36 hours once used. After that, a resource request with the same client token is treated as a new request. If you do not specify a client token, one is generated for inclusion in the request.
RoleArn	The ARN of the IAM role used to perform this resource operation. The role specified must have the permissions required for this operation. If you do not specify a role, a temporary session is created using your AWS user credentials.
TypeVersionId	For private resource types, the type version to use in this resource operation. If you do not specify a resource version, the default version is used.

Permissions

To operate on the connectors resource, the following permissions are required:

Create

kafkaconnect:CreateConnector,
kafkaconnect:DescribeConnector,
kafkaconnect:TagResource,
kafkaconnect:ListTagsForResource,
iam:CreateServiceLinkedRole,
iam:PassRole,
ec2:CreateNetworkInterface,
ec2:DescribeSecurityGroups,
ec2:DescribeSubnets,
ec2:DescribeVpcs,
logs:CreateLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries,
logs:PutResourcePolicy,
logs:DescribeResourcePolicies,
logs:DescribeLogGroups,
s3:GetBucketPolicy,
s3:PutBucketPolicy,
firehose:TagDeliveryStream

Read

kafkaconnect:DescribeConnector,
kafkaconnect:ListTagsForResource

Delete

kafkaconnect:DeleteConnector,
kafkaconnect:DescribeConnector,
logs:DeleteLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries

Update

kafkaconnect:UpdateConnector,
kafkaconnect:DescribeConnector,
kafkaconnect:DescribeConnectorOperation,
kafkaconnect:TagResource,
kafkaconnect:ListTagsForResource,
kafkaconnect:UntagResource,
iam:CreateServiceLinkedRole,
logs:UpdateLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries,
logs:PutResourcePolicy,
logs:DescribeResourcePolicies,
logs:DescribeLogGroups,
s3:GetBucketPolicy,
s3:PutBucketPolicy,
firehose:TagDeliveryStream

List

kafkaconnect:ListConnectors