connectors

Creates, updates, deletes or gets a connector resource or lists connectors in a region

Overview

Name	connectors
Type	Resource
Description	Resource Type definition for AWS::KafkaConnect::Connector
Id	awscc.kafkaconnect.connectors

Fields

get (all properties)

Name	Datatype	Description
▶capacity	object	Information about the capacity allocated to the connector.
connector_arn	string	Amazon Resource Name for the created Connector.
connector_configuration	object	The configuration for the connector.
connector_description	string	A summary description of the connector.
connector_name	string	The name of the connector.
▶kafka_cluster	object	Details of how to connect to the Kafka cluster.
▶kafka_cluster_client_authentication	object	Details of the client authentication used by the Kafka cluster.
▶kafka_cluster_encryption_in_transit	object	Details of encryption in transit to the Kafka cluster.
kafka_connect_version	string	The version of Kafka Connect. It has to be compatible with both the Kafka cluster's version and the plugins.
▶log_delivery	object	Details of what logs are delivered and where they are delivered.
▶plugins	array	List of plugins to use with the connector.
service_execution_role_arn	string	The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon S3 objects and other external resources.
▶tags	array	A collection of tags associated with a resource
▶worker_configuration	object	The configuration of the workers, which are the processes that run the connector logic.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
connector_arn	string	Amazon Resource Name for the created Connector.
region	string	AWS region.

For more information, see AWS::KafkaConnect::Connector.

Methods

Name	Resource	Accessible by	Required Params
create_resource	connectors	INSERT	Capacity, ConnectorConfiguration, ConnectorName, KafkaConnectVersion, KafkaCluster, KafkaClusterClientAuthentication, KafkaClusterEncryptionInTransit, Plugins, ServiceExecutionRoleArn, region
delete_resource	connectors	DELETE	Identifier, region
update_resource	connectors	UPDATE	Identifier, PatchDocument, region
list_resources	connectors_list_only	SELECT	region
get_resource	connectors	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual connector.

SELECT
  region,
  capacity,
  connector_arn,
  connector_configuration,
  connector_description,
  connector_name,
  kafka_cluster,
  kafka_cluster_client_authentication,
  kafka_cluster_encryption_in_transit,
  kafka_connect_version,
  log_delivery,
  plugins,
  service_execution_role_arn,
  tags,
  worker_configuration
FROM awscc.kafkaconnect.connectors
WHERE
  region = 'us-east-1' AND
  Identifier = '{{ connector_arn }}';

list (identifiers only)

Lists all connectors in a region.

SELECT
  region,
  connector_arn
FROM awscc.kafkaconnect.connectors_list_only
WHERE
  region = 'us-east-1';

INSERT example

Use the following StackQL query and manifest file to create a new connector resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.kafkaconnect.connectors (
  Capacity,
  ConnectorConfiguration,
  ConnectorName,
  KafkaCluster,
  KafkaClusterClientAuthentication,
  KafkaClusterEncryptionInTransit,
  KafkaConnectVersion,
  Plugins,
  ServiceExecutionRoleArn,
  region
)
SELECT
  '{{ capacity }}',
  '{{ connector_configuration }}',
  '{{ connector_name }}',
  '{{ kafka_cluster }}',
  '{{ kafka_cluster_client_authentication }}',
  '{{ kafka_cluster_encryption_in_transit }}',
  '{{ kafka_connect_version }}',
  '{{ plugins }}',
  '{{ service_execution_role_arn }}',
  '{{ region }}';

All Properties

/*+ create */
INSERT INTO awscc.kafkaconnect.connectors (
  Capacity,
  ConnectorConfiguration,
  ConnectorDescription,
  ConnectorName,
  KafkaCluster,
  KafkaClusterClientAuthentication,
  KafkaClusterEncryptionInTransit,
  KafkaConnectVersion,
  LogDelivery,
  Plugins,
  ServiceExecutionRoleArn,
  Tags,
  WorkerConfiguration,
  region
)
SELECT
  '{{ capacity }}',
  '{{ connector_configuration }}',
  '{{ connector_description }}',
  '{{ connector_name }}',
  '{{ kafka_cluster }}',
  '{{ kafka_cluster_client_authentication }}',
  '{{ kafka_cluster_encryption_in_transit }}',
  '{{ kafka_connect_version }}',
  '{{ log_delivery }}',
  '{{ plugins }}',
  '{{ service_execution_role_arn }}',
  '{{ tags }}',
  '{{ worker_configuration }}',
  '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: connector
    props:
      - name: capacity
        value:
          auto_scaling:
            max_worker_count: '{{ max_worker_count }}'
            min_worker_count: '{{ min_worker_count }}'
            scale_in_policy:
              cpu_utilization_percentage: '{{ cpu_utilization_percentage }}'
            scale_out_policy:
              cpu_utilization_percentage: '{{ cpu_utilization_percentage }}'
            mcu_count: '{{ mcu_count }}'
          provisioned_capacity:
            mcu_count: '{{ mcu_count }}'
            worker_count: '{{ worker_count }}'
      - name: connector_configuration
        value: {}
      - name: connector_description
        value: '{{ connector_description }}'
      - name: connector_name
        value: '{{ connector_name }}'
      - name: kafka_cluster
        value:
          apache_kafka_cluster:
            bootstrap_servers: '{{ bootstrap_servers }}'
            vpc:
              security_groups:
                - '{{ security_groups[0] }}'
              subnets:
                - '{{ subnets[0] }}'
      - name: kafka_cluster_client_authentication
        value:
          authentication_type: '{{ authentication_type }}'
      - name: kafka_cluster_encryption_in_transit
        value:
          encryption_type: '{{ encryption_type }}'
      - name: kafka_connect_version
        value: '{{ kafka_connect_version }}'
      - name: log_delivery
        value:
          worker_log_delivery:
            cloud_watch_logs:
              enabled: '{{ enabled }}'
              log_group: '{{ log_group }}'
            firehose:
              delivery_stream: '{{ delivery_stream }}'
              enabled: '{{ enabled }}'
            s3:
              bucket: '{{ bucket }}'
              enabled: '{{ enabled }}'
              prefix: '{{ prefix }}'
      - name: plugins
        value:
          - custom_plugin:
              name: '{{ name }}'
              description: '{{ description }}'
              content_type: '{{ content_type }}'
              location:
                s3_location:
                  bucket_arn: '{{ bucket_arn }}'
                  file_key: '{{ file_key }}'
                  object_version: '{{ object_version }}'
              tags:
                - key: '{{ key }}'
                  value: '{{ value }}'
      - name: service_execution_role_arn
        value: '{{ service_execution_role_arn }}'
      - name: tags
        value:
          - null
      - name: worker_configuration
        value:
          name: '{{ name }}'
          description: '{{ description }}'
          properties_file_content: '{{ properties_file_content }}'
          tags:
            - null

UPDATE example

Use the following StackQL query and manifest file to update a connector resource, using stack-deploy.

/*+ update */
UPDATE awscc.kafkaconnect.connectors
SET PatchDocument = string('{{ {
    "Capacity": capacity,
    "ConnectorConfiguration": connector_configuration,
    "Tags": tags
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ connector_arn }}';

DELETE example

/*+ delete */
DELETE FROM awscc.kafkaconnect.connectors
WHERE
  Identifier = '{{ connector_arn }}' AND
  region = 'us-east-1';

Permissions

To operate on the connectors resource, the following permissions are required:

Create

kafkaconnect:CreateConnector,
kafkaconnect:DescribeConnector,
kafkaconnect:TagResource,
kafkaconnect:ListTagsForResource,
iam:CreateServiceLinkedRole,
iam:PassRole,
ec2:CreateNetworkInterface,
ec2:DescribeSecurityGroups,
ec2:DescribeSubnets,
ec2:DescribeVpcs,
logs:CreateLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries,
logs:PutResourcePolicy,
logs:DescribeResourcePolicies,
logs:DescribeLogGroups,
s3:GetBucketPolicy,
s3:PutBucketPolicy,
firehose:TagDeliveryStream

Read

kafkaconnect:DescribeConnector,
kafkaconnect:ListTagsForResource

Delete

kafkaconnect:DeleteConnector,
kafkaconnect:DescribeConnector,
logs:DeleteLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries

Update

kafkaconnect:UpdateConnector,
kafkaconnect:DescribeConnector,
kafkaconnect:DescribeConnectorOperation,
kafkaconnect:TagResource,
kafkaconnect:ListTagsForResource,
kafkaconnect:UntagResource,
iam:CreateServiceLinkedRole,
logs:UpdateLogDelivery,
logs:GetLogDelivery,
logs:ListLogDeliveries,
logs:PutResourcePolicy,
logs:DescribeResourcePolicies,
logs:DescribeLogGroups,
s3:GetBucketPolicy,
s3:PutBucketPolicy,
firehose:TagDeliveryStream

List

kafkaconnect:ListConnectors