delivery_streams

Creates, updates, deletes or gets a delivery_stream resource or lists delivery_streams in a region

Overview

Name	delivery_streams
Type	Resource
Description	Resource Type definition for AWS::KinesisFirehose::DeliveryStream
Id	awscc.kinesisfirehose.delivery_streams

Fields

get (all properties)

Name	Datatype	Description
▶delivery_stream_encryption_configuration_input	object
▶http_endpoint_destination_configuration	object
▶kinesis_stream_source_configuration	object
delivery_stream_type	string
▶iceberg_destination_configuration	object
▶redshift_destination_configuration	object
▶amazonopensearchservice_destination_configuration	object
▶msk_source_configuration	object
▶direct_put_source_configuration	object
▶splunk_destination_configuration	object
▶extended_s3_destination_configuration	object
▶amazon_open_search_serverless_destination_configuration	object
▶elasticsearch_destination_configuration	object
▶snowflake_destination_configuration	object
▶database_source_configuration	object
▶s3_destination_configuration	object
delivery_stream_name	string
arn	string
▶tags	array
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
delivery_stream_name	string
region	string	AWS region.

For more information, see AWS::KinesisFirehose::DeliveryStream.

Methods

Name	Resource	Accessible by	Required Params
create_resource	delivery_streams	INSERT	region
delete_resource	delivery_streams	DELETE	Identifier, region
update_resource	delivery_streams	UPDATE	Identifier, PatchDocument, region
list_resources	delivery_streams_list_only	SELECT	region
get_resource	delivery_streams	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual delivery_stream.

SELECT
  region,
  delivery_stream_encryption_configuration_input,
  http_endpoint_destination_configuration,
  kinesis_stream_source_configuration,
  delivery_stream_type,
  iceberg_destination_configuration,
  redshift_destination_configuration,
  amazonopensearchservice_destination_configuration,
  msk_source_configuration,
  direct_put_source_configuration,
  splunk_destination_configuration,
  extended_s3_destination_configuration,
  amazon_open_search_serverless_destination_configuration,
  elasticsearch_destination_configuration,
  snowflake_destination_configuration,
  database_source_configuration,
  s3_destination_configuration,
  delivery_stream_name,
  arn,
  tags
FROM awscc.kinesisfirehose.delivery_streams
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ delivery_stream_name }}';

list (identifiers only)

Lists all delivery_streams in a region.

SELECT
  region,
  delivery_stream_name
FROM awscc.kinesisfirehose.delivery_streams_list_only
WHERE
  region = '{{ region }}';

INSERT example

Use the following StackQL query and manifest file to create a new delivery_stream resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.kinesisfirehose.delivery_streams (
  DeliveryStreamEncryptionConfigurationInput,
  HttpEndpointDestinationConfiguration,
  KinesisStreamSourceConfiguration,
  DeliveryStreamType,
  IcebergDestinationConfiguration,
  RedshiftDestinationConfiguration,
  AmazonopensearchserviceDestinationConfiguration,
  MSKSourceConfiguration,
  DirectPutSourceConfiguration,
  SplunkDestinationConfiguration,
  ExtendedS3DestinationConfiguration,
  AmazonOpenSearchServerlessDestinationConfiguration,
  ElasticsearchDestinationConfiguration,
  SnowflakeDestinationConfiguration,
  DatabaseSourceConfiguration,
  S3DestinationConfiguration,
  DeliveryStreamName,
  Tags,
  region
)
SELECT
  '{{ delivery_stream_encryption_configuration_input }}',
  '{{ http_endpoint_destination_configuration }}',
  '{{ kinesis_stream_source_configuration }}',
  '{{ delivery_stream_type }}',
  '{{ iceberg_destination_configuration }}',
  '{{ redshift_destination_configuration }}',
  '{{ amazonopensearchservice_destination_configuration }}',
  '{{ msk_source_configuration }}',
  '{{ direct_put_source_configuration }}',
  '{{ splunk_destination_configuration }}',
  '{{ extended_s3_destination_configuration }}',
  '{{ amazon_open_search_serverless_destination_configuration }}',
  '{{ elasticsearch_destination_configuration }}',
  '{{ snowflake_destination_configuration }}',
  '{{ database_source_configuration }}',
  '{{ s3_destination_configuration }}',
  '{{ delivery_stream_name }}',
  '{{ tags }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

All Properties

/*+ create */
INSERT INTO awscc.kinesisfirehose.delivery_streams (
  DeliveryStreamEncryptionConfigurationInput,
  HttpEndpointDestinationConfiguration,
  KinesisStreamSourceConfiguration,
  DeliveryStreamType,
  IcebergDestinationConfiguration,
  RedshiftDestinationConfiguration,
  AmazonopensearchserviceDestinationConfiguration,
  MSKSourceConfiguration,
  DirectPutSourceConfiguration,
  SplunkDestinationConfiguration,
  ExtendedS3DestinationConfiguration,
  AmazonOpenSearchServerlessDestinationConfiguration,
  ElasticsearchDestinationConfiguration,
  SnowflakeDestinationConfiguration,
  DatabaseSourceConfiguration,
  S3DestinationConfiguration,
  DeliveryStreamName,
  Tags,
  region
)
SELECT
  '{{ delivery_stream_encryption_configuration_input }}',
  '{{ http_endpoint_destination_configuration }}',
  '{{ kinesis_stream_source_configuration }}',
  '{{ delivery_stream_type }}',
  '{{ iceberg_destination_configuration }}',
  '{{ redshift_destination_configuration }}',
  '{{ amazonopensearchservice_destination_configuration }}',
  '{{ msk_source_configuration }}',
  '{{ direct_put_source_configuration }}',
  '{{ splunk_destination_configuration }}',
  '{{ extended_s3_destination_configuration }}',
  '{{ amazon_open_search_serverless_destination_configuration }}',
  '{{ elasticsearch_destination_configuration }}',
  '{{ snowflake_destination_configuration }}',
  '{{ database_source_configuration }}',
  '{{ s3_destination_configuration }}',
  '{{ delivery_stream_name }}',
  '{{ tags }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Manifest

version: 1
name: stack name
description: stack description
providers:
- aws
globals:
- name: region
  value: '{{ vars.AWS_REGION }}'
resources:
- name: delivery_stream
  props:
    - name: delivery_stream_encryption_configuration_input
      value:
        key_type: '{{ key_type }}'
        key_arn: '{{ key_arn }}'
    - name: http_endpoint_destination_configuration
      value:
        request_configuration:
          common_attributes:
            - attribute_value: '{{ attribute_value }}'
              attribute_name: '{{ attribute_name }}'
          content_encoding: '{{ content_encoding }}'
        s3_configuration:
          error_output_prefix: '{{ error_output_prefix }}'
          bucket_arn: '{{ bucket_arn }}'
          buffering_hints:
            interval_in_seconds: '{{ interval_in_seconds }}'
            size_in_mbs: '{{ size_in_mbs }}'
          compression_format: '{{ compression_format }}'
          encryption_configuration:
            kms_encryption_config:
              aws_kms_key_arn: '{{ aws_kms_key_arn }}'
            no_encryption_config: '{{ no_encryption_config }}'
          prefix: '{{ prefix }}'
          cloud_watch_logging_options:
            log_stream_name: '{{ log_stream_name }}'
            enabled: '{{ enabled }}'
            log_group_name: '{{ log_group_name }}'
          role_arn: '{{ role_arn }}'
        buffering_hints: null
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        secrets_manager_configuration:
          secret_arn: '{{ secret_arn }}'
          enabled: '{{ enabled }}'
          role_arn: '{{ role_arn }}'
        endpoint_configuration:
          access_key: '{{ access_key }}'
          url: '{{ url }}'
          name: '{{ name }}'
        processing_configuration:
          enabled: '{{ enabled }}'
          processors:
            - type: '{{ type }}'
              parameters:
                - parameter_value: '{{ parameter_value }}'
                  parameter_name: '{{ parameter_name }}'
        role_arn: '{{ role_arn }}'
        cloud_watch_logging_options: null
        s3_backup_mode: '{{ s3_backup_mode }}'
    - name: kinesis_stream_source_configuration
      value:
        kinesis_stream_arn: '{{ kinesis_stream_arn }}'
        role_arn: '{{ role_arn }}'
    - name: delivery_stream_type
      value: '{{ delivery_stream_type }}'
    - name: iceberg_destination_configuration
      value:
        catalog_configuration:
          catalog_arn: '{{ catalog_arn }}'
          warehouse_location: '{{ warehouse_location }}'
        s3_configuration: null
        destination_table_configuration_list:
          - destination_database_name: '{{ destination_database_name }}'
            s3_error_output_prefix: '{{ s3_error_output_prefix }}'
            destination_table_name: '{{ destination_table_name }}'
            unique_keys:
              - '{{ unique_keys[0] }}'
            partition_spec:
              identity:
                - source_name: '{{ source_name }}'
        buffering_hints: null
        table_creation_configuration:
          enabled: '{{ enabled }}'
        retry_options: null
        s3_backup_mode: '{{ s3_backup_mode }}'
        processing_configuration: null
        schema_evolution_configuration:
          enabled: '{{ enabled }}'
        append_only: '{{ append_only }}'
        cloud_watch_logging_options: null
        role_arn: '{{ role_arn }}'
    - name: redshift_destination_configuration
      value:
        s3_backup_configuration: null
        s3_configuration: null
        username: '{{ username }}'
        copy_command:
          data_table_name: '{{ data_table_name }}'
          copy_options: '{{ copy_options }}'
          data_table_columns: '{{ data_table_columns }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        secrets_manager_configuration: null
        processing_configuration: null
        cloud_watch_logging_options: null
        cluster_jdb_curl: '{{ cluster_jdb_curl }}'
        role_arn: '{{ role_arn }}'
        password: '{{ password }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
    - name: amazonopensearchservice_destination_configuration
      value:
        type_name: '{{ type_name }}'
        index_rotation_period: '{{ index_rotation_period }}'
        processing_configuration: null
        cluster_endpoint: '{{ cluster_endpoint }}'
        domain_arn: '{{ domain_arn }}'
        role_arn: '{{ role_arn }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
        index_name: '{{ index_name }}'
        document_id_options:
          default_document_id_format: '{{ default_document_id_format }}'
        s3_configuration: null
        buffering_hints:
          interval_in_seconds: '{{ interval_in_seconds }}'
          size_in_mbs: '{{ size_in_mbs }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        vpc_configuration:
          subnet_ids:
            - '{{ subnet_ids[0] }}'
          security_group_ids:
            - '{{ security_group_ids[0] }}'
          role_arn: '{{ role_arn }}'
        cloud_watch_logging_options: null
    - name: msk_source_configuration
      value:
        authentication_configuration:
          connectivity: '{{ connectivity }}'
          role_arn: '{{ role_arn }}'
        read_from_timestamp: '{{ read_from_timestamp }}'
        msk_cluster_arn: '{{ msk_cluster_arn }}'
        topic_name: '{{ topic_name }}'
    - name: direct_put_source_configuration
      value:
        throughput_hint_in_mbs: '{{ throughput_hint_in_mbs }}'
    - name: splunk_destination_configuration
      value:
        h_ec_endpoint: '{{ h_ec_endpoint }}'
        s3_configuration: null
        buffering_hints:
          interval_in_seconds: '{{ interval_in_seconds }}'
          size_in_mbs: '{{ size_in_mbs }}'
        h_ec_token: '{{ h_ec_token }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        h_ec_endpoint_type: '{{ h_ec_endpoint_type }}'
        secrets_manager_configuration: null
        h_ecacknowledgment_timeout_in_seconds: '{{ h_ecacknowledgment_timeout_in_seconds }}'
        processing_configuration: null
        cloud_watch_logging_options: null
        s3_backup_mode: '{{ s3_backup_mode }}'
    - name: extended_s3_destination_configuration
      value:
        error_output_prefix: '{{ error_output_prefix }}'
        s3_backup_configuration: null
        bucket_arn: '{{ bucket_arn }}'
        compression_format: '{{ compression_format }}'
        data_format_conversion_configuration:
          input_format_configuration:
            deserializer:
              hive_json_ser_de:
                timestamp_formats:
                  - '{{ timestamp_formats[0] }}'
              open_xjson_ser_de:
                convert_dots_in_json_keys_to_underscores: '{{ convert_dots_in_json_keys_to_underscores }}'
                column_to_json_key_mappings: {}
                case_insensitive: '{{ case_insensitive }}'
          enabled: '{{ enabled }}'
          schema_configuration:
            version_id: '{{ version_id }}'
            table_name: '{{ table_name }}'
            database_name: '{{ database_name }}'
            region: '{{ region }}'
            catalog_id: '{{ catalog_id }}'
            role_arn: '{{ role_arn }}'
          output_format_configuration:
            serializer:
              orc_ser_de:
                padding_tolerance: null
                compression: '{{ compression }}'
                stripe_size_bytes: '{{ stripe_size_bytes }}'
                bloom_filter_columns:
                  - '{{ bloom_filter_columns[0] }}'
                bloom_filter_false_positive_probability: null
                enable_padding: '{{ enable_padding }}'
                format_version: '{{ format_version }}'
                row_index_stride: '{{ row_index_stride }}'
                block_size_bytes: '{{ block_size_bytes }}'
                dictionary_key_threshold: null
              parquet_ser_de:
                compression: '{{ compression }}'
                block_size_bytes: '{{ block_size_bytes }}'
                enable_dictionary_compression: '{{ enable_dictionary_compression }}'
                page_size_bytes: '{{ page_size_bytes }}'
                max_padding_bytes: '{{ max_padding_bytes }}'
                writer_version: '{{ writer_version }}'
        encryption_configuration: null
        custom_time_zone: '{{ custom_time_zone }}'
        dynamic_partitioning_configuration:
          enabled: '{{ enabled }}'
          retry_options: null
        prefix: '{{ prefix }}'
        processing_configuration: null
        role_arn: '{{ role_arn }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
        buffering_hints: null
        file_extension: '{{ file_extension }}'
        cloud_watch_logging_options: null
    - name: amazon_open_search_serverless_destination_configuration
      value:
        index_name: '{{ index_name }}'
        s3_configuration: null
        buffering_hints:
          interval_in_seconds: '{{ interval_in_seconds }}'
          size_in_mbs: '{{ size_in_mbs }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        collection_endpoint: '{{ collection_endpoint }}'
        vpc_configuration: null
        processing_configuration: null
        cloud_watch_logging_options: null
        role_arn: '{{ role_arn }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
    - name: elasticsearch_destination_configuration
      value:
        type_name: '{{ type_name }}'
        index_rotation_period: '{{ index_rotation_period }}'
        processing_configuration: null
        cluster_endpoint: '{{ cluster_endpoint }}'
        domain_arn: '{{ domain_arn }}'
        role_arn: '{{ role_arn }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
        index_name: '{{ index_name }}'
        document_id_options: null
        s3_configuration: null
        buffering_hints:
          interval_in_seconds: '{{ interval_in_seconds }}'
          size_in_mbs: '{{ size_in_mbs }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        vpc_configuration: null
        cloud_watch_logging_options: null
    - name: snowflake_destination_configuration
      value:
        private_key: '{{ private_key }}'
        user: '{{ user }}'
        table: '{{ table }}'
        snowflake_vpc_configuration:
          private_link_vpce_id: '{{ private_link_vpce_id }}'
        data_loading_option: '{{ data_loading_option }}'
        schema: '{{ schema }}'
        content_column_name: '{{ content_column_name }}'
        secrets_manager_configuration: null
        snowflake_role_configuration:
          snowflake_role: '{{ snowflake_role }}'
          enabled: '{{ enabled }}'
        processing_configuration: null
        account_url: '{{ account_url }}'
        role_arn: '{{ role_arn }}'
        s3_backup_mode: '{{ s3_backup_mode }}'
        s3_configuration: null
        buffering_hints:
          interval_in_seconds: '{{ interval_in_seconds }}'
          size_in_mbs: '{{ size_in_mbs }}'
        meta_data_column_name: '{{ meta_data_column_name }}'
        database: '{{ database }}'
        retry_options:
          duration_in_seconds: '{{ duration_in_seconds }}'
        key_passphrase: '{{ key_passphrase }}'
        cloud_watch_logging_options: null
    - name: database_source_configuration
      value:
        digest: '{{ digest }}'
        port: '{{ port }}'
        public_certificate: '{{ public_certificate }}'
        columns:
          exclude:
            - '{{ exclude[0] }}'
          include:
            - null
        type: '{{ type }}'
        surrogate_keys:
          - '{{ surrogate_keys[0] }}'
        databases:
          exclude:
            - '{{ exclude[0] }}'
          include:
            - null
        endpoint: '{{ endpoint }}'
        s_sl_mode: '{{ s_sl_mode }}'
        snapshot_watermark_table: '{{ snapshot_watermark_table }}'
        database_source_authentication_configuration:
          secrets_manager_configuration: null
        tables:
          exclude:
            - null
          include:
            - null
        database_source_vpc_configuration:
          vpc_endpoint_service_name: '{{ vpc_endpoint_service_name }}'
    - name: s3_destination_configuration
      value: null
    - name: delivery_stream_name
      value: '{{ delivery_stream_name }}'
    - name: tags
      value:
        - value: '{{ value }}'
          key: '{{ key }}'

UPDATE example

Use the following StackQL query and manifest file to update a delivery_stream resource, using stack-deploy.

/*+ update */
UPDATE awscc.kinesisfirehose.delivery_streams
SET PatchDocument = string('{{ {
    "DeliveryStreamEncryptionConfigurationInput": delivery_stream_encryption_configuration_input,
    "HttpEndpointDestinationConfiguration": http_endpoint_destination_configuration,
    "RedshiftDestinationConfiguration": redshift_destination_configuration,
    "SplunkDestinationConfiguration": splunk_destination_configuration,
    "ExtendedS3DestinationConfiguration": extended_s3_destination_configuration,
    "S3DestinationConfiguration": s3_destination_configuration,
    "Tags": tags
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ delivery_stream_name }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

DELETE example

/*+ delete */
DELETE FROM awscc.kinesisfirehose.delivery_streams
WHERE
  Identifier = '{{ delivery_stream_name }}' AND
  region = '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Additional Parameters

Mutable resources in the Cloud Control provider support additional optional parameters which can be supplied with INSERT, UPDATE, or DELETE operations. These include:

Parameter	Description
ClientToken	A unique identifier to ensure the idempotency of the resource request. This allows the provider to accurately distinguish between retries and new requests. A client token is valid for 36 hours once used. After that, a resource request with the same client token is treated as a new request. If you do not specify a client token, one is generated for inclusion in the request.
RoleArn	The ARN of the IAM role used to perform this resource operation. The role specified must have the permissions required for this operation. If you do not specify a role, a temporary session is created using your AWS user credentials.
TypeVersionId	For private resource types, the type version to use in this resource operation. If you do not specify a resource version, the default version is used.

Permissions

To operate on the delivery_streams resource, the following permissions are required:

Read

firehose:DescribeDeliveryStream,
firehose:ListTagsForDeliveryStream

Create

firehose:CreateDeliveryStream,
firehose:DescribeDeliveryStream,
iam:GetRole,
iam:PassRole,
kms:CreateGrant,
kms:DescribeKey,
firehose:TagDeliveryStream

Update

firehose:UpdateDestination,
firehose:DescribeDeliveryStream,
firehose:StartDeliveryStreamEncryption,
firehose:StopDeliveryStreamEncryption,
firehose:ListTagsForDeliveryStream,
firehose:TagDeliveryStream,
firehose:UntagDeliveryStream,
kms:CreateGrant,
kms:RevokeGrant,
kms:DescribeKey,
iam:PassRole

List

firehose:ListDeliveryStreams

Delete

firehose:DeleteDeliveryStream,
firehose:DescribeDeliveryStream,
kms:RevokeGrant,
kms:DescribeKey