data_sources

Creates, updates, deletes or gets a data_source resource or lists data_sources in a region

Overview

Name	data_sources
Type	Resource
Description	A data source is used to import technical metadata of assets (data) from the source databases or data warehouses into Amazon DataZone.
Id	awscc.datazone.data_sources

Fields

get (all properties)

Name	Datatype	Description
▶asset_forms_input	array	The metadata forms that are to be attached to the assets that this data source works with.
connection_id	string	The unique identifier of a connection used to fetch relevant parameters from connection during Datasource run
connection_identifier	string	The unique identifier of a connection used to fetch relevant parameters from connection during Datasource run
created_at	string	The timestamp of when the data source was created.
description	string	The description of the data source.
domain_id	string	The ID of the Amazon DataZone domain where the data source is created.
domain_identifier	string	The ID of the Amazon DataZone domain where the data source is created.
enable_setting	string	Specifies whether the data source is enabled.
environment_id	string	The unique identifier of the Amazon DataZone environment to which the data source publishes assets.
environment_identifier	string	The unique identifier of the Amazon DataZone environment to which the data source publishes assets.
id	string	The unique identifier of the data source.
configuration	object	Configuration of the data source. It can be set to either glueRunConfiguration or redshiftRunConfiguration.
last_run_asset_count	number	The number of assets created by the data source during its last run.
last_run_at	string	The timestamp that specifies when the data source was last run.
last_run_status	string	The status of the last run of this data source.
name	string	The name of the data source.
project_id	string	The ID of the Amazon DataZone project to which the data source is added.
project_identifier	string	The identifier of the Amazon DataZone project in which you want to add the data source.
publish_on_import	boolean	Specifies whether the assets that this data source creates in the inventory are to be also automatically published to the catalog.
▶recommendation	object	Specifies whether the business name generation is to be enabled for this data source.
▶schedule	object	The schedule of the data source runs.
status	string	The status of the data source.
type	string	The type of the data source.
updated_at	string	The timestamp of when this data source was updated.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
domain_id	string	The ID of the Amazon DataZone domain where the data source is created.
id	string	The unique identifier of the data source.
region	string	AWS region.

For more information, see AWS::DataZone::DataSource.

Methods

Name	Resource	Accessible by	Required Params
create_resource	data_sources	INSERT	Name, DomainIdentifier, ProjectIdentifier, Type, region
delete_resource	data_sources	DELETE	Identifier, region
update_resource	data_sources	UPDATE	Identifier, PatchDocument, region
list_resources	data_sources_list_only	SELECT	region
get_resource	data_sources	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual data_source.

SELECT
  region,
  asset_forms_input,
  connection_id,
  connection_identifier,
  created_at,
  description,
  domain_id,
  domain_identifier,
  enable_setting,
  environment_id,
  environment_identifier,
  id,
  configuration,
  last_run_asset_count,
  last_run_at,
  last_run_status,
  name,
  project_id,
  project_identifier,
  publish_on_import,
  recommendation,
  schedule,
  status,
  type,
  updated_at
FROM awscc.datazone.data_sources
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ domain_id }}|{{ id }}';

list (identifiers only)

Lists all data_sources in a region.

SELECT
  region,
  domain_id,
  id
FROM awscc.datazone.data_sources_list_only
WHERE
  region = '{{ region }}';

INSERT example

Use the following StackQL query and manifest file to create a new data_source resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.datazone.data_sources (
  DomainIdentifier,
  Name,
  ProjectIdentifier,
  Type,
  region
)
SELECT
  '{{ domain_identifier }}',
  '{{ name }}',
  '{{ project_identifier }}',
  '{{ type }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

All Properties

/*+ create */
INSERT INTO awscc.datazone.data_sources (
  AssetFormsInput,
  ConnectionIdentifier,
  Description,
  DomainIdentifier,
  EnableSetting,
  EnvironmentIdentifier,
  Configuration,
  Name,
  ProjectIdentifier,
  PublishOnImport,
  Recommendation,
  Schedule,
  Type,
  region
)
SELECT
  '{{ asset_forms_input }}',
  '{{ connection_identifier }}',
  '{{ description }}',
  '{{ domain_identifier }}',
  '{{ enable_setting }}',
  '{{ environment_identifier }}',
  '{{ configuration }}',
  '{{ name }}',
  '{{ project_identifier }}',
  '{{ publish_on_import }}',
  '{{ recommendation }}',
  '{{ schedule }}',
  '{{ type }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Manifest

version: 1
name: stack name
description: stack description
providers:
- aws
globals:
- name: region
  value: '{{ vars.AWS_REGION }}'
resources:
- name: data_source
  props:
    - name: asset_forms_input
      value:
        - form_name: '{{ form_name }}'
          type_identifier: '{{ type_identifier }}'
          type_revision: '{{ type_revision }}'
          content: '{{ content }}'
    - name: connection_identifier
      value: '{{ connection_identifier }}'
    - name: description
      value: '{{ description }}'
    - name: domain_identifier
      value: '{{ domain_identifier }}'
    - name: enable_setting
      value: '{{ enable_setting }}'
    - name: environment_identifier
      value: '{{ environment_identifier }}'
    - name: configuration
      value: null
    - name: name
      value: '{{ name }}'
    - name: project_identifier
      value: '{{ project_identifier }}'
    - name: publish_on_import
      value: '{{ publish_on_import }}'
    - name: recommendation
      value:
        enable_business_name_generation: '{{ enable_business_name_generation }}'
    - name: schedule
      value:
        timezone: '{{ timezone }}'
        schedule: '{{ schedule }}'
    - name: type
      value: '{{ type }}'

UPDATE example

Use the following StackQL query and manifest file to update a data_source resource, using stack-deploy.

/*+ update */
UPDATE awscc.datazone.data_sources
SET PatchDocument = string('{{ {
    "AssetFormsInput": asset_forms_input,
    "Description": description,
    "EnableSetting": enable_setting,
    "Configuration": configuration,
    "Name": name,
    "PublishOnImport": publish_on_import,
    "Recommendation": recommendation,
    "Schedule": schedule
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ domain_id }}|{{ id }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

DELETE example

/*+ delete */
DELETE FROM awscc.datazone.data_sources
WHERE
  Identifier = '{{ domain_id }}|{{ id }}' AND
  region = '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Additional Parameters

Mutable resources in the Cloud Control provider support additional optional parameters which can be supplied with INSERT, UPDATE, or DELETE operations. These include:

Parameter	Description
ClientToken	A unique identifier to ensure the idempotency of the resource request. This allows the provider to accurately distinguish between retries and new requests. A client token is valid for 36 hours once used. After that, a resource request with the same client token is treated as a new request. If you do not specify a client token, one is generated for inclusion in the request.
RoleArn	The ARN of the IAM role used to perform this resource operation. The role specified must have the permissions required for this operation. If you do not specify a role, a temporary session is created using your AWS user credentials.
TypeVersionId	For private resource types, the type version to use in this resource operation. If you do not specify a resource version, the default version is used.

Permissions

To operate on the data_sources resource, the following permissions are required:

Create

datazone:CreateDataSource,
iam:PassRole,
datazone:GetDataSource,
datazone:DeleteDataSource

Read

datazone:GetDataSource

Update

datazone:UpdateDataSource,
datazone:GetDataSource,
datazone:DeleteDataSource

Delete

datazone:DeleteDataSource,
datazone:GetDataSource

List

datazone:ListDataSources