data_quality_job_definitions

Creates, updates, deletes or gets a data_quality_job_definition resource or lists data_quality_job_definitions in a region

Overview

Name	data_quality_job_definitions
Type	Resource
Description	Resource Type definition for AWS::SageMaker::DataQualityJobDefinition
Id	awscc.sagemaker.data_quality_job_definitions

Fields

get (all properties)

Name	Datatype	Description
job_definition_arn	string	The Amazon Resource Name (ARN) of job definition.
job_definition_name	string	The name of the job definition.
▶data_quality_baseline_config	object	Baseline configuration used to validate that the data conforms to the specified constraints and statistics.
▶data_quality_app_specification	object	Container image configuration object for the monitoring job.
▶data_quality_job_input	object	The inputs for a monitoring job.
▶data_quality_job_output_config	object	The output configuration for monitoring jobs.
▶job_resources	object	Identifies the resources to deploy for a monitoring job.
▶network_config	object	Networking options for a job, such as network traffic encryption between containers, whether to allow inbound and outbound network calls to and from containers, and the VPC subnets and security groups to use for VPC-enabled jobs.
endpoint_name	string	The name of the endpoint used to run the monitoring job.
role_arn	string	The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
▶stopping_condition	object	Configures conditions under which the processing job should be stopped, such as how long the processing job has been running. After the condition is met, the processing job is stopped.
▶tags	array	An array of key-value pairs to apply to this resource.
creation_time	string	The time at which the job definition was created.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
job_definition_arn	string	The Amazon Resource Name (ARN) of job definition.
region	string	AWS region.

For more information, see AWS::SageMaker::DataQualityJobDefinition.

Methods

Name	Resource	Accessible by	Required Params
create_resource	data_quality_job_definitions	INSERT	DataQualityAppSpecification, DataQualityJobInput, DataQualityJobOutputConfig, JobResources, RoleArn, region
delete_resource	data_quality_job_definitions	DELETE	Identifier, region
list_resources	data_quality_job_definitions_list_only	SELECT	region
get_resource	data_quality_job_definitions	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual data_quality_job_definition.

SELECT
  region,
  job_definition_arn,
  job_definition_name,
  data_quality_baseline_config,
  data_quality_app_specification,
  data_quality_job_input,
  data_quality_job_output_config,
  job_resources,
  network_config,
  endpoint_name,
  role_arn,
  stopping_condition,
  tags,
  creation_time
FROM awscc.sagemaker.data_quality_job_definitions
WHERE
  region = 'us-east-1' AND
  Identifier = '{{ job_definition_arn }}';

list (identifiers only)

Lists all data_quality_job_definitions in a region.

SELECT
  region,
  job_definition_arn
FROM awscc.sagemaker.data_quality_job_definitions_list_only
WHERE
  region = 'us-east-1';

INSERT example

Use the following StackQL query and manifest file to create a new data_quality_job_definition resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.sagemaker.data_quality_job_definitions (
  DataQualityAppSpecification,
  DataQualityJobInput,
  DataQualityJobOutputConfig,
  JobResources,
  RoleArn,
  region
)
SELECT
  '{{ data_quality_app_specification }}',
  '{{ data_quality_job_input }}',
  '{{ data_quality_job_output_config }}',
  '{{ job_resources }}',
  '{{ role_arn }}',
  '{{ region }}';

All Properties

/*+ create */
INSERT INTO awscc.sagemaker.data_quality_job_definitions (
  JobDefinitionName,
  DataQualityBaselineConfig,
  DataQualityAppSpecification,
  DataQualityJobInput,
  DataQualityJobOutputConfig,
  JobResources,
  NetworkConfig,
  EndpointName,
  RoleArn,
  StoppingCondition,
  Tags,
  region
)
SELECT
  '{{ job_definition_name }}',
  '{{ data_quality_baseline_config }}',
  '{{ data_quality_app_specification }}',
  '{{ data_quality_job_input }}',
  '{{ data_quality_job_output_config }}',
  '{{ job_resources }}',
  '{{ network_config }}',
  '{{ endpoint_name }}',
  '{{ role_arn }}',
  '{{ stopping_condition }}',
  '{{ tags }}',
  '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: data_quality_job_definition
    props:
      - name: job_definition_name
        value: '{{ job_definition_name }}'
      - name: data_quality_baseline_config
        value:
          baselining_job_name: '{{ baselining_job_name }}'
          constraints_resource:
            s3_uri: '{{ s3_uri }}'
          statistics_resource:
            s3_uri: null
      - name: data_quality_app_specification
        value:
          container_arguments:
            - '{{ container_arguments[0] }}'
          container_entrypoint:
            - '{{ container_entrypoint[0] }}'
          image_uri: '{{ image_uri }}'
          post_analytics_processor_source_uri: null
          record_preprocessor_source_uri: null
          environment: {}
      - name: data_quality_job_input
        value:
          endpoint_input:
            endpoint_name: '{{ endpoint_name }}'
            local_path: '{{ local_path }}'
            s3_data_distribution_type: '{{ s3_data_distribution_type }}'
            s3_input_mode: '{{ s3_input_mode }}'
            exclude_features_attribute: '{{ exclude_features_attribute }}'
          batch_transform_input:
            data_captured_destination_s3_uri: '{{ data_captured_destination_s3_uri }}'
            dataset_format:
              csv:
                header: '{{ header }}'
              json:
                line: '{{ line }}'
              parquet: '{{ parquet }}'
            local_path: '{{ local_path }}'
            s3_data_distribution_type: '{{ s3_data_distribution_type }}'
            s3_input_mode: '{{ s3_input_mode }}'
            exclude_features_attribute: '{{ exclude_features_attribute }}'
      - name: data_quality_job_output_config
        value:
          kms_key_id: '{{ kms_key_id }}'
          monitoring_outputs:
            - s3_output:
                local_path: '{{ local_path }}'
                s3_upload_mode: '{{ s3_upload_mode }}'
                s3_uri: '{{ s3_uri }}'
      - name: job_resources
        value:
          cluster_config:
            instance_count: '{{ instance_count }}'
            instance_type: '{{ instance_type }}'
            volume_size_in_gb: '{{ volume_size_in_gb }}'
            volume_kms_key_id: '{{ volume_kms_key_id }}'
      - name: network_config
        value:
          enable_inter_container_traffic_encryption: '{{ enable_inter_container_traffic_encryption }}'
          enable_network_isolation: '{{ enable_network_isolation }}'
          vpc_config:
            security_group_ids:
              - '{{ security_group_ids[0] }}'
            subnets:
              - '{{ subnets[0] }}'
      - name: endpoint_name
        value: null
      - name: role_arn
        value: '{{ role_arn }}'
      - name: stopping_condition
        value:
          max_runtime_in_seconds: '{{ max_runtime_in_seconds }}'
      - name: tags
        value:
          - value: '{{ value }}'
            key: '{{ key }}'

DELETE example

/*+ delete */
DELETE FROM awscc.sagemaker.data_quality_job_definitions
WHERE
  Identifier = '{{ job_definition_arn }}' AND
  region = 'us-east-1';

Permissions

To operate on the data_quality_job_definitions resource, the following permissions are required:

Create

sagemaker:CreateDataQualityJobDefinition,
sagemaker:DescribeDataQualityJobDefinition,
sagemaker:AddTags,
sagemaker:ListTags,
iam:PassRole

Delete

sagemaker:DeleteDataQualityJobDefinition

Read

sagemaker:DescribeDataQualityJobDefinition,
sagemaker:ListTags

List

sagemaker:ListDataQualityJobDefinitions,
sagemaker:ListTags