data_quality_job_definitions

Creates, updates, deletes or gets a data_quality_job_definition resource or lists data_quality_job_definitions in a region

Overview

Name	data_quality_job_definitions
Type	Resource
Description	Resource Type definition for AWS::SageMaker::DataQualityJobDefinition
Id	awscc.sagemaker.data_quality_job_definitions

Fields

Name	Datatype	Description
job_definition_arn	string	The Amazon Resource Name (ARN) of job definition.
job_definition_name	string	The name of the job definition.
▶data_quality_baseline_config	object	Baseline configuration used to validate that the data conforms to the specified constraints and statistics.
▶data_quality_app_specification	object	Container image configuration object for the monitoring job.
▶data_quality_job_input	object	The inputs for a monitoring job.
▶data_quality_job_output_config	object	The output configuration for monitoring jobs.
▶job_resources	object	Identifies the resources to deploy for a monitoring job.
▶network_config	object	Networking options for a job, such as network traffic encryption between containers, whether to allow inbound and outbound network calls to and from containers, and the VPC subnets and security groups to use for VPC-enabled jobs.
endpoint_name	string	The name of the endpoint used to run the monitoring job.
role_arn	string	The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
▶stopping_condition	object	Configures conditions under which the processing job should be stopped, such as how long the processing job has been running. After the condition is met, the processing job is stopped.
▶tags	array	An array of key-value pairs to apply to this resource.
creation_time	string	The time at which the job definition was created.
region	string	AWS region.

For more information, see AWS::SageMaker::DataQualityJobDefinition.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	DataQualityAppSpecification, DataQualityJobInput, DataQualityJobOutputConfig, JobResources, RoleArn, region
delete_resource	DELETE	data__Identifier, region
list_resources	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

Gets all properties from an individual data_quality_job_definition.

SELECT
region,
job_definition_arn,
job_definition_name,
data_quality_baseline_config,
data_quality_app_specification,
data_quality_job_input,
data_quality_job_output_config,
job_resources,
network_config,
endpoint_name,
role_arn,
stopping_condition,
tags,
creation_time
FROM awscc.sagemaker.data_quality_job_definitions
WHERE region = 'us-east-1' AND data__Identifier = '<JobDefinitionArn>';

INSERT example

Use the following StackQL query and manifest file to create a new data_quality_job_definition resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.sagemaker.data_quality_job_definitions (
 DataQualityAppSpecification,
 DataQualityJobInput,
 DataQualityJobOutputConfig,
 JobResources,
 RoleArn,
 region
)
SELECT 
'{{ DataQualityAppSpecification }}',
 '{{ DataQualityJobInput }}',
 '{{ DataQualityJobOutputConfig }}',
 '{{ JobResources }}',
 '{{ RoleArn }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO awscc.sagemaker.data_quality_job_definitions (
 JobDefinitionName,
 DataQualityBaselineConfig,
 DataQualityAppSpecification,
 DataQualityJobInput,
 DataQualityJobOutputConfig,
 JobResources,
 NetworkConfig,
 EndpointName,
 RoleArn,
 StoppingCondition,
 Tags,
 region
)
SELECT 
 '{{ JobDefinitionName }}',
 '{{ DataQualityBaselineConfig }}',
 '{{ DataQualityAppSpecification }}',
 '{{ DataQualityJobInput }}',
 '{{ DataQualityJobOutputConfig }}',
 '{{ JobResources }}',
 '{{ NetworkConfig }}',
 '{{ EndpointName }}',
 '{{ RoleArn }}',
 '{{ StoppingCondition }}',
 '{{ Tags }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: data_quality_job_definition
    props:
      - name: JobDefinitionName
        value: '{{ JobDefinitionName }}'
      - name: DataQualityBaselineConfig
        value:
          BaseliningJobName: '{{ BaseliningJobName }}'
          ConstraintsResource:
            S3Uri: '{{ S3Uri }}'
          StatisticsResource:
            S3Uri: null
      - name: DataQualityAppSpecification
        value:
          ContainerArguments:
            - '{{ ContainerArguments[0] }}'
          ContainerEntrypoint:
            - '{{ ContainerEntrypoint[0] }}'
          ImageUri: '{{ ImageUri }}'
          PostAnalyticsProcessorSourceUri: null
          RecordPreprocessorSourceUri: null
          Environment: {}
      - name: DataQualityJobInput
        value:
          EndpointInput:
            EndpointName: '{{ EndpointName }}'
            LocalPath: '{{ LocalPath }}'
            S3DataDistributionType: '{{ S3DataDistributionType }}'
            S3InputMode: '{{ S3InputMode }}'
            ExcludeFeaturesAttribute: '{{ ExcludeFeaturesAttribute }}'
          BatchTransformInput:
            DataCapturedDestinationS3Uri: '{{ DataCapturedDestinationS3Uri }}'
            DatasetFormat:
              Csv:
                Header: '{{ Header }}'
              Json:
                Line: '{{ Line }}'
              Parquet: '{{ Parquet }}'
            LocalPath: '{{ LocalPath }}'
            S3DataDistributionType: '{{ S3DataDistributionType }}'
            S3InputMode: '{{ S3InputMode }}'
            ExcludeFeaturesAttribute: '{{ ExcludeFeaturesAttribute }}'
      - name: DataQualityJobOutputConfig
        value:
          KmsKeyId: '{{ KmsKeyId }}'
          MonitoringOutputs:
            - S3Output:
                LocalPath: '{{ LocalPath }}'
                S3UploadMode: '{{ S3UploadMode }}'
                S3Uri: '{{ S3Uri }}'
      - name: JobResources
        value:
          ClusterConfig:
            InstanceCount: '{{ InstanceCount }}'
            InstanceType: '{{ InstanceType }}'
            VolumeSizeInGB: '{{ VolumeSizeInGB }}'
            VolumeKmsKeyId: '{{ VolumeKmsKeyId }}'
      - name: NetworkConfig
        value:
          EnableInterContainerTrafficEncryption: '{{ EnableInterContainerTrafficEncryption }}'
          EnableNetworkIsolation: '{{ EnableNetworkIsolation }}'
          VpcConfig:
            SecurityGroupIds:
              - '{{ SecurityGroupIds[0] }}'
            Subnets:
              - '{{ Subnets[0] }}'
      - name: EndpointName
        value: null
      - name: RoleArn
        value: '{{ RoleArn }}'
      - name: StoppingCondition
        value:
          MaxRuntimeInSeconds: '{{ MaxRuntimeInSeconds }}'
      - name: Tags
        value:
          - Value: '{{ Value }}'
            Key: '{{ Key }}'

DELETE example

/*+ delete */
DELETE FROM awscc.sagemaker.data_quality_job_definitions
WHERE data__Identifier = '<JobDefinitionArn>'
AND region = 'us-east-1';

Permissions

To operate on the data_quality_job_definitions resource, the following permissions are required:

Create

sagemaker:CreateDataQualityJobDefinition,
sagemaker:DescribeDataQualityJobDefinition,
sagemaker:AddTags,
sagemaker:ListTags,
iam:PassRole

Delete

sagemaker:DeleteDataQualityJobDefinition

Read

sagemaker:DescribeDataQualityJobDefinition,
sagemaker:ListTags

List

sagemaker:ListDataQualityJobDefinitions,
sagemaker:ListTags