jobs

Creates, updates, deletes or gets a job resource or lists jobs in a region

Overview

Name	jobs
Type	Resource
Description	Resource Type definition for AWS::Glue::Job
Id	awscc.glue.jobs

Fields

get (all properties)

Name	Datatype	Description
▶connections	object	Specifies the connections used by a job
max_retries	number	The maximum number of times to retry this job after a JobRun fails
description	string	A description of the job.
timeout	integer	The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
allocated_capacity	number	The number of capacity units that are allocated to this job.
name	string	The name you assign to the job definition
role	string	The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
default_arguments	object	The default arguments for this job, specified as name-value pairs.
▶notification_property	object	Specifies configuration properties of a notification.
worker_type	string	TThe type of predefined worker that is allocated when a job runs.
execution_class	string	Indicates whether the job is run with a standard or flexible execution class.
log_uri	string	This field is reserved for future use.
▶command	object	The code that executes a job.
glue_version	string	Glue version determines the versions of Apache Spark and Python that AWS Glue supports.
▶execution_property	object	The maximum number of concurrent runs that are allowed for this job.
security_configuration	string	The name of the SecurityConfiguration structure to be used with this job.
number_of_workers	integer	The number of workers of a defined workerType that are allocated when a job runs.
tags	object	The tags to use with this job.
max_capacity	number	The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
non_overridable_arguments	object	Non-overridable arguments for this job, specified as name-value pairs.
maintenance_window	string	Property description not available.
job_mode	string	Property description not available.
job_run_queuing_enabled	boolean	Property description not available.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
name	string	The name you assign to the job definition
region	string	AWS region.

For more information, see AWS::Glue::Job.

Methods

Name	Resource	Accessible by	Required Params
create_resource	jobs	INSERT	Role, Command, region
delete_resource	jobs	DELETE	Identifier, region
update_resource	jobs	UPDATE	Identifier, PatchDocument, region
list_resources	jobs_list_only	SELECT	region
get_resource	jobs	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual job.

SELECT
  region,
  connections,
  max_retries,
  description,
  timeout,
  allocated_capacity,
  name,
  role,
  default_arguments,
  notification_property,
  worker_type,
  execution_class,
  log_uri,
  command,
  glue_version,
  execution_property,
  security_configuration,
  number_of_workers,
  tags,
  max_capacity,
  non_overridable_arguments,
  maintenance_window,
  job_mode,
  job_run_queuing_enabled
FROM awscc.glue.jobs
WHERE
  region = 'us-east-1' AND
  Identifier = '{{ name }}';

list (identifiers only)

Lists all jobs in a region.

SELECT
  region,
  name
FROM awscc.glue.jobs_list_only
WHERE
  region = 'us-east-1';

INSERT example

Use the following StackQL query and manifest file to create a new job resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.glue.jobs (
  Role,
  Command,
  region
)
SELECT
  '{{ role }}',
  '{{ command }}',
  '{{ region }}';

All Properties

/*+ create */
INSERT INTO awscc.glue.jobs (
  Connections,
  MaxRetries,
  Description,
  Timeout,
  AllocatedCapacity,
  Name,
  Role,
  DefaultArguments,
  NotificationProperty,
  WorkerType,
  ExecutionClass,
  LogUri,
  Command,
  GlueVersion,
  ExecutionProperty,
  SecurityConfiguration,
  NumberOfWorkers,
  Tags,
  MaxCapacity,
  NonOverridableArguments,
  MaintenanceWindow,
  JobMode,
  JobRunQueuingEnabled,
  region
)
SELECT
  '{{ connections }}',
  '{{ max_retries }}',
  '{{ description }}',
  '{{ timeout }}',
  '{{ allocated_capacity }}',
  '{{ name }}',
  '{{ role }}',
  '{{ default_arguments }}',
  '{{ notification_property }}',
  '{{ worker_type }}',
  '{{ execution_class }}',
  '{{ log_uri }}',
  '{{ command }}',
  '{{ glue_version }}',
  '{{ execution_property }}',
  '{{ security_configuration }}',
  '{{ number_of_workers }}',
  '{{ tags }}',
  '{{ max_capacity }}',
  '{{ non_overridable_arguments }}',
  '{{ maintenance_window }}',
  '{{ job_mode }}',
  '{{ job_run_queuing_enabled }}',
  '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: job
    props:
      - name: connections
        value:
          connections:
            - '{{ connections[0] }}'
      - name: max_retries
        value: null
      - name: description
        value: '{{ description }}'
      - name: timeout
        value: '{{ timeout }}'
      - name: allocated_capacity
        value: null
      - name: name
        value: '{{ name }}'
      - name: role
        value: '{{ role }}'
      - name: default_arguments
        value: {}
      - name: notification_property
        value:
          notify_delay_after: '{{ notify_delay_after }}'
      - name: worker_type
        value: '{{ worker_type }}'
      - name: execution_class
        value: '{{ execution_class }}'
      - name: log_uri
        value: '{{ log_uri }}'
      - name: command
        value:
          name: '{{ name }}'
          python_version: '{{ python_version }}'
          runtime: '{{ runtime }}'
          script_location: '{{ script_location }}'
      - name: glue_version
        value: '{{ glue_version }}'
      - name: execution_property
        value:
          max_concurrent_runs: null
      - name: security_configuration
        value: '{{ security_configuration }}'
      - name: number_of_workers
        value: '{{ number_of_workers }}'
      - name: tags
        value: {}
      - name: max_capacity
        value: null
      - name: non_overridable_arguments
        value: {}
      - name: maintenance_window
        value: '{{ maintenance_window }}'
      - name: job_mode
        value: '{{ job_mode }}'
      - name: job_run_queuing_enabled
        value: '{{ job_run_queuing_enabled }}'

UPDATE example

Use the following StackQL query and manifest file to update a job resource, using stack-deploy.

/*+ update */
UPDATE awscc.glue.jobs
SET PatchDocument = string('{{ {
    "Connections": connections,
    "MaxRetries": max_retries,
    "Description": description,
    "Timeout": timeout,
    "AllocatedCapacity": allocated_capacity,
    "Role": role,
    "DefaultArguments": default_arguments,
    "NotificationProperty": notification_property,
    "WorkerType": worker_type,
    "ExecutionClass": execution_class,
    "LogUri": log_uri,
    "Command": command,
    "GlueVersion": glue_version,
    "ExecutionProperty": execution_property,
    "SecurityConfiguration": security_configuration,
    "NumberOfWorkers": number_of_workers,
    "Tags": tags,
    "MaxCapacity": max_capacity,
    "NonOverridableArguments": non_overridable_arguments,
    "MaintenanceWindow": maintenance_window,
    "JobMode": job_mode,
    "JobRunQueuingEnabled": job_run_queuing_enabled
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ name }}';

DELETE example

/*+ delete */
DELETE FROM awscc.glue.jobs
WHERE
  Identifier = '{{ name }}' AND
  region = 'us-east-1';

Permissions

To operate on the jobs resource, the following permissions are required:

Create

iam:GetRole,
iam:PassRole,
glue:CreateJob,
glue:GetJob,
glue:TagResource

Read

glue:GetJob,
glue:GetTags

Delete

glue:DeleteJob,
glue:GetJob,
glue:UntagResource

Update

iam:GetRole,
iam:PassRole,
glue:UpdateJob,
glue:UntagResource,
glue:TagResource

List

glue:ListJobs