jobs

Creates, updates, deletes or gets a job resource or lists jobs in a region

Overview

Name	jobs
Type	Resource
Description	Resource Type definition for AWS::Glue::Job
Id	awscc.glue.jobs

Fields

get (all properties)

Name	Datatype	Description
▶connections	object	Specifies the connections used by a job
max_retries	number	The maximum number of times to retry this job after a JobRun fails
description	string	A description of the job.
timeout	integer	The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
allocated_capacity	number	The number of capacity units that are allocated to this job.
name	string	The name you assign to the job definition
role	string	The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
default_arguments	object	The default arguments for this job, specified as name-value pairs.
▶notification_property	object	Specifies configuration properties of a notification.
worker_type	string	TThe type of predefined worker that is allocated when a job runs.
execution_class	string	Indicates whether the job is run with a standard or flexible execution class.
log_uri	string	This field is reserved for future use.
▶command	object	The code that executes a job.
glue_version	string	Glue version determines the versions of Apache Spark and Python that AWS Glue supports.
▶execution_property	object	The maximum number of concurrent runs that are allowed for this job.
security_configuration	string	The name of the SecurityConfiguration structure to be used with this job.
number_of_workers	integer	The number of workers of a defined workerType that are allocated when a job runs.
tags	object	The tags to use with this job.
max_capacity	number	The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
non_overridable_arguments	object	Non-overridable arguments for this job, specified as name-value pairs.
maintenance_window	string	Property description not available.
job_mode	string	Property description not available.
job_run_queuing_enabled	boolean	Property description not available.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
name	string	The name you assign to the job definition
region	string	AWS region.

For more information, see AWS::Glue::Job.

Methods

Name	Resource	Accessible by	Required Params
create_resource	jobs	INSERT	Role, Command, region
delete_resource	jobs	DELETE	Identifier, region
update_resource	jobs	UPDATE	Identifier, PatchDocument, region
list_resources	jobs_list_only	SELECT	region
get_resource	jobs	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual job.

SELECT
  region,
  connections,
  max_retries,
  description,
  timeout,
  allocated_capacity,
  name,
  role,
  default_arguments,
  notification_property,
  worker_type,
  execution_class,
  log_uri,
  command,
  glue_version,
  execution_property,
  security_configuration,
  number_of_workers,
  tags,
  max_capacity,
  non_overridable_arguments,
  maintenance_window,
  job_mode,
  job_run_queuing_enabled
FROM awscc.glue.jobs
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ name }}';

list (identifiers only)

Lists all jobs in a region.

SELECT
  region,
  name
FROM awscc.glue.jobs_list_only
WHERE
  region = '{{ region }}';

INSERT example

Use the following StackQL query and manifest file to create a new job resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.glue.jobs (
  Role,
  Command,
  region
)
SELECT
  '{{ role }}',
  '{{ command }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

All Properties

/*+ create */
INSERT INTO awscc.glue.jobs (
  Connections,
  MaxRetries,
  Description,
  Timeout,
  AllocatedCapacity,
  Name,
  Role,
  DefaultArguments,
  NotificationProperty,
  WorkerType,
  ExecutionClass,
  LogUri,
  Command,
  GlueVersion,
  ExecutionProperty,
  SecurityConfiguration,
  NumberOfWorkers,
  Tags,
  MaxCapacity,
  NonOverridableArguments,
  MaintenanceWindow,
  JobMode,
  JobRunQueuingEnabled,
  region
)
SELECT
  '{{ connections }}',
  '{{ max_retries }}',
  '{{ description }}',
  '{{ timeout }}',
  '{{ allocated_capacity }}',
  '{{ name }}',
  '{{ role }}',
  '{{ default_arguments }}',
  '{{ notification_property }}',
  '{{ worker_type }}',
  '{{ execution_class }}',
  '{{ log_uri }}',
  '{{ command }}',
  '{{ glue_version }}',
  '{{ execution_property }}',
  '{{ security_configuration }}',
  '{{ number_of_workers }}',
  '{{ tags }}',
  '{{ max_capacity }}',
  '{{ non_overridable_arguments }}',
  '{{ maintenance_window }}',
  '{{ job_mode }}',
  '{{ job_run_queuing_enabled }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Manifest

version: 1
name: stack name
description: stack description
providers:
- aws
globals:
- name: region
  value: '{{ vars.AWS_REGION }}'
resources:
- name: job
  props:
    - name: connections
      value:
        connections:
          - '{{ connections[0] }}'
    - name: max_retries
      value: null
    - name: description
      value: '{{ description }}'
    - name: timeout
      value: '{{ timeout }}'
    - name: allocated_capacity
      value: null
    - name: name
      value: '{{ name }}'
    - name: role
      value: '{{ role }}'
    - name: default_arguments
      value: {}
    - name: notification_property
      value:
        notify_delay_after: '{{ notify_delay_after }}'
    - name: worker_type
      value: '{{ worker_type }}'
    - name: execution_class
      value: '{{ execution_class }}'
    - name: log_uri
      value: '{{ log_uri }}'
    - name: command
      value:
        name: '{{ name }}'
        python_version: '{{ python_version }}'
        runtime: '{{ runtime }}'
        script_location: '{{ script_location }}'
    - name: glue_version
      value: '{{ glue_version }}'
    - name: execution_property
      value:
        max_concurrent_runs: null
    - name: security_configuration
      value: '{{ security_configuration }}'
    - name: number_of_workers
      value: '{{ number_of_workers }}'
    - name: tags
      value: {}
    - name: max_capacity
      value: null
    - name: non_overridable_arguments
      value: {}
    - name: maintenance_window
      value: '{{ maintenance_window }}'
    - name: job_mode
      value: '{{ job_mode }}'
    - name: job_run_queuing_enabled
      value: '{{ job_run_queuing_enabled }}'

UPDATE example

Use the following StackQL query and manifest file to update a job resource, using stack-deploy.

/*+ update */
UPDATE awscc.glue.jobs
SET PatchDocument = string('{{ {
    "Connections": connections,
    "MaxRetries": max_retries,
    "Description": description,
    "Timeout": timeout,
    "AllocatedCapacity": allocated_capacity,
    "Role": role,
    "DefaultArguments": default_arguments,
    "NotificationProperty": notification_property,
    "WorkerType": worker_type,
    "ExecutionClass": execution_class,
    "LogUri": log_uri,
    "Command": command,
    "GlueVersion": glue_version,
    "ExecutionProperty": execution_property,
    "SecurityConfiguration": security_configuration,
    "NumberOfWorkers": number_of_workers,
    "Tags": tags,
    "MaxCapacity": max_capacity,
    "NonOverridableArguments": non_overridable_arguments,
    "MaintenanceWindow": maintenance_window,
    "JobMode": job_mode,
    "JobRunQueuingEnabled": job_run_queuing_enabled
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ name }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

DELETE example

/*+ delete */
DELETE FROM awscc.glue.jobs
WHERE
  Identifier = '{{ name }}' AND
  region = '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Additional Parameters

Mutable resources in the Cloud Control provider support additional optional parameters which can be supplied with INSERT, UPDATE, or DELETE operations. These include:

Parameter	Description
ClientToken	A unique identifier to ensure the idempotency of the resource request. This allows the provider to accurately distinguish between retries and new requests. A client token is valid for 36 hours once used. After that, a resource request with the same client token is treated as a new request. If you do not specify a client token, one is generated for inclusion in the request.
RoleArn	The ARN of the IAM role used to perform this resource operation. The role specified must have the permissions required for this operation. If you do not specify a role, a temporary session is created using your AWS user credentials.
TypeVersionId	For private resource types, the type version to use in this resource operation. If you do not specify a resource version, the default version is used.

Permissions

To operate on the jobs resource, the following permissions are required:

Create

iam:GetRole,
iam:PassRole,
glue:CreateJob,
glue:GetJob,
glue:TagResource

Read

glue:GetJob,
glue:GetTags

Delete

glue:DeleteJob,
glue:GetJob,
glue:UntagResource

Update

iam:GetRole,
iam:PassRole,
glue:UpdateJob,
glue:UntagResource,
glue:TagResource

List

glue:ListJobs