location_hdfs

Creates, updates, deletes or gets a location_hdf resource or lists location_hdfs in a region

Overview

Name	location_hdfs
Type	Resource
Description	Resource schema for AWS::DataSync::LocationHDFS.
Id	awscc.datasync.location_hdfs

Fields

get (all properties)

Name	Datatype	Description
▶name_nodes	array	An array of Name Node(s) of the HDFS location.
block_size	integer	Size of chunks (blocks) in bytes that the data is divided into when stored in the HDFS cluster.
replication_factor	integer	Number of copies of each block that exists inside the HDFS cluster.
kms_key_provider_uri	string	The identifier for the Key Management Server where the encryption keys that encrypt data inside HDFS clusters are stored.
▶qop_configuration	object	Configuration information for RPC Protection and Data Transfer Protection. These parameters can be set to AUTHENTICATION, INTEGRITY, or PRIVACY. The default value is PRIVACY.
authentication_type	string	The authentication mode used to determine identity of user.
simple_user	string	The user name that has read and write permissions on the specified HDFS cluster.
kerberos_principal	string	The unique identity, or principal, to which Kerberos can assign tickets.
kerberos_keytab	string	The Base64 string representation of the Keytab file.
kerberos_krb5_conf	string	The string representation of the Krb5Conf file, or the presigned URL to access the Krb5.conf file within an S3 bucket.
▶tags	array	An array of key-value pairs to apply to this resource.
agent_arns	array	ARN(s) of the agent(s) to use for an HDFS location.
subdirectory	string	The subdirectory in HDFS that is used to read data from the HDFS source location or write data to the HDFS destination.
location_arn	string	The Amazon Resource Name (ARN) of the HDFS location.
location_uri	string	The URL of the HDFS location that was described.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
location_arn	string	The Amazon Resource Name (ARN) of the HDFS location.
region	string	AWS region.

For more information, see AWS::DataSync::LocationHDFS.

Methods

Name	Resource	Accessible by	Required Params
create_resource	location_hdfs	INSERT	NameNodes, AuthenticationType, AgentArns, region
delete_resource	location_hdfs	DELETE	Identifier, region
update_resource	location_hdfs	UPDATE	Identifier, PatchDocument, region
list_resources	location_hdfs_list_only	SELECT	region
get_resource	location_hdfs	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual location_hdf.

SELECT
  region,
  name_nodes,
  block_size,
  replication_factor,
  kms_key_provider_uri,
  qop_configuration,
  authentication_type,
  simple_user,
  kerberos_principal,
  kerberos_keytab,
  kerberos_krb5_conf,
  tags,
  agent_arns,
  subdirectory,
  location_arn,
  location_uri
FROM awscc.datasync.location_hdfs
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ location_arn }}';

list (identifiers only)

Lists all location_hdfs in a region.

SELECT
  region,
  location_arn
FROM awscc.datasync.location_hdfs_list_only
WHERE
  region = '{{ region }}';

INSERT example

Use the following StackQL query and manifest file to create a new location_hdf resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.datasync.location_hdfs (
  NameNodes,
  AuthenticationType,
  AgentArns,
  region
)
SELECT
  '{{ name_nodes }}',
  '{{ authentication_type }}',
  '{{ agent_arns }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

All Properties

/*+ create */
INSERT INTO awscc.datasync.location_hdfs (
  NameNodes,
  BlockSize,
  ReplicationFactor,
  KmsKeyProviderUri,
  QopConfiguration,
  AuthenticationType,
  SimpleUser,
  KerberosPrincipal,
  KerberosKeytab,
  KerberosKrb5Conf,
  Tags,
  AgentArns,
  Subdirectory,
  region
)
SELECT
  '{{ name_nodes }}',
  '{{ block_size }}',
  '{{ replication_factor }}',
  '{{ kms_key_provider_uri }}',
  '{{ qop_configuration }}',
  '{{ authentication_type }}',
  '{{ simple_user }}',
  '{{ kerberos_principal }}',
  '{{ kerberos_keytab }}',
  '{{ kerberos_krb5_conf }}',
  '{{ tags }}',
  '{{ agent_arns }}',
  '{{ subdirectory }}',
  '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Manifest

version: 1
name: stack name
description: stack description
providers:
- aws
globals:
- name: region
  value: '{{ vars.AWS_REGION }}'
resources:
- name: location_hdf
  props:
    - name: name_nodes
      value:
        - hostname: '{{ hostname }}'
          port: '{{ port }}'
    - name: block_size
      value: '{{ block_size }}'
    - name: replication_factor
      value: '{{ replication_factor }}'
    - name: kms_key_provider_uri
      value: '{{ kms_key_provider_uri }}'
    - name: qop_configuration
      value:
        rpc_protection: '{{ rpc_protection }}'
        data_transfer_protection: '{{ data_transfer_protection }}'
    - name: authentication_type
      value: '{{ authentication_type }}'
    - name: simple_user
      value: '{{ simple_user }}'
    - name: kerberos_principal
      value: '{{ kerberos_principal }}'
    - name: kerberos_keytab
      value: '{{ kerberos_keytab }}'
    - name: kerberos_krb5_conf
      value: '{{ kerberos_krb5_conf }}'
    - name: tags
      value:
        - key: '{{ key }}'
          value: '{{ value }}'
    - name: agent_arns
      value:
        - '{{ agent_arns[0] }}'
    - name: subdirectory
      value: '{{ subdirectory }}'

UPDATE example

Use the following StackQL query and manifest file to update a location_hdf resource, using stack-deploy.

/*+ update */
UPDATE awscc.datasync.location_hdfs
SET PatchDocument = string('{{ {
    "NameNodes": name_nodes,
    "BlockSize": block_size,
    "ReplicationFactor": replication_factor,
    "KmsKeyProviderUri": kms_key_provider_uri,
    "QopConfiguration": qop_configuration,
    "AuthenticationType": authentication_type,
    "SimpleUser": simple_user,
    "KerberosPrincipal": kerberos_principal,
    "KerberosKeytab": kerberos_keytab,
    "KerberosKrb5Conf": kerberos_krb5_conf,
    "Tags": tags,
    "AgentArns": agent_arns,
    "Subdirectory": subdirectory
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ location_arn }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

DELETE example

/*+ delete */
DELETE FROM awscc.datasync.location_hdfs
WHERE
  Identifier = '{{ location_arn }}' AND
  region = '{{ region }}'
RETURNING
  ErrorCode,
  EventTime,
  Identifier,
  Operation,
  OperationStatus,
  RequestToken,
  ResourceModel,
  RetryAfter,
  StatusMessage,
  TypeName
;

Additional Parameters

Mutable resources in the Cloud Control provider support additional optional parameters which can be supplied with INSERT, UPDATE, or DELETE operations. These include:

Parameter	Description
ClientToken	A unique identifier to ensure the idempotency of the resource request. This allows the provider to accurately distinguish between retries and new requests. A client token is valid for 36 hours once used. After that, a resource request with the same client token is treated as a new request. If you do not specify a client token, one is generated for inclusion in the request.
RoleArn	The ARN of the IAM role used to perform this resource operation. The role specified must have the permissions required for this operation. If you do not specify a role, a temporary session is created using your AWS user credentials.
TypeVersionId	For private resource types, the type version to use in this resource operation. If you do not specify a resource version, the default version is used.

Permissions

To operate on the location_hdfs resource, the following permissions are required:

Create

datasync:CreateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:TagResource,
datasync:ListTagsForResource

Read

datasync:DescribeLocationHdfs,
datasync:ListTagsForResource

Update

datasync:UpdateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:ListTagsForResource,
datasync:TagResource,
datasync:UntagResource

Delete

datasync:DeleteLocation

List

datasync:ListLocations