location_hdfs

Creates, updates, deletes or gets a location_hdf resource or lists location_hdfs in a region

Overview

Name	location_hdfs
Type	Resource
Description	Resource schema for AWS::DataSync::LocationHDFS.
Id	awscc.datasync.location_hdfs

Fields

get (all properties)

Name	Datatype	Description
▶name_nodes	array	An array of Name Node(s) of the HDFS location.
block_size	integer	Size of chunks (blocks) in bytes that the data is divided into when stored in the HDFS cluster.
replication_factor	integer	Number of copies of each block that exists inside the HDFS cluster.
kms_key_provider_uri	string	The identifier for the Key Management Server where the encryption keys that encrypt data inside HDFS clusters are stored.
▶qop_configuration	object	Configuration information for RPC Protection and Data Transfer Protection. These parameters can be set to AUTHENTICATION, INTEGRITY, or PRIVACY. The default value is PRIVACY.
authentication_type	string	The authentication mode used to determine identity of user.
simple_user	string	The user name that has read and write permissions on the specified HDFS cluster.
kerberos_principal	string	The unique identity, or principal, to which Kerberos can assign tickets.
kerberos_keytab	string	The Base64 string representation of the Keytab file.
kerberos_krb5_conf	string	The string representation of the Krb5Conf file, or the presigned URL to access the Krb5.conf file within an S3 bucket.
▶tags	array	An array of key-value pairs to apply to this resource.
agent_arns	array	ARN(s) of the agent(s) to use for an HDFS location.
subdirectory	string	The subdirectory in HDFS that is used to read data from the HDFS source location or write data to the HDFS destination.
location_arn	string	The Amazon Resource Name (ARN) of the HDFS location.
location_uri	string	The URL of the HDFS location that was described.
region	string	AWS region.

list (identifiers only)

Name	Datatype	Description
location_arn	string	The Amazon Resource Name (ARN) of the HDFS location.
region	string	AWS region.

For more information, see AWS::DataSync::LocationHDFS.

Methods

Name	Resource	Accessible by	Required Params
create_resource	location_hdfs	INSERT	NameNodes, AuthenticationType, AgentArns, region
delete_resource	location_hdfs	DELETE	Identifier, region
update_resource	location_hdfs	UPDATE	Identifier, PatchDocument, region
list_resources	location_hdfs_list_only	SELECT	region
get_resource	location_hdfs	SELECT	Identifier, region

SELECT examples

get (all properties)

Gets all properties from an individual location_hdf.

SELECT
  region,
  name_nodes,
  block_size,
  replication_factor,
  kms_key_provider_uri,
  qop_configuration,
  authentication_type,
  simple_user,
  kerberos_principal,
  kerberos_keytab,
  kerberos_krb5_conf,
  tags,
  agent_arns,
  subdirectory,
  location_arn,
  location_uri
FROM awscc.datasync.location_hdfs
WHERE
  region = 'us-east-1' AND
  Identifier = '{{ location_arn }}';

list (identifiers only)

Lists all location_hdfs in a region.

SELECT
  region,
  location_arn
FROM awscc.datasync.location_hdfs_list_only
WHERE
  region = 'us-east-1';

INSERT example

Use the following StackQL query and manifest file to create a new location_hdf resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO awscc.datasync.location_hdfs (
  NameNodes,
  AuthenticationType,
  AgentArns,
  region
)
SELECT
  '{{ name_nodes }}',
  '{{ authentication_type }}',
  '{{ agent_arns }}',
  '{{ region }}';

All Properties

/*+ create */
INSERT INTO awscc.datasync.location_hdfs (
  NameNodes,
  BlockSize,
  ReplicationFactor,
  KmsKeyProviderUri,
  QopConfiguration,
  AuthenticationType,
  SimpleUser,
  KerberosPrincipal,
  KerberosKeytab,
  KerberosKrb5Conf,
  Tags,
  AgentArns,
  Subdirectory,
  region
)
SELECT
  '{{ name_nodes }}',
  '{{ block_size }}',
  '{{ replication_factor }}',
  '{{ kms_key_provider_uri }}',
  '{{ qop_configuration }}',
  '{{ authentication_type }}',
  '{{ simple_user }}',
  '{{ kerberos_principal }}',
  '{{ kerberos_keytab }}',
  '{{ kerberos_krb5_conf }}',
  '{{ tags }}',
  '{{ agent_arns }}',
  '{{ subdirectory }}',
  '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: location_hdf
    props:
      - name: name_nodes
        value:
          - hostname: '{{ hostname }}'
            port: '{{ port }}'
      - name: block_size
        value: '{{ block_size }}'
      - name: replication_factor
        value: '{{ replication_factor }}'
      - name: kms_key_provider_uri
        value: '{{ kms_key_provider_uri }}'
      - name: qop_configuration
        value:
          rpc_protection: '{{ rpc_protection }}'
          data_transfer_protection: '{{ data_transfer_protection }}'
      - name: authentication_type
        value: '{{ authentication_type }}'
      - name: simple_user
        value: '{{ simple_user }}'
      - name: kerberos_principal
        value: '{{ kerberos_principal }}'
      - name: kerberos_keytab
        value: '{{ kerberos_keytab }}'
      - name: kerberos_krb5_conf
        value: '{{ kerberos_krb5_conf }}'
      - name: tags
        value:
          - key: '{{ key }}'
            value: '{{ value }}'
      - name: agent_arns
        value:
          - '{{ agent_arns[0] }}'
      - name: subdirectory
        value: '{{ subdirectory }}'

UPDATE example

Use the following StackQL query and manifest file to update a location_hdf resource, using stack-deploy.

/*+ update */
UPDATE awscc.datasync.location_hdfs
SET PatchDocument = string('{{ {
    "NameNodes": name_nodes,
    "BlockSize": block_size,
    "ReplicationFactor": replication_factor,
    "KmsKeyProviderUri": kms_key_provider_uri,
    "QopConfiguration": qop_configuration,
    "AuthenticationType": authentication_type,
    "SimpleUser": simple_user,
    "KerberosPrincipal": kerberos_principal,
    "KerberosKeytab": kerberos_keytab,
    "KerberosKrb5Conf": kerberos_krb5_conf,
    "Tags": tags,
    "AgentArns": agent_arns,
    "Subdirectory": subdirectory
} | generate_patch_document }}')
WHERE
  region = '{{ region }}' AND
  Identifier = '{{ location_arn }}';

DELETE example

/*+ delete */
DELETE FROM awscc.datasync.location_hdfs
WHERE
  Identifier = '{{ location_arn }}' AND
  region = 'us-east-1';

Permissions

To operate on the location_hdfs resource, the following permissions are required:

Create

datasync:CreateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:TagResource,
datasync:ListTagsForResource

Read

datasync:DescribeLocationHdfs,
datasync:ListTagsForResource

Update

datasync:UpdateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:ListTagsForResource,
datasync:TagResource,
datasync:UntagResource

Delete

datasync:DeleteLocation

List

datasync:ListLocations