Cmdlets for Elasticsearch

Build 25.0.9434

Establishing a Connection

With the CData Cmdlets users can install a data module, set the connection properties, and start scripting. This section provides examples of using our Elasticsearch Cmdlets with native PowerShell cmdlets, like the CSV import and export cmdlets.

Connecting to Elasticsearch Service

Set the following to connect to data:

  • Server should be set to the IP Address or domain of the Elasticsearch instance. The Server could also be set to a comma-delimited list of node addresses or hostnames from a single cluster. 
    Server=01.02.03.04 
  OR 
Server=01.01.01.01:1234,02.02.02.02:5678
  • Port should be set to the configured port for the Elasticsearch instance. If you include a port in a node specification for the Server property, that included port will take precedence over the specification for Port for that node only.

The cmdlet uses X-Pack Security for authentication and TLS/SSL encryption. You can prefix the server value with "https://" to connect using TLS/SSL.

Connecting to Amazon OpenSearch Service

Set the following to connect to data:

  • Server should be set to the Endpoint URL for the Amazon ES instance.
  • Port should be set to 443.
  • AWSRegion should be set to the Amazon AWS region where the Elasticsearch instance is being hosted (the cmdlet will attempt to automatically identify the region based on the Server value).

The cmdlet uses X-Pack Security for authentication and TLS/SSL encryption.

Note: Requests are signed using AWS Signature Version 4.

Authenticating to Elasticsearch

In addition to standard connection properties, select one of the below authentication methods to authenticate.

Obtain AWS Keys

To obtain the credentials for an IAM user:
  1. Sign into the IAM console.
  2. In the navigation pane, select Users.
  3. To create or manage the access keys for a user, select the user and then navigate to the Security Credentials tab.
To obtain the credentials for your AWS root account:
  1. Sign into the AWS Management console with the credentials for your root account.
  2. Select your account name or number.
  3. In the menu that displays, select My Security Credentials.
  4. To manage or create root account access keys, click Continue to Security Credentials and expand the "Access Keys" section.

Standard Authentication

Set the AuthScheme to Basic, and set User and Password properties and/or use PKI (public key infrastructure) to authenticate. Once the cmdlet is connected, X-Pack performs user authentication and grants role permissions based on the realms you have configured.

To use PKI, set the SSLClientCert, SSLClientCertType, SSLClientCertSubject, and SSLClientCertPassword properties.

Note: TLS/SSL and client authentication must be enabled on X-Pack to use PKI.

Securing Elasticsearch Connections

To enable TLS/SSL in the cmdlet, set UseSSL to true;.

Root Credentials

To authenticate using account root credentials, set these parameters:

Note: Amazon discourages the use of this authentication scheme for anything but simple tests. The account root credentials have the full permissions of the user, making this the least secure authentication method.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the cmdlet to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

Temporary Credentials

To authenticate using temporary credentials, specify the following:

The cmdlet can now request resources using the same permissions provided by long-term credentials (such as IAM user credentials) for the lifespan of the temporary credentials.

To authenticate using both temporary credentials and an IAM role, set all the parameters described above, and specify these additional parameters:

  • AWSRoleARN: The Role ARN for the role you'd like to authenticate with. This prompts the cmdlet to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the cmdlet to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

Using AWS From an EC2 Instance

Set AuthScheme to AwsEC2Roles.

If you are using the cmdlet from an EC2 Instance and have an IAM Role assigned to the instance, you can use the IAM Role to authenticate. Since the cmdlet automatically obtains your IAM Role credentials and authenticates with them, it is not necessary to specify AWSAccessKey and AWSSecretKey.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

IMDSv2 Support

The Elasticsearch cmdlet now supports IMDSv2. Unlike IMDSv1, the new version requires an authentication token. Endpoints and response are the same in both versions.

In IMDSv2, the Elasticsearch cmdlet first attempts to retrieve the IMDSv2 metadata token and then uses it to call AWS metadata endpoints. If it is unable to retrieve the token, the cmdlet reverts to IMDSv1.

Note that this method of authentication is only possible with Opensearch Service, and not with Elasticsearch.

AWS IAM Roles

To authenticate through AWS, set AuthScheme to AwsIAMRoles.

To authenticate as an AWS role, set these properties:

  • AWSAccessKey: The access key of the IAM user to assume the role for.
  • AWSSecretKey: The secret key of the IAM user to assume the role for.
  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the cmdlet to attempt to retrieve credentials for the specified role.
  • AWSExternalId (optional): Only required if you are assuming a role in another AWS account.

If multi-factor authentication is required, specify the following:

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.
  • MFASerialNumber: The serial number of the MFA device if one is being used.
  • MFAToken: The temporary token available from your MFA device.
This causes the cmdlet to submit the MFA credentials in the request to retrieve temporary authentication credentials.

Note: If you want to control the duration of the temporary credentials, set the TemporaryTokenDuration property (default: 3600 seconds).

Note: In some circumstances it might be preferable to use an IAM role for authentication, rather than the direct security credentials of an AWS root user. If you are specifying the AWSAccessKey and AWSSecretKey of an AWS root user, you cannot use roles.

Kerberos

Please see Using Kerberos for details on how to authenticate with Kerberos.

API Key

To authenticate using APIKey set the following:

  • AuthScheme: Set this to APIKey.
  • APIKey: Set this to APIKey returned from Elasticsearch.
  • APIKeyId: Set this to the Id returned alongside APIKey.

Creating a Connection Object

You can then use the Connect-Elasticsearch cmdlet to create a connection object that can be passed to other cmdlets: 

$conn = Connect-Elasticsearch -Server '127.0.0.1' -Port 9200

Retrieving Data

The Select-Elasticsearch cmdlet provides a native PowerShell interface for retrieving data: 

$results = Select-Elasticsearch -Connection $conn -Table "[CData].[Elasticsearch].Employee" -Columns @("Id, Name") -Where "Industry='Floppy Disks'"
The Invoke-Elasticsearch cmdlet provides an SQL interface. This cmdlet can be used to execute an SQL query via the Query parameter.

Piping Cmdlet Output

The cmdlets return row objects to the pipeline one row at a time. The following line exports results to a CSV file: 

Select-Elasticsearch -Connection $conn -Table [CData].[Elasticsearch].Employee -Where "Industry = 'Floppy Disks'" | Select -Property * -ExcludeProperty Connection,Table,Columns | Export-Csv -Path c:\my[CData].[Elasticsearch].EmployeeData.csv -NoTypeInformation

You will notice that we piped the results from Select-Elasticsearch into a Select-Object cmdlet and excluded some properties before piping them into an Export-CSV cmdlet. We do this because the CData Cmdlets append Connection, Table, and Columns information onto each row object in the result set, and we do not necessarily want that information in our CSV file.

However, this makes it easy to pipe the output of one cmdlet to another. The following is an example of converting a result set to JSON: 

 
PS C:\> $conn  = Connect-Elasticsearch -Server '127.0.0.1' -Port 9200
PS C:\> $row = Select-Elasticsearch -Connection $conn -Table "[CData].[Elasticsearch].Employee" -Columns (Id, Name) -Where "Industry = 'Floppy Disks'" | select -first 1
PS C:\> $row | ConvertTo-Json
{
  "Connection":  {

  },
  "Table":  "[CData].[Elasticsearch].Employee",
  "Columns":  [

  ],
  "Id":  "MyId",
  "Name":  "MyName"
}

Deleting Data

The following line deletes any records that match the criteria: 

Select-Elasticsearch -Connection $conn -Table [CData].[Elasticsearch].Employee -Where "Industry = 'Floppy Disks'" | Remove-Elasticsearch

Modifying Data

The cmdlets make data transformation easy as well as data cleansing. The following example loads data from a CSV file into Elasticsearch, checking first whether a record already exists and needs to be updated instead of inserted. 

Import-Csv -Path C:\My[CData].[Elasticsearch].EmployeeUpdates.csv | %{
  $record = Select-Elasticsearch -Connection $conn -Table [CData].[Elasticsearch].Employee -Where ("Id = `'"+$_.Id+"`'")
  if($record){
    Update-Elasticsearch -Connection $conn -Table [CData].[Elasticsearch].Employee -Columns @("Id","Name") -Values @($_.Id, $_.Name) -Where "Id  = `'$_.Id`'"
  }else{
    Add-Elasticsearch -Connection $conn -Table [CData].[Elasticsearch].Employee -Columns @("Id","Name") -Values @($_.Id, $_.Name)
  }
}

Copyright (c) 2025 CData Software, Inc. - All rights reserved.
Build 25.0.9434