Add-AzureHDInsightConfigValues

Syntax

Add-AzureHDInsightConfigValues
   -Config <AzureHDInsightConfig>
   [-Core <Hashtable>]
   [-Yarn <Hashtable>]
   [-Hdfs <Hashtable>]
   [-Hive <AzureHDInsightHiveConfiguration>]
   [-MapReduce <AzureHDInsightMapReduceConfiguration>]
   [-Oozie <AzureHDInsightOozieConfiguration>]
   [-Storm <Hashtable>]
   [-Spark <Hashtable>]
   [-HBase <AzureHDInsightHBaseConfiguration>]
   [-Profile <AzureSMProfile>]
   [<CommonParameters>]

Description

This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.

For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell. For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight. For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.

The Add-AzureHDInsightConfigValues cmdlet adds a Hadoop configuration value customization, such as Core-site.xml or Hive-site.xml, or a Hive shared library customization to an Azure HDInsight cluster configuration.

The cmdlet adds custom configuration values to a specified configuration object. The custom settings are added to the configuration files of the relevant Hadoop services when the cluster is deployed.

Examples

Example 1: Configure a cluster

PS C:\>$HiveConfigValues = New-Object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightHiveConfiguration'
PS C:\> $HiveConfigValues.Configuration = @{ hive.exec.compress.output = true }
PS C:\> $HiveConfigValues.AdditionalLibraries = New-Object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightDefaultStorageAccount'
PS C:\> $HiveConfigValues.AdditionalLibraries.StorageAccountName = "MyStorageAccount.blob.core.windows.net"
PS C:\> $HiveConfigValues.AdditionalLibraries.StorageAccountKey = (Get-AzureStorageKey -StorageAccountName "MyStorageAccount").Primary
PS C:\> $HiveConfigValues.AdditionalLibraries.StorageContainerName = "MySharedLibContainer"
PS C:\> $OozieConfigValues = New-Object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightOozieConfiguration'
PS C:\> $OozieConfigValues.Configuration = @{ hive.exec.compress.output = true }
PS C:\> $MapredConfigValues = New-Object 'Microsoft.WindowsAzure.Management.HDInsight.Cmdlet.DataObjects.AzureHDInsightMapReduceConfiguration'
PS C:\> $MapredConfigValues.Configuration = @{ mapred.map.max.attempts = 2 }
PS C:\> $MapredConfigValues.CapacitySchedulerConfiguration = @{ mapred.capacity-scheduler.init-poll-interval = 1000 }
PS C:\> $Config = New-AzureHDInsightClusterConfig -ClusterSizeInNodes 4
    | Set-AzureHDInsightDefaultStorage -StorageAccountName MyStorageAccount.blob.core.windows.net -StorageAccountKey (Get-AzureStorageKey -StorageAccountName "MyStorageAccount").Primary -StorageContainerName "MyStorageContainer"
    | Add-AzureHDInsightConfigValues -Core @{ io.file.buffer.size = 300000 } -MapReduce $MapredConfigValues -Hive $HiveConfigValues -Oozie $OozieConfigValues
PS C:\> $Config | New-AzureHDInsightCluster -Subscription $SubId -Credential $Creds -Name "MyCluster" -Location "North Europe"

The first command creates a new AzureHDInsightHiveConfiguration object, and then stores it in the $HiveConfigValues variable.

The next five commands create configuration values for Hive and store those values as members of $HiveConfigValues.

The seventh command creates an AzureHDInsightOozieConfiguration object, and then stores it in the $OozieConfigValues variable. The eighth command creates a configuration value for Oozie, and then stores that values as a member of $OozieConfigValues.

The ninth command creates an AzureHDInsightMapReduceConfiguration object, and then stores it in the $MapredConfigValues variable. The next two commands create configuration values for MapReduce and store those values as members of $MapredConfigValues.

The twelfth command uses the New-AzureHDInsightClusterConfig cmdlet to create an HDInsight cluster configuration, and then stores it in the $Config variable. The command uses the pipeline operator to pass $Config to the Set-AzureHDInsightDefaultStorage cmdlet to update the default storage setting and to the Add-AzureHDInsightConfigValues cmdlet to add the new configuration values to the cluster configuration.

The final command uses the pipeline operator to pass $Config to the New-AzureHDInsightCluster cmdlet to create a new HDInsight cluster with the customized settings.

Parameters

-Config

-Core

-HBase

-Hdfs

-Hive

-MapReduce

-Oozie

-Profile

-Spark

-Storm

-Yarn

Related Links

• New-AzureHDInsightCluster
• New-AzureHDInsightClusterConfig
• Set-AzureHDInsightDefaultStorage