Update-AzSynapseSparkPool

Syntax

Update-AzSynapseSparkPool
      [-ResourceGroupName <String>]
      -WorkspaceName <String>
      -Name <String>
      [-Tag <Hashtable>]
      [-EnableAutoScale <Boolean>]
      [-AutoScaleMinNodeCount <Int32>]
      [-AutoScaleMaxNodeCount <Int32>]
      [-EnableAutoPause <Boolean>]
      [-AutoPauseDelayInMinute <Int32>]
      [-NodeCount <Int32>]
      [-EnableIsolatedCompute <Boolean>]
      [-NodeSize <String>]
      [-EnableDynamicExecutorAllocation <Boolean>]
      [-MinExecutorCount <Int32>]
      [-MaxExecutorCount <Int32>]
      [-SparkVersion <String>]
      [-LibraryRequirementsFilePath <String>]
      [-SparkConfiguration <PSSparkConfigurationResource>]
      [-PackageAction <PackageActionType>]
      [-Package <System.Collections.Generic.List`1[Microsoft.Azure.Commands.Synapse.Models.WorkspacePackages.PSSynapseWorkspacePackage]>]
      [-ForceApplySetting]
      [-AsJob]
      [-DefaultProfile <IAzureContextContainer>]
      [-WhatIf]
      [-Confirm]
      [<CommonParameters>]

Update-AzSynapseSparkPool
      -Name <String>
      -WorkspaceObject <PSSynapseWorkspace>
      [-Tag <Hashtable>]
      [-EnableAutoScale <Boolean>]
      [-AutoScaleMinNodeCount <Int32>]
      [-AutoScaleMaxNodeCount <Int32>]
      [-EnableAutoPause <Boolean>]
      [-AutoPauseDelayInMinute <Int32>]
      [-NodeCount <Int32>]
      [-EnableIsolatedCompute <Boolean>]
      [-NodeSize <String>]
      [-EnableDynamicExecutorAllocation <Boolean>]
      [-MinExecutorCount <Int32>]
      [-MaxExecutorCount <Int32>]
      [-SparkVersion <String>]
      [-LibraryRequirementsFilePath <String>]
      [-SparkConfiguration <PSSparkConfigurationResource>]
      [-PackageAction <PackageActionType>]
      [-Package <System.Collections.Generic.List`1[Microsoft.Azure.Commands.Synapse.Models.WorkspacePackages.PSSynapseWorkspacePackage]>]
      [-ForceApplySetting]
      [-AsJob]
      [-DefaultProfile <IAzureContextContainer>]
      [-WhatIf]
      [-Confirm]
      [<CommonParameters>]

Update-AzSynapseSparkPool
      -InputObject <PSSynapseSparkPool>
      [-Tag <Hashtable>]
      [-EnableAutoScale <Boolean>]
      [-AutoScaleMinNodeCount <Int32>]
      [-AutoScaleMaxNodeCount <Int32>]
      [-EnableAutoPause <Boolean>]
      [-AutoPauseDelayInMinute <Int32>]
      [-NodeCount <Int32>]
      [-EnableIsolatedCompute <Boolean>]
      [-NodeSize <String>]
      [-EnableDynamicExecutorAllocation <Boolean>]
      [-MinExecutorCount <Int32>]
      [-MaxExecutorCount <Int32>]
      [-SparkVersion <String>]
      [-LibraryRequirementsFilePath <String>]
      [-SparkConfiguration <PSSparkConfigurationResource>]
      [-PackageAction <PackageActionType>]
      [-Package <System.Collections.Generic.List`1[Microsoft.Azure.Commands.Synapse.Models.WorkspacePackages.PSSynapseWorkspacePackage]>]
      [-ForceApplySetting]
      [-AsJob]
      [-DefaultProfile <IAzureContextContainer>]
      [-WhatIf]
      [-Confirm]
      [<CommonParameters>]

Update-AzSynapseSparkPool
      -ResourceId <String>
      [-Tag <Hashtable>]
      [-EnableAutoScale <Boolean>]
      [-AutoScaleMinNodeCount <Int32>]
      [-AutoScaleMaxNodeCount <Int32>]
      [-EnableAutoPause <Boolean>]
      [-AutoPauseDelayInMinute <Int32>]
      [-NodeCount <Int32>]
      [-EnableIsolatedCompute <Boolean>]
      [-NodeSize <String>]
      [-EnableDynamicExecutorAllocation <Boolean>]
      [-MinExecutorCount <Int32>]
      [-MaxExecutorCount <Int32>]
      [-SparkVersion <String>]
      [-LibraryRequirementsFilePath <String>]
      [-SparkConfiguration <PSSparkConfigurationResource>]
      [-PackageAction <PackageActionType>]
      [-Package <System.Collections.Generic.List`1[Microsoft.Azure.Commands.Synapse.Models.WorkspacePackages.PSSynapseWorkspacePackage]>]
      [-ForceApplySetting]
      [-AsJob]
      [-DefaultProfile <IAzureContextContainer>]
      [-WhatIf]
      [-Confirm]
      [<CommonParameters>]

Description

The Update-AzSynapseSparkPool cmdlet updates an Apache Spark pool in Azure Synapse Analytics.

Examples

Example 1

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -Tag @{"key" = "value"} -NodeCount 5 -NodeSize Medium

This command updates an Apache Spark pool in Azure Synapse Analytics.

Example 2

$pool = Get-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$pool | Update-AzSynapseSparkPool -Tag @{"key" = "value1"}

This command updates an Apache Spark pool in Azure Synapse Analytics through pipeline.

Example 3

$ws = Get-AzSynapseWorkspace -Name ContosoWorkspace
$ws | Update-AzSynapseSparkPool -Name ContosoSparkPool -Tag @{"key" = "value2"}

This command updates an Apache Spark pool in Azure Synapse Analytics through pipeline.

Example 4

Update-AzSynapseSparkPool -ResourceId /subscriptions/a0a0a0a0-bbbb-cccc-dddd-e1e1e1e1e1e1/resourceGroups/ContosoResourceGroup/providers/Microsoft.Synapse/workspaces/ContosoWorkspace/bigDataPools/ContosoSparkPool -Tag @{"key" = "value3"}

This command updates an Apache Spark pool in Azure Synapse Analytics with resource ID.

Example 5

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableAutoScale $true -AutoScaleMinNodeCount 3 -AutoScaleMaxNodeCount 7

This command enables auto-scale for an Apache Spark pool in Azure Synapse Analytics.

Example 6

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableAutoScale $false

This command disables auto-scale for an Apache Spark pool in Azure Synapse Analytics.

Example 7

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableAutoPause $true -AutoPauseDelayInMinute 15

This command enables auto-pause for an Apache Spark pool in Azure Synapse Analytics.

Example 8

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableAutoPause $false

This command disables auto-pause for an Apache Spark pool in Azure Synapse Analytics.

Example 9

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableDynamicExecutorAllocation $true -MinExecutorCount 1 -MaxExecutorCount 5

This command enables dynamic executor allocation and specify min executor count and max executor count for an Apache Spark pool in Azure Synapse Analytics.

Example 10

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableDynamicExecutorAllocation $false

This command disables dynamic executor allocation for an Apache Spark pool in Azure Synapse Analytics.

Example 11

$packages = Get-AzSynapseWorkspacePackage -WorkspaceName ContosoWorkspace
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -PackageAction Add -Package $packages

The first command retrieves workspace packages. The second command links these workspace packages to an Apache Spark pool in Azure Synapse Analytics.

Example 12

$package = Get-AzSynapseWorkspacePackage -WorkspaceName ContosoWorkspace -Name ContosoPackage
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -PackageAction Remove -Package $package

The first command retrieves workspace packages named ContosoPackage. The second command removes the workspace package from an Apache Spark pool in Azure Synapse Analytics.

Example 13

$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$pool | Update-AzSynapseSparkPool -PackageAction Remove -Package $pool.WorkspacePackages

The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The second command removes all workspace packages that are linked to that Apache Spark pool.

Example 14

$workspace_packages = Get-AzSynapseWorkspacePackage -WorkspaceName ContosoWorkspace

$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$library_names = $pool.WorkspacePackages | Where-Object {$_.name -notlike "new_package-*"} | ForEach-Object {$_.name}
$library_names += "new_package-2.0-py3-none-any.whl"

$new_pool_packages = @($workspace_packages | Where-Object {$_.name -in $library_names})
Update-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -PackageAction Set -Package $new_pool_packages

The first command retrieves the packages available in the workspace. The second command group retrieves the spark pool to get the packages currently linked to this pool and removes all versions of the package starting with new_package- from the retrieved list. The new version of the package is then added to this list. In the third group of commands the package list, containing only package names, is tranformed into a list of workspace packages by filtering the list of available workspace_packages accordingly and is then linked to the spark pool.

Example 15

$config = Get-AzSynapseSparkConfiguration -WorkspaceName ContosoWorkspace -Name ContosoSparkConfig1
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -Tag @{"key" = "value"} -NodeCount 5 -NodeSize Medium -SparkConfiguration $configs

This command updates an Apache Spark pool in Azure Synapse Analytics and specify a Spark configuration for the Spark pool.

Example 16

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -NodeSize small -ForceApplySetting

This command updates an Apache Spark pool in Azure Synapse Analytics, set NodeSize to small for the spark pool and force stop any running jobs in the Spark pool to apply this new setting.

Example 17

$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$pool | Update-AzSynapseSparkPool -PackageAction Remove -Package $pool.WorkspacePackages -ForceApplySetting

The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The second command removes all workspace packages that are linked to that Apache Spark pool and force stop any running jobs in the Spark pool to apply this new setting.

Example 18

Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableIsolatedCompute $true -NodeSize XXXLarge

This command enables isolated compute and specify node size to XXXLarge(80 vCPU / 504 GB) for an Apache Spark pool in Azure Synapse Analytics.

Parameters

-AsJob

-AutoPauseDelayInMinute

-AutoScaleMaxNodeCount

-AutoScaleMinNodeCount

-Confirm

-DefaultProfile

-EnableAutoPause

-EnableAutoScale

-EnableDynamicExecutorAllocation

-EnableIsolatedCompute

-ForceApplySetting

-InputObject

-LibraryRequirementsFilePath

-MaxExecutorCount

-MinExecutorCount

-Name

-NodeCount

-NodeSize

-Package

-PackageAction

-ResourceGroupName

-ResourceId

-SparkConfiguration

-SparkVersion

-Tag

-WhatIf

-WorkspaceName

-WorkspaceObject

Inputs

PSSynapseWorkspace

PSSynapseSparkPool

Outputs

PSSynapseSparkPool