Skip to main content

SparkCustomPool

Properties

PropertyTypeAttributesDescription
autoScaleSparkCustomPoolAutoScaleRequiredAuto-scale properties
dynamicExecutorAllocationSparkCustomPoolDynamicExecutorAllocationRequiredDynamic executor allocation properties
namestringRequiredThe Spark Custom Pool name
nodeFamily'MemoryOptimized'RequiredThe node family
nodeSize'Small' | 'Medium' | 'Large' | 'XLarge' | 'XXLarge'RequiredThe node size
type'Workspace'RequiredThe Spark Custom Pool type
workspaceIdstringRequired, IdentifierThe Workspace ID
idstringRead-only, IdentifierThe Spark Custom Pool ID

SparkCustomPoolAutoScale

PropertyTypeAttributesDescription
enabledboolRequiredThe status of the auto scale
maxNodeCountintRequiredThe maximum node count
minNodeCountintRequiredThe minimum node count

SparkCustomPoolDynamicExecutorAllocation

PropertyTypeAttributesDescription
enabledboolRequiredThe status of the dynamic executor allocation
maxExecutorsintThe maximum executors. Required when enabled is true
minExecutorsintThe minimum executors. Required when enabled is true