SparkCustomPool
Properties
| Property | Type | Attributes | Description |
|---|---|---|---|
autoScale | SparkCustomPoolAutoScale | Required | Auto-scale properties |
dynamicExecutorAllocation | SparkCustomPoolDynamicExecutorAllocation | Required | Dynamic executor allocation properties |
name | string | Required | The Spark Custom Pool name |
nodeFamily | 'MemoryOptimized' | Required | The node family |
nodeSize | 'Small' | 'Medium' | 'Large' | 'XLarge' | 'XXLarge' | Required | The node size |
type | 'Workspace' | Required | The Spark Custom Pool type |
workspaceId | string | Required, Identifier | The Workspace ID |
id | string | Read-only, Identifier | The Spark Custom Pool ID |
SparkCustomPoolAutoScale
| Property | Type | Attributes | Description |
|---|---|---|---|
enabled | bool | Required | The status of the auto scale |
maxNodeCount | int | Required | The maximum node count |
minNodeCount | int | Required | The minimum node count |
SparkCustomPoolDynamicExecutorAllocation
| Property | Type | Attributes | Description |
|---|---|---|---|
enabled | bool | Required | The status of the dynamic executor allocation |
maxExecutors | int | The maximum executors. Required when enabled is true | |
minExecutors | int | The minimum executors. Required when enabled is true |