OutputDatasetConfig Class
Represent how to copy the output of a job to a remote storage location and be promoted to a Dataset.
This is the base class used to represent how to copy the output of a job to a remote storage location, whether to register it as a named and versioned Dataset, and whether to apply any additional transformations to the Dataset that was created.
You should not be creating instances of this class directly but instead should use the appropriate subclass.
Initialize a OutputDatasetConfig.
- Inheritance
-
builtins.objectOutputDatasetConfig
Constructor
OutputDatasetConfig(mode, name=None, **kwargs)Parameters
| Name | Description |
|---|---|
|
mode
Required
|
The mode in which to copy the output to the remote storage. |
|
name
Required
|
The name of the output specific to the run it will be produced in. |
Methods
| as_input |
Specify how to consume the output as an input in subsequent pipeline steps. |
| register_on_complete |
Register the output as a new version of a named Dataset after the run has ran. If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset. |
as_input
Specify how to consume the output as an input in subsequent pipeline steps.
as_input(name=None)Parameters
| Name | Description |
|---|---|
|
name
|
The name of the input specific to the run. Default value: None
|
Returns
| Type | Description |
|---|---|
|
A DatasetConsumptionConfig instance describing how to deliver the input data. |
register_on_complete
Register the output as a new version of a named Dataset after the run has ran.
If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset.
register_on_complete(name, description=None, tags=None)Parameters
| Name | Description |
|---|---|
|
name
Required
|
The Dataset name to register the output under. |
|
description
Required
|
The description for the Dataset. |
|
tags
Required
|
A list of tags to be assigned to the Dataset. |
Returns
| Type | Description |
|---|---|
|
A new OutputDatasetConfig instance with the registration information. |
Attributes
name
Name of the output.
Returns
| Type | Description |
|---|---|
|
Name of the output. |