Workflow
A workflow is an event-driven application comprised of integration and management steps and transitions between them. Possible step attributes:
- Name and description.
- Filters for input and output data, i.e., jq templates that filter the workflow state either provided to or resulting from the step.
- Actions to perform within a step.
- Execution timeout.
- Retry policy applied if the step throws an error. If a step cannot be completed due to running out of quota or ends with an error, Workflows will retry the step according to the retry policy. Eventually the step may succeed or end with an error if the last execution attempt fails. If the step ends with an error, the whole execution gets the
Failedstatus and no further steps are performed.
The steps are described:
- In the YaWL specification. The description of each step, except for the last one, must name the next step.
- Using the specification constructor.
In the workflow settings, you can:
- Specify a user network to run the workflow in. The workflow will have access to resources in this network.
- Specify a service account to use to access private resources, such as Yandex Data Streams data streams.
- Enable Public workflow. This will allow any user to run the workflow. See details here.
Workflow state
The initial workflow state is a JSON object with the input key whose value is the workflow input data. If the input data is a JSON object, all keys from the workflow input data are added in addition to the input key.
Examples of input data and initial states of a workflow
| Payload | Workflow state |
|---|---|
{"a": "b", "c": 12} |
{"input": {"a": "b", "c": 12}, "a": "b", "c": 12} |
[1, 2, 3] |
{"input": [1, 2, 3]} |
During the execution:
- The workflow state is delivered to the step as input data (
input). - Output data (
output) of each step is added to the workflow state in the order of executing the steps.
With jq templates, you can filter:
- Workflow state delivered to the next step as input data (
input). - Output data (
output) added to the workflow state.
Note
A workflow state at any given time is a JSON object.
The output data of each step is added to the workflow state by merging it with the current state using top-level keys. Therefore, some of the state data can be overwritten. For example, if before the step execution the state was a JSON object of the form {“numbers“: [1,2,3,4], “strings”: [“a”, “b”, “c”]}, and the step's output was {“strings”: [“d”, “e”]}, the new state will be a JSON object in {“numbers“: [1,2,3,4], “strings”: [“d”, “e”]} format.
The workflow result is represented by the output data of the last step for which the output field value can be set.
Workflow state during the Parallel step
In the Parallel step, the workflow state is copied to each branch and changes independently within each one. Before filtering, the output data of the Parallel step is an object where keys are the unique names of execution branches, and values are the output data of the last steps of those branches.
Workflow state during the Foreach step
After the input data is filtered, the Foreach step accepts at its input a JSON array consisting of JSON objects. After executing a sequence of steps in do, a result is generated for each array element. The result is a JSON object which gets the same position in the array the original JSON object was at. This means that the input and output fields are required for the Foreach step.
input: To convert a state from a JSON object to an array.output: To convert the step execution result from an array into a JSON object that can be added to the workflow state.
Use cases
- Automatic data upload to Yandex SpeechSense using Yandex Workflows
- Configuring Workflows integrated with Yandex Tracker, Yandex Cloud AI Studio, and Yandex Cloud Postbox