YaWL specification
Workflow
For the workflow JSON schema, visit our GitHub repo
Field name | Type | Required | Description |
---|---|---|---|
yawl |
string |
Yes | Specification language version. Possible values: 1.0 . |
start |
string |
Yes | ID of the step to start off the workflow execution. |
defaultRetryPolicy |
RetryPolicy | No | Retry policy applied by default to any step throwing an error during execution. |
steps |
map<string, Step> |
Yes | Description of workflow steps. Object where key is the step ID selected by the user, and value is the object describing the step parameters. |
Step object
Field name | Type | Required | Description |
---|---|---|---|
title |
string |
No | Step name. |
description |
string |
No | Step description. |
<step_type> |
string(FunctionCall| ContainerCall| HTTPCall| GRPCCall| YDBDocument| YDS| YMQ| FoundationModelsCall| ObjectStorage| Switch| Foreach| Parallel| Success| Fail| NoOp) |
Yes | Step specification. Possible parameters depend on selected <step_type> . |
Integration steps
Common fields
The fields described herein are available for all integration steps.
Field name | Type | Required | Default value | Description |
---|---|---|---|---|
input |
string |
No | Overall state of the workflow | A jq expression to filter the workflow state fed into the step. |
output |
string |
No | Step outputs | A jq expression to filter the step outputs added into the workflow state. |
next |
string |
No | No | ID of the next step. |
retryPolicy |
RetryPolicy | No | defaultRetryPolicy , if set on the workflow level |
Retry policy applied if a step throws an error during execution. |
timeout |
Duration |
No | 15 minutes | Maximum step execution time. |
RetryPolicy object
Field name | Type | Required | Default value | Description |
---|---|---|---|---|
errorList |
WorkflowError[] |
Yes | [] |
List of errors for which the step will be retried. For more information, see Possible error codes. |
errorListMode |
INCLUDE/EXCLUDE |
No | INCLUDE |
Error selection mode: INCLUDE to retry on errors listed in error_list ; EXCLUDE to retry on any error other than those listed in error_list . |
initialDelay |
Duration |
No | 1s |
Initial value for a delay between retries. |
backoffRate |
double |
No | 1.0 |
Multiplier for time between each next retry. |
retryCount |
int |
No | 0 |
Maximum number of retries. |
maxDelay |
Duration |
No | 1s |
Maximum delay between retries. |
HTTPCall
HTTP call to a specified endpoint.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
url |
string |
Yes | No | Yes | Request URL. |
method |
enum |
No | GET |
No | Request method. |
body |
string |
No | "" |
Yes | Request body. |
headers |
map<string, string> |
No | {} |
Yes: in header values | Request headers. |
query |
map<string, string> |
No | {} |
Yes: in query parameter values | Request query parameters. |
GRPCCall
Only for services with gRPC reflection
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
endpoint |
string |
Yes | No | Yes | Server address |
method |
string |
Yes | No | Yes | gRPC service and method. |
useServiceAccount |
bool |
No | false |
No | If true , the IAM token of the service account specified in the workflow settings will be added to request headers. |
body |
string |
No | "" |
Yes | Request body. |
headers |
map<string, string> |
No | {} |
Yes: in header values | Request headers. |
YMQ
Sending messages to the Yandex Message Queue queue.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
queueArn |
string |
Yes | No | No | ARN of the queue. |
put |
YmqPut | Yes | No | No | Configuring the put action to add messages to the queue. |
YmqPut object
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
body |
string |
No | "" |
Yes | Message body. |
YDS
Sending messages to the Yandex Data Streams topic.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
database |
string |
Yes | No | No | Database ID. |
topic |
string |
Yes | No | No | Topic ID. |
put |
object |
Yes | YdsPut | No | Configuring the put action to send messages to the data stream. |
YdsPut object
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
body |
string |
No | "" |
Yes | Message body. |
partitionKey |
string |
No | "" |
Yes | Shard key. |
YDBDocument
Interacting with document tables in the Yandex Managed Service for YDB database. The get
, put
, and update
fields are mutually exclusive: you can use only one of them.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
database |
string |
Yes | No | No | Database ID. |
tableName |
string |
Yes | No | No | Table name. |
get |
YdbDocumentGet | No | No | No | Configuring the get action to select entries from the table. |
put |
YdbDocumentPut | No | No | No | Configuring the put action to add entries to the table. |
update |
YdbDocumentUpdate | No | No | No | Configuring the update action to update entries in the table. |
YdbDocumentGet
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
key |
string |
Yes | No | Yes | Primary key value for a database item. |
YdbDocumentPut
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
item |
string |
Yes | No | Yes | JSON-serialized data item to insert into a table. |
YdbDocumentUpdate
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
key |
string |
Yes | No | Yes | Primary key value for a database item. |
expression |
string |
Yes | No | Yes | Expression to describe the updates to the attributes of a database item. For more information, see UpdateExpression. |
expressionAttributeValues |
string |
No | "" |
Yes | Values for attributes used in the expression. For more information, see ExpressionAttributeValues. |
FunctionCall
Invoking the Yandex Cloud Functions function.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
functionId |
string |
Yes | No | No | Function ID. |
ContainerCall
Invoking the Yandex Serverless Containers container.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
containerId |
string |
Yes | No | No | Container ID. |
path |
string |
No | "" |
Yes | Request path. |
method |
enum |
No | GET |
Yes | HTTP method of the request. |
body |
string |
No | "" |
Yes | Request body. |
headers |
map<string, string> |
No | {} |
Yes: in header values | Request headers. |
query |
map<string, string> |
No | {} |
Yes: in query parameter values | Request query parameters. |
FoundationModelsCall
Integration with Yandex Foundation Models. Currently, the only supported integration is the one with YandexGPT API for text generation.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
modelUrl |
string |
Yes | No | No | ID of the model to call. |
generate |
FoundationModelsCallGenerate | Yes | No | No | Configuring the generate action to generate a text. |
FoundationModelsCallGenerate object
The json
and messages
are mutually exclusive: you can either specify a JSON string or explicitly list the messages.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
temperature |
double |
No | 0 |
No | With a higher temperature, you get a more creative and randomized response from the model. This parameter accepts values between 0 and 1, inclusive. |
maxTokens |
int64 |
Yes | No | No | Maximum number of tokens to generate. Allows limiting the model's response if needed. |
json |
string |
No | "" |
Yes | Context for the model, as a JSON string. For more information, see the messages field description in the Foundation Models documentation. |
messages |
FoundationModelsCallGenerateMessage[] | No | "" |
No | Context for the model, as a list of input messages. |
FoundationModelsCallGenerateMessage object
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
role |
string |
Yes | No | Yes | Message sender ID. For more information, see TextGeneration.completion. |
text |
string |
Yes | No | Yes | Message text. For more information, see TextGeneration.completion. |
ObjectStorage
Interacting with the Yandex Object Storage objects. The put
and get
fields are mutually exclusive: you can perform only one of these actions on an object.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
bucket |
string |
Yes | No | No | Bucket name. |
object |
string |
Yes | No | Yes | Object name. |
put |
ObjectStoragePut | No | No | No | Configuring the put action to add the object to the bucket. |
get |
{} |
No | No | No | Configuring the get action to get the object from the bucket. |
ObjectStoragePut object
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
content |
string |
Yes | No | Yes | Object contents. |
Control steps
Switch
Selecting the further execution path. Only one path can be selected: the one for which the condition first returns true
. If all conditions have returned false
, and no value is set in the default
field, the run will be terminated with an error.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
input |
string |
No | Overall state of the workflow | Yes | A jq expression to filter the workflow state fed into the step. |
choices |
Choice[] | Yes | No | No | List of possible further execution paths. |
default |
string |
No | No | No | ID of the step to execute if none of the conditions specified in choices return true . |
Choice object
Field name | Type | Required | Description |
---|---|---|---|
condition |
string |
Yes | Condition in the form of a jq expression that returns either true or false string. |
next |
string |
Yes | ID of the step to execute if the condition returns true . |
Foreach
Executes the sequence of steps specified in do
on each input data item. Outputs are an array of execution results of the steps specified in do
. In the next
field in do
, only steps listed in do
can be specified. Read more about the workflow state during the Foreach step.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
input |
string |
Yes | No | Yes | A jq expression that forms an array of objects. If the result is not an array of objects, the run will be terminated with an error. |
output |
string |
Yes | No | Yes | A jq expression that forms an object to contain the foreach output. If the result is not an object, the run will be terminated with an error. |
do |
ForeachDo | Yes | No | No | Sequence of steps to apply to each input data item. |
next |
string |
No | No | No | ID of the next step. |
ForeachDo object
Field name | Type | Required | Description |
---|---|---|---|
start |
string |
Yes | ID of the step to start the execution from. |
steps |
map<string, Step> |
Yes | Description of the steps. Object where key is the step ID, and value is the object describing the step parameters. The structure is similar to the steps field in the high-level specification. |
Parallel
Executes multiple branches (sequences of steps) concurrently. Execution result is an object where key is the execution branch name, and value is the execution branch outputs. Read more about the workflow state during the Parallel step.
Field name | Type | Required | Default value | Templating is supported | Description |
---|---|---|---|---|---|
input |
string |
No | Overall state of the workflow | Yes | A jq expression to filter the workflow state fed into the step. |
output |
string |
No | Step output data | Yes | A jq expression to filter the step outputs added into the workflow state. |
branches |
map<string, Branch> |
Yes | No | No | Object containing description of execution branches. Key: branch ID; value: description of steps in the branch. |
next |
string |
No | No | No | ID of the next step. |
Branch object
Field name | Type | Required | Description |
---|---|---|---|
start |
string |
Yes | ID of the step to start off the branch execution. |
steps |
map<string, Step> |
Yes | Description of steps in the execution branch. Object where key is the step ID, and value is the object describing the step parameters. The structure is similar to the steps field in the high-level specification. |
Success
Successfully completes the workflow. If placed inside Foreach or Parallel, terminates the whole run, not just its current branch.
Fail
Terminates the workflow with an error. If placed inside Foreach or Parallel, it will terminate the whole workflow, not just its current branch.
Field name | Type | Required | Templating is supported | Description |
---|---|---|---|---|
error |
string |
Yes | Yes | Error message. |
NoOp
This step does nothing. You may need it, e.g., to successfully complete a pipeline if the condition from the default
field is met in Switch, or as a placeholder for one of the steps when prototyping the workflow.
Field name | Type | Required | Templating is supported | Description |
---|---|---|---|---|
output |
string |
No | Yes | A jq expression to filter the step's output data added into the workflow state. |
next |
string |
No | No | ID of the next step. |
Specification example
For the workflow covered by the YaWL specification below, execution is contingent upon input data (input
).
Payload | Result |
---|---|
{"final_action": "success"} |
Success |
{"final_action": "fail"} |
fail now! error |
Other inputs | Error code: STEP_NO_CHOICE_MATCHED, message: no condition is true, and there is no default |
YaWL specification
yawl: "0.1"
start: parallel_step
steps:
parallel_step:
parallel:
next: join_post_and_users
branches:
fetch_posts_branch:
start: fetch_posts
steps:
fetch_posts:
httpCall:
url: https://jsonplaceholder.typicode.com/posts
method: GET
next: filter_posts
output: '\({"posts": .})'
filter_posts:
functionCall:
functionId: b09kpe9j2c5l********
retryPolicy:
errorList:
- HTTP_CALL_502
input: |-
\({
"posts": .posts,
"action": "filter"
})
fetch_users_branch:
start: fetch_users
steps:
fetch_users:
httpCall:
url: https://jsonplaceholder.typicode.com/users
method: GET
retryPolicy:
errorList:
- HTTP_CALL_500
- HTTP_CALL_502
- HTTP_CALL_429
backoffRate: 2.0
initialDelay: 2s
retryCount: 5
output: '\({"users": .})'
join_post_and_users:
functionCall:
next: crop_long_posts
functionId: b09kpe9j2c5l5********
input: |-
\({
"posts": .fetch_posts_branch.posts,
"users": .fetch_users_branch.users,
"action": "join"
})
crop_long_posts:
foreach:
next: grpc_call
input: \(.user_posts)
do:
start: filter_long_posts
steps:
filter_long_posts:
switch:
choices:
- condition: |-
.body | length > 160
next: call_crop_long_posts
default:
next: do_nothing
call_crop_long_posts:
containerCall:
containerId: flh16b3vmu3n********
body: |-
\({
"user_post": .,
"action": "crop"
})
do_nothing:
noOp:
next: ymq_write
ymq_write:
ymq:
put:
body: |-
\(.)
queueArn: "yrn:yc:ymq:ru-central1:aoehdt6d6hbk********:test-queue"
output: |-
\({
"queue_res": .
})
output: |-
\({
"user_posts": .
})
grpc_call:
grpcCall:
useServiceAccount: true
endpoint: 'serverless-functions.api.cloud.yandex.net:443'
method: yandex.cloud.serverless.functions.v1.FunctionService/List
body: |-
\({
"folder_id": "aoehdt6d6hbk********"
})
next: ydb_call
ydb_call:
ydbDocument:
database: "/ru-central1/aoedgvjds14c********/cc8dg7eqfuod********"
update:
key: |-
\({
"x": 123
})
expression: |-
SET name = :name
expressionAttributeValues: |-
\({
":name": "myname1"
})
tableName: "doc-table"
next: yagpt_call
yagpt_call:
foundationModelsCall:
modelUrl: gpt://aoehdt6d6hbk********/yandexgpt/latest
generate:
maxTokens: 100
messages:
messages:
- role: system
text: "Define the language of this text"
- role: user
text: \(.posts.[0].body)
next: yds_step
yds_step:
yds:
topic: test-topic
database: /ru-central1/aoedgvjds14c********/cc8dg7eqfuod********
put:
body: "Hello world!"
partitionKey: \(. | tostring | length)
next: storage_step
storage_step:
objectStorage:
bucket: werelaxe-public-bucket
object: file
put:
content: \(.)
next: final_parallel
final_parallel:
parallel:
branches:
wait_branch:
start: wait_call
steps:
wait_call:
functionCall:
functionId: b09kpe9j2c5l********
input: |-
\({
"action": "wait",
"delay": 10
})
terminate_branch:
start: terminate_switch
steps:
terminate_switch:
switch:
choices:
- condition: .final_action == "success"
next: success_step
- condition: .final_action == "fail"
next: fail_step
success_step:
success: {}
fail_step:
fail:
errorMessage: "fail now!"
output: \(.terminate_branch.fetch_posts_branch)