Have you ever encountered how difficult it could be to manage variables on a workflow that can be distributed across multiple resources? How to manage the dependency? How to get information in and out of a container or through firewalls?
For those questions, Activeeon offers multiple solutions that adapt to all your use cases. You will learn here about the main ways to share variables between tasks whether or not they are executed within a container, behind a firewall or in different resource providers.
Result variables
The first variables of interest are the results and result variables. They enable you to send information from one task to its children.
Anywhere in a task (usually at the end) you can set a value to a reserved variable named result. This value will be available in tasks directly depending on it from the variable results (which is an array).
For advanced users, the variable results will contain the content of the result variables in the same order it is declared in the dependency list in the xml file.
Example:
Parent tasks: result="MY_VAR"
Child tasks: results[i] (results[0], results[1], … depending on the number of parent tasks)
Doc: Result variables
Job variables / Input variables
One of the key elements of a good workflow solution is the ability to build generic templates that adapts to input variables.
Those variables are available everywhere in the workflow. They can be changed within the workflow execution.
Those variables can have a default value, a model that define their type and can be overriden on job submission via the GUI, CLI or the Rest API.
Example:
Any tasks: variables.get(“JOB_VARIABLE_TEST”)
Doc: Job variables
Script variables / Variables within workflow
You may also wonder how to create a variable within a task and make it available in the overall workflow. In that case, ProActive offers script variables with behave the same way as job variables.
Nothing more to add, it is that simple :)
Or, maybe… There is one thing you may want to know. For script variables, only dependent tasks can edit the same script variable, you need to be aware of this while developing your workflow.
And another thing you may be interested in. The variables are updated only upon task successful completion. If a variable is set at the beginning of one-hour task, this variable will be available only when this task completes properly. (For more advanced users, if you manually throw an error, this is considered as a successful task completion. Here is a workflow to test it.)
Example:
Any tasks: variables.put(“JOB_VARIABLE_TEST”, “some value”)
Any tasks: variables.get(“JOB_VARIABLE_TEST”)
Doc: Script variables
Task variables
Task variables are similar to job variables except that their scope is limited to a single task.
Overall, task variables override job variables, this means that if a job variable and a task variable are defined with the same name, the task variable value will be used inside the task, and the job variable value will be used elsewhere in the job.
However, the task variables have one specific feature regarding inheritance. Indeed, they can choose to inherit their value from job variables if it exists. In that case any change made to the variable within the task will not be visible elsewhere.
Watch out, if a task variable is edited within the task, it will become a job variable and other tasks with have access to it. To avoid any conflict, it is a good practice to use the result variable.
For a complete test of task variables, download, import this workflow and observer the results.
The task variables are particularly relevant to build generic tasks that can be reuse when building new workflow. For instance, they are used to build database connectors and gather all the input parameters required for a correct execution.
Doc: Task variables
Third-party credentials
The credentials are set in the scheduler within the user account. They are accessible by all workflows executed by this user. Two users can consequently execute the same workflow but have different behaviors.
Their content is encrypted until the execution of the tasks. You can use it to store database credentials and password that may be relevant for your tests.
Example:
From any task in any workflow: mysqlPassword = credentials.get('MY_MYSQL_PASSWORD')
Doc: Credentials
Generic Information
In addition to variables, the generic information is another system of key/value storage available within ProActive. Their value cannot be modified.
In general, the most common use case for generic information is to share information with the scheduler. It can for instance impact the scheduling loop with the START_AT generic information key.
They can also be used to provide metadata on the workflow. For instance, users can add a link to the documentation or setup relevant parameters to be used in the catalog (icon, project, etc.).
Doc: Generic information
Summary
In conclusion, to fit your use cases, Activeeon has developed multiple type of variables. It is key to really understand the scope and behavior of each of them to ensure consistent behavior and build standard templates.
If you want a quick summary, you can check the table below.
| Result variables | Job variables | Script variables | Task variables | Credentials | Generic information | |
| Scope | task and child tasks | any tasks | any tasks | a task | any tasks | NA |
| Variables | results and result | custom name | custom name | custom name | custom name | custom name |
| Context | Share a variable with child tasks |
Input parameters | Variables global to the job | Most commonly used to build generic workflows |
Encrypted variables | Workflow metadata |
To go further, you can check how to use the variable models to ensure your variables respect a certain pattern or learn how to use the variables mentioned above within the different languages provided my ProActive.
No comments:
Post a Comment