ProActive workflows and orchestration
The ProActive Scheduler is an open source software to orchestrate tasks in your hybrid cloud infrastructure. With an easy to use GUI and an easy setup on master
and nodes (just executing one sh file or why not getting it in containers
directly) it fits very well into the Docker ecosystem, which I find very user friendly.
Getting Docker Compose support into ProActive
The ProActive software uses the concept of JSR 223 script
engines. I used that fact to implement a Docker Compose executing JSR 223 compatible
script engine, which now drives the Docker Compose support. To add Docker Compose support simply add a jar file to your ProActive installation. You can find the Docker Compose script engine here, but don't worry it is already inside the containers we will use in this walk-through.
Running ProActive
You need to have docker 1.6 and Docker Compose installed, if not:
follow the installation guides for docker and Docker Compose.
Run ProActive with Docker Compose! I prepared a yaml file (download it here):
Run ProActive with Docker Compose! I prepared a yaml file (download it here):
The ‘proactiveScheduler’ start the ProActive master with a web
interface which will be accessible on port 8080. Additionally it exposes the
port 64738, to which the ProActive node will connect. The scheduler manages the nodes and keeps track of the tasks.
The node will execute and monitor tasks, so we need at least one node to execute our Docker Compose tasks.
The node will execute and monitor tasks, so we need at least one node to execute our Docker Compose tasks.
The ‘proactiveNode’ is linked to the proactiveScheduler and
connects with the command ‘-r pnp://proactiveScheduler:64738’; check out the node's and scheduler's dockerfile.
The docker.sock is mounted inside the container to execute containers on your local docker daemon.
The docker.sock is mounted inside the container to execute containers on your local docker daemon.
Now: Execute the yaml file!
Executing a Docker Compose script with the ProActive GUI
After executing the yaml file wait until the web interface is started and go to http://127.0.0.1:8080/studio and log in with the standard account (admin/admin). Click on create, to create a new workflow:
After creating your first workflow click on open (green button):
Now that the workflow is open: import the Docker Compose task by clicking import; download it here.
Note: The Docker Compose script engine is not selectable by the GUI, which will change soon, so for now, the xml files have to be edited in a text editor and imported.
More details: The GUI checks whether the script engine is valid as soon as the script is edited, that will remove the language="docker-compose" in the xml file.
Note: The Docker Compose script engine is not selectable by the GUI, which will change soon, so for now, the xml files have to be edited in a text editor and imported.
More details: The GUI checks whether the script engine is valid as soon as the script is edited, that will remove the language="docker-compose" in the xml file.
After importing you will see a task which contains a Docker Compose script:
It executes (in the dockerfile/ubuntu image), which will print "It worked!!!!" on the screen:
Note: I added the ->sleep 5<- because the echo command returned so fast that Docker Compose tried to attach to the container which had already stopped, that cause it do wait forever.
After clicking execute got to http://127.0.0.1:8080/scheduler/
to see the progress of the Docker Compose script. Select the submitted job and you will see:
After the job is finished you can access the output by
clicking the ‘Fetch output’ button in the Output tab:
Not glamorous but we can see the ‘It worked!!!’ in the line ‘[36mubuntuEcho_1
| [0mIt worked!!!’
Variables in Docker Compose workflows
The new yaml file inside the task is:
- Create a new workflow
- import the xml file (download it here).
Execute the workflow and check the output in the http://127.0.0.1:8080/scheduler/ interface. The variable has been replaced and the the echo will now output the content of the variable.
Follow @tobwiens