Pilot with your Applications (ETL, CI/CD, etc.)

As any software solution nowadays, the key for company success is the ability to integrate all solutions into one. Indeed, as technology evolves, software solutions are becoming more and more specialized and require tight integration between each other to bring true value.

ActiveEon ProActive has been developed with an open approach. The solution is easy to integrate in any architecture and connect to external services. To be more precise, the solution has an comprehensive open Rest API to let any third party application integrate with it. On the other side, tasks can be developed in any language to execute rest calls or simply execute a command line in the relevant host.

Benefits of using ProActive as a pilot

Without going in too much detail, some of the benefits of using ProActive to pilot third party software are:

• Shared resources, allocate resource dynamically depending on each service and application needs,
• Priority, preempt resources from low priority tasks to give them to urgent ones,
• Multi-language, automate business workflows through custom made scripts written in the most suited language,
• Automation of preparation tasks before starting third party services,
• Error handling, monitor and manage errors at a higher level for more control

How simple is it?

Nowadays, most companies are providing ways to connect to their system through API, CLI or SDKs.

• API - If the company is providing an API, a ProActive task will be responsible for connecting and submitting Rest calls to the ETL endpoints. In Groovy, it is as simple as json = ['curl', '-X', 'GET', '--header', '', 'http://api.giphy.com/v1/stickers/search?q='+input+'&api_key=dc6zaTOxFJmzC'].execute().text
• SDK - If the company is providing a library in Java/Python/etc., a ProActive task in the relevant language will be responsible for connecting and submitting relevant request to the ETL service. In that case, the library will have to be loaded within the ProActive folder or using a fork environment such as with Docker.
• CLI - If the company is providing a CLI, a ProActive task will be responsible for connecting and submitting requests to the CLI service. In that case. a selection script may be used to select the relevant host and execute the command within it or as explained above a Docker container with the relevant SDK can be used.

Do not hesitate to try these solutions on our user platform.

Successful integrations at customers

What about ETL

In the case of an ETL, ProActive can be configured to check information before and after starting an ETL job. This would ensure the job will be executed properly.

At Home Office, ProActive is used to pilot Talend one of the leading ETL provider.

Informatica is providing a CLI name PMCMD which can be used to pilot the ETL service. However, many other solutions are available, their store is a great place to select the tools that will support your integration with ProActive.

A simple CI/CD

As anyone knows, CI/CD needs to be fully automated. However, the more automation happens and testing is implemented the more processes and previously standalone scripts now needs to go through the pipeline for more secure deployment.

Even so ProActive does not need to be upgraded on a regular basis, the workflows built also need to go through some stages before being available in production. They may for instance depends on some github dependencies (latest data science algorithm to fetch).

ProActive can also be used in another way. The integration of a resource manager with a scheduler enables the solution to deploy complex environments on demand. The CI/CD pipeline can leverage this capability to, for instance, deploy thousands of nodes in different regions and monitor a service behavior and performance.