Monday, November 27, 2017

Scheduling Recurring Jobs - Best Practices

This article aims to briefly present the best practices and suggest tools to plan and monitor recurring jobs with Activeeon’s solution. The main concerns are addressed through different features and services:

  • Schedule management through the Job Planner service,
  • Workflow validity and management through the Catalog,
  • Notification on event with an integrated feature to be more proactive,
  • Requirements checks prior to execution through a selection script to avoid unnecessary issues.

Job Planner

The Job Planner is a service included in ProActive to manage recurring jobs. The main benefits are:

  • Dedicated and centralized interface,
  • Clear forecast of workflows,
  • Simplified management of exceptions: additional executions and/or exclusion periods.

Generate cron expressions

If CRON expressions are complex for you, please visit this webpage which will help you generate one.

To check the meaning behind cron expressions, crontab.guru is a great website translating a cron expression into readable text.

Build an ICS calendar for exceptions

Multiple options are available depending on your environment and company policies.

Manually

This website enables you to create ics based on events that you create through their web interface.

Other calendar clients and servers also give the possibility to export a calendar. You can then create a custom calendar, download it and give it access to the job planner.

Google Calendar

This solution is ideal for G Suite users. Calendars can easily be shared and roles can be assigned for read and write purposes. It is also simple to expose a calendar through a url and get a constant connection with the job planner.

Outlook

For Outlook, depending on your version multiple options are available to expose your calendar. The goal is to make sure the job planner can access it. Multiple documentation and tutorial are available online such as this one or this one.

Private Calendar Server

Finally, it is also possible to get the same behavior as a Google Calendar with a private calendar server.

As an example, here is some open source code provided by Apple and here is another provided by Owncloud.

How to

An article as already been posted to give an example on how to setup the job planner. Do not hesitate to check it here. For the official documentation, please follow this url

The web UI is in progress. In the meantime an endpoint has been available for advance interactions. Check out the swagger to understand how it works and test it.

Important note: Do not forget to refresh the job planner when an update is done to your calendar. Tip: This process can also be automated with a job planner workflows ;)

The Job Planner is consequently the ideal service to manage recurring jobs and easily plan exceptions.

Catalog

The Catalog is a service included in ProActive to manage revisions of workflows. The main benefits are:

  • dedicated and centralized interface to organize workflows,
  • System for revision control,
  • Share functionality between users within selected group,
  • Workflow exposed via url.

Organization and RBAC (Role Based Access Control)

In the catalog the workflows are organized in buckets.

Each bucket belong to a group of users. Each member of the group can read and edit the workflows stored within the bucket. For more information, check out the documentation.

Revision

In this shared environments, keeping track of the changes is required. The service includes a revision system with commit messages to understand what has been done. It is possible to roll back to an older version if the current version of the workflow includes multiple bugs.

Two web interfaces are available to manage revisions and update workflows. For more information, click here.

Bulk import

The latest versions of ProActive include a script which is launched at start time which will automatically import the workflows according to some metadata.

For a concrete example, the default workflows within the buckets are following this principle. Use the instructions on this github to create a bulk import.

External Interface

The Catalog automatically exposes workflows. This enable users to integrate those with other services such as the Job Planner.

Two url structures are available. The first one takes into account the revision and commit time such as:
https://try.activeeon.com/catalog/buckets/1000/resources/2_minutes_cron/revisions/1510835710880/raw
And the second will take the latest revision available for a specific workflow:
https://try.activeeon.com/catalog/buckets/1000/resources/2_minutes_cron/raw

How To

This previous article show you the main Rest calls that can be used to manage the workflows, buckets and revisions.

For more information, the documentation is available here.

Notification on Errors

The Notification feature enables sysadmins to be notified on specific events:

  • Job change priority,
  • Job In-Error,
  • Job paused,
  • Job pending to finished,
  • Job pending to running,
  • Job restarted from error,
  • Job resumed,
  • Job running to finished,
  • Job submitted.
The configuration is simple and would bring proactivity to the solution. For instance, an error can automatically trigger an email to the relevant team.

How To

Follow the documentation section to quickly setup notifications.

Selection Script

The selection script feature enable to perform some checks prior to execute the task. For example it could be relevant to ensure the resource can connect to a selected database or ensure that a file is available on the system.

This is particularly useful to ensure that task that should have been done externally have been correctly completed.

A few examples are available in the following folder: PROACTIVE_HOME/samples/scripts/selection. For those using the try version online find some examples on github

How To

For more information, follow the documentation.



For more information or an actual demo, please, do not hesitate to come back to us and send us an email!

No comments:

Post a Comment