Logo
  1. Potens.io
  2. Magnus
  3. Concepts & Components

Articles > Workflow

  • Workflow
  • Parameters
  • Workflows Browser
  • Workflows History
  • Permissions
  • Dashboard

Overview

Scheduled daily in the morning, a sequence of tasks are executed to extract data from Google BigQuery, merge data points with metadata, calculate metrics, loop through attributes, save data to a table, export the data to an FTP server and Google Cloud Storage, email the process owner of process status, and distribute an HTML report via email to a distribution group.  All of these actions are available and made possible because of Magnus Workflow.

Magnus Workflow manages Tasks, scheduling, and ownership of processes that primary bridge the gap between Google BigQuery data and ETL tools.  Workflows can consist of any of the available Tasks in any combination to perform complex, scheduled, automated functions.  Workflows are scheduled by specifying frequency and time period of run.  Workflow control is in the hand of creators who own the process and are able to delegate and share with other users.

Workflows feature shortcuts, search/replace, and parameters to make process design easier.  Shortcuts allow designers to quickly jump to tasks.  Search and Replace features an easy way to update any text within a workflow.  Parameters help generalize values so that they can be used in any task though out the Workflow.

Header

01.2.png

 

A workflow has the following properties:

Workflow ID Unique name of the workflow
Description Used to describe purpose of workflow (optional)
Owner Account under which workflow is run when scheduled
Primarily the user who created workflow
User account is case-sensitive!
Delegates Users who can edit and run workflow
Added by Owner or other Delegates
Optional 
User accounts are case-sensitive
Shared With Users who can run workflow via Execute Workflow Task
Added by Owner or Delegates
Optional 
User accounts are case-sensitive
Lifetime Time period during which scheduled workflow is executed
02.png
Schedule

When to execute a workflow
Requires Lifetime to be set and Workflow to be enabled

Please consider: Try not to schedule on the exact hour, half hour, or quarter hour. 
Spread the time out such as 9:04 or 9:32 instead of 9:00 or 9:30

Optional

Can be set using drop down options
03.png

Or by setting Quartz Cron (Advanced)
04.png

Time Zone

By default, the Workflow is executed in Magnus’ server time zone, which is US/Eastern.  You can change the time zone in which to execute the Workflow.  This affects when the Workflow is executed and the time-related built-in parameters.

If the Workflow with US/Eastern time zone is scheduled to run at each day at 5 AM from 6/1/2018 to 6/2/2018, then it will run on 6/1/2018 and 6/2/2018 at 5 AM US/Eastern time.  The following built-in parameters will reflect the date and time in US/Eastern time zone during the execution: var_date, var_daydate, var_year, var_month, var_day, var_hour, var_yesterday, var_timezone. 
Note: 
- The time zone selected in not applied to Script Task, so you will need to utilize the built-in parameter <var_timezone> in your script body to do any date time manipulations.
- The time zone selected is not inherited by the child Workflow in Execute Workflow Task.  For example, if the parent Workflow has US/Eastern time zone and the child Workflow has US/Pacific time zone, then during execution the parent Workflow will be executed with US/Eastern time zone and thus its built-in parameters will reflect the date and time in US/Eastern time zone, while the child Workflow will be executed with US/Pacific time zone and its built-in parameters will reflect the date and time in US/Pacific time zone.
-The time zone selected only applies to this Workflow. If you want to change the time zone for other Workflows, you will have to update each of them individually.

Enabled Enable/Disable workflow schedule.
Billing Project From

Owner or Delegate User can choose what Billing Project a specific workflow should use

Run-User: use the Magnus-assigned billing project of the user who is running the Workflow. 
Owner: use the Magnus-assigned billing project of the Workflow Owner. 
Custom: Specify a billing project. Parameter is not allowed

Cascade Billing Project

If checked, during workflow execution, the billing project ID that is derived from the setting “Billing Project From” will be passed down to child and descendent workflows. During the execution of the descendent workflows, the cascaded billing project will be used. This applies for descendent workflows that are executed via Execute Workflow Task or Remote Workflow Execution. The user who is running the workflows must have required permission in the cascaded billing project.
For example, if this is a workflow chain w1 > w2 > w3 > w4 > w5, where the top level workflow w1 calls w2, and w2 calls w3, and so on. If you want all 5 workflows to be executed in a custom billing project, then you will set the custom billing project and cascade option in w1.

28.png

Then all 5 workflows will be run under the billing project my-custom-billing-project.

Public Checked: the Workflow is viewable by all users who have same or set to have visibility to Magnus-assigned billing project as the Workflow Owner 
Unchecked: the workflow is viewable only by its owners/delegates/shared users
Save   Save new or modified Workflow
Exit  Exit workflow.  Prompts to save if there are unsaved changes 
Run 

Runs workflow.  Automatically saves if there are unsaved changes

Workflow will run under Owner Account when scheduled

Workflow will run under User Account who Manually Run Workflow 
Workflow Header 

05.png

If clicked twice Tasks will expand/collapse 

Show/Hide Disabled Tasks   06.png Clicking icon will show/hide disabled Tasks
Archived   

Option becomes available when Archive icon is clicked from the Workflows Browser

Check indicates to Archive the workflow on save

Archiving a workflow removes the workflow from listing in Workflow Browser
To see archived workflows check Show Archived from Workflow Browser

 

Left Vertical Toolbar

Toolbar is available when adding new or editing existing workflow

07.png Go back to Magnus home page
08.png Run workflow manually
09.png Save workflow changes
10.png Exit workflow
11.png

Jump to task

Names of tasks in workflow will display in list
Selecting option will jump to task within workflow

12.png

13.png Collapse/Expand all tasks
14.png Hide/Show disabled tasks
15.png Show Workflow Execution History
16.png

Show Workflow Versions
Modification date and the user who did the modification will display in a list.
Click on a particular version to view it
17.png

 

Find and Replace

Text within workflows are searchable and can be replaced by using Find/Replace feature

Find/Replace Collapsed
18.png

Find/Replace Expanded
19.png

Find input box The target keyword to be searched for within workflow
Replace input box Text used to replace found keyword
Match case Checked keyword search is case sensitive
Unchecked keyword search is case insensitive
Match whole word Checked entire keyword matches are displayed
Unchecked partial keyword matches are displayed
Find button

Clicking initiates Find

Results will be displayed

Selections can be checked from Results to determine which matches to Replace

Select all and Unselect all are available for quick selections

20.png


Parameters

Parameters can be used throughout the workflow.
To add a custom parameter, follow the steps below:

1. Click to open Parameters tool 
21.png
22.png

2. Built-in parameters are system defined parameters
3. Custom parameters are user defined parameters

a. Click Add to create custom parameter 
23.png

b. Parameter is always created with var_ prefix 
24.png

c. Parameter can be deleted if x is clicked
d. Parameter type can be string, number, Boolean, or record

Workflow footer
25.png

Add Task container

List of individual tasks that can be added to workflow

Clicking on a specific Task button will add that task to the workflow

Add to position Displays list of existing Tasks from which a selection can be made to insert added Task into that position
Save Save new or modified Workflow
Exit Exit workflow.  Prompts to save if changes have been made
Run

Runs workflow.  Prompts to save if changes have been made

Workflow will run under Owner Account when scheduled

Workflow will run under User Account who Manually Run Workflow
Copy

Creates copy of saved workflow

Available to Owner and Admin

Share Link Workflow shortcut link is a link that is auto generated by the system and can be used to access the workflow immediately from the browser

 

Share Workflow Link (Permalink)

At the bottom of the Workflow Editor, there is a button titled as “Share Link” as shown below:
26.png

If the button was clicked then the system will show a text box that will appear with the shortcut link highlighted/selected: 
27.png

Now, user can copy and use this link to reference this workflow, share it with anyone who entitled 

On-Save Notification 

Any time – someone saves workflow – email notification can be sent to the owner of that workflow and cc to user who saved workflow.

There is a magic parameter available that allows enable/disable this feature for given workflow - var_sys_save_notification_enabled – by default it is false.

To set the recipients, add and set parameter var_sys_save_notification_recipients to a comma-separated list of e-mail addresses (default is workflow owner and user who did the change)

You can add this parameter and set it to false to disable notification

Notification Email will have details on Tasks that were modified , deleted or added

Ownership Change Notification

Magnus administrators have the ability to change workflow ownership.  When there is a change in workflow ownership, previous owner and new owner will be notified via email about the change.

Failure Notification

It is important to have your respective workflows properly set with graceful alerting/notification toward its owners and consumers.

Magnus has all you need for this:

To fully control workflow notification options follow below instructions:

To disable notification, add and set boolean parameter var_sys_notification_enabled to false (default is true) 

To set the recipients, add and set parameter var_sys_notification_recipients to a comma-separated list of e-mail addresses (default is workflow owner)

To set when notification is sent, add and set parameter var_sys_notification_event to failure, success or completion (default is failure)

 To get this notification on manual runs and/or scheduled runs, set var_sys_notification_scope to manual, scheduled or all (default is scheduled)

To get this notification on all other modes of executions (via Execute Workflow Task, Trigger or Remote), set var_sys_notification_scope to all

To notify your company's Potens administrators as well, please contact them to mark the workflow mission critical. Such workflow is indicated by the bell icon:
29.png

Please make sure you use this features when needed, especially for business critical workflows

Workflow Lifetime & Long-Term Workflows

By default, the maximum lifetime for a workflow is 93 days (Client Admins can request different limit for their Org). 

Long-Term Workflows are workflows with Lifetime set for 90 or more days.

Instructions for setting / extending Long-Term Workflows:

Following below instructions will ensure that your workflow's schedule is not interrupted, and workflow will continue to run for another long-term duration, and you will get another expiration notification when lifetime will expire again.

So, open the workflow in Workflow Editor (shortcut - click on workflow name within the respective Home Page Module) and follow below steps:

  1. To extend the Lifetime of the Long-Term Workflow.
    • Click on the magic wand icon to the right of Lifetime to extend it to next Long-Term duration.
      For example, a workflow set to 90 days - will be extended to another 90 days. 
      If workflow set for 1 year – will be extended to another 1 year.  
    • It is also possible to set a new Lifetime manually. You are still limited to whatever duration is currently set.  
      IMPORTANT: if Lifetime will be (purposely or accidentally) set to duration lower than previously set – next time that duration will be a limit. 
      For example, workflow was set for 1 year.  Then Lifetime was changed to 6 months. Next time you will want to extend or change Lifetime duration for that workflow – 6 months will be the limit


  2. To set non-Long-Term Workflow as a Long-Term Workflows with 90-93 days.
    • Click on the magic wand icon to the right of Lifetime to extend it to 93 days.
      It is also possible to set new Lifetime manually. Still you are limited by 93 days  

  3. To set non-Long-Term Workflow as a Long-Term Workflows with lifetime of more than 93 days.
    Send request to your Org Potens.io Admins - Admins are not limited in setting Lifetime   

Size Limitation

Workflow size is limited to 5 MB; user cannot save a Workflow that is more than 5 MB in size.  To keep Workflows within the size limit, it is recommended to clear any large Workflow-owned custom parameters at the end of the Workflow execution.

Cascade Billing Project

Once Cascade Billing Project is enabled, it is applied downstream in the workflow chain.

Example #1:
Say you have a workflow chain w1 > w2 > w3 > w4 > w5, where w1 calls w2, w2 calls w3, w3 calls w4, and w4 calls w5.

The workflow headers are:

30.png

31.png

32.png

33.png

34.png

When the workflow chain executes, the billing project used for each workflow will be:

  1. w1, w2 – billing project used will be that from the Run-User.
  2. w3, w4, w5 – billing project used will be project-3 because Cascade Billing Project is enabled at w3. And once that is enabled, the billing project is trickled downstream, overriding the billing project setting of w4 (project-4) and w5 (project-5).


Example #2:
Say you have a workflow chain:
w1 > w2 > w3 > w4
> w5 > w6

In this workflow chain, w1 calls w2, w2 calls w3 and w5, w3 calls w4, and w5 calls w6.

The workflow headers are:

30.png

31.png

32.png

33.png

35.png

36.png

When the workflow chain executes, the billing project used for each workflow will be:

  1. w1, w2, w5, w6 – billing project used will be that from the Run-User.
  2. w3, w4 – billing project used will be project-3 because Cascade Billing Project is enabled at w3. And once that is enabled, the billing project is trickled downstream, overriding the billing project setting of w4 (project-4).

Batch Mode

Tasks can be Batch Deleted or Batch Copied within the Workflow Editor after entering into Batch Mode.

To enter Batch Mode click the “Enter Batch Mode” menu item from the left side navigation bar.37.png

After entering into Batch Mode all tasks within the workflow will appear collapsed with checkboxes aligned to the right side for selecting multiple tasks.
38.png

The Batch Mode toolbars are accessible above and below the list of tasks. 

Click the “Exit Batch Mode” button to return to the default view.
39.png

Click “Delete Selected” to remove selected tasks from the workflow.
40.png

Click “Copy Selected” to duplicate selected tasks.
41.png

When selecting tasks the Selection Button is available to check all, none or invert the current selections. 

Clicking the down arrow on the right side of the Select All button will open or close the menu.
42.png
43.png
44.png

Clicking on the Copy Selected button after selecting tasks opens the Batch Copy modal.
45.png

The Destination selector allows you to choose the target workflow for the batch copy.
Currently only “This Workflow” is allowed.
46.png

The Copy to Position selector allows you to select the position of the copied tasks within the target workflow. Copied tasks will be inserted in order above the selected task position. Selecting the Last option will insert the copied tasks at the bottom of the tasks list.
47.png

Copied tasks are suffixed with “_COPY”. The suffixed names can be edited until the workflow is saved.

Copied tasks are also highlighted green.
48.png

The highlight persists after exiting Batch Mode up until until the workflow is successfully saved.
49.png 

Potens.io
Launch Magnus | Goliath