Execute Workflow Task

Overview

Simplify processes by creating a template that can be re-used by other workflows.  Generalize a process by using parameters for dynamic input, creating tasks that work on a common input/output schema, and branch out complicated workflows.  

Magnus provides Execute Workflow to accomplish this by allowing other Workflows, such as those constructed as templates, to be executed by other Workflows.

Magnus Workflows can be designed to perform specific tasks such as extracting and transforming Google BigQuery data.  These Workflows can be part of a greater scope that require performing many similar or related tasks.

Magnus features Execute Workflow Task enabling a Workflow to trigger execution of an external Workflow.  Combined with parameters, whose values can be assigned in one Workflow and passed to another, Execute Workflow Task consolidates and simplifies creating otherwise repetitive and potentially bloated Workflows.

Execute Workflow Task allows execution of owned, deleted, or shared workflows.  Parameters available in the selected Workflow to execute can be set to inherit value, set value, or be assigned the value of parameter from a drop down selection.

Front Panel

Flip Panel

Add Execute Workflow Task

1. Under Add Task, select the desired position from the "Add to position" drop down list, then click +Workflow. 

2. Specify the child workflow to execute. 

Selecting a workflow to execute

a. From the type drop down list, select the child workflow type:

i. Owned: show workflows that the user owns
ii. Delegated: show workflows that the user is delegate of
ii. Shared: show workflows that the user is shared with. The prefix (s) indicates shared workflow. 

iv. All: show all of the above

b. Once the type is selected, select the child workflow to execute. 

c. Specify the option "Wait if Running" 

Running Workflows

i. In Magnus, workflow that is not a shared workflow can have one and only one instance running at any given time. See Workflows for more details.

ii. If the child workflow selected is a shared workflow, the option "Wait if Running" does not apply. This is because multiple instances of a shared workflow can run concurrently. So, there is no waiting needed.

iii. If the child workflow selected is not a shared workflow, the option "Wait if Running" dictates how to handle the case when there is already an instance of the child workflow running.

1. If the option "Wait if Running" is checked, and there is already an instance of the child workflow running, the calling parent workflow will wait up to 60 minutes for that instance to complete. If the calling parent workflow has waited more than 60 minutes and that instance is still running, the Execute Workflow Task will fail. It is up to the user to ignore the failure or not by specifying the option "Ignore Failure".

2. If the option "Wait if Running" is not checked, and there is already an instance of the child workflow running, the Execute Workflow Task will fail. It is up to the user to ignore the failure or not by specifying the option "Ignore Failure".

3. If the child workflow should be executed under a different billing project than the parent workflow, specify the billing project.

4. To cascade the child billing project to descendent workflows, check the option.

Arguments

1. By default, when an Execute Workflow task is run and the specified child workflow is executed, custom parameters used throughout the child workflow will have their own run time values. 
User can overwrite custom parameters that are not workflow-owned (in child workflow) and pass in different values for the custom parameters.

2. Click the link Arguments to set the values for the custom parameters of the child workflow to be executed. 

3. The Arguments advanced panel displays all custom parameters of the child workflow. 

4. The columns displayed are:

Return Parameters

1. Return parameters from child workflow can be utilized by subsequent tasks in parent workflow by associating it to the parent workflow’s parameters.
2. To associate child workflow’s return parameters to parent workflow’s parameters, click on the Arguments link. (screen_2)
   
3. Expand “Return parameters” section. (screen_3)
   
4. Create custom parameters in parent workflow. (Screen_4)
   
5. Associate parent workflow’s parameters to child workflow’s return parameters. (Screen_5)
   
6. Utilize the output from child workflow’s return parameters in subsequent task. (Screen_6)
   
7. If the association is no longer desired, just clear the input field. (screen_7)
   

Parameter Details

a. Parameter Name: name of the custom parameter
b. Type: type of the custom parameter
c. Value: value of the custom parameter. The value displayed here only reflects the design time value, the run time value could be different.
d. Overwrite action:

i. inherit-var: Leave the parameter value as it is according to the logic and flow of the child workflow execution, do not change it.
ii. set-value: Overwrite with the new value specified in the Value textbox. The scope of the overwrite is limited to the Execute Workflow task. The child workflow is not modified.

iii. <var_parameter>: Overwrite with the run time value of the parameter selected. The scope of the overwrite is limited to the Execute Workflow task. The child workflow is not modified.

5. Sometimes the child workflow is modified and its list of custom parameters is changed. To get the most up-to-date list of custom parameters of the child workflow, click the button  to refresh the list.

 Permissions

Permission to execute a child workflow is granted if any one of the following conditions is met:

1. Parent workflow owner is either owner or delegate or in "share with" list of child workflow
2. At least one of parent workflow delegate is in "share with" list of child workflow
3. Current run-user is an admin

Nested Execute Workflow Task

Nested Execute Workflow Task is allowed up to 5 levels. That is, this is the maximum level allowed:

• Parent workflow calls
  • Child workflow 1 calls
    • Child workflow 2 calls
      • Child workflow 3 calls
        • Child workflow 4 calls
          • Child workflow 5

State of Child Workflow

1. If the child workflow is not a shared workflow, the state of the child workflow will be saved after it is executed via the Execute Workflow task.

a. Its built-in parameter var_runId will be incremented
b. Its built-in parameter var_prevTime will be updated with the run time.
c. Its custom parameters that are owned by the workflow will reflect the most recent state after the execution.

2. If the child workflow is a shared workflow, the state of the child workflow will not be saved after it is executed via the Execute Workflow task

Execution Mode

Execute Workflow Task executes the child Workflow synchronously.  That is, the Task will wait for the child Workflow to complete before going on to the next Task.  In order to execute the child Workflow asynchronously, meaning not to wait of its completion, user can use Remote Workflow Execution v2 API.  To convert the existing Execute Workflow Task into an API Task for Remote Workflow Execution v2: 

1. Click on the Advanced Properties icon:
   
2. Click on Asynchronous Execution Mode:
   
3. Click Yes to insert an API Task:
   
4. Now the new API Task is inserted:
   
5. After verifying that the new API Task looks ok, such as checking the parameter overrides, delete the old Execute Workflow Task as that is not needed anymore.
6. Since now that the child Workflow will be executed via Remote Workflow Execution v2, permission requirements for Remote Workflow Execution v2 apply.  See here for more details. 

Considerations for Workflow Template

When creating a Workflow template to be shared with other users, it is good practice to run the Workflow template once, so that any Workflow-owned parameters will not be exposed to other users.