1. Potens.io
  2. Magnus
  3. Concepts & Components

Articles > Parameters

Overview

Magnus provides Parameters to help template Workflow Tasks.  
Built-in, Magic, and Custom Parameters are available to use.  
Built-in Parameters exist in all Workflows and include date and time values.  
Magic Parameters are hidden values that when enabled perform Workflow functions.  
Custom Parameters are open ended and created by the developer.  
Developers can specify name, data type, and value of the Custom Parameter.

Parameters

All parameters begin with var_

Built-in Parameters

01.png

These built-in parameters are readily available:

Name Definition Example
var_date Today's date 20161027
var_daydate Today's date with - 2016-10-27
var_year Today's year yyyy 2016
var_month Today's month mm 10
var_day Today's day dd 27
var_hour Current hour hh 21
var_yesterday Yesterday's date 20161026
var_runId This is an integer that is incremented by 1 every time the workflow is executed 1
var_time Current Unix timestamp 1477598967719
var_prevTime The unix timestamp the workflow was executed previously  1477593242891 
var_workflowID The workflow ID  myWorkflow 
var_taskId The ID of the current running task  myTask 
var_timezone Time zone GB-Eire 

 

Magic Parameters

Magic parameters are specific predefined custom parameters. To set certain features for a workflow, you create custom parameters with predefined names and values.
These magic parameters are available:

 

Name Definition Values Type
var_{taskid}_output     String
var_{taskid}_return     String
var_{taskid}_failed Failed status of TaskID true or false  
var_{taskid}_failure_message Failure Message of TaskID   String
var_iterationid Iteration Count (auto-incremented with each next iteration; reset for each new loop)   Boolean
var_sys_save_notification_enabled When the workflow is updated, send an email to the workflow owner and the user who made the change. 
By default, this feature is off.		 true
false (default)		  
var_sys_save_notification_recipients The comma separated recipients of the email notification for var_sys_save_notification_enabled   Boolean
var_sys_notification_enabled When the workflow is executed via schedule (scope) and it has failed (event), send an email to the workflow owner (recipient). 

By default, this feature is on. 

To customize the scope, event, recipient, or email body for this notification, set the appropriate magic parameters: 
var_sys_notification_scope 
var_sys_notification_event 
var_sys_notification_recipients		 true (default) 
false		 Boolean
var_sys_notification_scope The scope of the workflow execution to notify on.  
To be notified for workflow execution from Execute Workflow Task or Remote Execution or Triggered, select the option “all”		 scheduled (default) 
manual 
all		 String
var_sys_notification_event The event of the workflow execution to notify on failure (default) 
success 
completion 
all 		 String
var_sys_notification_recipients The comma separated recipients of the email notification for var_sys_notification_enabled   String
var_sys_notification_extra  Any extra information you want to include in the email notification for var_sys_notification_enabled   String
var_sys_workflow_prevtask_failed  This parameter will have the value of true if the previous task failed. It will have the value of false if the previous succeeded true 
false 		 Boolean
var_sys_workflow_prevtask_failure_message This parameter will contain the error message from the previous task if the previous task has failed   String
var_sys_double_eval_params If you want to double evaluate parameters, set this parameter to true true 
false (default) 		 Boolean 

 

 

Example for var_sys_double_eval_params:
When the task b runs, it will run this query because of double parameter evaluation:
SELECT 'red'
02.png
03.png

Custom Parameters

Naming

  1. The name of a custom parameter must begin with a-z or A-Z.
  2. The name of a custom parameter can contain characters: a-z, A-Z, 0-9, or _
     

Please consider: Clear out assigned Parameters

Returned data can be assigned to a Parameter. Parameters can store a large amount of data which is saved with the Workflow when executed

To minimize resource usage consider including a task that clears out assigned Parameters when values are no longer needed in the Workflow

You can create custom parameters of type: string, number, boolean, and record.
You can access the fields in a record parameter by index (1-based) or by field name:

<var_record[1]>

<var_record[price]>


You can access parameters (built-in/magic/custom) in two ways:

  1. <var_param_name>
    This gives you the parameter value as it is
  2. :var_param_name

This gives you the parameter value. Additionally, if the parameter is of type string, it will enclose the parameter value with quotes. For example, if you use the parameter in a query in a BigQuery task, it will enclose it in single quotes. If you use the parameter in a Script task body, it will enclose it in double quotes. 

Record Parameter

To create

  1. To create a custom parameter of Record type, click the Add button in Parameters Panel:
    04.png
  2. Select record from the drop down type:
    05.png

If this record parameter will be used as the output of a Task (example:  BigQuery Task or Script Task), there is no need to specify the value and schema of the record parameter at design time because the value and schema of the record parameter will be derived at run time.  Otherwise, the value and schema of the record parameter can be specified:

  1. Click on the text box to bring up the schema/value popup:
    06.png
    07.png

The format of schema is a comma delimited list of field name and type:

fieldName type [,fieldName type …]

where type can be: string, number, or Boolean

  1. The format of value is a comma delimited list of value:
    fieldValue [,fieldValue …]
  2. For example, this is a record with 4 fields:

    08.png
  3. If the string value contains comma, wrap the string with quotes.  
    For example: the first field is the string "apple,strawberry,watermelon":

    09.png

  4. To escape double quote in a string value, use 2 double quotes

     10.png

To access

  1. The field of a record parameter can be accessed by either index (1-based) or field name:
    1. By index:
      This returns the first field of the record
      :var_rec[1] or <var_rec[1]> 
    2. By field name:
      This returns the specified field of the record
      :var_rec[fieldName] or <var_rec[fieldName]> 
  2. If no subscript is provided, the first field is returned.  For example:
    <var_rec>
    is equivalent to <var_rec[1]>
  3. The index 0 returns the last field.  For example:
    <var_rec[0]>
    is equivalent to <var_rec[4]> following the example where the record has 4 fields.
  4. :var_rec[#] returns the field count of the record parameter.
  5. If the subscript has no index or field name:
    1. Script Task: 
      :var_rec[]
      is translated to the appropriate data type with respect to the Script Language.  See Script Task for more details.
    2. Other Task:
      :var_rec[]
      is translated to the string representation of the record parameter. 

Pivot Info Popup

11.png

Previously by clicking on record parameter’s value – Info Popup was shown with schema and values, like below

 12.png

Now, we have added Pivot / Un-Pivot Button so you all elements of record more easily explored

13.png

14.png

 

Workflow-Owned

  1. A Custom Parameter becomes workflow-owned when it is assigned as the output/return parameter of a Task.  For example:
    1. Parameter as Destination in a BigQuery Task:
       15.png
    2. Return Parameter in a Script Task:
       16.png
    3. Response Parameter in an API Call Task:
       17.png
  2. A Custom Parameter that is workflow-owned is indicated by the reset icon next to the parameter's name:
     18.png
  3. Clicking on the reset icon 19.png will reset the parameter to the original value.
  4. The value of the parameter is read-only at design time.  It cannot be edited at design time other than to reset it by clicking the reset icon.

 

Limits

  1. The name of a custom parameter cannot exceed 100 characters.
  2. When creating a new custom parameter, you will be asked to re-enter the parameter name if it exceeds 100 characters:
    sh1.png

  3. You can save up to 18,000 characters in a custom parameter value.
  4. At design time when updating a Workflow, you can enter up to 18,000 characters for the value of any custom parameter.
    sh2.png
  5. When the state of the Workflow is saved at the end of a Workflow execution, if the Workflow-owned custom parameters have values greater than 18,000 characters, the truncated values will be saved. Truncation only happens at the saving of the Workflow state.  There is no truncation while the Workflow is executing; the full value of the parameter is available to any Task that consumes it. 

    For example:
    Here is a Workflow that contains a BigQuery Task and a Script Task:
    sh3.png

    This Workflow has one Workflow-owned custom parameter:
    sh4.png

    During Workflow execution, if the output parameter var_data from the BigQuery Task has value of more than 18,000 characters, the Script Task or any other Task that consumes var_data will see the full value of var_data without any truncation.

    It is only when the Workflow execution is completed and when the Workflow state is being saved, the value of var_data will be truncated.

    The truncated parameter value is presented in the textbox after the Workflow execution is completed:
    sh5.png

  6. You can save up to 3,000 characters in a custom parameter schema.
  7. At design time when updating a Workflow, you can enter up to 3,000 characters for the schema of any custom parameter.
    sh6.png
  8. When the state of the Workflow is saved at the end of a Workflow execution, if the Workflow-owned custom parameters have schema greater than 3,000 characters, the truncated schemas will be saved. Truncation only happens at the saving of the Workflow state.  There is no truncation while the Workflow is executing; the full schema of the parameter is available to any Task that consumes it. 

    For example:
    Here is a Workflow that contains a BigQuery Task and a Script Task:
    sh3.png
    This Workflow has one Workflow-owned custom parameter:
    sh4.png

    During Workflow execution, if the output parameter var_data from the BigQuery Task has schema of more than 3,000 characters, the Script Task or any other Task that consumes var_data will see the full schema of var_data without any truncation.

    It is only when the Workflow execution is completed and when the Workflow state is being saved, the schema of var_data will be truncated.

    The truncated parameter schema is presented in the schema textbox after the Workflow execution is completed:
    sh7.png