Magnus provides Workflow Triggers as a way to initiate a Workflow execution based on some predefined cloud events. When the watched event is fired, the Workflow is triggered to execute via Remote Workflow Execution v2 via queue. That is:
- the request to execute is queued and processed by the Fair Scheduler
- the quota for max enqueued requests per user applies
- multiple instances of the Workflow is allowed
- the Workflow state is not saved after execution
Cloud Storage Event
When a file is created, updated, or deleted in the watched Google Cloud Storage bucket, the Workflow is triggered to execute with information about the file and event.
Prerequisites before creating the Trigger
- Trigger must be enabled for the organization. Please contact your organization admins.
- Before creating the Trigger, user must have already granted offline access to Magnus. Otherwise, saving of the trigger will fail.
- The service account service-[Project Number of Organization’s Trigger Cloud Project]@gcf-admin-robot.iam.gserviceaccount.com must be Storage Admin of the project that the watched Google Cloud Storage (GCS) bucket belongs to. Otherwise, saving of the trigger will fail.
Add a Workflow Trigger
- Make sure that prerequisites are met.
- From Magnus Homepage, click on Triggers to go to the Triggers Browser.
- Click on the Add Trigger icon on the upper left corner.
- Specify Trigger properties.
A Trigger has the following properties:
[Required] Unique name of the Trigger
[Optional] Description of the Trigger
[Required] Account under which the Workflow is run when triggered if the Workflow is not locked to run always as the Workflow owner.
This is case-sensitive.
[Optional] Period during which the Trigger is enabled.
If not specified, the Trigger is always enabled.
Time zone of the Trigger lifetime. For example, if the Trigger lifetime is 6/1/2018 to 6/2/2018 and the time zone is US/Eastern, then the Trigger will be enabled from 6/1/2018 to 6/2/2018 US/Eastern time. The Trigger time zone does not affect Workflow time zone.
- Choose GCS as Trigger type
Specify GCS event properties.
A GCS event has the following properties:
Description [Optional] Description of the event Source GS Bucket [Required] The Google Cloud Storage bucket to put a watch on Source GS Folder
[Required] The folder or file pattern to watch for.
For example, to put a watch on gs://myBucket/fruits/apple_*.txt, specify /fruits/apple_*.txt as the Source GS Folder.
/a/b: This matches the object b in folder a only, such as /a/b.
/a/b/: This matches any objects in folder /a/b/, like /a/b/c or /a/b/c.txt. It doesn't match objects in further child subfolders, like /a/b/c/d.txt
/a/b/*.txt: This matches any .txt objects in folder /a/b/, like /a/b/c.txt. It doesn't match objects in further child subfolders, such as /a/b/c/d.txt
/a/*/*.txt: This matches any .txt objects in immediate child folders of a, like /a/b/c.txt or /a/d/c.txt. It doesn’t match objects in further child subfolders, such as /a/b/e/c.txt
/a/*/*/*.txt: This matches any .txt objects in immediate grandchild folders of a, like /a/b/e/c.txt or /a/d/f/c.txt
State [Required] The qualifying file state to trigger the Workflow.
- Specify the Workflow.
It has the following properties:
[Optional] Description of the response.
[Required] The Workflow to trigger. Only owned or delegated Workflows are allowed.
- Specify parameter overrides to pass to the Workflow during execution.
- Click on the Arguments link.
- Select a value:
- inherit-var: inherit the parameter value from the Workflow
- set-value: set a custom value for the parameter
- var_data_*: these are built-in parameters that contain information about the GCS object that triggered the event. See below for definitions.
- var_event_*: these are built-in parameters that contain information about the event. See below for definitions.
- Click on the Arguments link.
- Click Save to save the trigger. This could take up to a few minutes.
You will end up with something like below:
The Trigger will be enabled few minutes after saving.
All Triggers are listed in Triggers Browser. Triggers Browser lists Triggers by Trigger Type, Name, Owner, Workflow to be execute, Created By and Created At
Action Icons are available for each Trigger providing a quick way to view, show Workflow to execute history, and delete.
Trigger information can be filtered.
- Take user to Magnus main landing page.
- Add new Trigger.
- Refresh icon: Refresh the page to the latest results.
- Content Search is disabled and not supported for now
- Scope selector: Owned Triggers option only.
- Filter: filter Triggers based on the input text on client side.
Row example and action buttons
Results can be sorted ascending or descending by clicking the small arrow next to each column title
- Trigger Type
- Trigger Name.
- Owner for the trigger.
- View Trigger , read only mode.
- Delete Trigger.
- View the history of the Workflow.
- The name of the workflow that is going to be executed by the Trigger.
- Who created Trigger.
- When Trigger was created.
- Status of the Trigger:
- failed: Trigger creation failed and by hovering over the status more information will be displayed.
- deploying: Trigger is being saved and deployed and during that it can’t be deleted (Delete action icon will be disabled).
- deleting: Trigger is being deleted.
- ready: Trigger was a successfully created.
- Icon indicating GCS Event based Trigger
- Icon indicating Pub/Sub Event based Trigger
- The first line will show a summary how many Triggers were fetched, and only 30 Triggers per page will be displayed.
- The second line will show the number of pages and by clicking different numbers it will show the next 30 Triggers and so on.
- Exit button will take user to the Magnus main landing page.
Definitions of Built-in Parameters
|var_data_kind||The kind of item this is.|
|var_data_id||The ID of the object, including the bucket name, object name, and generation number.|
|var_data_name||The name of the object.|
|var_data_bucket||The name of the bucket containing this object.|
|var_data_generation||The content generation of this object. Used for object versioning.|
|var_data_contentType||Content-Type of the object data|
|var_data_timeCreated||The creation time of the object in RFC 3339 format.|
|var_data_updated||The modification time of the object metadata in RFC 3339 format.|
|var_data_size||Content-Length of the data in bytes.|
|var_data_md5Hash||MD5 hash of the data; encoded using base64|
|var_data_timeDeleted||The deletion time of the object in RFC 3339 format.|
|var_event_id||A unique ID for the event|
|var_event_timestamp||The date/time this event was created in ISO 8601 format|
|var_event_type||The type of the event.|
Workflow Triggers are backed by Google Cloud Functions. When a Workflow Trigger is created, a respective GCF is created in the organization's Google Cloud project. When event invokes the GCF, the GCF in turns makes a call to remotely execute the Workflow. Per the nature of GCF, it is guaranteed to be invoked at least once for each incoming event (https://cloud.google.com/functions/docs/concepts/exec#execution_guarantees). It is recommended to make the target Workflow idempotent so that when an incoming event invokes it multiple times, the desired results will still be achieved.