1. Potens.io
  2. Magnus
  3. Meet Magnus

Articles > Release Notes

July 30, 2021

  1. Magnus. API Task: added support for binary response type

    mceclip0.png

    Few use cases as an example (from actual Feature Requests):
    • User can download an image from tableau API and upload that image to google cloud storage or use it in Email Task to send as a report, etc.
    • User can download an image (max API response in Magnus is 32 MB) via API Task and use that output with Vision API to annotate that image

    Check out How-To Workflow for details:
    HowTo_Download_and_Process_Binary_API_Response
    https://magnus.potens.io/?l=SG93VG9fRG93bmxvYWRfYW5kX1Byb2Nlc3NfQmluYXJ5X0FQSV9SZXNwb25zZQ==  

  2. Magnus. Workflow Editor: added system parameter selector

    mceclip1.png

    mceclip2.png

June 15, 2021

  1. Home Page - Activity Tab: Added “Workflow delete” and “Trigger delete” to filter types.

    mceclip0.png

  2. Workflow Editor:  Unintended effects when pressing Enter 
    Fixes an issue where pressing enter key when focused on an input element in the Workflow Editor or Diff Tool causes the page to redirect.

  3. Dark Mode: Fix Workflow Editor background (edge case caused by new Home Page) 
    Fixes an issue where toggling dark mode doesn’t also toggle the Editors background color.

  4. Purge Archived Workflows Process: When deleting too old, archived workflows, delete associated triggers as well.

  5. Email Notification: Added Magnus Engine version to workflow failure/success email notifications 

May 27, 2021

  1. Magnus: new Home Page – cockpit style with quick views on what is going on, inventory, activities, alerts, etc. 

  2. Magnus. Workflow Editor - Auto copy Shared Link to clipboard when Share Link button is clicked

April 7, 2021

  1. Magnus: Workflow Labels for API Task 

    Workflow Labels now propagated to API Task (this feature previously was released for all Task types having explicit call to BigQuery API)
    If API Task starts a BQ job, propagated workflow labels will automatically be included in the request body sent to the BQ service. If a propagated workflow label key is already present in the API task request labels, the label already in the request has precedence. Propagated workflow labels are included up to the google cloud label limit (64), any in excess are excluded. 

  2. Admin Suite: Transfer workflow roles in bulk 

    1.png

    Admin users can transfer a user’s workflows and triggers roles to a different user by completing Transfer Roles Form on the Users Page. 

    1.png

    Upon Completion, Admin and involved users will get Notification Email with all relevant details. 
    All submitted requests with details and results of the transfer are available in Roles Transfer Log Page.

    1.png

March 18, 2021

  1. Workflow Labels. 

    1.png

    New panel added in Workflow Editor – Labels Panel – so user can add up to 10 labels (key and optional value) to be then used for search in Workflow Browser  

    1.png   

    Workflow can be searched by whole label using key:value format or by key only using key: format or by value only using :value format 

    Most importantly – user can propagate labels down to BigQuery Jobs executed by workflow – for this label must ne set using respective arrow-down icon – it should be green to be propagated. When label propagated – it can be used to slice and dice cost and usage by label – for example via use of INFORMATION_SCHEMA.JOBS_BY_* (see more at Getting jobs metadata using INFORMATION_SCHEMA) or in Workflow History Details or in Goliath’s Job History 

    Note, there is one built-in label with key workflowid with respective value of workflowid. It is set to be always propagated. 

  2.  Timeline Dashboard. Multiple Fixes and Improvements 

February 18, 2021

  1. Magnus: Dashboard – Added Next Run At (Column, Info, Visualization) 
    “Next Run At” column/info added in Workflows Section of the Inventory Dashboard 
    1.png

    …and in the Workflow Info Popup indicating the time of the next execution for the workflows. 

    1.png

    Additionally an icon has been placed on the timeline indicating the next run with some metrics/stats (on icon hover) and Timeline now can be sorted by Next Run At

    1.png
  2. Multiple fixes and improvements based on your feedback. 

February 3, 2021

  1. IntelliSense – (Magnus, Goliath) Added and updated Keywords to reflect most recent new and updated BigQuery features.  
    Added keywords: SUBSTRING, ASCII, CHR, INITCAP, INSTR, LAST_DAY, LEFT, OCTET_LENGTH, REGEXP_INSTR, REGEXP_SUBSTR, RIGHT, SOUNDEX, TRANSLATE, UNICODE, KEYS.NEW_KEYSET, KEYS.ADD_KEY_FROM_RAW_BYTES, AEAD.DECRYPT_BYTES, AEAD.DECRYPT_STRING, AEAD.ENCRYPT, ST_CLUSTERDBSCAN, ARRAY, ML.ARIMA_COEFFICIENTS, ML.FEATURE_IMPORTANCE. Added keyword statements CREATE EXTERNAL TABLE, DROP EXTERNAL TABLE, ALTER TABLE ADD COLUMN, EXPORT DATA, TRUNCATE TABLE, ADD COLUMN, ADD COLUMN IF NOT EXISTS, EXPORT DATA, EXPORT MODEL. 
    Updated keyword descriptions for ST_GEOGFROMGEOJSON, ST_GEOGFROMTEXT, REGEXP_EXTRACT, ST_NPOINTS, JSON_QUERY, JSON_VALUE.

  2. Processing locations – (Magnus, Goliath) Added new locations
    Added Iowa (us-central1), Oregon (us-west1), Belgium (europe-west1), and Netherlands (europe-west4)
  3. Multiple fixes and improvements based on your feedback. 

January 27, 2021

  1. Magnus: Remove/Cancel Queued Workflows – user can prevent a queued workflow from running with the “Remove Workflow/Iteration from Queue” button in the History Details Panel

    1.png 1.png

    Only Admins, Workflow Owner and the User who ran the workflow are allowed to cancel. The button is not shown otherwise.
    Any workflows waiting on such a cancelled workflow will raise failure with message “workflow cancelled”, or “iteration cancelled” in the case of an async loop iteration. It is up to user to handle the error as desired
  2. Magnus: Custom Billing Projects per Task - added billing project selector dialog so user can select from the list of available billing projects.     
  3. Multiple fixes and improvements based on users' feedback. 

January 21, 2021 

  1. Custom Billing Projects per Task - User can specify a billing project per Task where applicable, different from the workflow’s default billing project.
    Previously, this was available for Execute Workflow Task (and obviously in API Task as it is part of the request body) only – now it is available for all Tasks where GCP billing involved (namely BigQuery Task, Cloud Storage Export Task and FTP Out Task).
    Respective controls can be found on the Flip side of Task Panel:

    mceclip0.png

    Billing project can be set using parameter – so you can make it a function of some condition (like time of the day, etc.)
    Note: Task level billing project is ignored in case if/when Billing Project Cascading is enabled on Workflow level

    mceclip1.png

January 13, 2021 

  1. Loop Task – Added new options for Iteration Data
    • Iteration over Number Range    
    • Iteration over Date Range

      mceclip15.png
      mceclip16.png
  2. Go-To Task within Loop Task – Added more Action options
    • Continue Loop – skip the rest of the tasks in Loop and continue execution to the next iteration
    • Exit Loop – skip the rest of the tasks in the Loop, skip the rest of the iterations and jump to the next Task after the Loop Task

      mceclip17.png

December 21, 2020 

  1. Workflow Browser – Workflow Tree View: Added support to recognize child workflows called via remote API as API Task. Previously only child workflows called via Execute Workflow were recognized here.  

    mceclip10.pngmceclip11.png

    In above example - previously, only “export async” Execute Workflow Task was recognized as a call to “pre_cache_advertiser_campaign_export” workflow and thus was ‘drilled’ down to that workflow definition and shown in Tree View.
    The “export_async_COPY1” API Task was not recognized as a child workflow call and thus was presented just as API Task with one line.
    Now, we recognize that this particular API Task actually does remote call to child workflow thus user can see that workflow definition in Tree View.

  2. Workflows Browser: Added Next Run At column – to show next scheduled time (obviously for scheduled workflows only).

    mceclip12.png

    Additional feature here – by clicking on date-time value in this column user can see all involved child workflows including nested calls (this is a trimmed down version of  Workflow Tree View with ONLY child workflows shown to have a glance on what else workflows are involved) 

    mceclip14.png

    Below is yet another example with more complex case – so you can appreciate this trimmed-down version and focus on child workflows calls only

    mceclip13.png

December 15, 2020 

  1. Incremental Authorization:  During signup, user is asked for the most basic scope, bigquery.readonly, to perform actions within the applications. Some actions may need additional permissions and when initiated - a prompt requesting to grant them is presented. This flow is known as incremental authorization and introduced mostly to relax required permissions for trial users and new clients onboarding. Check out documentation here -  https://potensio.zendesk.com/hc/en-us/articles/360052386432-Incremental-Authorization-

    Note: this is Beta release – please let us know if any issue noticed

    Important recommendation: Current Poten.io Clients who are adding new users after incremental authorization has been released (early December 2020) can recommend these users to grant the below scopes in advance to bypass incremental authorization process

    • bigquery
    • storage.full_control
    • cloud-platform
    • drive scopes

  2. Number of fixes and improvements. Some items:
    • Improved Errors extraction for Failed Dashboard in Magnus
    • Fixed rowcount parameter for UPDATE DML in BigQuery Task
    • Improved failure notification emails
    • and many more …   

August 25, 2020 

  1. Billing Project for Execute Workflow Task:  Support for use of custom parameter in Billing Project attribute in Execute Workflow Task.

    mceclip9.png

    This allows to easily control the billing project on the weekend and after hours vs. working hours to save some more $$$ 

  2. Locked Workflows: Workflow can be locked into read-only mode by admins
    Admin can Lock and Unlock workflows from within the Workflow Browser.
    Locked workflow are marked with lock icon to the left of workflow id in Workflow Browser and Inventory Dashboard. Also lock icon appears to the left of Save button in Workflow Editor
    The Workflow Editor is locked into read-only mode until unlocked by an admin. While workflow is locked for editing  - it still can be run by any available means

July 14, 2020 

  1. Dark theme – finally Dark Theme available in Magnus
    Use CTRL+M to switch between Light and Dark Themes

    For example, Light Mode
    mceclip0.png
    vs. Dark Mode
    mceclip1.png
    For the best experience consider also using respective settings in System Preferences
    mceclip2.png
  2. Row Count output parameter for BigQuery Task
    User can select a parameter to receive the produced or affected row count.
    mceclip3.png

    mceclip4.png

  3. Diff Tool: ability to shift/slide down/up pair of versions 
    Users may shift the pair of Workflow versions to compare by clicking  the up or down buttons in the Diff Tool.
    mceclip5.png

  4. Documentation links: Diff Tool – Documentation link and title fields are now treated as independent within the Diff Tool.
    The workflow documentation links are now highlighted to indicate differences independently for Link and Title fields.

  5. Dashboard: Cron Helper for Schedule Column – A readable Cron expression tooltip was added to the Inventory Dashboard.
    A tooltip with a human readable form of each Cron expression is viewable upon hovering over the Inventory Dashboard workflows table schedule column.
    mceclip6.png
  6. Null parameter substitution (IMPORTANT – behavior change/fix)
    Substitute null parameter value with “null” or “nil” (depends on context) during parameter substitution. 
    Currently substitution is skipped if parameter value is null. 

  7. Recent Changes Dashboard:
    Last # days drop down has been added to recent changes dashboard.
    The drop down impacts recent changes dashboard only.

    mceclip7.png

    Options include last 7 days, last 14 days, last 30 days, last 60 days, last 90 days, last 120 days, and last 180 days.

    mceclip8.png

  8. Number of fixes and improvements (like few example below)
    1. Fix/improve parameter not defined warning
      Fix parameter not defined false alarm with record when using :var_rec without subscripts.
    2. Inventory Dashboard: Workflows Table Columns
      Inventory Dashboard table column widths were reduced and tooltips were added.
      Workflow Owner, Delegate and  Schedule columns were made narrower.
      Tooltips were added to show additional data that is cropped by the narrowed columns
    3. Fail fast if no refresh token 
      Fail immediately when Task requires access token but user does not have refresh token or valid access token

 



June 23, 2020 

  1. Set Workflow’s Documentation Link – user can now set / manage a workflow documentation link from the left menu which is shown at the top of the workflow. 

    mceclip0.png

    mceclip1.png

  2. Schedule validation – Added schedule cron validation on workflow save.  

    mceclip0.png

June 18, 2020  

  1. Loop Task: Async Mode  - Beta Feature! Work in Progress! 
    Async Mode can be enabled on a Loop Task, and allows the loop to run multiple iterations at the same time, rather than one at a time in sequential order. Concurrent Iterations specifies how many loop iterations can run in parallel. For example, when Async Mode is enabled and Concurrent Iterations is 5, it means it will run up to 5 loop iterations in parallel; when one iteration finishes, it will start another iteration, until 5 are running or the loop is completed. Parameters within the async loop can be referenced as usual, however any parameter updates within the async loop will be accessible only within that iteration and will not persist to the next iterations or after the loop completes.

    User can set Async Mode on the Flip Side of Loop Task 
    mceclip2.png

    If Loop is set for Async Mode – respective icon indicates this on Task’s Front Side 
    mceclip3.png

  2. Timeline Dashboard – Added visualization for Async Loop’s iterations.
    Async Mode is available and all async iteration executions are displayed in the Timeline Dashboard. Such Async executions are identified by a dotted border. Popups for async executions display the workflow name and iteration start time along with the task name and iteration. Clicking on an async execution displays the iteration run details. Async executions are available in the timeline dashboard only.

    mceclip4.png

    another example
    mceclip5.png

  3. Fair Scheduler
    Increased execution frequency of Fair Scheduler and Dequeuer execution
    Now it is every 10 sec vs. previous 1 min 
    Note: Fair Scheduler is used for handling queued remote workflows executions. Regular Scheduler is still limited to 1 min

  4. Cascade Workflow Kill – When “killing” a workflow, it will now cascade kills/cancels all descendent workflows (child workflows, grandchildren, and so on)
    When a workflow is explicitly “killed”, the system will also find all of its child workflows, and child workflows of those child workflows and so on, and will kill all such descendent workflows. If any of these workflows are pending in the queue and have not yet started execution, they will be cancelled and never start.
    Note: Kill Workflow feature available for Org Admins only!

  5. Helper / Explainer for Quartz Cron Expression 
    Present a quartz cron expression as a human readable text / descriptions that clearly explains when it will be executed 
    Now supported in Workflow Editor only 
    More cron patterns supported:
    • optional “Years” position 
    • Some “Days” patterns: “L”, “LW”, “L-3”, “7#4”, “31W”, “3L”, etc.  
    • And more 
      Let us know if you identify any cron being not properly described 

      mceclip6.png

June 2, 2020  

  1. History Browser: Added Days dropdown to limit how many days to look back to get history for.
    mceclip7.png

  2. API Task: Added option to pass ID Token instead of Access Token.
    This is to cover use case of calling GCF (Google Cloud Function) via http

    mceclip8.png

  3. API Task: Added options to specify max attempts count and sleep duration (below shown values are max values and used as default).

    mceclip9.png

  4. API Wizard: API grouped now by Categories: Advertising; AI and Machine Learning; Big Data; Blog & CMS; Compute; Data Analytics; Developer Tools; Firebase; G Suite; Identity, Access and Security; Maps; Mobile; Networking; Operations and Management Tools; Social; Storage and Database; YouTube

    mceclip10.png

  5. Helper for Cron Quartz:  Advanced entries have their value converted from CRON expressions into human readable descriptions.
    A tooltip has also been added when hovering over Schedule field.

    mceclip11.png

  6. BigQuery and Script Tasks: Code Fold / Unfold – Text editors for query and scripts can have code folded and unfolded.

  7. Timeline Dashboard: Sorting - The number of Daily Executions and Daily Total Duration for each workflow is available to be sorted by

    mceclip12.png
  8. Inventory Dashboard: Added Backup and Archive Menu Items.

    mceclip13.png

  9. Recent Changes Dashboard – A new Dashboard added that lists workflows that have been created or changed recently. 

    mceclip14.png

May 13, 2020 

  1. [NEW] Inventory Dashboard -Added inventory section in Magnus Dashboard
    Inventory dashboard displays detailed list of workflow and triggers associated to the user/organization as well as summary and usage info. The list includes owned and delegated workflows, triggers, and (for admins only) users. Org Admin can see given user’s Inventory as well as whole Org Inventory. 

    mceclip15.png

  2. Timeline Dashboard- Added execution counts per workflow 
    In timeline the workflow name is listed along with swim lanes. The number of executions were added in parentheses to the right of the workflow name.

    mceclip16.png

  3. [NEW] Backup and Restore workflows- User can backup / restore workflows through GCS. 
    User can back up just most recent version or whole workflow with all its versions. 
    Restore can be done as a new workflow or as a new version to existing workflow 

    Available in Workflow Browser in Contextual Menu and Left Toolbar
    mceclip17.png

    Also available in Left Toolbar in Workflow Editor 
    mceclip18.png

  4. [NEW] Batch Delete Tasks- Multiple tasks can be deleted at once
    Users can enter batch mode and select multiple tasks for deletion 

    mceclip19.png

  5. [NEW] Batch Copy Tasks- Multiple tasks can be copied at once
    Users can enter batch mode and select multiple tasks to copy

  6. [NEW] Continuous workflows- If an instance of a workflow is already running, and the scheduler tries to execute the workflow, the scheduler will silently skip the execution.  Previously this would cause "Another instance of the workflow is already running" error. 

    If user wants to see such rejected executions – user need to explicitly express this in Workflow Header   

    mceclip20.png

  7. Number of fixes and improvements  

April 21, 2020 

  1. [NEW] BigQuery Task:  Added support for STRUCT, ARRAY, or GEOGRAPHYas output to Destination Parameter
    When querying STRUCT, ARRAY, or GEOGRAPHY data types into a Destination Parameter, they will be translated into respective string representations that make them easily reusable in next Tasks. Previously these types were not supported as a output to parameters and would cause the BigQuery Task to fail.

  2. [NEW] Reservation Grantees API:  Added API to update Reservation Grantees metadata
    Org Admin can call this new API to update Reservation Grantees metadata.  Historical states of Reservation Grantees are used in Dashboard for BigQuery cost calculation.

  3. [NEW] Added feature of Private Keys rotation for service accounts (available for Potens Org Admins)

  4. Number of fixes and improvements    
    • Parameters Drag and Drop – Fix colors/styles, added cancel esc key, fix ghost position, and highlight moved variables.
    • Some UI Issues on Zoom
    • Many more …

April 8, 2020 

  1. Magnus Dashboard – Failed Workflows
    Added new Dashboard Page that provides main stats about failed workflows

  2. Magnus Dashboard – Workflows Timeline
    Few fixes and improvements

  3. Added feature of Private Keys rotation for service accounts (available for Potens Admins)

  4. Number of fixes and improvements

March 11, 2020 

  1. Magnus Dashboard
    Provides visualization (along with some stats) of workflow executions, success, failures, inventory and changes
  2. Drag & Drop Tasks & Parameters
    Allows to change order of Tasks / Parameters in Workflow by simply using Drag & Drop approach

February 10, 2020 

Workflow Versions Diff Tool extended for Text-Based entries for below Tasks

  1. BigQuery Task – Query Text
  2. API Task – Request Body & Request Headers
  3. E-Mail Task – Email Body
  4. Script Task – Script Text
  5. And few more text-based elements

Note: Diff Tool is still in Beta, so please let us know when you see something needs to be fixed – especially if it is not cosmetic o)  

February 4, 2020 

  1. Workflow Versions Diff Tool v1 Beta 

January 9, 2020 

New in January 2020 and Q4 2019

  1. Resumable Workflows 

  2. Mission Critical Workflows 

  3. Long Term Workflows – Smart Lifetime & Updates 

  4. Improved Org Admins Actions & Admin Tool 

  5. BigQuery Task: Support for Scripting and Stored Procedures 

  6. Execute Workflow Task: Billing Projects Option for 

  7. Slot Usage and Consumed SlotTime Metrics

 

September 17, 2019 

  1. Workflow Billing: Added cascade billing project option to workflow header in workflow editor

  2. API Wizard: Added API Key as a Credential options to API wizard
  3. Some minor improvements / fixes
     

August 28, 2019 

  1. API Task: Extend API Wizard with APIs requiring additional non-sensitive or non-restrictive scopes
    About 30 more APIs are available for use in API Wizard  
    mceclip2.png

  2. Some minor improvements / fixes 
    • Fixed exposure of incorrect / not-applicable compressions in Export Tasks
    • Email Task: Allow empty email address in recipients list

August 13, 2019 

  1. Workflow Browser: Improved coverage of Last Run At attribute in Workflow Browser

    Till recently, Last Run At attribute was ONLY available for workflows run manually or directly by schedule or via Execute Workflow Task by owner or delegate or queue less-remote execution (v1). Other runs like via trigger, remote (v2) runs, via Execute Workflow Task by “shared with” users were not accounted
    Since now, all invocations are counted, so users can see better / correct and UpToDate picture of their workflows’ usage 

  2. BigQuery Task: Output to Parameter now supports date/time BQ types: Timestamp, Datetime, Date, Time

    We still don’t support ARRAY and STRUCT data types to be used as output to parameters. Will be addressed in future releases

  3. Some minor improvements / fixes
    • Lifetime validation falsely identifies duration of 93 days as exceeding the max (93 days) allowed when end-of-daylight-saving lands within the lifetime
    • API Wizard conflicted with some Chrome Extensions

  4. API Wizard – is a great tool on its own developed by Potens.io Team

    Wizard currently supports about 160 Google APIs with 110 of them ready to be used w/o any extra permissions to be given. See list below

    mceclip0.png

    Note: To use given API – it needs to be enabled – so contact Magnus Team for this

    In next release we plan to add 30 more APIs to be available for immediate use in API Wizard  

    mceclip2.png
 

August 6, 2019 

New in July 2019

  • BigQuery Task: Full support for Query Prefix

    SQL variant, either legacy SQL or standard SQL, cab be set by including a prefix to your query
    #legacySQL - Runs the query using legacy SQL

    #standardSQL - Runs the query using standard SQL

    Now, user can have BigQuery Task in the Loop running queries stored / generated with mixed SQL versions/dialects (Legacy vs Standard)

  • Email Task: Added Email Priority Selector

    mceclip0.png

  • API Task: Added API Wizard

    mceclip1.png

  • History Details: Navigate to Child and Parent workflow in History Details

    Previously troubleshooting of below execution of parent / children workflows was possible only by opening History Details for each respective workflow one-by-one     
     mceclip2.png

    Now, user can start from any workflow and then navigate up to parent workflow or down to child workflow directly within workflow’s History Details 

    mceclip3.png

  • Workflow Execution Notification Email: Added direct link to workflow
  • Some minor improvements / fixes

June 26, 2019 

New in June 2019

  • Execute Workflow Task: Added support for Return Parameters

    1.png

  • Workflow Browser: Added Manual Run feature
    No need to Open Workflow anymore for Manual Run

    2.png

  • Service Account registration and use to run workflows: JSON private key file is now supported (in addition to previously supported p12)
  • Single Sign On (SSO): Added SSO to support integration with 3rd party tools/sites
  • Minor improvements / fixes (few examples below)
    • Hub Task: increased max item’s timeout to 6 hours.
    • History Details: when workflow killed while being within Loop Task – that Loop Task and all nested Tasks were not shown in History Details which prevented user from effective troubleshooting 
    • Remote Workflow Execution: Improved stability for Remote Execution – based on quite an edge case from one of our DLP client 

April 4, 2019 

  • BigQuery Task: Added processing locations São Paulo (southamerica-east1) and Zürich (europe-west6)

  • Add Workflow: New button added to Goliath - "Automate with Magnus" which allows to copy Goliath's workspace into Magnus and create new workflow off of that workspace for further automation

March 20, 2019 

  • Workflow Browser - Added “All Above” option

    02-01.png

  • Copy Workflow functionality now available for Workflow’s Delegates

    02-02.png

  • Minor improvements / fixes 
    As an example: API Task - Template Selector: pre-select the first template when Service selected

March 13, 2019 

  • API Task - Added more API templates:
    1. BigQuery: Create External Table from GCS with Autodetect
    2. BigQuery: Create External Table from GCS with Schema
    3. Data Transfer: Export/Import To/From Cloud SQL
    4. DoubleClick Bid Manager: Retrieve Entities in SDF format
    5. DoubleClick Bid Manager: Retrieve Line Items in CSV format
    6. DoubleClick Bid Manager: Upload Line Items in CSV format
    7. DoubleClick Bid Manager: Run a Stored Query to generate a report
    8. Firebase: Generate short Dynamic Links
    9. Firebase: Get analytics data for short Dynamic Links
  • Script Task - Added XML support for Golang and JavaScript
  • Workflow Editor: Added Workflow Tree button to left nav bar

February 20, 2019 

  • API Task - Added more API templates:
    • PubSub: Publish Message
    • Magnus: Get History
    • Artificial Intelligence - Natural Language: Annotate Text
    • Artificial Intelligence - Natural Language: Analyze Entities
    • Artificial Intelligence - Translation: Translate Ttext
    • Artificial Intelligence - Vision: Annotate Image
    • Dataflow: Bulk Compress GCS files
    • Dataflow: Bulk Decompress GCS files
    • Dataflow: WordCount Example
    • Dataflow: GCS Text to BQ
  • Workflow Task - Added Asynchronous Execution Mode
  • Cost attribution for BigQuery Tasks in Async mode is now presented in History Details

February 14, 2019 

  • GCS Export Task - Added Asynchronous Execution Mode along with Collect Feature
  • Workflow and History Browsers – light cosmetic changes: some UI controls are regrouped
  • API Task - Added more API templates:
    • BQ: Copy a BQ table
    • BQ: Create a BQ View
    • BQ: Run a BQ query
    • GCS: Copy an object
    • GCS: Get metadata of an object
    • GCS: Download an object
    • GCS: Get a list of objects from a bucket
    • GCS: Delete an object
  • Added new Processing Locations (Los Angeles, Hong Kong)
  • Improved cost attribution for BigQuery Tasks in Async mode

February 6, 2019 

  • BigQuery Task – Added Asynchronous Execution Mode along with Collect Feature
  • Added New Task Type – Hub Task
  • API Task – Added Collect Feature
  • API Task – Added two more Templates
    • Remote execute of Magnus Workflow via Queue
    • Export BigQuery table to Google Cloud Storage
  • Few fixes/improvements:
    • New recently introduced Processing Locations
    • Fix for BigQuery Job Cancel Feature

January 24, 2019 

  • Templates are introduced in API Task 
  • Improved features / performance of Workflow Browser and History Browser
  • Workflow Editor is now opened in a new tab when view/edit/archive/add icon is clicked.
    History Browser is now opened in a new tab when history icon is clicked
  • Few fixes/improvements:
    • Over-validation for GCS buckets when creating/updating Trigger
    • Improved performance of BQ Task to poll BQ job status the first time at 1 sec, then at 5 sec, 10 sec, up to 1 minute interval. Before the first poll was at 5 sec

December 12, 2018 (Q4'18)

  • Triggers are now updatable
  • MySQL Task is now executed within SQL transaction
  • FTP Task: user can specify custom port in FTP path
  • Loop Task: sleep time between iterations reduced to 100 ms
  • Workflow Editor:
    • Added “Extend Workflow Lifetime” button for quick extension to another 90 days – only active long-term workflows are qualifie
    • “Show Workflow Execution History” action icon in left Tool Bar now opens History Browser for respective workflow in new Tab
  • “Magnus Long-term Workflows About to Expire” Notification Email now has direct link to respective workflows
  • Added new Processing Locations to BQ Task, GS Export Task, FTP Task, FTP-GS Task.
    • London, Singapore, Sydney, Northern Virginia, Finland and Taiwan
  • Waiting in Queue Remotely Executed Workflows are now listed in History Browser with empty Start and Finish dates and with “Pending” as a Status and respective indicator
  • Magnus v5 – massive upgrade to Google Cloud SQL powered backend
  • Limits for Workflow’s Parameters:
    • parameter name up to 100 chars.
    • parameter value up to 18000 chars.
    • parameter schema up to 3000 chars.
    • At the end of workflow execution, parameter value and schema of workflow-owned parameters are truncated if length exceeds max allowed
  • Fixes, improvements – let us know if you see something you asked / reported but not addressed yet. Few examples below:
    • Issue with Custom Projects when there is an expired user either owner or delegate
    • Issue with Email Task attachment size limit
    • Edge case: Workflow/History Browser: “workflow not found”
    • Edge case: exposing extra workflows names in Execute Workflow Task
    • History Browser in hierarchical mode: Fixed UI NaN bug for BQ cost.

August 13, 2018 (July, 2018)

 

June 19, 2018  (Q2 2018)

  • Created number of detailed Tutorials Articles and Videos 
  • Added Time Zone support.
    1. User can choose Time Zone to use for representing dates/times in UI
    2. User can set Time Zone to use for scheduling on per Workflow basis
  • Added Lifetime’s End Date to Workflow Browser
    When End Date is closer than 7 days, it is being red-colored so easy to catch!
  • Enable copy of Get Started Example Workflows
  • Introduced various size based limits
    1. FTP Task, the file cannot be more than 32 MB (recently increase to 100MB)
    2. Magnus Remote API, the request body cannot be more than 32 MB.
  • Remote APIs Optimization
  • Fixed validation issue for BigQuery Task with View as destination
  • Improved FTP-to-GCS Task
  • Workflow Editor:
    1. Parameters Panel expanded by default
    2. Each Task Panel now supplied with respective (context sensitive) Help Link (icon)
    3. Built-in parameters are no longer editable in Parameters Panel
    4. Ability to add link to Workflow’s documentation (by request only)
  • Welcome email and personalized Hello Magnus Workflow for new Potens Users
  • Trigger v2 (for Potens.io)
  • Auto login in when referred from Signup Page
  • Many/many more fixes, improvements – let us know if you see something you asked / reported but not addressed yet

January 4, 2018

  • On Workflow Save – if a newer version is detected (due to team’s collaborative work on same workflow or if user simply opened same workflow editor in different sessions and “lost track” of where there most recent changes done) user asked for confirmation (save or cancel)
  • Added new Action Icons to Workflow Editor’s Toolbar
    • Share Workflow; Copy Workflow; Disable/Enable All Tasks
  • Added Some Limitations for
    • Email Task - Max Attachment Size – 25MB
    • FTP-GS Task - Max File Size – 32MB
    • API Task - Max API Response Size – 32MB
  • User facing bugs fixed
    • Manual run of a long-running workflows does not show Workflow Completed Popup
    • When creating new workflow/trigger, if task added - user is not able to correct invalid workflow ID/trigger ID
  • Some more internal fixes and improvements
  • The rest is massive Potens.io related upgrades – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com ) 

December 13, 2017

  • On Workflow Save – notify user if a newer version is detected 
  • Fixed bug for Run/Save
  • Fixed bug Lifetime
  • Some more fixes and improvements
  • The rest is massive Potens.io related upgrade – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com ) 

December 6, 2017

  • Standard SQL is a default SQL dialect for newly created BigQuery Tasks
  • Billing Tier setting is retired so is removed now from BigQuery Task UI 
  • The rest is massive Potens.io related upgrade – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com )

November 1, 2017

  • Roles rearranged between Admin/Advanced Admin and Sys Admin
  • Updated scope for Admin views in Workflow/History/Trigger Browsers
  • Added consumption logic for Licensed and Free Trial Users 
  • Added more infrastructure items into config

September 26, 2017

  • Workflow lifetime expiration notification 
    (only for workflows with 90-93 days lifetime forced by recently introduced 90 days rule)

August 16, 2017

  • Email Task: multiple GCS attachments
  • Email Task: optional GCS attachments
  • Workflow Versions: Load/Copy Selected Previous Version
  • Fixes and Improvements

August 9, 2017

  • Workflow Versions
  • History Details Refresh
  • Fixes and Improvements

August 2, 2017

  • Get Started Examples 

July 19, 2017

  • Open in Goliath 
  • Loop Task in History Details
  • Conversational Pub/Sub Event based Triggers – Public Beta
  • Execute Workflow Task – Nested Calls Limit
  • New How To Articles created for wiki and will be available shortly

July 6, 2017

  • Open in Goliath (Beta)
  • Changes in Misc Task (Beta)
  • Loop Task in History Details (Beta)

June 8, 2017

  • GCS Event based Triggers – Public Beta
  • Conversational Pub/Sub Event based Triggers – Private Beta
  • On-Save Notification – more Enhancements
  • How to use Magnus with AWS – example code fix to properly treat all AWS regions – `Thank you!` to  Hiroshi Yabu (Xumo)
  • Some small fixes / improvements 

May 31, 2017

  • On-Save Notification Enhancements
  • Workflow Tree Explorer
  • Remote Workflow Execution v2 with Fair Scheduler
  • GCS Triggers Support – in Private Beta
  • Many small fixes / improvements

April 12, 2017

  • Breakpoint Task
  • Added Icons for IntelliSense Popup 
  • Many small fixes / improvements

March 30, 2017

  • User’s Default Billing Project (as per Magnus configuration) is now visible / available in UI.
  • Some small fixes / improvements 

March 22, 2017

  • New Feature: Ability to set Billing Project for Workflow
  • Added IntelliSense Support for new Standard SQL Functions introduced in recent BigQuery Releases

March 8, 2017

  • Added IntelliSense Support for new Standard SQL Functions
  • Remote Workflow Execution: Added remote API for getting workflow execution status by History ID
  • Introduced Billing vs Viewable Projects Concept

February 9, 2017

  • "Add Workflow" is available now from within Workflow Browser
  • History Details: Run Info Section displays parameters overrides and run user/mode
  • Workflows and history visibility based on billing project
  • Added Python code example for Remote Execution implementation 
  • Some small fixes

January 24, 2017

  • Workflow Execution via remote call
  • Workflow Execution with Passing Parameters via remote call

January 18, 2017

  • Access/Jump to Workflow History directly from within Workflow Editor
  • Ability to Pivot Record Parameter Info Popup
  • Some UX fixes, improvements – you will most likely notice by yourself 

January 5, 2017

  • Deployment Push Notification
  • Increase Workflow ID Length up to 50
    Under popular demand we had to increase size of that attribute so it is easier for you to have more descriptive IDs
  • UX fixes, improvement

December 28, 2016

  • IntelliSense and Help on Hover for ALL Legacy and Standard SQL Functions
  • Link to Documentation on Hover for ALL Legacy and Standard SQL Functions
  • Better support for System Parameters  
      No need any more to remember all valid options for specific sys parameter. 
      They are available to select from drop-down 
  • Added Billing Tier to Cost Calculation - in effect since 01/01/2017
  • Multiple UX fixes, improvements

December 15, 2016

  • IntelliSense for Parameters extended to Magic and Sys Parameters
  • IntelliSense for Parameters extended to Script, API Call and Email Tasks as well as UDF Editor (In-line mode) in BigQuery Task
  • Multiple fixes and improvements. E.g.
    • Improved error messaging for Script Task
    • Improved Keyword/Functions recognition in Rich Editor / IntelliSense
    • Double Slash is recognized as comments sign for BigQuery Legacy SQL

December 7, 2016

  • BigQuery Task: Mixed Priority
  • BigQuery Rich Editor: IntelliSense with Tooltips and Tooltips on Hover in BigQuery Standard Mode
  • IntelliSense support for Parameters with “<” + “Ctrl-Space”

November 30, 2016

  • Workflow Browser: Tasks Icons in Tasks Tooltips 
  • Light Viant Branding for Start/Login Page and Re-Login Pop-Up

November 29, 2016

  • Script Task: show standard output in History Details Popup
  • Task Details in History Browser: added Task Start Time for each Task
  • Added Tasks’ icons in Workflow Editor and History Browser (Workflow Details and Task Details)
  • BigQuery Editor: 
    • Preserve Auto Formatting (keywords capitalization) via CTRL+SHIFT+F
    • Help Hint on Keywords hover
    • Help Hints while in Auto-complete / Intellisense Mode (CTRL+Space)

November 10, 2016

  • Share Workflow via Permalink
  • Updated Home Page
  • Various small fixes

November 9, 2016

  • Company-wide release of Magnus v.3!
  • Full User Documentation

October 25, 2016 

  • Run-Time Notification to cover workflows called via Execute Workflow Task
  • Cost, Billing Tier and other stats from Child Workflows are rolled up to Parent Workflow when it is shown collapsed in History Browser’s Hierarchical Mode
  • Find/Replace and Parameters Panels are now always Visible/Accessible
  • Workflow Editor got Vertical Floating Toolbar with frequent used Shortcuts
  • Contextual Tab Titles
  • Various small fixes 

October 12, 2016 

  • JavaScript for GoTo Task’s Expression Builder
  • Navigation thru Task Details in History Details (History Browser)
  • DML Support in BigQuery Task
  • Various fixes 

October 5, 2016 

  • Last Run At new column in Workflows Browser
  • Task Details link in History Details for below tasks (more tasks added)
    1. GoTo Task
    2. Export to MySQL Task
    3. FTP Task (FTP To)
    4. FTP-to-GS Task (FTP From)

September 28, 2016 

  • Cancel Job for BigQuery Task in History Details (History Browser)
  • On-Save Notifications Support (in addition to previously ONLY available On-Run Notifications)
  • Task Details link in History Details for below tasks (previously available ONLY for BigQuery Task)
    1. Email Task
    2. Execute Workflow Task
    3. API Task
    4. Script Task

September 21, 2016 

  • New Magic Parameters
  • Multiple Rich Editor enhancements
  • Many Improvements and Bugs Fixing

September 7, 2016 

  • Full Size” Button for Rich Editor (to compliment F11 that is for PC only as of now)
  • Max Billing bytes option – allows to set billing limit on per query level
  • Admin support for recently introduced Secure/Locked Workflows features
  • Multiple fixes / improvements across
    1. History Browser: Hierarchical View
    2. Workflow Editor: Improved parameters validation
    3. Workflow Editor: fixes for Chrome Search (CTRL-F) and Magnus’s built-in Find/Replace feature
    4. Many more …

August 24, 2016 

  • Secure/Locked Workflows Support
  • Rich Editors added to all modes and popups
  • Standard SQL mode is available now for View as an Destination in BigQuery Task

June 28, 2016 

  • BigQuery Task. Rich Editors 
    • Lines Numbering
    • Active Line
    • Code Coloring
    • Auto-completion
    • Matching bracket
    • Commenting / Uncommenting (Ctlr + / or Cmd + /)
    • Full Screen mode (F11 / Esc)

June 21, 2016 

  •  1. Script Task. Rich Editors for all modes (Go, JavaScript, Lua)
    • Lines Numbering
    • Active Line
    • Code Coloring
    • Auto-completion (for JavaScript only for now)
    • Matching bracket
    • Commenting / Uncommenting (Ctlr + / or Cmd + /)
    • Full Screen mode (F11 / Esc)
  • Script Task. Go-lang Mode: 
    • Added packages:
      • crypto/md5
      • bytes
    • Added aliases for fmt.Fprint* functions
      • xFprint, xFprintf, xFprintln
  • Magnus’s output “magic” parameter automatically conforms to chosen SQL Version of BigQuery Task 

July 6, 2016 

  • BigQuery Task: Support for Inline Code for UDF Resources
  • Workflow History Browser: Hierarchical View for Workflows run with Execute Workflow Task
  • Few bugs are fixed

June 14, 2016 

  • BigQuery Task: Support for UDF Source as external files in Cloud Storage
  • By popular demand - Copy Workflow - most asked feature in last many months! 

May 25, 2016 

  • Find / Replace Feature in Workflow Design / Edit Mode
  • BigQuery Task: support for
    1. Standard SQL
    2. Max Allowed Billing Tier
    3. Parametrized Retention
  • Execute Workflow Task: added workflows type selector  

May 18, 2016 

  • Workflow History Browser: Added Workflow Filter by user’s access 
  • Workflow Browser: Search Workflow by Content
  • Support for configurable Billing Projects
  • Number of improvements and optimizations in UI and Backend …

May 11, 2016 

  • Workflow Design Time Improvements
  • White List based sharing Workflow with domain
  • Multiple UI improvements
  • Weekly reports distributed to producers of data and consumers of data under Magnus Self-Serve

May 4, 2016 

  • More options for FTP-to-GS Task:
    • ftp multiple files using wildcard
    • ftp whole folder
    • support for return parameter with list of successfully loaded files

April 28, 2016 

  • New Feature: Shared Workflows 

April 20, 2016 

  • New Task Type: FTP-to-GS Task
  • New Feature for Email Task: File in Cloud Storage as an email attachment 
  • Simple how-to example is ready – see workflow “HOWTO_EXPORT_IMPORT”

April 13, 2016 

  • New Feature: Failure Escalation for Mission Critical Magnus Workflows
  • GCS Export Task: Fix to address recent adjustments in cloud projects permissions

April 7, 2016 

  • Go support for Script Task 
  • Retention Options

March 8, 2016

  • “Advanced” Mode for BigQuery Editor 

March 2, 2016 

  • Workflow Search in Workflow History Browser  
  • JavaScript support in Script Task

January 6, 2016 

  • Mostly tuning, optimization 
  • Public access to FTP Task. This is on demand so far
  • Mini “New Tasks” Bar within Loop Task

December 10, 2015 

  • Enhanced API Task – allows supporting wide range of REST API, including AWS (s3, ec2, etc.) ones 
  • Billing Tier in Workflow History and History Details

October 1, 2015 (v.2 Release)

  •  See announcement!

September 9, 2015 (v.2 Beta)

  • Greatly Improved stability and exceptions (mostly from google cloud) handling - that was priority #1 for v2
    • Multiple fixes and improvements, mostly backend related
    • Retry logic and many more
  • Better and friendlier workflow UI including but not limited to:
    • Collapsing/expanding tasks; all and individual
    • Hiding disabled tasks
    • Targeted positioning of new task
    • Advanced Properties panel for some tasks
  • New BETA Features
    • Loop Task: Loop Specific On-Failure Behavior. Skip to next iteration
    • Record Parameter: Referencing Items Count and Fields Name
    • GoTo Task: Expressions based evaluation
    • Script Task: Lua is the option for scripting as of v2 release. Passing workflow’s parameters in and getting result out to parameter(s) is supported
    • MySQL Export Task: this is to support [for now] light (below 500MB at a time) data export from GBQ to MySQL. Please note: Performance improvement will be a subject for v4, after which 500MB limit will be eliminated
    • FTP Task: FTP, sFTP and FTPs are supported. For DW internal use for now. Will be opened to public in v3
    • API Task: For Advanced Users - any BigQuery and/or Cloud Storage APIs can now be part of your workflow as a separate Task – still in testing. Planned to be available to public by Final v2.

August 24, 2015 (v.2 Alpha) 

  • Expand / Collapse Workflow’s Tasks
  • Expand / Collapse Single Tasks
  • Hide/Show Disabled Tasks
  • Add New Task to selected position
  • Task Behavior On-Failure
  • GS Export Task - Additional Options for Export Format
  • Email Task - HTML and Attachment Support
  • Loop Task - Loop Support
  • Misc Task
  • Execute Workflow Task
  • Record Type Parameter

May 20, 2015 

  •  Email Task 

May 20, 2015 (v.1 Beta)

  • DW Self-Serve Tool Beta is available!

May 4, 2015

  •  Workflow Notifications 

April 13, 2015 (v.1 Alpha) 

  • Multi-Steps Workflows
  • Orchestration & Scheduling
  • Different Tasks Types to implement most common flow patterns
  • Extensions/Extras/Parameters
  • Simple UI
  • Safety & Security
  • New Features
  • Sneak Preview Videos