Release Notes

March 29, 2023 

New in Magnus  

New in Potens.io this Week: 
Magnus:

1. Go-To Task. Multiple IFs are now allowed 
   Added multiple IF/ELSEIF comparisons and a THEN task for each, in addition to one final ELSE task. User can add as many If comparisons as they need, as well as modify, reorder, and remove. 
   
   
   
2. BigQuery Trigger. New Events added – Table Created and Table Truncated 
   
   
   
3. Triggers. Built-in Parameters are available now in Triggers
   Allow use of certain built-in parameters (such as var_date, etc.) when creating/updating triggers.
   
   
   
4. Triggers. Pause Trigger - added ability to disable Execute Workflow Task 
   
   
   
5. Triggers. Trigger Expiration Notification - notify users of about to expire long term triggers 
   
   Goliath. Added some BigQuery GA Features:  
   
   
6. Authorized Datasets – Datasets can now be authorized via Add Access.
   https://cloud.google.com/bigquery/docs/authorized-datasets
7. Case Insensitive – Option added to Create Dataset and info added to dataset popup.
   https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#creating_a_case-insensitive_dataset
8. Default Collation – Options added to Create Dataset, Modify Dataset Info, Create Empty Table, and Modify Table Info. Info added to popups.
   https://cloud.google.com/bigquery/docs/reference/standard-sql/collation-concepts

March 23, 2023 

New in Potens.io this Week: 

1. Magnus. Workflow Trigger - added new trigger event. 
   When watched workflow completes execution, another workflow is triggered to run.
2. Goliath. IntelliSense for Wildcard Tables
   Tables with wildcards (*) are now identified within IntelliSense and CTRL + Click now supported to navigate to the appropriate node in the Navigator.

March 2, 2023 

New in Magnus  

• Org Hierarchy – added support for parent-child orgs relationship.
  The parent-child organizational structure enables collaboration between organizations by allowing the delegation of users from a parent org to child orgs. This structure is useful for organizations that have multiple departments or teams or even initiatives/projects that needs to be [for whatever reason it might be] handled separately. It also covers scenario when organization (e.g., Viant) has multiple clients (e.g., DLP Clients) which needs to be treated as a separate org, but sales engineers still need have access to client’s org as Users or Admins to help/support Client’s needs. Yet another potential use is to separate production, stage, and dev space for Magnus developers. There are plenty of real-life applications of such structure.
  
  
• Snowflake API support. v 0.2 – still in Beta – added more features 
  
  • Role Selection for Granted Token (was PUBLIC only in v 0.1)
  • Some Request Fields auto-filled with starters (URL, Body, Header) 
  • Warning in UI of Token Expiration
  • Notification via Email of Token Expiration (in alpha)
  • Support in Admin Suite (still WIP – to be available in next v 0.3)

February 9, 2023 

New in Magnus  

• Support for Snowflake API in Magnus via Task API (Beta v 0.1) 
• Some minor fixes and improvements

September 13, 2022

New in Potens

• Magnus: (New Feature) Workflow Debugger  
  Control Panel: Start/Pause/Stop/Exit Debug, Run, Step Into/Over/Out
  Runtime Parameters Panel: monitor all parameters (custom, built-in, magic, sys, etc.) in real time  
  Runtime Output Panel: detailed debug log in real time, tasks executions details, stats  
  Watch List Panel: set Parameters, JavaScript Expressions to watch in real time; set Break on change / conditional breakpoints  
  
  

August 22, 2022

New in Potens

• Magnus: BigQuery Triggers
  - On Insert/Delete/Update Row(s)
  - On Delete/Expire Table(s)
  
  
• Magnus: Rename parameter
  Added ability to rename parameter in workflow editor
  
  
• Goliath: Information Schema Views (ISV)
  Support for ISV in Navigator and IntelliSense
  
  
• Goliath: Support for BigLake, BigQuery Omni Data Sources
  Explore, Create and Manage Connections and respective External Tables:
  - Cloud SQL - MySQL and PostgreSQL
  - BigQuery Omni - BigLake on AWS
  - Cloud Resources – BigLake and Remote Functions

June 28, 2022 

New in Potens 

• Magnus: Workflow Billing Project – Support for list of billing projects in Workflow 
  If user supplies a comma-separated list of billing projects (for custom billing project at workflow level, or for the specific tasks), Magnus will select one randomly at run time. 
  
  Improvements/upgrades
  
  
• Goliath: IntelliSense – Improve table reference identification in query text
  Previously, tables in query text would take time and interaction to be identified in IntelliSense.
  Especially when added right after page load.
  Changes have been made to improve identifying tables in query text.
  
  
• Processing Locations - Added Santiago to the list of processing locations
  
  
• Magnus: Better message when creating new workflow with existing already workflow name 
  Better message when creating new workflow and the workflow ID already exists
  
  
• Migration to Google Identity Platform
  Completed migration to new Platform and respective library for users’ login and auth.  Previously used libraries to be retired by Google

June 20, 2022 

New in Potens 

• Goliath: Some keyboard shortcuts extended to tables references with parameters
  1. 1. Highlight table name in Editor (Ctrl)
        
     2. Locate table in Navigator (Ctrl-Click)
        
     3. Locate & Preview (Ctrl-Shift-Click) table
        
        
        
• Magnus: Workflow execution time alert
  Added new magic parameter var_sys_execution_time_alert_minutes.   
  When value is set to greater than 0 and workflow’s execution time is greater than or equals to that value - alert is sent to workflow owner (recipient by default). As always, recipients can be set using var_sys_notification_recipients magic parameter
  
  

Some fixes / improvements

• IntelliSense – updated with all recently introduced by BigQuery Team keywords, phrases, syntax, etc. 
• Goliath: Snapshot Table – Added Preview Menu Option. Snapshots can now be previewed
• Admin Suite: Groups Update Log – Added new column “Updated By” 
• Export Table (Goliath), GCS Export Task (Magnus) – Different minimum scopes are required now for exporting to GCS. 
  Previously devstorage scopes were enough to export to GCS. Now bigquery or cloudplatform scopes are needed. This is reflected in incremental auth when performing these actions.
• Goliath: Pick Datasets Modal – respect spacing (compact, normal, large) settings
• Goliath: Mark as Favorite – Parameters values were not saved when option is checked
  Previously, checking parameters when marking as favorite would not save parameters and their values. Now parameters are correctly saved.
• Admin Suite: Roles Transfer Log does not expand – fixed bug where single quote in toUser column or null value in request json body prevent entry from expanding when clicked on the plus icon
• Magnus: Dashboard – Incorrect values in cards when selecting another user (not session User – which is feature available for Admins)
  When selecting a different user, the values in each card was incorrect. It would show values for the current user. This is fixed so that now the correct count for the user is displayed.
• Magnus: Workflow editor – fixed edge case bug when workflow save was allowed even when there is no change
• Github export/import – fixed bug when devstorage scope is asked even when it is not needed
• Magnus Dashboard: add messaging for workflows with executions that are not rendered.  
  Added a warning modal listing “problem workflows” with daily executions greater than 2000 which are not visualized. 
• and more ...

May 25, 2022 

New in Potens 

• Goliath Command Palette - user interface element that gives users quick access to most commands, actions, settings available in Goliath – use Shift+Ctrl+K (Win) or ⇧⌃K (Mac) to invoke Command Palette
  The command palette lists available commands along brief descriptions and documentation links.
  
  
  
  
• Some fixes / improvements 
  • Goliath: Listing Google Cloud Storage now available under devstorage.readonly scope
  • Goliath: Table Preview under GoBQ mode
  • Goliath: Choppy Highlighting when cursor is moving over object icons
  • Goliath Pin/Unpin: Mispositioned Cursor leads to wrong pin to be act upon
  • Magnus Failure Dashboard: Improved “By Error” and “By Workflow Error” reports
  • Magnus: Improved API Template Selector
  • Magnus: Default Timezone is not sticking beyond user's session
  • Magnus: Improved handling mis-formatted email addresses
  • Magnus: fixed record parameter handling when converting Execute Workflow Task to API Task
  • Magnus: support standard BQ table syntax in inputs such as in bq task destination table name.
    Supports project.dataset.table in addition to project:dataset.table
  • Magnus: add support for Exact Search in workflow browser to disable searching for labels and instead allow searching for literal colons.
  • Admin Suite: Service account key presence indication – On Users page, added indication if valid private key or hmac key is registered for service accounts
    and more …

March 9, 2022 

New in Potens 

• Magnus: Kill Workflow - Allow users to kill workflow from the history details dialog.
  This feature previously was available for Magnus Admins only.
  Now it is available for Admins, Owner, Delegates, and the run user (such as in the case of a shared workflow).
  Kill Workflow button can be found by clicking on the Workflow Running status line at the bottom of the history details dialog, if the workflow is currently running and they have the necessary permission.
  
  
  
  
• Goliath: Built-in Parameters – Added var_yesterdaydate to list of built-in parameters
  var_yesterday displays yesterday’s date as YYYY-MM-DD.

March 2, 2022 

New in Potens  

• Magnus. More Integration with GitHub
  User can set GitHub location associated with the workflow, pull, commit, and auto commit changes.
  
  
  
• Magnus. New built-in parameters: var_yesterdayDate and var_shareLink
  var_yesterdayDate is the same as var_yesterday but with hyphens separating each component (2022-02-24 instead of 20220224)
  var_shareLink is URL to the workflow in Magnus, the same one from the Share Link button.
  
  
  
  
• Magnus. Support for User Groups
  Added Groups support to workflows’ Delegates and sharedWith users. 
  Magnus Admins can now manage allowed Groups via Admin Suite, so then Groups can be used in Workflows  
  
  
• Admin Suite. Bulk Role Transfer API
  Bulk Role Transfer now can be invoked from outside the Potens.io via respective API.
  API for initiating Transfer process as well as API to retrieve status/state of Transfer Process now available for remote execution by Admin account
  See https://wiki.viantinc.com/display/PAS/Role+Transfer+APIs for details
  
  
• Magnus Editor. Workflow Graph View 
  Added a graphical navigator to the right of the workflow editor which allows quicker access to tasks by scrolling to a task when clicking its icon in the graph. Graph View allows to see whole workflow outline at once. Also, when workflow is in edit mode, Graph View reflect changes on fly, etc.
  
  
  
  
• Goliath. More Integration with GitHub 
  User can set GitHub location associated with given workspace/tab, commit changes, commit on run, auto commit on run, etc..
  
  
  
  Sidenote: Why Integration with GitHub is important!? Just few points …
  
  • GitHub is a repository
  • It makes it easy to contribute to your and your Team’s projects
    There is a wiki and issue tracker that makes it easy to include more in-depth documentation and get feedback about your project. If you want to contribute, you just fork a project, make your changes and then send them a pull request using GitHub web interface.
  • Track changes in your code across versions
    When multiple people collaborate on a project, it’s hard to keep track revisions—who changed what, when, and where those files are stored. GitHub takes care of this problem by keeping track of all the changes that have been pushed to the repository. You can have a version history of your code so that previous versions are not lost with every iteration.
  • Documentation
    By using GitHub, you make it easier to get excellent documentation about your project. 
  • And many more …

February 2, 2022 

New in Magnus – January 2022 

• Rename Task in Workflow editor – added ability for user to rename task 
  
  
  
  When user renames using this feature, references to the old task ID in the workflow definition will be adjusted for you.  They are:
  • Dependencies to the old task ID are updated to the new task ID
  • References to magic parameters of the old task ID are updated to the new task ID.  The magic parameters are: var_{taskId}_output, var_{taskId}_return, var_{taskId}_failed, var_{taskId}_failure_message
• Export / Import Workflows - Integration with GitHub (Phase 1) 
  Added ability to export workflow to GitHub and import from GitHub it in the import/export dialogs.
  Magnus allows users to export a workflow to GitHub and import it back into Magnus. From the Import and Export dialogs in the left navigation menu or the workflow menu in the workflow browser, you can connect to GitHub, explore your repositories, branches, and files, and import/export your workflows. 
  
  
  
  Note: for Trial Users Public GitHub is used  

January 5, 2022 

New in Magnus – December 2021

• New magic parameters – Added run-time magic parameters: var_sys_runUser, var_sys_runMode, var_sys_historyId, var_sys_parent
• Signed URLs for Email Task - Added ability to generate Signed URLs for GCS files in Email Task.
  Instead of attaching the files to email - user can choose to generate Signed URLs for the GCS files 
  A Signed URL is an URL that provides limited permission and time to make a request.
  It contains authentication information in the query string, allowing users without credentials to perform specific action on a resource.
  Anyone provided with the Signed URLs can view or download the GCS files within a time limit, and without needing to sign into Google.
  This feature introduced to support scenarios when attached file(s) exceed limitations that most providers have.
• Remote API to Lock/Unlock workflows – Added Remote API for Org Admins to Lock/Unlock workflows.
• Timeline Dashboard – User can now save Timeline Dashboard as PDF
• Dashboards (Success, Inventory, Recent Changes) – User can now Export the whole Dashboard to Google Spreadsheet 

Admin Suite

• Download as CSV – Added the ability to download Orgs, Users, Billing Projects, Configuration and Roles Transfer Log info as CSV.
• Service Account HMAC Key – Added ability for Admins to add/update HMAC key of service account (required for generating Signed URLs)

November 8, 2021

New in Magnus – October 2021

1. Comments on Save Workflow, Version Comments & View Change Log 

   When workflow’s version is Saved – user can provide Comments for What and why was created/changed; What else needs to be done in next versions, etc. It is free text so anything relevant can be supplied. In addition to user’s comments - Magnus auto-generates some high-level change log entries for user to keep track of what is going on. User can enter comments multiple times during work on same version – by clicking Save – entering comments and then instead of confirming Save – just click Continue – so no need in remembering all small details and rather do commenting incrementally (comments and auto-generated entries are available for new versions only)  
   
   
   ‘View Change Log’ – menu item was added in Workflow Browser
   
   
   
   
2. Import/Export Workflow – User can work and collaborate on workflow outside of Magnus and get it back when ready.
   Magnus now support Export workflow’s latest version as YAML file to further work on workflow outside of Magnus UI.
   
   
   
   When ready, user can Import edited file back to Magnus as a new version
   
   
3. Download Content of Browsers, Dashboards, Activity
   User can download whole content of Browsers and Dashboards as a CSV file to conduct analysis on usage, inventory, etc.
   Supported pages: Workflows and History Browsers; Failed Workflows, Inventory and Recent Changes Dashboards; Activity Tab on Home Page
   
   
   
4. Home Page Customization
   User can customize Welcome and Alerts Tabs of the Home Page
   
   
   
   Cards can be rearranged. User can disable/enable (hide/show) Cards of their choice as well as resize them
   
   
   
   
5. MoreMagic Parameters
   New magic parameters were introduced:
   var_sys_owner, var_sys_delegates, var_sys_sharedWith,
   var_sys_description, var_sys_lifetime, var_sys_schedule, var_sys_qCron,
   var_sys_isPublic, var_sys_isLocked, var_sys_isCritical
   
   Also default value of var_sys_save_notification_recipients magic parameter has been changed to owner + delegates

October 13, 2021

1. Added contextual menu in History Browser to perform multiple actions previously available from with Workflow Browser only.
   
   
2. Optimized performance for Workflow Editor and Diff Tool
   
   
3. Improved performance for loading Workflows Browser, History Browser and History Details

July 30, 2021

1. Magnus. API Task: added support for binary response type
   
   
   
   Few use cases as an example (from actual Feature Requests):
   
   • User can download an image from tableau API and upload that image to google cloud storage or use it in Email Task to send as a report, etc.
   • User can download an image (max API response in Magnus is 32 MB) via API Task and use that output with Vision API to annotate that image

   Check out How-To Workflow for details:
   HowTo_Download_and_Process_Binary_API_Response
   https://magnus.potens.io/?l=SG93VG9fRG93bmxvYWRfYW5kX1Byb2Nlc3NfQmluYXJ5X0FQSV9SZXNwb25zZQ==  

2. Magnus. Workflow Editor: added system parameter selector
   
   
   
   

June 15, 2021

1. Home Page - Activity Tab: Added “Workflow delete” and “Trigger delete” to filter types.
   
   
   
   
2. Workflow Editor:  Unintended effects when pressing Enter 
   Fixes an issue where pressing enter key when focused on an input element in the Workflow Editor or Diff Tool causes the page to redirect.
   
   
3. Dark Mode: Fix Workflow Editor background (edge case caused by new Home Page) 
   Fixes an issue where toggling dark mode doesn’t also toggle the Editors background color.
   
   
4. Purge Archived Workflows Process: When deleting too old, archived workflows, delete associated triggers as well.
   
   
5. Email Notification: Added Magnus Engine version to workflow failure/success email notifications 

May 27, 2021

1. Magnus: new Home Page – cockpit style with quick views on what is going on, inventory, activities, alerts, etc. 
   
   
2. Magnus. Workflow Editor - Auto copy Shared Link to clipboard when Share Link button is clicked

April 7, 2021

1. Magnus: Workflow Labels for API Task 
   
   Workflow Labels now propagated to API Task (this feature previously was released for all Task types having explicit call to BigQuery API)
   If API Task starts a BQ job, propagated workflow labels will automatically be included in the request body sent to the BQ service. If a propagated workflow label key is already present in the API task request labels, the label already in the request has precedence. Propagated workflow labels are included up to the google cloud label limit (64), any in excess are excluded. 
   
   
2. Admin Suite: Transfer workflow roles in bulk 
   
   
   
   Admin users can transfer a user’s workflows and triggers roles to a different user by completing Transfer Roles Form on the Users Page. 
   
   
   
   Upon Completion, Admin and involved users will get Notification Email with all relevant details. 
   All submitted requests with details and results of the transfer are available in Roles Transfer Log Page.
   
   

March 18, 2021

1. Workflow Labels. 
   
   
   
   New panel added in Workflow Editor – Labels Panel – so user can add up to 10 labels (key and optional value) to be then used for search in Workflow Browser  
   
      
   
   Workflow can be searched by whole label using key:value format or by key only using key: format or by value only using :value format 
   
   Most importantly – user can propagate labels down to BigQuery Jobs executed by workflow – for this label must ne set using respective arrow-down icon – it should be green to be propagated. When label propagated – it can be used to slice and dice cost and usage by label – for example via use of INFORMATION_SCHEMA.JOBS_BY_* (see more at Getting jobs metadata using INFORMATION_SCHEMA) or in Workflow History Details or in Goliath’s Job History 
   
   Note, there is one built-in label with key workflowid with respective value of workflowid. It is set to be always propagated. 
   
   
2.  Timeline Dashboard. Multiple Fixes and Improvements 
   
   

February 18, 2021

1. Magnus: Dashboard – Added Next Run At (Column, Info, Visualization) 
   “Next Run At” column/info added in Workflows Section of the Inventory Dashboard 
   
   
   …and in the Workflow Info Popup indicating the time of the next execution for the workflows. 
   
   
   
   Additionally an icon has been placed on the timeline indicating the next run with some metrics/stats (on icon hover) and Timeline now can be sorted by Next Run At
   
   
2. Multiple fixes and improvements based on your feedback. 

February 3, 2021

1. IntelliSense – (Magnus, Goliath) Added and updated Keywords to reflect most recent new and updated BigQuery features.  
   Added keywords: SUBSTRING, ASCII, CHR, INITCAP, INSTR, LAST_DAY, LEFT, OCTET_LENGTH, REGEXP_INSTR, REGEXP_SUBSTR, RIGHT, SOUNDEX, TRANSLATE, UNICODE, KEYS.NEW_KEYSET, KEYS.ADD_KEY_FROM_RAW_BYTES, AEAD.DECRYPT_BYTES, AEAD.DECRYPT_STRING, AEAD.ENCRYPT, ST_CLUSTERDBSCAN, ARRAY, ML.ARIMA_COEFFICIENTS, ML.FEATURE_IMPORTANCE. Added keyword statements CREATE EXTERNAL TABLE, DROP EXTERNAL TABLE, ALTER TABLE ADD COLUMN, EXPORT DATA, TRUNCATE TABLE, ADD COLUMN, ADD COLUMN IF NOT EXISTS, EXPORT DATA, EXPORT MODEL. 
   Updated keyword descriptions for ST_GEOGFROMGEOJSON, ST_GEOGFROMTEXT, REGEXP_EXTRACT, ST_NPOINTS, JSON_QUERY, JSON_VALUE.
   
   
2. Processing locations – (Magnus, Goliath) Added new locations
   Added Iowa (us-central1), Oregon (us-west1), Belgium (europe-west1), and Netherlands (europe-west4)
   
3. Multiple fixes and improvements based on your feedback. 

January 27, 2021

1. Magnus: Remove/Cancel Queued Workflows – user can prevent a queued workflow from running with the “Remove Workflow/Iteration from Queue” button in the History Details Panel
   
    
   
   Only Admins, Workflow Owner and the User who ran the workflow are allowed to cancel. The button is not shown otherwise.
   Any workflows waiting on such a cancelled workflow will raise failure with message “workflow cancelled”, or “iteration cancelled” in the case of an async loop iteration. It is up to user to handle the error as desired
2. Magnus: Custom Billing Projects per Task - added billing project selector dialog so user can select from the list of available billing projects.     
3. Multiple fixes and improvements based on users' feedback. 

January 21, 2021 

1. Custom Billing Projects per Task - User can specify a billing project per Task where applicable, different from the workflow’s default billing project.
   Previously, this was available for Execute Workflow Task (and obviously in API Task as it is part of the request body) only – now it is available for all Tasks where GCP billing involved (namely BigQuery Task, Cloud Storage Export Task and FTP Out Task).
   Respective controls can be found on the Flip side of Task Panel:
   
   
   
   Billing project can be set using parameter – so you can make it a function of some condition (like time of the day, etc.)
   Note: Task level billing project is ignored in case if/when Billing Project Cascading is enabled on Workflow level
   
   
   
   

January 13, 2021 

1. Loop Task – Added new options for Iteration Data
   
   • Iteration over Number Range    
   • Iteration over Date Range
     
     
     
     
2. Go-To Task within Loop Task – Added more Action options
   • Continue Loop – skip the rest of the tasks in Loop and continue execution to the next iteration
   • Exit Loop – skip the rest of the tasks in the Loop, skip the rest of the iterations and jump to the next Task after the Loop Task
     
     
     
     

December 21, 2020 

1. Workflow Browser – Workflow Tree View: Added support to recognize child workflows called via remote API as API Task. Previously only child workflows called via Execute Workflow were recognized here.  
   
   
   
   In above example - previously, only “export async” Execute Workflow Task was recognized as a call to “pre_cache_advertiser_campaign_export” workflow and thus was ‘drilled’ down to that workflow definition and shown in Tree View.
   The “export_async_COPY1” API Task was not recognized as a child workflow call and thus was presented just as API Task with one line.
   Now, we recognize that this particular API Task actually does remote call to child workflow thus user can see that workflow definition in Tree View.
   
   
2. Workflows Browser: Added Next Run At column – to show next scheduled time (obviously for scheduled workflows only).
   
   
   
   Additional feature here – by clicking on date-time value in this column user can see all involved child workflows including nested calls (this is a trimmed down version of  Workflow Tree View with ONLY child workflows shown to have a glance on what else workflows are involved) 
   
   
   
   Below is yet another example with more complex case – so you can appreciate this trimmed-down version and focus on child workflows calls only
   
   

December 15, 2020 

1. Incremental Authorization:  During signup, user is asked for the most basic scope, bigquery.readonly, to perform actions within the applications. Some actions may need additional permissions and when initiated - a prompt requesting to grant them is presented. This flow is known as incremental authorization and introduced mostly to relax required permissions for trial users and new clients onboarding. Check out documentation here -  https://potensio.zendesk.com/hc/en-us/articles/360052386432-Incremental-Authorization-
   
   Note: this is Beta release – please let us know if any issue noticed
   
   Important recommendation: Current Poten.io Clients who are adding new users after incremental authorization has been released (early December 2020) can recommend these users to grant the below scopes in advance to bypass incremental authorization process
   
   
   • bigquery
   • storage.full_control
   • cloud-platform
   • drive scopes
     
     
2. Number of fixes and improvements. Some items:

• Improved Errors extraction for Failed Dashboard in Magnus
• Fixed rowcount parameter for UPDATE DML in BigQuery Task
• Improved failure notification emails
• and many more …   

August 25, 2020 

1. Billing Project for Execute Workflow Task:  Support for use of custom parameter in Billing Project attribute in Execute Workflow Task.
   
   
   
   This allows to easily control the billing project on the weekend and after hours vs. working hours to save some more $$$ 
   
   
2. Locked Workflows: Workflow can be locked into read-only mode by admins
   Admin can Lock and Unlock workflows from within the Workflow Browser.
   Locked workflow are marked with lock icon to the left of workflow id in Workflow Browser and Inventory Dashboard. Also lock icon appears to the left of Save button in Workflow Editor
   The Workflow Editor is locked into read-only mode until unlocked by an admin. While workflow is locked for editing  - it still can be run by any available means

July 14, 2020 

1. Dark theme – finally Dark Theme available in Magnus
   Use CTRL+M to switch between Light and Dark Themes
   
   For example, Light Mode
   
   vs. Dark Mode
   
   For the best experience consider also using respective settings in System Preferences
   
2. Row Count output parameter for BigQuery Task
   User can select a parameter to receive the produced or affected row count.
   
   
   
   
   
3. Diff Tool: ability to shift/slide down/up pair of versions 
   Users may shift the pair of Workflow versions to compare by clicking  the up or down buttons in the Diff Tool.
   
   
   
4. Documentation links: Diff Tool – Documentation link and title fields are now treated as independent within the Diff Tool.
   The workflow documentation links are now highlighted to indicate differences independently for Link and Title fields.
   
   
5. Dashboard: Cron Helper for Schedule Column – A readable Cron expression tooltip was added to the Inventory Dashboard.
   A tooltip with a human readable form of each Cron expression is viewable upon hovering over the Inventory Dashboard workflows table schedule column.
   
6. Null parameter substitution (IMPORTANT – behavior change/fix)
   Substitute null parameter value with “null” or “nil” (depends on context) during parameter substitution. 
   Currently substitution is skipped if parameter value is null. 
   
   
7. Recent Changes Dashboard:
   Last # days drop down has been added to recent changes dashboard.
   The drop down impacts recent changes dashboard only.
   
   
   
   Options include last 7 days, last 14 days, last 30 days, last 60 days, last 90 days, last 120 days, and last 180 days.
   
   
   
   
8. Number of fixes and improvements (like few example below)
1. Fix/improve parameter not defined warning
   Fix parameter not defined false alarm with record when using :var_rec without subscripts.
2. Inventory Dashboard: Workflows Table Columns
   Inventory Dashboard table column widths were reduced and tooltips were added.
   Workflow Owner, Delegate and  Schedule columns were made narrower.
   Tooltips were added to show additional data that is cropped by the narrowed columns
3. Fail fast if no refresh token 
   Fail immediately when Task requires access token but user does not have refresh token or valid access token

June 23, 2020 

1. Set Workflow’s Documentation Link – user can now set / manage a workflow documentation link from the left menu which is shown at the top of the workflow. 
   
   
   
   
   
   
2. Schedule validation – Added schedule cron validation on workflow save.  
   
   

June 18, 2020  

1. Loop Task: Async Mode  - Beta Feature! Work in Progress! 
   Async Mode can be enabled on a Loop Task, and allows the loop to run multiple iterations at the same time, rather than one at a time in sequential order. Concurrent Iterations specifies how many loop iterations can run in parallel. For example, when Async Mode is enabled and Concurrent Iterations is 5, it means it will run up to 5 loop iterations in parallel; when one iteration finishes, it will start another iteration, until 5 are running or the loop is completed. Parameters within the async loop can be referenced as usual, however any parameter updates within the async loop will be accessible only within that iteration and will not persist to the next iterations or after the loop completes.
   
   User can set Async Mode on the Flip Side of Loop Task 
   
   
   If Loop is set for Async Mode – respective icon indicates this on Task’s Front Side 
   
   
   
2. Timeline Dashboard – Added visualization for Async Loop’s iterations.
   Async Mode is available and all async iteration executions are displayed in the Timeline Dashboard. Such Async executions are identified by a dotted border. Popups for async executions display the workflow name and iteration start time along with the task name and iteration. Clicking on an async execution displays the iteration run details. Async executions are available in the timeline dashboard only.
   
   
   
   another example
   
   
   
3. Fair Scheduler
   Increased execution frequency of Fair Scheduler and Dequeuer execution
   Now it is every 10 sec vs. previous 1 min 
   Note: Fair Scheduler is used for handling queued remote workflows executions. Regular Scheduler is still limited to 1 min
   
   
4. Cascade Workflow Kill – When “killing” a workflow, it will now cascade kills/cancels all descendent workflows (child workflows, grandchildren, and so on)
   When a workflow is explicitly “killed”, the system will also find all of its child workflows, and child workflows of those child workflows and so on, and will kill all such descendent workflows. If any of these workflows are pending in the queue and have not yet started execution, they will be cancelled and never start.
   Note: Kill Workflow feature available for Org Admins only!
   
   
5. Helper / Explainer for Quartz Cron Expression 
   Present a quartz cron expression as a human readable text / descriptions that clearly explains when it will be executed 
   Now supported in Workflow Editor only 
   More cron patterns supported:

• optional “Years” position 
• Some “Days” patterns: “L”, “LW”, “L-3”, “7#4”, “31W”, “3L”, etc.  
• And more 
  Let us know if you identify any cron being not properly described 
  
  
  

June 2, 2020  

1. History Browser: Added Days dropdown to limit how many days to look back to get history for.
   
   
   
2. API Task: Added option to pass ID Token instead of Access Token.
   This is to cover use case of calling GCF (Google Cloud Function) via http
   
   
   
   
3. API Task: Added options to specify max attempts count and sleep duration (below shown values are max values and used as default).
   
   
   
   
4. API Wizard: API grouped now by Categories: Advertising; AI and Machine Learning; Big Data; Blog & CMS; Compute; Data Analytics; Developer Tools; Firebase; G Suite; Identity, Access and Security; Maps; Mobile; Networking; Operations and Management Tools; Social; Storage and Database; YouTube
   
   
   
   
5. Helper for Cron Quartz:  Advanced entries have their value converted from CRON expressions into human readable descriptions.
   A tooltip has also been added when hovering over Schedule field.
   
   
   
   
6. BigQuery and Script Tasks: Code Fold / Unfold – Text editors for query and scripts can have code folded and unfolded.
   
   
7. Timeline Dashboard: Sorting - The number of Daily Executions and Daily Total Duration for each workflow is available to be sorted by
   
   
8. Inventory Dashboard: Added Backup and Archive Menu Items.
   
   
   
   
9. Recent Changes Dashboard – A new Dashboard added that lists workflows that have been created or changed recently. 
   
   

May 13, 2020 

1. [NEW] Inventory Dashboard -Added inventory section in Magnus Dashboard
   Inventory dashboard displays detailed list of workflow and triggers associated to the user/organization as well as summary and usage info. The list includes owned and delegated workflows, triggers, and (for admins only) users. Org Admin can see given user’s Inventory as well as whole Org Inventory. 
   
   
   
   
2. Timeline Dashboard- Added execution counts per workflow 
   In timeline the workflow name is listed along with swim lanes. The number of executions were added in parentheses to the right of the workflow name.
   
   
   
   
3. [NEW] Backup and Restore workflows- User can backup / restore workflows through GCS. 
   User can back up just most recent version or whole workflow with all its versions. 
   Restore can be done as a new workflow or as a new version to existing workflow 
   
   Available in Workflow Browser in Contextual Menu and Left Toolbar
   
   
   Also available in Left Toolbar in Workflow Editor 
   
   
   
4. [NEW] Batch Delete Tasks- Multiple tasks can be deleted at once
   Users can enter batch mode and select multiple tasks for deletion 
   
   
   
   
5. [NEW] Batch Copy Tasks- Multiple tasks can be copied at once
   Users can enter batch mode and select multiple tasks to copy
   
   
6. [NEW] Continuous workflows- If an instance of a workflow is already running, and the scheduler tries to execute the workflow, the scheduler will silently skip the execution.  Previously this would cause "Another instance of the workflow is already running" error. 
   
   If user wants to see such rejected executions – user need to explicitly express this in Workflow Header   
   
   
   
   
7. Number of fixes and improvements  

April 21, 2020 

1. [NEW] BigQuery Task:  Added support for STRUCT, ARRAY, or GEOGRAPHYas output to Destination Parameter
   When querying STRUCT, ARRAY, or GEOGRAPHY data types into a Destination Parameter, they will be translated into respective string representations that make them easily reusable in next Tasks. Previously these types were not supported as a output to parameters and would cause the BigQuery Task to fail.
   
   
2. [NEW] Reservation Grantees API:  Added API to update Reservation Grantees metadata
   Org Admin can call this new API to update Reservation Grantees metadata.  Historical states of Reservation Grantees are used in Dashboard for BigQuery cost calculation.
   
   
3. [NEW] Added feature of Private Keys rotation for service accounts (available for Potens Org Admins)
   
   
4. Number of fixes and improvements    
   • Parameters Drag and Drop – Fix colors/styles, added cancel esc key, fix ghost position, and highlight moved variables.
   • Some UI Issues on Zoom
   • Many more …
     

April 8, 2020 

1. Magnus Dashboard – Failed Workflows
   Added new Dashboard Page that provides main stats about failed workflows
   
   
2. Magnus Dashboard – Workflows Timeline
   Few fixes and improvements
   
   
3. Added feature of Private Keys rotation for service accounts (available for Potens Admins)
   
   
4. Number of fixes and improvements

March 11, 2020 

1. Magnus Dashboard
   Provides visualization (along with some stats) of workflow executions, success, failures, inventory and changes
2. Drag & Drop Tasks & Parameters
   Allows to change order of Tasks / Parameters in Workflow by simply using Drag & Drop approach

February 10, 2020 

Workflow Versions Diff Tool extended for Text-Based entries for below Tasks

1. BigQuery Task – Query Text
2. API Task – Request Body & Request Headers
3. E-Mail Task – Email Body
4. Script Task – Script Text
5. And few more text-based elements

Note: Diff Tool is still in Beta, so please let us know when you see something needs to be fixed – especially if it is not cosmetic o)  

February 4, 2020 

1. Workflow Versions Diff Tool v1 Beta 

January 9, 2020 

New in January 2020 and Q4 2019

1. Resumable Workflows 

2. Mission Critical Workflows 

3. Long Term Workflows – Smart Lifetime & Updates 

4. Improved Org Admins Actions & Admin Tool 

5. BigQuery Task: Support for Scripting and Stored Procedures 

6. Execute Workflow Task: Billing Projects Option for 

7. Slot Usage and Consumed SlotTime Metrics

September 17, 2019 

1. Workflow Billing: Added cascade billing project option to workflow header in workflow editor
   
   
2. API Wizard: Added API Key as a Credential options to API wizard
   
3. Some minor improvements / fixes
    

August 28, 2019 

1. API Task: Extend API Wizard with APIs requiring additional non-sensitive or non-restrictive scopes
   About 30 more APIs are available for use in API Wizard  
   
   
   
2. Some minor improvements / fixes 
   • Fixed exposure of incorrect / not-applicable compressions in Export Tasks
   • Email Task: Allow empty email address in recipients list
     
     

August 13, 2019 

1. Workflow Browser: Improved coverage of Last Run At attribute in Workflow Browser
   
   Till recently, Last Run At attribute was ONLY available for workflows run manually or directly by schedule or via Execute Workflow Task by owner or delegate or queue less-remote execution (v1). Other runs like via trigger, remote (v2) runs, via Execute Workflow Task by “shared with” users were not accounted
   Since now, all invocations are counted, so users can see better / correct and UpToDate picture of their workflows’ usage 
   
   
2. BigQuery Task: Output to Parameter now supports date/time BQ types: Timestamp, Datetime, Date, Time
   
   We still don’t support ARRAY and STRUCT data types to be used as output to parameters. Will be addressed in future releases
   
   
3. Some minor improvements / fixes
• Lifetime validation falsely identifies duration of 93 days as exceeding the max (93 days) allowed when end-of-daylight-saving lands within the lifetime
• API Wizard conflicted with some Chrome Extensions
  
  
4. API Wizard – is a great tool on its own developed by Potens.io Team
   
   Wizard currently supports about 160 Google APIs with 110 of them ready to be used w/o any extra permissions to be given. See list below
   
   
   
   Note: To use given API – it needs to be enabled – so contact Magnus Team for this
   
   In next release we plan to add 30 more APIs to be available for immediate use in API Wizard  
   
   
   

 

August 6, 2019 

New in July 2019

• BigQuery Task: Full support for Query Prefix
  
  SQL variant, either legacy SQL or standard SQL, cab be set by including a prefix to your query
  

  #legacySQL - Runs the query using legacy SQL
  
  #standardSQL - Runs the query using standard SQL

  Now, user can have BigQuery Task in the Loop running queries stored / generated with mixed SQL versions/dialects (Legacy vs Standard)
  

• Email Task: Added Email Priority Selector
  
  
  
  
• API Task: Added API Wizard
  
  
  
  
• History Details: Navigate to Child and Parent workflow in History Details
  
  Previously troubleshooting of below execution of parent / children workflows was possible only by opening History Details for each respective workflow one-by-one     
   
  
  Now, user can start from any workflow and then navigate up to parent workflow or down to child workflow directly within workflow’s History Details 
  
  
  
  
• Workflow Execution Notification Email: Added direct link to workflow
• Some minor improvements / fixes

June 26, 2019 

New in June 2019

• Execute Workflow Task: Added support for Return Parameters
  
  
  
  
• Workflow Browser: Added Manual Run feature
  No need to Open Workflow anymore for Manual Run
  
  
  
  
• Service Account registration and use to run workflows: JSON private key file is now supported (in addition to previously supported p12)
• Single Sign On (SSO): Added SSO to support integration with 3rd party tools/sites
• Minor improvements / fixes (few examples below)
  • Hub Task: increased max item’s timeout to 6 hours.
  • History Details: when workflow killed while being within Loop Task – that Loop Task and all nested Tasks were not shown in History Details which prevented user from effective troubleshooting 
  • Remote Workflow Execution: Improved stability for Remote Execution – based on quite an edge case from one of our DLP client 

April 4, 2019 

• BigQuery Task: Added processing locations São Paulo (southamerica-east1) and Zürich (europe-west6)
  
  
• Add Workflow: New button added to Goliath - "Automate with Magnus" which allows to copy Goliath's workspace into Magnus and create new workflow off of that workspace for further automation

March 20, 2019 

• Workflow Browser - Added “All Above” option
  
  
  
  
• Copy Workflow functionality now available for Workflow’s Delegates
  
  
  
  
• Minor improvements / fixes 
  As an example: API Task - Template Selector: pre-select the first template when Service selected

March 13, 2019 

• API Task - Added more API templates:
  1. BigQuery: Create External Table from GCS with Autodetect
  2. BigQuery: Create External Table from GCS with Schema
  3. Data Transfer: Export/Import To/From Cloud SQL
  4. DoubleClick Bid Manager: Retrieve Entities in SDF format
  5. DoubleClick Bid Manager: Retrieve Line Items in CSV format
  6. DoubleClick Bid Manager: Upload Line Items in CSV format
  7. DoubleClick Bid Manager: Run a Stored Query to generate a report
  8. Firebase: Generate short Dynamic Links
  9. Firebase: Get analytics data for short Dynamic Links
• Script Task - Added XML support for Golang and JavaScript
• Workflow Editor: Added Workflow Tree button to left nav bar

February 20, 2019 

• API Task - Added more API templates:
  • PubSub: Publish Message
  • Magnus: Get History
  • Artificial Intelligence - Natural Language: Annotate Text
  • Artificial Intelligence - Natural Language: Analyze Entities
  • Artificial Intelligence - Translation: Translate Ttext
  • Artificial Intelligence - Vision: Annotate Image
  • Dataflow: Bulk Compress GCS files
  • Dataflow: Bulk Decompress GCS files
  • Dataflow: WordCount Example
  • Dataflow: GCS Text to BQ
• Workflow Task - Added Asynchronous Execution Mode
• Cost attribution for BigQuery Tasks in Async mode is now presented in History Details

February 14, 2019 

• GCS Export Task - Added Asynchronous Execution Mode along with Collect Feature
• Workflow and History Browsers – light cosmetic changes: some UI controls are regrouped
• API Task - Added more API templates:
  • BQ: Copy a BQ table
  • BQ: Create a BQ View
  • BQ: Run a BQ query
  • GCS: Copy an object
  • GCS: Get metadata of an object
  • GCS: Download an object
  • GCS: Get a list of objects from a bucket
  • GCS: Delete an object
• Added new Processing Locations (Los Angeles, Hong Kong)
• Improved cost attribution for BigQuery Tasks in Async mode

February 6, 2019 

• BigQuery Task – Added Asynchronous Execution Mode along with Collect Feature
• Added New Task Type – Hub Task
• API Task – Added Collect Feature
• API Task – Added two more Templates
  • Remote execute of Magnus Workflow via Queue
  • Export BigQuery table to Google Cloud Storage
• Few fixes/improvements:
  • New recently introduced Processing Locations
  • Fix for BigQuery Job Cancel Feature

January 24, 2019 

• Templates are introduced in API Task 
• Improved features / performance of Workflow Browser and History Browser
• Workflow Editor is now opened in a new tab when view/edit/archive/add icon is clicked.
  History Browser is now opened in a new tab when history icon is clicked
• Few fixes/improvements:
  • Over-validation for GCS buckets when creating/updating Trigger
  • Improved performance of BQ Task to poll BQ job status the first time at 1 sec, then at 5 sec, 10 sec, up to 1 minute interval. Before the first poll was at 5 sec

December 12, 2018 (Q4'18)

• Triggers are now updatable
• MySQL Task is now executed within SQL transaction
• FTP Task: user can specify custom port in FTP path
• Loop Task: sleep time between iterations reduced to 100 ms
• Workflow Editor:
  • Added “Extend Workflow Lifetime” button for quick extension to another 90 days – only active long-term workflows are qualifie
  • “Show Workflow Execution History” action icon in left Tool Bar now opens History Browser for respective workflow in new Tab
• “Magnus Long-term Workflows About to Expire” Notification Email now has direct link to respective workflows
• Added new Processing Locations to BQ Task, GS Export Task, FTP Task, FTP-GS Task.
  • London, Singapore, Sydney, Northern Virginia, Finland and Taiwan
• Waiting in Queue Remotely Executed Workflows are now listed in History Browser with empty Start and Finish dates and with “Pending” as a Status and respective indicator
• Magnus v5 – massive upgrade to Google Cloud SQL powered backend
• Limits for Workflow’s Parameters:
  • parameter name up to 100 chars.
  • parameter value up to 18000 chars.
  • parameter schema up to 3000 chars.
  • At the end of workflow execution, parameter value and schema of workflow-owned parameters are truncated if length exceeds max allowed
• Fixes, improvements – let us know if you see something you asked / reported but not addressed yet. Few examples below:
  • Issue with Custom Projects when there is an expired user either owner or delegate
  • Issue with Email Task attachment size limit
  • Edge case: Workflow/History Browser: “workflow not found”
  • Edge case: exposing extra workflows names in Execute Workflow Task
  • History Browser in hierarchical mode: Fixed UI NaN bug for BQ cost.

August 13, 2018 (July, 2018)

• GCS Trigger: Added Wildcard Support for Folders names
• Email Task: Added Reply-To attribute with parameters support (if empty or null, in run-time, set to workflow owner)
• Processing Location: added full support in BQ Task, GS Export Task, and FTP Export Task
• API Task: added support for DoubleClick Bid Manager API and Firebase Dynamic Links API (only for internal Magnus, not available in Potens.io)
• GS Export and FTP Tasks: Added DEFLATE and SNAPPY compressions for AVRO format
• Trigger Editor: improved UX - Parameters panel expanded by default
• Workflow Editor: Improved UX – clear messaging / signaling about workflow not being scheduled and/or enabled
• Workflow History: improved / stable BigQuery Task details View availability 
• Magnus v5 - Next Generation – Google Cloud SQL powered backend: work in progress 
• Many/many more fixes, improvements – let us know if you see something you asked / reported but not addressed yet
• Below new BigQuery features - recently announced during Google Cloud Next’18 are natively supported by Magnus
  • BigQuery ML, available now in beta
    • How to train and predict regression and classification ML models using only SQL — using BigQuery ML
    • How to Do Predictive Analytics in BigQuery (Cloud Next '18)
    • How to do online prediction with BigQuery ML
    • Don’t get the Google Cloud Bill Shock!
  • BigQuery Clustering, in beta
    • Optimizing BigQuery: Cluster your tables
  • BigQuery GIS, in public alpha
    • Using BigQuery’s new geospatial functions to interpolate temperatures
    • How Geotab Drives Smart City Innovation, Using BigQuery (Cloud Next '18)

June 19, 2018  (Q2 2018)

• Created number of detailed Tutorials Articles and Videos 
• Added Time Zone support.
  1. User can choose Time Zone to use for representing dates/times in UI
  2. User can set Time Zone to use for scheduling on per Workflow basis
• Added Lifetime’s End Date to Workflow Browser
  When End Date is closer than 7 days, it is being red-colored so easy to catch!
• Enable copy of Get Started Example Workflows
• Introduced various size based limits
  1. FTP Task, the file cannot be more than 32 MB (recently increase to 100MB)
  2. Magnus Remote API, the request body cannot be more than 32 MB.
• Remote APIs Optimization
• Fixed validation issue for BigQuery Task with View as destination
• Improved FTP-to-GCS Task
• Workflow Editor:
  1. Parameters Panel expanded by default
  2. Each Task Panel now supplied with respective (context sensitive) Help Link (icon)
  3. Built-in parameters are no longer editable in Parameters Panel
  4. Ability to add link to Workflow’s documentation (by request only)
• Welcome email and personalized Hello Magnus Workflow for new Potens Users
• Trigger v2 (for Potens.io)
• Auto login in when referred from Signup Page
• Many/many more fixes, improvements – let us know if you see something you asked / reported but not addressed yet

January 4, 2018

• On Workflow Save – if a newer version is detected (due to team’s collaborative work on same workflow or if user simply opened same workflow editor in different sessions and “lost track” of where there most recent changes done) user asked for confirmation (save or cancel)
• Added new Action Icons to Workflow Editor’s Toolbar
  • Share Workflow; Copy Workflow; Disable/Enable All Tasks
• Added Some Limitations for
  • Email Task - Max Attachment Size – 25MB
  • FTP-GS Task - Max File Size – 32MB
  • API Task - Max API Response Size – 32MB
• User facing bugs fixed
  • Manual run of a long-running workflows does not show Workflow Completed Popup
  • When creating new workflow/trigger, if task added - user is not able to correct invalid workflow ID/trigger ID
• Some more internal fixes and improvements
• The rest is massive Potens.io related upgrades – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com ) 

December 13, 2017

• On Workflow Save – notify user if a newer version is detected 
• Fixed bug for Run/Save
• Fixed bug Lifetime
• Some more fixes and improvements
• The rest is massive Potens.io related upgrade – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com ) 

December 6, 2017

• Standard SQL is a default SQL dialect for newly created BigQuery Tasks
• Billing Tier setting is retired so is removed now from BigQuery Task UI 
• The rest is massive Potens.io related upgrade – supposed to be transparent for internal users (vindicotech.com, adelphic.com, viantadcloud.com )

November 1, 2017

• Roles rearranged between Admin/Advanced Admin and Sys Admin
• Updated scope for Admin views in Workflow/History/Trigger Browsers
• Added consumption logic for Licensed and Free Trial Users 
• Added more infrastructure items into config

September 26, 2017

• Workflow lifetime expiration notification 
  (only for workflows with 90-93 days lifetime forced by recently introduced 90 days rule)

August 16, 2017

• Email Task: multiple GCS attachments
• Email Task: optional GCS attachments
• Workflow Versions: Load/Copy Selected Previous Version
• Fixes and Improvements

August 9, 2017

• Workflow Versions
• History Details Refresh
• Fixes and Improvements

August 2, 2017

• Get Started Examples 

July 19, 2017

• Open in Goliath 
• Loop Task in History Details
• Conversational Pub/Sub Event based Triggers – Public Beta
• Execute Workflow Task – Nested Calls Limit
• New How To Articles created for wiki and will be available shortly

July 6, 2017

• Open in Goliath (Beta)
• Changes in Misc Task (Beta)
• Loop Task in History Details (Beta)

June 8, 2017

• GCS Event based Triggers – Public Beta
• Conversational Pub/Sub Event based Triggers – Private Beta
• On-Save Notification – more Enhancements
• How to use Magnus with AWS – example code fix to properly treat all AWS regions – `Thank you!` to  Hiroshi Yabu (Xumo)
• Some small fixes / improvements 

May 31, 2017

• On-Save Notification Enhancements
• Workflow Tree Explorer
• Remote Workflow Execution v2 with Fair Scheduler
• GCS Triggers Support – in Private Beta
• Many small fixes / improvements

April 12, 2017

• Breakpoint Task
• Added Icons for IntelliSense Popup 
• Many small fixes / improvements

March 30, 2017

• User’s Default Billing Project (as per Magnus configuration) is now visible / available in UI.
• Some small fixes / improvements 

March 22, 2017

• New Feature: Ability to set Billing Project for Workflow
• Added IntelliSense Support for new Standard SQL Functions introduced in recent BigQuery Releases

March 8, 2017

• Added IntelliSense Support for new Standard SQL Functions
• Remote Workflow Execution: Added remote API for getting workflow execution status by History ID
• Introduced Billing vs Viewable Projects Concept

February 9, 2017

• "Add Workflow" is available now from within Workflow Browser
• History Details: Run Info Section displays parameters overrides and run user/mode
• Workflows and history visibility based on billing project
• Added Python code example for Remote Execution implementation 
• Some small fixes

January 24, 2017

• Workflow Execution via remote call
• Workflow Execution with Passing Parameters via remote call

January 18, 2017

• Access/Jump to Workflow History directly from within Workflow Editor
• Ability to Pivot Record Parameter Info Popup
• Some UX fixes, improvements – you will most likely notice by yourself 

January 5, 2017

• Deployment Push Notification
• Increase Workflow ID Length up to 50
  Under popular demand we had to increase size of that attribute so it is easier for you to have more descriptive IDs
• UX fixes, improvement

December 28, 2016

• IntelliSense and Help on Hover for ALL Legacy and Standard SQL Functions
• Link to Documentation on Hover for ALL Legacy and Standard SQL Functions
• Better support for System Parameters  
    No need any more to remember all valid options for specific sys parameter. 
    They are available to select from drop-down 
• Added Billing Tier to Cost Calculation - in effect since 01/01/2017
• Multiple UX fixes, improvements

December 15, 2016

• IntelliSense for Parameters extended to Magic and Sys Parameters
• IntelliSense for Parameters extended to Script, API Call and Email Tasks as well as UDF Editor (In-line mode) in BigQuery Task
• Multiple fixes and improvements. E.g.
  • Improved error messaging for Script Task
  • Improved Keyword/Functions recognition in Rich Editor / IntelliSense
  • Double Slash is recognized as comments sign for BigQuery Legacy SQL

December 7, 2016

• BigQuery Task: Mixed Priority
• BigQuery Rich Editor: IntelliSense with Tooltips and Tooltips on Hover in BigQuery Standard Mode
• IntelliSense support for Parameters with “<” + “Ctrl-Space”

November 30, 2016

• Workflow Browser: Tasks Icons in Tasks Tooltips 
• Light Viant Branding for Start/Login Page and Re-Login Pop-Up

November 29, 2016

• Script Task: show standard output in History Details Popup
• Task Details in History Browser: added Task Start Time for each Task
• Added Tasks’ icons in Workflow Editor and History Browser (Workflow Details and Task Details)
• BigQuery Editor: 
  • Preserve Auto Formatting (keywords capitalization) via CTRL+SHIFT+F
  • Help Hint on Keywords hover
  • Help Hints while in Auto-complete / Intellisense Mode (CTRL+Space)

November 10, 2016

• Share Workflow via Permalink
• Updated Home Page
• Various small fixes

November 9, 2016

• Company-wide release of Magnus v.3!
• Full User Documentation

October 25, 2016 

• Run-Time Notification to cover workflows called via Execute Workflow Task
• Cost, Billing Tier and other stats from Child Workflows are rolled up to Parent Workflow when it is shown collapsed in History Browser’s Hierarchical Mode
• Find/Replace and Parameters Panels are now always Visible/Accessible
• Workflow Editor got Vertical Floating Toolbar with frequent used Shortcuts
• Contextual Tab Titles
• Various small fixes 

October 12, 2016 

• JavaScript for GoTo Task’s Expression Builder
• Navigation thru Task Details in History Details (History Browser)
• DML Support in BigQuery Task
• Various fixes 

October 5, 2016 

• Last Run At new column in Workflows Browser
• Task Details link in History Details for below tasks (more tasks added)
  1. GoTo Task
  2. Export to MySQL Task
  3. FTP Task (FTP To)
  4. FTP-to-GS Task (FTP From)

September 28, 2016 

• Cancel Job for BigQuery Task in History Details (History Browser)
• On-Save Notifications Support (in addition to previously ONLY available On-Run Notifications)
• Task Details link in History Details for below tasks (previously available ONLY for BigQuery Task)
  1. Email Task
  2. Execute Workflow Task
  3. API Task
  4. Script Task

September 21, 2016 

• New Magic Parameters
• Multiple Rich Editor enhancements
• Many Improvements and Bugs Fixing

September 7, 2016 

• Full Size” Button for Rich Editor (to compliment F11 that is for PC only as of now)
• Max Billing bytes option – allows to set billing limit on per query level
• Admin support for recently introduced Secure/Locked Workflows features
• Multiple fixes / improvements across
  1. History Browser: Hierarchical View
  2. Workflow Editor: Improved parameters validation
  3. Workflow Editor: fixes for Chrome Search (CTRL-F) and Magnus’s built-in Find/Replace feature
  4. Many more …

August 24, 2016 

• Secure/Locked Workflows Support
• Rich Editors added to all modes and popups
• Standard SQL mode is available now for View as an Destination in BigQuery Task

June 28, 2016 

• BigQuery Task. Rich Editors 
  • Lines Numbering
  • Active Line
  • Code Coloring
  • Auto-completion
  • Matching bracket
  • Commenting / Uncommenting (Ctlr + / or Cmd + /)
  • Full Screen mode (F11 / Esc)

June 21, 2016 

•  1. Script Task. Rich Editors for all modes (Go, JavaScript, Lua)
  • Lines Numbering
  • Active Line
  • Code Coloring
  • Auto-completion (for JavaScript only for now)
  • Matching bracket
  • Commenting / Uncommenting (Ctlr + / or Cmd + /)
  • Full Screen mode (F11 / Esc)
• Script Task. Go-lang Mode: 
  • Added packages:
    • crypto/md5
    • bytes
  • Added aliases for fmt.Fprint* functions
    • xFprint, xFprintf, xFprintln
• Magnus’s output “magic” parameter automatically conforms to chosen SQL Version of BigQuery Task 

July 6, 2016 

• BigQuery Task: Support for Inline Code for UDF Resources
• Workflow History Browser: Hierarchical View for Workflows run with Execute Workflow Task
• Few bugs are fixed

June 14, 2016 

• BigQuery Task: Support for UDF Source as external files in Cloud Storage
• By popular demand - Copy Workflow - most asked feature in last many months! 

May 25, 2016 

• Find / Replace Feature in Workflow Design / Edit Mode
• BigQuery Task: support for
  1. Standard SQL
  2. Max Allowed Billing Tier
  3. Parametrized Retention
• Execute Workflow Task: added workflows type selector  

May 18, 2016 

• Workflow History Browser: Added Workflow Filter by user’s access 
• Workflow Browser: Search Workflow by Content
• Support for configurable Billing Projects
• Number of improvements and optimizations in UI and Backend …

May 11, 2016 

• Workflow Design Time Improvements
• White List based sharing Workflow with domain
• Multiple UI improvements
• Weekly reports distributed to producers of data and consumers of data under Magnus Self-Serve

May 4, 2016 

• More options for FTP-to-GS Task:
  • ftp multiple files using wildcard
  • ftp whole folder
  • support for return parameter with list of successfully loaded files

April 28, 2016 

• New Feature: Shared Workflows 

April 20, 2016 

• New Task Type: FTP-to-GS Task
• New Feature for Email Task: File in Cloud Storage as an email attachment 
• Simple how-to example is ready – see workflow “HOWTO_EXPORT_IMPORT”

April 13, 2016 

• New Feature: Failure Escalation for Mission Critical Magnus Workflows
• GCS Export Task: Fix to address recent adjustments in cloud projects permissions

April 7, 2016 

• Go support for Script Task 
• Retention Options

March 8, 2016

• “Advanced” Mode for BigQuery Editor 

March 2, 2016 

• Workflow Search in Workflow History Browser  
• JavaScript support in Script Task

January 6, 2016 

• Mostly tuning, optimization 
• Public access to FTP Task. This is on demand so far
• Mini “New Tasks” Bar within Loop Task

December 10, 2015 

• Enhanced API Task – allows supporting wide range of REST API, including AWS (s3, ec2, etc.) ones 
• Billing Tier in Workflow History and History Details

October 1, 2015 (v.2 Release)

•  See announcement!

September 9, 2015 (v.2 Beta)

• Greatly Improved stability and exceptions (mostly from google cloud) handling - that was priority #1 for v2
  • Multiple fixes and improvements, mostly backend related
  • Retry logic and many more
• Better and friendlier workflow UI including but not limited to:
  • Collapsing/expanding tasks; all and individual
  • Hiding disabled tasks
  • Targeted positioning of new task
  • Advanced Properties panel for some tasks
• New BETA Features
  • Loop Task: Loop Specific On-Failure Behavior. Skip to next iteration
  • Record Parameter: Referencing Items Count and Fields Name
  • GoTo Task: Expressions based evaluation
  • Script Task: Lua is the option for scripting as of v2 release. Passing workflow’s parameters in and getting result out to parameter(s) is supported
  • MySQL Export Task: this is to support [for now] light (below 500MB at a time) data export from GBQ to MySQL. Please note: Performance improvement will be a subject for v4, after which 500MB limit will be eliminated
  • FTP Task: FTP, sFTP and FTPs are supported. For DW internal use for now. Will be opened to public in v3
  • API Task: For Advanced Users - any BigQuery and/or Cloud Storage APIs can now be part of your workflow as a separate Task – still in testing. Planned to be available to public by Final v2.

August 24, 2015 (v.2 Alpha) 

• Expand / Collapse Workflow’s Tasks
• Expand / Collapse Single Tasks
• Hide/Show Disabled Tasks
• Add New Task to selected position
• Task Behavior On-Failure
• GS Export Task - Additional Options for Export Format
• Email Task - HTML and Attachment Support
• Loop Task - Loop Support
• Misc Task
• Execute Workflow Task
• Record Type Parameter

May 20, 2015 

•  Email Task 

May 20, 2015 (v.1 Beta)

• DW Self-Serve Tool Beta is available!

May 4, 2015

•  Workflow Notifications 

April 13, 2015 (v.1 Alpha) 

• Multi-Steps Workflows
• Orchestration & Scheduling
• Different Tasks Types to implement most common flow patterns
• Extensions/Extras/Parameters
• Simple UI
• Safety & Security
• New Features
• Sneak Preview Videos