Navigator

Overview

Navigator lists accessible datasets for the current project as well as respective tables/views and their schemas(fields).

It also allows to filter datasets by name and labels; group and sort tables.

Hovering over Dataset or Table shows respective Info PopUp.

Right click on Project Title or Dataset /Table/View shows Contextual Menu with all available actions for that Object

See quick details below

Billing Project

1.  The navigation header displays the currently selected billing project. 2.   Right clicking on project header or left clicking on more options icon displays Context Menu.

Data Project

1.  The navigation header displays the currently selected project 2.   The count of displayed datasets out of total number of datasets 3. Right clicking on project header displays Context Menu Clicking on the project header displays the Project Browser.

Filtering

Filter by Dataset ID

Datasets can be filtered by ID 1.   In Filter by ID input box enter any portion of dataset’s name to be filtered.  2.   To remove filter either remove text from the Filter by ID input box or click clear filter datasets icon to remove all text 3.    Dataset List is filtered as text is input into the Filter by ID input box.   You can use Regular Expression in Filter by ID box

Filter by Labels

To help organize projects, arbitrary labels can be added to datasets. After datasets labeled, user can search for datasets based on label values. For example, labels can be used to group datasets by purpose, environment, department, and so on. Datasets can be filtered by label.  Labels are key value pairs that can be assigned to datasets to help organize and identify datasets.  1.  To filter by label click the Filter by Label input box.  A drop down of available labels will display.  Typing will filter the drop down list.  Click on a label from the drop down list to select it.  2.  With a label selected in the input box click the + icon to add the filter.  The + icon will only be active if an existing label is selected. 3.  The added filter will display under the Filter by Label section.  4.  Datasets with the label will now be displayed. 5.  Label value is optional.  To add value, first add label by clicking in Filter by Label input box and selecting a label from the drop down list.  With a label selected click in the value (optional) input box.  A list of available values associated to selected label is listed.  Click on a value from the drop down list to select it.  With label and value present in their respective input fields click the + icon (2) to filter.  The added filter will display under the Filter by Label section (1).  Datasets with both the selected label and value will be displayed.  Multiple values can be added to a single label filter. To add multiple values, click on the value input box again to see the list of remaining values.  Clicking on another value will add it to the list of values to filter on. 6.  Clear filter by labels.  Added filters will display under the Filter by Label section.  Clicking on Remove label filter will remove the selected Datasets can be filtered by ID and Labels at the same time

Billing Project Context Menu

Right click on the project name or left click more options icon in Nav to display the context menu.

Item	Action
Switch Project	Changes project costs are billed to

Switch Billing Project 

1. From billing project context menu select Switch Billing Project
2. Then select desired project

Data Project Context Menu

Right click on the project name or left click more options icon in Nav to display the context menu.

Item	Action
Switch Project	Changes project and loads datasets in Nav
Browse	Displays Project Browser
Recent Jobs	Displays list of Recent Jobs
Create Dataset	Opens dialogue to create dataset
Share Link	Opens dialogue to create and copy link to project
Refresh	Refreshes list of datasets
Recently Run	List of queries previous run
Favorite	List of queries selected as favorite
GoBQ Examples	Starter templates for GoBQ scripts

Switch Project

Switch Data Project contains a list of projects user has access to

Data Project List Data Project list may contain up to 4 sections. a. List projects user has been granted any project role. A project will not be listed if a user has not been granted any project role. Permissions granted only on dataset or table are not a project role. b. If a user has not been granted any project role and has only been granted permissions on a dataset or table they will need to manually add the project to Goliath. The project will not automatically appear in section #a. Section #a only lists projects where the user has been granted any project role. See #a above. To add a project, click on Display project. Projects added via Display project will display in this section. c. List of temporarily added projects. This section includes projects that are available for session only. This section is normally populated when a shared link was used for a project the user does not have in list a or b. d. Add/Remove project options add or remove custom projects for use with Goliath Switch Data Project From project context menu select Switch Data Project Then select desired project         Add Data Project From project context menu select Data Switch Project Select Display project ... Type project ID in Enter Project ID field Click OK Remove Data Project From data project context menu select Switch Data Project Select Remove project display ... In Remove Project Dialog select project from drop down to remove Click Confirm to remove

Recently Run

a. The 5 most recent queries executed in Goliath are listed in Recently Run.  The first portion of the query is used in the list.  Left clicking on a query in the list inserts the query, options, and parameters into the workspace. b. Hovering over a query listing will display a preview of the query

Favorite

	Any Query or GoBQ Script can be marked as favorite. The workspace (query, script, options, and parameters) are available to be saved when selecting favorite. To mark as Favorite Prepare query or script Click on star icon (left of query text area).  Filled star indicates a saved favorite.  Empty star indicates workspace not saved as favorite A Favorite dialogue box will appear Enter the name for the favorite Options (table name, SQL version, and other options under Show Options section can be included (checked) or not (unchecked) Parameters are included if checked and not included when unchecked Click Cancel to exit without favoriting Click Favorite to save as favorite
	Favorite List a. Favorites are listed in Favorite context menu.  Left clicking a favorite will load the it into the workspace b. Hovering over a favorite name will display a preview of the query
	Delete Favorite A filled star represents a loaded favorite An empty star indicates workspace is not saved as favorite To delete Favorite: Load a favorite Click on the star icon

GoBQ Examples

GoBQ examples are templated scripts that can be used as starting points for developing robust scripts that interact with BigQuery and Cloud Storage. Additional information can be found in wiki section User Manual > GoBQ. 1. BigQuery GoBQ examples are templated scripts that interact with BigQuery (projects, datasets, tables, etc) 2. Storage GoBQ examples are templated scripts that interact with Google Cloud Storage

Create Dataset

Create Dataset Dialog prompts for information used to create a new dataset
Click OK to submit information.
A Result Dialog will appear indicating success or failure.

Input	Description
Project	Name of Project where Dataset is to be created. Required.
Dataset	Name of Dataset to create. Required.
Description	Brief description of Dataset.  Optional
Choose Data Location	Drop down of data center geo location (US/EU/asia-northeast1)
Data Expiration (days)	Setting for default table expiration in number of days.  Optional
Partition Expiration (days)	Setting for default table partition expiration in number of days.  Optional

Datasets

Dataset Info

Hover over a dataset to display dataset information

Item	Description
ID
Creation Time
Last Modified
Data Location	displayed if available
Dataset Size	displayed once dataset opened *
Long-term Size	displayed once dataset opened *
Total Rows	displayed once dataset opened *
Expiration	displayed if available
Owner	Dataset Owner

* - reset on Project or Dataset refresh

When a dataset is open both tables and table metadata are collected. Popup information such as Last Modified, Dataset Size, Long-term Size, and Total Rows are obtained from table metadata. While table metadata is gathered a loading icon will display along with a the number of tables processed out of total tables. Once table metadata is complete the loading icons and number of tables processed out of total tables are removed.

Context Menu

Dataset Context Menu is accessed by right-clicking on dataset name  or left clicking on other options menu icon.

Item

Action

Browse

Displays Dataset Browser

Share Dataset

Modify Access List

1. Project where dataset is located
2. Dataset being shared
3. Current list of entity types and access levels. Access levels for existing entities can be updated and removed using the drop-down menus.
4. Section for adding users

a. Select entity type
b. Based on type add the user’s entity such as email or name
c. Select level of permission
d. Click add
e. Repeat for each entity
f. Click save when ready to submit

Input	Description
Entity Type (4a)	User by e-mail Group by e-mail Domain All Authenticated Users Authorized View
Entity (4b)	Corresponding name for entity type
Access Level (4c)	Can View Can Edit Is Owner

Modify Dataset Info

Modify description, expiration, labels

1. Project where dataset is located
2. Dataset being modified
3. Modify Description by adding or removing text
4. Add default expiration for dataset tables by adding number
5. Add default partition expiration for dataset tables by adding number
6. Existing dataset labels

a. Remove existing labels by clicking on the remove icon
b. Edit existing values by updating the text

7. Associate dataset with labels by

a. Adding input for key
b. Optionally associate key with a value
c. Click add to include label on save
d. Repeat above to add additional labels

8. Click OK to submit modification

Delete Dataset

1. Project where dataset is located
2. Dataset to be deleted
3. Notice of the number of tables that will be deleted along with dataset
4. To delete dataset enter the name of the dataset and then click OK

Script Tables List

Insert script list into query text area

Load Table Data

Load data from a local file or cloud storage to GBQ destination

When loading data from a local file the size limit is 100 MB for Latin1 encoded files and 10 MB for non-Latin1 encoded files.
For file sizes greater than these limits first load the file to Google Cloud Storage and from there load to GBQ destination.

1. To load table data begin with the Source File section which contains options for handling the data file

a. Select where you want to load data from (local file or Google Cloud Storage)

b. Select the file you want to upload (either local or Google Cloud Storage location)

c. Specify the format (CSV, JSON, AVRO)

d. Add a numeric value to indicate how many errors are acceptable for the load before the job should fail

e. Check this box to ignore entries in source that do not match data schema in destination.  These values will be ignored and not loaded

f. Select the delimiter of the source file (Comma, Tab, Pipe, Other).  If Other is selected enter the custom delimiter

g. Enter a number of rows to skip, such as headers, that are not to be loaded

h. Check if data contains new lines within quotes

i. Check to ignore missing optional columns

2. Continue by specifying the destination table configuration

a. Project where the table will be created

b. Dataset where the table will be created

c. Enter the name of the table

d. Select from drop down what should happen if the table exists (Fail, Append to it, Overwrite it)

e. Enter a number value to indicate how many days before table expires.

f.  Check to have the system auto-determine the schema based on source file.

g. If auto-detect schema is unchecked manually type the schema of the table to be created.  Format is JSON and must have name and type
Example:
  {
  "name": "myCol1",
  "type": "INTEGER"
  }

h. Select Partition Type which specifies partitioning with None being no partition and Day being partition by day.  See Partitioned Tables.

i.  If a partition type is specified select from drop down to specify partition field 

j. Check to require queries to include a partition filter

k. Add expiration, in days, for partition.  If no partition expiration then leave blank.

l. Enter up to four column names, separated by commas, to indicate clustering of data.

3. Processing Location identifies where the job will be processed.

 a. Location options.

4. Submit by clicking OK.

Create Empty Table

Creates empty table

To create an empty table fill out the destination information.

1. Project where the table will be created.
2. Dataset where the table will be created.
3. Enter the name of the table.
4. Enter a number value to indicate how many days before table expires.
5. Type the schema of the table to be created.  Format is JSON and must have name and type
   Example:
   {
   "name": "myCol1",
   "type": "INTEGER"
   }
6. Select Partition Type which specifies partitioning with None being no partition and Day being partition by day.  See Partitioned Tables.
7. If a partition type is specified select from drop down to specify partition field.
8. Check to require queries to include a partition filter.
9. Add expiration, in days, for partition.  If no partition expiration then leave blank.
10. Enter up to four column names, separated by commas, to indicate clusting of data.
11. Submit by clicking OK.

Create External Table

Creates an external table

To create an external table fill out the information.

1. Select from drop down specifying data source type.
2. Enter the source location of data.
3. Select from drop down to specify data source format type.
4. Enter a number value to indicate how many errors to encounter before failing.
5. Check this box to ignore entries in source that do not match data schema in destination.  These values will be ignored and not loaded.
6. Select the delimiter of the source file (Comma, Tab, Pipe, Other).  If Other is selected enter the custom delimiter.
7. Enter a number of rows to skip, such as headers, that are not to be loaded.
8. Check if data contains new lines within quotes.
9. Check to ignore missing optional columns.
10. Project where the external table will created.
11. Dataset where the external table will be created.
12. Enter the name of the table.
13. Select from drop down what should happen if the table exists (Fail, Append to it, Overwrite it).
14. Enter a number value to indicate how many days before table expires.
15. Check to have the system auto-determine the schema based on source file.
16. If auto-detect schema is unchecked manually type the schema of the table to be created.  Format is JSON and must have name and type.
    
    Example:
      {
      "name": "myCol1",
      "type": "INTEGER"
      }
17. Processing Location identifies where the job will be processed.
18. Submit by clicking OK

Create Model

Click to insert template into text area to create a model.

Share Link

Shares a direct link to dataset by copying URL to clipboard.

Refresh

Refreshes list of tables.

Table

Grouping & Sorting

Tables are listed under their respective dataset.  

Tables are sorted by case sensitive name.

Table list is paged.  Currently datasets with greater than 2,000 items are paged.
An additional page of 2,000 is loaded by clicking on Show More.

Table names ending in date formats such as YYYYMMDD are consolidated by year and month depending on their quantity.
Currently if only one table meets the format criteria it is not consolidated and will instead display the full name.

Tables are prepended with an icon:

Icon	Represents
	Table
	View
	Folder for grouped tables
	Model

Expanding tables

Columns are displayed by left clicking on a table to expand it. 

Folders also have a context menu (right-click)

Menu Item	Action
Expand	Expands selected node
Collapse	Closes selected node

Expanding datasets (left-click) displays list of tables associated to dataset.  A count of tables is displayed to right of dataset name.  Tables have table icon appended before name. Views (not pictured) have eye icon appended before name. Sorting selections are available.  To sort table list make selection from sort dropdown. Sorting is available once all table metadata is processed. Sort Option Name Size Rows Created Modified Location Expired Type Click on AZ icon to switch between ascending and descending order. Note: For Sort Option Name the top level name is sorted only. Children of consolidated tables are not sorted.

Table's Schema

Columns are displayed under the table name.  The name of the column is displayed Along with the data type which is color coded for visual representation Columns with record data types can be further expanded (indicated with + or – icon before table name) Insert the column to query text area by double clicking column

Table Info

Hover over a table to display popup information

Item	Description
Type	View or table
ID
Table Size
Number of Rows
Long-term Size	Size considered “long-term storage”
Creation Time
Last Modified
Data Location	displayed if available
Expiration	displayed if available
Estimated Size	Streaming tables only.  Buffer size
Estimated Rows	Streaming tables only.  Buffer row count
Oldest Entry Time	Streaming tables only.  Latest entry into buffer
Partition Type	Partitioned tables only

Context Menu

Table Context Menu is accessed by right-clicking on table name

Menu Item	Action
Preview Table	Switches to Preview Mode and displays table's data for user to explore
Query Table
Query Table > Insert table	Inserts table to query area at cursor
Query Table > Insert all fields	Inserts all table fields to query area at cursor
Query Table > Insert picked fields	Inserts selected fields from list to query area at cursor
Query Table > Insert “select from table”	Inserts select from selected table to query area at cursor
Query Table > Insert “select all fields from table”	Inserts all columns from table to query area at cursor
Query Table > Insert “select picked fields from table”	Inserts selected columns from table to query area at cursor
Copy Table	Copy selected table to new destination 1. To copy a table confirm source information is correct a. Project where table is b. Dataset where table is c. Table to be copied 2. Fill in information for Destination a. Add Project where table is to be copied to b. Add Dataset where table is to be copied to c. Name the table d. Select what happens if table already exists (Fail, Append to it, Overwrite it) e. Select what happens if table doesn’t exist (Create It, Fail) 3. Processing Location  a. Identifies where the job will be processed. Options include US, EU, and asia-northeast. 4. Click OK to submit request
Export Table	Export table to cloud storage 1. To export a table to cloud storage confirm source information is correct a. Project where table is b. Dataset where table is c. Table to be exported 2. Fill in information for Destination a. Select format to export to (CSV, JSON, AVRO) b. Select compression of file (None, GZIP) c. Select delimiter of file (Comma, Tab, Pipe, Other) d. Check to include headers in output e. Provide URI to Google Cloud Storage (URI and filename) 3. Processing Location  a. Identifies where the job will be processed. Options include US, EU, and asia-northeast. 4. Click OK to submit request
Modify Table Info	Modify table description, retention, and labels. A table’s description, retention, and labels can be modified Project where table is located Dataset where table is located Table to be modified Add, edit, or remove text to modify description Date time will be listed here if the table has an expiration date Add a numeric value to modify expiration or enter 0 to remove an existing expiration If expiration value is given then provide unit (Hours, Days) Existing table labels a. Remove existing labels by clicking on the remove icon b. Edit existing values by updating the text Associate table with labels by a. Add input for key b. Optionally associate key with a value c. Click add to include label on save
Delete Table	To delete a table verify the correct table is selected and then click Confirm
Show Schema	Display table schema, modify, or add new field Table schema is presented in JSON format Click on Modify to edit the schema Click Add Field to  add columns to schema Modifying Schema Modify mode allows editing the schema Click OK to submit changes Adding field A field can be appended to end of the root or a repeated column Enter the name of the field Click drop down to select data type Click drop down to assign NULLABLE or REPEATED Click submit to add field
Share Link	Click on Copy Link to copy URL to clipboard that links directly to table

View Context Menu

View Context Menu is accessed by right-clicking on view name

Menu Item	Action
Show View	Inserts view query to text area.  Not available in preview mode
Query View > Insert view	Inserts view to query area at cursor
Query View > Insert all fields	Inserts all fields to query area at cursor
Query View > Insert picked fields	Inserts selected fields from list to query area at cursor
Query View > Insert “select from view”	Inserts select from selected view to query area
Query View > Insert “select all fields from view”	Inserts all columns from view to query text area
Query View > Insert “select picked fields from view”	Inserts selected columns from view to query text area
Modify View Info	Modify view description, retention, and labels.  See Modify Table Info for details
Delete View
Show Schema
Share Link	Click on Copy Link to copy URL to clipboard that links directly to view

 

Model Context Menu

Model Context Menu is accessed by right-clicking on model name or let clicking on other options menu icon.

Menu Item	Action
Query Model > Retrieve Training Information	Inserts starter template to query text area that provides training information on model.
Query Model > Evaluate Metrics (Template)	Inserts starter template to query text area that evaluates model metrics.
Query Model > Evaluate Logistic Regression Model (Template)	Inserts starter template to query text area that evaluates regression-specific metrics.
Query Model > Produce Confusion Matrix (Template)	Inserts starter template to query text area that provides a confusion matrix.
Query Model > Predict Outcome (Template)	Inserts starter template to query text area that predicts outcomes.
Query Model > Inspect Input Features	Inserts starter template to query text area that provides information on input features used to trail model.
Query Model> Inspect Weights Info	Inserts starter template to query text area that provides weights used by model during prediction.
Train Model	Inserts starter template to query text area that trains model.
Copy Model
Modify Model Info
Delete Model
Show Schema
Share Link	Click on Copy Link to copy URL to clipboard that links directly to model.

Storage

Goliath displays Google Cloud Storage buckets and folders in the navigator.  To display Storage items select GCS Storage menu item from the Project Context Menu

Storage Context Menu

When Storage is displayed in the navigator the Project Context Menu will change to handle Storage related tasks.  Menu items that are not relevant to Storage are grayed out.

Switch Project

The project listings remain the same.  However when selecting a project to switch to the navigator will load Storage buckets associated to that project instead of it’s datasets.

Buckets

Bucket menu item is unique to Storage Context Menu.  Similar to functionality for Switch Projects Buckets the menu item lists individual Storage Buckets added to display by the user. 

• To switch to a Bucket click on the bucket from the Buckets list. 
• To add a bucket select Display bucket… from the menu and enter the bucket name.  Click on Remember checkbox to save the bucket and have it listed in the Buckets list. 
• To remove a saved bucket click on Remove bucket display… and select the bucket from the drop down menu.

Browse, Recent Jobs, Create Dataset, and Share Link are disabled and not available when displaying Storage in the navigator 

Refresh will refresh all buckets listed in the navigator 

Recently Run, Favorite, and GoBQ Examples remain the same both Dataset and Storage navigation 

GBQ Datasets option becomes available when in Storage navigation.  Clicking on GBQ Datasets will switch to dataset listing in the navigator.

Buckets

Storage navigator lists buckets associated to the displayed project.

User Filter by ID to filter buckets

Single left click on bucket to expand object

Hover over bucket to obtain popup with metadata

Right click on a bucket to open Bucket Context Menu

Bucket Objects

Buckets contain files.  The file path is broken into folders and files.  For example a file path with folder1/folder2/filename.txt will be displayed in the navigator under a folder named folder1 and a subfolder named folder2 that contains a file titled filename.txt

In the navigator folders have a folder icon and files have a file icon

Single left click to expand a folder

Right click on a file to open File Context Menu

Hover over file to obtain popup with file metadata