New LookML — Deep dive into the new syntax and the new IDE


(Nate Pickens) #1

Introduction

New LookML introduces a refreshed syntax along with a full-featured IDE. We’ll take a look at the syntax first, exploring what is different from YAML LookML and why. Then we’ll explore all of the new features in the improved IDE.

Syntax

New LookML, while looking very similar to YAML, is entirely its own language. Its simple and consistent syntax takes the good from YAML but leaves out the things that often trip up LookML developers. Let’s take a look at how it differs from YAML.

Curly braces (instead of indentation) for hierarchy

New LookML solves the whitespace-significant indentation confusion by doing away with it entirely. With new LookML, hierarchy is clearly designated with a simple set of curly braces. So, for example, a measure definition looks like this:

measure: average_sale_price {
  type: average
  sql: ${sale_price}
}

Two semicolons (instead of indentation) to end chunks of SQL and HTML

Rather than reducing indentation to signify the end of a chunk of SQL or HTML, new LookML designates the end with two semicolons:

dimension: total_amount_of_order_usd {
  type: number
  sql:
    (SELECT SUM(order_items.sale_price)
    FROM order_items
    WHERE order_items.order_id = orders.id) ;;
}

dimension: ...

So the obvious question is: why two semicolons? Why not just use curly braces like everywhere else? Curly braces are going to appear in SQL due to Looker’s ${...} field substitution syntax. So using curly braces would make it very easy to accidentally mismatch them and either end a chunk of SQL too early or never at all.

OK but really, why two semicolons? Put simply: because two semicolons right next to each other are extremely unlikely to appear in a SQL query or HTML snippet, making it unlikely that the chunk will accidentally be terminated. (If two semicolons do happen to be used in a chunk of SQL, they can be escaped by placing a backslash between the two: ;\;).

No more dashes

There’s no need to put a dash at the beginning of the line when defining explores, views, dimensions and measures, etc. Because we couldn’t find any reason to require one! :slight_smile:

Strings are always quoted

There’s no more guessing about whether you should quote or not quote a string. If a parameter’s value is a string, it is enclosed in double quotes.

measure: average_sale_price {
  label: "Avg Sale Price"
  ...
}

Lists are enclosed in square brackets and items separated with commas

Items can optionally be placed on separate lines, but the syntax remains the same (as opposed to YAML switching to a dash at the beginning of each line rather than commas). Trailing commas (a comma following the last item) are also allowed—new LookML won’t be mad at you!

drill_fields: [
    user.id,
    user.email,
]

No more joins:, fields:, and sets: groupings

The items within these groupings are simply defined directly within the parent. So, for example, when defining dimensions and measures, they are placed directly within a view—not underneath a fields: section defined within that view:

view: orders {
  dimension: id {
    type: number
    primary_key: true
    sql: ${TABLE}.id
  }

  ...
}

A more helpful IDE

Beyond the more consistent and understandable syntax, the true magic of new LookML shines in the form of the more full-featured IDE that comes along with it. There are five major enhancements.

Auto suggest

As you type, the IDE automatically suggests both parameters (e.g. explore or dimension) and values for those parameters (when possible values are known, such as with type). These suggestions are context-sensitive, so for example when suggesting for a dimension’s type value, the suggestions will only include valid dimension types (not measure types).

Additionally, field suggestions appear within ${...} inside a SQL block. (Note that the fields suggested are limited to the current view, since within the context of a specific view it is not known what other views may ultimately get joined in to the explore where it is used.)

Auto suggest automatically appears wherever it can be shown. To close it, press the escape key on your keyboard. To pop it up at any point, simply press Ctrl+Space.

On-the-fly error checking

Many errors are now surfaced on the fly, much like spell checking. The problematic area is underlined in red and a red X appears on the gutter, which when hovered over reveals a description of the problem.

Note that while this should greatly reduce how often the LookML Validator must be used, it is still required to perform a full model validation. Some errors, such as an invalid field reference due to a missing join, require a holistic look at the model and therefore are only surfaced when the LookML Validator is run.

Context-sensitive help pane

The context-sensitive help pane that appears on the righthand side of the IDE dynamically updates to reveal documentation based on where your cursor is placed within the file. It provides a description of the LookML parameter, along with all possible children elements or list of allowed values. Each item within that list is also a link to the full Looker.com documentation page for that specific parameter.

Automatic formatting

One of the benefits of moving to curly braces instead of whitespace-significant indentation is that incorrect indentation can be easily corrected. An entire file’s formatting can be fixed with the press of the “Format File” button in the gear drop-down in the upper-right corner of the IDE.

Folding

A small arrow appears in the gutter next to the start of each “chunk” of LookML. Pressing these allows that section to be folded or unfolded. The gear drop-down in the upper righthand corner of the IDE also provides options to “fold” and “unfold” all sections within the current file, so you can easily focus in on just the section you care about and get the other stuff out of the way.

Related Info

For related info, check out these articles:

More to come…

Stay tuned for more IDE enhancements that make developing in LookML even faster and easier!


New LookML — How to convert a "YAML" LookML project
New LookML — Why?
The New LookML IDE Shows New and Pre-Existing Parameters
Looker 4.0 Release Notes
Legacy Feature Notice: Old Style YAML LookML (Looker 4.4+)
New LookML — Why?
(Thomas Vo) #2

Super informative post and great update. The new IDE will make it so much easier for our staff to learn how to code!


(Rick Cobb) #3

Are there open-source parsers / generators for this format? We have a number of tools we’re still supporting for handling our two big open issues (separate Discourse articles discuss those); those tools work by using generic YAML parsers to read LookML files and output new versions of those files (e.g., with different model declarations, or limits on the number of elements in specific dashboards for specific customer audiences).

We’re trying to live with old LookML as long as possible, but the IDE appears to rename files willy-nilly; our PRs often need a ton of cleanup to avoid problems with our legacy tools.


(allegra) #4

@cobbr2 Hi Rick! It sounds like what you are hoping for is a generator/converter you can use internally rather than just relying on Looker to do this for you-- this is not possible yet, but something that I can mention to the product team! However, it sounds like you may be experiencing some issues with the current IDE-- I will up offline to get a better sense of what’s going on!


(fabio) #5

Hi Rick! Not sure if this is still relevant, but I have just shared my LookML parser here: Parser for new LookML?