Introducing lkml, a speedy LookML parser in pure Python

I’m excited to debut lkml, a pure Python parser for LookML.

You can run lkml from the command line (it will output a JSON string) or import it as a Python package (it will output a nested dictionary).

@fabio at Looker already built a fantastic, open-source parser in Node, but I find that the Node dependency sometimes makes it difficult to adopt for Python-based workflows that data people are more familiar with.

I decided to embrace the challenge of hand-writing a parser in Python without any external dependencies or libraries.

I didn’t have access to the actual grammar used for LookML, so I reverse engineered it myself. Don’t worry, I’ve tested lkml on over 160K lines of public LookML I downloaded through the GitHub API and it works like a charm!

lkml is fast too. Excluding file I/O, it parses a typical file in a handful of milliseconds. Based on my tests, it’s at parity with the Node parser.

Lastly, lkml has a full unit test suite with CI.

Installation

lkml is up on pip, so the following command will install it

pip install lkml

Please also check out the project on GitHub for a more detailed description and instructions. Contributions and issues are welcome! If you build anything interesting with this parser, please send me a link!

9 Likes

Huge, Josh! That’s commitment.

Can’t wait to see what gets built out now that the Python-speaking masses (like me :snake: ) have easy access to this.

1 Like

Hi Josh. That’s a neat work. How are you using it? I remember some of my users were interested in doing somewhat reverse: generate/update LookML if the table structure changes.

Thanks! I released it so that others could use it to build or improve linters, syntax highlighting, CI tools, and support more Python-based “built on Looker” projects. There are a couple other open-source projects that I know of that are considering implementing it behind the scenes.

There’s no functionality to do the reverse at the moment (“dump” instead of “load”), but the project is definitely open for contributions!

@joshtemple This is great. I’ve tested this with lookml-tools and I will be swapping out the node LookML parser with this Python lkml parser in the next release – hopefully in the next week
Carl

2 Likes

lookml-tools 2.0.0 is released and it now uses lkml as its LookML parser.

1 Like