Experimental Lab Feature: Importing Files From Other LookML Projects

Project Import

There exists an experimental labs feature, Project Import, that lets you import files from other LookML projects into your current project. This lets you use model files, view files, and other files in more than one project.

There are several use cases for this, including:

  • Building on top of an installed Looker Block without having to go and make direct changes to it. That way if Looker makes changes to the Block, you can pull the changes in really easily because all of the stuff you’ve added to it is kept nice and neat in a separate repo.

  • Maintain a base project that is just auto-generated based on the database schema. Then you put all your custom dimensions, measures, etc. in a separate project that just includes all the stuff from the auto-generated project. That way you can regularly regenerate the base model/project as the database schema changes without overwriting all your custom stuff. (This is similar to the Block use case described above.)

  • You can encapsulate common/shared stuff in a single project and then include it in multiple other projects. Maybe you’ve got a certain table that is common to multiple databases. You could put the view for it in one project and maintain it in that one place, but have multiple other projects make use of it.

Enabling the Feature

To enable this feature, navigate to the Labs page in the Admin section of Looker, and set the Project Import toggle to ON:

Creating a Project Manifest File

All projects which import files from other projects must have a project manifest file. We will use it to detail which other projects are known and usable by the current project.

In the Add Files section of the sidebar there is a new entry, Create Project Manifest. This option only appears if you don’t yet have a project manifest file; there can only be one per project.

This will add a Project section to the sidebar and a new file called manifest.lkml.

Importing a Project

Projects are imported by specifying them in the manifest. There are two methods of doing so: locally and remotely.

Local Import

In a local import, the imported project must be present on the same Looker instance. Versioning is dynamic, meaning that changes in the imported project will immediately impact the project that imports it. This is a good option when the same team owns both projects and can validate changes in both places before pushing to production.

To import locally, you must specify the current project as well as the project to import.

# This project
project_name: "my_project"

# The project to import
local_dependency: {
  project: "my_other_project"
}

Remote Import

In a remote import, the imported project does not need to be on the same instance. Rather, the project is imported through its remote Git repository. Versioning is static, meaning that changes in the imported project will not automatically be reflected in the project that imports it; the desired version of the imported project is explicitly indicated. This is a good option for using Looker Blocks.

Important!

As of 5.18, remote imports can only be performed on public repositories.

To import a remote repository, you must specify its URL and the commit SHA representing the version of the repository you want. Here’s an example:

remote_dependency: ga_360_block {
  url: "git://github.com/llooker/google_ga360.git"
  ref: "4be130a28f3776c2bf67a9acc637e65c11231bcc"
}

Some notes:

  • To get the URL for a public repository, just copy its HTTPS URL and replace https:// with git://.
  • Commit SHAs can be found anywhere you can browse the commit history. The Git service provider often makes the SHA clearly visible and copyable. For example, I found the SHA above at https://github.com/llooker/google_ga360/commits/master.
  • The name (in this case, “ga_360_block”) can be anything you like. It will be used when you include its files in the following step.

Including Files from an Imported Project

Once a project is imported via the manifest, its files are available to be included in models and views via include.

To include them, you must specify which imported project the files are from. Do this by adding /<imported_project_name>/ at the beginning of the include string. For example:

include: "/my_other_project/imported_view.view.lkml"
include: "/ga_360_block/ga_block.view.lkml"

As always, you can include wildcards. So the following example:

include: "/IMPORTED_PROJECT/*.view.lkml"

would include all view files in the IMPORTED_PROJECT project.

Q&A

Can I Include a Model File?

Not directly. Models for a project can only come from that project.

If you want to reuse or extend explores across projects, in the imported project you could put your explores in a dedicated file that does not end with “.model.lkml”, like “.explore.lkml” (not a special file type by any means but useful for organization). You could then include that file as described above.

What If I Include Files that Are Also Including Files?

In this case, all the includes are resolved before being passed on to the next project that includes it.

For example, if in your current project, you import a file A from another project proj_A and the imported file contains an include parameter that includes file B from proj_B, then file B will be included into file A before file A is imported into the current project.

2 Likes

Does Looker have plans to take this out of Labs and make it a native feature?

1 Like

Hi @sschanche There isn’t a current ETA on when this will move out of Labs. Any updates for that change should be posted in outgoing, future release notes. All release notes can be found here.

In 5.18 we released git (remote) project import. I’ve updated the original post here to describe the new functionality.

Hi, is there an example of using the explore.lkml method to import explores from another model in the way described above?

Hey Zander,

What we mean by this, is that you can define a “base” or “common” model file with explore definitions. Then rename it to anything that doesn’t end in model.lkml, the reasoning for this is that the file suffix tells Looker how to interpret the file. So if you rename it to end in other.lkml, like base.other.lkml, Looker will interpret this file as an “Other” file as opposed to a “Model” like here:

57%20PM

I have a simple example of a parent explore, and a child explore extending it.

57%20PM

Now to reuse or extend the explores defined in this base.other.lkml file, you can simply add it to the include parameter.

01%20PM

If you are doing this across projects, be sure to add in the necessary project name to fully scope files.

Have a great day!

1 Like

What about adding custom attributes to an auto-generated dimension (such as for example, hidden). How is that done?

The first thing that comes to my mind is to use extends to redefine those fields. That does lose you some of the automated approach of the auto-generation though.