User Guide

Getting Started

Help Center

Documentation

Community

Training

DynamoDb integration

etl
(Shavkat Aynurin) #1

Hi there,

Could you suggest the best and easiest way to load one small dataset from DynamoDb into Looker?

The dataset represented with one table of time-series data, up to 20 fields, not more than 100k rows in total. The data needs to be updated one or two times a day. And this doesn’t need to be a production-ready integration, we just need a quick and dirty way to get the data and play with it. If we decide that this data looks good in looker we will consider implementing a more robust integration.

(George Fraser) #2

If you haven’t already, you’re going to need to set up a data warehouse to sit between DynamoDB and Looker. If you don’t have a data warehouse, probably the easiest/cheapest one to just set up and play around with one thing is going to be BigQuery.

To get the data into the warehouse, I would recommend my own company :yum: Fivetran. We have a DynamoDB connector, and a two-week free trial. Since you’re basically just playing around right now, I’d recommend you just click the “Free Trial” link on the home page, hook up DynamoDB and BigQuery, and the data will load automatically. The trial is two weeks but the data will still be in BigQuery forever, so you can effectively get a one-time load for free. Someone on the sales team will reach out, just let them know you’re experimenting with DynamoDB, it’s fine…and our sales people really do try to be helpful, its a good idea to jump on a call with them and discuss your use case, even if you set everything up on your own.

(Shavkat Aynurin) #3

Thanks for the suggestion, George, I’ll have a look, and we were actually considering Fivetran :slight_smile:

Interesting though if there is a shorter way than proxying with BigQuery :).

#4

Hey @Shavkat_Aynurin

As @George_Fraser mentions, the data from DynamoDB will need to be piped into a database (BigQuery or otherwise) for Looker to query against. Looker does not store any data itself as it is simply an application that sits on top of a database and generates queries to run against the database based on the LookML defined.