Looker over presto requires additional queries for totals

on-hold
performance
normal_priority
reply

(Daniel Rubio) #1

Hi all,

We are testing looker over Presto and we found that total (distinct) calculations require additional (and sequential) queries. This is an issue because it harms the response time. There are some SQL model workarounds we could create but before that, I would like to know if:

  1. Is this only happening with presto?
  2. There is any enhancement planned?
  3. Are you guys using any workaround for this?
    Thanks in advance!

Dani


(Izzy) #2

This is the way that totals work across all database dialects— We run the normal query, and fire off a totally separate one to get the totals. You can see this reflected in the SQL tab of the results where there’s an entirely new SELECT for the totals.

I’m not sure if there’s plans for enhancing this, but I’ll check in on it. I’m curious what kind of response time discrepancies you’re seeing between queries with/without totals!


(Daniel Rubio) #3

Thanks Izzy!
Time to result basically doubles. If you want I could provide some evidence of this. I would be happy to understand if there is any plan on enhancing this :slight_smile:

Thanks in advance,

Dani


(Ronak Bhatt) #4

Very poor implementation if this is the case


(Izzy) #5

My general guess would be that since the overall totals query would be less performance-intensive than the original more complex query, it wouldn’t be a full doubling, but I see what you mean. I’ll check into the internal implementation and see if we kick them off at the same time on the database, and if not, why. I’ll update here!


(Daniel Rubio) #6

Hi @izzy,

Any updates on this? Thanks in advance!

Dani


(Izzy) #7

Ooh, this is interesting actually. I’d been testing on bigquery and was seeing almost precisely the same query time, but I took the time to get a presto test set up, and you’re right— Pretty much double.

I’m going to take that and go see what I can dig up about how we talk to Presto. On your actual database, do you see the queries kicked off at the same time and running concurrently, or one after the other?


(Daniel Rubio) #8

One after the another. However I would like looker to run one single query rather than 2 (1 for the columns and another for the column totals).

Many thanks @izzy!