Per-User Query Throttle (3.38+)


(Abby West) #1

What the Query Throttler is & Why We’re Implementing It

As of Looker 3.38, Looker enforces a per-user query throttle. In 3.38 and 3.40 the value is a default of 5 queries per person. In 3.42+ the value is a default of 15 queries per person. This means that each user sends a maximum of 15 queries to the database at any given time. If a user has requested more than 15 queries (by opening a large dashboard or many simultaneous Explore windows), additional queries queue up behind the first 15, with the next query executing as soon as soon as the query ahead of it completes.

This goal of this feature is to prevent any individual user from overwhelming (and thereby locking up) the database, impacting query performance for others who are using Looker. In particular, customers who were experiencing PoolTimeout errors should find that this prevents their system from getting into that state.

Adjusting the Defaults for your Company

We estimate that the defaults will work well for the majority of Looker customers, especially those who have a medium to large number of users. That said, a default of 15 queries per person might not feel right for your company’s usage. If, for example, you have a small number of users who might want to execute lots of queries at once, you could consider raising the per-user limit.

Raising the per-query limit requires executing commands in the start-up script. If you’re self-hosted, you can modify these settings yourself. If Looker hosts your instance, please contact support@looker.com to update your settings.

The following settings show the defaults for the per-user, and also give an indication of how to update them. Scroll to the right to see the full information on each line.

opt :per_user_query_limit,   'Limits number of concurrent queries per user',      :short => :none, :type => :int, :default => 5
opt :per_user_query_timeout, 'Length of per-user timeout to wait for connection', :short => :none, :type => :int, :default => 600
opt :scheduler_query_limit,   'Limits number of concurrent scheduled queries',      :short => :none, :type => :int, :default => 10
opt :scheduler_query_timeout, 'Length of scheduler timeout to wait for connection', :short => :none, :type => :int, :default => 1200

Note: This is just one place where timeouts could be occurring on your Looker. If you have a proxy or loadbalancer in front of your Looker, there may be a timeout set there as well. Read more about Queueing and Timeouts in Looker.


Looker 3.38 Release Notes
(Ken Yeoh) #2

So, where do we put these parameters?
I’m looking here: http://www.looker.com/docs/setup-and-management/on-prem-install/looker-startup-options
But it’s not immediately clear to me where to put these, or what format they need to be in


#3

@kyeohhubspot we talked about this on chat, but for everyone else’s benefit, you can add this to the LOOKERARGS portion of the startup script. This would look something like:

LOOKERARGS="--per-user-query-limit=5"

(Ken Yeoh) #4

Hi @lindsey
I got this error when starting up
Looker Error: unknown argument '--per_user_query_limit'.
We are on version 3.40


#5

@kyeohhubspot Ah I made a typo above (since fixed). You will want to replace the _ with -.

You can run java -jar looker.jar --help to see all the options.


(Todd) #6

thanks for detailing this out, I think this would be a good candidate for a runtime option via the admin interface.

i’m bummed i can’t easily test this w/o scaling down my cluster and restarting a node and scaling back up.


(peter.whitehead) #7

Hi @tfoley,

Thank you for your feedback! I can see your use case for this. I will reach out to our product team and let them know you would like to see this option on the admin page.