Created: 2011-07-18 06:39
Updated: 2018-07-08 01:44


GAE Fast/Slow Queue is a useful little decorator/utility designed to help minimize App Engine taskqueues' degradation of user-facing request performance.

It optimizes taskqueue behavior by helping you separate fast-running tasks from slow-running and splitting the two into separate queues.

gae_fast_slow_queue is designed to replace common App Engine taskqueue/mapper situations like this:

def daily_activity_summary(user):
    if user.has_recent_activity():
        # Some slow task

...with this:

    lambda user: user.has_recent_activity()
def daily_activity_summary(user):
    # Some slow task


  1. Drop the following into your queue.yaml (you can change the default rates, the names are all that matters):
- name: fast-background-queue
  rate: 10/s

- name: slow-background-queue
  rate: 5/s
  1. Add fast_slow_queue to your project.

  2. Use the fast_slow_queue.handler decorator as shown in the example above for any common task queue entry point such as a mapreduce task, and always use fast_slow_queue.QUEUE_NAME when specifying the queue to be used:

    handler_spec = "daily_activity_summary",
    reader_spec = "mapreduce.input_readers.DatastoreInputReader",
    reader_parameters = {"entity_kind": "models.UserData"},
    queue_name = fast_slow_queue.QUEUE_NAME,

See for more about how this helps App Engine scale and why your users care.

This code is MIT licensed.

Cookies help us deliver our services. By using our services, you agree to our use of cookies Learn more