Ruby 2.1: Out-of-Band GC

Ruby 2.1’s GC is better than ever, but ruby still uses a stop-the-world GC implementation. This means collections triggered during request processing will add latency to your response time. One way to mitigate this is by running GC in-between requests, i.e. “Out-of-Band”.

OOBGC is a popular technique, first introduced by Unicorn and later integrated into Passenger. Traditionally, these out-of-band collectors force a GC every N requests. While this works well, it requires careful tuning and can add CPU pressure if unnecessary collections occur too often.

In kiji (twitter’s REE fork), @evanweaver introduced GC.preemptive_start as an alternative to the “every N requests” model. This new method could make more intelligent decisions about OOBGC based on the size of the heap and the number of free slots. We’ve long used a similar trick in our 1.9.3 fork to optimize OOBGC on github.com.

When we upgraded to a patched 2.1.0 in production earlier this month, I translated these techniques into a new OOBGC for RGenGC. Powered by 2.1’s new tracepoint GC hooks, it understands both lazy vs immediate sweeping and major vs minor GC in order to make the best descision about when a collection is required.

Using the new OOBGC is simple:

require 'gctools/oobgc'
GC::OOB.run() # run this after every request body is flushed

or if you’re using Unicorn:

# in config.ru
require 'gctools/oobgc'
if defined?(Unicorn::HttpRequest)
  use GC::OOB::UnicornMiddleware
end

OOBGC results

With ruby 2.1, our average OOBGC pause time (`oobgc.mean`) went from 125ms to 50ms thanks to RGenGC. The number of out-of-band collections (`oobgc.count`) also went down, since the new OOBGC only runs when necessary.

The overall result is much less CPU time (`oobgc.sum`) spent doing GC work between requests.

GC during requests

After our 2.1 upgrade, we’re performing GC during requests 2-3x more often than before (`gc.time.count`). However since all major collections can happen preemptively, only minor GCs happen during requests making the average GC pause only 25ms (`gc.time.mean`).

The overall result is reduced in-request GC overhead (`gc.time.sum`), even though GC happens more often.

Note: Even with the best OOBGC, collections during requests are inevitable (especially on large requests with lots of allocations). The GC’s job during these requests is to control memory growth, so I do not recommend disabling ruby’s GC during requests.