Rails 7.1 makes ActiveRecord query cache an LRU

Application performance is one of the top most priority for any developer. If you are using Ruby on Rails, caching is one of the best tools Rails provides to improve performance.

Rails ActiveRecord query cache is a feature that caches the result set returned by each query. If the Rails server reencounters the same query for that request, it will use the cached result instead of querying it on the database again. This is beneficial for reducing the number of queries on the database and boosting performance.

Note: ActiveRecord QueryCache is enabled by default.

Can query caching slow down your application?

Yes, it can. Caching a significant amount of data occupies a substantial portion of your system's memory. As a result, memory exhaustion problems may arise, potentially causing your server to crash.

To resolve the memory exhaustion issue, Rails decided to modify the implementation of the ActiveRecord Query Cache and adopt the Least Recently Used (LRU) caching strategy. LRU is a common caching strategy that evicts elements from the cache to make room for new elements when the cache is full. It removes the least recently used items first.

Before Rails 7.1

Let's assume a Rails application with a User model, and it has millions of records. A user has one role and has many addresses, as shown below:

class User < ApplicationRecord
  has_one: role
  has_many: addresses
end

The application runs a background job, where all the user data gets fetched and analyzed to generate stats and reports for admins.

class Admin::AnalyzeUserDataJob < ApplicationJob
  def perform
    User.active.each do |user|
      addresses = user.addresses
      role = user.role
      # ...
      # code to generate some stats and report
      # N number of queries
    end
  end
end

When the background job is running, the query results are cached. Due to caching, all the user objects will live in the memory for the entire job duration.

For millions of records, the memory consumption will be too high, which can kill the background job process. If the application is using docker, the docker instance can die.

Query Caching never had a limit on how many query results needed to be cached. With such heavy queries, bloating memory issues will be frequent. Or if a heavy background job like the above has N queries, all the N queries get cached in the memory.

A workaround for this, before Rails 7.1, is to disable query caching, as shown in the code below:

class Admin::AnalyzeUserDataJob < ApplicationJob
  around_perform do |_job, block|
    ActiveRecord::Base.uncached do
      block.call
    end
  end

  def perform
    User.active.each do |user|
      addresses = user.addresses
      role = user.role
      # ...
      # code to generate some stats and report
      # N number of queries
    end
  end
end

The around_perform callback can also be extracted to ApplicationJob, if you want to disable query caching for all the background jobs.

In Rails 7.1

In Rails 7.1, significant changes are made to the ActiveRecord query cache implementation, introducing an LRU (Least Recently Used) caching strategy. The query cache continues storing query results in memory, but when a certain threshold gets reached, the system removes the least recently accessed queries to prevent excessive memory usage.

Before Rails 7.1, there was no limit on the query cache size. However, in Rails 7.1, a default limit of 100 queries has been imposed. If desired, you can customize the query cache size by setting the query_cache key in the database.yml file as shown below:

development:
  adapter: postgresql
  query_cache: 500
  ...

Returning to the previous example involving a heavy background job with N queries. With Rails 7.1, not all N queries get cached in memory. Instead, only the 100 most recently used queries are cached. It ensures that sufficient memory space is available and prevents memory bloating.

If you wish to disable query caching altogether, you can set query_cache to false in the database.yml file:

development:
  adapter: postgresql
  query_cache: false
  ...

To know more about this change, please look at this PR.