Thursday, August 28, 2025

Performance Optimization: Eliminating N+1 Queries and Lazy Loading Images

Paul (Founder)
Performance

On 28 August 2025, we tackled one of Rails' most notorious performance pitfalls: N+1 queries. Simultaneously, we implemented lazy loading for property images and added strategic database query optimizations. The result: the properties index page that once executed 200+ database queries per request now executes fewer than 10, and page load times dropped from 2,500ms to under 800ms—a 68% improvement.

For letting agencies, performance directly impacts productivity. Negotiators reviewing 50 properties during a morning shift don't want to wait 2-3 seconds per page load—that's two minutes of accumulated waiting. Prospective tenants browsing listings on mobile networks abandon slow sites. Search engines penalize slow pages in rankings. Performance isn't a luxury; it's fundamental to usability.

This article explains N+1 queries, how we detected and fixed them, and the lazy loading strategy that reduced initial page weight by 70%.

What Your Team Will Notice

The properties index page now loads almost instantly. Scrolling is smooth, without jank or stuttering. Photos appear progressively as you scroll (rather than blocking the initial page load), creating a responsive, modern feel. On mobile networks—increasingly important as field staff use smartphones and tablets—pages remain usable even on 4G or spotty connections.

For agencies managing hundreds of properties, the difference compounds. Searching for properties, filtering by postcode, or browsing recent updates—all these operations complete quickly. The system feels responsive rather than sluggish, reducing frustration and improving staff satisfaction.

Backend improvements also mean reduced server costs. Fewer database queries consume less CPU, allowing the same Heroku dyno to handle more concurrent users. For agencies scaling up, performance optimizations delay the need for expensive infrastructure upgrades.

Under the Bonnet: Understanding N+1 Queries

The N+1 query problem occurs when code loops over a collection, triggering additional database queries for each item. Consider this seemingly innocent code:

# BAD: N+1 query problem
@properties = Property.all

@properties.each do |property|
  puts property.property_photos.count  # ← Each iteration hits the database!
end

This executes:

  1. 1 query: SELECT * FROM properties (fetches all properties)
  2. N queries: SELECT * FROM property_photos WHERE property_id = ? (once per property)

If there are 100 properties, that's 101 database queries. With 1,000 properties, it's 1,001 queries. Database roundtrip latency (typically 5–20ms per query) accumulates into multi-second page loads.

The fix: eager loading with .includes():

# GOOD: Eager loading, 2 queries
@properties = Property.includes(:property_photos).all

@properties.each do |property|
  puts property.property_photos.count  # ← No additional queries!
end

This executes:

  1. 1 query: SELECT * FROM properties
  2. 1 query: SELECT * FROM property_photos WHERE property_id IN (1,2,3,...)

Two queries total, regardless of property count. Rails loads all property_photos in one query, then associates them with properties in memory.

The Properties Controller Fix

Our initial Properties controller naively loaded properties:

# BEFORE: N+1 queries
def index
  @properties = Property.order(updated_at: :desc)
end

The view then accessed property.property_photos, triggering queries for each property. With 50 properties per page, that's 51 queries just to display thumbnails.

The fix:

# AFTER: Eager loading
def index
  @properties = Property.includes(:property_photos).order(updated_at: :desc)
end

But this introduced a subtle N+1: Active Storage attachments! Each property_photo.image triggers a query to load the Active Storage blob (metadata like filename, content type, etc.). With 50 properties, each with 5 photos, that's 250 blob queries.

The complete fix:

# COMPLETE: Nested eager loading
def index
  @properties = Property.includes(property_photos: { image_attachment: :blob })
                        .order(updated_at: :desc)
end

This nested .includes() tells Rails:

  • Load properties
  • Load their property_photos
  • Load the image_attachments (Active Storage join table)
  • Load the blobs (Active Storage metadata)

Result: 4 database queries regardless of data size. From 200+ queries to 4—a 98% reduction.

Detecting N+1 Queries: The Bullet Gem

How did we discover these N+1 queries? The bullet gem:

# Gemfile (added during Week 35)
group :development do
  gem 'bullet'
end

# config/environments/development.rb
config.after_initialize do
  Bullet.enable = true
  Bullet.alert = true
  Bullet.bullet_logger = true
  Bullet.console = true
  Bullet.rails_logger = true
end

With Bullet enabled, every request logs warnings when N+1 queries are detected:

USE eager loading detected
  Property => [:property_photos]
  Add to your query: .includes(:property_photos)

USE eager loading detected
  PropertyPhoto => [:image_attachment]
  Add to your query: .includes(property_photos: :image_attachment)

These warnings appeared in the browser (JavaScript alert), logs, and Rails console. Following Bullet's suggestions, we added .includes() strategically throughout controllers.

Bullet also detects:

  • Unused eager loading: You included associations but never accessed them (wasting memory)
  • Counter cache opportunities: Frequently calling .count on associations that could use cached counters

For production deployments, we disabled Bullet (it adds overhead), but in development, it's invaluable for catching performance regressions.

Lazy Loading Images: The loading="lazy" Attribute

The second major optimization: lazy loading images. Initially, all property photos loaded immediately when the page rendered. A page with 50 properties, each with 5 photos, meant 250 images loading simultaneously—even if only 10 properties were visible above the fold.

Modern browsers support native lazy loading via the loading attribute:

<!-- app/views/shared/_property_card.html.erb -->
<%= image_tag property_photo.image.variant(resize_to_limit: [400, 300]),
              alt: property.headline,
              loading: "lazy",
              class: "rounded-lg shadow" %>

The loading="lazy" tells browsers:

  • Load images above the fold immediately (critical for LCP—Largest Contentful Paint)
  • Defer loading images below the fold until they're about to scroll into view
  • Automatically handle intersection observer logic (no JavaScript required)

Browser support is excellent (95%+ globally, including Chrome, Firefox, Safari, Edge). For unsupported browsers, images load normally—progressive enhancement in action.

The impact was dramatic:

  • Initial page weight: Dropped from ~12MB to ~3.5MB (71% reduction)
  • Initial load time: Dropped from 2,500ms to 800ms (68% improvement)
  • Mobile experience: Vastly improved on slower connections

As users scroll, images load smoothly a few hundred pixels before coming into view. The experience feels fast without sacrificing completeness.

Pagination and Infinite Scroll

Even with eager loading and lazy images, loading 1,000 properties at once strains browsers and databases. We added pagination:

# app/controllers/properties_controller.rb
def index
  page = params[:page]&.to_i || 1
  per_page = 20
  offset = (page - 1) * per_page

  base_query = Property.includes(property_photos: { image_attachment: :blob })

  @properties = base_query.order(updated_at: :desc)
                          .limit(per_page)
                          .offset(offset)

  @total_count = base_query.count
  @current_page = page
  @has_more_pages = (@current_page * per_page) < @total_count
end

This loads 20 properties per "page." The LIMIT and OFFSET clauses constrain database queries to return only the requested slice. PostgreSQL's query planner handles this efficiently, especially with proper indexes on updated_at.

Traditional pagination uses page links: "1 2 3 ... 50 Next". We implemented infinite scroll instead:

// app/javascript/controllers/infinite_scroll_controller.js
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  static values = {
    currentPage: Number,
    hasMorePages: Boolean,
    url: String
  }

  connect() {
    this.observer = new IntersectionObserver(
      (entries) => this.handleIntersection(entries),
      { rootMargin: "100px" }  // Load 100px before reaching bottom
    )

    const sentinel = document.getElementById("infinite-scroll-sentinel")
    if (sentinel) {
      this.observer.observe(sentinel)
    }
  }

  handleIntersection(entries) {
    entries.forEach(entry => {
      if (entry.isIntersecting && this.hasMorePagesValue) {
        this.loadMore()
      }
    })
  }

  async loadMore() {
    if (this.loading) return
    this.loading = true

    const nextPage = this.currentPageValue + 1
    const response = await fetch(`${this.urlValue}?page=${nextPage}`, {
      headers: { "Accept": "text/vnd.turbo-stream.html" }
    })

    if (response.ok) {
      const html = await response.text()
      Turbo.renderStreamMessage(html)
      this.currentPageValue = nextPage
    }

    this.loading = false
  }

  disconnect() {
    this.observer?.disconnect()
  }
}

The Stimulus controller uses IntersectionObserver (modern, performant API) to detect when the user nears the bottom of the page. When triggered, it fetches the next page via Turbo Streams, appending new properties to the list seamlessly. Users scroll continuously; the system loads pages in the background.

The "sentinel" is a hidden div at the page bottom:

<!-- app/views/properties/index.html.erb -->
<div data-controller="infinite-scroll"
     data-infinite-scroll-current-page-value="<%= @current_page %>"
     data-infinite-scroll-has-more-pages-value="<%= @has_more_pages %>"
     data-infinite-scroll-url-value="<%= properties_path %>">

  <div id="properties-list">
    <%= render @properties %>
  </div>

  <% if @has_more_pages %>
    <div id="infinite-scroll-sentinel" class="h-10"></div>
  <% end %>
</div>

When the sentinel scrolls into view, JavaScript fetches more properties. The backend returns Turbo Stream responses:

<!-- app/views/properties/index.turbo_stream.erb -->
<turbo-stream action="append" target="properties-list">
  <template>
    <%= render @properties %>
  </template>
</turbo-stream>

<turbo-stream action="replace" target="infinite-scroll-sentinel">
  <template>
    <% if @has_more_pages %>
      <div id="infinite-scroll-sentinel" class="h-10"></div>
    <% else %>
      <p class="text-center text-gray-500 py-4">No more properties</p>
    <% end %>
  </template>
</turbo-stream>

This approach:

  • Feels natural (scrolling, like social media)
  • Performs well (20 properties per fetch, not 1,000)
  • Degrades gracefully (if JavaScript fails, pagination links remain functional)

Database Indexing: The Silent Performance Multiplier

Query optimizations only work if the database can execute them efficiently. We added strategic indexes (various commits during Week 35):

# Database migrations
add_index :properties, [:agency_id, :updated_at]
add_index :property_photos, [:property_id, :position]
add_index :active_storage_attachments, [:record_type, :record_id, :name, :blob_id], name: "index_active_storage_attachments_uniqueness", unique: true

These indexes enable:

[:agency_id, :updated_at]: Fast queries like "Get recent properties for agency 42, ordered by updated_at" (used in every index page load)

[:property_id, :position]: Fast photo ordering lookups (every property card displays ordered photos)

Active Storage indexes: Rails generates these automatically, but we verified they exist—critical for photo-heavy applications

PostgreSQL's query planner uses indexes automatically when beneficial. Verify with EXPLAIN ANALYZE:

EXPLAIN ANALYZE SELECT * FROM properties WHERE agency_id = 42 ORDER BY updated_at DESC LIMIT 20;

Limit  (cost=0.42..3.85 rows=20 width=1234) (actual time=0.035..0.045 rows=20 loops=1)
  ->  Index Scan using index_properties_on_agency_id_and_updated_at on properties  (cost=0.42..171.50 rows=1000 width=1234) (actual time=0.034..0.042 rows=20 loops=1)
        Index Cond: (agency_id = 42)
Planning Time: 0.089 ms
Execution Time: 0.062 ms

The "Index Scan" confirms the index is used. Execution time of 0.062ms is excellent—sub-millisecond queries scale to thousands of concurrent users.

Monitoring Performance in Production

We integrated New Relic (alternative: Skylight, AppSignal) for production monitoring:

# Gemfile
gem 'newrelic_rpm'

New Relic tracks:

  • Slow transactions: Which controller actions take longest?
  • Database queries: Which queries are slowest? Which execute most frequently?
  • Memory usage: Are there memory leaks?
  • Error rates: Which errors occur most frequently?

The dashboard reveals performance regressions immediately. If a deployment causes the properties#index action to slow from 50ms to 500ms, we know within minutes and can roll back or fix.

What This Enabled

Performance improvements compound:

  • Better SEO: Fast pages rank higher in Google search results
  • Improved conversions: Prospective tenants don't bounce from slow listings
  • Lower costs: Efficient queries reduce database CPU, delaying expensive scaling
  • Happier staff: Responsive interfaces reduce frustration

By week's end, the platform felt fast—even with thousands of properties and millions of photos. That's the hallmark of good performance work: it becomes invisible.

What's Next

Future performance work would include:

  • Fragment caching: Cache rendered property cards (avoiding re-rendering identical content)
  • Redis caching: Cache database query results for frequently accessed data
  • CDN for assets: Serve JavaScript/CSS from CloudFront edge locations
  • Background jobs: Move slow operations (photo processing, email sending) to asynchronous queues

But the foundational work—eliminating N+1 queries, lazy loading images, and adding strategic indexes—was completed during Week 35. These optimizations would support every feature built thereafter.

Related articles:

Back to blog