undeniablyrob

Spoiler Alert: I found build_stubbed not viable as a replacement of create​in our test suite.

To remove the dependency of the database in tests, I would recommend RSpec’s instance_double instead.

Read on to find out why.

I was excited to find out that FactoryGirl has methods to create in-memory objects instead of hitting the database. Running a lot of queries for a test can be painfully slow, particularly when you’re object graph is large.

I’m working on a codebase that has strong test coverage. We made the decision some time ago to leverage FactoryGirl.create in building our object graphs. At the time, this was the right call. Our codebase was much smaller and had fewer engineers involved. Now, as we scale, we’re searching for the next step. To give some context, we have around 12k tests, and an average of 125 queries per test. Our test suite takes around 65 minutes to run serially (16 minutes in parallel with 8 threads). We’re moving in the direction of the 10-minute build, and we’ve got some work to do.

Enter FactoryGirl.build. Now I’m intrigued. I quickly find out that .build stubs out the top-level object in-memory, but still creates all relational objects in the database (i.e., all associations are built with .create). That’s not going to make a dent in our test suite.

Enter FactoryGirl.build_stubbed.

build_stubbed is the younger, more hip sibling to build; it instantiates and assigns attributes just like build, but that’s where the similarities end. It makes objects look like they’ve been persisted, creates associations with the build_stubbed strategy

-Josh Clayton, Thoughtbot
Use Factory Girl’s build_stubbed for a Faster Test Suite

My fingers dive quickly into a mass find+replace of FactoryGirl.create to FactoryGirl.build_stubbed. Can I get the test suite to run without any queries? And if so, what’s the impact on performance and memory usage?

TIL: the equivalent of the after(:create) hook is after(:build_stubbed) after(:stub)(thanks Dmitriy Vasin).

After updating all of those in my factories and tests, I run a handful of tests. All green. Now a larger sample. Mostly green, with a handful of failures. The failures all have something in common: those tests all have association proxies, e.g.,

FactoryGirl.define do
  factory :blog do
    ...
    
    trait :with_posts do |blog, evaluator|
      FactoryGirl.build_stubbed(:post, blog: blog)
    end
  end
end

And in the test:

# some_blog_spec.rb
let(:blog) { FactoryGirl.build_stubbed(:blog, :with_posts) }

it "is some trite example"
  expect(blog.posts.count).to equal(1)
end

# test output:
NoMethodError: undefined method `count' for nil:NilClass

Ok. So that’s strange. And the runtime error isn’t especially helpful in debugging. Let me try something a little more direct in the factory:

trait :with_posts do |blog, evaluator|
  FactoryGirl.build_stubbed(:post, blog: blog)
  blog.posts << FactoryGirl.build_stubbed(:post, blog: blog)
end

# test output:
RuntimeError:
 stubbed models are not allowed to access the database - Post#save

That led to this incredibly detailed post by Aaron K. on StackOverflow.

Fix #1: Do Not Use FactoryGirl to Create Stubs

Fix #2: Clear the id field

Fix #3: Create your own definition of new_record?

I’m inclined to use instance_double in general with mocks in RSpec in the long term. But in the short term, I’m curious if I can get FactoryGirl.build_stubbed to work. So I try clearing the id field.

trait :with_posts do |blog, evaluator|
  blog.id = nil
  blog.posts << FactoryGirl.build_stubbed(:post, blog: blog)
end

# test output
Passed!

Ok. That feels dirty. It’s certainly a shim, but this is a proof-of-concept so I continue forward.

And that’s when build_stubbed began to fall apart. The main issue I found was that many persistence methods are disabled with stubbed objects in FactoryGirl. Also, before_validation calls are also skipped.

At the end of the day, I didn’t find a quick, clear path to using build_stubbed in our existing test suite. This may be a sign of tight coupling. There’s likely some refactoring we can do. My gut says use RSpec Mocks (specifically, instance_double to stub out database calls in unit tests).

What about you? What’s your experience with build_stubbed, integration tests, and/or writing Rails tests without depending on the database?