Comprehensive Guide to SimpleCov Gem

When developing software, especially in a team setting, understanding how much of your code is exercised by your tests can be crucial. This is where code coverage tools come in handy. One such tool for Ruby is SimpleCov, a popular gem that provides detailed code coverage reports for your Ruby projects.

This comprehensive guide will dive deep into SimpleCov, exploring its features, installation process, configuration options, and how to interpret its reports effectively. Whether you're new to code coverage or a seasoned developer looking to improve your testing practices, this guide will help you harness the power of SimpleCov to write better tests and improve the quality of your Ruby code.

What is SimpleCov?

SimpleCov is a code coverage analysis tool for Ruby. It allows you to measure how much of your Ruby code your tests execute. Code coverage is expressed as a percentage, indicating the proportion of your code exercised by your test suite. A higher code coverage percentage typically indicates more comprehensive testing, though it's important to note that high coverage does not guarantee the absence of bugs.

SimpleCov works by tracking which lines of code are executed during your test suite's run. It generates a report highlighting the percentage of code covered, as well as specific files and lines not covered by your tests. This information can be invaluable for identifying untested or poorly tested code, helping you focus your testing efforts where they are most needed.

How does SimpleCov work?

SimpleCov works by hooking into the Ruby Coverage library, a standard code coverage analysis library. Running your test suite with SimpleCov enabled, instruments your code to track which lines are executed. After the test run completes, SimpleCov collects this data and generates reports in various formats, such as HTML, JSON, and more.

Installation

First, you need to add SimpleCov to your Gemfile and install it using Bundler.

gem "simplecov", require: false, group: :test

> bundle install

Configuration (optional)

SimpleCov provides configuration options to customize its behaviour. You can create a configuration file (e.g., .simplecov) in your project directory to specify settings such as output format, coverage thresholds, and filters.

Integration with Test Suite

You typically integrate SimpleCov with your test suite by requiring it at the top of your test helper file (e.g., spec_helper.rb for RSpec or test_helper.rb for Minitest).

require "simplecov"

SimpleCov.start

SimpleCov offers ready-made groups for Rails development for common areas like Controllers, Models, and Helpers. These predefined groups simplify your initial setup and help you analyze code coverage more effectively. To benefit from these, update the first two lines of your test_helper as shown below.

require "simplecov"

SimpleCov.start "rails"

Note:

Due to the way SimpleCov integrates with Ruby's Coverage library, it requires initialization before any application code is loaded. If you start SimpleCov later, it will miss coverage data for previously executed code. Hence, the SimpleCov.start must be added at the top of your test helper file.

Execute tests

With SimpleCov enabled, run your test suite as usual. SimpleCov will automatically track code coverage during the test execution.

View Reports

After the test run is completed, SimpleCov generates coverage reports. You can view these reports in your terminal or open the HTML report in a web browser for a more detailed analysis. The reports are generated in the index.html file, which you can find in the application's coverage directory. If you use a Mac, run the following command to view the report in your terminal.

open coverage/index.html

Coverage filters

You can refine your coverage report by excluding specific files or directories. This is achieved using the add_filter method. The provided configuration demonstrates the following filters:

require "simplecov"

SimpleCov.start do
  add_filter "spec/"
  add_filter "config/"
  add_filter "node_modules/"
  add_filter "tmp"
end

The filter excludes non-code files and directories often found in Rails projects, such as tests, configuration files, and documentation. This helps focus the coverage report on the application logic itself.

The syntax add_filter "spec/" will remove all files that match "spec/" in their path. You can use a Regular expression to filter out files that contain specific words like:

SimpleCov.start do
  add_filter %r{^/spec/}
end

This simple regex filter will remove all files that start with /spec/ in their path.

You also can pass a block for filtering the files. Block filters are provided with a SimpleCov::SourceFile instance and anticipate your block to yield true (to exclude the file from the result) or false (to retain the result).

SimpleCov.start do
  add_filter do |source_file|
    source_file.lines.count < 10
  end
end

In the above example, the filter will remove all files with less than 10 lines of code.

Creating custom filters is straightforward. Subclass SimpleCov::Filter and implement a matches?(source_file) method. During filter execution, returning true from this method removes the specified source_file. In the example below, the filter_argument method is configured within the SimpleCov::Filter initialize method, where it is assigned a value of 10.

class LineFilter < SimpleCov::Filter
  def matches?(source_file)
    source_file.lines.count < filter_argument
  end
end

SimpleCov.add_filter LineFilter.new(10)

Grouping files and directories

Groups categorize similar file types, allowing you to view them independently. Enhance your coverage report's organization by grouping files. The provided configuration showcases the following groups:

require "simplecov"

SimpleCov.start do
  add_filter "spec/"
  add_filter "config/"
  add_filter "node_modules/"
  add_filter "tmp"

  add_group "Model Concerns", "app/model/concerns"
  add_group "Mailers", "app/mailers"
  add_group "Helpers", "app/helpers"
  add_group "Long files" do |source_file|
    source_file.lines.count > 200
  end
  add_group "Short files", LineFilter.new(10)
end

In the example above, you can see how coverage reports can be grouped based on directories.

.simplecov file

When consolidating various test suite results (such as Test/Unit and Cucumber) into one report using SimpleCov, typically, you'd need to configure your settings twice—once in test_helper.rb and once in env.rb.

To streamline this process, create a file named .simplecov in your project root. Then, you only need to retain the require "simplecov" statement in each test setup helper file (at the beginning), while relocating the SimpleCov.start code along with all customized configuration options into .simplecov.

# .simplecov

SimpleCov.start "rails" do
  add_filter "spec/"
  add_filter "config/"
  add_filter "node_modules/"
  add_filter "tmp"

  add_group "Model Concerns", "app/model/concerns"
  add_group "Mailers", "app/mailers"
  add_group "Helpers", "app/helpers"
end

# test/test_helper.rb

require "simplecov"

# features/support/env.rb
require "simplecov"

Features of SimpleCov

SimpleCov offers several features to help you gain insights into your code coverage:

• Coverage Metrics: SimpleCov provides metrics such as line coverage, branch coverage, and overall coverage percentage, giving you a comprehensive view of your test coverage.

• Customizable Reports: You can customize the format and content of the coverage reports generated by SimpleCov to suit your preferences and requirements.

• Integration with CI/CD: SimpleCov integrates seamlessly with continuous integration (CI) and continuous deployment (CD) pipelines, allowing you to track code coverage trends over time and ensure that new code contributions maintain adequate test coverage.

• Filtering: SimpleCov allows you to filter out specific files or directories from coverage analysis, enabling you to focus on the relevant parts of your codebase.

• Thresholds: You can set coverage thresholds to define minimum acceptable coverage levels for your project. SimpleCov will alert you if coverage falls below these thresholds, helping you maintain code quality standards.

  1. Minimum coverage

     You have the option to specify the minimum expected coverage percentage. SimpleCov will generate a non-zero result if this requirement is not met.

     SimpleCov.minimum_coverage 95
     
     # same as above (the default is to check line coverage)
     SimpleCov.minimum_coverage line: 95
     
     # check for a minimum line coverage of 95% and minimum 85% branch coverage
     SimpleCov.minimum_coverage line: 95, branch: 85

  2. Minimum coverage by file

     You have the ability to set the minimum expected coverage percentage on a per-file basis. If this requirement is not fulfilled, SimpleCov will produce a non-zero result. This feature is beneficial for maintaining relatively consistent coverage across the codebase.

     SimpleCov.minimum_coverage_by_file 90
     
     # same as above (the default is to check line coverage by file)
     SimpleCov.minimum_coverage_by_file line: 90

  3. Maximum coverage drop

     You can set the maximum allowed percentage drop in coverage at one time. SimpleCov will yield a non-zero result if this threshold is surpassed.

     SimpleCov.maximum_coverage_drop 5
     
     # same as above (the default is to check line drop)
     SimpleCov.maximum_coverage_drop line: 5
     
     # check for a maximum line drop of 5% and maximum 10% branch drop
     SimpleCov.maximum_coverage_drop line: 5, branch: 10

Conclusion

SimpleCov is a valuable tool for Ruby developers seeking to improve the quality and reliability of their code. By providing detailed insights into test coverage, SimpleCov empowers developers to write better tests, identify areas for improvement, and maintain code quality standards throughout the development lifecycle. By integrating SimpleCov into your workflow and following best practices for code coverage analysis, you can ensure that your Ruby projects are well-tested and resilient against bugs and regressions. To learn more about the gem, please check this link.