Packages for Testing your R Package
Testing your R package is crucial, and thankfully it only gets easier with time, thanks to experience… and awesome packages helping you setup and improve tests! In this post, we shall offer a roundup of packages for testing R packages, first in a section about general testing setup, and then in a section about testing “peculiar” stuff.
General package testing infrastructure
If you’re brand-new to unit testing your R package, I’d recommend reading this chapter from Hadley Wickham’s book about R packages.
There’s an R package called
RUnit for unit testing, but in the whole post we’ll mention resources around the
testthat package since it’s the one we use in our packages, and arguably the most popular one.
testthat is great! Don’t hesitate to reads its docs again if you started using it a while ago, since the latest major release added the
teardown() functions to run code before and after all tests, very handy.
To setup testing in an existing package i.e. creating the test folder and adding
testthat as a dependency, run
usethis::use_testthat(). In our WIP
pRojects package, we set up the tests directory for you so you don’t forget. Then, in any case, add new tests for a function using
testthis package might help make your testing workflow even smoother. In particular,
test_this() “reloads the package and runs tests associated with the currently open R script file.”, and there’s also a function for opening the test file associated with the current R script. Edit: as of version 2.0.0
devtools itself features functions for testing single files.
Assess your tests
To get a sense of how good your tests are, check these out:
covrcomputes the test coverage i.e. the percentage of lines of code that are covered by tests.
covrallows skipping some lines. If run on Travis or Appveyor, for instance, it can send a report to online coverage tools such as CodeCov or Coveralls, allowing you to visualize the coverage. At Locke Data we mostly don’t run
covrlocally but instead have it run on Travis. To set that up,
usethis::use_coverage(). Here is Coveralls report for
HIBPwnedand the corresponding badge. See a CodeCov report for comparison.
covrpagecreates a detailed coverage report that can serve as a README for your test folder. We’ve done that for
HIBPwnedso now without clicking in a coverage report, thanks to “detailed test results”, you can see the tests associated with each context.
Test all the things
Now, sometimes you might encounter cases of things that you don’t quite know how to test. Here’s a small list, but please comment about anything we’ve forgotten!
Sometimes you need to test whether your package works as expected “if something happens”, “if a thing has this value” and can’t rely on arguments. E.g. what happens if the environment variable
GITHUB_PAT doesn’t exist, or if a dependency isn’t installed? In such cases, what you might be after is mocking. The
testthat package itself has a
with_mock() function, but it’s now recommended to rather use the
If the mocking you need to perform is e.g. mimicking a 404 result from an API, or saving a web API response and replay it to not have to re-query the API at each test, you can use:
webmockrlike we did for
vcrdocs. It works for both
vcrdocs include a list of packages using
vcrfor testing in the wild.
Test plot outputs
You can test your plot outputs haven’t changed by using
vdiffr. To set things up you need to run
Test Shiny apps
Fear not, there’s a whole package dedicated to help you test Shiny apps! Check out
Test RStudio add-ins
remedy package by ThinkR has tests for its add-ins. See in particular the
scratch_file() function. The tests need to be run only when RStudio is available so a helper defines a
skip_if_not_rstudio() function. Thanks to Colin Fay and Jonathan Sidi for showing it to me.
I’ll be honest, I haven’t seen examples of this in the wild, which is not surprising given I don’t use htmlwidgets a lot. Still, worth mentioning are:
viztesttests htmlwidgets based on screenshots.
the idea to test the content of the html before or after interactions.
rdomcan be a part of such a workflow:
xml2to scrape the result +
testthatof course. Thanks to David Gohel for telling me this!
Test interactive behavior?
The ecosystem for R package testing is getting more and more complete and automatic, which is very exciting. To deploy your tests, you’ll need to learn about continuous integration, which thankfully is also an area with exciting developments. Maybe a subject for another post… In the meantime, feel free to tell us about your favourite resources for testing packages!