Testing database interactions using Go
When it comes to integration or end-to-end testing, there is no more place for the mocks ss everything tested around real components or at least stubs for integration. Sometimes this might seem hard, even more so when you have to deal with databases.
The simplest example of the test case that includes a database in it is some web server that returns a record data by its id. To test this case, the record should be present in the test database before the actual test run.
First of all, you should check that the database is clean and ready to receive the required data. The test should run in the isolated suite, which includes only the required data and doesn’t rely on any previous data it may contain. If, for example, a test case is about pagination of the records, assert might fail if there are some other records that were not cleaned and are not supposed to be there.
The second step is to insert the actual data you are dealing with into the database before the test run. You are going to do this either by executing some statements or uploading schema.
Go is a new language. Although it borrows ideas from existing languages, it has unusual properties that make effective Go programs different in character from programs written in its relatives.
As a former Ruby on Rails developer, I appreciate the practices and solutions like FactoryBot for fixtures, DatabaseCleaner and etc, that Rails provide for testing interactions with a database. All the good things that work and are used in other languages can find their way in Go.
Here I will introduce some packages that share the ideas:
Basically this package mimics the “Rails’ way” of writing tests for database applications, where sample data is kept in fixtures files. Before the execution of every test, the test database is cleaned and the fixture data is loaded into the database.
The testfixtures work out-of-the-box. It’s doing both things, it cleans a database before each test and then loads the data into the database. No parallel support.
2. dbcleaner
Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood and makes to make sure the tests can run in parallel without racing issues.
The dbcleaner allows clearing database before each test. It allows running tests in parallel.
3. go-txdb
Package txdb is a single transaction-based database sql driver. When the connection is opened, it starts a transaction and all operations performed on this sql.DB will be within that transaction. If concurrent actions are performed, the lock is acquired and connection is always released the statements and rows are not holding the connection.
The go-txdb allows to run tests inside the SQL transactions. This means what each test will run in the isolated database instance, this allows running tests in parallel.
4. polluter
This package was mainly created for testing purposes, to give the ability to seed a database with records from simple .yaml files. Polluter respects the order in files, so you can handle foreign_keys just by placing them in the right order.
The polluter gives the ability to seed a database with the data from YAML files. This package was created to be used with the database cleaning tool or transactional database.
Even though it looks too simple and could do more, it does everything that it should. If you want some random data you could pre-generate files with data using some faker package. When it comes to foreign_key constraint violation, polluter respects the order in files, so if you have placed your data in the proper order, everything will work fine.
To make your data as easy to use as possible, ids in tests should be hard-coded. If the key is auto-incremented, it could be managed with changing the initial value. For example, for MySQL with executing ALTER TABLE table_name AUTO_INCREMENT = 10;
The testfixtures package provides everything you need out-of-the-box. But I prefer to use more flexible solutions:
- For acceptance testing — database cleaner with polluter
- For integration testing — go-txdb with polluter.
This code example shows the usage of the polluter with the go-txdb. Here all the goals are reached.
Each test runs in the isolated transaction and is seeded with the required data. So each test is independent and other tests afterward will not fail regardless of the order of execution. This also allows running all tests in parallel without the fear of the race conditions.
This is the best what I found for now when it comes to testing the database interactions, I hope you enjoyed it.
