Notes about Unit Testing - Principles, Practices, and Patterns

These are my notes about a book by Vladimir Khorikov, Unit Testing - Principles, Practices, and Patterns. The book was not bad, but there was not much new in it (except chapters 1 and 3). Khorikov is a C# MPV (most valuable professional) at Microsoft. C# .NET culture seems to be less classicist TDD and more mockist testing afterwards compared to Java. Good chunk of the book is about realization that mockist tests are bad, but the approach seems to be tests after production code instead of TDD still. Since the book was written in 2020, there is no mention about LLMs or AI-assisted development in it yet. Still, many of the principles in the book are still valid today. On a technical note - the book has great chapter Summaries. You don't even have to read the whole book, as the Summaries cover the most important thoughts. I've read the book anyway. There were small tips all over the place. 

In the past Khorikov noticed how much test code was needed to test just 3 lines of production code. He knew that something was off. 

Chapter 1 - The goal of unit testing

The ratio of production code to test code can be anywhere between 1:1 and 1:3. #reassuring The discussion has shifted from Whether to write tests, to How to write good tests. We'll get the most value from the book if we write enterprise software, i.e. with complex business logic, not very data heavy and not performance hungry. 

The goal of unit testing is to enable sustainable growth of the software. Code tends to rot over time. As complexity grows, the codebase becomes less readable. To keep it maintainable, we need to use refactoring and tests enable that.

Both production code and test code are liabilities, not assets. We need to refactor both and there is a difference between good tests and bad ones. We should keep good tests only. Good tests are:

• Part of development pipeline.
• Target only important parts of code base.
• Provides maximum value with minimum maintenance costs.

We need to be able to recognize good tests and be able to refactor bad test to a good one. Ability to unit test something is a good negative indicator (it is a design smell if we are not able to test something) as well as code coverage (we should have at least 60% of code base unit tested).

Chapter 2 - What is a unit test?

A unit test (definition):

• Verifies single unit of behavior,
• Does it quickly,
• And does it in isolation from other tests.

An integration test is a test which is not a unit test - so has one or more of the properties unfulfilled. End to end test are tests against a fully integrated environment.

There is long and IMHO unnecessary discussion about the Detroit and London schools of unit testing. London testing, although IMHO very rich is reduced in this book to mocking everything except the current class / method. Unsurprisingly this is not a good idea and Detroit = classicist school wins. I agree with the result, but I didn't like the reduction of the London school to be a mockist testing. Mockist testing is IMHO a misunderstanding coming from generic developers about the London school of TDD.

Kharikov recommends two books - TDD by Example from Kent Beck and Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce. I completely agree with both recommendations. 

Chapter 3 - The anatomy of a unit test

Kharikov uses AAA pattern - arrange, act, assert. The only note about the Given-When-Then pattern is that it is more readable to non-programmers. I agree and that's why Java world seems to favor Given-When-Then instead. Kharikov says we might even start with the assert part, but notes that it is only viable when doing TDD (which makes me wonder whether he uses TDD).

There should be only one Act part in a unit tests. More that one Act part in a test could mean a wrong encapsulation design which might lead to invariant violations of the tested class. That said, it is sometimes beneficial to have more Act parts in integration tests, because they can be slow some times and this might speed things up. 

We should avoid IF statements in the tests. IF in a test means that the test is testing more than one thing. There is no exception from this rule for integration tests. The assert part should use fluent assertions.

The Arrange section should be the largest. We should beware Act sections longer than one line. The sections should be split by a newline or a section comment.

Each test should tell a story.

When multiple tests have a common arrange part, we shouldn't use the setUp method for this purpose. setUp is best reserved in a common abstract parent class for the tests. Explicit shared code via extracted method or a class (Object Mother) is preferable. 

The most prominent and the least helpful naming pattern for the tests is:

methodUnderTest_scenario_expectedResult

Much better pattern are simple English phrases: #interesting

sum_of_two_numbers() 

We shouldn't use a rigid naming policy, but we should name the tests as if we were describing the name to a non-programmer. #important

Chapter 4 - The four pillars of a good unit test

A good unit tests:

1. Protect against regression bugs.
• The more production code test executes the better.
2. Has low resistance towards against production code refactoring.
3. Gives fast feedback.
4. Is maintainable.
• How hard is to understand the test? The smaller the test, the more readable it is.
• How hard is to run the test? The fewer out-of-process dependencies the better.

We should beware the false alarms, i.e. when a test fails incorrectly, because over time, we'll stop listening to them. They are as important over time, for medium to large projects, as false negatives (unnoticed bugs).

It is impossible for a test to score maximum score in all 4 attributes, because the first 3 attributes are mutually exclusive. The 2nd point is usually non-negotiable. The tradeoff is usually between protection against regressions and fast feedback. Unit tests, integration tests and end-to-end tests are on the spectrum from fastest to slowest tests. 

Unit tests are best suited for testing complex algorithms and our domain model. In most trivial applications, there could be no unit tests. Otherwise, the test pyramid advocates for more unit tests, less integration tests and even less end-to-end tests.

We should favor black-box over white-box testing as much as possible.

Chapter 5 - Mocks and test fragility

Asserting interaction with stubs is a common anti-pattern leading to fragile tests. We should verify the end result instead. We can verify against mocks if the verification is about the observable system behavior, like a call to external service. Just internal calls should not be verified.

Making an API well-designed automatically improves unit tests.

According to Kharikov, a typical application follows a hexagonal architecture. The domain model is in the code, not depending on application layer. Application layer contains all integrations, such as with the database and no logic. No logic also means ideally no IFs.

Classical school of testing recommends avoiding shared dependencies between tests.

It is an antipattern to leak implementation details in the APIs.

Chapter 6 - Styles of unit testing

A functional architecture is even more extreme than the hexagonal architecture. The functional core is immutable, containing only pure functions. This architecture separates business logic from the side effects.

Output-based testing sends the inputs to the SUT and verifies the outputs from it. It assumes there are no side-effects. It leads to best quality of the tests. State-based testing verifies the state of the system after the operation is completed. The worst is communication-based testing, which verifies interactions between components.

Chapter 7 - Refactoring toward valuable unit tests

We should unit test only domain model and complex algorithms. They should have just a few (if any) collaborators. For the rest of the code, integration tests are a better fit.

We shouldn't write any tests for complex algorithmic code which also integrates to many collaborators. We should refactor it to algorithms / domain model and controllers with collaborators only, by using patterns like a Humble object. We can use a pattern of returning list of CanDos from the domain model and Do-s in the controllers to separate the business logic from the integration logic.

It is better not to write any test than to write a bad one.

Don't test preconditions if they don't have a business significance. 

Chapter 8 - Integration testing

Integration tests should be used for controllers, i.e. the code with many collaborators. They should cover one happy path and all edge cases should be covered by unit tests. Multi-step tests almost always belong to category of end-to-end tests.

Integration tests should use a real implementations of the dependencies we have full control of, e.g. our database and mocks for external dependencies out of our control. 

Logging should be tested only if it is a part of application observable behavior, not when it is an implementation detail.

Avoid circular dependencies in your codebase and don't cheat with dependency inversion. The cognitive load remains the same.

Chapter 9 - Mocking best practices

When mocking, verify only the interactions at the edge of the system. Mocks should be used in integration tests only, not in unit tests. We should always verify the number of the calls to the mock. We should only mock types that we own. We should verify there are no more calls to the mock in the test. We should duplicate constants and literals in the test code instead of referencing the production code, because otherwise we risk that we test nothing (tautology tests).

Chapter 10 - Testing the database

Database transactions also implement unit-of-work pattern. We should use at least 3 transactions in our tests - one for each section - arrange, act and assert. Introduction of an overarching transaction can lead to unrealistic tests.

We shouldn't modify more than one aggregate per business operation.

The database schema should be part of the versioned code in the versioning system (like git). Reference data should be part of git too. Each developer should have their separate database instance. Migrations such as Flyway or Liquibase are preferred over sharing the database state. Khorikov does not recommend running tests in parallel to avoid complexity of separating them. We should clean up the test data at the beginning of each test.

Chapter 11 - Unit testing anti-patterns

Don't expose private methods to enable unit testing. Instead, there might be an abstraction that is missing. Don't add to production code code that is used by tests only. When testing timestamps, inject an explicit dependency - either a service or as a plain value.