What is a mock

“Mock” is one of those words that means subtly different things to different people, so, to be specific, in this post I am using the following definition: a mock is a test double that stands in for a real component by satisfying its API and enforces assertions on how that API gets called during a test. The mocks that mockery generates would be Go examples.

Every mock writes a contract

Tests codify promises about functionality into executable logic that can verify that those promises are kept, effectively making them contracts with built-in enforcement. A test suite is a large collection of these promises that together should describe the full functionality of the system you have built and yell at you if you break any part of it. This makes the test suite an incredibly valuable part of a software project†, but there’s something important to keep in mind: what promises are your tests actually enforcing?

Taking a step back, let’s say you’re testing a component that needs a Repository or an RPC client or whatever. You could wire up the genuine article, but that requires additional plumbing, so you instead reach for a mock and configure it with the behaviour needed for this test. This is of course very convenient, but in so doing, you’ve also created a contract between the functionality you’re testing and its implementation. Probably a very strict one given how simple it is to add checks to your mocks, but are these the sorts of promises we want to codify into our test suite?

Useful (and not so useful) promises

The best tests, in my opinion, make promises about what your software achieves. They describe outcomes, both on the happy-path and on the sad-path.

As an example, imagine you’ve got some code that retries a flaky network call. There are a few ways to describe a test for this functionality, but let’s imagine two options:

• “The record still gets created even when the network is a bit flaky” ‡
• “The code calls the upstream API up to three times when a call fails”

These two are both ways to approach testing the same logic, but the promises they enforce are quite different. In the first case, the test is verifying that a desired outcome is being achieved, but in the second case, it is instead verifying that you are using a specific mechanism for achieving the desired outcome. So the day that you change that mechanism, even if it’s as minor as increasing the retry count§, this test is likely to fail. Yet, despite the failing test, the software is actually still doing its job successfully, it’s just going about it in a different way. Once you build up enough of these mechanistic tests, your test suite ends up becoming less of a way of validating that your software works and more of a way of validating that you haven’t changed anything, which is a big problem when you want to change something. On the other hand, tests that promise outcomes are much more durable across software change. You might need to tweak them here or there, but unless you make massive changes in how your application functions, they should still broadly work.

Thinking back to the promises our mocks are making, which one of these categories do they fall into? In my experience, mocks lend themselves heavily to the latter category of strict mechanistic contracts that enforce that a feature is implemented a certain way rather than that the feature works, effectively welding that implementation in place.

The revenge of the mock

Early in a project, this just seems like a largely academic issue without any practical significance. Your system is small, the mocks are quick to write, the tests are green, and everybody’s happy. But of course, you’re at the start, writing tests is already easy. You haven’t built up years of complex business logic or the residue of abstractions that didn’t quite fit but aren’t enough of a problem to be worth refactoring out. In effect, the mocking framework is giving you a discount on something that isn’t expensive yet.

Later on though, when the system is big and tangled and genuinely hard to reason about, precisely the moment the test suite is supposed to be saving your bacon, you start paying the cost of all that welding you did with your mocks. You go to refactor something, the observable behaviour is identical, you’re just changing how it achieves that goal, but suddenly dozens of tests go red.

The tests didn’t fail because anything broke, but because a bunch of mocks asserted that method A was called and now it’s method B, even though it never actually mattered whether it was A or B. You’re now spending your afternoon (or tokens) updating mock expectations to match the new implementation, which means your tests aren’t protecting you during the refactor at all, but are instead levying a toll on you for having the chutzpah to try to make an improvement to your codebase.

Which gets at the thing I find most insidious about all this: the web of mocks welding your implementations into place is largely invisible as the welds accumulate right up until the day you try to move something, and then every weld you didn’t think you were making announces itself at once.

So what do you do

For starters, I am not saying “never mock.” Sometimes you do actually want to write a mechanistic test. For example, if you’re writing a circuit breaker library, using a mock is a perfectly reasonable way to test that the breaker opens after N failed calls (or whatever the criteria is). And even if you don’t really want to write a mechanistic test, sometimes reaching for a tool that we’d rather not use is also just an unavoidable reality of the systems we work on, and deciding when and where to accept that kind of trade-off is a large part of the discipline of software engineering.

What I am saying is that mocks are a valid tool, but also one that is quite easy to burn yourself with without ever realizing you were holding a torch. Unfortunately, the appropriate type of double to use is too situational and too project-dependent for there to be a universal way to defuse the hazard. That said, I do think awareness can go a long way to help keep your codebase safe. So these days, when I add an expectation to a mock, I try to stop and ask myself what kind of promise I’m actually making. Am I making a durable promise about what the software achieves, or am I promising that the implementation will always look the way it looks today? Put another way, I try to imagine a scenario where the implementation’s call count, call order, precise arguments, etc. changed but the caller-observable behaviour stayed the same and ask myself: does this scenario make sense? If so, should this test fail?

Both types of promises are sometimes worth making. I’ve even written tests that exist for the sole purpose of welding a code structure or decision in place††; however, they’re very different promises, and the mocks can make it dangerously easy to make the second type of promise when you were trying to make the first. That’s mostly what’s made me wary of leaning on mocks too heavily: it’s not that they’re wrong, but that they make it so easy to weld yourself into place without noticing that you’re doing it.

So maybe think about loosening up those promises a bit and reach for something other than a mocking library. Here are a few options:

1. Use the actual dependency (sociable unit testing style). In my opinion, this is the best option when it’s practical. It can be a bit unwieldy for sad-path testing in particular though.
2. Write a simple stub. This is great for very simple single-purpose dependencies, and it lets you take advantage of your own laziness to help keep you from over-specifying behaviour. This is what I normally do for testing sad-paths since it’s usually extremely easy to write a stub that just fails.
3. Take the time to write a fake (e.g. a simple in-memory data store in place of a cache) that duplicates the dependency’s functionality in a simplified manner. This can be extremely helpful for other reasons (reducing flakiness, speeding up test execution, etc.), but it does often require a bit of development effort and ongoing care to avoid having the fake’s behaviour drift from the real implementation.

And I know that some people are probably thinking that this is all well and good, but your interfaces are too complicated to stub or fake. And to that I say, sure, that may very well be the case, but that might also be because the interface itself is doing too much and you need to narrow its scope. After all, a lot of application architecture concepts exist to make code more testable, so maybe embrace the practicality of writing a fake or stub as a forcing function for simplifying your interfaces and narrowing their scope.

† In many circumstances I’d argue that it is even more valuable than the software itself. After all, if you have a good test suite, you can largely recreate the software (and probably do so fairly quickly with coding agents and a healthy token budget), but if you have software without a test suite, you can barely do anything with it for fear of unexpectedly breaking it.

‡ Since I hate vague hand-wavy examples that lead you to think “well that sounds good but how do you actually do that in practice?”, it’s actually not all that hard to write tests for this. In Go, one approach would be to re-use your happy-path test, but inject an HTTP client with an http.Transport that returns an error the first time it makes a request to a given URL rather than executing the request, which simulates a flaky network. The rest of the happy-path test should then be largely reusable as-is to validate that the functionality still works. It is tempting to make the transport randomly drop requests instead, which you could do if you wanted to, but this is likely to result in a flaky hard-to-debug test suite since the test behaviour will not be consistent between executions.

§ This isn’t a contrived example. This was a very irritating issue on a project I worked on once, and it was made all the more irritating by how it interacted with our two (!) different systems for mocking HTTP interactions with external services.

†† This is rarely a good idea, but when I’ve done it, it has been when something is very critical in a non-obvious way or has a lot of underlying subtleties that are easy to miss. It’s also always accompanied by a long comment explaining the situation, but, as we all know, people don’t always read the comments. And even pre-LLM, it wasn’t unheard of for automated refactoring tools to erroneously modify areas of code that they weren’t supposed to. So, sometimes, making a test that is extremely rigid is called for when you want some extra insurance that changes to the relevant code are being done with care and knowledge of the consequences.