Fakes

Published: Jun 1, 2023
Updated: Jun 1, 2023

• Test
• Swe

Summary

A fake doesn’t use a mock framework; instead fake is a lightweight implementation of an API that behaves like the real implementation, but isn’t suitable for production (e.g. an in-memory database). You shouldn’t need to write your own fakes often since fakes should usually be created and maintained by the owner of the real implementation¹.

Example

Fakes can be used when you can’t use a real implementation in your test, e.g. if the real implementation is too slow or it talks over the network.

A fake might not exist for an object you need to use in a test. Writing one can be challenging because you need to ensure that it has similar behavior to the real implementation, now and in the future.

Fakes can be a powerful tool for testing

• Executes quickly
• Allows you to effectively test your code without the drawbacks of using real implementations.

A single fake has the power to radically improve the testing experience of an API.

• If you scale that to a large number of fakes for all sorts of APIs, fakes can provide an enormous boost to engineering velocity across a software organization.

In a software organization where fakes are rare:

• Velocity will be slower because engineers can end up struggling with using real implementations that lead to slow and flaky tests.
• Engineers might resort to other test double techniques such as stubbing or interaction testing which can result in tests that are unclear, brittle, and less effective.
• If a fake is not available, first ask the owners of the API to create one.
  • The owners might not be familiar with the concept of fakes, or they might not realize the benefit they provide to users of an API.
  • If the owners of an API are unwilling or unable to create a fake, you might be able to write your own.
    • One way to do this is to wrap all calls to the API in a single class and then create a fake version of the class that doesn’t talk to the API. Doing this can also be much simpler than creating a fake for the entire API because often you’ll need to use only a subset of the API’s behavior anyway.

Finally, you could decide to settle on using a real implementation or resort to other test double techniques.

• In some cases, you can think of a fake as an optimization: if tests are too slow using a real implementation, you can create a fake to make them run faster.
• But if the speedup from a fake doesn’t outweigh the work it would take to create and maintain the fake, it would be better to stick with using the real implementation.

A fake requires more effort and more domain experience to create because it needs to behave similarly to the real implementation.

A fake also requires maintenance. Whenever the behavior of the real implementation changes, the fake must also be updated to match this behavior. Because of this, the team that owns the real implementation should write and maintain a fake.

If a team is considering writing a fake, a trade-off needs to be made on whether the productivity improvements that will result from the use of the fake outweigh the costs of writing and maintaining it. If there are only a handful of users, it might not be worth their time, whereas if there are hundreds of users, it can result in an obvious productivity improvement.

To reduce the number of fakes that need to be maintained, a fake should typically be created only at the root of the code that isn’t feasible for use in tests.

When a database can not be used in tests, a fake should exist for the database API itself rather than for each class that calls the database API.

Maintaining a fake can be burdensome if its implementation needs to be duplicated across programming languages, such as for a service that has client libraries that allow the service to be invoked from different languages.

• One solution for this case is to create a single fake service implementation and have tests configure the client libraries to send requests to this fake service.
• This approach is more heavyweight compared to having the fake written entirely in memory because it requires the test to communicate across processes.
• However, it can be a reasonable trade-off to make, as long as the tests can still execute quickly.

A fake should maintain fidelity to the API contracts of the real implementation. The fake must have perfect fidelity to the real implementation, but only from the perspective of the test.

Summary

For any given input to an API, a fake should return the same output and perform the same state changes of its corresponding real implementation.

Examples

• For a real implementation of database.save(itemId), if an item is successfully saved when its ID does not yet exist, but an error is produced when the ID already exists, the fake must conform to this same behavior.

• A fake for a hashing API doesn’t need to guarantee that the hash value for a given input is exactly the same as the hash value that is generated by the real implementation, tests likely don’t care about the specific hash value, only that the hash value is unique for a given input. If the contract of the hashing API doesn’t make guarantees of what specific hash values will be returned, the fake is still conforming to the contract even if it doesn’t have perfect fidelity to the real implementation.

• Other examples where perfect fidelity typically might not be useful for fakes include latency and resource consumption.

  • However, a fake cannot be used if you need to explicitly test for these constraints (e.g., a performance test that verifies the latency of a function call), so you would need to resort to other mechanisms, such as by using a real implementation instead of a fake.

• A fake might not need to have 100% of the functionality of its corresponding real implementation, especially if such behavior is not needed by most tests (e.g., error handling code for rare edge cases).

  • It is best to have the fake fail fast in this case;
  • For example, raise an error if an unsupported code path is executed.
  • This failure communicates to the engineer that the fake is not appropriate in this situation.

• A fake must have its own tests to ensure that it conforms to the API of its corresponding real implementation.
• A fake without tests might initially provide realistic behavior, but without tests, this behavior can diverge over time as the real implementation evolves.
• One approach to writing tests for fakes involves writing tests against the API’s public interface and running those tests against both the real implementation and the fake (these are known as contract tests).
• The tests that run against the real implementation will likely be slower, but their downside is minimized because they need to be run only by the owners of the fake.