Dependency Injection & Testing

I’ve been using the Dependency Injector library as the DI container at work these days as part of a new project we’ve been rolling out. It’s an excellent library which makes things easier for us and will keep injecting dependencies easy as the application grows, (it also reminds me of the good ol’ .NET days where DI has been pretty much the standard routine for a long time). The library is pretty much feature complete for our use case, and most of the stuff that I’ve expected from a DI library is already present in it. There have been instances where I thought the library is missing something, only to find it present by RTFM or going through the open Github issues.

For all the unit tests we’ve been writing, the standard combination of Pytest and Mock has been working well. We mock some dependencies, run the test, and make sure it passes.

Recently I had to write integration tests which try to test out a more holistic, and pretty much end to end flow of the code path. Some of the classes in the flow I wanted to test were 4-5 levels deep. One option would be to generate such classes, injecting other classes these require manually, and that would’ve made some sense if we were only looking to inject something a level deep. In order to inject dependencies further down the graph, it makes sense to use the DI container itself, and pass down the mocked services.

Obviously, the entire idea behind doing an integration tests is that one needs to keep the flow as close to the original code flow as possible, especially trying not to mock everything as that ends up just being a very brittle and complicated unit test. Using DI container in this regard help as it can inject all the services, and then there are probably 1-2 you can mock out, and then assert the statements in the class under test.

In my case, one particular flow doesn’t end up using the database, and will follow a different code path depending if the code is running in production or in staging. So I was able to do two things:

  1. Stub out the database by passing a blank implementation
  2. Change the application to use the test configuration.

I searched through the github issues and there was a discussion regarding something similar to my needs. The gist is that each Container provides an override method which allows you to pass in a class that overrides part of the container. Great thing is that a container can be overridden in multiple ways:

  1. A class that overrides the existing container
  2. A decorator
  3. A context manager

The examples of 1. and 2. are present in the official docs, while the example of 3. is present in the github issue that I linked above. I ended up using the third approach wherein I override the container with test data inside the context manager. I chose this because it’s much cleaner, and overridden container reverts to the original when the context closes, allowing other test cases to override the container again.

The Code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# The top level application where the containers are registered
import application

from unittest.mock import MagicMock
from dependency_injector import containers, providers


# This will live outside the test file obviously, but we'll test out
# the feature_class provided by this container
class FeatureContainer(containers.DeclarativeContainer):
    application_configuration = providers.resource(Configuration)

    # An example which shows that the DbPool is also registered in an outer
    # container using the Configuration class
    database_connection_pool = providers.Dependency(instance_of=DbPool)

    feature_class = providers.Singleton(FeatureClass, application_configuration)


class TestClass:
    def test(self):
        # Get the test configuration
        test_configuration = TestConfiguration()

        mocked_db_pool = MagicMock(spec=DbPool)

        with app.container.application_container.override(
            FeatureContainer(
                feature_class=FeatureClass(test_configuration),
                database_connection_pool=mocked_db_pool,
            )
        ):

            feature_class = application.container.feature_container.feature_class()

            mock_request = MagicMock(spec=FeatureParam)
            mock_request.param1 = "param1"
            mock_request.param2 = "param2"

            response = feature_class.method_to_test_with(mock_request)

            assert response == True

The code is pretty straight forward. I have a Sanic application which has the top level container registered. We’re going to test out the feature_class. The container FeatureContainer provides the services for the Feature module. In a real world app, there are going to be multiple classes inside a module. In the app I’ve been working with, this particular class had a database_connection_pool, a cache service, a few repositories, a few services, etc.

With the context manager option, we just need to override the class that needs to change. Other classes don’t really need to be overridden.

A catch is the database_connection_pool which uses the instance_of parameter. Unless you’re also providing a default instance, this would require a class to be passed in, otherwise the test will throw error.

Once everything’s set up, we just create an instance of the feature_class with the test configuration passed into the container, and then test the method we want to test.

Caveats

A couple of things stand out in the code. The first is that I have to import the entire application, which I’m not a fan of. But given this is an integration test, and these are supposed to be much lower in number than unit tests, this is something I’m okay to live with. Obviously, as the size of the application grows, this is going to a problem, but it’s something I’m willing to deal with when the time comes.

Second, the example is a bit simplified. If there are multiple provider.Dependency then we’ll have to override all of them because of the way the library works. But one can always work around that by having a default implementation set. For this flow, we had no need for a database, but in other tests I’d probably roll out an in-memory database or a separate test database.

Having said that, I like this particular library for our needs currently and have no complaints what so ever.