Home Blog Notes Activities

Weekly Notes

Docs Aren't Enough, Docker Volume Validation, and Speeding up the Test Suite

Documentation Isn't Enough

I maintain a fairly popular open source project for Recipe management and Meal Planning. Recently I've started testing the new Version 1 beta release with some of our core user-base. With this version release there's a significant change in how Mealie is deployed on their server. Without a doubt it has some added complexity, and while I thought that the documentation I provided did a really good job at breaking down the steps, it wasn't enough and many users got caught up in some of the complexities of setting the new version up. Lesson learned, sometimes documentation isn't enough to combat the complexity because either they didn't read it, or what I see as a clear and concise explanation isn't so clear and concise. So what do we do then? Bake checks into the application!

Docker Volume Validation

One of the biggest hick-ups in deployment of the new version of Mealie is the docker volume mounts. In the new version the mealie-frontend and mealie-api containers share a volume mount so the proxy server in mealie-frontend can easily server up the images and reduce load on the API server. This was working excellently in the previous version in significantly increased performance overall. That said, mounting the volume on the backend and the frontend containers provided to be a constant source of problems for users on discord. It became very clear that we needed some way to validate the settings.

Inspired by the dns challenge certificates are issued I was able to provide an easy way to the end-user to validate their volume was configured correctly. The process works something like this:

  1. Client issues a API request to the backend endpoint to generate a temporary file at a know location on disk. In this case /app/data/docker-validation/validate.txt. The temporary file contains a randomly generated text that is then returned to the client when it's generated. This also starts a job in the background that will clean up with file in 60 seconds.
  2. After the client receives the randomly generated text, it then requests the validate.txt file which is served up from the mounted volume on the mealie-frontend container.
  3. The client then reads the text file and compares the two strings an ensures that they are identical.

This has proved to be a very efficient and simple way to validate that they have in-fact mounted the same volume to both containers.

Speeding Up Your Test Suite

One of the more obvious once you hear it ways to speed up your test suite is to disable password hashing in testing! Mealie's test suite was originally running around 50-60 seconds, after implementing a simple Protocol class I was able to cut that in half to 25 seconds on average, which is a huge time saver during development. Bonus points, because the implementation is very simple.

from functools import lru_cachefrom typing import Protocolfrom passlib.context import CryptContextfrom mealie.core.config import get_app_settingsclass Hasher(Protocol):    def hash(self, password: str) -> str:        ...    def verify(self, password: str, hashed: str) -> bool:        ...class FakeHasher:    def hash(self, password: str) -> str:        return password    def verify(self, password: str, hashed: str) -> bool:        return password == hashedclass PasslibHasher:    def __init__(self) -> None:        self.ctx = CryptContext(schemes=["bcrypt"], deprecated="auto")    def hash(self, password: str) -> str:        return self.ctx.hash(password)    def verify(self, password: str, hashed: str) -> bool:        return self.ctx.verify(password, hashed)@lru_cache(maxsize=1)def get_hasher() -> Hasher:    settings = get_app_settings()    if settings.TESTING:        return FakeHasher()    return PasslibHasher()

Here's a couple things to note...

  1. We're using a Hasher Protocol class to describe the qualities that a hasher would have. You can also do this with a similar thing with an abstract class.
  2. The FakeHasher class passes the plain text through on the hash function and on the verify method it uses a plain comparison operator to verify they match.
  3. The PasslibHasher class uses the CryptoContext provided by passlib to handle the the hashing and verifying. We are providing a wrapper to ensure that our API layer stays the same and we fulfill the contract in the Protocol class.
  4. The get_hasher function determined which hasher is returned, and since we don't expect the settings.TESTING env variable to ever change we can implement a lru_cache to speed up the subsequent reads.

If you decide to implement this in your code base, be sure to also implement some tests and hard failures around the get_hasher function to ensure that it operates as expected. The last thing you want to do is start storing raw passwords in the production database.