Docs Aren't Enough, Docker Volume Validation, and Speeding up the Test Suite
I maintain a fairly popular open source project for Recipe management and Meal Planning. Recently I've started testing the new Version 1 beta release with some of our core user-base. With this version release there's a significant change in how Mealie is deployed on their server. Without a doubt it has some added complexity, and while I thought that the documentation I provided did a really good job at breaking down the steps, it wasn't enough and many users got caught up in some of the complexities of setting the new version up. Lesson learned, sometimes documentation isn't enough to combat the complexity because either they didn't read it, or what I see as a clear and concise explanation isn't so clear and concise. So what do we do then? Bake checks into the application!
One of the biggest hick-ups in deployment of the new version of Mealie is the docker volume mounts. In the new version the
mealie-api containers share a volume mount so the proxy server in
mealie-frontend can easily server up the images and reduce load on the API server. This was working excellently in the previous version in significantly increased performance overall. That said, mounting the volume on the backend and the frontend containers provided to be a constant source of problems for users on discord. It became very clear that we needed some way to validate the settings.
Inspired by the dns challenge certificates are issued I was able to provide an easy way to the end-user to validate their volume was configured correctly. The process works something like this:
/app/data/docker-validation/validate.txt. The temporary file contains a randomly generated text that is then returned to the client when it's generated. This also starts a job in the background that will clean up with file in 60 seconds.
validate.txtfile which is served up from the mounted volume on the
This has proved to be a very efficient and simple way to validate that they have in-fact mounted the same volume to both containers.
One of the more obvious once you hear it ways to speed up your test suite is to disable password hashing in testing! Mealie's test suite was originally running around 50-60 seconds, after implementing a simple Protocol class I was able to cut that in half to 25 seconds on average, which is a huge time saver during development. Bonus points, because the implementation is very simple.
from functools import lru_cachefrom typing import Protocolfrom passlib.context import CryptContextfrom mealie.core.config import get_app_settingsclass Hasher(Protocol): def hash(self, password: str) -> str: ... def verify(self, password: str, hashed: str) -> bool: ...class FakeHasher: def hash(self, password: str) -> str: return password def verify(self, password: str, hashed: str) -> bool: return password == hashedclass PasslibHasher: def __init__(self) -> None: self.ctx = CryptContext(schemes=["bcrypt"], deprecated="auto") def hash(self, password: str) -> str: return self.ctx.hash(password) def verify(self, password: str, hashed: str) -> bool: return self.ctx.verify(password, hashed)@lru_cache(maxsize=1)def get_hasher() -> Hasher: settings = get_app_settings() if settings.TESTING: return FakeHasher() return PasslibHasher()
Here's a couple things to note...
get_hasherfunction determined which hasher is returned, and since we don't expect the
settings.TESTINGenv variable to ever change we can implement a
lru_cacheto speed up the subsequent reads.
If you decide to implement this in your code base, be sure to also implement some tests and hard failures around the
get_hasher function to ensure that it operates as expected. The last thing you want to do is start storing raw passwords in the production database.