Run minimal docker tests on every commit #3150

Is your feature request related to a problem? Please describe.
Currently our docker tests are only run on release, and they install a bunch of test dependencies to enable running our test application suite in the docker. This means that it is possible that our tests miss that we broke the docker image on main branch, and it is also possible that some dependency is missing in the docker, but it gets installed when we put the testing dependencies into the docker.

Describe the solution you'd like
We could also have a minimal docker test that runs on every commit on main, just testing basic functionality like does opening a page work.

Describe alternatives you've considered

Additional context