Running Typescript Tests with Playwright in Docker

I spent some hours this week on getting Playwright setup to run tests for the website of my upcoming micro VPN service ESPLink.

I needed the tests to execute in a Docker container and while there is a Docker image and some documentation, I was a little disappointed by the lack of any detail.

So lets fill in this gap.

Setup

Go ahead and follow the instructions for installing Playwright, but opt to not install the Playwright browsers. We are going to be using the Docker image to run them.

I assume you already have a Docker Compose file for setting up and running your local test environment. We will amend this file with an entry for the Playwright test runner container.

Add the following snippet to your compose.yml

 1  playwright:
 2    image: mcr.microsoft.com/playwright:v1.43.0-jammy
 3    command: npx playwright test
 4    working_dir: /app
 5    ipc: host
 6    profiles: [tests]
 7    environment:
 8      APP_PORT: ${YOUR_APP_HTTP_PORT}
 9      APP_HOST: ${YOUR_APP_CONTAINER_NAME}
10    ports:
11      - localhost:9323:9323
12    depends_on:
13      - ${YOUR_APP_CONTAINER_NAME}
14    volumes:
15      - type: bind
16        source: ${YOUR_APP_ROOT_DIRECTORY}
17        target: /app

Let's take a close look at this configuraiton in the next section.

Dissecting the compose.yml

This configuration has a couple of arguments that you will either need to provide to your docker compose invocation or replace with their corresponding values in your setup. But we'll get to those in just a second, let's look at the basics first.

1    image: mcr.microsoft.com/playwright:v1.43.0-jammy
2    command: npx playwright test
3    working_dir: /app

This sets up the container with the correct image from the Microsoft registry and sets the command for actually running the tests within the container. It also sets the working directory to a canonical location.

1    ipc: host

The little Playwright Docker documentation that exists sets the IPC mode of the container explicitly to host, so I adopted this. It doesn't seem required to me, but the documentation mentions that Chrome might run out of memory without it, so I didn't gamble and just left it.

1    profiles: [tests]

Setting a test profile for this container will prevent it from being started with the regular docker compose up invocation and only if the tests profile was explicitly selected. You can learn more about profiles in the Docker Compose documentation.

1    environment:
2      APP_PORT: ${YOUR_APP_HTTP_PORT}
3      APP_HOST: ${YOUR_APP_CONTAINER_NAME}

Here we set two environment variables for consumption in our tests. These will allow us to navigate the browsers to the correct URL when running our tests. You could hard-code those in your tests, but that would make changing these later much more cumbersome.

${YOUR_APP_CONTAINER_NAME} is the name of your application container or HTTP server hosting the site you want to test, e.g. your node or nginx container and ${YOUR_APP_HTTP_PORT} is the port your application or HTTP server is listening on.

1    ports:
2      - localhost:9323:9323

We expose port 9323 from the container to localhost on the host. This is the port that the test runner will publish the HTML test report on.

1    depends_on:
2      - ${YOUR_APP_CONTAINER_NAME}

Set your application container or webserver as a dependency so that it can start up before your test runner starts. Consider using the service_healthy condition if the startup of your application takes some time. You can learn more about how to use service_healthy and healthcheck in the Docker Compose documentation.

1    volumes:
2      - type: bind
3        source: ${YOUR_APP_ROOT_DIRECTORY}
4        target: /app

Bind your application root into the container. ${YOUR_APP_ROOT_DIRECTORY} is the directory that contains your node_modules and tests directories.

You can now run your tests with the following command:

1docker compose --profile tests run -P playwright

The -P argument will make sure to publish the ports to your host machine so you can inspect the resulting HTML report. To be able to actually access the web server from outside the container we need to instruct playwright to bind to a non-local interface. We do this with a small change in the playwright.config.ts file:

1  reporter: [['html', { host: '0.0.0.0' }]],

This setup should now work and if all you want to do is manually run some tests, this will likely be good enough for you. But there are some limitations:

1. This will reuse and run your tests against your regular local development environment. This might be acceptable for you, but it could very much be a show stopper.

2. There won't be any isolation between test runs either. Again, this might be acceptable, but may also not be.

3. Upon test failure, this will host the test run report on an HTTP server and block. While this is great when manually invocing it, it's not great if you are trying to utilize the test invocation as part of a script (e.g. Git pre-commit or pre-push hooks).

So let's isolate test runs from the development environment and each other and also setup some git hooks to automatically run the tests before pushing.

Isolation

The easiest way to achieve test isolation is to run the tests using a dedicated project within Docker. Doing this is as simple as passing the --project project-name flag to the docker compose run invocation. In addition we will need to drop the -P flag to avoid exposing any ports to the host that might conflict with ports from the development environment.

1docker compose --profile tests --project tests run playwright

This will run the tests in it's own Docker Compose "instance" called "tests". If you want to isolate test runs from each other you could use a semi-random project-name, e.g. based on /dev/random output. This might be useful if you need to run multiple instances of the tests in parallel, let's say on a CI server or similar.

There's now two new issues though. Having removed the -P will make it impossible to access the web server hosting the test results. At the same time, the playwright invocation will still block to make the web server available when the tests fail.

We can prevent the latter by changing the default configuration in playwright.config.ts to not automatically open the html reporter on failure:

1  reporter: [['html', { host: '0.0.0.0', open: 'never' }]],

This will still generate the html report, but will not automatically spin up the web server anymore.

To still view the the html report, simply run the following command:

1docker --profile tests --project tests  run --no-deps -P playwright npx playwright show-report --host 0.0.0.0

This will spin up the playwright web server hosting the html report (npx playwright show-report), without starting any of the other Docker Compose dependencies (--no-deps). The --host 0.0.0.0 argument is required because Playwright will not read the reporter settings from playwright.config.ts when invoked like this and will bind to the local loopback interface inside the container by default.

Given the increasing number of commands with long argument lists it's probably adviseable to make use of the scripts section of your package.json or to add a Makefile to your project. I personally prefer Makefiles, but most people will probably find the package.json approach more convenient.

1  "scripts": {
2    "tests": "docker compose --profile tests --project tests run playwright",
3    "report": "docker compose --profile tests --project tests run --no-deps -P playwright npx playwright show-report --host 0.0.0.0"
4  }

To achieve isolation between test runs we wil need to bring the Docker Compose project down again. It would be amazing if docker compose run had a --rm or --down option to do this automatically once the run completes, but it currently does not, so we'll need to take care of it ourselves.

The tricky part is to run the tests, bring down the prject and then return the test runner's exit code. Otherwise we won't be able to use the test runner effecticvely from scripts like Git hooks.

The easiest way is probably to write a small shell script for this.

1#!/bin/sh
2
3docker compose --profile tests --project tests run playwright
4RESULT=$?
5docker compose --profile tests --project tests down
6exit $RESULT

Don't forget to replace the "tests" script in your package.json with this shell script.

Now we can run npm run tests to run the tests and npm run report to open the HTML report. The tests will run in their own Docker Compose project which will spin down and get removed again after each test run.

Automatically running tests

Setting up a push hook is as simple as creating a script called pre-push in .git/hooks, making sure it's executable and calling npm run tests from it. E.g.

1npm run tests || exit 1

npm run tests will return a non-zero exist code when the tests fail, which will cause the pre-push hook to exit with an exit code of 1. Using || exit 1 is neccesary here, becase the Git hooks reserve the exit code 2 and require it not to be returned by the hook itself. Since we do not control the exit code of the playwright invocation directly, we need to make sure to not return anything other than 0 or 1 here.

Now you can run git push and your tests will automatically run and git push will abort should your tests fail.

Wrapup

And that's it. If you like content like this, please make sure to checkouy my YouTube channel, too. If you are interested in Home Automation, Home Assistant and ESPHome, you might also be interested in the ESPLink Micro VPN service I'm building or ESPHome Web App (Github), a standalone web application for provisioning, configuring and controlling ESPHome based devices.