57 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
7ecb26cc92 Release version 1.1.11
Some checks failed
ci / tests (push) Has been cancelled
ci / detect-release (push) Has been cancelled
ci / publish-image (push) Has been cancelled
ci / tag-stable (push) Has been cancelled
2026-02-14 15:03:35 +01:00
Kevin Veen-Birkenbach
5d4a2d59db Fix installer selectors for setupSuperUser UI variants 2026-02-14 15:00:34 +01:00
Kevin Veen-Birkenbach
1847c14b63 Release version 1.1.10 2026-02-14 12:19:43 +01:00
Kevin Veen-Birkenbach
29e812f584 fix(installer): harden navigation races in setupSuperUser flow 2026-02-14 12:14:06 +01:00
Kevin Veen-Birkenbach
1c8de40a05 Release version 1.1.9 2026-02-14 11:04:30 +01:00
Kevin Veen-Birkenbach
4bfa7433f4 ci: run on all branches but release only from main via git tag detection
- Trigger CI on push for all branches and on pull_request
- Detect SemVer release tags (vX.Y.Z) via git tag --points-at
- Run publish-image and stable-tag only for tagged commits on main
- Pass version_tag and sha to reusable workflows
- Prevent tag pushes from triggering additional workflows

https://chatgpt.com/share/e/699044d3-c1d8-8013-a40d-974d1fc69974
2026-02-14 11:02:58 +01:00
Kevin Veen-Birkenbach
00c012e553 Release version 1.1.8 2026-02-14 10:53:40 +01:00
Kevin Veen-Birkenbach
1bebeb8abc ci: make ci.yml the single coordinator workflow
- Trigger ci on push and pull_request only
- Convert publish-image and stable-tag to reusable workflows (workflow_call)
- Add detect-release job for strict SemVer tag detection (vX.Y.Z)
- Run tests first, then publish image, then move stable tag
- Remove direct tag/push triggers from publish-image and stable-tag

https://chatgpt.com/share/e/699044d3-c1d8-8013-a40d-974d1fc69974
2026-02-14 10:47:56 +01:00
Kevin Veen-Birkenbach
01d1626cf2 ci(docker): publish image without leading 'v' in version tag
Strip the leading 'v' from git tags (e.g. v1.1.7 -> 1.1.7)
when pushing Docker images to GHCR.

https://chatgpt.com/share/e/699041b9-8610-8013-8d04-fbef93b10d3c
2026-02-14 10:34:41 +01:00
Kevin Veen-Birkenbach
5bbe78b272 Release version 1.1.7 2026-02-14 05:43:32 +01:00
Kevin Veen-Birkenbach
865d5155d5 Lint 2026-02-14 05:39:01 +01:00
Kevin Veen-Birkenbach
209037cd64 Harden compose installer timeouts and e2e stack diagnostics 2026-02-14 05:37:29 +01:00
Kevin Veen-Birkenbach
ba2d84b6cb Fix nix e2e to use compose run instead of exec 2026-02-14 05:08:12 +01:00
Kevin Veen-Birkenbach
4f5c41753f Release version 1.1.6 2026-02-14 04:54:10 +01:00
Kevin Veen-Birkenbach
aac01810a1 Add tables-step timeout env knobs to compose, docs, and e2e 2026-02-14 04:52:26 +01:00
Kevin Veen-Birkenbach
fb42167b89 Release version 1.1.5 2026-02-14 04:49:49 +01:00
Kevin Veen-Birkenbach
7836dbacf9 Harden web installer flow for nix e2e 2026-02-14 04:45:00 +01:00
Kevin Veen-Birkenbach
d380b1493c Lint 2026-02-14 03:47:37 +01:00
Kevin Veen-Birkenbach
e81c5262b0 Release version 1.1.4
Some checks failed
ci / tests (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2026-02-13 18:32:19 +01:00
Kevin Veen-Birkenbach
f35ea04d66 Resolve merge conflict and preserve installer hardening 2026-02-13 18:24:34 +01:00
Kevin Veen-Birkenbach
37a17b536d Harden installer readiness and fix e2e healthcheck 2026-02-13 15:20:18 +01:00
Kevin Veen-Birkenbach
c80fdf8d01 Release version 1.1.3 2026-02-12 01:27:16 +01:00
Kevin Veen-Birkenbach
276833bd16 fix(matomo-bootstrap): increase Playwright step wait from 200ms to 1000ms to reduce CI flakiness
Increase page.wait_for_timeout from 200ms to 1000ms in WebInstaller to mitigate race conditions during Matomo web installation steps in slower CI environments.

https://chatgpt.com/share/698d1e2f-1f40-800f-92bc-10a736358b40
2026-02-12 01:26:09 +01:00
Kevin Veen-Birkenbach
9e267ec83f Added github sponsor buttons
Some checks failed
ci / tests (push) Has been cancelled
2026-01-02 13:00:00 +01:00
Kevin Veen-Birkenbach
20274985bc Release version 1.1.2
Some checks failed
ci / tests (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-24 17:28:48 +01:00
Kevin Veen-Birkenbach
cf473d4f3f Ruff formated
Some checks failed
ci / tests (push) Has been cancelled
2025-12-24 17:25:07 +01:00
Kevin Veen-Birkenbach
84323bd2aa test: add integration tests for installer warning detection
Some checks failed
ci / tests (push) Has been cancelled
- add make target test-integration and run it in reusable CI workflow
- add integration unittest covering _page_warnings stderr output + deduplication
- surface Matomo installer warnings during Playwright flow (stderr only)

https://chatgpt.com/share/694c1371-365c-800f-bdf8-ede2e850e648
2025-12-24 17:22:50 +01:00
Kevin Veen-Birkenbach
1a65ceb015 Release version 1.1.1
Some checks failed
ci / tests (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-24 08:39:59 +01:00
Kevin Veen-Birkenbach
81746f4b26 ci: publish Docker images for version tags and stable releases
Some checks failed
ci / tests (push) Has been cancelled
- Publish images on semantic version tags (vX.Y.Z) as :vX.Y.Z and :latest
- Publish :stable image via workflow_run after successful Stable Tag workflow
- Build stable image from exact commit marked as stable
- Remove duplicate build steps and unify tag computation

https://chatgpt.com/share/694b9758-58fc-800f-a586-8f3a341ece9d
2025-12-24 08:33:37 +01:00
Kevin Veen-Birkenbach
45867ece38 Release version 1.1.0
Some checks failed
ci / tests (push) Has been cancelled
publish-image / tests (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-23 22:07:11 +01:00
Kevin Veen-Birkenbach
33ed86adf1 Updated README.md
Some checks failed
ci / tests (push) Has been cancelled
2025-12-23 22:05:52 +01:00
Kevin Veen-Birkenbach
f86f84ff93 Added missing constraints.txt
Some checks failed
ci / tests (push) Has been cancelled
2025-12-23 21:54:55 +01:00
Kevin Veen-Birkenbach
b429644d9e Relax Playwright dependency for Nix, pin exact version in Docker
Some checks failed
ci / tests (push) Has been cancelled
- Allow playwright>=1.46.0,<2 in pyproject.toml for Nix compatibility
- Add pip constraints.txt to pin playwright==1.46.0 in container builds
- Enforce constraints during Docker image build

https://chatgpt.com/share/694b0180-2734-800f-830e-44e15d0a527d
2025-12-23 21:54:16 +01:00
Kevin Veen-Birkenbach
9fbdce2972 Solved
Some checks failed
ci / tests (push) Has been cancelled
Error: Invalid installation targets: 'chromium-headless-shell'. Expecting one of: chromium, chrome, chrome-beta, msedge, msedge-beta, msedge-dev, firefox, webkit
make: *** [Makefile:86: playwright-install] Error 1
2025-12-23 21:38:27 +01:00
Kevin Veen-Birkenbach
1b3ee2c3fd fix: pin Playwright to 1.46.0 to match Docker base image
Some checks failed
ci / tests (push) Has been cancelled
The Docker image is based on mcr.microsoft.com/playwright/python:v1.46.0.
Using an open-ended dependency (playwright>=1.46.0) allowed newer
Playwright versions to be installed in CI, causing driver/browser
mismatches and bootstrap crashes (exit code 3).

Pinning Playwright to 1.46.0 ensures version consistency between
Python package and container image and fixes the docker-compose
E2E stack failure.

https://chatgpt.com/share/694afaed-7c1c-800f-a2da-ec8c06e6ebe1
2025-12-23 21:26:03 +01:00
Kevin Veen-Birkenbach
1f448f4457 test(e2e): make root docker-compose stack port-configurable
Some checks failed
ci / tests (push) Has been cancelled
Parameterize Matomo’s published port via MATOMO_PORT to avoid CI collisions,
update installer site URL accordingly, and adapt the root docker-compose E2E
test to run on a non-default port with explicit environment injection.

https://chatgpt.com/share/694af650-a484-800f-ace7-0a634d57b0a0
2025-12-23 21:16:51 +01:00
Kevin Veen-Birkenbach
7fa8b580d2 test(e2e): validate root docker-compose stack bootstrap flow
Some checks failed
ci / tests (push) Has been cancelled
Adds an end-to-end test that brings up the root docker-compose.yml stack,
runs the one-shot bootstrap container, verifies token-only stdout, and
checks the token via Matomo API, with full cleanup via down -v.

https://chatgpt.com/share/694af650-a484-800f-ace7-0a634d57b0a0
2025-12-23 21:06:16 +01:00
Kevin Veen-Birkenbach
bf69c110a7 fix(nix): use nixpkgs playwright-driver and disable Playwright browser downloads
Some checks failed
ci / tests (push) Has been cancelled
https://chatgpt.com/share/694af26e-fb6c-800f-b0e3-e710cd035798
2025-12-23 20:49:57 +01:00
Kevin Veen-Birkenbach
a582e8be13 ci: split reusable workflow into lint + e2e jobs
Some checks failed
ci / tests (push) Has been cancelled
- Run ruff in a dedicated lint job (faster feedback)
- Keep e2e separate and install dependencies only there
- Force Playwright Chromium install to avoid cache-related missing binaries
- Trigger CI on PRs and pushes to main

https://chatgpt.com/share/694ae842-1588-800f-9418-31e7d02ac45e
2025-12-23 20:35:09 +01:00
Kevin Veen-Birkenbach
e38051a92f fix(playwright): install chromium-headless-shell for headless runs
Some checks failed
CI / test (push) Has been cancelled
Playwright v1.46 expects the separate chromium_headless_shell binary in
headless mode. Install chromium-headless-shell alongside chromium in both
the Makefile and Nix flake installer to prevent ENOENT launch errors.

https://chatgpt.com/share/694ae842-1588-800f-9418-31e7d02ac45e
2025-12-23 20:06:30 +01:00
Kevin Veen-Birkenbach
8162d337b5 nix: disable pythonImportsCheck to avoid Playwright side effects during build
Some checks failed
CI / test (push) Has been cancelled
https://chatgpt.com/share/694ae5ee-de04-800f-9944-9d7deb74f300
2025-12-23 19:56:42 +01:00
Kevin Veen-Birkenbach
bac453c435 ci: add GHCR publish workflow gated by tests
Some checks failed
CI / test (push) Has been cancelled
- build & push image on tag
- publish :latest only for semver tags
- multi-arch (amd64, arm64)

https://chatgpt.com/share/694ae4b0-ca28-800f-863c-0916a2f62a43
2025-12-23 19:51:10 +01:00
Kevin Veen-Birkenbach
a2010cd914 feat(container): add pinned Playwright Docker image and compose stack for Matomo bootstrap
Some checks failed
CI / test (push) Has been cancelled
- Add Dockerfile based on pinned Playwright image (v1.46.0-jammy) for reproducible browser runtime
- Introduce docker-compose stack (MariaDB + Matomo + one-shot bootstrap)
- Extend Makefile with container image and stack management targets
- Add env.sample for environment-driven bootstrap configuration
- Relax Playwright dependency to >=1.46.0 to keep Nix builds compatible
- Add E2E test ensuring docker-compose bootstrap exits with 0 and prints token
2025-12-23 19:41:31 +01:00
Kevin Veen-Birkenbach
f270a5c7c6 Release version 1.0.1
Some checks failed
CI / test (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-23 17:50:24 +01:00
Kevin Veen-Birkenbach
5e5b6c8933 fix(nix/e2e): keep Playwright install output off stdout
Some checks failed
CI / test (push) Has been cancelled
- Build matomo-bootstrap with setuptools/wheel in Nix
- Run playwright install via a dedicated pythonPlaywright env and redirect logs to stderr
- Add Nix-based E2E path: nix runner service + persistent nix/home volumes + host networking
- Add E2E test that runs bootstrap via `nix run` and asserts token-only stdout

https://chatgpt.com/share/694ab489-7028-800f-8398-a19a99faffd0
2025-12-23 17:44:41 +01:00
Kevin Veen-Birkenbach
1af480ee91 chore(nix): pin nixpkgs via flake.lock for reproducible runs
https://chatgpt.com/share/694ab489-7028-800f-8398-a19a99faffd0
2025-12-23 17:29:03 +01:00
Kevin Veen-Birkenbach
4f7de18a11 Release version 1.0.0
Some checks failed
CI / test (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-23 13:33:39 +01:00
Kevin Veen-Birkenbach
1a65077d0c Deactivated Mirror git.veen.world temporary due to db issues
Some checks failed
CI / test (push) Has been cancelled
2025-12-23 13:32:15 +01:00
Kevin Veen-Birkenbach
83967ab61f Added funding 2025-12-23 13:04:17 +01:00
Kevin Veen-Birkenbach
50914dea8b docs(readme): add installation and usage instructions for nix and python
Document how to install and use matomo-bootstrap via Nix flakes and pip,
describe CLI usage, environment variables, debug mode, E2E workflow,
and add author attribution.

https://chatgpt.com/share/694a84dd-aa30-800f-b45e-8a30e66e1c9b
2025-12-23 13:02:33 +01:00
Kevin Veen-Birkenbach
c9d4a5a9a4 feat(nix): expose matomo-bootstrap as flake package and app
- add flake packages and apps for matomo-bootstrap
- provide nix run support for CLI execution
- add Playwright browser installer app
- simplify devShell (remove pytest, keep ruff only)
- align pyproject license with MIT LICENSE
- register stable CLI entry point via project.scripts
2025-12-23 12:58:48 +01:00
Kevin Veen-Birkenbach
5f645cbfbf Changed License to MIT 2025-12-23 12:51:53 +01:00
Kevin Veen-Birkenbach
47dc84238f Added Mirrors 2025-12-23 12:50:17 +01:00
Kevin Veen-Birkenbach
d41129b6bd ci: deduplicate workflows using reusable test pipeline and stable tagging
- Introduce reusable-test workflow for Ruff + E2E tests
- Refactor CI to call reusable workflow (no duplicated steps)
- Add stable-tag workflow to promote version tags to stable after successful tests

https://chatgpt.com/share/694a7f81-d96c-800f-88cb-7b25b4cdfe1a
2025-12-23 12:46:30 +01:00
Kevin Veen-Birkenbach
482ac3377d ci: add GitHub Actions workflow with ruff and E2E tests
- Add CI workflow running Ruff lint/format checks
- Run full E2E cycle (Docker Compose + Playwright + tests)
- Refactor code formatting to satisfy Ruff (line breaks, readability)
- Use sys.executable in tests for interpreter-agnostic execution

https://chatgpt.com/share/694a7f81-d96c-800f-88cb-7b25b4cdfe1a
2025-12-23 12:39:39 +01:00
Kevin Veen-Birkenbach
5dbb1857c9 refactor: remove bootstrap wrappers and split into config/service/installers
- Replace bootstrap wrapper with config-driven service orchestration
- Introduce Config dataclass for centralized env/CLI validation
- Add MatomoApi service for session login + app token creation
- Move Playwright installer into installers/web and drop old install package
- Refactor HttpClient to unify HTTP handling and debug to stderr
- Make E2E tests use sys.executable instead of hardcoded python3
2025-12-23 12:33:42 +01:00
Kevin Veen-Birkenbach
92a2ee1d96 fix(cli): keep stdout clean by sending installer/auth logs to stderr
- route all installer and auth debug output to stderr
- ensure CLI stdout contains only the generated token
- fix E2E test failure caused by mixed log/token output

https://chatgpt.com/share/694a7b30-3dd4-800f-ba48-ae7083cfa4d8
2025-12-23 12:24:54 +01:00
39 changed files with 3155 additions and 566 deletions

7
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,7 @@
github: kevinveenbirkenbach
patreon: kevinveenbirkenbach
buy_me_a_coffee: kevinveenbirkenbach
custom: https://s.veen.world/paypaldonate

76
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,76 @@
name: ci
on:
pull_request: {}
push:
branches:
- "**"
permissions:
contents: write
packages: write
jobs:
tests:
uses: ./.github/workflows/reusable-test.yml
with:
python-version: "3.12"
matomo-token-description: "ci-token"
detect-release:
# Only consider releases on main branch pushes (not PRs, not other branches)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
outputs:
is_semver_tag: ${{ steps.detect.outputs.is_semver_tag }}
version_tag: ${{ steps.detect.outputs.version_tag }}
steps:
- name: Checkout (full history for tags)
uses: actions/checkout@v4
with:
fetch-depth: 0
- id: detect
shell: bash
run: |
set -euo pipefail
git fetch --tags --force
# Tags that point to the current commit
TAGS="$(git tag --points-at "$GITHUB_SHA" || true)"
# Pick the first strict SemVer tag: vX.Y.Z
VERSION_TAG="$(echo "$TAGS" | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -n1 || true)"
if [ -n "$VERSION_TAG" ]; then
echo "is_semver_tag=true" >> "$GITHUB_OUTPUT"
echo "version_tag=$VERSION_TAG" >> "$GITHUB_OUTPUT"
echo "Release tag detected on this commit: $VERSION_TAG"
else
echo "is_semver_tag=false" >> "$GITHUB_OUTPUT"
echo "version_tag=" >> "$GITHUB_OUTPUT"
echo "No SemVer tag on this commit."
fi
publish-image:
# Only on main, and only if detect-release found a SemVer tag on this commit
if: needs.detect-release.outputs.is_semver_tag == 'true'
needs: [tests, detect-release]
uses: ./.github/workflows/publish-image.yml
with:
version_tag: ${{ needs.detect-release.outputs.version_tag }}
sha: ${{ github.sha }}
permissions:
contents: read
packages: write
tag-stable:
# Only after tests + publish succeeded
if: needs.detect-release.outputs.is_semver_tag == 'true'
needs: [tests, detect-release, publish-image]
uses: ./.github/workflows/stable-tag.yml
with:
version_tag: ${{ needs.detect-release.outputs.version_tag }}
sha: ${{ github.sha }}
permissions:
contents: write

59
.github/workflows/publish-image.yml vendored Normal file
View File

@@ -0,0 +1,59 @@
name: publish-image
on:
workflow_call:
inputs:
version_tag:
type: string
required: true
sha:
type: string
required: true
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout (exact commit)
uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ inputs.sha }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Compute tags
id: meta
shell: bash
run: |
set -euo pipefail
IMAGE="ghcr.io/${{ github.repository }}"
RAW_TAG="${{ inputs.version_tag }}" # e.g. v1.1.8
TAG="${RAW_TAG#v}" # -> 1.1.8
echo "tags=$IMAGE:$TAG,$IMAGE:latest" >> "$GITHUB_OUTPUT"
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile
push: true
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max

117
.github/workflows/reusable-test.yml vendored Normal file
View File

@@ -0,0 +1,117 @@
name: reusable-test
on:
workflow_call:
inputs:
python-version:
type: string
required: false
default: "3.12"
matomo-url:
type: string
required: false
default: "http://127.0.0.1:8080"
matomo-admin-user:
type: string
required: false
default: "administrator"
matomo-admin-password:
type: string
required: false
default: "AdminSecret123!"
matomo-admin-email:
type: string
required: false
default: "administrator@example.org"
matomo-token-description:
type: string
required: false
default: "ci-token"
jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
cache: "pip"
- name: Install lint deps
run: |
python -m pip install --upgrade pip
pip install ruff
- name: Ruff
run: |
ruff check .
ruff format --check .
integration:
runs-on: ubuntu-latest
needs: lint
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Integration tests
run: make test-integration
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
cache: "pip"
- name: Install system deps (curl)
run: |
sudo apt-get update
sudo apt-get install -y curl
- name: Install Python deps (editable + e2e)
run: |
python -m pip install --upgrade pip
pip install -e ".[e2e]"
- name: Install Playwright Chromium
run: |
python -m playwright install --with-deps --force chromium
- name: Run E2E (docker compose)
env:
MATOMO_URL: ${{ inputs.matomo-url }}
MATOMO_ADMIN_USER: ${{ inputs.matomo-admin-user }}
MATOMO_ADMIN_PASSWORD: ${{ inputs.matomo-admin-password }}
MATOMO_ADMIN_EMAIL: ${{ inputs.matomo-admin-email }}
MATOMO_TOKEN_DESCRIPTION: ${{ inputs.matomo-token-description }}
run: |
make e2e
- name: Docker logs (on failure)
if: failure()
run: |
docker compose -f tests/e2e/docker-compose.yml ps || true
docker compose -f tests/e2e/docker-compose.yml logs --no-color --tail=300 matomo || true
docker compose -f tests/e2e/docker-compose.yml logs --no-color --tail=300 db || true
- name: Cleanup (always)
if: always()
run: |
docker compose -f tests/e2e/docker-compose.yml down -v || true

39
.github/workflows/stable-tag.yml vendored Normal file
View File

@@ -0,0 +1,39 @@
name: Stable Tag
on:
workflow_call:
inputs:
version_tag:
type: string
required: true
sha:
type: string
required: true
permissions:
contents: write
jobs:
tag-stable:
runs-on: ubuntu-latest
steps:
- name: Checkout (full history for tags)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Move stable tag to the release commit
shell: bash
run: |
set -euo pipefail
echo "Release tag: ${{ inputs.version_tag }}"
echo "Commit: ${{ inputs.sha }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --tags --force
git tag -fa stable -m "stable -> ${{ inputs.version_tag }} (${{ inputs.sha }})" "${{ inputs.sha }}"
git push --force origin stable

71
CHANGELOG.md Normal file
View File

@@ -0,0 +1,71 @@
## [1.1.11] - 2026-02-14
* This release improves matomo-bootstrap installer resilience by adding robust setupSuperUser field and button detection to prevent intermittent bootstrap failures.
## [1.1.10] - 2026-02-14
* This release fixes a reproducible Playwright navigation race in the Matomo installer (setupSuperUser), hardens the Next/Continue flow, and adds integration tests for transient locator errors and progress detection without a visible Next button.
## [1.1.9] - 2026-02-14
* Reworked CI to run on all branches while restricting Docker image publishing and stable tagging to tagged commits on main, using git-based SemVer detection.
## [1.1.8] - 2026-02-14
* Refactored CI to use a single coordinator workflow with strict SemVer-based release gating and adjusted Docker image publishing to strip the leading v from version tags.
## [1.1.7] - 2026-02-14
* Harden compose installer timeouts and e2e stack diagnostics
## [1.1.6] - 2026-02-14
* Add installer table-step timeout env vars (MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S, MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S) to compose/docs and e2e checks.
## [1.1.5] - 2026-02-14
* Harden web installer flow for nix e2e
## [1.1.4] - 2026-02-13
* This release hardens Matomo bootstrap by adding installer UI readiness waits/retries.
## [1.1.3] - 2026-02-12
* Increase Playwright step wait from 200ms to 1000ms to improve CI stability during Matomo installation.
## [1.1.2] - 2025-12-24
* **Improved error visibility during Matomo installation**: When the setup fails (for example due to an invalid admin email or missing required fields), the installer now **prints the actual Matomo error messages to the logs**, instead of failing with a generic error.
## [1.1.1] - 2025-12-24
* Improved Docker image publishing: automatic `vX.Y.Z`, `latest`, and `stable` tags for releases.
## [1.1.0] - 2025-12-23
* Implemented bootstrap docker image to auto install matomo in docker compose
## [1.0.1] - 2025-12-23
* * Support for running `matomo-bootstrap` **fully via Nix** in a clean, containerized environment.
* A **token-only stdout contract**: the bootstrap command now prints only the API token, making it safe for automation.
* Reproducible Nix builds via a pinned `flake.lock`.
## [1.0.0] - 2025-12-23
* 🥳

21
Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
# Playwright Python image with Chromium + all required OS dependencies
# Version should roughly match your playwright requirement
FROM mcr.microsoft.com/playwright/python:v1.46.0-jammy
# Keep stdout clean (token-only), logs go to stderr
ENV PYTHONUNBUFFERED=1
WORKDIR /app
# Install matomo-bootstrap
# Option A: from PyPI (recommended once published)
# RUN pip install --no-cache-dir matomo-bootstrap==1.0.1
# Option B: build from source (current repo)
COPY pyproject.toml README.md LICENSE /app/
COPY constraints.txt /app/
COPY src /app/src
RUN pip install --no-cache-dir -c /app/constraints.txt .
# Default entrypoint: environment-driven bootstrap
ENTRYPOINT ["matomo-bootstrap"]

View File

@@ -1 +1,7 @@
All rights reserved by Kevin Veen-Birkenbach Copyright 2025 Kevin Veen-Birkenbach
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -1,2 +1,3 @@
git@github.com:kevinveenbirkenbach/matomo-bootstrap.git git@github.com:kevinveenbirkenbach/matomo-bootstrap.git
https://pypi.org/project/matomo-bootstrap/ ssh://git@code.infinito.nexus:2201/kevinveenbirkenbach/matomo-bootstrap.git
https://pypi.org/project/matomo-bootstrap/

184
Makefile
View File

@@ -1,4 +1,8 @@
PYTHON ?= python3 PYTHON ?= python3
# ----------------------------
# E2E (existing)
# ----------------------------
COMPOSE_FILE := tests/e2e/docker-compose.yml COMPOSE_FILE := tests/e2e/docker-compose.yml
COMPOSE := docker compose -f $(COMPOSE_FILE) COMPOSE := docker compose -f $(COMPOSE_FILE)
@@ -13,11 +17,30 @@ MATOMO_ADMIN_PASSWORD ?= AdminSecret123!
MATOMO_ADMIN_EMAIL ?= administrator@example.org MATOMO_ADMIN_EMAIL ?= administrator@example.org
MATOMO_TOKEN_DESCRIPTION ?= e2e-make-token MATOMO_TOKEN_DESCRIPTION ?= e2e-make-token
.PHONY: help venv deps-e2e playwright-install e2e-up e2e-install e2e-test e2e-down e2e logs clean # ----------------------------
# Container image (production-like)
# ----------------------------
IMAGE_NAME ?= ghcr.io/kevinveenbirkenbach/matomo-bootstrap
IMAGE_VERSION ?= 1.0.1
# Optional .env file for container runs
ENV_FILE ?= .env
# ----------------------------
# docker-compose stack (Matomo + MariaDB + Bootstrap)
# ----------------------------
COMPOSE_STACK_FILE ?= docker-compose.yml
COMPOSE_STACK := docker compose -f $(COMPOSE_STACK_FILE)
.PHONY: help \
venv deps-e2e playwright-install e2e-up e2e-install e2e-test e2e-down e2e logs clean \
test-integration \
image-build image-run image-shell image-push image-clean \
stack-up stack-down stack-logs stack-ps stack-bootstrap stack-rebootstrap stack-clean stack-reset
help: help:
@echo "Targets:" @echo "Targets:"
@echo " venv Create local venv in $(VENV_DIR)" @echo " venv Create local venv in $(VENV_DIR)"
@echo " deps-e2e Install package + E2E deps into venv" @echo " deps-e2e Install package + E2E deps into venv"
@echo " playwright-install Install Chromium for Playwright (inside venv)" @echo " playwright-install Install Chromium for Playwright (inside venv)"
@echo " e2e-up Start Matomo + DB for E2E tests" @echo " e2e-up Start Matomo + DB for E2E tests"
@@ -25,15 +48,76 @@ help:
@echo " e2e-test Run E2E tests (unittest)" @echo " e2e-test Run E2E tests (unittest)"
@echo " e2e-down Stop and remove E2E containers" @echo " e2e-down Stop and remove E2E containers"
@echo " e2e Full cycle: up → install → test → down" @echo " e2e Full cycle: up → install → test → down"
@echo " logs Show Matomo logs" @echo " logs Show Matomo logs (E2E compose)"
@echo " clean Stop containers + remove venv" @echo " clean Stop E2E containers + remove venv"
@echo " test-integration Run integration tests (unittest)"
@echo "" @echo ""
@echo "Variables (override like: make e2e MATOMO_URL=http://127.0.0.1:8081):" @echo "Container image targets:"
@echo " MATOMO_URL, MATOMO_ADMIN_USER, MATOMO_ADMIN_PASSWORD, MATOMO_ADMIN_EMAIL, MATOMO_TOKEN_DESCRIPTION" @echo " image-build Build matomo-bootstrap container image"
@echo " image-run Run container bootstrap using $(ENV_FILE) (token-only stdout)"
@echo " image-shell Start interactive shell in container"
@echo " image-push Push image tags ($(IMAGE_VERSION) + latest)"
@echo " image-clean Remove local image tags"
@echo ""
@echo "docker-compose stack targets (docker-compose.yml):"
@echo " stack-up Start MariaDB + Matomo (no bootstrap)"
@echo " stack-bootstrap Run one-shot bootstrap (prints token to stdout)"
@echo " stack-reset Full reset: down -v → build → up → bootstrap"
@echo " stack-down Stop stack"
@echo " stack-clean Stop stack and REMOVE volumes (DANGER)"
@echo " stack-logs Follow Matomo logs (stack)"
@echo " stack-ps Show stack status"
@echo ""
@echo "Variables:"
@echo " E2E: MATOMO_URL, MATOMO_ADMIN_USER, MATOMO_ADMIN_PASSWORD, MATOMO_ADMIN_EMAIL, MATOMO_TOKEN_DESCRIPTION"
@echo " IMG: IMAGE_NAME, IMAGE_VERSION, ENV_FILE"
@echo " STK: COMPOSE_STACK_FILE"
# ----------------------------
# E2E targets
# ----------------------------
venv: venv:
@test -x "$(VENV_PY)" || ($(PYTHON) -m venv $(VENV_DIR)) @set -e; \
@$(VENV_PIP) -q install -U pip setuptools wheel >/dev/null if [ ! -d "$(VENV_DIR)" ]; then \
echo "Creating $(VENV_DIR) ..."; \
$(PYTHON) -m venv "$(VENV_DIR)"; \
fi; \
if ! [ -x "$(VENV_PY)" ] || ! "$(VENV_PY)" -V >/dev/null 2>&1; then \
echo "Repairing $(VENV_PY) symlink ..."; \
fix_target=""; \
for cand in "$(VENV_DIR)/bin/python3.14" "$(VENV_DIR)/bin/python3.13" "$(VENV_DIR)/bin/python3.12" "$(VENV_DIR)/bin/python3.11" "$(VENV_DIR)/bin/python3.10"; do \
if [ -x "$$cand" ]; then \
fix_target="$$(basename "$$cand")"; \
break; \
fi; \
done; \
if [ -z "$$fix_target" ] && [ -x "$(VENV_PIP)" ]; then \
shebang="$$(head -n1 "$(VENV_PIP)" | sed 's/^#!//')"; \
if [ -n "$$shebang" ] && [ -x "$$shebang" ]; then \
fix_target="$$(basename "$$shebang")"; \
fi; \
fi; \
if [ -n "$$fix_target" ] && [ -x "$(VENV_DIR)/bin/$$fix_target" ]; then \
ln -sfn "$$fix_target" "$(VENV_PY)"; \
ln -sfn "$$fix_target" "$(VENV_DIR)/bin/python3"; \
fi; \
fi; \
if ! [ -x "$(VENV_PIP)" ] || ! "$(VENV_PIP)" --version >/dev/null 2>&1; then \
echo "Repairing pip via ensurepip ..."; \
"$(VENV_PY)" -m ensurepip --upgrade >/dev/null 2>&1 || true; \
fi; \
if ! [ -x "$(VENV_PY)" ] || ! "$(VENV_PY)" -V >/dev/null 2>&1; then \
echo "ERROR: Could not repair $(VENV_PY) in existing $(VENV_DIR)."; \
echo "Run 'make clean' once or remove $(VENV_DIR) manually."; \
exit 2; \
fi; \
if ! [ -x "$(VENV_PIP)" ] || ! "$(VENV_PIP)" --version >/dev/null 2>&1; then \
echo "ERROR: Could not repair $(VENV_PIP) in existing $(VENV_DIR)."; \
echo "Run 'make clean' once or remove $(VENV_DIR) manually."; \
exit 2; \
fi; \
"$(VENV_PIP)" -q install -U pip setuptools wheel >/dev/null
deps-e2e: venv deps-e2e: venv
@$(VENV_PIP) install -e ".[e2e]" @$(VENV_PIP) install -e ".[e2e]"
@@ -72,6 +156,11 @@ e2e-test: deps-e2e
e2e-down: e2e-down:
$(COMPOSE) down -v $(COMPOSE) down -v
e2e-nix:
docker compose -f tests/e2e/docker-compose.yml up -d
python3 -m unittest -v tests/e2e/test_bootstrap_nix.py
docker compose -f tests/e2e/docker-compose.yml down -v
e2e: e2e-up e2e-install e2e-test e2e-down e2e: e2e-up e2e-install e2e-test e2e-down
logs: logs:
@@ -79,3 +168,82 @@ logs:
clean: e2e-down clean: e2e-down
rm -rf $(VENV_DIR) rm -rf $(VENV_DIR)
# ----------------------------
# Integration tests
# ----------------------------
test-integration:
PYTHONPATH=src $(PYTHON) -m unittest discover -s tests/integration -v
# ----------------------------
# Container image workflow
# ----------------------------
image-build:
docker build -t $(IMAGE_NAME):$(IMAGE_VERSION) -t $(IMAGE_NAME):latest .
image-run:
@test -f "$(ENV_FILE)" || (echo "Missing $(ENV_FILE). Create it from env.sample."; exit 1)
docker run --rm \
--env-file "$(ENV_FILE)" \
--network host \
$(IMAGE_NAME):$(IMAGE_VERSION)
image-shell:
@test -f "$(ENV_FILE)" || (echo "Missing $(ENV_FILE). Create it from env.sample."; exit 1)
docker run --rm -it \
--env-file "$(ENV_FILE)" \
--network host \
--entrypoint /bin/bash \
$(IMAGE_NAME):$(IMAGE_VERSION)
image-push:
docker push $(IMAGE_NAME):$(IMAGE_VERSION)
docker push $(IMAGE_NAME):latest
image-clean:
docker rmi $(IMAGE_NAME):$(IMAGE_VERSION) $(IMAGE_NAME):latest || true
# ----------------------------
# docker-compose stack workflow
# ----------------------------
## Start MariaDB + Matomo (without bootstrap)
stack-up:
$(COMPOSE_STACK) up -d db matomo
@echo "Matomo is starting on http://127.0.0.1:8080"
## Run one-shot bootstrap (prints token to stdout)
stack-bootstrap:
$(COMPOSE_STACK) run --rm bootstrap
## Re-run bootstrap (forces a fresh one-shot run)
stack-rebootstrap:
$(COMPOSE_STACK) rm -f bootstrap || true
$(COMPOSE_STACK) run --rm bootstrap
## Follow Matomo logs (stack)
stack-logs:
$(COMPOSE_STACK) logs -f matomo
## Show running services (stack)
stack-ps:
$(COMPOSE_STACK) ps
## Stop stack
stack-down:
$(COMPOSE_STACK) down
## Stop stack and REMOVE volumes (DANGER)
stack-clean:
$(COMPOSE_STACK) down -v
## Full reset: down -v → rebuild bootstrap → up → bootstrap
stack-reset:
$(COMPOSE_STACK) down -v
$(COMPOSE_STACK) build --no-cache bootstrap
$(COMPOSE_STACK) up -d db matomo
@echo "Waiting for Matomo to become reachable..."
@sleep 10
$(COMPOSE_STACK) run --rm bootstrap

317
README.md
View File

@@ -1,6 +1,319 @@
# matomo-bootstrap # matomo-bootstrap
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
Homepage: https://github.com/kevinveenbirkenbach/matomo-bootstrap
Headless bootstrap tooling for **Matomo**. Automates **first-time installation** and **API token provisioning** for fresh Matomo instances.
---
## Features
- 🚀 **Fully headless Matomo installation**
- Drives the official Matomo web installer using **Playwright**
- Automatically skips the installer if Matomo is already installed
- 🔐 **API token provisioning**
- Creates an **app-specific token** via an authenticated Matomo session
- Compatible with **Matomo 5.3.x** Docker images
- 🧪 **E2E-tested**
- Docker-based end-to-end tests included
- ❄️ **First-class Nix support**
- Flake-based packaging and pinned `flake.lock`
- Uses `nixpkgs` browsers via `playwright-driver` (no Playwright downloads)
- 🧼 **Token-only stdout contract**
- **stdout contains only the token** (safe for scripting)
- Logs go to **stderr**
---
## Requirements
- A running Matomo instance (e.g. via Docker)
- For fresh installs:
- Chromium (provided by Playwright or by the Playwright base container image)
---
## Installation
### Nix (recommended)
Run directly from the repository:
```bash
nix run github:kevinveenbirkenbach/matomo-bootstrap
```
In Nix mode, browsers are provided via `nixpkgs` (`playwright-driver`) and Playwright downloads are disabled.
---
### Python / pip
Requires **Python ≥ 3.10**:
```bash
pip install matomo-bootstrap
python -m playwright install chromium
```
---
### Docker image (GHCR)
Pull the prebuilt image:
```bash
docker pull ghcr.io/kevinveenbirkenbach/matomo-bootstrap:stable
# or:
docker pull ghcr.io/kevinveenbirkenbach/matomo-bootstrap:latest
```
---
## Usage
### CLI
```bash
matomo-bootstrap \
--base-url http://127.0.0.1:8080 \
--admin-user administrator \
--admin-password 'AdminSecret123!' \
--admin-email administrator@example.org \
--token-description my-ci-token
```
On success, the command prints **only the token** to stdout:
```text
6c7a8c2b0e9e4a3c8e1d0c4e8a6b9f21
```
---
### Environment variables
All options can be provided via environment variables:
```bash
export MATOMO_URL=http://127.0.0.1:8080
export MATOMO_ADMIN_USER=administrator
export MATOMO_ADMIN_PASSWORD='AdminSecret123!'
export MATOMO_ADMIN_EMAIL=administrator@example.org
export MATOMO_TOKEN_DESCRIPTION=my-ci-token
matomo-bootstrap
```
---
### Debug mode
Enable verbose logs (**stderr only**):
```bash
matomo-bootstrap --debug
```
---
## Docker Compose integration (one-shot bootstrap)
### Why “one-shot”?
The bootstrap container is meant to:
1. Run once,
2. Print the token to stdout,
3. Exit with code `0`.
You should **not** start it automatically on every `docker compose up`.
Instead, start Matomo normally, then run the bootstrap via `docker compose run`.
The cleanest Compose pattern is to put `bootstrap` behind a **profile**.
---
### Example `docker-compose.yml` (recommended: `profiles`)
```yaml
services:
db:
image: mariadb:11
container_name: matomo-db
restart: unless-stopped
environment:
MARIADB_DATABASE: matomo
MARIADB_USER: matomo
MARIADB_PASSWORD: matomo_pw
MARIADB_ROOT_PASSWORD: root_pw
volumes:
- mariadb_data:/var/lib/mysql
healthcheck:
test: ["CMD-SHELL", "mariadb-admin ping -uroot -proot_pw --silent"]
interval: 5s
timeout: 3s
retries: 60
matomo:
image: matomo:5.3.2
container_name: matomo
restart: unless-stopped
depends_on:
db:
condition: service_healthy
ports:
- "${MATOMO_PORT:-8080}:80"
environment:
MATOMO_DATABASE_HOST: db
MATOMO_DATABASE_ADAPTER: mysql
MATOMO_DATABASE_USERNAME: matomo
MATOMO_DATABASE_PASSWORD: matomo_pw
MATOMO_DATABASE_DBNAME: matomo
volumes:
- matomo_data:/var/www/html
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://127.0.0.1/ >/dev/null || exit 1"]
interval: 10s
timeout: 5s
retries: 60
bootstrap:
# This prevents automatic startup during a normal `docker compose up`
profiles: ["bootstrap"]
# Option A: use the published image (recommended)
image: ghcr.io/kevinveenbirkenbach/matomo-bootstrap:1.0.1
# Option B: build locally from the repository checkout
# build:
# context: .
# dockerfile: Dockerfile
# image: matomo-bootstrap:local
container_name: matomo-bootstrap
depends_on:
matomo:
condition: service_started
environment:
# inside the compose network, Matomo is reachable via the service name
MATOMO_URL: "http://matomo"
MATOMO_ADMIN_USER: "administrator"
MATOMO_ADMIN_PASSWORD: "AdminSecret123!"
MATOMO_ADMIN_EMAIL: "administrator@example.org"
MATOMO_TOKEN_DESCRIPTION: "docker-compose-bootstrap"
# Values used by the recorded installer flow
MATOMO_SITE_NAME: "Matomo (docker-compose)"
MATOMO_SITE_URL: "http://127.0.0.1:${MATOMO_PORT:-8080}"
MATOMO_TIMEZONE: "Germany - Berlin"
# Optional stability knobs
MATOMO_TIMEOUT: "60"
MATOMO_PLAYWRIGHT_HEADLESS: "1"
MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS: "60000"
MATOMO_PLAYWRIGHT_SLOWMO_MS: "0"
MATOMO_INSTALLER_READY_TIMEOUT_S: "240"
MATOMO_INSTALLER_STEP_TIMEOUT_S: "45"
MATOMO_INSTALLER_STEP_DEADLINE_S: "240"
MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S: "240"
MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S: "180"
MATOMO_INSTALLER_DEBUG_DIR: "/tmp/matomo-bootstrap"
restart: "no"
volumes:
mariadb_data:
matomo_data:
```
---
### Commands
Start DB + Matomo **without** bootstrap:
```bash
docker compose up -d db matomo
```
Run bootstrap once (prints token to stdout):
```bash
docker compose --profile bootstrap run --rm bootstrap
```
Re-run bootstrap (creates a new token by default):
```bash
docker compose --profile bootstrap run --rm bootstrap
```
---
## Idempotency / avoiding new tokens on every run
By default, `UsersManager.createAppSpecificTokenAuth` creates a new token each time.
If you want strictly idempotent runs in automation, you can provide an existing token
and make the bootstrap return it instead of creating a new one:
```bash
export MATOMO_BOOTSTRAP_TOKEN_AUTH="0123456789abcdef..."
matomo-bootstrap
```
> This is useful for CI re-runs or configuration management tools.
---
## How it works
1. **Reachability check**
* waits until Matomo responds via HTTP (any status is considered “reachable”)
2. **Installation (if needed)**
* uses a recorded Playwright flow to complete the Matomo web installer
* waits until installer controls are interactive before clicking next steps
* writes screenshot/HTML debug artifacts on installer failure
3. **Authentication**
* logs in using Matomos `Login.logme` controller (cookie session)
4. **Token creation**
* calls `UsersManager.createAppSpecificTokenAuth`
5. **Output**
* prints the token to stdout (token-only contract)
---
## End-to-end tests
Run the full E2E cycle locally:
```bash
make e2e
```
This will:
1. Start Matomo + MariaDB via Docker
2. Install Matomo headlessly
3. Create an API token
4. Validate the token via the Matomo API
5. Tear everything down again
---
## Author ## Author
Kevin Veen-Birkenbach <kevin@veen.world>
**Kevin Veen-Birkenbach**
[https://www.veen.world/](https://www.veen.world/)
---
## License
MIT — see [LICENSE](LICENSE)

1
constraints.txt Normal file
View File

@@ -0,0 +1 @@
playwright==1.46.0

80
docker-compose.yml Normal file
View File

@@ -0,0 +1,80 @@
services:
db:
image: mariadb:11
container_name: matomo-db
restart: unless-stopped
environment:
MARIADB_DATABASE: matomo
MARIADB_USER: matomo
MARIADB_PASSWORD: matomo_pw
MARIADB_ROOT_PASSWORD: root_pw
volumes:
- mariadb_data:/var/lib/mysql
healthcheck:
test: ["CMD-SHELL", "mariadb-admin ping -uroot -proot_pw --silent"]
interval: 5s
timeout: 3s
retries: 60
matomo:
image: matomo:5.3.2
container_name: matomo
restart: unless-stopped
depends_on:
db:
condition: service_healthy
ports:
- "${MATOMO_PORT:-8080}:80"
environment:
MATOMO_DATABASE_HOST: db
MATOMO_DATABASE_ADAPTER: mysql
MATOMO_DATABASE_USERNAME: matomo
MATOMO_DATABASE_PASSWORD: matomo_pw
MATOMO_DATABASE_DBNAME: matomo
volumes:
- matomo_data:/var/www/html
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://127.0.0.1/ >/dev/null || exit 1"]
interval: 10s
timeout: 5s
retries: 60
bootstrap:
build:
context: .
dockerfile: Dockerfile
image: matomo-bootstrap:local
container_name: matomo-bootstrap
depends_on:
matomo:
condition: service_healthy
environment:
MATOMO_URL: "http://matomo"
MATOMO_ADMIN_USER: "administrator"
MATOMO_ADMIN_PASSWORD: "AdminSecret123!"
MATOMO_ADMIN_EMAIL: "administrator@example.org"
MATOMO_TOKEN_DESCRIPTION: "docker-compose-bootstrap"
# Installer flow values
MATOMO_SITE_NAME: "Matomo (docker-compose)"
MATOMO_SITE_URL: "http://127.0.0.1:${MATOMO_PORT:-8080}"
MATOMO_TIMEZONE: "Germany - Berlin"
# Optional stability knobs
MATOMO_TIMEOUT: "60"
MATOMO_PLAYWRIGHT_HEADLESS: "1"
MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS: "60000"
MATOMO_PLAYWRIGHT_SLOWMO_MS: "0"
MATOMO_INSTALLER_READY_TIMEOUT_S: "240"
MATOMO_INSTALLER_STEP_TIMEOUT_S: "45"
MATOMO_INSTALLER_STEP_DEADLINE_S: "240"
MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S: "240"
MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S: "180"
MATOMO_INSTALLER_DEBUG_DIR: "/tmp/matomo-bootstrap"
# bootstrap is a one-shot command that prints the token and exits
# if you want to re-run, do: docker compose run --rm bootstrap
restart: "no"
volumes:
mariadb_data:
matomo_data:

37
env.sample Normal file
View File

@@ -0,0 +1,37 @@
# --- REQUIRED ---
MATOMO_URL=http://127.0.0.1:8080
MATOMO_ADMIN_USER=administrator
MATOMO_ADMIN_PASSWORD=AdminSecret123!
MATOMO_ADMIN_EMAIL=administrator@example.org
# --- OPTIONAL ---
# Description for the app-specific token
MATOMO_TOKEN_DESCRIPTION=ansible-bootstrap
# Timeout (seconds)
MATOMO_TIMEOUT=30
# Debug logs to stderr (stdout stays token-only)
# MATOMO_DEBUG=1
# If set, bootstrap will NOT create a new token
# but return this one instead (idempotent runs)
# MATOMO_BOOTSTRAP_TOKEN_AUTH=0123456789abcdef...
# Values used by the recorded installer flow
MATOMO_SITE_NAME=Matomo
MATOMO_SITE_URL=http://127.0.0.1:8080
MATOMO_TIMEZONE=Germany - Berlin
# Playwright knobs (usually not needed)
# MATOMO_PLAYWRIGHT_HEADLESS=1
# MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS=60000
# MATOMO_PLAYWRIGHT_SLOWMO_MS=0
# Installer readiness / step guards
# MATOMO_INSTALLER_READY_TIMEOUT_S=240
# MATOMO_INSTALLER_STEP_TIMEOUT_S=45
# MATOMO_INSTALLER_STEP_DEADLINE_S=240
# MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S=240
# MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S=180
# MATOMO_INSTALLER_DEBUG_DIR=/tmp/matomo-bootstrap

27
flake.lock generated Normal file
View File

@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1766309749,
"narHash": "sha256-3xY8CZ4rSnQ0NqGhMKAy5vgC+2IVK0NoVEzDoOh4DA4=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a6531044f6d0bef691ea18d4d4ce44d0daa6e816",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

136
flake.nix
View File

@@ -1,11 +1,135 @@
{ {
description = "matomo-bootstrap"; description = "matomo-bootstrap";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, nixpkgs }: outputs = { self, nixpkgs }:
let system = "x86_64-linux"; pkgs = import nixpkgs { inherit system; }; let
in { systems = [ "x86_64-linux" "aarch64-linux" ];
devShells.${system}.default = pkgs.mkShell { forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
packages = with pkgs; [ python312 python312Packages.pytest python312Packages.ruff ]; in
}; {
packages = forAllSystems (system:
let
pkgs = import nixpkgs { inherit system; };
python = pkgs.python312;
playwrightDriver = pkgs.playwright-driver;
in
rec {
matomo-bootstrap = python.pkgs.buildPythonApplication {
pname = "matomo-bootstrap";
version = "1.1.11"; # keep in sync with pyproject.toml
pyproject = true;
src = self;
# disable import-check phase (prevents Playwright/installer side effects)
pythonImportsCheck = [ ];
nativeBuildInputs =
(with python.pkgs; [
setuptools
wheel
])
++ [
pkgs.makeWrapper
];
propagatedBuildInputs = with python.pkgs; [
playwright
];
doCheck = false;
# IMPORTANT (Nix):
# Do NOT let Playwright download ubuntu/fhs browser binaries into ~/.cache/ms-playwright.
# Instead, point Playwright to nixpkgs-provided browsers (playwright-driver).
#
# This fixes errors like:
# BrowserType.launch ... headless_shell ENOENT
#
# ...which happens when Playwright downloads a fallback ubuntu build that cannot run on NixOS.
postFixup = ''
wrapProgram "$out/bin/matomo-bootstrap" \
--set PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD 1 \
--set PLAYWRIGHT_BROWSERS_PATH "${playwrightDriver.browsers}"
'';
meta = with pkgs.lib; {
description = "Headless bootstrap tooling for Matomo (installation + API token provisioning)";
homepage = "https://github.com/kevinveenbirkenbach/matomo-bootstrap";
license = licenses.mit;
mainProgram = "matomo-bootstrap";
};
};
default = matomo-bootstrap;
}
);
apps = forAllSystems (system:
let
pkgs = import nixpkgs { inherit system; };
python = pkgs.python312;
playwrightDriver = pkgs.playwright-driver;
pythonPlaywright = python.withPackages (ps: [
ps.playwright
]);
matomo = self.packages.${system}.matomo-bootstrap;
playwright-install = pkgs.writeShellApplication {
name = "matomo-bootstrap-playwright-install";
runtimeInputs = [ pythonPlaywright ];
text = ''
# Nix mode: NO browser downloads.
#
# Playwright upstream "install" downloads ubuntu/fhs browser binaries into ~/.cache/ms-playwright.
# Those binaries often don't run on NixOS, producing ENOENT on launch (missing loader/libs).
#
# We keep this app for backwards-compat (tests/docs call it), but it is intentionally a NO-OP.
#
# IMPORTANT: Do not print anything to stdout (tests expect token-only stdout).
{
echo "Playwright browsers are provided by nixpkgs (playwright-driver)."
echo "Using PLAYWRIGHT_BROWSERS_PATH=${playwrightDriver.browsers}"
echo "Set PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 to prevent downloads."
} 1>&2
exit 0
'';
};
in
{
matomo-bootstrap = {
type = "app";
program = "${matomo}/bin/matomo-bootstrap";
};
matomo-bootstrap-playwright-install = {
type = "app";
program = "${playwright-install}/bin/matomo-bootstrap-playwright-install";
};
default = self.apps.${system}.matomo-bootstrap;
}
);
devShells = forAllSystems (system:
let
pkgs = import nixpkgs { inherit system; };
python = pkgs.python312;
in
{
default = pkgs.mkShell {
packages = with pkgs; [
python
python.pkgs.ruff
];
};
}
);
}; };
} }

View File

@@ -4,22 +4,27 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "matomo-bootstrap" name = "matomo-bootstrap"
version = "0.1.0" version = "1.1.11"
description = "Headless bootstrap tooling for Matomo (installation + API token provisioning)" description = "Headless bootstrap tooling for Matomo (installation + API token provisioning)"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"
authors = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }] authors = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
license = { text = "All rights reserved by Kevin Veen-Birkenbach" } license = "MIT"
urls = { Homepage = "https://github.com/kevinveenbirkenbach/matomo-bootstrap" } urls = { Homepage = "https://github.com/kevinveenbirkenbach/matomo-bootstrap" }
# Playwright is needed at runtime to run the web installer when Matomo is not yet installed. dependencies = ["playwright>=1.46.0,<2"]
dependencies = [
"playwright>=1.40.0", # Provides a stable CLI name for Nix + pip installs:
] [project.scripts]
matomo-bootstrap = "matomo_bootstrap.__main__:main"
[project.optional-dependencies] [project.optional-dependencies]
e2e = [] e2e = []
dev = [
"ruff",
]
[tool.setuptools] [tool.setuptools]
package-dir = { "" = "src" } package-dir = { "" = "src" }

View File

@@ -4,4 +4,5 @@ Headless bootstrap tooling for Matomo:
- readiness checks - readiness checks
- admin/API token provisioning - admin/API token provisioning
""" """
__all__ = [] __all__ = []

View File

@@ -1,16 +1,25 @@
from .cli import parse_args from __future__ import annotations
from .bootstrap import run_bootstrap
from .errors import BootstrapError
import sys import sys
from .cli import parse_args
from .config import config_from_env_and_args
from .errors import BootstrapError
from .service import run
def main() -> int: def main() -> int:
args = parse_args() args = parse_args()
try: try:
token = run_bootstrap(args) config = config_from_env_and_args(args)
token = run(config)
print(token) print(token)
return 0 return 0
except ValueError as exc:
# config validation errors
print(f"[ERROR] {exc}", file=sys.stderr)
return 2
except BootstrapError as exc: except BootstrapError as exc:
print(f"[ERROR] {exc}", file=sys.stderr) print(f"[ERROR] {exc}", file=sys.stderr)
return 2 return 2

View File

@@ -1,106 +0,0 @@
import hashlib
import json
import os
import urllib.error
from .errors import TokenCreationError
from .http import HttpClient
def _md5(text: str) -> str:
return hashlib.md5(text.encode("utf-8")).hexdigest()
def _try_json(body: str) -> object:
try:
return json.loads(body)
except json.JSONDecodeError as exc:
raise TokenCreationError(f"Invalid JSON from Matomo API: {body[:400]}") from exc
def _login_via_logme(client: HttpClient, admin_user: str, admin_password: str, debug: bool) -> None:
"""
Create an authenticated Matomo session (cookie jar) using the classic Login controller.
Matomo accepts the md5 hashed password in the `password` parameter for action=logme.
We rely on urllib's opener to follow redirects and store cookies.
If this ever stops working in a future Matomo version, the next step would be:
- GET the login page, extract CSRF/nonce, then POST the login form.
"""
md5_password = _md5(admin_password)
# Hit the login endpoint; cookies should be set in the client's CookieJar.
# We treat any HTTP response as "we reached the login controller" later API call will tell us if session is valid.
try:
status, body = client.get(
"/index.php",
{
"module": "Login",
"action": "logme",
"login": admin_user,
"password": md5_password,
},
)
if debug:
print(f"[auth] login via logme returned HTTP {status} (body preview: {body[:120]!r})")
except urllib.error.HTTPError as exc:
# Even 4xx/5xx can still set cookies; continue and let the API call validate.
if debug:
try:
err_body = exc.read().decode("utf-8", errors="replace")
except Exception:
err_body = ""
print(f"[auth] login via logme raised HTTPError {exc.code} (body preview: {err_body[:120]!r})")
def create_app_token_via_session(
*,
client: HttpClient,
admin_user: str,
admin_password: str,
description: str,
debug: bool = False,
) -> str:
"""
Create an app-specific token using an authenticated SESSION (cookies),
not via UsersManager.getTokenAuth (removed/not available in Matomo 5.3.x images).
If MATOMO_BOOTSTRAP_TOKEN_AUTH is already set, we return it.
"""
env_token = os.environ.get("MATOMO_BOOTSTRAP_TOKEN_AUTH")
if env_token:
if debug:
print("[auth] Using MATOMO_BOOTSTRAP_TOKEN_AUTH from environment.")
return env_token
# 1) Establish logged-in session
_login_via_logme(client, admin_user, admin_password, debug=debug)
# 2) Use the session cookie to create an app specific token
status, body = client.post(
"/index.php",
{
"module": "API",
"method": "UsersManager.createAppSpecificTokenAuth",
"userLogin": admin_user,
"passwordConfirmation": admin_password,
"description": description,
"format": "json",
},
)
if debug:
print(f"[auth] createAppSpecificTokenAuth HTTP {status} body[:200]={body[:200]!r}")
if status != 200:
raise TokenCreationError(f"HTTP {status} during token creation: {body[:400]}")
data = _try_json(body)
token = data.get("value") if isinstance(data, dict) else None
if not token:
# Matomo may return {"result":"error","message":"..."}.
raise TokenCreationError(f"Unexpected response from token creation: {data}")
return str(token)

View File

@@ -1,37 +0,0 @@
from argparse import Namespace
from .api_tokens import create_app_token_via_session
from .health import assert_matomo_ready
from .http import HttpClient
from .install.web_installer import ensure_installed
def run_bootstrap(args: Namespace) -> str:
# 1) Ensure Matomo is installed (NO-OP if already installed)
ensure_installed(
base_url=args.base_url,
admin_user=args.admin_user,
admin_password=args.admin_password,
admin_email=args.admin_email,
debug=args.debug,
)
# 2) Now the UI/API should be reachable and "installed"
assert_matomo_ready(args.base_url, timeout=args.timeout)
# 3) Create app-specific token via authenticated session (cookie-based)
client = HttpClient(
base_url=args.base_url,
timeout=args.timeout,
debug=args.debug,
)
token = create_app_token_via_session(
client=client,
admin_user=args.admin_user,
admin_password=args.admin_password,
description=args.token_description,
debug=args.debug,
)
return token

View File

@@ -30,23 +30,21 @@ def parse_args() -> argparse.Namespace:
p.add_argument( p.add_argument(
"--token-description", "--token-description",
default=os.environ.get("MATOMO_TOKEN_DESCRIPTION", "matomo-bootstrap"), default=os.environ.get("MATOMO_TOKEN_DESCRIPTION", "matomo-bootstrap"),
help="App token description",
) )
p.add_argument("--timeout", type=int, default=int(os.environ.get("MATOMO_TIMEOUT", "20"))) p.add_argument(
p.add_argument("--debug", action="store_true") "--timeout",
type=int,
default=int(os.environ.get("MATOMO_TIMEOUT", "20")),
help="Network timeout in seconds (or MATOMO_TIMEOUT env)",
)
p.add_argument("--debug", action="store_true", help="Enable debug logs on stderr")
args = p.parse_args() # Optional (future use)
p.add_argument(
"--matomo-container-name",
default=os.environ.get("MATOMO_CONTAINER_NAME"),
help="Matomo container name (optional; also MATOMO_CONTAINER_NAME env)",
)
missing = [] return p.parse_args()
if not args.base_url:
missing.append("--base-url (or MATOMO_URL)")
if not args.admin_user:
missing.append("--admin-user (or MATOMO_ADMIN_USER)")
if not args.admin_password:
missing.append("--admin-password (or MATOMO_ADMIN_PASSWORD)")
if not args.admin_email:
missing.append("--admin-email (or MATOMO_ADMIN_EMAIL)")
if missing:
p.error("missing required values: " + ", ".join(missing))
return args

View File

@@ -0,0 +1,75 @@
from __future__ import annotations
from dataclasses import dataclass
import os
@dataclass(frozen=True)
class Config:
base_url: str
admin_user: str
admin_password: str
admin_email: str
token_description: str = "matomo-bootstrap"
timeout: int = 20
debug: bool = False
matomo_container_name: str | None = (
None # optional, for future console installer usage
)
def config_from_env_and_args(args) -> Config:
"""
Build a Config object from CLI args (preferred) and environment variables (fallback).
"""
base_url = getattr(args, "base_url", None) or os.environ.get("MATOMO_URL")
admin_user = getattr(args, "admin_user", None) or os.environ.get(
"MATOMO_ADMIN_USER"
)
admin_password = getattr(args, "admin_password", None) or os.environ.get(
"MATOMO_ADMIN_PASSWORD"
)
admin_email = getattr(args, "admin_email", None) or os.environ.get(
"MATOMO_ADMIN_EMAIL"
)
token_description = (
getattr(args, "token_description", None)
or os.environ.get("MATOMO_TOKEN_DESCRIPTION")
or "matomo-bootstrap"
)
timeout = int(
getattr(args, "timeout", None) or os.environ.get("MATOMO_TIMEOUT") or "20"
)
debug = bool(getattr(args, "debug", False))
matomo_container_name = (
getattr(args, "matomo_container_name", None)
or os.environ.get("MATOMO_CONTAINER_NAME")
or None
)
missing: list[str] = []
if not base_url:
missing.append("--base-url (or MATOMO_URL)")
if not admin_user:
missing.append("--admin-user (or MATOMO_ADMIN_USER)")
if not admin_password:
missing.append("--admin-password (or MATOMO_ADMIN_PASSWORD)")
if not admin_email:
missing.append("--admin-email (or MATOMO_ADMIN_EMAIL)")
if missing:
raise ValueError("missing required values: " + ", ".join(missing))
return Config(
base_url=str(base_url),
admin_user=str(admin_user),
admin_password=str(admin_password),
admin_email=str(admin_email),
token_description=str(token_description),
timeout=timeout,
debug=debug,
matomo_container_name=matomo_container_name,
)

View File

@@ -1,4 +1,7 @@
from __future__ import annotations
import urllib.request import urllib.request
from .errors import MatomoNotReadyError from .errors import MatomoNotReadyError
@@ -9,5 +12,6 @@ def assert_matomo_ready(base_url: str, timeout: int = 10) -> None:
except Exception as exc: except Exception as exc:
raise MatomoNotReadyError(f"Matomo not reachable: {exc}") from exc raise MatomoNotReadyError(f"Matomo not reachable: {exc}") from exc
if "Matomo" not in html and "piwik" not in html.lower(): lower = html.lower()
if "matomo" not in lower and "piwik" not in lower:
raise MatomoNotReadyError("Matomo UI not detected at base URL") raise MatomoNotReadyError("Matomo UI not detected at base URL")

View File

@@ -1,4 +1,7 @@
from __future__ import annotations
import http.cookiejar import http.cookiejar
import sys
import urllib.error import urllib.error
import urllib.parse import urllib.parse
import urllib.request import urllib.request
@@ -16,15 +19,11 @@ class HttpClient:
urllib.request.HTTPCookieProcessor(self.cookies) urllib.request.HTTPCookieProcessor(self.cookies)
) )
def get(self, path: str, params: Dict[str, str]) -> Tuple[int, str]: def _dbg(self, msg: str) -> None:
qs = urllib.parse.urlencode(params)
url = f"{self.base_url}{path}?{qs}"
if self.debug: if self.debug:
print(f"[HTTP] GET {url}") print(msg, file=sys.stderr)
req = urllib.request.Request(url, method="GET")
def _open(self, req: urllib.request.Request) -> Tuple[int, str]:
try: try:
with self.opener.open(req, timeout=self.timeout) as resp: with self.opener.open(req, timeout=self.timeout) as resp:
body = resp.read().decode("utf-8", errors="replace") body = resp.read().decode("utf-8", errors="replace")
@@ -37,22 +36,25 @@ class HttpClient:
body = str(exc) body = str(exc)
return exc.code, body return exc.code, body
def get(self, path: str, params: Dict[str, str]) -> Tuple[int, str]:
qs = urllib.parse.urlencode(params)
if path == "/":
url = f"{self.base_url}/"
else:
url = f"{self.base_url}{path}"
if qs:
url = f"{url}?{qs}"
self._dbg(f"[HTTP] GET {url}")
req = urllib.request.Request(url, method="GET")
return self._open(req)
def post(self, path: str, data: Dict[str, str]) -> Tuple[int, str]: def post(self, path: str, data: Dict[str, str]) -> Tuple[int, str]:
url = self.base_url + path url = self.base_url + path
encoded = urllib.parse.urlencode(data).encode() encoded = urllib.parse.urlencode(data).encode()
if self.debug: self._dbg(f"[HTTP] POST {url} keys={list(data.keys())}")
print(f"[HTTP] POST {url} keys={list(data.keys())}")
req = urllib.request.Request(url, data=encoded, method="POST") req = urllib.request.Request(url, data=encoded, method="POST")
return self._open(req)
try:
with self.opener.open(req, timeout=self.timeout) as resp:
body = resp.read().decode("utf-8", errors="replace")
return resp.status, body
except urllib.error.HTTPError as exc:
try:
body = exc.read().decode("utf-8", errors="replace")
except Exception:
body = str(exc)
return exc.code, body

View File

@@ -1,113 +0,0 @@
import subprocess
import time
from typing import Optional
MATOMO_ROOT = "/var/www/html"
CONSOLE = f"{MATOMO_ROOT}/console"
def _run(cmd: list[str]) -> subprocess.CompletedProcess:
return subprocess.run(cmd, text=True, capture_output=True)
def _container_state(container_name: str) -> str:
res = _run(["docker", "inspect", "-f", "{{.State.Status}}", container_name])
return (res.stdout or "").strip()
def _wait_container_running(container_name: str, timeout: int = 60) -> None:
last = ""
for _ in range(timeout):
state = _container_state(container_name)
last = state
if state == "running":
return
time.sleep(1)
raise RuntimeError(f"Container '{container_name}' did not become running (last state: {last})")
def _exec(container_name: str, argv: list[str]) -> subprocess.CompletedProcess:
return _run(["docker", "exec", container_name, *argv])
def _sh(container_name: str, script: str) -> subprocess.CompletedProcess:
# Use sh -lc so PATH + cwd behave more like interactive container sessions
return _exec(container_name, ["sh", "-lc", script])
def _console_exists(container_name: str) -> bool:
res = _sh(container_name, f"test -x {CONSOLE} && echo yes || echo no")
return (res.stdout or "").strip() == "yes"
def _is_installed(container_name: str) -> bool:
res = _sh(container_name, f"test -f {MATOMO_ROOT}/config/config.ini.php && echo yes || echo no")
return (res.stdout or "").strip() == "yes"
def _console_list(container_name: str) -> str:
# --no-ansi for stable parsing
res = _sh(container_name, f"{CONSOLE} list --no-ansi 2>/dev/null || true")
return (res.stdout or "") + "\n" + (res.stderr or "")
def _has_command(console_list_output: str, command: str) -> bool:
# cheap but robust enough
return f" {command} " in console_list_output or f"\n{command}\n" in console_list_output or command in console_list_output
def ensure_installed_via_console(
*,
container_name: str,
admin_user: str,
admin_password: str,
admin_email: str,
debug: bool = False,
) -> None:
"""
Ensure Matomo is installed using the container's console if possible.
If no known install command exists, we do NOT guess: we raise with diagnostics.
"""
_wait_container_running(container_name, timeout=90)
if _is_installed(container_name):
if debug:
print("[install] Matomo already installed (config.ini.php exists).")
return
if not _console_exists(container_name):
raise RuntimeError(f"Matomo console not found/executable at {CONSOLE} inside container '{container_name}'.")
listing = _console_list(container_name)
if debug:
print("[install] Matomo console list obtained.")
# Matomo versions differ; we discover what exists.
# Historically: core:install. Your earlier log showed it does NOT exist in 5.3.2 image.
# Therefore we refuse to guess and provide the list in the exception.
if _has_command(listing, "core:install"):
# If this ever exists, use it.
cmd = (
f"{CONSOLE} core:install --no-ansi "
f"--database-host=db "
f"--database-username=matomo "
f"--database-password=matomo_pw "
f"--database-name=matomo "
f"--login={admin_user} "
f"--password={admin_password} "
f"--email={admin_email} "
f"--url=http://localhost "
)
res = _sh(container_name, cmd)
if res.returncode != 0:
raise RuntimeError(f"Matomo CLI install failed.\nexit={res.returncode}\nstdout:\n{res.stdout}\nstderr:\n{res.stderr}")
return
# No install command -> fail with diagnostics (dont keep burning time).
raise RuntimeError(
"Matomo is not installed yet, but no supported CLI install command was found in this image.\n"
"This Matomo image likely expects the web installer.\n"
"\n[console list]\n"
f"{listing}\n"
)

View File

@@ -1,238 +0,0 @@
import os
import time
import urllib.error
import urllib.request
# Optional knobs (mostly for debugging / CI stability)
PLAYWRIGHT_HEADLESS = os.environ.get("MATOMO_PLAYWRIGHT_HEADLESS", "1").strip() not in ("0", "false", "False")
PLAYWRIGHT_SLOWMO_MS = int(os.environ.get("MATOMO_PLAYWRIGHT_SLOWMO_MS", "0"))
PLAYWRIGHT_NAV_TIMEOUT_MS = int(os.environ.get("MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS", "60000"))
# Values used by the installer flow (recorded)
DEFAULT_SITE_NAME = os.environ.get("MATOMO_SITE_NAME", "localhost")
DEFAULT_SITE_URL = os.environ.get("MATOMO_SITE_URL", "http://localhost")
DEFAULT_TIMEZONE = os.environ.get("MATOMO_TIMEZONE", "Germany - Berlin")
DEFAULT_ECOMMERCE = os.environ.get("MATOMO_ECOMMERCE", "Ecommerce enabled")
def wait_http(url: str, timeout: int = 180) -> None:
"""
Consider Matomo 'reachable' as soon as the HTTP server answers - even with 500.
urllib raises HTTPError for 4xx/5xx, so we must treat that as reachability too.
"""
print(f"[install] Waiting for Matomo HTTP at {url} ...")
last_err: Exception | None = None
for i in range(timeout):
try:
with urllib.request.urlopen(url, timeout=2) as resp:
_ = resp.read(128)
print("[install] Matomo HTTP reachable (2xx/3xx).")
return
except urllib.error.HTTPError as exc:
print(f"[install] Matomo HTTP reachable (HTTP {exc.code}).")
return
except Exception as exc:
last_err = exc
if i % 5 == 0:
print(f"[install] still waiting ({i}/{timeout}) … ({type(exc).__name__})")
time.sleep(1)
raise RuntimeError(f"Matomo did not become reachable after {timeout}s: {url} ({last_err})")
def is_installed(url: str) -> bool:
"""
Heuristic:
- installed instances typically render login module links
- installer renders 'installation' wizard content
"""
try:
with urllib.request.urlopen(url, timeout=5) as resp:
html = resp.read().decode(errors="ignore").lower()
return ("module=login" in html) or ("matomo login" in html) or ("matomo/login" in html)
except urllib.error.HTTPError as exc:
try:
html = exc.read().decode(errors="ignore").lower()
return ("module=login" in html) or ("matomo login" in html) or ("matomo/login" in html)
except Exception:
return False
except Exception:
return False
def ensure_installed(
base_url: str,
admin_user: str,
admin_password: str,
admin_email: str,
debug: bool = False,
) -> None:
"""
Ensure Matomo is installed.
NO-OP if already installed.
This implementation ONLY uses the Playwright web installer (recorded flow).
"""
wait_http(base_url)
if is_installed(base_url):
if debug:
print("[install] Matomo already looks installed. Skipping installer.")
return
from playwright.sync_api import sync_playwright
print("[install] Running Matomo web installer via Playwright (recorded flow)...")
with sync_playwright() as p:
browser = p.chromium.launch(
headless=PLAYWRIGHT_HEADLESS,
slow_mo=PLAYWRIGHT_SLOWMO_MS if PLAYWRIGHT_SLOWMO_MS > 0 else None,
)
context = browser.new_context()
page = context.new_page()
page.set_default_navigation_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
page.set_default_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
def _dbg(msg: str) -> None:
if debug:
print(f"[install] {msg}")
def click_next() -> None:
"""
Matomo installer mixes link/button variants and sometimes includes '»'.
We try common variants in a robust order.
"""
candidates = [
("link", "Next »"),
("button", "Next »"),
("link", "Next"),
("button", "Next"),
("link", "Continue"),
("button", "Continue"),
("link", "Proceed"),
("button", "Proceed"),
("link", "Start Installation"),
("button", "Start Installation"),
("link", "Weiter"),
("button", "Weiter"),
("link", "Fortfahren"),
("button", "Fortfahren"),
]
for role, name in candidates:
loc = page.get_by_role(role, name=name)
if loc.count() > 0:
_dbg(f"click_next(): {role} '{name}'")
loc.first.click()
return
# last resort: some pages use same text but different element types
loc = page.get_by_text("Next", exact=False)
if loc.count() > 0:
_dbg("click_next(): fallback text 'Next'")
loc.first.click()
return
raise RuntimeError("Could not find a Next/Continue control in the installer UI.")
# --- Recorded-ish flow, but made variable-based + more stable ---
page.goto(base_url, wait_until="domcontentloaded")
# The first few screens can vary slightly (welcome/system check/db etc.).
# In your recording, you clicked through multiple Next pages without DB input (env already set in container).
# We mimic that: keep clicking "Next" until we see the superuser fields.
#
# Stop condition: superuser login field appears.
def superuser_form_visible() -> bool:
# In your recording, the superuser "Login" field was "#login-0".
return page.locator("#login-0").count() > 0
# Click next until the superuser page shows up (cap to avoid infinite loops).
for _ in range(12):
if superuser_form_visible():
break
click_next()
page.wait_for_load_state("domcontentloaded")
page.wait_for_timeout(200)
else:
raise RuntimeError("Installer did not reach superuser step (login-0 not found).")
# Superuser step
page.locator("#login-0").click()
page.locator("#login-0").fill(admin_user)
page.locator("#password-0").click()
page.locator("#password-0").fill(admin_password)
# Repeat password (some versions have it)
if page.locator("#password_bis-0").count() > 0:
page.locator("#password_bis-0").click()
page.locator("#password_bis-0").fill(admin_password)
page.locator("#email-0").click()
page.locator("#email-0").fill(admin_email)
# Next
page.get_by_role("button", name="Next »").click()
# First website
if page.locator("#siteName-0").count() > 0:
page.locator("#siteName-0").click()
page.locator("#siteName-0").fill(DEFAULT_SITE_NAME)
if page.locator("#url-0").count() > 0:
page.locator("#url-0").click()
page.locator("#url-0").fill(DEFAULT_SITE_URL)
# Timezone dropdown (best-effort)
try:
# recording: page.get_by_role("combobox").first.click() then listbox text
page.get_by_role("combobox").first.click()
page.get_by_role("listbox").get_by_text(DEFAULT_TIMEZONE).click()
except Exception:
_dbg("Timezone selection skipped (not found / changed UI).")
# Ecommerce dropdown (best-effort)
try:
# recording: combobox nth(2)
page.get_by_role("combobox").nth(2).click()
page.get_by_role("listbox").get_by_text(DEFAULT_ECOMMERCE).click()
except Exception:
_dbg("Ecommerce selection skipped (not found / changed UI).")
# Next pages to finish
click_next()
page.wait_for_load_state("domcontentloaded")
# In recording: Next link, then Continue to Matomo button
if page.get_by_role("link", name="Next »").count() > 0:
page.get_by_role("link", name="Next »").click()
if page.get_by_role("button", name="Continue to Matomo »").count() > 0:
page.get_by_role("button", name="Continue to Matomo »").click()
# Optional: login once (not strictly required for token flow, but harmless and matches your recording).
# Some UIs have fancy-icon labels; we follow your recorded selectors best-effort.
try:
user_box = page.get_by_role("textbox", name=" Username or e-mail")
pass_box = page.get_by_role("textbox", name=" Password")
if user_box.count() > 0 and pass_box.count() > 0:
user_box.click()
user_box.fill(admin_user)
pass_box.fill(admin_password)
if page.get_by_role("button", name="Sign in").count() > 0:
page.get_by_role("button", name="Sign in").click()
except Exception:
_dbg("Post-install login skipped (UI differs).")
context.close()
browser.close()
time.sleep(1)
if not is_installed(base_url):
raise RuntimeError("[install] Installer did not reach installed state.")
print("[install] Installation finished.")

View File

@@ -0,0 +1 @@
__all__ = []

View File

@@ -0,0 +1,11 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from ..config import Config
class Installer(ABC):
@abstractmethod
def ensure_installed(self, config: Config) -> None:
raise NotImplementedError

View File

@@ -0,0 +1,988 @@
from __future__ import annotations
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from .base import Installer
from ..config import Config
# Optional knobs (mostly for debugging / CI stability)
PLAYWRIGHT_HEADLESS = os.environ.get("MATOMO_PLAYWRIGHT_HEADLESS", "1").strip() not in (
"0",
"false",
"False",
)
PLAYWRIGHT_SLOWMO_MS = int(os.environ.get("MATOMO_PLAYWRIGHT_SLOWMO_MS", "0"))
PLAYWRIGHT_NAV_TIMEOUT_MS = int(
os.environ.get("MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS", "60000")
)
INSTALLER_READY_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_READY_TIMEOUT_S", "180")
)
INSTALLER_STEP_TIMEOUT_S = int(os.environ.get("MATOMO_INSTALLER_STEP_TIMEOUT_S", "30"))
INSTALLER_STEP_DEADLINE_S = int(
os.environ.get("MATOMO_INSTALLER_STEP_DEADLINE_S", "180")
)
INSTALLER_TABLES_CREATION_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S", "180")
)
INSTALLER_TABLES_ERASE_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S", "120")
)
INSTALLER_DEBUG_DIR = os.environ.get(
"MATOMO_INSTALLER_DEBUG_DIR", "/tmp/matomo-bootstrap"
).rstrip("/")
# Values used by the installer flow (recorded)
DEFAULT_SITE_NAME = os.environ.get("MATOMO_SITE_NAME", "localhost")
DEFAULT_SITE_URL = os.environ.get("MATOMO_SITE_URL", "http://localhost")
DEFAULT_TIMEZONE = os.environ.get("MATOMO_TIMEZONE", "Germany - Berlin")
DEFAULT_ECOMMERCE = os.environ.get("MATOMO_ECOMMERCE", "Ecommerce enabled")
NEXT_BUTTON_CANDIDATES: list[tuple[str, str]] = [
("link", "Next »"),
("button", "Next »"),
("link", "Next"),
("button", "Next"),
("link", "Continue"),
("button", "Continue"),
("link", "Proceed"),
("button", "Proceed"),
("link", "Start Installation"),
("button", "Start Installation"),
("link", "Weiter"),
("button", "Weiter"),
("link", "Fortfahren"),
("button", "Fortfahren"),
]
CONTINUE_TO_MATOMO_CANDIDATES: list[tuple[str, str]] = [
("button", "Continue to Matomo »"),
("button", "Continue to Matomo"),
("link", "Continue to Matomo »"),
("link", "Continue to Matomo"),
]
SUPERUSER_LOGIN_SELECTORS = (
"#login-0",
"#login",
"input[name='login']",
"form#generalsetupform input[name='login']",
)
SUPERUSER_PASSWORD_SELECTORS = (
"#password-0",
"#password",
"input[name='password']",
"form#generalsetupform input[name='password']",
)
SUPERUSER_PASSWORD_REPEAT_SELECTORS = (
"#password_bis-0",
"#password_bis",
"input[name='password_bis']",
"form#generalsetupform input[name='password_bis']",
)
SUPERUSER_EMAIL_SELECTORS = (
"#email-0",
"#email",
"input[name='email']",
"form#generalsetupform input[name='email']",
)
SUPERUSER_SUBMIT_SELECTORS = (
"#submit-0",
"#submit",
"form#generalsetupform button[type='submit']",
"form#generalsetupform input[type='submit']",
)
FIRST_WEBSITE_NAME_SELECTORS = (
"#siteName-0",
"#siteName",
"input[name='siteName']",
"form#websitesetupform input[name='siteName']",
)
FIRST_WEBSITE_URL_SELECTORS = (
"#url-0",
"#url",
"input[name='url']",
"form#websitesetupform input[name='url']",
)
def _log(msg: str) -> None:
# IMPORTANT: logs must not pollute stdout (tests expect only token on stdout)
print(msg, file=sys.stderr)
_TRANSIENT_NAVIGATION_ERROR_SNIPPETS = (
"Execution context was destroyed",
"most likely because of a navigation",
"Cannot find context with specified id",
"Frame was detached",
)
def _is_transient_navigation_error(exc: Exception) -> bool:
msg = str(exc)
return any(snippet in msg for snippet in _TRANSIENT_NAVIGATION_ERROR_SNIPPETS)
def _count_locator(
locator, *, timeout_s: float = 2.0, retry_interval_s: float = 0.1
) -> int:
deadline = time.time() + timeout_s
while True:
try:
return locator.count()
except Exception as exc:
if _is_transient_navigation_error(exc) and time.time() < deadline:
time.sleep(retry_interval_s)
continue
raise
def _page_warnings(page, *, prefix: str = "[install]") -> list[str]:
"""
Detect Matomo installer warnings/errors on the current page.
- Does NOT change any click logic.
- Prints found warnings/errors to stderr (stdout stays clean).
- Returns a de-duplicated list of warning/error texts (empty if none found).
"""
def _safe(s: str | None) -> str:
return (s or "").strip()
# Helpful context (doesn't spam much, but makes failures traceable)
try:
url = page.url
except Exception:
url = "<unknown-url>"
try:
title = page.title()
except Exception:
title = "<unknown-title>"
selectors = [
# your originals
".warning",
".alert.alert-danger",
".alert.alert-warning",
".notification",
".message_container",
# common Matomo / UI patterns seen across versions
"#notificationContainer",
".system-check-error",
".system-check-warning",
".form-errors",
".error",
".errorMessage",
".invalid-feedback",
".help-block.error",
".ui-state-error",
".alert-danger",
".alert-warning",
"[role='alert']",
]
texts: list[str] = []
for sel in selectors:
loc = page.locator(sel)
try:
n = loc.count()
except Exception:
n = 0
if n <= 0:
continue
# collect all matches (not only .first)
for i in range(min(n, 50)): # avoid insane spam if page is weird
try:
t = _safe(loc.nth(i).inner_text())
except Exception:
t = ""
if t:
texts.append(t)
# Also catch HTML5 validation bubbles / inline field errors
# (Sometimes Matomo marks invalid inputs with aria-invalid + sibling text)
try:
invalid = page.locator("[aria-invalid='true']")
n_invalid = invalid.count()
except Exception:
n_invalid = 0
if n_invalid > 0:
texts.append(f"{n_invalid} field(s) marked aria-invalid=true.")
# De-duplicate while preserving order
seen: set[str] = set()
out: list[str] = []
for t in texts:
if t not in seen:
seen.add(t)
out.append(t)
if out:
print(
f"{prefix} page warnings/errors detected @ {url} ({title}):",
file=sys.stderr,
)
for idx, t in enumerate(out, 1):
print(f"{prefix} {idx}) {t}", file=sys.stderr)
return out
def _wait_dom_settled(page) -> None:
try:
page.wait_for_load_state("domcontentloaded")
except Exception:
pass
try:
# Best effort: helps when the UI needs a bit more rendering time.
page.wait_for_load_state("networkidle", timeout=2_000)
except Exception:
pass
page.wait_for_timeout(250)
def _get_step_hint(url: str) -> str:
try:
parsed = urllib.parse.urlparse(url)
qs = urllib.parse.parse_qs(parsed.query)
module = (qs.get("module") or [""])[0]
action = (qs.get("action") or [""])[0]
if module or action:
return f"{module}:{action}"
return parsed.path or url
except Exception:
return url
def _safe_page_snapshot_name() -> str:
return time.strftime("%Y%m%d-%H%M%S")
def _dump_failure_artifacts(page, reason: str) -> None:
os.makedirs(INSTALLER_DEBUG_DIR, exist_ok=True)
stamp = _safe_page_snapshot_name()
base = f"{INSTALLER_DEBUG_DIR}/installer-failure-{stamp}"
screenshot_path = f"{base}.png"
html_path = f"{base}.html"
meta_path = f"{base}.txt"
try:
page.screenshot(path=screenshot_path, full_page=True)
except Exception as exc:
_log(f"[install] Could not write screenshot: {exc}")
screenshot_path = "<unavailable>"
try:
html = page.content()
with open(html_path, "w", encoding="utf-8") as f:
f.write(html)
except Exception as exc:
_log(f"[install] Could not write HTML snapshot: {exc}")
html_path = "<unavailable>"
try:
url = page.url
except Exception:
url = "<unknown-url>"
try:
title = page.title()
except Exception:
title = "<unknown-title>"
try:
with open(meta_path, "w", encoding="utf-8") as f:
f.write(f"reason: {reason}\n")
f.write(f"url: {url}\n")
f.write(f"title: {title}\n")
f.write(f"step_hint: {_get_step_hint(url)}\n")
except Exception as exc:
_log(f"[install] Could not write metadata snapshot: {exc}")
meta_path = "<unavailable>"
_log("[install] Debug artifacts written:")
_log(f"[install] screenshot: {screenshot_path}")
_log(f"[install] html: {html_path}")
_log(f"[install] meta: {meta_path}")
def _first_next_locator(page):
for role, name in NEXT_BUTTON_CANDIDATES:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc) > 0 and loc.first.is_visible():
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Next", exact=False)
try:
if _count_locator(text_loc) > 0 and text_loc.first.is_visible():
return text_loc.first, "text:Next*"
except Exception:
pass
return None, ""
def _first_present_css_locator(page, selectors, *, timeout_s: float = 0.2):
for selector in selectors:
loc = page.locator(selector)
try:
if _count_locator(loc, timeout_s=timeout_s) > 0:
return loc.first, f"css:{selector}"
except Exception:
continue
return None, ""
def _first_continue_to_matomo_locator(page, *, timeout_s: float = 0.2):
for role, name in CONTINUE_TO_MATOMO_CANDIDATES:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc, timeout_s=timeout_s) > 0 and loc.first.is_visible():
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Continue to Matomo", exact=False)
try:
if (
_count_locator(text_loc, timeout_s=timeout_s) > 0
and text_loc.first.is_visible()
):
return text_loc.first, "text:Continue to Matomo*"
except Exception:
pass
return None, ""
def _has_superuser_login_field(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_present_css_locator(
page, SUPERUSER_LOGIN_SELECTORS, timeout_s=timeout_s
)
return loc is not None
def _has_first_website_name_field(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_present_css_locator(
page, FIRST_WEBSITE_NAME_SELECTORS, timeout_s=timeout_s
)
return loc is not None
def _has_continue_to_matomo_action(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_continue_to_matomo_locator(page, timeout_s=timeout_s)
return loc is not None
def _fill_required_input(page, selectors, value: str, *, label: str) -> None:
loc, _ = _first_present_css_locator(page, selectors, timeout_s=1.0)
if loc is None:
raise RuntimeError(
f"Could not locate required installer field '{label}' "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
try:
loc.click(timeout=2_000)
except Exception:
pass
loc.fill(value)
def _fill_optional_input(page, selectors, value: str) -> bool:
loc, _ = _first_present_css_locator(page, selectors, timeout_s=0.5)
if loc is None:
return False
try:
loc.click(timeout=2_000)
except Exception:
pass
loc.fill(value)
return True
def _installer_interactive(page) -> bool:
checks = [
_has_superuser_login_field(page),
_has_first_website_name_field(page),
_has_continue_to_matomo_action(page),
]
loc, _ = _first_next_locator(page)
return any(checks) or loc is not None
def _wait_for_installer_interactive(page, *, timeout_s: int) -> None:
_log(f"[install] Waiting for interactive installer UI (timeout={timeout_s}s)...")
deadline = time.time() + timeout_s
while time.time() < deadline:
_wait_dom_settled(page)
if _installer_interactive(page):
_log("[install] Installer UI looks interactive.")
return
page.wait_for_timeout(300)
raise RuntimeError(
f"Installer UI did not become interactive within {timeout_s}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
def _click_next_with_wait(page, *, timeout_s: int) -> str:
before_url = page.url
before_step = _get_step_hint(before_url)
last_warning_log_at = 0.0
deadline = time.time() + timeout_s
while time.time() < deadline:
loc, label = _first_next_locator(page)
if loc is not None:
try:
loc.click(timeout=2_000)
except Exception:
page.wait_for_timeout(250)
continue
_wait_dom_settled(page)
after_url = page.url
after_step = _get_step_hint(after_url)
_log(
f"[install] Clicked {label}; step {before_step} -> {after_step} "
f"(url {before_url} -> {after_url})"
)
return after_step
_wait_dom_settled(page)
current_url = page.url
current_step = _get_step_hint(current_url)
if current_url != before_url or current_step != before_step:
_log(
"[install] Installer progressed without explicit click; "
f"step {before_step} -> {current_step} "
f"(url {before_url} -> {current_url})"
)
return current_step
# Some installer transitions render the next form asynchronously without
# exposing another "Next" control yet. Treat this as progress.
if _has_superuser_login_field(page, timeout_s=0.2):
_log(
"[install] Superuser form became available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
if _has_first_website_name_field(page, timeout_s=0.2):
_log(
"[install] First website form became available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
if _has_continue_to_matomo_action(page, timeout_s=0.2):
_log(
"[install] Continue-to-Matomo action is available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
now = time.time()
if now - last_warning_log_at >= 5:
_page_warnings(page)
last_warning_log_at = now
page.wait_for_timeout(300)
raise RuntimeError(
"Could not find a Next/Continue control in the installer UI "
f"within {timeout_s}s (url={page.url}, step={_get_step_hint(page.url)})."
)
def _first_erase_tables_locator(page):
css_loc = page.locator("#eraseAllTables")
try:
if _count_locator(css_loc) > 0:
return css_loc.first, "css:#eraseAllTables"
except Exception:
pass
for role, name in [
("link", "Delete the detected tables »"),
("button", "Delete the detected tables »"),
("link", "Delete the detected tables"),
("button", "Delete the detected tables"),
]:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc) > 0:
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Delete the detected tables", exact=False)
try:
if _count_locator(text_loc) > 0:
return text_loc.first, "text:Delete the detected tables*"
except Exception:
pass
return None, ""
def _resolve_tables_creation_conflict(page, *, timeout_s: int) -> bool:
before_url = page.url
before_step = _get_step_hint(before_url)
if "tablesCreation" not in before_step:
return False
loc, label = _first_erase_tables_locator(page)
if loc is None:
return False
_log(
"[install] Detected existing tables during tablesCreation. "
f"Trying cleanup via {label}."
)
def _cleanup_url() -> str | None:
try:
href = page.locator("#eraseAllTables").first.get_attribute("href")
if href:
return urllib.parse.urljoin(page.url, href)
except Exception:
pass
try:
parsed = urllib.parse.urlparse(page.url)
qs = urllib.parse.parse_qs(parsed.query, keep_blank_values=True)
if (qs.get("action") or [""])[0] != "tablesCreation":
return None
qs["deleteTables"] = ["1"]
return urllib.parse.urlunparse(
parsed._replace(query=urllib.parse.urlencode(qs, doseq=True))
)
except Exception:
return None
deadline = time.time() + timeout_s
while time.time() < deadline:
accepted_dialog = False
def _accept_dialog(dialog) -> None:
nonlocal accepted_dialog
accepted_dialog = True
try:
_log(f"[install] Accepting installer dialog: {dialog.message}")
except Exception:
_log("[install] Accepting installer dialog.")
try:
dialog.accept()
except Exception:
pass
page.on("dialog", _accept_dialog)
try:
loc.click(timeout=2_000, force=True)
_wait_dom_settled(page)
except Exception as exc:
_log(f"[install] Cleanup click via {label} failed: {exc}")
cleanup_url = _cleanup_url()
if cleanup_url:
try:
page.goto(cleanup_url, wait_until="domcontentloaded")
_wait_dom_settled(page)
_log(
"[install] Triggered existing-table cleanup via URL fallback: "
f"{cleanup_url}"
)
except Exception as nav_exc:
_log(
"[install] Cleanup URL fallback failed: "
f"{cleanup_url} ({nav_exc})"
)
finally:
page.remove_listener("dialog", _accept_dialog)
if accepted_dialog:
_log("[install] Existing-table cleanup dialog accepted.")
_wait_dom_settled(page)
current_url = page.url
current_step = _get_step_hint(current_url)
if current_url != before_url or current_step != before_step:
_log(
"[install] Existing-table cleanup progressed installer; "
f"step {before_step} -> {current_step} "
f"(url {before_url} -> {current_url})"
)
return True
remaining_loc, _ = _first_erase_tables_locator(page)
if remaining_loc is None:
_log("[install] Existing-table cleanup control is gone.")
return True
loc = remaining_loc
page.wait_for_timeout(500)
raise RuntimeError(
"Detected existing Matomo tables but cleanup did not complete "
f"within {timeout_s}s (url={page.url}, step={_get_step_hint(page.url)})."
)
def wait_http(url: str, timeout: int = 180) -> None:
"""
Consider Matomo 'reachable' as soon as the HTTP server answers - even with 500.
urllib raises HTTPError for 4xx/5xx, so we must treat that as reachability too.
"""
_log(f"[install] Waiting for Matomo HTTP at {url} ...")
last_err: Exception | None = None
for i in range(timeout):
try:
with urllib.request.urlopen(url, timeout=2) as resp:
_ = resp.read(128)
_log("[install] Matomo HTTP reachable (2xx/3xx).")
return
except urllib.error.HTTPError as exc:
_log(f"[install] Matomo HTTP reachable (HTTP {exc.code}).")
return
except Exception as exc:
last_err = exc
if i % 5 == 0:
_log(
f"[install] still waiting ({i}/{timeout}) … ({type(exc).__name__})"
)
time.sleep(1)
raise RuntimeError(
f"Matomo did not become reachable after {timeout}s: {url} ({last_err})"
)
def is_installed(url: str) -> bool:
"""
Heuristic:
- installed instances typically render login module links
- installer renders 'installation' wizard content
"""
try:
with urllib.request.urlopen(url, timeout=5) as resp:
html = resp.read().decode(errors="ignore").lower()
return (
("module=login" in html)
or ("matomo login" in html)
or ("matomo/login" in html)
)
except urllib.error.HTTPError as exc:
try:
html = exc.read().decode(errors="ignore").lower()
return (
("module=login" in html)
or ("matomo login" in html)
or ("matomo/login" in html)
)
except Exception:
return False
except Exception:
return False
class WebInstaller(Installer):
def ensure_installed(self, config: Config) -> None:
"""
Ensure Matomo is installed. NO-OP if already installed.
Uses Playwright to drive the web installer (recorded flow).
"""
base_url = config.base_url
wait_http(base_url)
if is_installed(base_url):
_log("[install] Matomo already looks installed. Skipping installer.")
return
from playwright.sync_api import sync_playwright
_log("[install] Running Matomo web installer via Playwright (recorded flow)...")
with sync_playwright() as p:
browser = p.chromium.launch(
headless=PLAYWRIGHT_HEADLESS,
slow_mo=PLAYWRIGHT_SLOWMO_MS if PLAYWRIGHT_SLOWMO_MS > 0 else None,
)
context = browser.new_context()
page = context.new_page()
page.set_default_navigation_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
page.set_default_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
try:
page.goto(base_url, wait_until="domcontentloaded")
_wait_for_installer_interactive(
page, timeout_s=INSTALLER_READY_TIMEOUT_S
)
_page_warnings(page)
progress_deadline = time.time() + INSTALLER_STEP_DEADLINE_S
while not _has_superuser_login_field(page):
if time.time() >= progress_deadline:
raise RuntimeError(
"Installer did not reach superuser step "
f"within {INSTALLER_STEP_DEADLINE_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
if _resolve_tables_creation_conflict(
page, timeout_s=INSTALLER_TABLES_ERASE_TIMEOUT_S
):
_page_warnings(page)
continue
step_timeout = INSTALLER_STEP_TIMEOUT_S
if "tablesCreation" in _get_step_hint(page.url):
step_timeout = max(
step_timeout, INSTALLER_TABLES_CREATION_TIMEOUT_S
)
_click_next_with_wait(page, timeout_s=step_timeout)
_page_warnings(page)
_fill_required_input(
page,
SUPERUSER_LOGIN_SELECTORS,
config.admin_user,
label="superuser login",
)
_fill_required_input(
page,
SUPERUSER_PASSWORD_SELECTORS,
config.admin_password,
label="superuser password",
)
_fill_optional_input(
page, SUPERUSER_PASSWORD_REPEAT_SELECTORS, config.admin_password
)
_fill_required_input(
page,
SUPERUSER_EMAIL_SELECTORS,
config.admin_email,
label="superuser email",
)
_page_warnings(page)
submitted_superuser = False
try:
submitted_superuser = bool(
page.evaluate(
"""
([user, password, email]) => {
const form = document.querySelector("form#generalsetupform");
if (!form) return false;
const loginInput = form.querySelector("input[name='login']");
const passwordInput = form.querySelector("input[name='password']");
const repeatPasswordInput = form.querySelector("input[name='password_bis']");
const emailInput = form.querySelector("input[name='email']");
if (!loginInput || !passwordInput || !emailInput) return false;
loginInput.value = user;
passwordInput.value = password;
if (repeatPasswordInput) {
repeatPasswordInput.value = password;
}
emailInput.value = email;
if (typeof form.requestSubmit === "function") {
form.requestSubmit();
} else {
form.submit();
}
return true;
}
""",
[
config.admin_user,
config.admin_password,
config.admin_email,
],
)
)
except Exception:
submitted_superuser = False
if submitted_superuser:
_wait_dom_settled(page)
_log("[install] Submitted superuser form via form.requestSubmit().")
else:
submit_loc, submit_label = _first_present_css_locator(
page, SUPERUSER_SUBMIT_SELECTORS, timeout_s=0.5
)
if submit_loc is not None:
submit_loc.click(timeout=2_000)
_wait_dom_settled(page)
_log(
"[install] Submitted superuser form via "
f"{submit_label} fallback."
)
else:
_click_next_with_wait(page, timeout_s=INSTALLER_STEP_TIMEOUT_S)
superuser_progress_deadline = time.time() + INSTALLER_STEP_TIMEOUT_S
while time.time() < superuser_progress_deadline:
_wait_dom_settled(page)
if not _has_superuser_login_field(page):
break
page.wait_for_timeout(300)
if _has_superuser_login_field(page):
_page_warnings(page)
raise RuntimeError(
"Superuser form submit did not progress to first website setup "
f"within {INSTALLER_STEP_TIMEOUT_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
_page_warnings(page)
submitted_first_website = False
try:
submitted_first_website = bool(
page.evaluate(
"""
([siteName, siteUrl, timezoneLabel, ecommerceLabel]) => {
const form = document.querySelector("form#websitesetupform");
if (!form) return false;
const siteNameInput = form.querySelector("input[name='siteName']");
const siteUrlInput = form.querySelector("input[name='url']");
if (!siteNameInput || !siteUrlInput) return false;
siteNameInput.value = siteName;
siteUrlInput.value = siteUrl;
const timezoneSelect = form.querySelector("select[name='timezone']");
if (timezoneSelect) {
const timezoneOption = Array.from(timezoneSelect.options).find(
(opt) => (opt.textContent || "").trim() === timezoneLabel
);
if (timezoneOption) {
timezoneSelect.value = timezoneOption.value;
}
}
const ecommerceSelect = form.querySelector("select[name='ecommerce']");
if (ecommerceSelect) {
const ecommerceOption = Array.from(ecommerceSelect.options).find(
(opt) => (opt.textContent || "").trim() === ecommerceLabel
);
if (ecommerceOption) {
ecommerceSelect.value = ecommerceOption.value;
}
}
if (typeof form.requestSubmit === "function") {
form.requestSubmit();
} else {
form.submit();
}
return true;
}
""",
[
DEFAULT_SITE_NAME,
DEFAULT_SITE_URL,
DEFAULT_TIMEZONE,
DEFAULT_ECOMMERCE,
],
)
)
except Exception:
submitted_first_website = False
if submitted_first_website:
_wait_dom_settled(page)
_log(
"[install] Submitted first website form via form.requestSubmit()."
)
else:
_fill_optional_input(
page, FIRST_WEBSITE_NAME_SELECTORS, DEFAULT_SITE_NAME
)
_fill_optional_input(
page, FIRST_WEBSITE_URL_SELECTORS, DEFAULT_SITE_URL
)
_page_warnings(page)
try:
comboboxes = page.get_by_role("combobox")
if _count_locator(comboboxes) > 0:
comboboxes.first.click(timeout=2_000)
page.get_by_role("listbox").get_by_text(
DEFAULT_TIMEZONE
).click(timeout=2_000)
except Exception:
_log("Timezone selection skipped (not found / changed UI).")
try:
comboboxes = page.get_by_role("combobox")
if _count_locator(comboboxes) > 2:
comboboxes.nth(2).click(timeout=2_000)
page.get_by_role("listbox").get_by_text(
DEFAULT_ECOMMERCE
).click(timeout=2_000)
except Exception:
_log("Ecommerce selection skipped (not found / changed UI).")
_page_warnings(page)
_click_next_with_wait(page, timeout_s=INSTALLER_STEP_TIMEOUT_S)
first_website_progress_deadline = time.time() + INSTALLER_STEP_TIMEOUT_S
while time.time() < first_website_progress_deadline:
_wait_dom_settled(page)
if not _has_first_website_name_field(page):
break
page.wait_for_timeout(300)
if _has_first_website_name_field(page):
_page_warnings(page)
raise RuntimeError(
"First website form submit did not progress to tracking code "
f"within {INSTALLER_STEP_TIMEOUT_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
_page_warnings(page)
if _count_locator(page.get_by_role("link", name="Next »")) > 0:
page.get_by_role("link", name="Next »").click()
_wait_dom_settled(page)
_page_warnings(page)
continue_loc, _ = _first_continue_to_matomo_locator(page)
if continue_loc is not None:
continue_loc.click()
_wait_dom_settled(page)
_page_warnings(page)
page.wait_for_timeout(1_000)
if not is_installed(base_url):
_page_warnings(page)
raise RuntimeError(
"[install] Installer did not reach installed state."
)
except Exception as exc:
_dump_failure_artifacts(page, reason=str(exc))
raise
finally:
context.close()
browser.close()
_log("[install] Installation finished.")

View File

@@ -0,0 +1,125 @@
from __future__ import annotations
import hashlib
import json
import os
import sys
import urllib.error
from .errors import MatomoNotReadyError, TokenCreationError
from .http import HttpClient
def _md5(text: str) -> str:
return hashlib.md5(text.encode("utf-8")).hexdigest()
def _try_json(body: str) -> object:
try:
return json.loads(body)
except json.JSONDecodeError as exc:
raise TokenCreationError(f"Invalid JSON from Matomo API: {body[:400]}") from exc
def _dbg(msg: str, enabled: bool) -> None:
if enabled:
# Keep stdout clean (tests expect only token on stdout).
print(msg, file=sys.stderr)
class MatomoApi:
def __init__(self, *, client: HttpClient, debug: bool = False):
self.client = client
self.debug = debug
def assert_ready(self, timeout: int = 10) -> None:
"""
Minimal readiness check: Matomo UI should be reachable and look like Matomo.
"""
try:
status, body = self.client.get("/", {})
except Exception as exc: # pragma: no cover
raise MatomoNotReadyError(f"Matomo not reachable: {exc}") from exc
_dbg(f"[ready] GET / -> HTTP {status}", self.debug)
html = (body or "").lower()
if "matomo" not in html and "piwik" not in html:
raise MatomoNotReadyError("Matomo UI not detected at base URL")
def login_via_logme(self, admin_user: str, admin_password: str) -> None:
"""
Create an authenticated Matomo session (cookie jar) using Login controller.
Matomo accepts md5 hashed password in `password` parameter for action=logme.
"""
md5_password = _md5(admin_password)
try:
status, body = self.client.get(
"/index.php",
{
"module": "Login",
"action": "logme",
"login": admin_user,
"password": md5_password,
},
)
_dbg(f"[auth] logme HTTP {status} body[:120]={body[:120]!r}", self.debug)
except urllib.error.HTTPError as exc:
# Even 4xx/5xx can still set cookies; continue and let the API call validate.
try:
err_body = exc.read().decode("utf-8", errors="replace")
except Exception:
err_body = ""
_dbg(
f"[auth] logme HTTPError {exc.code} body[:120]={err_body[:120]!r}",
self.debug,
)
def create_app_specific_token(
self,
*,
admin_user: str,
admin_password: str,
description: str,
) -> str:
"""
Create an app-specific token using an authenticated session (cookies),
not UsersManager.getTokenAuth (not available in Matomo 5.3.x images).
"""
env_token = os.environ.get("MATOMO_BOOTSTRAP_TOKEN_AUTH")
if env_token:
_dbg(
"[auth] Using MATOMO_BOOTSTRAP_TOKEN_AUTH from environment.", self.debug
)
return env_token
self.login_via_logme(admin_user, admin_password)
status, body = self.client.post(
"/index.php",
{
"module": "API",
"method": "UsersManager.createAppSpecificTokenAuth",
"userLogin": admin_user,
"passwordConfirmation": admin_password,
"description": description,
"format": "json",
},
)
_dbg(
f"[auth] createAppSpecificTokenAuth HTTP {status} body[:200]={body[:200]!r}",
self.debug,
)
if status != 200:
raise TokenCreationError(
f"HTTP {status} during token creation: {body[:400]}"
)
data = _try_json(body)
token = data.get("value") if isinstance(data, dict) else None
if not token:
raise TokenCreationError(f"Unexpected response from token creation: {data}")
return str(token)

View File

@@ -0,0 +1,33 @@
from __future__ import annotations
from .config import Config
from .http import HttpClient
from .matomo_api import MatomoApi
from .installers.web import WebInstaller
def run(config: Config) -> str:
"""
Orchestrate:
1) Ensure Matomo is installed (NO-OP if installed)
2) Ensure Matomo is reachable/ready
3) Create an app-specific token using an authenticated session
"""
installer = WebInstaller()
installer.ensure_installed(config)
client = HttpClient(
base_url=config.base_url,
timeout=config.timeout,
debug=config.debug,
)
api = MatomoApi(client=client, debug=config.debug)
api.assert_ready(timeout=config.timeout)
token = api.create_app_specific_token(
admin_user=config.admin_user,
admin_password=config.admin_password,
description=config.token_description,
)
return token

View File

@@ -27,3 +27,50 @@ services:
MATOMO_DATABASE_USERNAME: matomo MATOMO_DATABASE_USERNAME: matomo
MATOMO_DATABASE_PASSWORD: matomo_pw MATOMO_DATABASE_PASSWORD: matomo_pw
MATOMO_DATABASE_DBNAME: matomo MATOMO_DATABASE_DBNAME: matomo
nix:
image: nixos/nix:latest
container_name: e2e-nix
depends_on:
matomo:
condition: service_started
# Run as root to avoid /nix big-lock permission issues
user: "0:0"
working_dir: /work
volumes:
# Project root as flake
- ../../:/work:ro
# Nix store (removed by docker compose down -v)
- e2e_nix_store:/nix
# HOME/XDG for nix + playwright
- e2e_nix_home:/tmp/home
environment:
NIX_CONFIG: "experimental-features = nix-command flakes"
TERM: "xterm"
HOME: "/tmp/home"
USER: "root"
LOGNAME: "root"
XDG_CACHE_HOME: "/tmp/home/.cache"
XDG_CONFIG_HOME: "/tmp/home/.config"
XDG_DATA_HOME: "/tmp/home/.local/share"
MATOMO_SITE_NAME: "Matomo E2E"
MATOMO_SITE_URL: "http://127.0.0.1:8080"
MATOMO_TIMEZONE: "Germany - Berlin"
command: >
sh -lc "mkdir -p /tmp/home/.cache /tmp/home/.config /tmp/home/.local/share;
tail -f /dev/null"
# Allow access to host-published Matomo port
network_mode: host
volumes:
e2e_nix_store:
e2e_nix_home:

View File

@@ -1,6 +1,7 @@
import json import json
import os import os
import subprocess import subprocess
import sys
import unittest import unittest
import urllib.request import urllib.request
@@ -14,7 +15,7 @@ ADMIN_EMAIL = os.environ.get("MATOMO_ADMIN_EMAIL", "administrator@example.org")
class TestMatomoBootstrapE2E(unittest.TestCase): class TestMatomoBootstrapE2E(unittest.TestCase):
def test_bootstrap_creates_api_token(self) -> None: def test_bootstrap_creates_api_token(self) -> None:
cmd = [ cmd = [
"python3", sys.executable,
"-m", "-m",
"matomo_bootstrap", "matomo_bootstrap",
"--base-url", "--base-url",
@@ -29,12 +30,18 @@ class TestMatomoBootstrapE2E(unittest.TestCase):
"e2e-test-token", "e2e-test-token",
] ]
token = subprocess.check_output( token = (
cmd, subprocess.check_output(
env={**os.environ, "PYTHONPATH": "src"}, cmd,
).decode().strip() env={**os.environ, "PYTHONPATH": "src"},
)
.decode()
.strip()
)
self.assertRegex(token, r"^[a-f0-9]{32,64}$", f"Expected token_auth, got: {token}") self.assertRegex(
token, r"^[a-f0-9]{32,64}$", f"Expected token_auth, got: {token}"
)
api_url = ( api_url = (
f"{MATOMO_URL}/index.php" f"{MATOMO_URL}/index.php"

View File

@@ -0,0 +1,107 @@
import os
import re
import subprocess
import textwrap
import unittest
MATOMO_URL = os.environ.get("MATOMO_URL", "http://127.0.0.1:8080")
ADMIN_USER = os.environ.get("MATOMO_ADMIN_USER", "administrator")
ADMIN_PASSWORD = os.environ.get("MATOMO_ADMIN_PASSWORD", "AdminSecret123!")
ADMIN_EMAIL = os.environ.get("MATOMO_ADMIN_EMAIL", "administrator@example.org")
TOKEN_RE = re.compile(r"^[a-f0-9]{32,64}$")
class TestMatomoBootstrapE2ENix(unittest.TestCase):
def test_bootstrap_creates_api_token_via_nix(self) -> None:
script = textwrap.dedent(
f"""\
set -eux
export NIX_CONFIG='experimental-features = nix-command flakes'
export TERM='xterm'
# Improve CI resilience for slow installer pages.
export MATOMO_INSTALLER_READY_TIMEOUT_S="${{MATOMO_INSTALLER_READY_TIMEOUT_S:-240}}"
export MATOMO_INSTALLER_STEP_TIMEOUT_S="${{MATOMO_INSTALLER_STEP_TIMEOUT_S:-45}}"
export MATOMO_INSTALLER_STEP_DEADLINE_S="${{MATOMO_INSTALLER_STEP_DEADLINE_S:-240}}"
export MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S="${{MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S:-240}}"
export MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S="${{MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S:-180}}"
export MATOMO_INSTALLER_DEBUG_DIR="${{MATOMO_INSTALLER_DEBUG_DIR:-/tmp/matomo-bootstrap}}"
# Make sure we have a writable HOME (compose already sets HOME=/tmp/home)
mkdir -p "$HOME" "$HOME/.cache" "$HOME/.config" "$HOME/.local/share"
# IMPORTANT:
# Nix flakes read the local repo as git+file:///work.
# Git refuses if the repo is not owned by the current user (root in the container).
# Mark it as safe explicitly.
git config --global --add safe.directory /work
# Preflight checks to surface "command not executable" failures (exit 126) clearly.
playwright_app="$(nix eval --raw .#apps.x86_64-linux.matomo-bootstrap-playwright-install.program)"
bootstrap_app="$(nix eval --raw .#apps.x86_64-linux.matomo-bootstrap.program)"
if [ -e "$playwright_app" ]; then
test -x "$playwright_app"
fi
if [ -e "$bootstrap_app" ]; then
test -x "$bootstrap_app"
fi
# 1) Install Playwright Chromium (cached in the container environment)
nix run --no-write-lock-file -L .#matomo-bootstrap-playwright-install
# 2) Run bootstrap (must print ONLY token)
nix run --no-write-lock-file -L .#matomo-bootstrap -- \\
--base-url '{MATOMO_URL}' \\
--admin-user '{ADMIN_USER}' \\
--admin-password '{ADMIN_PASSWORD}' \\
--admin-email '{ADMIN_EMAIL}' \\
--token-description 'e2e-test-token-nix'
"""
)
cmd = [
"docker",
"compose",
"-f",
"tests/e2e/docker-compose.yml",
# Use `run` instead of `exec` to avoid runtime-specific
# `/etc/group` lookup issues seen with nix image + compose exec.
"run",
"--rm",
"--no-deps",
"-T",
"nix",
"sh",
"-lc",
script,
]
result = subprocess.run(
cmd,
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
if result.returncode != 0:
self.fail(
"nix bootstrap command failed\n"
f"exit={result.returncode}\n"
f"stdout:\n{result.stdout}\n"
f"stderr:\n{result.stderr}"
)
stdout_lines = [
line.strip() for line in result.stdout.splitlines() if line.strip()
]
token = stdout_lines[-1] if stdout_lines else ""
self.assertRegex(
token,
TOKEN_RE,
f"Expected token on last stdout line, got stdout={result.stdout!r}",
)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,222 @@
import json
import os
import subprocess
import time
import unittest
import urllib.request
COMPOSE_FILE = os.environ.get("MATOMO_STACK_COMPOSE_FILE", "docker-compose.yml")
# Pick a non-default port to avoid collisions with other CI stacks that use 8080
MATOMO_PORT = os.environ.get("MATOMO_PORT", "18080")
MATOMO_HOST_URL = os.environ.get("MATOMO_STACK_URL", f"http://127.0.0.1:{MATOMO_PORT}")
# How long we wait for Matomo HTTP to respond at all (seconds)
WAIT_TIMEOUT_SECONDS = int(os.environ.get("MATOMO_STACK_WAIT_TIMEOUT", "180"))
def _run(
cmd: list[str],
*,
check: bool = True,
extra_env: dict[str, str] | None = None,
) -> subprocess.CompletedProcess:
return subprocess.run(
cmd,
check=check,
env={**os.environ, **(extra_env or {})},
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
def _compose_cmd(*args: str) -> list[str]:
return ["docker", "compose", "-f", COMPOSE_FILE, *args]
def _wait_for_http_any_status(url: str, timeout_s: int) -> None:
"""
Consider the service "up" once the HTTP server answers anything.
urllib raises HTTPError on 4xx/5xx, but that's still "reachable".
"""
deadline = time.time() + timeout_s
last_exc: Exception | None = None
while time.time() < deadline:
try:
with urllib.request.urlopen(url, timeout=2) as resp:
_ = resp.read(64)
return
except Exception as exc: # includes HTTPError
last_exc = exc
time.sleep(1)
raise RuntimeError(f"Matomo did not become reachable at {url} ({last_exc})")
def _extract_service_block(compose_config: str, service_name: str) -> str:
lines = compose_config.splitlines()
marker = f" {service_name}:"
start = -1
for idx, line in enumerate(lines):
if line == marker:
start = idx
break
if start < 0:
raise AssertionError(
f"service block not found in compose config: {service_name}"
)
end = len(lines)
for idx in range(start + 1, len(lines)):
line = lines[idx]
if line.startswith(" ") and not line.startswith(" "):
end = idx
break
return "\n".join(lines[start:end])
class TestRootDockerComposeStack(unittest.TestCase):
"""
E2E test for repository root docker-compose.yml:
1) docker compose down -v
2) docker compose build bootstrap
3) docker compose up -d db matomo
4) wait for Matomo HTTP on host port (default 8080, overridden here)
5) docker compose run --rm bootstrap -> token on stdout
6) validate token via Matomo API call
7) docker compose down -v (cleanup)
"""
def setUp(self) -> None:
# Always start from a clean slate (also clears volumes)
_run(
_compose_cmd("down", "-v"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
def tearDown(self) -> None:
# Cleanup even if assertions fail
_run(
_compose_cmd("down", "-v"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
def test_root_docker_compose_yml_stack_bootstraps_and_token_works(self) -> None:
# Build bootstrap image from Dockerfile (as defined in docker-compose.yml)
build = _run(
_compose_cmd("build", "bootstrap"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(
build.returncode,
0,
f"compose build failed\nstdout:\n{build.stdout}\nstderr:\n{build.stderr}",
)
# Start db + matomo (bootstrap is one-shot and started via "run")
up = _run(
_compose_cmd("up", "-d", "db", "matomo"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(
up.returncode,
0,
f"compose up failed\nstdout:\n{up.stdout}\nstderr:\n{up.stderr}",
)
# Wait until Matomo answers on the published port
_wait_for_http_any_status(MATOMO_HOST_URL + "/", WAIT_TIMEOUT_SECONDS)
# Run bootstrap: it should print ONLY the token to stdout.
# Retry once because first-run installer startup can be flaky on slow CI.
boot_attempts: list[subprocess.CompletedProcess] = []
for _ in range(2):
boot = _run(
_compose_cmd("run", "--rm", "bootstrap"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
boot_attempts.append(boot)
if boot.returncode == 0:
break
time.sleep(5)
if boot.returncode != 0:
matomo_logs = _run(
_compose_cmd("logs", "--no-color", "--tail=200", "matomo"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
attempts_dump = "\n\n".join(
[
(
f"[attempt {i}] rc={attempt.returncode}\n"
f"stdout:\n{attempt.stdout}\n"
f"stderr:\n{attempt.stderr}"
)
for i, attempt in enumerate(boot_attempts, 1)
]
)
self.fail(
"bootstrap container failed after retry.\n"
f"{attempts_dump}\n\n"
f"[matomo logs]\n{matomo_logs.stdout}\n{matomo_logs.stderr}"
)
token = (boot.stdout or "").strip()
self.assertRegex(
token,
r"^[a-f0-9]{32,64}$",
f"Expected token_auth on stdout, got stdout={boot.stdout!r} stderr={boot.stderr!r}",
)
# Verify token works against Matomo API
api_url = (
f"{MATOMO_HOST_URL}/index.php"
f"?module=API&method=SitesManager.getSitesWithAtLeastViewAccess"
f"&format=json&token_auth={token}"
)
with urllib.request.urlopen(api_url, timeout=10) as resp:
data = json.loads(resp.read().decode("utf-8", errors="replace"))
self.assertIsInstance(data, list)
class TestRootDockerComposeDefinition(unittest.TestCase):
def test_bootstrap_service_waits_for_healthy_matomo_and_has_readiness_knobs(
self,
) -> None:
cfg = _run(
_compose_cmd("config"),
check=True,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(cfg.returncode, 0, cfg.stderr)
bootstrap_block = _extract_service_block(cfg.stdout, "bootstrap")
self.assertIn("depends_on:", bootstrap_block)
self.assertIn("matomo:", bootstrap_block)
self.assertIn("condition: service_healthy", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_READY_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_STEP_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_STEP_DEADLINE_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S:", bootstrap_block)
matomo_block = _extract_service_block(cfg.stdout, "matomo")
self.assertIn("healthcheck:", matomo_block)
self.assertIn("curl -fsS http://127.0.0.1/ >/dev/null || exit 1", matomo_block)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,173 @@
import unittest
from matomo_bootstrap.installers.web import _click_next_with_wait, _count_locator
class _FlakyLocator:
def __init__(self, outcomes):
self._outcomes = list(outcomes)
self.calls = 0
def count(self) -> int:
self.calls += 1
outcome = self._outcomes.pop(0)
if isinstance(outcome, Exception):
raise outcome
return int(outcome)
class _StaticLocator:
def __init__(self, page, selector: str):
self._page = page
self._selector = selector
def count(self) -> int:
if self._selector == "#login-0":
return 1 if self._page.login_visible else 0
if self._selector == "#siteName-0":
return 0
return 0
@property
def first(self):
return self
def is_visible(self) -> bool:
return self.count() > 0
class _RoleLocator:
def __init__(self, count_value: int):
self._count_value = count_value
def count(self) -> int:
return self._count_value
@property
def first(self):
return self
def is_visible(self) -> bool:
return self._count_value > 0
class _NameOnlyStaticLocator:
def __init__(self, page, selector: str):
self._page = page
self._selector = selector
def count(self) -> int:
if self._selector == "input[name='login']":
return 1 if self._page.login_visible else 0
if self._selector == "input[name='siteName']":
return 0
return 0
@property
def first(self):
return self
def is_visible(self) -> bool:
return self.count() > 0
class _NoNextButLoginAppearsPage:
def __init__(self):
self.url = "http://matomo/index.php?action=setupSuperUser&module=Installation"
self.login_visible = False
self._wait_calls = 0
def locator(self, selector: str):
return _StaticLocator(self, selector)
def get_by_role(self, role: str, name: str):
return _RoleLocator(0)
def get_by_text(self, *_args, **_kwargs):
return _RoleLocator(0)
def title(self) -> str:
return "setupSuperUser"
def wait_for_load_state(self, *_args, **_kwargs):
return None
def wait_for_timeout(self, *_args, **_kwargs):
self._wait_calls += 1
if self._wait_calls >= 1:
self.login_visible = True
class _NoNextButNamedLoginAppearsPage:
def __init__(self):
self.url = "http://matomo/index.php?action=setupSuperUser&module=Installation"
self.login_visible = False
self._wait_calls = 0
def locator(self, selector: str):
return _NameOnlyStaticLocator(self, selector)
def get_by_role(self, role: str, name: str):
return _RoleLocator(0)
def get_by_text(self, *_args, **_kwargs):
return _RoleLocator(0)
def title(self) -> str:
return "setupSuperUser"
def wait_for_load_state(self, *_args, **_kwargs):
return None
def wait_for_timeout(self, *_args, **_kwargs):
self._wait_calls += 1
if self._wait_calls >= 1:
self.login_visible = True
class TestWebInstallerLocatorCountIntegration(unittest.TestCase):
def test_retries_transient_navigation_error(self) -> None:
locator = _FlakyLocator(
[
RuntimeError(
"Locator.count: Execution context was destroyed, most likely because of a navigation"
),
RuntimeError(
"Locator.count: Execution context was destroyed, most likely because of a navigation"
),
1,
]
)
result = _count_locator(locator, timeout_s=0.5, retry_interval_s=0.0)
self.assertEqual(result, 1)
self.assertEqual(locator.calls, 3)
def test_raises_non_transient_error_without_retry(self) -> None:
locator = _FlakyLocator([RuntimeError("Locator is not attached to DOM")])
with self.assertRaises(RuntimeError):
_count_locator(locator, timeout_s=0.5, retry_interval_s=0.0)
self.assertEqual(locator.calls, 1)
def test_click_next_wait_treats_login_form_as_progress(self) -> None:
page = _NoNextButLoginAppearsPage()
step = _click_next_with_wait(page, timeout_s=1)
self.assertEqual(step, "Installation:setupSuperUser")
self.assertTrue(page.login_visible)
def test_click_next_wait_treats_named_login_form_as_progress(self) -> None:
page = _NoNextButNamedLoginAppearsPage()
step = _click_next_with_wait(page, timeout_s=1)
self.assertEqual(step, "Installation:setupSuperUser")
self.assertTrue(page.login_visible)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,128 @@
import io
import unittest
from contextlib import redirect_stderr
# Import the function under test.
# This keeps the test close to real integration behavior without requiring Playwright.
from matomo_bootstrap.installers.web import _page_warnings
class _FakeLocatorNth:
def __init__(self, text: str):
self._text = text
def inner_text(self) -> str:
return self._text
class _FakeLocator:
def __init__(self, texts: list[str]):
self._texts = texts
def count(self) -> int:
return len(self._texts)
def nth(self, i: int) -> _FakeLocatorNth:
return _FakeLocatorNth(self._texts[i])
class _FakePage:
"""
Minimal Playwright-like page stub:
- locator(selector) -> object with count() / nth(i).inner_text()
- url, title()
"""
def __init__(self, *, url: str, title: str, selector_texts: dict[str, list[str]]):
self.url = url
self._title = title
self._selector_texts = selector_texts
def title(self) -> str:
return self._title
def locator(self, selector: str) -> _FakeLocator:
return _FakeLocator(self._selector_texts.get(selector, []))
class TestWebInstallerWarningsIntegration(unittest.TestCase):
def test_detects_bootstrap_alert_warning_block(self) -> None:
"""
Matomo installer commonly renders validation errors like:
<div class="alert alert-warning"> ... <ul><li>...</li></ul> ... </div>
We must detect and print those messages to stderr.
"""
page = _FakePage(
url="http://matomo/index.php?action=setupSuperUser&module=Installation",
title="Superuser",
selector_texts={
# The key selector from the observed DOM
".alert.alert-warning": [
"Please fix the following errors:\n"
"Password required\n"
"Password (repeat) required\n"
"The email doesn't have a valid format."
],
},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
# Function must return the warning text
self.assertEqual(len(warnings), 1)
self.assertIn("Please fix the following errors:", warnings[0])
self.assertIn("The email doesn't have a valid format.", warnings[0])
# And it must print it to stderr (stdout must remain token-only in the app)
out = buf.getvalue()
self.assertIn("[install] page warnings/errors detected", out)
self.assertIn("Superuser", out)
self.assertIn("The email doesn't have a valid format.", out)
def test_deduplicates_repeated_warning_blocks(self) -> None:
"""
Some Matomo versions repeat the same alert in multiple containers.
We must return/log each unique text only once.
"""
repeated = (
"Please fix the following errors:\nThe email doesn't have a valid format."
)
page = _FakePage(
url="http://matomo/index.php?action=setupSuperUser&module=Installation",
title="Superuser",
selector_texts={
".alert.alert-warning": [repeated, repeated],
},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
self.assertEqual(warnings, [repeated])
out = buf.getvalue()
# Only a single numbered entry should be printed
self.assertIn("[install] 1) ", out)
self.assertNotIn("[install] 2) ", out)
def test_no_output_when_no_warnings(self) -> None:
page = _FakePage(
url="http://matomo/",
title="Welcome",
selector_texts={},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
self.assertEqual(warnings, [])
self.assertEqual(buf.getvalue(), "")
if __name__ == "__main__":
unittest.main()