27 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
7ecb26cc92 Release version 1.1.11
Some checks failed
ci / tests (push) Has been cancelled
ci / detect-release (push) Has been cancelled
ci / publish-image (push) Has been cancelled
ci / tag-stable (push) Has been cancelled
2026-02-14 15:03:35 +01:00
Kevin Veen-Birkenbach
5d4a2d59db Fix installer selectors for setupSuperUser UI variants 2026-02-14 15:00:34 +01:00
Kevin Veen-Birkenbach
1847c14b63 Release version 1.1.10 2026-02-14 12:19:43 +01:00
Kevin Veen-Birkenbach
29e812f584 fix(installer): harden navigation races in setupSuperUser flow 2026-02-14 12:14:06 +01:00
Kevin Veen-Birkenbach
1c8de40a05 Release version 1.1.9 2026-02-14 11:04:30 +01:00
Kevin Veen-Birkenbach
4bfa7433f4 ci: run on all branches but release only from main via git tag detection
- Trigger CI on push for all branches and on pull_request
- Detect SemVer release tags (vX.Y.Z) via git tag --points-at
- Run publish-image and stable-tag only for tagged commits on main
- Pass version_tag and sha to reusable workflows
- Prevent tag pushes from triggering additional workflows

https://chatgpt.com/share/e/699044d3-c1d8-8013-a40d-974d1fc69974
2026-02-14 11:02:58 +01:00
Kevin Veen-Birkenbach
00c012e553 Release version 1.1.8 2026-02-14 10:53:40 +01:00
Kevin Veen-Birkenbach
1bebeb8abc ci: make ci.yml the single coordinator workflow
- Trigger ci on push and pull_request only
- Convert publish-image and stable-tag to reusable workflows (workflow_call)
- Add detect-release job for strict SemVer tag detection (vX.Y.Z)
- Run tests first, then publish image, then move stable tag
- Remove direct tag/push triggers from publish-image and stable-tag

https://chatgpt.com/share/e/699044d3-c1d8-8013-a40d-974d1fc69974
2026-02-14 10:47:56 +01:00
Kevin Veen-Birkenbach
01d1626cf2 ci(docker): publish image without leading 'v' in version tag
Strip the leading 'v' from git tags (e.g. v1.1.7 -> 1.1.7)
when pushing Docker images to GHCR.

https://chatgpt.com/share/e/699041b9-8610-8013-8d04-fbef93b10d3c
2026-02-14 10:34:41 +01:00
Kevin Veen-Birkenbach
5bbe78b272 Release version 1.1.7 2026-02-14 05:43:32 +01:00
Kevin Veen-Birkenbach
865d5155d5 Lint 2026-02-14 05:39:01 +01:00
Kevin Veen-Birkenbach
209037cd64 Harden compose installer timeouts and e2e stack diagnostics 2026-02-14 05:37:29 +01:00
Kevin Veen-Birkenbach
ba2d84b6cb Fix nix e2e to use compose run instead of exec 2026-02-14 05:08:12 +01:00
Kevin Veen-Birkenbach
4f5c41753f Release version 1.1.6 2026-02-14 04:54:10 +01:00
Kevin Veen-Birkenbach
aac01810a1 Add tables-step timeout env knobs to compose, docs, and e2e 2026-02-14 04:52:26 +01:00
Kevin Veen-Birkenbach
fb42167b89 Release version 1.1.5 2026-02-14 04:49:49 +01:00
Kevin Veen-Birkenbach
7836dbacf9 Harden web installer flow for nix e2e 2026-02-14 04:45:00 +01:00
Kevin Veen-Birkenbach
d380b1493c Lint 2026-02-14 03:47:37 +01:00
Kevin Veen-Birkenbach
e81c5262b0 Release version 1.1.4
Some checks failed
ci / tests (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2026-02-13 18:32:19 +01:00
Kevin Veen-Birkenbach
f35ea04d66 Resolve merge conflict and preserve installer hardening 2026-02-13 18:24:34 +01:00
Kevin Veen-Birkenbach
37a17b536d Harden installer readiness and fix e2e healthcheck 2026-02-13 15:20:18 +01:00
Kevin Veen-Birkenbach
c80fdf8d01 Release version 1.1.3 2026-02-12 01:27:16 +01:00
Kevin Veen-Birkenbach
276833bd16 fix(matomo-bootstrap): increase Playwright step wait from 200ms to 1000ms to reduce CI flakiness
Increase page.wait_for_timeout from 200ms to 1000ms in WebInstaller to mitigate race conditions during Matomo web installation steps in slower CI environments.

https://chatgpt.com/share/698d1e2f-1f40-800f-92bc-10a736358b40
2026-02-12 01:26:09 +01:00
Kevin Veen-Birkenbach
9e267ec83f Added github sponsor buttons
Some checks failed
ci / tests (push) Has been cancelled
2026-01-02 13:00:00 +01:00
Kevin Veen-Birkenbach
20274985bc Release version 1.1.2
Some checks failed
ci / tests (push) Has been cancelled
publish-image / build-and-push (push) Has been cancelled
Stable Tag / test (push) Has been cancelled
Stable Tag / tag-stable (push) Has been cancelled
2025-12-24 17:28:48 +01:00
Kevin Veen-Birkenbach
cf473d4f3f Ruff formated
Some checks failed
ci / tests (push) Has been cancelled
2025-12-24 17:25:07 +01:00
Kevin Veen-Birkenbach
84323bd2aa test: add integration tests for installer warning detection
Some checks failed
ci / tests (push) Has been cancelled
- add make target test-integration and run it in reusable CI workflow
- add integration unittest covering _page_warnings stderr output + deduplication
- surface Matomo installer warnings during Playwright flow (stderr only)

https://chatgpt.com/share/694c1371-365c-800f-bdf8-ede2e850e648
2025-12-24 17:22:50 +01:00
17 changed files with 1551 additions and 172 deletions

View File

@@ -1,11 +1,76 @@
name: ci
on:
pull_request:
pull_request: {}
push:
branches:
- main
- "**"
permissions:
contents: write
packages: write
jobs:
tests:
uses: ./.github/workflows/reusable-test.yml
with:
python-version: "3.12"
matomo-token-description: "ci-token"
detect-release:
# Only consider releases on main branch pushes (not PRs, not other branches)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
outputs:
is_semver_tag: ${{ steps.detect.outputs.is_semver_tag }}
version_tag: ${{ steps.detect.outputs.version_tag }}
steps:
- name: Checkout (full history for tags)
uses: actions/checkout@v4
with:
fetch-depth: 0
- id: detect
shell: bash
run: |
set -euo pipefail
git fetch --tags --force
# Tags that point to the current commit
TAGS="$(git tag --points-at "$GITHUB_SHA" || true)"
# Pick the first strict SemVer tag: vX.Y.Z
VERSION_TAG="$(echo "$TAGS" | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -n1 || true)"
if [ -n "$VERSION_TAG" ]; then
echo "is_semver_tag=true" >> "$GITHUB_OUTPUT"
echo "version_tag=$VERSION_TAG" >> "$GITHUB_OUTPUT"
echo "Release tag detected on this commit: $VERSION_TAG"
else
echo "is_semver_tag=false" >> "$GITHUB_OUTPUT"
echo "version_tag=" >> "$GITHUB_OUTPUT"
echo "No SemVer tag on this commit."
fi
publish-image:
# Only on main, and only if detect-release found a SemVer tag on this commit
if: needs.detect-release.outputs.is_semver_tag == 'true'
needs: [tests, detect-release]
uses: ./.github/workflows/publish-image.yml
with:
version_tag: ${{ needs.detect-release.outputs.version_tag }}
sha: ${{ github.sha }}
permissions:
contents: read
packages: write
tag-stable:
# Only after tests + publish succeeded
if: needs.detect-release.outputs.is_semver_tag == 'true'
needs: [tests, detect-release, publish-image]
uses: ./.github/workflows/stable-tag.yml
with:
version_tag: ${{ needs.detect-release.outputs.version_tag }}
sha: ${{ github.sha }}
permissions:
contents: write

View File

@@ -1,32 +1,28 @@
name: publish-image
on:
push:
tags:
- "v*.*.*"
workflow_run:
workflows: ["Stable Tag"] # MUST match stable-tag.yml -> name: Stable Tag
types: [completed]
workflow_call:
inputs:
version_tag:
type: string
required: true
sha:
type: string
required: true
jobs:
build-and-push:
if: |
(github.event_name == 'push') ||
(github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success')
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
- name: Checkout (exact commit)
uses: actions/checkout@v4
with:
# push: checks out the tag ref
# workflow_run: checks out the exact commit that the Stable Tag workflow ran on
ref: ${{ github.event_name == 'workflow_run' && github.event.workflow_run.head_sha || github.ref }}
fetch-depth: 0
ref: ${{ inputs.sha }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -47,13 +43,9 @@ jobs:
run: |
set -euo pipefail
IMAGE="ghcr.io/${{ github.repository }}"
if [ "${{ github.event_name }}" = "push" ]; then
TAG="${{ github.ref_name }}" # e.g. v1.1.0
RAW_TAG="${{ inputs.version_tag }}" # e.g. v1.1.8
TAG="${RAW_TAG#v}" # -> 1.1.8
echo "tags=$IMAGE:$TAG,$IMAGE:latest" >> "$GITHUB_OUTPUT"
else
echo "tags=$IMAGE:stable" >> "$GITHUB_OUTPUT"
fi
- name: Build and push
uses: docker/build-push-action@v6

View File

@@ -52,6 +52,21 @@ jobs:
ruff check .
ruff format --check .
integration:
runs-on: ubuntu-latest
needs: lint
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
- name: Integration tests
run: make test-integration
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30

View File

@@ -1,24 +1,21 @@
name: Stable Tag
on:
push:
tags:
- "v*"
workflow_call:
inputs:
version_tag:
type: string
required: true
sha:
type: string
required: true
permissions:
contents: write
jobs:
test:
uses: ./.github/workflows/reusable-test.yml
with:
python-version: "3.12"
matomo-token-description: "stable-ci-token"
tag-stable:
runs-on: ubuntu-latest
needs: test
if: startsWith(github.ref, 'refs/tags/v')
steps:
- name: Checkout (full history for tags)
@@ -26,17 +23,17 @@ jobs:
with:
fetch-depth: 0
- name: Move stable tag to this version tag commit
- name: Move stable tag to the release commit
shell: bash
run: |
set -euo pipefail
echo "Triggered by tag: ${GITHUB_REF_NAME}"
echo "Commit: ${GITHUB_SHA}"
echo "Release tag: ${{ inputs.version_tag }}"
echo "Commit: ${{ inputs.sha }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --tags --force
git tag -fa stable -m "stable -> ${GITHUB_REF_NAME} (${GITHUB_SHA})" "${GITHUB_SHA}"
git tag -fa stable -m "stable -> ${{ inputs.version_tag }} (${{ inputs.sha }})" "${{ inputs.sha }}"
git push --force origin stable

View File

@@ -1,3 +1,53 @@
## [1.1.11] - 2026-02-14
* This release improves matomo-bootstrap installer resilience by adding robust setupSuperUser field and button detection to prevent intermittent bootstrap failures.
## [1.1.10] - 2026-02-14
* This release fixes a reproducible Playwright navigation race in the Matomo installer (setupSuperUser), hardens the Next/Continue flow, and adds integration tests for transient locator errors and progress detection without a visible Next button.
## [1.1.9] - 2026-02-14
* Reworked CI to run on all branches while restricting Docker image publishing and stable tagging to tagged commits on main, using git-based SemVer detection.
## [1.1.8] - 2026-02-14
* Refactored CI to use a single coordinator workflow with strict SemVer-based release gating and adjusted Docker image publishing to strip the leading v from version tags.
## [1.1.7] - 2026-02-14
* Harden compose installer timeouts and e2e stack diagnostics
## [1.1.6] - 2026-02-14
* Add installer table-step timeout env vars (MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S, MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S) to compose/docs and e2e checks.
## [1.1.5] - 2026-02-14
* Harden web installer flow for nix e2e
## [1.1.4] - 2026-02-13
* This release hardens Matomo bootstrap by adding installer UI readiness waits/retries.
## [1.1.3] - 2026-02-12
* Increase Playwright step wait from 200ms to 1000ms to improve CI stability during Matomo installation.
## [1.1.2] - 2025-12-24
* **Improved error visibility during Matomo installation**: When the setup fails (for example due to an invalid admin email or missing required fields), the installer now **prints the actual Matomo error messages to the logs**, instead of failing with a generic error.
## [1.1.1] - 2025-12-24
* Improved Docker image publishing: automatic `vX.Y.Z`, `latest`, and `stable` tags for releases.

View File

@@ -34,6 +34,7 @@ COMPOSE_STACK := docker compose -f $(COMPOSE_STACK_FILE)
.PHONY: help \
venv deps-e2e playwright-install e2e-up e2e-install e2e-test e2e-down e2e logs clean \
test-integration \
image-build image-run image-shell image-push image-clean \
stack-up stack-down stack-logs stack-ps stack-bootstrap stack-rebootstrap stack-clean stack-reset
@@ -49,6 +50,7 @@ help:
@echo " e2e Full cycle: up → install → test → down"
@echo " logs Show Matomo logs (E2E compose)"
@echo " clean Stop E2E containers + remove venv"
@echo " test-integration Run integration tests (unittest)"
@echo ""
@echo "Container image targets:"
@echo " image-build Build matomo-bootstrap container image"
@@ -76,8 +78,46 @@ help:
# ----------------------------
venv:
@test -x "$(VENV_PY)" || ($(PYTHON) -m venv $(VENV_DIR))
@$(VENV_PIP) -q install -U pip setuptools wheel >/dev/null
@set -e; \
if [ ! -d "$(VENV_DIR)" ]; then \
echo "Creating $(VENV_DIR) ..."; \
$(PYTHON) -m venv "$(VENV_DIR)"; \
fi; \
if ! [ -x "$(VENV_PY)" ] || ! "$(VENV_PY)" -V >/dev/null 2>&1; then \
echo "Repairing $(VENV_PY) symlink ..."; \
fix_target=""; \
for cand in "$(VENV_DIR)/bin/python3.14" "$(VENV_DIR)/bin/python3.13" "$(VENV_DIR)/bin/python3.12" "$(VENV_DIR)/bin/python3.11" "$(VENV_DIR)/bin/python3.10"; do \
if [ -x "$$cand" ]; then \
fix_target="$$(basename "$$cand")"; \
break; \
fi; \
done; \
if [ -z "$$fix_target" ] && [ -x "$(VENV_PIP)" ]; then \
shebang="$$(head -n1 "$(VENV_PIP)" | sed 's/^#!//')"; \
if [ -n "$$shebang" ] && [ -x "$$shebang" ]; then \
fix_target="$$(basename "$$shebang")"; \
fi; \
fi; \
if [ -n "$$fix_target" ] && [ -x "$(VENV_DIR)/bin/$$fix_target" ]; then \
ln -sfn "$$fix_target" "$(VENV_PY)"; \
ln -sfn "$$fix_target" "$(VENV_DIR)/bin/python3"; \
fi; \
fi; \
if ! [ -x "$(VENV_PIP)" ] || ! "$(VENV_PIP)" --version >/dev/null 2>&1; then \
echo "Repairing pip via ensurepip ..."; \
"$(VENV_PY)" -m ensurepip --upgrade >/dev/null 2>&1 || true; \
fi; \
if ! [ -x "$(VENV_PY)" ] || ! "$(VENV_PY)" -V >/dev/null 2>&1; then \
echo "ERROR: Could not repair $(VENV_PY) in existing $(VENV_DIR)."; \
echo "Run 'make clean' once or remove $(VENV_DIR) manually."; \
exit 2; \
fi; \
if ! [ -x "$(VENV_PIP)" ] || ! "$(VENV_PIP)" --version >/dev/null 2>&1; then \
echo "ERROR: Could not repair $(VENV_PIP) in existing $(VENV_DIR)."; \
echo "Run 'make clean' once or remove $(VENV_DIR) manually."; \
exit 2; \
fi; \
"$(VENV_PIP)" -q install -U pip setuptools wheel >/dev/null
deps-e2e: venv
@$(VENV_PIP) install -e ".[e2e]"
@@ -129,6 +169,13 @@ logs:
clean: e2e-down
rm -rf $(VENV_DIR)
# ----------------------------
# Integration tests
# ----------------------------
test-integration:
PYTHONPATH=src $(PYTHON) -m unittest discover -s tests/integration -v
# ----------------------------
# Container image workflow
# ----------------------------

View File

@@ -1,4 +1,6 @@
# matomo-bootstrap
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
Headless bootstrap tooling for **Matomo**. Automates **first-time installation** and **API token provisioning** for fresh Matomo instances.
---
@@ -170,7 +172,7 @@ services:
volumes:
- matomo_data:/var/www/html
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/ >/dev/null || exit 1"]
test: ["CMD-SHELL", "curl -fsS http://127.0.0.1/ >/dev/null || exit 1"]
interval: 10s
timeout: 5s
retries: 60
@@ -207,10 +209,16 @@ services:
MATOMO_TIMEZONE: "Germany - Berlin"
# Optional stability knobs
MATOMO_TIMEOUT: "30"
MATOMO_TIMEOUT: "60"
MATOMO_PLAYWRIGHT_HEADLESS: "1"
MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS: "60000"
MATOMO_PLAYWRIGHT_SLOWMO_MS: "0"
MATOMO_INSTALLER_READY_TIMEOUT_S: "240"
MATOMO_INSTALLER_STEP_TIMEOUT_S: "45"
MATOMO_INSTALLER_STEP_DEADLINE_S: "240"
MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S: "240"
MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S: "180"
MATOMO_INSTALLER_DEBUG_DIR: "/tmp/matomo-bootstrap"
restart: "no"
@@ -267,6 +275,8 @@ matomo-bootstrap
2. **Installation (if needed)**
* uses a recorded Playwright flow to complete the Matomo web installer
* waits until installer controls are interactive before clicking next steps
* writes screenshot/HTML debug artifacts on installer failure
3. **Authentication**
* logs in using Matomos `Login.logme` controller (cookie session)

View File

@@ -34,7 +34,7 @@ services:
volumes:
- matomo_data:/var/www/html
healthcheck:
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/ >/dev/null || exit 1"]
test: ["CMD-SHELL", "curl -fsS http://127.0.0.1/ >/dev/null || exit 1"]
interval: 10s
timeout: 5s
retries: 60
@@ -47,7 +47,7 @@ services:
container_name: matomo-bootstrap
depends_on:
matomo:
condition: service_started
condition: service_healthy
environment:
MATOMO_URL: "http://matomo"
MATOMO_ADMIN_USER: "administrator"
@@ -61,10 +61,16 @@ services:
MATOMO_TIMEZONE: "Germany - Berlin"
# Optional stability knobs
MATOMO_TIMEOUT: "30"
MATOMO_TIMEOUT: "60"
MATOMO_PLAYWRIGHT_HEADLESS: "1"
MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS: "60000"
MATOMO_PLAYWRIGHT_SLOWMO_MS: "0"
MATOMO_INSTALLER_READY_TIMEOUT_S: "240"
MATOMO_INSTALLER_STEP_TIMEOUT_S: "45"
MATOMO_INSTALLER_STEP_DEADLINE_S: "240"
MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S: "240"
MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S: "180"
MATOMO_INSTALLER_DEBUG_DIR: "/tmp/matomo-bootstrap"
# bootstrap is a one-shot command that prints the token and exits
# if you want to re-run, do: docker compose run --rm bootstrap
restart: "no"

View File

@@ -27,3 +27,11 @@ MATOMO_TIMEZONE=Germany - Berlin
# MATOMO_PLAYWRIGHT_HEADLESS=1
# MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS=60000
# MATOMO_PLAYWRIGHT_SLOWMO_MS=0
# Installer readiness / step guards
# MATOMO_INSTALLER_READY_TIMEOUT_S=240
# MATOMO_INSTALLER_STEP_TIMEOUT_S=45
# MATOMO_INSTALLER_STEP_DEADLINE_S=240
# MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S=240
# MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S=180
# MATOMO_INSTALLER_DEBUG_DIR=/tmp/matomo-bootstrap

View File

@@ -20,7 +20,7 @@
rec {
matomo-bootstrap = python.pkgs.buildPythonApplication {
pname = "matomo-bootstrap";
version = "1.1.1"; # keep in sync with pyproject.toml
version = "1.1.11"; # keep in sync with pyproject.toml
pyproject = true;
src = self;

View File

@@ -4,12 +4,12 @@ build-backend = "setuptools.build_meta"
[project]
name = "matomo-bootstrap"
version = "1.1.1"
version = "1.1.11"
description = "Headless bootstrap tooling for Matomo (installation + API token provisioning)"
readme = "README.md"
requires-python = ">=3.10"
authors = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
license = { text = "MIT" }
license = "MIT"
urls = { Homepage = "https://github.com/kevinveenbirkenbach/matomo-bootstrap" }
dependencies = ["playwright>=1.46.0,<2"]

View File

@@ -4,6 +4,7 @@ import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from .base import Installer
@@ -20,6 +21,22 @@ PLAYWRIGHT_SLOWMO_MS = int(os.environ.get("MATOMO_PLAYWRIGHT_SLOWMO_MS", "0"))
PLAYWRIGHT_NAV_TIMEOUT_MS = int(
os.environ.get("MATOMO_PLAYWRIGHT_NAV_TIMEOUT_MS", "60000")
)
INSTALLER_READY_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_READY_TIMEOUT_S", "180")
)
INSTALLER_STEP_TIMEOUT_S = int(os.environ.get("MATOMO_INSTALLER_STEP_TIMEOUT_S", "30"))
INSTALLER_STEP_DEADLINE_S = int(
os.environ.get("MATOMO_INSTALLER_STEP_DEADLINE_S", "180")
)
INSTALLER_TABLES_CREATION_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S", "180")
)
INSTALLER_TABLES_ERASE_TIMEOUT_S = int(
os.environ.get("MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S", "120")
)
INSTALLER_DEBUG_DIR = os.environ.get(
"MATOMO_INSTALLER_DEBUG_DIR", "/tmp/matomo-bootstrap"
).rstrip("/")
# Values used by the installer flow (recorded)
DEFAULT_SITE_NAME = os.environ.get("MATOMO_SITE_NAME", "localhost")
@@ -27,12 +44,602 @@ DEFAULT_SITE_URL = os.environ.get("MATOMO_SITE_URL", "http://localhost")
DEFAULT_TIMEZONE = os.environ.get("MATOMO_TIMEZONE", "Germany - Berlin")
DEFAULT_ECOMMERCE = os.environ.get("MATOMO_ECOMMERCE", "Ecommerce enabled")
NEXT_BUTTON_CANDIDATES: list[tuple[str, str]] = [
("link", "Next »"),
("button", "Next »"),
("link", "Next"),
("button", "Next"),
("link", "Continue"),
("button", "Continue"),
("link", "Proceed"),
("button", "Proceed"),
("link", "Start Installation"),
("button", "Start Installation"),
("link", "Weiter"),
("button", "Weiter"),
("link", "Fortfahren"),
("button", "Fortfahren"),
]
CONTINUE_TO_MATOMO_CANDIDATES: list[tuple[str, str]] = [
("button", "Continue to Matomo »"),
("button", "Continue to Matomo"),
("link", "Continue to Matomo »"),
("link", "Continue to Matomo"),
]
SUPERUSER_LOGIN_SELECTORS = (
"#login-0",
"#login",
"input[name='login']",
"form#generalsetupform input[name='login']",
)
SUPERUSER_PASSWORD_SELECTORS = (
"#password-0",
"#password",
"input[name='password']",
"form#generalsetupform input[name='password']",
)
SUPERUSER_PASSWORD_REPEAT_SELECTORS = (
"#password_bis-0",
"#password_bis",
"input[name='password_bis']",
"form#generalsetupform input[name='password_bis']",
)
SUPERUSER_EMAIL_SELECTORS = (
"#email-0",
"#email",
"input[name='email']",
"form#generalsetupform input[name='email']",
)
SUPERUSER_SUBMIT_SELECTORS = (
"#submit-0",
"#submit",
"form#generalsetupform button[type='submit']",
"form#generalsetupform input[type='submit']",
)
FIRST_WEBSITE_NAME_SELECTORS = (
"#siteName-0",
"#siteName",
"input[name='siteName']",
"form#websitesetupform input[name='siteName']",
)
FIRST_WEBSITE_URL_SELECTORS = (
"#url-0",
"#url",
"input[name='url']",
"form#websitesetupform input[name='url']",
)
def _log(msg: str) -> None:
# IMPORTANT: logs must not pollute stdout (tests expect only token on stdout)
print(msg, file=sys.stderr)
_TRANSIENT_NAVIGATION_ERROR_SNIPPETS = (
"Execution context was destroyed",
"most likely because of a navigation",
"Cannot find context with specified id",
"Frame was detached",
)
def _is_transient_navigation_error(exc: Exception) -> bool:
msg = str(exc)
return any(snippet in msg for snippet in _TRANSIENT_NAVIGATION_ERROR_SNIPPETS)
def _count_locator(
locator, *, timeout_s: float = 2.0, retry_interval_s: float = 0.1
) -> int:
deadline = time.time() + timeout_s
while True:
try:
return locator.count()
except Exception as exc:
if _is_transient_navigation_error(exc) and time.time() < deadline:
time.sleep(retry_interval_s)
continue
raise
def _page_warnings(page, *, prefix: str = "[install]") -> list[str]:
"""
Detect Matomo installer warnings/errors on the current page.
- Does NOT change any click logic.
- Prints found warnings/errors to stderr (stdout stays clean).
- Returns a de-duplicated list of warning/error texts (empty if none found).
"""
def _safe(s: str | None) -> str:
return (s or "").strip()
# Helpful context (doesn't spam much, but makes failures traceable)
try:
url = page.url
except Exception:
url = "<unknown-url>"
try:
title = page.title()
except Exception:
title = "<unknown-title>"
selectors = [
# your originals
".warning",
".alert.alert-danger",
".alert.alert-warning",
".notification",
".message_container",
# common Matomo / UI patterns seen across versions
"#notificationContainer",
".system-check-error",
".system-check-warning",
".form-errors",
".error",
".errorMessage",
".invalid-feedback",
".help-block.error",
".ui-state-error",
".alert-danger",
".alert-warning",
"[role='alert']",
]
texts: list[str] = []
for sel in selectors:
loc = page.locator(sel)
try:
n = loc.count()
except Exception:
n = 0
if n <= 0:
continue
# collect all matches (not only .first)
for i in range(min(n, 50)): # avoid insane spam if page is weird
try:
t = _safe(loc.nth(i).inner_text())
except Exception:
t = ""
if t:
texts.append(t)
# Also catch HTML5 validation bubbles / inline field errors
# (Sometimes Matomo marks invalid inputs with aria-invalid + sibling text)
try:
invalid = page.locator("[aria-invalid='true']")
n_invalid = invalid.count()
except Exception:
n_invalid = 0
if n_invalid > 0:
texts.append(f"{n_invalid} field(s) marked aria-invalid=true.")
# De-duplicate while preserving order
seen: set[str] = set()
out: list[str] = []
for t in texts:
if t not in seen:
seen.add(t)
out.append(t)
if out:
print(
f"{prefix} page warnings/errors detected @ {url} ({title}):",
file=sys.stderr,
)
for idx, t in enumerate(out, 1):
print(f"{prefix} {idx}) {t}", file=sys.stderr)
return out
def _wait_dom_settled(page) -> None:
try:
page.wait_for_load_state("domcontentloaded")
except Exception:
pass
try:
# Best effort: helps when the UI needs a bit more rendering time.
page.wait_for_load_state("networkidle", timeout=2_000)
except Exception:
pass
page.wait_for_timeout(250)
def _get_step_hint(url: str) -> str:
try:
parsed = urllib.parse.urlparse(url)
qs = urllib.parse.parse_qs(parsed.query)
module = (qs.get("module") or [""])[0]
action = (qs.get("action") or [""])[0]
if module or action:
return f"{module}:{action}"
return parsed.path or url
except Exception:
return url
def _safe_page_snapshot_name() -> str:
return time.strftime("%Y%m%d-%H%M%S")
def _dump_failure_artifacts(page, reason: str) -> None:
os.makedirs(INSTALLER_DEBUG_DIR, exist_ok=True)
stamp = _safe_page_snapshot_name()
base = f"{INSTALLER_DEBUG_DIR}/installer-failure-{stamp}"
screenshot_path = f"{base}.png"
html_path = f"{base}.html"
meta_path = f"{base}.txt"
try:
page.screenshot(path=screenshot_path, full_page=True)
except Exception as exc:
_log(f"[install] Could not write screenshot: {exc}")
screenshot_path = "<unavailable>"
try:
html = page.content()
with open(html_path, "w", encoding="utf-8") as f:
f.write(html)
except Exception as exc:
_log(f"[install] Could not write HTML snapshot: {exc}")
html_path = "<unavailable>"
try:
url = page.url
except Exception:
url = "<unknown-url>"
try:
title = page.title()
except Exception:
title = "<unknown-title>"
try:
with open(meta_path, "w", encoding="utf-8") as f:
f.write(f"reason: {reason}\n")
f.write(f"url: {url}\n")
f.write(f"title: {title}\n")
f.write(f"step_hint: {_get_step_hint(url)}\n")
except Exception as exc:
_log(f"[install] Could not write metadata snapshot: {exc}")
meta_path = "<unavailable>"
_log("[install] Debug artifacts written:")
_log(f"[install] screenshot: {screenshot_path}")
_log(f"[install] html: {html_path}")
_log(f"[install] meta: {meta_path}")
def _first_next_locator(page):
for role, name in NEXT_BUTTON_CANDIDATES:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc) > 0 and loc.first.is_visible():
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Next", exact=False)
try:
if _count_locator(text_loc) > 0 and text_loc.first.is_visible():
return text_loc.first, "text:Next*"
except Exception:
pass
return None, ""
def _first_present_css_locator(page, selectors, *, timeout_s: float = 0.2):
for selector in selectors:
loc = page.locator(selector)
try:
if _count_locator(loc, timeout_s=timeout_s) > 0:
return loc.first, f"css:{selector}"
except Exception:
continue
return None, ""
def _first_continue_to_matomo_locator(page, *, timeout_s: float = 0.2):
for role, name in CONTINUE_TO_MATOMO_CANDIDATES:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc, timeout_s=timeout_s) > 0 and loc.first.is_visible():
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Continue to Matomo", exact=False)
try:
if (
_count_locator(text_loc, timeout_s=timeout_s) > 0
and text_loc.first.is_visible()
):
return text_loc.first, "text:Continue to Matomo*"
except Exception:
pass
return None, ""
def _has_superuser_login_field(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_present_css_locator(
page, SUPERUSER_LOGIN_SELECTORS, timeout_s=timeout_s
)
return loc is not None
def _has_first_website_name_field(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_present_css_locator(
page, FIRST_WEBSITE_NAME_SELECTORS, timeout_s=timeout_s
)
return loc is not None
def _has_continue_to_matomo_action(page, *, timeout_s: float = 0.2) -> bool:
loc, _ = _first_continue_to_matomo_locator(page, timeout_s=timeout_s)
return loc is not None
def _fill_required_input(page, selectors, value: str, *, label: str) -> None:
loc, _ = _first_present_css_locator(page, selectors, timeout_s=1.0)
if loc is None:
raise RuntimeError(
f"Could not locate required installer field '{label}' "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
try:
loc.click(timeout=2_000)
except Exception:
pass
loc.fill(value)
def _fill_optional_input(page, selectors, value: str) -> bool:
loc, _ = _first_present_css_locator(page, selectors, timeout_s=0.5)
if loc is None:
return False
try:
loc.click(timeout=2_000)
except Exception:
pass
loc.fill(value)
return True
def _installer_interactive(page) -> bool:
checks = [
_has_superuser_login_field(page),
_has_first_website_name_field(page),
_has_continue_to_matomo_action(page),
]
loc, _ = _first_next_locator(page)
return any(checks) or loc is not None
def _wait_for_installer_interactive(page, *, timeout_s: int) -> None:
_log(f"[install] Waiting for interactive installer UI (timeout={timeout_s}s)...")
deadline = time.time() + timeout_s
while time.time() < deadline:
_wait_dom_settled(page)
if _installer_interactive(page):
_log("[install] Installer UI looks interactive.")
return
page.wait_for_timeout(300)
raise RuntimeError(
f"Installer UI did not become interactive within {timeout_s}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
def _click_next_with_wait(page, *, timeout_s: int) -> str:
before_url = page.url
before_step = _get_step_hint(before_url)
last_warning_log_at = 0.0
deadline = time.time() + timeout_s
while time.time() < deadline:
loc, label = _first_next_locator(page)
if loc is not None:
try:
loc.click(timeout=2_000)
except Exception:
page.wait_for_timeout(250)
continue
_wait_dom_settled(page)
after_url = page.url
after_step = _get_step_hint(after_url)
_log(
f"[install] Clicked {label}; step {before_step} -> {after_step} "
f"(url {before_url} -> {after_url})"
)
return after_step
_wait_dom_settled(page)
current_url = page.url
current_step = _get_step_hint(current_url)
if current_url != before_url or current_step != before_step:
_log(
"[install] Installer progressed without explicit click; "
f"step {before_step} -> {current_step} "
f"(url {before_url} -> {current_url})"
)
return current_step
# Some installer transitions render the next form asynchronously without
# exposing another "Next" control yet. Treat this as progress.
if _has_superuser_login_field(page, timeout_s=0.2):
_log(
"[install] Superuser form became available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
if _has_first_website_name_field(page, timeout_s=0.2):
_log(
"[install] First website form became available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
if _has_continue_to_matomo_action(page, timeout_s=0.2):
_log(
"[install] Continue-to-Matomo action is available without explicit click; "
f"staying on step {current_step} (url {current_url})"
)
return current_step
now = time.time()
if now - last_warning_log_at >= 5:
_page_warnings(page)
last_warning_log_at = now
page.wait_for_timeout(300)
raise RuntimeError(
"Could not find a Next/Continue control in the installer UI "
f"within {timeout_s}s (url={page.url}, step={_get_step_hint(page.url)})."
)
def _first_erase_tables_locator(page):
css_loc = page.locator("#eraseAllTables")
try:
if _count_locator(css_loc) > 0:
return css_loc.first, "css:#eraseAllTables"
except Exception:
pass
for role, name in [
("link", "Delete the detected tables »"),
("button", "Delete the detected tables »"),
("link", "Delete the detected tables"),
("button", "Delete the detected tables"),
]:
loc = page.get_by_role(role, name=name)
try:
if _count_locator(loc) > 0:
return loc.first, f"{role}:{name}"
except Exception:
continue
text_loc = page.get_by_text("Delete the detected tables", exact=False)
try:
if _count_locator(text_loc) > 0:
return text_loc.first, "text:Delete the detected tables*"
except Exception:
pass
return None, ""
def _resolve_tables_creation_conflict(page, *, timeout_s: int) -> bool:
before_url = page.url
before_step = _get_step_hint(before_url)
if "tablesCreation" not in before_step:
return False
loc, label = _first_erase_tables_locator(page)
if loc is None:
return False
_log(
"[install] Detected existing tables during tablesCreation. "
f"Trying cleanup via {label}."
)
def _cleanup_url() -> str | None:
try:
href = page.locator("#eraseAllTables").first.get_attribute("href")
if href:
return urllib.parse.urljoin(page.url, href)
except Exception:
pass
try:
parsed = urllib.parse.urlparse(page.url)
qs = urllib.parse.parse_qs(parsed.query, keep_blank_values=True)
if (qs.get("action") or [""])[0] != "tablesCreation":
return None
qs["deleteTables"] = ["1"]
return urllib.parse.urlunparse(
parsed._replace(query=urllib.parse.urlencode(qs, doseq=True))
)
except Exception:
return None
deadline = time.time() + timeout_s
while time.time() < deadline:
accepted_dialog = False
def _accept_dialog(dialog) -> None:
nonlocal accepted_dialog
accepted_dialog = True
try:
_log(f"[install] Accepting installer dialog: {dialog.message}")
except Exception:
_log("[install] Accepting installer dialog.")
try:
dialog.accept()
except Exception:
pass
page.on("dialog", _accept_dialog)
try:
loc.click(timeout=2_000, force=True)
_wait_dom_settled(page)
except Exception as exc:
_log(f"[install] Cleanup click via {label} failed: {exc}")
cleanup_url = _cleanup_url()
if cleanup_url:
try:
page.goto(cleanup_url, wait_until="domcontentloaded")
_wait_dom_settled(page)
_log(
"[install] Triggered existing-table cleanup via URL fallback: "
f"{cleanup_url}"
)
except Exception as nav_exc:
_log(
"[install] Cleanup URL fallback failed: "
f"{cleanup_url} ({nav_exc})"
)
finally:
page.remove_listener("dialog", _accept_dialog)
if accepted_dialog:
_log("[install] Existing-table cleanup dialog accepted.")
_wait_dom_settled(page)
current_url = page.url
current_step = _get_step_hint(current_url)
if current_url != before_url or current_step != before_step:
_log(
"[install] Existing-table cleanup progressed installer; "
f"step {before_step} -> {current_step} "
f"(url {before_url} -> {current_url})"
)
return True
remaining_loc, _ = _first_erase_tables_locator(page)
if remaining_loc is None:
_log("[install] Existing-table cleanup control is gone.")
return True
loc = remaining_loc
page.wait_for_timeout(500)
raise RuntimeError(
"Detected existing Matomo tables but cleanup did not complete "
f"within {timeout_s}s (url={page.url}, step={_get_step_hint(page.url)})."
)
def wait_http(url: str, timeout: int = 180) -> None:
"""
Consider Matomo 'reachable' as soon as the HTTP server answers - even with 500.
@@ -102,7 +709,6 @@ class WebInstaller(Installer):
wait_http(base_url)
if is_installed(base_url):
if config.debug:
_log("[install] Matomo already looks installed. Skipping installer.")
return
@@ -120,117 +726,263 @@ class WebInstaller(Installer):
page.set_default_navigation_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
page.set_default_timeout(PLAYWRIGHT_NAV_TIMEOUT_MS)
def _dbg(msg: str) -> None:
if config.debug:
_log(f"[install] {msg}")
def click_next() -> None:
"""
Matomo installer mixes link/button variants and sometimes includes '»'.
We try common variants in a robust order.
"""
candidates = [
("link", "Next »"),
("button", "Next »"),
("link", "Next"),
("button", "Next"),
("link", "Continue"),
("button", "Continue"),
("link", "Proceed"),
("button", "Proceed"),
("link", "Start Installation"),
("button", "Start Installation"),
("link", "Weiter"),
("button", "Weiter"),
("link", "Fortfahren"),
("button", "Fortfahren"),
]
for role, name in candidates:
loc = page.get_by_role(role, name=name)
if loc.count() > 0:
_dbg(f"click_next(): {role} '{name}'")
loc.first.click()
return
loc = page.get_by_text("Next", exact=False)
if loc.count() > 0:
_dbg("click_next(): fallback text 'Next'")
loc.first.click()
return
raise RuntimeError(
"Could not find a Next/Continue control in the installer UI."
)
try:
page.goto(base_url, wait_until="domcontentloaded")
_wait_for_installer_interactive(
page, timeout_s=INSTALLER_READY_TIMEOUT_S
)
_page_warnings(page)
def superuser_form_visible() -> bool:
return page.locator("#login-0").count() > 0
progress_deadline = time.time() + INSTALLER_STEP_DEADLINE_S
for _ in range(12):
if superuser_form_visible():
break
click_next()
page.wait_for_load_state("domcontentloaded")
page.wait_for_timeout(200)
else:
while not _has_superuser_login_field(page):
if time.time() >= progress_deadline:
raise RuntimeError(
"Installer did not reach superuser step (login-0 not found)."
"Installer did not reach superuser step "
f"within {INSTALLER_STEP_DEADLINE_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
if _resolve_tables_creation_conflict(
page, timeout_s=INSTALLER_TABLES_ERASE_TIMEOUT_S
):
_page_warnings(page)
continue
step_timeout = INSTALLER_STEP_TIMEOUT_S
if "tablesCreation" in _get_step_hint(page.url):
step_timeout = max(
step_timeout, INSTALLER_TABLES_CREATION_TIMEOUT_S
)
_click_next_with_wait(page, timeout_s=step_timeout)
_page_warnings(page)
_fill_required_input(
page,
SUPERUSER_LOGIN_SELECTORS,
config.admin_user,
label="superuser login",
)
_fill_required_input(
page,
SUPERUSER_PASSWORD_SELECTORS,
config.admin_password,
label="superuser password",
)
_fill_optional_input(
page, SUPERUSER_PASSWORD_REPEAT_SELECTORS, config.admin_password
)
_fill_required_input(
page,
SUPERUSER_EMAIL_SELECTORS,
config.admin_email,
label="superuser email",
)
_page_warnings(page)
submitted_superuser = False
try:
submitted_superuser = bool(
page.evaluate(
"""
([user, password, email]) => {
const form = document.querySelector("form#generalsetupform");
if (!form) return false;
const loginInput = form.querySelector("input[name='login']");
const passwordInput = form.querySelector("input[name='password']");
const repeatPasswordInput = form.querySelector("input[name='password_bis']");
const emailInput = form.querySelector("input[name='email']");
if (!loginInput || !passwordInput || !emailInput) return false;
loginInput.value = user;
passwordInput.value = password;
if (repeatPasswordInput) {
repeatPasswordInput.value = password;
}
emailInput.value = email;
if (typeof form.requestSubmit === "function") {
form.requestSubmit();
} else {
form.submit();
}
return true;
}
""",
[
config.admin_user,
config.admin_password,
config.admin_email,
],
)
)
except Exception:
submitted_superuser = False
if submitted_superuser:
_wait_dom_settled(page)
_log("[install] Submitted superuser form via form.requestSubmit().")
else:
submit_loc, submit_label = _first_present_css_locator(
page, SUPERUSER_SUBMIT_SELECTORS, timeout_s=0.5
)
if submit_loc is not None:
submit_loc.click(timeout=2_000)
_wait_dom_settled(page)
_log(
"[install] Submitted superuser form via "
f"{submit_label} fallback."
)
else:
_click_next_with_wait(page, timeout_s=INSTALLER_STEP_TIMEOUT_S)
superuser_progress_deadline = time.time() + INSTALLER_STEP_TIMEOUT_S
while time.time() < superuser_progress_deadline:
_wait_dom_settled(page)
if not _has_superuser_login_field(page):
break
page.wait_for_timeout(300)
if _has_superuser_login_field(page):
_page_warnings(page)
raise RuntimeError(
"Superuser form submit did not progress to first website setup "
f"within {INSTALLER_STEP_TIMEOUT_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
page.locator("#login-0").click()
page.locator("#login-0").fill(config.admin_user)
_page_warnings(page)
page.locator("#password-0").click()
page.locator("#password-0").fill(config.admin_password)
submitted_first_website = False
try:
submitted_first_website = bool(
page.evaluate(
"""
([siteName, siteUrl, timezoneLabel, ecommerceLabel]) => {
const form = document.querySelector("form#websitesetupform");
if (!form) return false;
if page.locator("#password_bis-0").count() > 0:
page.locator("#password_bis-0").click()
page.locator("#password_bis-0").fill(config.admin_password)
const siteNameInput = form.querySelector("input[name='siteName']");
const siteUrlInput = form.querySelector("input[name='url']");
if (!siteNameInput || !siteUrlInput) return false;
page.locator("#email-0").click()
page.locator("#email-0").fill(config.admin_email)
siteNameInput.value = siteName;
siteUrlInput.value = siteUrl;
if page.get_by_role("button", name="Next »").count() > 0:
page.get_by_role("button", name="Next »").click()
const timezoneSelect = form.querySelector("select[name='timezone']");
if (timezoneSelect) {
const timezoneOption = Array.from(timezoneSelect.options).find(
(opt) => (opt.textContent || "").trim() === timezoneLabel
);
if (timezoneOption) {
timezoneSelect.value = timezoneOption.value;
}
}
const ecommerceSelect = form.querySelector("select[name='ecommerce']");
if (ecommerceSelect) {
const ecommerceOption = Array.from(ecommerceSelect.options).find(
(opt) => (opt.textContent || "").trim() === ecommerceLabel
);
if (ecommerceOption) {
ecommerceSelect.value = ecommerceOption.value;
}
}
if (typeof form.requestSubmit === "function") {
form.requestSubmit();
} else {
form.submit();
}
return true;
}
""",
[
DEFAULT_SITE_NAME,
DEFAULT_SITE_URL,
DEFAULT_TIMEZONE,
DEFAULT_ECOMMERCE,
],
)
)
except Exception:
submitted_first_website = False
if submitted_first_website:
_wait_dom_settled(page)
_log(
"[install] Submitted first website form via form.requestSubmit()."
)
else:
click_next()
_fill_optional_input(
page, FIRST_WEBSITE_NAME_SELECTORS, DEFAULT_SITE_NAME
)
_fill_optional_input(
page, FIRST_WEBSITE_URL_SELECTORS, DEFAULT_SITE_URL
)
if page.locator("#siteName-0").count() > 0:
page.locator("#siteName-0").click()
page.locator("#siteName-0").fill(DEFAULT_SITE_NAME)
if page.locator("#url-0").count() > 0:
page.locator("#url-0").click()
page.locator("#url-0").fill(DEFAULT_SITE_URL)
_page_warnings(page)
try:
page.get_by_role("combobox").first.click()
page.get_by_role("listbox").get_by_text(DEFAULT_TIMEZONE).click()
comboboxes = page.get_by_role("combobox")
if _count_locator(comboboxes) > 0:
comboboxes.first.click(timeout=2_000)
page.get_by_role("listbox").get_by_text(
DEFAULT_TIMEZONE
).click(timeout=2_000)
except Exception:
_dbg("Timezone selection skipped (not found / changed UI).")
_log("Timezone selection skipped (not found / changed UI).")
try:
page.get_by_role("combobox").nth(2).click()
page.get_by_role("listbox").get_by_text(DEFAULT_ECOMMERCE).click()
comboboxes = page.get_by_role("combobox")
if _count_locator(comboboxes) > 2:
comboboxes.nth(2).click(timeout=2_000)
page.get_by_role("listbox").get_by_text(
DEFAULT_ECOMMERCE
).click(timeout=2_000)
except Exception:
_dbg("Ecommerce selection skipped (not found / changed UI).")
_log("Ecommerce selection skipped (not found / changed UI).")
click_next()
page.wait_for_load_state("domcontentloaded")
_page_warnings(page)
if page.get_by_role("link", name="Next »").count() > 0:
_click_next_with_wait(page, timeout_s=INSTALLER_STEP_TIMEOUT_S)
first_website_progress_deadline = time.time() + INSTALLER_STEP_TIMEOUT_S
while time.time() < first_website_progress_deadline:
_wait_dom_settled(page)
if not _has_first_website_name_field(page):
break
page.wait_for_timeout(300)
if _has_first_website_name_field(page):
_page_warnings(page)
raise RuntimeError(
"First website form submit did not progress to tracking code "
f"within {INSTALLER_STEP_TIMEOUT_S}s "
f"(url={page.url}, step={_get_step_hint(page.url)})."
)
_page_warnings(page)
if _count_locator(page.get_by_role("link", name="Next »")) > 0:
page.get_by_role("link", name="Next »").click()
_wait_dom_settled(page)
_page_warnings(page)
if page.get_by_role("button", name="Continue to Matomo »").count() > 0:
page.get_by_role("button", name="Continue to Matomo »").click()
continue_loc, _ = _first_continue_to_matomo_locator(page)
if continue_loc is not None:
continue_loc.click()
_wait_dom_settled(page)
_page_warnings(page)
page.wait_for_timeout(1_000)
if not is_installed(base_url):
_page_warnings(page)
raise RuntimeError(
"[install] Installer did not reach installed state."
)
except Exception as exc:
_dump_failure_artifacts(page, reason=str(exc))
raise
finally:
context.close()
browser.close()
time.sleep(1)
if not is_installed(base_url):
raise RuntimeError("[install] Installer did not reach installed state.")
_log("[install] Installation finished.")

View File

@@ -1,5 +1,7 @@
import os
import re
import subprocess
import textwrap
import unittest
@@ -7,14 +9,24 @@ MATOMO_URL = os.environ.get("MATOMO_URL", "http://127.0.0.1:8080")
ADMIN_USER = os.environ.get("MATOMO_ADMIN_USER", "administrator")
ADMIN_PASSWORD = os.environ.get("MATOMO_ADMIN_PASSWORD", "AdminSecret123!")
ADMIN_EMAIL = os.environ.get("MATOMO_ADMIN_EMAIL", "administrator@example.org")
TOKEN_RE = re.compile(r"^[a-f0-9]{32,64}$")
class TestMatomoBootstrapE2ENix(unittest.TestCase):
def test_bootstrap_creates_api_token_via_nix(self) -> None:
script = f"""set -euo pipefail
script = textwrap.dedent(
f"""\
set -eux
export NIX_CONFIG='experimental-features = nix-command flakes'
export TERM='xterm'
# Improve CI resilience for slow installer pages.
export MATOMO_INSTALLER_READY_TIMEOUT_S="${{MATOMO_INSTALLER_READY_TIMEOUT_S:-240}}"
export MATOMO_INSTALLER_STEP_TIMEOUT_S="${{MATOMO_INSTALLER_STEP_TIMEOUT_S:-45}}"
export MATOMO_INSTALLER_STEP_DEADLINE_S="${{MATOMO_INSTALLER_STEP_DEADLINE_S:-240}}"
export MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S="${{MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S:-240}}"
export MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S="${{MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S:-180}}"
export MATOMO_INSTALLER_DEBUG_DIR="${{MATOMO_INSTALLER_DEBUG_DIR:-/tmp/matomo-bootstrap}}"
# Make sure we have a writable HOME (compose already sets HOME=/tmp/home)
mkdir -p "$HOME" "$HOME/.cache" "$HOME/.config" "$HOME/.local/share"
@@ -25,6 +37,16 @@ mkdir -p "$HOME" "$HOME/.cache" "$HOME/.config" "$HOME/.local/share"
# Mark it as safe explicitly.
git config --global --add safe.directory /work
# Preflight checks to surface "command not executable" failures (exit 126) clearly.
playwright_app="$(nix eval --raw .#apps.x86_64-linux.matomo-bootstrap-playwright-install.program)"
bootstrap_app="$(nix eval --raw .#apps.x86_64-linux.matomo-bootstrap.program)"
if [ -e "$playwright_app" ]; then
test -x "$playwright_app"
fi
if [ -e "$bootstrap_app" ]; then
test -x "$bootstrap_app"
fi
# 1) Install Playwright Chromium (cached in the container environment)
nix run --no-write-lock-file -L .#matomo-bootstrap-playwright-install
@@ -36,13 +58,18 @@ nix run --no-write-lock-file -L .#matomo-bootstrap -- \\
--admin-email '{ADMIN_EMAIL}' \\
--token-description 'e2e-test-token-nix'
"""
)
cmd = [
"docker",
"compose",
"-f",
"tests/e2e/docker-compose.yml",
"exec",
# Use `run` instead of `exec` to avoid runtime-specific
# `/etc/group` lookup issues seen with nix image + compose exec.
"run",
"--rm",
"--no-deps",
"-T",
"nix",
"sh",
@@ -50,8 +77,30 @@ nix run --no-write-lock-file -L .#matomo-bootstrap -- \\
script,
]
token = subprocess.check_output(cmd).decode().strip()
self.assertRegex(token, r"^[a-f0-9]{32,64}$")
result = subprocess.run(
cmd,
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
if result.returncode != 0:
self.fail(
"nix bootstrap command failed\n"
f"exit={result.returncode}\n"
f"stdout:\n{result.stdout}\n"
f"stderr:\n{result.stderr}"
)
stdout_lines = [
line.strip() for line in result.stdout.splitlines() if line.strip()
]
token = stdout_lines[-1] if stdout_lines else ""
self.assertRegex(
token,
TOKEN_RE,
f"Expected token on last stdout line, got stdout={result.stdout!r}",
)
if __name__ == "__main__":

View File

@@ -56,6 +56,29 @@ def _wait_for_http_any_status(url: str, timeout_s: int) -> None:
raise RuntimeError(f"Matomo did not become reachable at {url} ({last_exc})")
def _extract_service_block(compose_config: str, service_name: str) -> str:
lines = compose_config.splitlines()
marker = f" {service_name}:"
start = -1
for idx, line in enumerate(lines):
if line == marker:
start = idx
break
if start < 0:
raise AssertionError(
f"service block not found in compose config: {service_name}"
)
end = len(lines)
for idx in range(start + 1, len(lines)):
line = lines[idx]
if line.startswith(" ") and not line.startswith(" "):
end = idx
break
return "\n".join(lines[start:end])
class TestRootDockerComposeStack(unittest.TestCase):
"""
E2E test for repository root docker-compose.yml:
@@ -89,28 +112,65 @@ class TestRootDockerComposeStack(unittest.TestCase):
# Build bootstrap image from Dockerfile (as defined in docker-compose.yml)
build = _run(
_compose_cmd("build", "bootstrap"),
check=True,
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(build.returncode, 0, build.stderr)
self.assertEqual(
build.returncode,
0,
f"compose build failed\nstdout:\n{build.stdout}\nstderr:\n{build.stderr}",
)
# Start db + matomo (bootstrap is one-shot and started via "run")
up = _run(
_compose_cmd("up", "-d", "db", "matomo"),
check=True,
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(up.returncode, 0, up.stderr)
self.assertEqual(
up.returncode,
0,
f"compose up failed\nstdout:\n{up.stdout}\nstderr:\n{up.stderr}",
)
# Wait until Matomo answers on the published port
_wait_for_http_any_status(MATOMO_HOST_URL + "/", WAIT_TIMEOUT_SECONDS)
# Run bootstrap: it should print ONLY the token to stdout
# Run bootstrap: it should print ONLY the token to stdout.
# Retry once because first-run installer startup can be flaky on slow CI.
boot_attempts: list[subprocess.CompletedProcess] = []
for _ in range(2):
boot = _run(
_compose_cmd("run", "--rm", "bootstrap"),
check=True,
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
boot_attempts.append(boot)
if boot.returncode == 0:
break
time.sleep(5)
if boot.returncode != 0:
matomo_logs = _run(
_compose_cmd("logs", "--no-color", "--tail=200", "matomo"),
check=False,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
attempts_dump = "\n\n".join(
[
(
f"[attempt {i}] rc={attempt.returncode}\n"
f"stdout:\n{attempt.stdout}\n"
f"stderr:\n{attempt.stderr}"
)
for i, attempt in enumerate(boot_attempts, 1)
]
)
self.fail(
"bootstrap container failed after retry.\n"
f"{attempts_dump}\n\n"
f"[matomo logs]\n{matomo_logs.stdout}\n{matomo_logs.stderr}"
)
token = (boot.stdout or "").strip()
self.assertRegex(
@@ -131,5 +191,32 @@ class TestRootDockerComposeStack(unittest.TestCase):
self.assertIsInstance(data, list)
class TestRootDockerComposeDefinition(unittest.TestCase):
def test_bootstrap_service_waits_for_healthy_matomo_and_has_readiness_knobs(
self,
) -> None:
cfg = _run(
_compose_cmd("config"),
check=True,
extra_env={"MATOMO_PORT": MATOMO_PORT},
)
self.assertEqual(cfg.returncode, 0, cfg.stderr)
bootstrap_block = _extract_service_block(cfg.stdout, "bootstrap")
self.assertIn("depends_on:", bootstrap_block)
self.assertIn("matomo:", bootstrap_block)
self.assertIn("condition: service_healthy", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_READY_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_STEP_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_STEP_DEADLINE_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_TABLES_CREATION_TIMEOUT_S:", bootstrap_block)
self.assertIn("MATOMO_INSTALLER_TABLES_ERASE_TIMEOUT_S:", bootstrap_block)
matomo_block = _extract_service_block(cfg.stdout, "matomo")
self.assertIn("healthcheck:", matomo_block)
self.assertIn("curl -fsS http://127.0.0.1/ >/dev/null || exit 1", matomo_block)
if __name__ == "__main__":
unittest.main()

View File

View File

@@ -0,0 +1,173 @@
import unittest
from matomo_bootstrap.installers.web import _click_next_with_wait, _count_locator
class _FlakyLocator:
def __init__(self, outcomes):
self._outcomes = list(outcomes)
self.calls = 0
def count(self) -> int:
self.calls += 1
outcome = self._outcomes.pop(0)
if isinstance(outcome, Exception):
raise outcome
return int(outcome)
class _StaticLocator:
def __init__(self, page, selector: str):
self._page = page
self._selector = selector
def count(self) -> int:
if self._selector == "#login-0":
return 1 if self._page.login_visible else 0
if self._selector == "#siteName-0":
return 0
return 0
@property
def first(self):
return self
def is_visible(self) -> bool:
return self.count() > 0
class _RoleLocator:
def __init__(self, count_value: int):
self._count_value = count_value
def count(self) -> int:
return self._count_value
@property
def first(self):
return self
def is_visible(self) -> bool:
return self._count_value > 0
class _NameOnlyStaticLocator:
def __init__(self, page, selector: str):
self._page = page
self._selector = selector
def count(self) -> int:
if self._selector == "input[name='login']":
return 1 if self._page.login_visible else 0
if self._selector == "input[name='siteName']":
return 0
return 0
@property
def first(self):
return self
def is_visible(self) -> bool:
return self.count() > 0
class _NoNextButLoginAppearsPage:
def __init__(self):
self.url = "http://matomo/index.php?action=setupSuperUser&module=Installation"
self.login_visible = False
self._wait_calls = 0
def locator(self, selector: str):
return _StaticLocator(self, selector)
def get_by_role(self, role: str, name: str):
return _RoleLocator(0)
def get_by_text(self, *_args, **_kwargs):
return _RoleLocator(0)
def title(self) -> str:
return "setupSuperUser"
def wait_for_load_state(self, *_args, **_kwargs):
return None
def wait_for_timeout(self, *_args, **_kwargs):
self._wait_calls += 1
if self._wait_calls >= 1:
self.login_visible = True
class _NoNextButNamedLoginAppearsPage:
def __init__(self):
self.url = "http://matomo/index.php?action=setupSuperUser&module=Installation"
self.login_visible = False
self._wait_calls = 0
def locator(self, selector: str):
return _NameOnlyStaticLocator(self, selector)
def get_by_role(self, role: str, name: str):
return _RoleLocator(0)
def get_by_text(self, *_args, **_kwargs):
return _RoleLocator(0)
def title(self) -> str:
return "setupSuperUser"
def wait_for_load_state(self, *_args, **_kwargs):
return None
def wait_for_timeout(self, *_args, **_kwargs):
self._wait_calls += 1
if self._wait_calls >= 1:
self.login_visible = True
class TestWebInstallerLocatorCountIntegration(unittest.TestCase):
def test_retries_transient_navigation_error(self) -> None:
locator = _FlakyLocator(
[
RuntimeError(
"Locator.count: Execution context was destroyed, most likely because of a navigation"
),
RuntimeError(
"Locator.count: Execution context was destroyed, most likely because of a navigation"
),
1,
]
)
result = _count_locator(locator, timeout_s=0.5, retry_interval_s=0.0)
self.assertEqual(result, 1)
self.assertEqual(locator.calls, 3)
def test_raises_non_transient_error_without_retry(self) -> None:
locator = _FlakyLocator([RuntimeError("Locator is not attached to DOM")])
with self.assertRaises(RuntimeError):
_count_locator(locator, timeout_s=0.5, retry_interval_s=0.0)
self.assertEqual(locator.calls, 1)
def test_click_next_wait_treats_login_form_as_progress(self) -> None:
page = _NoNextButLoginAppearsPage()
step = _click_next_with_wait(page, timeout_s=1)
self.assertEqual(step, "Installation:setupSuperUser")
self.assertTrue(page.login_visible)
def test_click_next_wait_treats_named_login_form_as_progress(self) -> None:
page = _NoNextButNamedLoginAppearsPage()
step = _click_next_with_wait(page, timeout_s=1)
self.assertEqual(step, "Installation:setupSuperUser")
self.assertTrue(page.login_visible)
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,128 @@
import io
import unittest
from contextlib import redirect_stderr
# Import the function under test.
# This keeps the test close to real integration behavior without requiring Playwright.
from matomo_bootstrap.installers.web import _page_warnings
class _FakeLocatorNth:
def __init__(self, text: str):
self._text = text
def inner_text(self) -> str:
return self._text
class _FakeLocator:
def __init__(self, texts: list[str]):
self._texts = texts
def count(self) -> int:
return len(self._texts)
def nth(self, i: int) -> _FakeLocatorNth:
return _FakeLocatorNth(self._texts[i])
class _FakePage:
"""
Minimal Playwright-like page stub:
- locator(selector) -> object with count() / nth(i).inner_text()
- url, title()
"""
def __init__(self, *, url: str, title: str, selector_texts: dict[str, list[str]]):
self.url = url
self._title = title
self._selector_texts = selector_texts
def title(self) -> str:
return self._title
def locator(self, selector: str) -> _FakeLocator:
return _FakeLocator(self._selector_texts.get(selector, []))
class TestWebInstallerWarningsIntegration(unittest.TestCase):
def test_detects_bootstrap_alert_warning_block(self) -> None:
"""
Matomo installer commonly renders validation errors like:
<div class="alert alert-warning"> ... <ul><li>...</li></ul> ... </div>
We must detect and print those messages to stderr.
"""
page = _FakePage(
url="http://matomo/index.php?action=setupSuperUser&module=Installation",
title="Superuser",
selector_texts={
# The key selector from the observed DOM
".alert.alert-warning": [
"Please fix the following errors:\n"
"Password required\n"
"Password (repeat) required\n"
"The email doesn't have a valid format."
],
},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
# Function must return the warning text
self.assertEqual(len(warnings), 1)
self.assertIn("Please fix the following errors:", warnings[0])
self.assertIn("The email doesn't have a valid format.", warnings[0])
# And it must print it to stderr (stdout must remain token-only in the app)
out = buf.getvalue()
self.assertIn("[install] page warnings/errors detected", out)
self.assertIn("Superuser", out)
self.assertIn("The email doesn't have a valid format.", out)
def test_deduplicates_repeated_warning_blocks(self) -> None:
"""
Some Matomo versions repeat the same alert in multiple containers.
We must return/log each unique text only once.
"""
repeated = (
"Please fix the following errors:\nThe email doesn't have a valid format."
)
page = _FakePage(
url="http://matomo/index.php?action=setupSuperUser&module=Installation",
title="Superuser",
selector_texts={
".alert.alert-warning": [repeated, repeated],
},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
self.assertEqual(warnings, [repeated])
out = buf.getvalue()
# Only a single numbered entry should be printed
self.assertIn("[install] 1) ", out)
self.assertNotIn("[install] 2) ", out)
def test_no_output_when_no_warnings(self) -> None:
page = _FakePage(
url="http://matomo/",
title="Welcome",
selector_texts={},
)
buf = io.StringIO()
with redirect_stderr(buf):
warnings = _page_warnings(page, prefix="[install]")
self.assertEqual(warnings, [])
self.assertEqual(buf.getvalue(), "")
if __name__ == "__main__":
unittest.main()