Commit Graph

20 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
33dfbf3a4d test(env-virtual): execute pkgmgr from Python venv instead of system launcher
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / linter-shell (push) Has been cancelled
Mark stable commit / linter-python (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
The virtual environment test no longer invokes the distro-installed pkgmgr launcher (Nix-based).
Instead, it explicitly installs and activates the Python venv via make setup-venv and runs pkgmgr from there.

This aligns the test with its actual purpose (venv validation), avoids accidental execution of the Nix launcher, and fixes the failure caused by the missing run_with_github_403_retry helper in the venv workflow.

https://chatgpt.com/share/693ee224-e838-800f-8fa0-45295b2f5e20
2025-12-14 17:12:48 +01:00
Kevin Veen-Birkenbach
7f262c6557 feat(install): add --update to re-run active-layer installers and improve Nix refresh logic
Some checks failed
CI / test-unit (push) Has been cancelled
CI / test-integration (push) Has been cancelled
CI / test-env-virtual (push) Has been cancelled
CI / test-env-nix (push) Has been cancelled
CI / test-e2e (push) Has been cancelled
CI / test-virgin-user (push) Has been cancelled
CI / test-virgin-root (push) Has been cancelled
CI / codesniffer-shellcheck (push) Has been cancelled
CI / codesniffer-ruff (push) Has been cancelled
* Add `force_update` to `RepoContext` and propagate it through install/update flows
* Add `pkgmgr install --update` to force re-running installers even if the same CLI layer is already loaded
* Enhance `NixFlakeInstaller` to ensure correct outputs (pkgmgr + optional default for package-manager) and support refresh/upgrade with index-based fallback remove+reinstall
* Make Python/Makefile installers emit an “upgraded” marker when `force_update` is used
* Add E2E tests for “three times install” scenarios (makefile, nix, venv) with shared run helper
* Fix git safe.directory wildcard quoting in E2E shell runner and minor cleanup/reordering of imports/comments

https://chatgpt.com/share/693db0b4-6ea4-800f-b44a-f03939c7fb9e
2025-12-13 19:30:06 +01:00
Kevin Veen-Birkenbach
0bc7a3ecc0 ci(nix): retry flake evaluation on GitHub API rate limits
Some checks failed
CI / test-unit (push) Has been cancelled
CI / test-integration (push) Has been cancelled
CI / test-env-virtual (push) Has been cancelled
CI / test-env-nix (push) Has been cancelled
CI / test-e2e (push) Has been cancelled
CI / test-virgin-user (push) Has been cancelled
CI / test-virgin-root (push) Has been cancelled
CI / codesniffer-shellcheck (push) Has been cancelled
CI / codesniffer-ruff (push) Has been cancelled
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / codesniffer-shellcheck (push) Has been cancelled
Mark stable commit / codesniffer-ruff (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
Add a reusable retry helper that detects GitHub API 403 rate-limit errors
during Nix flake evaluation and retries with exponential backoff.

Apply the retry logic to flake-only CI tests so transient GitHub rate
limits no longer cause random CI failures while preserving fast failure
for real errors.

https://chatgpt.com/share/693d7ec5-ac70-800f-a627-ef705c653ba1
2025-12-13 15:57:05 +01:00
Kevin Veen-Birkenbach
2c3efa7a27 Solved shellcheck quoting issue
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / codesniffer-shellcheck (push) Has been cancelled
Mark stable commit / codesniffer-ruff (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
2025-12-13 14:38:37 +01:00
Kevin Veen-Birkenbach
4e28eba883 refactor(ci,build,test): rename distro to PKGMGR_DISTRO for consistent environment handling
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / codesniffer-shellcheck (push) Has been cancelled
Mark stable commit / codesniffer-ruff (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
https://chatgpt.com/share/693d6b63-12cc-800f-b55f-abc52ee7fb52
2025-12-13 14:34:15 +01:00
Kevin Veen-Birkenbach
0dfbaa0f6b ci/docker: unify image build logic and run virgin tests across all distros
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
Refactor Dockerfile into multi-stage virgin/full targets and introduce a single
flag-based image build script. Standardize image naming, remove redundant build
scripts, and update Makefile targets accordingly. CI workflows now build missing
virgin images and run root and user tests consistently across all supported
distributions.

https://chatgpt.com/share/693c29d9-9b28-800f-a549-5661c783d968
2025-12-12 16:40:21 +01:00
Kevin Veen-Birkenbach
08ab9fb142 feat(ci): stabilize virgin Arch tests with Makefile install/setup and Nix Git safety
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
* Switch virgin root/user workflows to use *make install* + *make setup/setup-venv*
* Add Git *safe.directory /src* to avoid flake evaluation failures on mounted repos
* Enable Nix flake run in workflows and prepare */nix* for non-root execution
* Refactor Arch packaging to build in an isolated */tmp* directory via *aur_builder*
* Rename installer scripts (*run-** → *dependencies.sh* / *package.sh*) and adjust Docker entry + env var to *REINSTALL_PKGMGR*

https://chatgpt.com/share/693c29d9-9b28-800f-a549-5661c783d968
2025-12-12 15:42:25 +01:00
Kevin Veen-Birkenbach
324f6db1f3 ci: split container tests into virtualenv and Nix flake environments
Some checks failed
Mark stable commit / test-unit (push) Has been cancelled
Mark stable commit / test-integration (push) Has been cancelled
Mark stable commit / test-env-virtual (push) Has been cancelled
Mark stable commit / test-env-nix (push) Has been cancelled
Mark stable commit / test-e2e (push) Has been cancelled
Mark stable commit / test-virgin-user (push) Has been cancelled
Mark stable commit / test-virgin-root (push) Has been cancelled
Mark stable commit / mark-stable (push) Has been cancelled
Refactor CI to clearly separate virtualenv-based container tests from pure Nix flake tests across all distros (arch, debian, ubuntu, fedora, centos).
Introduce dedicated test-env-nix workflow and Makefile targets, rename former container tests to test-env-virtual, and update stable pipeline dependencies.
Improve Nix reliability in containers by fixing installer permissions and explicitly validating nix availability and version during image build and tests.
2025-12-12 12:15:40 +01:00
Kevin Veen-Birkenbach
39b16b87a8 CI: Add debugging instrumentation to identify container build/run anomalies
Some checks failed
CI / test-unit (push) Has been cancelled
CI / test-integration (push) Has been cancelled
CI / test-container (push) Has been cancelled
CI / test-e2e (push) Has been cancelled
CI / test-virgin-user (push) Has been cancelled
CI / test-virgin-root (push) Has been cancelled
- Added `git rev-parse HEAD` to test-container workflow to confirm the exact
  commit SHA used during CI runs.
- Updated Dockerfile to print BASE_IMAGE and OS release information during
  build for better reproducibility diagnostics.
- Extended test-container script to dump the first 40 lines of
  `docker image inspect` output, allowing verification of the image ID,
  creation time, and applied build args.

These additions help trace discrepancies between local builds and GitHub
Actions, ensuring we can detect mismatches in commit SHA, base image,
or container metadata.

https://chatgpt.com/share/693ae07a-8c58-800f-88e6-254cdb00b676
2025-12-11 17:27:57 +01:00
Kevin Veen-Birkenbach
5ca1adda7b Refactor CI distro handling and container build scripts
- Introduce a GitHub Actions matrix for `test-container` and `test-e2e`
  to run against arch, debian, ubuntu, fedora, and centos
- Run unit and integration tests only in the Arch container by passing
  `distro="arch"` via make in the corresponding workflows
- Replace the global DISTROS loop with a single `distro` variable in
  the Makefile, defaulting to `arch`, and export it for all scripts
- Update build scripts (build-image, build-image-no-cache, build-image-missing)
  to build images for the selected distro only
- Simplify test-container script to validate a single distro image using
  the `distro` environment variable
- Simplify E2E, unit, and integration test scripts to run against a
  single distro container instead of iterating over all distros

https://chatgpt.com/share/693acbba-9e30-800f-94fb-fea4489e9078
2025-12-11 14:48:36 +01:00
Kevin Veen-Birkenbach
acb18adf76 test: restore Dockerfile ENTRYPOINT for all test runs (fix Nix TLS on CentOS)
All test scripts (unit, integration, e2e) previously overwrote the Docker
ENTRYPOINT by using `--entrypoint bash`, which bypassed the container’s
startup logic in `docker-entry.sh`.

`docker-entry.sh` performs essential initialization steps such as:

- CA bundle auto-detection (NIX_SSL_CERT_FILE, SSL_CERT_FILE, etc.)
- Nix environment setup
- PATH adjustments and distro logging

By removing the explicit `--entrypoint bash` and invoking:

  bash -lc '...'

directly as the container command, the Dockerfile’s ENTRYPOINT is restored
and runs as intended before executing the test logic.

This fixes TLS issues in CentOS E2E runs where Nix was unable to fetch
flake inputs due to missing CA configuration.

https://chatgpt.com/share/693ac1f3-fb7c-800f-9e5c-b40c351a9f04
2025-12-11 14:06:39 +01:00
Kevin Veen-Birkenbach
bb0a801396 Fix Git safe.directory handling in E2E containers
- Mark /src and /src/.git as safe to satisfy newer Git ownership checks
- Add '*' as safe.directory for ephemeral test containers to avoid Nix flake failures

https://chatgpt.com/share/693a9e1f-1cc8-800f-9df4-90813cbb6bd5
2025-12-11 11:33:51 +01:00
Kevin Veen-Birkenbach
ee968efc4b Harden E2E test runner and fix Git safe.directory in containers
- Quote Nix store/cache volumes and distro image name in docker run
- Use strict bash flags (set -euo pipefail) inside test container
- Print distro ID robustly with fallback
- Configure /src as Git safe.directory when git is available

https://chatgpt.com/share/693a9c0e-59ec-800f-83a1-eec31bd76962
2025-12-11 11:25:11 +01:00
Kevin Veen-Birkenbach
389ec40163 Refine Nix dev shell, ensure PyYAML availability, fix Python invocation, and
expose pkgmgr.cli for Python 3.13 compatibility

- Add `.nix-dev-installed` to .gitignore
- Improve flake.nix:
  * unify pkgs/pyPkgs definitions
  * provide python311.withPackages including pip + PyYAML
  * remove unused pkgmgrPkg reference from devShell
  * fix PYTHONPATH export and devShell help message
- Update unit/integration test scripts to use `python3 -m unittest`
- Add top-level pkgmgr.__init__ exposing `cli` attribute for
  pkgutil.resolve_name compatibility under Python 3.13+
2025-12-11 09:33:55 +01:00
Kevin Veen-Birkenbach
6d7ee6fc04 Fix test scripts: ensure default distro and always run via bash
- Remove Makefile inline variable export (distro=arch) and invoke scripts via bash
- Add robust default in test-unit.sh and test-integration.sh:
    : "${distro:=arch}"
- Prevent "unbound variable" errors under `set -u` when no distro is provided
2025-12-10 21:09:18 +01:00
Kevin Veen-Birkenbach
5a022db0db Use dynamic distro selection for UNIT and INTEGRATION tests
- Pass `distro=arch` from Makefile into test scripts
- Replace hardcoded "arch" references with "${distro}"
- Update test-unit.sh and test-integration.sh to use dynamic image names
- Improve log output to reflect selected distro

https://chatgpt.com/share/6939c98a-d428-800f-8bb8-cf72e80ba80c
2025-12-10 20:27:03 +01:00
Kevin Veen-Birkenbach
37ac22e0b4 test: isolate Nix store/cache per distro to fix cross-distro manifest conflicts
- Replace shared Nix volumes with distro-specific volumes
  (pkgmgr_nix_store_<distro>, pkgmgr_nix_cache_<distro>)
- Prevent incompatible profile manifest versions between Ubuntu and Debian
- Update all test scripts (unit, integration, container, e2e)
- Remove unused global Nix volume variables from Makefile
- Improve consistency of test-e2e.sh formatting and environment handling
- Add Git safe.directory configuration for mounted /src to avoid ownership warnings
2025-12-10 20:07:41 +01:00
Kevin Veen-Birkenbach
78693225f1 test: share persistent Nix store across all test containers
This commit adds the `pkgmgr_nix_store` volume mount (`/nix`) to all test
runners (unit, integration, container sanity checks, and E2E tests).

Previously only the Arch-based E2E container mounted a persistent `/nix`
store, causing all other distros (Debian, Ubuntu, Fedora, CentOS, etc.)
to download the entire Nix closure repeatedly during test runs.

Changes:
- Add `-v pkgmgr_nix_store:/nix` to:
  - scripts/test/test-container.sh
  - scripts/test/test-e2e.sh (remove Arch-only condition)
  - scripts/test/test-unit.sh
  - scripts/test/test-integration.sh
- Ensures all test containers reuse the same Nix store.

Benefits:
- Significantly faster test execution after the first run.
- Prevents redundant downloads from cache.nixos.org.
- Ensures consistent Nix environments across all test distros.

No functional changes to pkgmgr itself; only test infrastructure improved.

https://chatgpt.com/share/693890f5-2f54-800f-b47e-1925da85b434
2025-12-09 22:13:01 +01:00
Kevin Veen-Birkenbach
ca5d0d22f3 feat(test): make unittest pattern configurable and pass TEST_PATTERN into containers
This update introduces a configurable TEST_PATTERN variable in the Makefile,
allowing selective execution of unit, integration, and E2E tests without
modifying scripts.

Key changes:
- Add TEST_PATTERN (default: test_*.py) to Makefile and export it.
- Inject TEST_PATTERN into all test containers via `-e TEST_PATTERN=...`.
- Update test-unit.sh, test-integration.sh, and test-e2e.sh to use
  `-p "$TEST_PATTERN"` instead of a hardcoded pattern.
- Ensure flexible test selection via:
      make test-e2e TEST_PATTERN=test_install_pkgmgr_shallow.py

This enables fast debugging, selective test runs, and better developer
experience while keeping full compatibility with CI defaults.

https://chatgpt.com/share/69385400-2f14-800f-b093-bb03c8ef9c7f
2025-12-09 17:53:10 +01:00
Kevin Veen-Birkenbach
f9943fafae Refactor container build and installation pipeline to use configurable Makefile parameters (e.g. DISTROS, base images) and propagate them through all build, install, and test scripts 2025-12-09 05:31:55 +01:00