Refactored the Nix initialization script to reduce duplicated code and
centralize the installation workflow. The core functionality remains
unchanged, but all installer calls now use a unified function with retry
support to ensure resilient downloads in CI and container environments.
Key improvements:
- Added download retry logic (5 minutes total, 20-second intervals)
- Consolidated installer invocation into `install_nix_with_retry`
- Reduced code duplication across container/host install paths
- Preserved existing installation behavior for all environments
- Maintained `nixbld` group and build-user handling
- Improved consistency and readability without altering semantics
This prevents intermittent failures such as:
“curl: (6) Could not resolve host: nixos.org”
and ensures stable, deterministic Nix setup in CI pipelines.
https://chatgpt.com/share/693b13ce-fdcc-800f-a7bc-81c67478edff
Implemented a robust retry mechanism for cloning the yay AUR helper during Arch dependency installation.
The new logic retries the git clone operation for up to 5 minutes with a 20-second pause between attempts, allowing the build to proceed even when GitHub intermittently returns HTTP 504 errors.
This improves the stability of Arch container builds, especially under network pressure or transient upstream outages.
The yay build process now only starts once the clone step completes successfully.
https://chatgpt.com/share/693b102b-fdb0-800f-9f2e-d4840f14d329
Implement `ensure_nix_build_group()` and use it in all code paths where Nix is installed as root.
This resolves Nix installation failures on Ubuntu containers (root, no systemd) where the installer aborts with:
```
error: the group 'nixbld' specified in 'build-users-group' does not exist
```
The fix standardizes creation of the `nixbld` group and `nixbld1..10` build users across:
* container root mode
* systemd host daemon installs
* root-on-host without systemd (Debian/Ubuntu CI case)
This makes Nix initialization deterministic across all test distros and fixes failing Ubuntu E2E runs.
https://chatgpt.com/share/693b0e1a-e5d4-800f-8a89-7d91108b0368
- Extend PATH probing to include /home/nix/.nix-profile/bin/nix (container mode).
- Automatically invoke init-nix.sh when nix is not found before first run.
- Ensure pkgmgr always attempts a one-time Nix initialization instead of failing prematurely.
- Improve error message to clarify that nix was still missing *after* initialization attempt.
- Keep existing flake-based execution path unchanged (exec nix run …).
This makes the wrapper fully reliable across Debian/Ubuntu package installs,
fresh containers, and minimal systems where Nix is not yet initialized.
https://chatgpt.com/share/693b005d-b250-800f-8830-ab71685f51b3
- Added `git rev-parse HEAD` to test-container workflow to confirm the exact
commit SHA used during CI runs.
- Updated Dockerfile to print BASE_IMAGE and OS release information during
build for better reproducibility diagnostics.
- Extended test-container script to dump the first 40 lines of
`docker image inspect` output, allowing verification of the image ID,
creation time, and applied build args.
These additions help trace discrepancies between local builds and GitHub
Actions, ensuring we can detect mismatches in commit SHA, base image,
or container metadata.
https://chatgpt.com/share/693ae07a-8c58-800f-88e6-254cdb00b676
- Introduce a GitHub Actions matrix for `test-container` and `test-e2e`
to run against arch, debian, ubuntu, fedora, and centos
- Run unit and integration tests only in the Arch container by passing
`distro="arch"` via make in the corresponding workflows
- Replace the global DISTROS loop with a single `distro` variable in
the Makefile, defaulting to `arch`, and export it for all scripts
- Update build scripts (build-image, build-image-no-cache, build-image-missing)
to build images for the selected distro only
- Simplify test-container script to validate a single distro image using
the `distro` environment variable
- Simplify E2E, unit, and integration test scripts to run against a
single distro container instead of iterating over all distros
https://chatgpt.com/share/693acbba-9e30-800f-94fb-fea4489e9078
All test scripts (unit, integration, e2e) previously overwrote the Docker
ENTRYPOINT by using `--entrypoint bash`, which bypassed the container’s
startup logic in `docker-entry.sh`.
`docker-entry.sh` performs essential initialization steps such as:
- CA bundle auto-detection (NIX_SSL_CERT_FILE, SSL_CERT_FILE, etc.)
- Nix environment setup
- PATH adjustments and distro logging
By removing the explicit `--entrypoint bash` and invoking:
bash -lc '...'
directly as the container command, the Dockerfile’s ENTRYPOINT is restored
and runs as intended before executing the test logic.
This fixes TLS issues in CentOS E2E runs where Nix was unable to fetch
flake inputs due to missing CA configuration.
https://chatgpt.com/share/693ac1f3-fb7c-800f-9e5c-b40c351a9f04
- Quote Nix store/cache volumes and distro image name in docker run
- Use strict bash flags (set -euo pipefail) inside test container
- Print distro ID robustly with fallback
- Configure /src as Git safe.directory when git is available
https://chatgpt.com/share/693a9c0e-59ec-800f-83a1-eec31bd76962
* Move Arch-specific ignore rules into `packaging/arch/.gitignore` and simplify top-level `.gitignore`/`.dockerignore`.
* Update Arch `PKGBUILD` to sync from the project root and drop `packaging/` from the installed tree.
* Fix OS-specific `package.sh` helpers to resolve the new `packaging/*` locations correctly for Arch, Debian/Ubuntu, Fedora, and CentOS.
- Add dev runner main.py that prefers local src/ over installed pkgmgr
- Move Arch/Debian/Fedora packaging files under packaging/* and update build scripts
- Adjust .gitignore/.dockerignore for new packaging paths and src/source/
- Improve config defaults discovery to support src/ layout and installed packages
- Update architecture diagram and add TODO overview for TAGS/MIRROR/SIGNING_KEY
https://chatgpt.com/share/693a76a0-e408-800f-9939-868524cbef4d
- Split virgin tests into separate root and user GitHub Actions workflows
(test-virgin-root, test-virgin-user) and adjust Arch container flows
- Introduce scripts/installation/venv-create.sh and reuse it from
scripts/installation/main.sh with separate root/system and user/dev paths
- Add PKGMGR architecture & setup map (assets/map.png) and section in README
with link to the up-to-date master page
- Simplify README by removing outdated Docker quickstart, usage examples,
and AI footer
- Extend .gitignore to exclude src/source artifacts
https://chatgpt.com/share/6939bbfe-5cb0-800f-8ea8-95628dc911f5
The init-nix.sh script previously hardcoded /usr/bin/bash as the login shell
for the 'nix' user, which exists on Arch but not on Debian. This caused the
Nix single-user installer (run via `su - nix`) to fail silently or break in
unpredictable ways on Debian-based images.
We now resolve the shell dynamically via `command -v bash` and fall back to
/bin/sh on minimal systems. This makes Nix installation deterministic across
Arch, Debian, Ubuntu, Fedora, CentOS and CI containers.
https://chatgpt.com/share/6939e97f-c93c-800f-887b-27c7e67ec46d
- Remove Makefile inline variable export (distro=arch) and invoke scripts via bash
- Add robust default in test-unit.sh and test-integration.sh:
: "${distro:=arch}"
- Prevent "unbound variable" errors under `set -u` when no distro is provided
- Pass `distro=arch` from Makefile into test scripts
- Replace hardcoded "arch" references with "${distro}"
- Update test-unit.sh and test-integration.sh to use dynamic image names
- Improve log output to reflect selected distro
https://chatgpt.com/share/6939c98a-d428-800f-8bb8-cf72e80ba80c
This commit adds the `pkgmgr_nix_store` volume mount (`/nix`) to all test
runners (unit, integration, container sanity checks, and E2E tests).
Previously only the Arch-based E2E container mounted a persistent `/nix`
store, causing all other distros (Debian, Ubuntu, Fedora, CentOS, etc.)
to download the entire Nix closure repeatedly during test runs.
Changes:
- Add `-v pkgmgr_nix_store:/nix` to:
- scripts/test/test-container.sh
- scripts/test/test-e2e.sh (remove Arch-only condition)
- scripts/test/test-unit.sh
- scripts/test/test-integration.sh
- Ensures all test containers reuse the same Nix store.
Benefits:
- Significantly faster test execution after the first run.
- Prevents redundant downloads from cache.nixos.org.
- Ensures consistent Nix environments across all test distros.
No functional changes to pkgmgr itself; only test infrastructure improved.
https://chatgpt.com/share/693890f5-2f54-800f-b47e-1925da85b434
- NixFlakeInstaller:
- Skip when running inside a Nix dev shell (IN_NIX_SHELL).
- Add PKGMGR_DISABLE_NIX_FLAKE_INSTALLER kill-switch for CI/debugging.
- Ensure run() respects supports() and handles preview/allow_failure cleanly.
- DebianControlInstaller:
- Introduce _privileged_prefix() to handle sudo vs. root vs. no elevation.
- Avoid hard-coded sudo usage and degrade gracefully when neither sudo nor
root is available.
- Improve messaging around build-dep and .deb installation.
- RpmSpecInstaller:
- Prepare rpmbuild tree and source tarball in ~/rpmbuild/SOURCES based on
Name/Version from the spec file.
- Reuse a helper to resolve the rpmbuild topdir.
- Install built RPMs via dnf/yum when available, falling back to rpm -Uvh
to avoid file conflicts during upgrades.
- PythonInstaller:
- Skip pip-based installation inside Nix dev shells (IN_NIX_SHELL).
- Add PKGMGR_DISABLE_PYTHON_INSTALLER kill-switch.
- Make pip command resolution explicit and overridable via PKGMGR_PIP.
- Type-hint supports() and run() with RepoContext/InstallContext.
- Docker entrypoint:
- Add robust CA bundle detection for Nix, Git, Python requests and curl.
- Export NIX_SSL_CERT_FILE, SSL_CERT_FILE, REQUESTS_CA_BUNDLE and
GIT_SSL_CAINFO from a single detected CA path.
- Improve logging and section comments in the entrypoint script."
https://chatgpt.com/share/69387df8-bda0-800f-a053-aa9e2999dc84
This update introduces a configurable TEST_PATTERN variable in the Makefile,
allowing selective execution of unit, integration, and E2E tests without
modifying scripts.
Key changes:
- Add TEST_PATTERN (default: test_*.py) to Makefile and export it.
- Inject TEST_PATTERN into all test containers via `-e TEST_PATTERN=...`.
- Update test-unit.sh, test-integration.sh, and test-e2e.sh to use
`-p "$TEST_PATTERN"` instead of a hardcoded pattern.
- Ensure flexible test selection via:
make test-e2e TEST_PATTERN=test_install_pkgmgr_shallow.py
This enables fast debugging, selective test runs, and better developer
experience while keeping full compatibility with CI defaults.
https://chatgpt.com/share/69385400-2f14-800f-b093-bb03c8ef9c7f
In GitHub's Fedora-based CI containers the directory /nix may already exist
(e.g. from the base image or a previous build layer) and is often owned by
root:root. In this situation the Nix single-user installer aborts with:
"directory /nix exists, but is not writable by you"
This caused the container build to fail during `init-nix.sh`, leaving no
working `nix` binary on PATH. As a result, the runtime wrapper
(pkmgr-wrapper.sh) reported:
"[pkgmgr-wrapper] ERROR: 'nix' binary not found on PATH."
Local runs did not show the issue because a previous installation had already
created /nix with correct ownership.
This commit makes container-mode Nix initialization fully idempotent:
• If /nix does not exist → create it with owner nix:nixbld (existing logic).
• If /nix exists but has wrong owner/group → forcibly chown -R nix:nixbld.
• A warning is emitted if /nix remains non-writable after correction.
This guarantees that the Nix installer always has writable access to /nix
and prevents the installer from aborting in CI. As a result, `pkgmgr --help`
works again inside Fedora CI containers.
https://chatgpt.com/share/69384149-9dc8-800f-8148-55817ece8e21
Changes included:
• GitHub Actions workflow: rename job from 'test-unit' to 'test-container' to match intent.
• RPM packaging: replace %{_libdir}/package-manager with a fixed /usr/lib/package-manager
to avoid lib/lib64 divergence on CentOS and ensure pkgmgr + Nix flake resolution works
consistently across distros.
• Docker entrypoint: add automatic CA-bundle detection and set NIX_SSL_CERT_FILE to fix
TLS issues on CentOS ('unable to get local issuer certificate') when Nix fetches flake
inputs.
These updates stabilize container-based tests and unify the runtime environment
for Fedora, CentOS, and other distributions.
Reference:
ChatGPT conversation: https://chatgpt.com/share/6937aa72-d33c-800f-a63f-c353e92de6b3
- Add .dockerignore rules to prevent Arch package artifacts from entering the build context
- Rework Dockerfile to remove stale package artifacts before makepkg and use a dev entry script
- Introduce docker-entry-dev.sh to always rebuild pkgmgr from the mounted /src tree
- Update Makefile 'test' target to rebuild pkgmgr inside the container before running tests
- Fix predictable makepkg failures caused by residual *.pkg.tar.* files
Conversation reference: https://chatgpt.com/share/6935e6e8-f3fc-800f-a4e9-7537114f13d1
References:
- Current ChatGPT conversation: https://chatgpt.com/share/6935d6d7-0ae4-800f-988a-44a50c17ba48
- Extended discussion: https://chatgpt.com/share/6935d734-fd84-800f-9755-290902b8cee8
Summary:
This commit performs a major cleanup and modernization of the installation pipeline:
1. Introduced a new capability-detection subsystem:
- Capabilities (python-runtime, make-install, nix-flake) are detected per installer/layer.
- Installers run only when they add new capabilities.
- Prevents duplicated work such as Python installers running when Nix already provides the runtime.
2. Removed deprecated pkgmgr.yml manifest installer:
- Dependency resolution is now delegated entirely to real package managers (Nix, pip, make, distro build tools).
- Simplifies layering and avoids unnecessary recursion.
3. Reworked OS-specific installers:
- Arch PKGBUILD now uses 'makepkg --syncdeps --cleanbuild --install --noconfirm'.
- Debian installer now builds proper .deb packages via dpkg-buildpackage + installs them.
- RPM installer now builds packages using rpmbuild and installs them via rpm.
4. Switched from remote GitHub flakes to local-flake execution:
- Wrapper now executes: nix run /usr/lib/package-manager#pkgmgr
- Avoids lock-file write attempts and improves reliability in CI.
5. Added bash -i based integration test:
- Correctly sources ~/.bashrc and evaluates alias + venv activation.
- ‘pkgmgr --help’ is now printed for debugging without failing tests.
6. Updated unit tests across all installers:
- Removed references to manifest installer.
- Adjusted expectations for new behaviors (makepkg, dpkg-buildpackage, rpmbuild).
- Added capability subsystem tests.
7. Improved flake.nix packaging logic:
- The entire project source tree is copied into the runtime closure.
- pkgmgr wrapper now executes runpy inside the packaged directory.
Together, these changes create a predictable, layered, capability-driven installer pipeline with consistent behavior across Arch, Debian, RPM, Nix, and Python layers.
