Compare commits

..

33 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
eeda944b73 ci: migrate tests to reusable workflows and introduce stable-tag pipeline
- convert all test workflows to reusable workflow_call
- add central CI workflow for branches and PRs
- add mark-stable workflow triggered on main pushes
- ensure stable tag updates only after all tests succeed
- remove duplicated triggers from test workflows
`

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 13:04:44 +01:00
Kevin Veen-Birkenbach
52cfbebba4 ci: make mark-stable robust for workflow_run
- fetch workflow_run runs without head_sha filter
- match by workflow name and head_sha in jq
- keep tagging logic and permissions unchanged

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:46:42 +01:00
Kevin Veen-Birkenbach
f4385807f1 e2e: disable Nix sandbox for cross-distro flake build test
- Update test_nix_build_pkgmgr.py to invoke
    nix --option sandbox false build .#pkgmgr -L
  to avoid sandbox/permission issues in Debian and Ubuntu containers.
- Keeps the test logic identical across all distros while ensuring
  consistent flake build behaviour during E2E runs.

https://chatgpt.com/share/693aa33f-4e3c-800f-86ec-99c38a07eacb
2025-12-11 12:45:04 +01:00
Kevin Veen-Birkenbach
e9e083c9dd ci: finalize mark-stable workflow fixes
- use correct GitHub API path (/repos/.../actions/runs)
- resolve repository via workflow_run.repository.full_name
- improve logging and safe no-tag exits
- ensure correct token handling and tag update logic

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:38:12 +01:00
Kevin Veen-Birkenbach
3218b2b39f ci: fix mark-stable workflow for workflow_run events
- use workflow_run.repository.full_name for gh API queries
- expose GITHUB_TOKEN as GH_TOKEN for the GitHub CLI
- improve log messages and keep tag skipped when checks are missing or failing
2025-12-11 12:26:29 +01:00
Kevin Veen-Birkenbach
ba296a79c9 ci: fix mark-stable permissions and ignore Nix result symlink
https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:16:34 +01:00
Kevin Veen-Birkenbach
62e05e2f5b ci: tag commit as stable after full test matrix
- add mark-stable workflow that runs on workflow_run for all test pipelines
- use GitHub API to ensure all required workflows succeeded before moving the 'stable' tag
- add Nix flake.lock to pin nixpkgs for reproducible builds

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:01:21 +01:00
Kevin Veen-Birkenbach
77d8b68ba5 Add E2E Nix flake build test across all distro containers
- Introduce tests/e2e/test_nix_build_pkgmgr.py to inspect the Nix environment
  and build the pkgmgr flake inside the container started by test-e2e.sh
- Run the same commands in every distro container: nix --version, sandbox
  config, id, and nix build .#pkgmgr -L
- Print stdout/stderr and assert the flake build succeeds for easier
  cross-distro Nix debugging

https://chatgpt.com/share/693aa33f-4e3c-800f-86ec-99c38a07eacb
2025-12-11 11:55:43 +01:00
Kevin Veen-Birkenbach
bb0a801396 Fix Git safe.directory handling in E2E containers
- Mark /src and /src/.git as safe to satisfy newer Git ownership checks
- Add '*' as safe.directory for ephemeral test containers to avoid Nix flake failures

https://chatgpt.com/share/693a9e1f-1cc8-800f-9df4-90813cbb6bd5
2025-12-11 11:33:51 +01:00
Kevin Veen-Birkenbach
ee968efc4b Harden E2E test runner and fix Git safe.directory in containers
- Quote Nix store/cache volumes and distro image name in docker run
- Use strict bash flags (set -euo pipefail) inside test container
- Print distro ID robustly with fallback
- Configure /src as Git safe.directory when git is available

https://chatgpt.com/share/693a9c0e-59ec-800f-83a1-eec31bd76962
2025-12-11 11:25:11 +01:00
Kevin Veen-Birkenbach
644b2b8fa0 Align Nix Python environment and add lazy CLI import
- Switch flake package and dev shell to Python 3.11 to match pyproject
- Ensure the python-with-deps environment is preferred on PATH in nix develop
- Introduce a lightweight pkgmgr __init__ with lazy loading of pkgmgr.cli
- Avoid pulling in CLI/config dependencies on plain `import pkgmgr`, fixing
  unit test imports and PyYAML availability in the Nix test containers

https://chatgpt.com/share/693a9723-27ac-800f-a6c2-c1bcc91b7dff
2025-12-11 11:04:12 +01:00
Kevin Veen-Birkenbach
0f74907f82 flake.nix: switch to generic python3 and remove side-effects from pkgmgr package root
- Replace hardcoded python311 references with generic python3 to avoid minor
  version pinning and ensure consistent interpreter selection across systems.
- Use python.pkgs instead of python311Packages in the build pipeline.
- Update devShell to use python3.withPackages, including pip and pyyaml.
- Add Python version echo in shellHook for improved debugging.
- Remove cli re-export from src/pkgmgr/__init__.py to eliminate heavy
  side-effects during import and prevent premature config loading in tests.
2025-12-11 10:30:19 +01:00
Kevin Veen-Birkenbach
5a8b1b11de arch packaging: exclude assets from PKGBUILD rsync
Exclude the assets/ directory from the PKGBUILD rsync step to avoid
permission issues (e.g. map.png) when building the Arch package in
Docker as aur_builder.

https://chatgpt.com/share/693a8c25-4464-800f-8d5e-5c4579d78b52
2025-12-11 10:17:14 +01:00
Kevin Veen-Birkenbach
389ec40163 Refine Nix dev shell, ensure PyYAML availability, fix Python invocation, and
expose pkgmgr.cli for Python 3.13 compatibility

- Add `.nix-dev-installed` to .gitignore
- Improve flake.nix:
  * unify pkgs/pyPkgs definitions
  * provide python311.withPackages including pip + PyYAML
  * remove unused pkgmgrPkg reference from devShell
  * fix PYTHONPATH export and devShell help message
- Update unit/integration test scripts to use `python3 -m unittest`
- Add top-level pkgmgr.__init__ exposing `cli` attribute for
  pkgutil.resolve_name compatibility under Python 3.13+
2025-12-11 09:33:55 +01:00
Kevin Veen-Birkenbach
1d03055491 Removed ignore files 2025-12-11 09:07:18 +01:00
Kevin Veen-Birkenbach
7775c6d974 Refine packaging layout and Arch build paths
* Move Arch-specific ignore rules into `packaging/arch/.gitignore` and simplify top-level `.gitignore`/`.dockerignore`.
* Update Arch `PKGBUILD` to sync from the project root and drop `packaging/` from the installed tree.
* Fix OS-specific `package.sh` helpers to resolve the new `packaging/*` locations correctly for Arch, Debian/Ubuntu, Fedora, and CentOS.
2025-12-11 09:04:17 +01:00
Kevin Veen-Birkenbach
a24a819511 Restructure repo layout, wiring src/ and packaging for local and distro builds
- Add dev runner main.py that prefers local src/ over installed pkgmgr
- Move Arch/Debian/Fedora packaging files under packaging/* and update build scripts
- Adjust .gitignore/.dockerignore for new packaging paths and src/source/
- Improve config defaults discovery to support src/ layout and installed packages
- Update architecture diagram and add TODO overview for TAGS/MIRROR/SIGNING_KEY

https://chatgpt.com/share/693a76a0-e408-800f-9939-868524cbef4d
2025-12-11 08:45:07 +01:00
Kevin Veen-Birkenbach
0a6c2f2988 Release version 0.9.1 2025-12-10 22:56:04 +01:00
Kevin Veen-Birkenbach
0c90e984ad Refine setup workflows and add architecture map
- Split virgin tests into separate root and user GitHub Actions workflows
  (test-virgin-root, test-virgin-user) and adjust Arch container flows
- Introduce scripts/installation/venv-create.sh and reuse it from
  scripts/installation/main.sh with separate root/system and user/dev paths
- Add PKGMGR architecture & setup map (assets/map.png) and section in README
  with link to the up-to-date master page
- Simplify README by removing outdated Docker quickstart, usage examples,
  and AI footer
- Extend .gitignore to exclude src/source artifacts

https://chatgpt.com/share/6939bbfe-5cb0-800f-8ea8-95628dc911f5
2025-12-10 22:51:40 +01:00
Kevin Veen-Birkenbach
0a0cbbfe6d fix(init-nix): create 'nix' user with a valid shell across all distros
The init-nix.sh script previously hardcoded /usr/bin/bash as the login shell
for the 'nix' user, which exists on Arch but not on Debian. This caused the
Nix single-user installer (run via `su - nix`) to fail silently or break in
unpredictable ways on Debian-based images.

We now resolve the shell dynamically via `command -v bash` and fall back to
/bin/sh on minimal systems. This makes Nix installation deterministic across
Arch, Debian, Ubuntu, Fedora, CentOS and CI containers.

https://chatgpt.com/share/6939e97f-c93c-800f-887b-27c7e67ec46d
2025-12-10 22:43:20 +01:00
Kevin Veen-Birkenbach
15c44cd484 Removed deprecated pkgmgr.yml 2025-12-10 21:34:33 +01:00
Kevin Veen-Birkenbach
6d7ee6fc04 Fix test scripts: ensure default distro and always run via bash
- Remove Makefile inline variable export (distro=arch) and invoke scripts via bash
- Add robust default in test-unit.sh and test-integration.sh:
    : "${distro:=arch}"
- Prevent "unbound variable" errors under `set -u` when no distro is provided
2025-12-10 21:09:18 +01:00
Kevin Veen-Birkenbach
5a022db0db Use dynamic distro selection for UNIT and INTEGRATION tests
- Pass `distro=arch` from Makefile into test scripts
- Replace hardcoded "arch" references with "${distro}"
- Update test-unit.sh and test-integration.sh to use dynamic image names
- Improve log output to reflect selected distro

https://chatgpt.com/share/6939c98a-d428-800f-8bb8-cf72e80ba80c
2025-12-10 20:27:03 +01:00
Kevin Veen-Birkenbach
37ac22e0b4 test: isolate Nix store/cache per distro to fix cross-distro manifest conflicts
- Replace shared Nix volumes with distro-specific volumes
  (pkgmgr_nix_store_<distro>, pkgmgr_nix_cache_<distro>)
- Prevent incompatible profile manifest versions between Ubuntu and Debian
- Update all test scripts (unit, integration, container, e2e)
- Remove unused global Nix volume variables from Makefile
- Improve consistency of test-e2e.sh formatting and environment handling
- Add Git safe.directory configuration for mounted /src to avoid ownership warnings
2025-12-10 20:07:41 +01:00
Kevin Veen-Birkenbach
bcea440e40 Fix path and shell repo directory resolution + add unit/E2E tests
- Introduce `_resolve_repository_directory()` to unify directory lookup
  (explicit `directory` key → fallback to `get_repo_dir()` using base dir)
- Fix `pkgmgr path` to avoid KeyError and behave consistently with
  other commands using lazy directory resolution
- Fix `pkgmgr shell` to use resolved directory and correctly emit cwd
- Add full E2E tests for `pkgmgr path --all` and `pkgmgr path pkgmgr`
- Add unit tests covering:
    * explicit directory usage
    * fallback resolution via get_repo_dir()
    * empty selection behavior
    * shell command cwd resolution
    * missing shell command error handling
2025-12-10 19:47:26 +01:00
Kevin Veen-Birkenbach
6edde2d65b Release version 0.9.0 2025-12-10 18:38:10 +01:00
Kevin Veen-Birkenbach
74189c1e14 Add virgin Nix flake E2E workflow and update .gitignore
- Introduce `test-nix-flake-e2e.yml` workflow to run a full Arch-based virgin
  environment test with Nix flakes enabled and shared Docker caches
- Ensure pkgmgr self-installation and flake-based installer path are exercised
- Update .gitignore with additional build artifacts, Debian packaging files,
  and pkgmgr output directories
2025-12-10 18:37:29 +01:00
Kevin Veen-Birkenbach
b5ddf7402a Release version 0.8.0 2025-12-10 17:32:00 +01:00
Kevin Veen-Birkenbach
900224ed2e Moved installer dir 2025-12-10 17:27:26 +01:00
Kevin Veen-Birkenbach
e290043089 Refine installer capability integration tests and documentation
- Adjust install_repos integration test to patch resolve_command_for_repo
  in the pipeline module and tighten DummyInstaller overrides
- Rewrite recursive capability integration tests to focus on layer
  ordering and capability shadowing across Makefile, Python, Nix
  and OS-package installers
- Extend recursive capabilities markdown with hierarchy diagram,
  capability matrix, scenario matrix and link to the external
  setup controller schema

https://chatgpt.com/share/69399857-4d84-800f-a636-6bcd1ab5e192
2025-12-10 17:23:33 +01:00
Kevin Veen-Birkenbach
a7fd37d646 Add unit tests for install pipeline, Nix flake installer, and command resolution
https://chatgpt.com/share/69399857-4d84-800f-a636-6bcd1ab5e192
2025-12-10 16:57:02 +01:00
Kevin Veen-Birkenbach
d4b00046d3 Refine installer layering and Python/Nix integration
- Introduce explicit CLI layer model (os-packages, nix, python, makefile)
  and central InstallationPipeline to orchestrate installers.
- Move installer orchestration out of install_repos() into
  pkgmgr.actions.repository.install.pipeline, using layer precedence and
  capability tracking.
- Add pkgmgr.actions.repository.install.layers to classify commands into
  layers and compare priorities.
- Rework PythonInstaller to always use isolated environments:
  PKGMGR_PIP override → active venv → per-repo venv under ~/.venvs/<identifier>,
  avoiding system Python and PEP 668 conflicts.
- Adjust NixFlakeInstaller to install flake outputs based on repository
  identity: pkgmgr/package-manager → pkgmgr (mandatory) + default (optional),
  all other repos → default (mandatory).
- Tighten MakefileInstaller behaviour, add global
  PKGMGR_DISABLE_MAKEFILE_INSTALLER switch, and simplify install target
  detection.
- Rewrite resolve_command_for_repo() with explicit Repository typing,
  better Python package detection, Nix/PATH resolution, and a
  library-only fallback instead of raising on missing CLI.
- Update flake.nix devShell to provide python3 with pip and add pip as a
  propagated build input.
- Remove deprecated/wip repository entries from config defaults and drop
  the unused config/wip.yml.

https://chatgpt.com/share/69399157-86d8-800f-9935-1a820893e908
2025-12-10 16:26:23 +01:00
Kevin Veen-Birkenbach
545d345ea4 core(command): implement explicit command=None bypass and add unit tests
This update introduces Variant B behavior in the command resolver:

- If a repository explicitly defines the key \"command\" (even if its value is None),
  resolve_command_for_repo() treats it as authoritative and returns immediately.
  This allows library-only repositories to declare:
      command: null
  which disables CLI resolution entirely.

- As a result, Python package repositories without installed CLI entry points
  no longer trigger SystemExit during update/install flows, as long as they set
  command: null in their repo configuration.

The resolution logic is now bypassed for such repositories, skipping:
  - Python package detection (src/*/__main__.py)
  - PATH/Nix/venv binary lookup
  - main.sh/main.py fallback evaluation

A new unit test suite has been added under
  tests/unit/pkgmgr/core/command/test_resolve.py
covering:

 1) Python package without installed command → SystemExit
 2) Python package with installed command → returned correctly
 3) Script repository fallback to main.py
 4) Explicit command overrides all logic

This commit stabilizes update/install flows and ensures library-only
repositories behave as intended when no CLI command is provided.

https://chatgpt.com/share/69394a53-bc78-800f-995d-21099a68dd60
2025-12-10 11:23:57 +01:00
137 changed files with 3112 additions and 1408 deletions

View File

@@ -25,7 +25,5 @@ venv/
.DS_Store
Thumbs.db
# Arch pkg artifacts
*.pkg.tar.*
# Logs
*.log
package-manager-*

26
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,26 @@
name: CI
on:
push:
branches-ignore:
- main
pull_request:
jobs:
test-unit:
uses: ./.github/workflows/test-unit.yml
test-integration:
uses: ./.github/workflows/test-integration.yml
test-container:
uses: ./.github/workflows/test-container.yml
test-e2e:
uses: ./.github/workflows/test-e2e.yml
test-virgin-user:
uses: ./.github/workflows/test-virgin-user.yml
test-virgin-root:
uses: ./.github/workflows/test-virgin-root.yml

64
.github/workflows/mark-stable.yml vendored Normal file
View File

@@ -0,0 +1,64 @@
name: Mark stable commit
on:
push:
branches:
- main
jobs:
test-unit:
uses: ./.github/workflows/test-unit.yml
test-integration:
uses: ./.github/workflows/test-integration.yml
test-container:
uses: ./.github/workflows/test-container.yml
test-e2e:
uses: ./.github/workflows/test-e2e.yml
test-virgin-user:
uses: ./.github/workflows/test-virgin-user.yml
test-virgin-root:
uses: ./.github/workflows/test-virgin-root.yml
mark-stable:
needs:
- test-unit
- test-integration
- test-container
- test-e2e
- test-virgin-user
- test-virgin-root
runs-on: ubuntu-latest
permissions:
contents: write # to move the tag
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Move 'stable' tag to this commit
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
echo "Tagging commit $GITHUB_SHA as stable…"
# delete local tag if exists
git tag -d stable 2>/dev/null || true
# delete remote tag if exists
git push origin :refs/tags/stable || true
# create new tag on this commit
git tag stable "$GITHUB_SHA"
git push origin stable
echo "✅ Stable tag updated."

View File

@@ -1,13 +1,7 @@
name: Test OS Containers
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-container:

View File

@@ -1,13 +1,7 @@
name: Test End-To-End
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-e2e:

View File

@@ -1,13 +1,7 @@
name: Test Code Integration
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-integration:

View File

@@ -1,13 +1,7 @@
name: Test Units
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-unit:

58
.github/workflows/test-virgin-root.yml vendored Normal file
View File

@@ -0,0 +1,58 @@
name: Test Virgin Root
on:
workflow_call:
jobs:
test-virgin-root:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Show Docker version
run: docker version
- name: Virgin Arch pkgmgr flake test (root)
run: |
set -euo pipefail
echo ">>> Starting virgin ArchLinux container test (root, with shared caches)..."
docker run --rm \
-v "$PWD":/src \
-v pkgmgr_repos:/root/Repositories \
-v pkgmgr_pip_cache:/root/.cache/pip \
-w /src \
archlinux:latest \
bash -lc '
set -euo pipefail
echo ">>> Updating and upgrading Arch system..."
pacman -Syu --noconfirm git python python-pip nix >/dev/null
echo ">>> Creating isolated virtual environment for pkgmgr..."
python -m venv /tmp/pkgmgr-venv
echo ">>> Activating virtual environment..."
source /tmp/pkgmgr-venv/bin/activate
echo ">>> Upgrading pip (cached)..."
python -m pip install --upgrade pip >/dev/null
echo ">>> Installing pkgmgr from current source tree (cached pip)..."
python -m pip install /src >/dev/null
echo ">>> Enabling Nix experimental features..."
export NIX_CONFIG="experimental-features = nix-command flakes"
echo ">>> Running: pkgmgr update pkgmgr --clone-mode shallow --no-verification"
pkgmgr update pkgmgr --clone-mode shallow --no-verification
echo ">>> Running: pkgmgr version pkgmgr"
pkgmgr version pkgmgr
echo ">>> Virgin Arch (root) test completed successfully."
'

73
.github/workflows/test-virgin-user.yml vendored Normal file
View File

@@ -0,0 +1,73 @@
name: Test Virgin User
on:
workflow_call:
jobs:
test-virgin-user:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Show Docker version
run: docker version
- name: Virgin Arch pkgmgr user test (non-root with sudo)
run: |
set -euo pipefail
echo ">>> Starting virgin ArchLinux container test (non-root user with sudo)..."
docker run --rm \
-v "$PWD":/src \
archlinux:latest \
bash -lc '
set -euo pipefail
echo ">>> [root] Updating and upgrading Arch system..."
pacman -Syu --noconfirm git python python-pip sudo base-devel debugedit
echo ">>> [root] Creating non-root user dev..."
useradd -m dev
echo ">>> [root] Allowing passwordless sudo for dev..."
echo "dev ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/dev
chmod 0440 /etc/sudoers.d/dev
echo ">>> [root] Adjusting ownership of /src for dev..."
chown -R dev:dev /src
echo ">>> [root] Running pkgmgr flow as non-root user dev..."
sudo -u dev env PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 bash -lc "
set -euo pipefail
cd /src
echo \">>> [dev] Using user: \$(whoami)\"
echo \">>> [dev] Running scripts/installation/main.sh...\"
bash scripts/installation/main.sh
echo \">>> [dev] Activating venv...\"
. \"\$HOME/.venvs/pkgmgr/bin/activate\"
echo \">>> [dev] Installing pkgmgr into venv via pip...\"
python -m pip install /src >/dev/null
echo \">>> [dev] PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=\$PKGMGR_DISABLE_NIX_FLAKE_INSTALLER\"
echo \">>> [dev] Updating managed repo package-manager via pkgmgr...\"
pkgmgr update pkgmgr --clone-mode shallow --no-verification
echo \">>> [dev] PATH:\"
echo \"\$PATH\"
echo \">>> [dev] which pkgmgr:\"
which pkgmgr || echo \">>> [dev] pkgmgr not found in PATH\"
echo \">>> [dev] Running: pkgmgr version pkgmgr\"
pkgmgr version pkgmgr
"
echo ">>> [root] Container flow finished."
'

13
.gitignore vendored
View File

@@ -1,9 +1,6 @@
# Prevents unwanted files from being committed to version control.
# Custom Config file
config/config.yaml
# Python bytecode
__pycache__/
*.pyc
@@ -17,6 +14,7 @@ venv/
dist/
build/*
*.egg-info/
package-manager-*
# Editor files
.vscode/
@@ -28,14 +26,9 @@ Thumbs.db
# Nix Cache to speed up tests
.nix/
.nix-dev-installed
# Ignore logs
*.log
package-manager-*
# debian
debian/package-manager/
debian/debhelper-build-stamp
debian/files
debian/.debhelper/
debian/package-manager.substvars
result

View File

@@ -1,3 +1,28 @@
## [0.9.1] - 2025-12-10
* * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
## [0.9.0] - 2025-12-10
* Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
## [0.8.0] - 2025-12-10
* **v0.7.15 — Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
## [0.7.14] - 2025-12-10
* Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.

View File

@@ -2,12 +2,6 @@
test build build-no-cache test-unit test-e2e test-integration \
test-container
# ------------------------------------------------------------
# Local Nix cache directories in the repo
# ------------------------------------------------------------
NIX_STORE_VOLUME := pkgmgr_nix_store
NIX_CACHE_VOLUME := pkgmgr_nix_cache
# ------------------------------------------------------------
# Distro list and base images
# (kept for documentation/reference; actual build logic is in scripts/build)
@@ -68,8 +62,8 @@ test-container: build-missing
build-missing:
@bash scripts/build/build-image-missing.sh
# Combined test target for local + CI (unit + e2e + integration)
test: test-container test-unit test-e2e test-integration
# Combined test target for local + CI (unit + integration + e2e)
test: test-container test-unit test-integration test-e2e
# ------------------------------------------------------------
# System install (native packages, calls scripts/installation/run-package.sh)

View File

@@ -24,6 +24,15 @@
- **Custom Aliases:**
Generate and manage custom aliases for easy command invocation.
## Architecture & Setup Map 🗺️
The following diagram provides a full overview of PKGMGRs package structure,
installation layers, and setup controller flow:
![PKGMGR Architecture](assets/map.png)
**Diagram status:** *Stand: 11. Dezember 2025*
**Always-up-to-date version:** https://s.veen.world/pkgmgrmp
## Installation ⚙️
@@ -51,55 +60,6 @@ The `make setup` command will:
- Install required packages from `requirements.txt`.
- Execute `python main.py install` to complete the installation.
## Docker Quickstart 🐳
Alternatively to installing locally, you can use Docker: build the image with
```bash
docker build --no-cache -t pkgmgr .
```
or alternativ pull it via
```bash
docker pull kevinveenbirkenbach/pkgmgr:latest
```
and then run
```bash
docker run --rm pkgmgr --help
```
## Usage 📖
Run the script with different commands. For example:
- **Install all packages:**
```bash
pkgmgr install --all
```
- **Pull updates for a specific repository:**
```bash
pkgmgr pull pkgmgr
```
- **Commit changes with extra Git parameters:**
```bash
pkgmgr commit pkgmgr -- -m "Your commit message"
```
- **List all configured packages:**
```bash
pkgmgr config show
```
- **Manage configuration:**
```bash
pkgmgr config init
pkgmgr config add
pkgmgr config edit
pkgmgr config delete <identifier>
pkgmgr config ignore <identifier> --set true
```
## License 📄
This project is licensed under the MIT License.
@@ -108,9 +68,3 @@ This project is licensed under the MIT License.
Kevin Veen-Birkenbach
[https://www.veen.world](https://www.veen.world)
---
**Repository:** [github.com/kevinveenbirkenbach/package-manager](https://github.com/kevinveenbirkenbach/package-manager)
*Created with AI 🤖 - [View conversation](https://chatgpt.com/share/67c728c4-92d0-800f-8945-003fa9bf27c6)*

7
TODO.md Normal file
View File

@@ -0,0 +1,7 @@
# to-dos
For the following checkout the implementation map:
- Implement TAGS
- Implement MIRROR
- Implement SIGNING_KEY

BIN
assets/map.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

View File

@@ -380,17 +380,6 @@ repositories:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
alias: infinito-presentation
description: This repository contains a Infinito.Nexus presentation designed for customers, end-users, investors, developers, and administrators, offering tailored content and insights for each group.
homepage: https://github.com/kevinveenbirkenbach/infinito-presentation
provider: github.com
repository: infinito-presentation
verified:
gpg_keys:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
description: A lightweight Python utility to generate dynamic color schemes from a single base color. Provides HSL-based color transformations for theming, UI design, and CSS variable generation. Optimized for integration in Python projects, Flask applications, and Ansible roles.
homepage: https://github.com/kevinveenbirkenbach/colorscheme-generator
@@ -599,17 +588,6 @@ repositories:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
desciption: Infinito Inventory Builder — a containerized web application that dynamically generates Ansible inventory files from invokable Infinito.Nexus roles through an interactive, browser-based interface.
homepage: https://github.com/kevinveenbirkenbach/infinito-inventory-builder
alias: invbuild
provider: github.com
repository: infinito-inventory-builder
verified:
gpg_keys:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
desciption: A simple Python CLI tool to safely rename Linux user accounts using usermod — including home directory migration and validation checks.
homepage: https://github.com/kevinveenbirkenbach/user-rename

View File

@@ -1,7 +0,0 @@
- account: kevinveenbirkenbach
alias: gkfdrtdtcntr
provider: github.com
repository: federated-to-central-social-network-bridge
verified:
gpg_keys:
- 44D8F11FD62F878E

27
flake.lock generated Normal file
View File

@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1765186076,
"narHash": "sha256-hM20uyap1a0M9d344I692r+ik4gTMyj60cQWO+hAYP8=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "addf7cf5f383a3101ecfba091b98d0a1263dc9b8",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

View File

@@ -26,12 +26,17 @@
packages = forAllSystems (system:
let
pkgs = nixpkgs.legacyPackages.${system};
# Single source of truth for pkgmgr: Python 3.11
# - Matches pyproject.toml: requires-python = ">=3.11"
# - Uses python311Packages so that PyYAML etc. are available
python = pkgs.python311;
pyPkgs = pkgs.python311Packages;
in
rec {
pkgmgr = pyPkgs.buildPythonApplication {
pname = "package-manager";
version = "0.7.14";
version = "0.9.1";
# Use the git repo as source
src = ./.;
@@ -45,18 +50,17 @@
pyPkgs.wheel
];
# Runtime dependencies (matches [project.dependencies])
# Runtime dependencies (matches [project.dependencies] in pyproject.toml)
propagatedBuildInputs = [
pyPkgs.pyyaml
# Add more here if needed, e.g.:
# pyPkgs.click
# pyPkgs.rich
pyPkgs.pip
];
doCheck = false;
pythonImportsCheck = [ "pkgmgr" ];
};
default = pkgmgr;
}
);
@@ -67,23 +71,42 @@
devShells = forAllSystems (system:
let
pkgs = nixpkgs.legacyPackages.${system};
pkgmgrPkg = self.packages.${system}.pkgmgr;
ansiblePkg =
if pkgs ? ansible-core then pkgs.ansible-core
else pkgs.ansible;
# Use the same Python version as the package (3.11)
python = pkgs.python311;
pythonWithDeps = python.withPackages (ps: [
ps.pip
ps.pyyaml
]);
in
{
default = pkgs.mkShell {
buildInputs = [
pkgmgrPkg
pythonWithDeps
pkgs.git
ansiblePkg
];
shellHook = ''
# Ensure our Python with dependencies is preferred on PATH
export PATH=${pythonWithDeps}/bin:$PATH
# Ensure src/ layout is importable:
# pkgmgr lives in ./src/pkgmgr
export PYTHONPATH="$PWD/src:${PYTHONPATH:-}"
# Also add repo root in case tools/tests rely on it
export PYTHONPATH="$PWD:$PYTHONPATH"
echo "Entered pkgmgr development shell for ${system}"
echo "pkgmgr CLI is available via the flake build"
echo "Python used in this shell:"
python --version
echo "pkgmgr CLI (from source) is available via:"
echo " python -m pkgmgr.cli --help"
'';
};
}

View File

@@ -1,4 +1,12 @@
#!/usr/bin/env python3
import sys
from pathlib import Path
# Ensure local src/ overrides installed package
ROOT = Path(__file__).resolve().parent
SRC = ROOT / "src"
if SRC.is_dir():
sys.path.insert(0, str(SRC))
from pkgmgr.cli import main

6
packaging/arch/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# Arch pkg artifacts
*.pkg.tar.*
*.log
package-manager-*
src/
pkg/

View File

@@ -1,7 +1,7 @@
# Maintainer: Kevin Veen-Birkenbach <info@veen.world>
pkgname=package-manager
pkgver=0.7.14
pkgver=0.9.1
pkgrel=1
pkgdesc="Local-flake wrapper for Kevin's package-manager (Nix-based)."
arch=('any')
@@ -15,7 +15,7 @@ makedepends=('rsync')
install=${pkgname}.install
# Local source checkout — avoids the tarball requirement.
# This assumes you build the package from inside the main project repository.
# We build from the project root (two levels above packaging/arch/).
source=()
sha256sums=()
@@ -24,12 +24,18 @@ _srcdir_name="source"
prepare() {
mkdir -p "$srcdir/$_srcdir_name"
local project_root
project_root="$(cd "$startdir/../.." && pwd)"
rsync -a \
--exclude=".git" \
--exclude=".github" \
--exclude="pkg" \
--exclude="srcpkg" \
"$startdir/" "$srcdir/$_srcdir_name/"
--exclude="packaging" \
--exclude="assets" \
"$project_root/" "$srcdir/$_srcdir_name/"
}
build() {
@@ -62,7 +68,8 @@ package() {
"$pkgdir/usr/lib/package-manager/PKGBUILD" \
"$pkgdir/usr/lib/package-manager/Dockerfile" \
"$pkgdir/usr/lib/package-manager/debian" \
"$pkgdir/usr/lib/package-manager/packaging" \
"$pkgdir/usr/lib/package-manager/.gitignore" \
"$pkgdir/usr/lib/package-manager/__pycache__" \
"$pkgdir/usr/lib/package-manager/.gitkeep"
"$pkgdir/usr/lib/package-manager/.gitkeep" || true
}

6
packaging/debian/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# debian
package-manager/
debhelper-build-stamp
files
.debhelper/
package-manager.substvars

View File

@@ -1,3 +1,31 @@
package-manager (0.9.1-1) unstable; urgency=medium
* * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 22:56:01 +0100
package-manager (0.9.0-1) unstable; urgency=medium
* Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 18:38:07 +0100
package-manager (0.8.0-1) unstable; urgency=medium
* **v0.7.15 — Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 17:31:57 +0100
package-manager (0.7.14-1) unstable; urgency=medium
* Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.

View File

@@ -1,5 +1,5 @@
Name: package-manager
Version: 0.7.14
Version: 0.9.1
Release: 1%{?dist}
Summary: Wrapper that runs Kevin's package-manager via Nix flake
@@ -77,6 +77,25 @@ echo ">>> package-manager removed. Nix itself was not removed."
/usr/lib/package-manager/
%changelog
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.9.1-1
- * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.9.0-1
- Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.8.0-1
- **v0.7.15 Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.7.14-1
- Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.

View File

@@ -1,7 +0,0 @@
version: 1
author: "Kevin Veen-Birkenbach"
url: "https://github.com/kevinveenbirkenbach/package-manager"
description: "A configurable Python-based package manager for managing multiple repositories via Bash."
dependencies: []

View File

@@ -1,294 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Repository installation pipeline for pkgmgr.
This module orchestrates the installation of repositories by:
1. Ensuring the repository directory exists (cloning if necessary).
2. Verifying the repository according to the configured policies.
3. Creating executable links using create_ink(), after resolving the
appropriate command via resolve_command_for_repo().
4. Running a sequence of modular installer components that handle
specific technologies or manifests (PKGBUILD, Nix flakes, Python
via pyproject.toml, Makefile, OS-specific package metadata).
The goal is to keep this file thin and delegate most logic to small,
focused installer classes.
"""
import os
from typing import List, Dict, Any
from pkgmgr.core.repository.identifier import get_repo_identifier
from pkgmgr.core.repository.dir import get_repo_dir
from pkgmgr.core.command.ink import create_ink
from pkgmgr.core.repository.verify import verify_repository
from pkgmgr.actions.repository.clone import clone_repos
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.core.command.resolve import resolve_command_for_repo
# Installer implementations
from pkgmgr.actions.repository.install.installers.os_packages import (
ArchPkgbuildInstaller,
DebianControlInstaller,
RpmSpecInstaller,
)
from pkgmgr.actions.repository.install.installers.nix_flake import NixFlakeInstaller
from pkgmgr.actions.repository.install.installers.python import PythonInstaller
from pkgmgr.actions.repository.install.installers.makefile import MakefileInstaller
# Layering:
# 1) OS packages: PKGBUILD / debian/control / RPM spec → os-deps.*
# 2) Nix flakes (flake.nix) → e.g. python-runtime, make-install
# 3) Python (pyproject.toml) → e.g. python-runtime, make-install
# 4) Makefile fallback → e.g. make-install
INSTALLERS = [
ArchPkgbuildInstaller(), # Arch
DebianControlInstaller(), # Debian/Ubuntu
RpmSpecInstaller(), # Fedora/RHEL/CentOS
NixFlakeInstaller(), # flake.nix (Nix layer)
PythonInstaller(), # pyproject.toml
MakefileInstaller(), # generic 'make install'
]
def _ensure_repo_dir(
repo: Dict[str, Any],
repositories_base_dir: str,
all_repos: List[Dict[str, Any]],
preview: bool,
no_verification: bool,
clone_mode: str,
identifier: str,
) -> str:
"""
Ensure the repository directory exists. If not, attempt to clone it.
Returns the repository directory path or an empty string if cloning failed.
"""
repo_dir = get_repo_dir(repositories_base_dir, repo)
if not os.path.exists(repo_dir):
print(f"Repository directory '{repo_dir}' does not exist. Cloning it now...")
clone_repos(
[repo],
repositories_base_dir,
all_repos,
preview,
no_verification,
clone_mode,
)
if not os.path.exists(repo_dir):
print(f"Cloning failed for repository {identifier}. Skipping installation.")
return ""
return repo_dir
def _verify_repo(
repo: Dict[str, Any],
repo_dir: str,
no_verification: bool,
identifier: str,
) -> bool:
"""
Verify the repository using verify_repository().
Returns True if installation should proceed, False if it should be skipped.
"""
verified_info = repo.get("verified")
verified_ok, errors, commit_hash, signing_key = verify_repository(
repo,
repo_dir,
mode="local",
no_verification=no_verification,
)
if not no_verification and verified_info and not verified_ok:
print(f"Warning: Verification failed for {identifier}:")
for err in errors:
print(f" - {err}")
choice = input("Proceed with installation? (y/N): ").strip().lower()
if choice != "y":
print(f"Skipping installation for {identifier}.")
return False
return True
def _create_context(
repo: Dict[str, Any],
identifier: str,
repo_dir: str,
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Dict[str, Any]],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> RepoContext:
"""
Build a RepoContext for the given repository and parameters.
"""
return RepoContext(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
def install_repos(
selected_repos: List[Dict[str, Any]],
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Dict[str, Any]],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> None:
"""
Install repositories by creating symbolic links and processing standard
manifest files (PKGBUILD, flake.nix, Python manifests, Makefile, etc.)
via dedicated installer components.
Any installer failure (SystemExit) is treated as fatal and will abort
the current installation.
"""
for repo in selected_repos:
identifier = get_repo_identifier(repo, all_repos)
repo_dir = _ensure_repo_dir(
repo=repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
no_verification=no_verification,
clone_mode=clone_mode,
identifier=identifier,
)
if not repo_dir:
continue
if not _verify_repo(
repo=repo,
repo_dir=repo_dir,
no_verification=no_verification,
identifier=identifier,
):
continue
ctx = _create_context(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
# ------------------------------------------------------------
# Resolve the command for this repository before creating the link.
# If no command is resolved, no link will be created.
# ------------------------------------------------------------
resolved_command = resolve_command_for_repo(
repo=repo,
repo_identifier=identifier,
repo_dir=repo_dir,
)
if resolved_command:
repo["command"] = resolved_command
else:
repo.pop("command", None)
# ------------------------------------------------------------
# Create the symlink using create_ink (if a command is set).
# ------------------------------------------------------------
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
# Track which logical capabilities have already been provided by
# earlier installers for this repository. This allows us to skip
# installers that would only duplicate work (e.g. Python runtime
# already provided by Nix flake → skip pyproject/Makefile).
provided_capabilities: set[str] = set()
# Run all installers that support this repository, but only if they
# provide at least one capability that is not yet satisfied.
for installer in INSTALLERS:
if not installer.supports(ctx):
continue
caps = installer.discover_capabilities(ctx)
# If the installer declares capabilities and *all* of them are
# already provided, we can safely skip it.
if caps and caps.issubset(provided_capabilities):
if not quiet:
print(
f"Skipping installer {installer.__class__.__name__} "
f"for {identifier} capabilities {caps} already provided."
)
continue
# ------------------------------------------------------------
# Debug output + clear error if an installer fails
# ------------------------------------------------------------
if not quiet:
print(
f"[pkgmgr] Running installer {installer.__class__.__name__} "
f"for {identifier} in '{repo_dir}' "
f"(new capabilities: {caps or ''})..."
)
try:
installer.run(ctx)
except SystemExit as exc:
exit_code = exc.code if isinstance(exc.code, int) else str(exc.code)
print(
f"[ERROR] Installer {installer.__class__.__name__} failed "
f"for repository {identifier} (dir: {repo_dir}) "
f"with exit code {exit_code}."
)
print(
"[ERROR] This usually means an underlying command failed "
"(e.g. 'make install', 'nix build', 'pip install', ...)."
)
print(
"[ERROR] Check the log above for the exact command output. "
"You can also run this repository in isolation via:\n"
f" pkgmgr install {identifier} --clone-mode shallow --no-verification"
)
# Re-raise so that CLI/tests fail clearly,
# but now with much more context.
raise
# Only merge capabilities if the installer succeeded
provided_capabilities.update(caps)

View File

@@ -1,19 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer package for pkgmgr.
This exposes all installer classes so users can import them directly from
pkgmgr.actions.repository.install.installers.
"""
from pkgmgr.actions.repository.install.installers.base import BaseInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.nix_flake import NixFlakeInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.python import PythonInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.makefile import MakefileInstaller # noqa: F401
# OS-specific installers
from pkgmgr.actions.repository.install.installers.os_packages.arch_pkgbuild import ArchPkgbuildInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.os_packages.debian_control import DebianControlInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.os_packages.rpm_spec import RpmSpecInstaller # noqa: F401

View File

@@ -1,93 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer that triggers `make install` if a Makefile is present and
the Makefile actually defines an 'install' target.
This is useful for repositories that expose a standard Makefile-based
installation step.
"""
import os
import re
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
class MakefileInstaller(BaseInstaller):
"""Run `make install` if a Makefile with an 'install' target exists."""
# Logical layer name, used by capability matchers.
layer = "makefile"
MAKEFILE_NAME = "Makefile"
def supports(self, ctx: RepoContext) -> bool:
"""Return True if a Makefile exists in the repository directory."""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
return os.path.exists(makefile_path)
def _has_install_target(self, makefile_path: str) -> bool:
"""
Check whether the Makefile defines an 'install' target.
We treat the presence of a real install target as either:
- a line starting with 'install:' (optionally preceded by whitespace), or
- a .PHONY line that lists 'install' as one of the targets.
"""
try:
with open(makefile_path, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
except OSError:
# If we cannot read the Makefile for some reason, assume no target.
return False
# install: ...
if re.search(r"^\s*install\s*:", content, flags=re.MULTILINE):
return True
# .PHONY: ... install ...
if re.search(r"^\s*\.PHONY\s*:\s*.*\binstall\b", content, flags=re.MULTILINE):
return True
return False
def run(self, ctx: RepoContext) -> None:
"""
Execute `make install` in the repository directory, but only if an
'install' target is actually defined in the Makefile.
Any failure in `make install` is treated as a fatal error and will
propagate as SystemExit from run_command().
"""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
if not os.path.exists(makefile_path):
# Should normally not happen if supports() was checked before,
# but keep this guard for robustness.
if not ctx.quiet:
print(
f"[pkgmgr] Makefile '{makefile_path}' not found, "
"skipping make install."
)
return
if not self._has_install_target(makefile_path):
if not ctx.quiet:
print(
"[pkgmgr] Skipping Makefile install: no 'install' target "
f"found in {makefile_path}."
)
return
if not ctx.quiet:
print(
f"[pkgmgr] Running 'make install' in {ctx.repo_dir} "
"(install target detected in Makefile)."
)
cmd = "make install"
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -1,127 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer for Python projects based on pyproject.toml.
Strategy:
- Determine a pip command in this order:
1. $PKGMGR_PIP (explicit override, e.g. ~/.venvs/pkgmgr/bin/pip)
2. sys.executable -m pip (current interpreter)
3. "pip" from PATH as last resort
- If pyproject.toml exists: pip install .
All installation failures are treated as fatal errors (SystemExit),
except when we explicitly skip the installer:
- If IN_NIX_SHELL is set, we assume Python is managed by Nix and
skip this installer entirely.
- If PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set, the installer is
globally disabled (useful for CI or debugging).
"""
from __future__ import annotations
import os
import sys
from typing import TYPE_CHECKING
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install import InstallContext
class PythonInstaller(BaseInstaller):
"""Install Python projects and dependencies via pip."""
# Logical layer name, used by capability matchers.
layer = "python"
def _in_nix_shell(self) -> bool:
"""
Return True if we appear to be running inside a Nix dev shell.
Nix sets IN_NIX_SHELL in `nix develop` environments. In that case
the Python environment is already provided by Nix, so we must not
attempt an additional pip-based installation.
"""
return bool(os.environ.get("IN_NIX_SHELL"))
def supports(self, ctx: "RepoContext") -> bool:
"""
Return True if this installer should handle the given repository.
Only pyproject.toml is supported as the single source of truth
for Python dependencies and packaging metadata.
The installer is *disabled* when:
- IN_NIX_SHELL is set (Python managed by Nix dev shell), or
- PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set.
"""
# 1) Skip in Nix dev shells Python is managed by the flake/devShell.
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected; skipping PythonInstaller. "
"Python runtime is provided by the Nix dev shell."
)
return False
# 2) Optional global kill-switch.
if os.environ.get("PKGMGR_DISABLE_PYTHON_INSTALLER") == "1":
print(
"[INFO] PKGMGR_DISABLE_PYTHON_INSTALLER=1 "
"PythonInstaller is disabled."
)
return False
repo_dir = ctx.repo_dir
return os.path.exists(os.path.join(repo_dir, "pyproject.toml"))
def _pip_cmd(self) -> str:
"""
Resolve the pip command to use.
Order:
1) PKGMGR_PIP (explicit override)
2) sys.executable -m pip
3) plain "pip"
"""
explicit = os.environ.get("PKGMGR_PIP", "").strip()
if explicit:
return explicit
if sys.executable:
return f"{sys.executable} -m pip"
return "pip"
def run(self, ctx: "InstallContext") -> None:
"""
Install Python project defined via pyproject.toml.
Any pip failure is propagated as SystemExit.
"""
# Extra guard in case run() is called directly without supports().
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected in PythonInstaller.run(); "
"skipping pip-based installation."
)
return
if not self.supports(ctx): # type: ignore[arg-type]
return
pip_cmd = self._pip_cmd()
pyproject = os.path.join(ctx.repo_dir, "pyproject.toml")
if os.path.exists(pyproject):
print(
f"pyproject.toml found in {ctx.identifier}, "
f"installing Python project..."
)
cmd = f"{pip_cmd} install ."
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -1,113 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Command resolver for repositories.
This module determines the correct command to expose via symlink.
It implements the following priority:
1. Explicit command in repo config → command
2. System package manager binary (/usr/...) → NO LINK (respect OS)
3. Nix profile binary (~/.nix-profile/bin/<id>) → command
4. Python / non-system console script on PATH → command
5. Fallback: repository's main.sh or main.py → command
6. If nothing is available → raise error
The actual symlink creation is handled by create_ink(). This resolver
only decides *what* should be used as the entrypoint, or whether no
link should be created at all.
"""
import os
import shutil
from typing import Optional
def resolve_command_for_repo(repo, repo_identifier: str, repo_dir: str) -> Optional[str]:
"""
Determine the command for this repository.
Returns:
str → path to the command (a symlink should be created)
None → do NOT create a link (e.g. system package already provides it)
On total failure (no suitable command found at any layer), this function
raises SystemExit with a descriptive error message.
"""
# ------------------------------------------------------------
# 1. Explicit command defined by repository config
# ------------------------------------------------------------
explicit = repo.get("command")
if explicit:
return explicit
home = os.path.expanduser("~")
def is_executable(path: str) -> bool:
return os.path.exists(path) and os.access(path, os.X_OK)
# ------------------------------------------------------------
# 2. System package manager binary via PATH
#
# If the binary lives under /usr/, we treat it as a system-managed
# package (e.g. installed via pacman/apt/yum). In that case, pkgmgr
# does NOT create a link at all and defers entirely to the OS.
# ------------------------------------------------------------
path_candidate = shutil.which(repo_identifier)
system_binary: Optional[str] = None
non_system_binary: Optional[str] = None
if path_candidate:
if path_candidate.startswith("/usr/"):
system_binary = path_candidate
else:
non_system_binary = path_candidate
if system_binary:
# Respect system package manager: do not create a link.
if repo.get("debug", False):
print(
f"[pkgmgr] System binary for '{repo_identifier}' found at "
f"{system_binary}; no symlink will be created."
)
return None
# ------------------------------------------------------------
# 3. Nix profile binary (~/.nix-profile/bin/<identifier>)
# ------------------------------------------------------------
nix_candidate = os.path.join(home, ".nix-profile", "bin", repo_identifier)
if is_executable(nix_candidate):
return nix_candidate
# ------------------------------------------------------------
# 4. Python / non-system console script on PATH
#
# Here we reuse the non-system PATH candidate (e.g. from a venv or
# a user-local install like ~/.local/bin). This is treated as a
# valid command target.
# ------------------------------------------------------------
if non_system_binary and is_executable(non_system_binary):
return non_system_binary
# ------------------------------------------------------------
# 5. Fallback: main.sh / main.py inside the repository
# ------------------------------------------------------------
main_sh = os.path.join(repo_dir, "main.sh")
main_py = os.path.join(repo_dir, "main.py")
if is_executable(main_sh):
return main_sh
if is_executable(main_py) or os.path.exists(main_py):
return main_py
# ------------------------------------------------------------
# 6. Nothing found → treat as a hard error
# ------------------------------------------------------------
raise SystemExit(
f"No executable command could be resolved for repository '{repo_identifier}'. "
"No explicit 'command' configured, no system-managed binary under /usr/, "
"no Nix profile binary, no non-system console script on PATH, and no "
"main.sh/main.py found in the repository."
)

View File

@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "package-manager"
version = "0.7.14"
version = "0.9.1"
description = "Kevin's package-manager tool (pkgmgr)"
readme = "README.md"
requires-python = ">=3.11"
@@ -39,13 +39,13 @@ pkgmgr = "pkgmgr.cli:main"
# -----------------------------
# setuptools configuration
# -----------------------------
# We use find_packages(), not a fixed list,
# and explicitly include pkgmgr* and config*
# Source layout: all packages live under "src/"
[tool.setuptools]
package-dir = { "" = "src", "config" = "config" }
[tool.setuptools.packages.find]
where = ["."]
where = ["src", "."]
include = ["pkgmgr*", "config*"]
# Ensure defaults.yaml is shipped inside wheels & nix builds
[tool.setuptools.package-data]
"config" = ["defaults.yaml"]

View File

@@ -94,7 +94,15 @@ if [[ "${IN_CONTAINER}" -eq 1 && "${EUID:-0}" -eq 0 ]]; then
# Ensure "nix" user (home at /home/nix)
if ! id nix >/dev/null 2>&1; then
echo "[init-nix] Creating user 'nix'..."
useradd -m -r -g nixbld -s /usr/bin/bash nix
# Resolve a valid shell path across distros:
# - Debian/Ubuntu: /bin/bash
# - Arch: /usr/bin/bash (often symlinked)
# Fall back to /bin/sh on ultra-minimal systems.
BASH_SHELL="$(command -v bash || true)"
if [[ -z "${BASH_SHELL}" ]]; then
BASH_SHELL="/bin/sh"
fi
useradd -m -r -g nixbld -s "${BASH_SHELL}" nix
fi
# Ensure /nix exists and is writable by the "nix" user.

View File

@@ -3,10 +3,21 @@ set -euo pipefail
echo "[arch/package] Building Arch package (makepkg --nodeps)..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
PKG_DIR="${PROJECT_ROOT}/packaging/arch"
if [[ ! -f "${PKG_DIR}/PKGBUILD" ]]; then
echo "[arch/package] ERROR: PKGBUILD not found in ${PKG_DIR}"
exit 1
fi
cd "${PKG_DIR}"
if id aur_builder >/dev/null 2>&1; then
echo "[arch/package] Using 'aur_builder' user for makepkg..."
chown -R aur_builder:aur_builder "$(pwd)"
su aur_builder -c "cd '$(pwd)' && rm -f package-manager-*.pkg.tar.* && makepkg --noconfirm --clean --nodeps"
chown -R aur_builder:aur_builder "${PKG_DIR}"
su aur_builder -c "cd '${PKG_DIR}' && rm -f package-manager-*.pkg.tar.* && makepkg --noconfirm --clean --nodeps"
else
echo "[arch/package] WARNING: user 'aur_builder' not found, running makepkg as current user..."
rm -f package-manager-*.pkg.tar.*

View File

@@ -4,8 +4,17 @@ set -euo pipefail
echo "[centos/package] Setting up rpmbuild directories..."
mkdir -p /root/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
SPEC_PATH="${PROJECT_ROOT}/packaging/fedora/package-manager.spec"
if [[ ! -f "${SPEC_PATH}" ]]; then
echo "[centos/package] ERROR: SPEC file not found: ${SPEC_PATH}"
exit 1
fi
echo "[centos/package] Extracting version from package-manager.spec..."
version="$(grep -E '^Version:' package-manager.spec | awk '{print $2}')"
version="$(grep -E '^Version:' "${SPEC_PATH}" | awk '{print $2}')"
if [[ -z "${version}" ]]; then
echo "ERROR: Version missing!"
exit 1
@@ -15,13 +24,13 @@ srcdir="package-manager-${version}"
echo "[centos/package] Preparing source tree: ${srcdir}"
rm -rf "/tmp/${srcdir}"
mkdir -p "/tmp/${srcdir}"
cp -a . "/tmp/${srcdir}/"
cp -a "${PROJECT_ROOT}/." "/tmp/${srcdir}/"
echo "[centos/package] Creating source tarball..."
tar czf "/root/rpmbuild/SOURCES/${srcdir}.tar.gz" -C /tmp "${srcdir}"
echo "[centos/package] Copying SPEC..."
cp package-manager.spec /root/rpmbuild/SPECS/
cp "${SPEC_PATH}" /root/rpmbuild/SPECS/
echo "[centos/package] Running rpmbuild..."
cd /root/rpmbuild/SPECS

View File

@@ -3,6 +3,25 @@ set -euo pipefail
echo "[debian/package] Building Debian package..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
BUILD_ROOT="/tmp/package-manager-debian-build"
rm -rf "${BUILD_ROOT}"
mkdir -p "${BUILD_ROOT}"
echo "[debian/package] Syncing project sources to ${BUILD_ROOT}..."
rsync -a \
--exclude 'packaging/debian' \
"${PROJECT_ROOT}/" "${BUILD_ROOT}/"
echo "[debian/package] Overlaying debian/ metadata from packaging/debian..."
mkdir -p "${BUILD_ROOT}/debian"
cp -a "${PROJECT_ROOT}/packaging/debian/." "${BUILD_ROOT}/debian/"
cd "${BUILD_ROOT}"
echo "[debian/package] Running dpkg-buildpackage..."
dpkg-buildpackage -us -uc -b
echo "[debian/package] Installing generated DEB package..."

View File

@@ -4,8 +4,17 @@ set -euo pipefail
echo "[fedora/package] Setting up rpmbuild directories..."
mkdir -p /root/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
SPEC_PATH="${PROJECT_ROOT}/packaging/fedora/package-manager.spec"
if [[ ! -f "${SPEC_PATH}" ]]; then
echo "[fedora/package] ERROR: SPEC file not found: ${SPEC_PATH}"
exit 1
fi
echo "[fedora/package] Extracting version from package-manager.spec..."
version="$(grep -E '^Version:' package-manager.spec | awk '{print $2}')"
version="$(grep -E '^Version:' "${SPEC_PATH}" | awk '{print $2}')"
if [[ -z "${version}" ]]; then
echo "ERROR: Version missing!"
exit 1
@@ -15,13 +24,13 @@ srcdir="package-manager-${version}"
echo "[fedora/package] Preparing source tree: ${srcdir}"
rm -rf "/tmp/${srcdir}"
mkdir -p "/tmp/${srcdir}"
cp -a . "/tmp/${srcdir}/"
cp -a "${PROJECT_ROOT}/." "/tmp/${srcdir}/"
echo "[fedora/package] Creating source tarball..."
tar czf "/root/rpmbuild/SOURCES/${srcdir}.tar.gz" -C /tmp "${srcdir}"
echo "[fedora/package] Copying SPEC..."
cp package-manager.spec /root/rpmbuild/SPECS/
cp "${SPEC_PATH}" /root/rpmbuild/SPECS/
echo "[fedora/package] Running rpmbuild..."
cd /root/rpmbuild/SPECS

View File

@@ -4,20 +4,22 @@ set -euo pipefail
# ------------------------------------------------------------
# main.sh
#
# Developer setup entrypoint.
# Developer / system setup entrypoint.
#
# Responsibilities:
# - If inside a Nix shell (IN_NIX_SHELL=1):
# * Skip venv creation and dependency installation
# * Run `python3 main.py install`
# - Otherwise:
# - If running as root (EUID=0):
# * Run system-level installer (run-package.sh)
# - Otherwise (normal user):
# * Create ~/.venvs/pkgmgr virtual environment if missing
# * Install Python dependencies into that venv
# * Append auto-activation to ~/.bashrc and ~/.zshrc
# * Run `main.py install` using the venv Python
# ------------------------------------------------------------
echo "[installation/main] Starting developer setup..."
echo "[installation/main] Starting setup..."
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "${PROJECT_ROOT}"
@@ -26,20 +28,34 @@ VENV_DIR="${HOME}/.venvs/pkgmgr"
RC_LINE='if [ -d "${HOME}/.venvs/pkgmgr" ]; then . "${HOME}/.venvs/pkgmgr/bin/activate"; if [ -n "${PS1:-}" ]; then echo "Global Python virtual environment '\''~/.venvs/pkgmgr'\'' activated."; fi; fi'
# ------------------------------------------------------------
# Nix shell mode: do not touch venv, only run main.py install
# 1) Nix shell mode: do not touch venv, only run main.py install
# ------------------------------------------------------------
if [[ -n "${IN_NIX_SHELL:-}" ]]; then
echo "[installation/main] Nix shell detected (IN_NIX_SHELL=1)."
echo "[installation/main] Skipping virtualenv creation and dependency installation."
echo "[installation/main] Running main.py install via system python3..."
python3 main.py install
echo "[installation/main] Developer setup finished (Nix mode)."
echo "[installation/main] Setup finished (Nix mode)."
exit 0
fi
# ------------------------------------------------------------
# Normal host mode: create/update venv and run main.py install
# 2) Root mode: system / distro-level installation
# ------------------------------------------------------------
if [[ "${EUID:-$(id -u)}" -eq 0 ]]; then
echo "[installation/main] Running as root (EUID=0)."
echo "[installation/main] Skipping user virtualenv and shell RC modifications."
echo "[installation/main] Delegating to scripts/installation/run-package.sh..."
bash scripts/installation/run-package.sh
echo "[installation/main] Root/system setup complete."
exit 0
fi
# ------------------------------------------------------------
# 3) Normal user mode: dev setup with venv
# ------------------------------------------------------------
echo "[installation/main] Running in normal user mode (developer setup)."
echo "[installation/main] Ensuring main.py is executable..."
chmod +x main.py || true
@@ -47,26 +63,8 @@ chmod +x main.py || true
echo "[installation/main] Ensuring global virtualenv root: ${HOME}/.venvs"
mkdir -p "${HOME}/.venvs"
if [[ ! -d "${VENV_DIR}" ]]; then
echo "[installation/main] Creating virtual environment at: ${VENV_DIR}"
python3 -m venv "${VENV_DIR}"
else
echo "[installation/main] Virtual environment already exists at: ${VENV_DIR}"
fi
echo "[installation/main] Installing Python tooling into venv..."
"${VENV_DIR}/bin/python" -m ensurepip --upgrade
"${VENV_DIR}/bin/pip" install --upgrade pip setuptools wheel
if [[ -f "requirements.txt" ]]; then
echo "[installation/main] Installing dependencies from requirements.txt..."
"${VENV_DIR}/bin/pip" install -r requirements.txt
elif [[ -f "_requirements.txt" ]]; then
echo "[installation/main] Installing dependencies from _requirements.txt..."
"${VENV_DIR}/bin/pip" install -r _requirements.txt
else
echo "[installation/main] No requirements.txt or _requirements.txt found. Skipping dependency installation."
fi
echo "[installation/main] Creating/updating virtualenv via helper..."
PKGMGR_VENV_DIR="${VENV_DIR}" bash scripts/installation/venv-create.sh
echo "[installation/main] Ensuring ~/.bashrc and ~/.zshrc exist..."
touch "${HOME}/.bashrc" "${HOME}/.zshrc"

View File

@@ -3,6 +3,25 @@ set -euo pipefail
echo "[ubuntu/package] Building Ubuntu (Debian-style) package..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
BUILD_ROOT="/tmp/package-manager-ubuntu-build"
rm -rf "${BUILD_ROOT}"
mkdir -p "${BUILD_ROOT}"
echo "[ubuntu/package] Syncing project sources to ${BUILD_ROOT}..."
rsync -a \
--exclude 'packaging/debian' \
"${PROJECT_ROOT}/" "${BUILD_ROOT}/"
echo "[ubuntu/package] Overlaying debian/ metadata from packaging/debian..."
mkdir -p "${BUILD_ROOT}/debian"
cp -a "${PROJECT_ROOT}/packaging/debian/." "${BUILD_ROOT}/debian/"
cd "${BUILD_ROOT}"
echo "[ubuntu/package] Running dpkg-buildpackage..."
dpkg-buildpackage -us -uc -b
echo "[ubuntu/package] Installing generated DEB package..."

View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -euo pipefail
# venv-create.sh
#
# Small helper to create/update a Python virtual environment for pkgmgr.
#
# Usage:
# PKGMGR_VENV_DIR=/home/dev/.venvs/pkgmgr bash scripts/installation/venv-create.sh
# or
# bash scripts/installation/venv-create.sh /home/dev/.venvs/pkgmgr
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "${PROJECT_ROOT}"
VENV_DIR="${PKGMGR_VENV_DIR:-${1:-${HOME}/.venvs/pkgmgr}}"
echo "[venv-create] Using VENV_DIR=${VENV_DIR}"
echo "[venv-create] Ensuring virtualenv parent directory exists..."
mkdir -p "$(dirname "${VENV_DIR}")"
if [[ ! -d "${VENV_DIR}" ]]; then
echo "[venv-create] Creating virtual environment at: ${VENV_DIR}"
python3 -m venv "${VENV_DIR}"
else
echo "[venv-create] Virtual environment already exists at: ${VENV_DIR}"
fi
echo "[venv-create] Installing Python tooling into venv..."
"${VENV_DIR}/bin/python" -m ensurepip --upgrade
"${VENV_DIR}/bin/pip" install --upgrade pip setuptools wheel
if [[ -f "requirements.txt" ]]; then
echo "[venv-create] Installing dependencies from requirements.txt..."
"${VENV_DIR}/bin/pip" install -r requirements.txt
elif [[ -f "_requirements.txt" ]]; then
echo "[venv-create] Installing dependencies from _requirements.txt..."
"${VENV_DIR}/bin/pip" install -r _requirements.txt
else
echo "[venv-create] No requirements.txt or _requirements.txt found. Skipping dependency installation."
fi
echo "[venv-create] Done."

View File

@@ -19,9 +19,9 @@ for distro in $DISTROS; do
# Run the command and capture the output
if OUTPUT=$(docker run --rm \
-e PKGMGR_DEV=1 \
-v pkgmgr_nix_store:/nix \
-v pkgmgr_nix_store_${distro}:/nix \
-v "$(pwd):/src" \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
"$IMAGE" 2>&1); then
echo "$OUTPUT"
echo

View File

@@ -10,43 +10,56 @@ for distro in $DISTROS; do
docker run --rm \
-v "$(pwd):/src" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v "pkgmgr_nix_store_${distro}:/nix" \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--workdir /src \
--entrypoint bash \
"package-manager-test-$distro" \
"package-manager-test-${distro}" \
-c '
set -e;
set -euo pipefail
# Load distro info
if [ -f /etc/os-release ]; then
. /etc/os-release;
fi;
. /etc/os-release
fi
echo "Running tests inside distro: $ID";
echo "Running tests inside distro: ${ID:-unknown}"
# Try to load nix environment
# Load Nix environment if available
if [ -f "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh" ]; then
. "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh";
. "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh"
fi
if [ -f "$HOME/.nix-profile/etc/profile.d/nix.sh" ]; then
. "$HOME/.nix-profile/etc/profile.d/nix.sh";
. "$HOME/.nix-profile/etc/profile.d/nix.sh"
fi
PATH="/nix/var/nix/profiles/default/bin:$HOME/.nix-profile/bin:$PATH";
PATH="/nix/var/nix/profiles/default/bin:$HOME/.nix-profile/bin:$PATH"
command -v nix >/dev/null || {
echo "ERROR: nix not found.";
exit 1;
echo "ERROR: nix not found."
exit 1
}
git config --global --add safe.directory /src || true;
# Mark the mounted repository as safe to avoid Git ownership errors.
# Newer Git (e.g. on Ubuntu) complains about the gitdir (/src/.git),
# older versions about the worktree (/src). Nix turns "." into the
# flake input "git+file:///src", which then uses Git under the hood.
if command -v git >/dev/null 2>&1; then
# Worktree path
git config --global --add safe.directory /src || true
# Gitdir path shown in the "dubious ownership" error
git config --global --add safe.directory /src/.git || true
# Ephemeral CI containers: allow all paths as a last resort
git config --global --add safe.directory '*' || true
fi
# Run the E2E tests inside the Nix development shell
nix develop .#default --no-write-lock-file -c \
python3 -m unittest discover \
-s /src/tests/e2e \
-p "$TEST_PATTERN";
-p "$TEST_PATTERN"
'
done

View File

@@ -1,24 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
: "${distro:=arch}"
echo "============================================================"
echo ">>> Running INTEGRATION tests in Arch container"
echo ">>> Running INTEGRATION tests in ${distro} container"
echo "============================================================"
docker run --rm \
-v "$(pwd):/src" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v pkgmgr_nix_store_${distro}:/nix \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
--workdir /src \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--entrypoint bash \
"package-manager-test-arch" \
"package-manager-test-${distro}" \
-c '
set -e;
git config --global --add safe.directory /src || true;
nix develop .#default --no-write-lock-file -c \
python -m unittest discover \
python3 -m unittest discover \
-s tests/integration \
-t /src \
-p "$TEST_PATTERN";

View File

@@ -1,24 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
: "${distro:=arch}"
echo "============================================================"
echo ">>> Running UNIT tests in Arch container"
echo ">>> Running UNIT tests in ${distro} container"
echo "============================================================"
docker run --rm \
-v "$(pwd):/src" \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
-v pkgmgr_nix_store_${distro}:/nix \
--workdir /src \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--entrypoint bash \
"package-manager-test-arch" \
"package-manager-test-${distro}" \
-c '
set -e;
git config --global --add safe.directory /src || true;
nix develop .#default --no-write-lock-file -c \
python -m unittest discover \
python3 -m unittest discover \
-s tests/unit \
-t /src \
-p "$TEST_PATTERN";

36
src/pkgmgr/__init__.py Normal file
View File

@@ -0,0 +1,36 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Top-level pkgmgr package.
We deliberately avoid importing heavy submodules (like the CLI)
on import to keep unit tests fast and to not require optional
dependencies (like PyYAML) unless they are actually used.
Accessing ``pkgmgr.cli`` will load the CLI module lazily via
``__getattr__``. This keeps patterns like
from pkgmgr import cli
working as expected in tests and entry points.
"""
from __future__ import annotations
from importlib import import_module
from typing import Any
__all__ = ["cli"]
def __getattr__(name: str) -> Any:
"""
Lazily expose ``pkgmgr.cli`` as attribute on the top-level package.
This keeps ``import pkgmgr`` lightweight while still allowing
``from pkgmgr import cli`` in tests and entry points.
"""
if name == "cli":
return import_module("pkgmgr.cli")
raise AttributeError(f"module 'pkgmgr' has no attribute {name!r}")

View File

@@ -0,0 +1,218 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
High-level entry point for repository installation.
Responsibilities:
- Ensure the repository directory exists (clone if necessary).
- Verify the repository (GPG / commit checks).
- Build a RepoContext object.
- Delegate the actual installation decision logic to InstallationPipeline.
"""
from __future__ import annotations
import os
from typing import Any, Dict, List
from pkgmgr.core.repository.identifier import get_repo_identifier
from pkgmgr.core.repository.dir import get_repo_dir
from pkgmgr.core.repository.verify import verify_repository
from pkgmgr.actions.repository.clone import clone_repos
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.os_packages import (
ArchPkgbuildInstaller,
DebianControlInstaller,
RpmSpecInstaller,
)
from pkgmgr.actions.install.installers.nix_flake import (
NixFlakeInstaller,
)
from pkgmgr.actions.install.installers.python import PythonInstaller
from pkgmgr.actions.install.installers.makefile import (
MakefileInstaller,
)
from pkgmgr.actions.install.pipeline import InstallationPipeline
Repository = Dict[str, Any]
# All available installers, in the order they should be considered.
INSTALLERS = [
ArchPkgbuildInstaller(),
DebianControlInstaller(),
RpmSpecInstaller(),
NixFlakeInstaller(),
PythonInstaller(),
MakefileInstaller(),
]
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _ensure_repo_dir(
repo: Repository,
repositories_base_dir: str,
all_repos: List[Repository],
preview: bool,
no_verification: bool,
clone_mode: str,
identifier: str,
) -> str | None:
"""
Compute and, if necessary, clone the repository directory.
Returns the absolute repository path or None if cloning ultimately failed.
"""
repo_dir = get_repo_dir(repositories_base_dir, repo)
if not os.path.exists(repo_dir):
print(
f"Repository directory '{repo_dir}' does not exist. "
f"Cloning it now..."
)
clone_repos(
[repo],
repositories_base_dir,
all_repos,
preview,
no_verification,
clone_mode,
)
if not os.path.exists(repo_dir):
print(
f"Cloning failed for repository {identifier}. "
f"Skipping installation."
)
return None
return repo_dir
def _verify_repo(
repo: Repository,
repo_dir: str,
no_verification: bool,
identifier: str,
) -> bool:
"""
Verify a repository using the configured verification data.
Returns True if verification is considered okay and installation may continue.
"""
verified_info = repo.get("verified")
verified_ok, errors, _commit_hash, _signing_key = verify_repository(
repo,
repo_dir,
mode="local",
no_verification=no_verification,
)
if not no_verification and verified_info and not verified_ok:
print(f"Warning: Verification failed for {identifier}:")
for err in errors:
print(f" - {err}")
choice = input("Continue anyway? [y/N]: ").strip().lower()
if choice != "y":
print(f"Skipping installation for {identifier}.")
return False
return True
def _create_context(
repo: Repository,
identifier: str,
repo_dir: str,
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Repository],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> RepoContext:
"""
Build a RepoContext instance for the given repository.
"""
return RepoContext(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def install_repos(
selected_repos: List[Repository],
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Repository],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> None:
"""
Install one or more repositories according to the configured installers
and the CLI layer precedence rules.
"""
pipeline = InstallationPipeline(INSTALLERS)
for repo in selected_repos:
identifier = get_repo_identifier(repo, all_repos)
repo_dir = _ensure_repo_dir(
repo=repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
no_verification=no_verification,
clone_mode=clone_mode,
identifier=identifier,
)
if not repo_dir:
continue
if not _verify_repo(
repo=repo,
repo_dir=repo_dir,
no_verification=no_verification,
identifier=identifier,
):
continue
ctx = _create_context(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
pipeline.run(ctx)

View File

@@ -38,7 +38,7 @@ from abc import ABC, abstractmethod
from typing import Iterable, TYPE_CHECKING
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.install.context import RepoContext
# ---------------------------------------------------------------------------

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer package for pkgmgr.
This exposes all installer classes so users can import them directly from
pkgmgr.actions.install.installers.
"""
from pkgmgr.actions.install.installers.base import BaseInstaller # noqa: F401
from pkgmgr.actions.install.installers.nix_flake import NixFlakeInstaller # noqa: F401
from pkgmgr.actions.install.installers.python import PythonInstaller # noqa: F401
from pkgmgr.actions.install.installers.makefile import MakefileInstaller # noqa: F401
# OS-specific installers
from pkgmgr.actions.install.installers.os_packages.arch_pkgbuild import ArchPkgbuildInstaller # noqa: F401
from pkgmgr.actions.install.installers.os_packages.debian_control import DebianControlInstaller # noqa: F401
from pkgmgr.actions.install.installers.os_packages.rpm_spec import RpmSpecInstaller # noqa: F401

View File

@@ -8,8 +8,8 @@ Base interface for all installer components in the pkgmgr installation pipeline.
from abc import ABC, abstractmethod
from typing import Set
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.capabilities import CAPABILITY_MATCHERS
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.capabilities import CAPABILITY_MATCHERS
class BaseInstaller(ABC):

View File

@@ -0,0 +1,97 @@
from __future__ import annotations
import os
import re
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
class MakefileInstaller(BaseInstaller):
"""
Generic installer that runs `make install` if a Makefile with an
install target is present.
Safety rules:
- If PKGMGR_DISABLE_MAKEFILE_INSTALLER=1 is set, this installer
is globally disabled.
- The higher-level InstallationPipeline ensures that Makefile
installation does not run if a stronger CLI layer already owns
the command (e.g. Nix or OS packages).
"""
layer = "makefile"
MAKEFILE_NAME = "Makefile"
def supports(self, ctx: RepoContext) -> bool:
"""
Return True if this repository has a Makefile and the installer
is not globally disabled.
"""
# Optional global kill switch.
if os.environ.get("PKGMGR_DISABLE_MAKEFILE_INSTALLER") == "1":
if not ctx.quiet:
print(
"[INFO] MakefileInstaller is disabled via "
"PKGMGR_DISABLE_MAKEFILE_INSTALLER."
)
return False
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
return os.path.exists(makefile_path)
def _has_install_target(self, makefile_path: str) -> bool:
"""
Heuristically check whether the Makefile defines an install target.
We look for:
- a plain 'install:' target, or
- any 'install-*:' style target.
"""
try:
with open(makefile_path, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
except OSError:
return False
# Simple heuristics: look for "install:" or targets starting with "install-"
if re.search(r"^install\s*:", content, flags=re.MULTILINE):
return True
if re.search(r"^install-[a-zA-Z0-9_-]*\s*:", content, flags=re.MULTILINE):
return True
return False
def run(self, ctx: RepoContext) -> None:
"""
Execute `make install` in the repository directory if an install
target exists.
"""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
if not os.path.exists(makefile_path):
if not ctx.quiet:
print(
f"[pkgmgr] Makefile '{makefile_path}' not found, "
"skipping MakefileInstaller."
)
return
if not self._has_install_target(makefile_path):
if not ctx.quiet:
print(
f"[pkgmgr] No 'install' target found in {makefile_path}."
)
return
if not ctx.quiet:
print(
f"[pkgmgr] Running 'make install' in {ctx.repo_dir} "
f"(MakefileInstaller)"
)
cmd = "make install"
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -10,28 +10,31 @@ installer will try to install profile outputs from the flake.
Behavior:
- If flake.nix is present and `nix` exists on PATH:
* First remove any existing `package-manager` profile entry (best-effort).
* Then install the flake outputs (`pkgmgr`, `default`) via `nix profile install`.
- Failure installing `pkgmgr` is treated as fatal.
- Failure installing `default` is logged as an error/warning but does not abort.
* Then install one or more flake outputs via `nix profile install`.
- For the package-manager repo:
* `pkgmgr` is mandatory (CLI), `default` is optional.
- For all other repos:
* `default` is mandatory.
Special handling for dev shells / CI:
- If IN_NIX_SHELL is set (e.g. inside `nix develop`), the installer is
disabled. In that environment the flake outputs are already provided
by the dev shell and we must not touch the user profile.
Special handling:
- If PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 is set, the installer is
globally disabled (useful for CI or debugging).
The higher-level InstallationPipeline and CLI-layer model decide when this
installer is allowed to run, based on where the current CLI comes from
(e.g. Nix, OS packages, Python, Makefile).
"""
import os
import shutil
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, List, Tuple
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install import InstallContext
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install import InstallContext
class NixFlakeInstaller(BaseInstaller):
@@ -43,33 +46,14 @@ class NixFlakeInstaller(BaseInstaller):
FLAKE_FILE = "flake.nix"
PROFILE_NAME = "package-manager"
def _in_nix_shell(self) -> bool:
"""
Return True if we appear to be running inside a Nix dev shell.
Nix sets IN_NIX_SHELL in `nix develop` environments. In that case
the flake outputs are already available, and touching the user
profile (nix profile install/remove) is undesirable.
"""
return bool(os.environ.get("IN_NIX_SHELL"))
def supports(self, ctx: "RepoContext") -> bool:
"""
Only support repositories that:
- Are NOT inside a Nix dev shell (IN_NIX_SHELL unset),
- Are NOT explicitly disabled via PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1,
- Have a flake.nix,
- And have the `nix` command available.
"""
# 1) Skip when running inside a dev shell flake is already active.
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected; skipping NixFlakeInstaller. "
"Flake outputs are provided by the development shell."
)
return False
# 2) Optional global kill-switch for CI or debugging.
# Optional global kill-switch for CI or debugging.
if os.environ.get("PKGMGR_DISABLE_NIX_FLAKE_INSTALLER") == "1":
print(
"[INFO] PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 "
@@ -77,11 +61,11 @@ class NixFlakeInstaller(BaseInstaller):
)
return False
# 3) Nix must be available.
# Nix must be available.
if shutil.which("nix") is None:
return False
# 4) flake.nix must exist in the repository.
# flake.nix must exist in the repository.
flake_path = os.path.join(ctx.repo_dir, self.FLAKE_FILE)
return os.path.exists(flake_path)
@@ -107,36 +91,56 @@ class NixFlakeInstaller(BaseInstaller):
# Unit tests explicitly assert this is swallowed
pass
def _profile_outputs(self, ctx: "RepoContext") -> List[Tuple[str, bool]]:
"""
Decide which flake outputs to install and whether failures are fatal.
Returns a list of (output_name, allow_failure) tuples.
Rules:
- For the package-manager repo (identifier 'pkgmgr' or 'package-manager'):
[("pkgmgr", False), ("default", True)]
- For all other repos:
[("default", False)]
"""
ident = ctx.identifier
if ident in {"pkgmgr", "package-manager"}:
# pkgmgr: main CLI output is "pkgmgr" (mandatory),
# "default" is nice-to-have (non-fatal).
return [("pkgmgr", False), ("default", True)]
# Generic repos: we expect a sensible "default" package/app.
# Failure to install it is considered fatal.
return [("default", False)]
def run(self, ctx: "InstallContext") -> None:
"""
Install Nix flake profile outputs (pkgmgr, default).
Install Nix flake profile outputs.
Any failure installing `pkgmgr` is treated as fatal (SystemExit).
A failure installing `default` is logged but does not abort.
For the package-manager repo, failure installing 'pkgmgr' is fatal,
failure installing 'default' is non-fatal.
For other repos, failure installing 'default' is fatal.
"""
# Extra guard in case run() is called directly without supports().
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected in run(); "
"skipping Nix flake profile installation."
)
return
# Reuse supports() to keep logic in one place
# Reuse supports() to keep logic in one place.
if not self.supports(ctx): # type: ignore[arg-type]
return
print("Nix flake detected, attempting to install profile outputs...")
outputs = self._profile_outputs(ctx) # list of (name, allow_failure)
# Handle the "already installed" case up-front:
print(
"Nix flake detected in "
f"{ctx.identifier}, attempting to install profile outputs: "
+ ", ".join(name for name, _ in outputs)
)
# Handle the "already installed" case up-front for the shared profile.
self._ensure_old_profile_removed(ctx) # type: ignore[arg-type]
for output in ("pkgmgr", "default"):
for output, allow_failure in outputs:
cmd = f"nix profile install {ctx.repo_dir}#{output}"
try:
# For 'default' we don't want the process to exit on error
allow_failure = output == "default"
run_command(
cmd,
cwd=ctx.repo_dir,
@@ -146,12 +150,11 @@ class NixFlakeInstaller(BaseInstaller):
print(f"Nix flake output '{output}' successfully installed.")
except SystemExit as e:
print(f"[Error] Failed to install Nix flake output '{output}': {e}")
if output == "pkgmgr":
# Broken main CLI install → fatal
if not allow_failure:
# Mandatory output failed → fatal for the pipeline.
raise
# For 'default' we log and continue
# Optional output failed → log and continue.
print(
"[Warning] Continuing despite failure to install 'default' "
"because 'pkgmgr' is already installed."
"[Warning] Continuing despite failure to install "
f"optional output '{output}'."
)
break

View File

@@ -3,8 +3,8 @@
import os
import shutil
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -19,8 +19,8 @@ import os
import shutil
from typing import List
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -21,8 +21,8 @@ import shutil
import tarfile
from typing import List, Optional, Tuple
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -0,0 +1,139 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
PythonInstaller — install Python projects defined via pyproject.toml.
Installation rules:
1. pip command resolution:
a) If PKGMGR_PIP is set → use it exactly as provided.
b) Else if running inside a virtualenv → use `sys.executable -m pip`.
c) Else → create/use a per-repository virtualenv under ~/.venvs/<repo>/.
2. Installation target:
- Always install into the resolved pip environment.
- Never modify system Python, never rely on --user.
- Nix-immutable systems (PEP 668) are automatically avoided because we
never touch system Python.
3. The installer is skipped when:
- PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set.
- The repository has no pyproject.toml.
All pip failures are treated as fatal.
"""
from __future__ import annotations
import os
import sys
import subprocess
from typing import TYPE_CHECKING
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install import InstallContext
class PythonInstaller(BaseInstaller):
"""Install Python projects and dependencies via pip using isolated environments."""
layer = "python"
# ----------------------------------------------------------------------
# Installer activation logic
# ----------------------------------------------------------------------
def supports(self, ctx: "RepoContext") -> bool:
"""
Return True if this installer should handle this repository.
The installer is active only when:
- A pyproject.toml exists in the repo, and
- PKGMGR_DISABLE_PYTHON_INSTALLER is not set.
"""
if os.environ.get("PKGMGR_DISABLE_PYTHON_INSTALLER") == "1":
print("[INFO] PythonInstaller disabled via PKGMGR_DISABLE_PYTHON_INSTALLER.")
return False
return os.path.exists(os.path.join(ctx.repo_dir, "pyproject.toml"))
# ----------------------------------------------------------------------
# Virtualenv handling
# ----------------------------------------------------------------------
def _in_virtualenv(self) -> bool:
"""Detect whether the current interpreter is inside a venv."""
if os.environ.get("VIRTUAL_ENV"):
return True
base = getattr(sys, "base_prefix", sys.prefix)
return sys.prefix != base
def _ensure_repo_venv(self, ctx: "InstallContext") -> str:
"""
Ensure that ~/.venvs/<identifier>/ exists and contains a minimal venv.
Returns the venv directory path.
"""
venv_dir = os.path.expanduser(f"~/.venvs/{ctx.identifier}")
python = sys.executable
if not os.path.isdir(venv_dir):
print(f"[python-installer] Creating virtualenv: {venv_dir}")
subprocess.check_call([python, "-m", "venv", venv_dir])
return venv_dir
# ----------------------------------------------------------------------
# pip command resolution
# ----------------------------------------------------------------------
def _pip_cmd(self, ctx: "InstallContext") -> str:
"""
Determine which pip command to use.
Priority:
1. PKGMGR_PIP override given by user or automation.
2. Active virtualenv → use sys.executable -m pip.
3. Per-repository venv → ~/.venvs/<repo>/bin/pip
"""
explicit = os.environ.get("PKGMGR_PIP", "").strip()
if explicit:
return explicit
if self._in_virtualenv():
return f"{sys.executable} -m pip"
venv_dir = self._ensure_repo_venv(ctx)
pip_path = os.path.join(venv_dir, "bin", "pip")
return pip_path
# ----------------------------------------------------------------------
# Execution
# ----------------------------------------------------------------------
def run(self, ctx: "InstallContext") -> None:
"""
Install the project defined by pyproject.toml.
Uses the resolved pip environment. Installation is isolated and never
touches system Python.
"""
if not self.supports(ctx): # type: ignore[arg-type]
return
pyproject = os.path.join(ctx.repo_dir, "pyproject.toml")
if not os.path.exists(pyproject):
return
print(f"[python-installer] Installing Python project for {ctx.identifier}...")
pip_cmd = self._pip_cmd(ctx)
# Final install command: ALWAYS isolated, never system-wide.
install_cmd = f"{pip_cmd} install ."
run_command(install_cmd, cwd=ctx.repo_dir, preview=ctx.preview)
print(f"[python-installer] Installation finished for {ctx.identifier}.")

View File

@@ -0,0 +1,91 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
CLI layer model for the pkgmgr installation pipeline.
We treat CLI entry points as coming from one of four conceptual layers:
- os-packages : system package managers (pacman/apt/dnf/…)
- nix : Nix flake / nix profile
- python : pip / virtualenv / user-local scripts
- makefile : repo-local Makefile / scripts inside the repo
The layer order defines precedence: higher layers "own" the CLI and
lower layers will not be executed once a higher-priority CLI exists.
"""
from __future__ import annotations
import os
from enum import Enum
from typing import Optional
class CliLayer(str, Enum):
OS_PACKAGES = "os-packages"
NIX = "nix"
PYTHON = "python"
MAKEFILE = "makefile"
# Highest priority first
CLI_LAYERS: list[CliLayer] = [
CliLayer.OS_PACKAGES,
CliLayer.NIX,
CliLayer.PYTHON,
CliLayer.MAKEFILE,
]
def layer_priority(layer: Optional[CliLayer]) -> int:
"""
Return a numeric priority index for a given layer.
Lower index → higher priority.
Unknown / None → very low priority.
"""
if layer is None:
return len(CLI_LAYERS)
try:
return CLI_LAYERS.index(layer)
except ValueError:
return len(CLI_LAYERS)
def classify_command_layer(command: str, repo_dir: str) -> CliLayer:
"""
Heuristically classify a resolved command path into a CLI layer.
Rules (best effort):
- /usr/... or /bin/... → os-packages
- /nix/store/... or ~/.nix-profile → nix
- ~/.local/bin/... → python
- inside repo_dir → makefile
- everything else → python (user/venv scripts, etc.)
"""
command_abs = os.path.abspath(os.path.expanduser(command))
repo_abs = os.path.abspath(repo_dir)
home = os.path.expanduser("~")
# OS package managers
if command_abs.startswith("/usr/") or command_abs.startswith("/bin/"):
return CliLayer.OS_PACKAGES
# Nix store / profile
if command_abs.startswith("/nix/store/") or command_abs.startswith(
os.path.join(home, ".nix-profile")
):
return CliLayer.NIX
# User-local bin
if command_abs.startswith(os.path.join(home, ".local", "bin")):
return CliLayer.PYTHON
# Inside the repository → usually a Makefile/script
if command_abs.startswith(repo_abs):
return CliLayer.MAKEFILE
# Fallback: treat as Python-style/user-level script
return CliLayer.PYTHON

View File

@@ -0,0 +1,257 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installation pipeline orchestration for repositories.
This module implements the "Setup Controller" logic:
1. Detect current CLI command for the repo (if any).
2. Classify it into a layer (os-packages, nix, python, makefile).
3. Iterate over installers in layer order:
- Skip installers whose layer is weaker than an already-loaded one.
- Run only installers that support() the repo and add new capabilities.
- After each installer, re-resolve the command and update the layer.
4. Maintain the repo["command"] field and create/update symlinks via create_ink().
The goal is to prevent conflicting installations and make the layering
behaviour explicit and testable.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Optional, Sequence, Set
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.actions.install.layers import (
CliLayer,
classify_command_layer,
layer_priority,
)
from pkgmgr.core.command.ink import create_ink
from pkgmgr.core.command.resolve import resolve_command_for_repo
@dataclass
class CommandState:
"""
Represents the current CLI state for a repository:
- command: absolute or relative path to the CLI entry point
- layer: which conceptual layer this command belongs to
"""
command: Optional[str]
layer: Optional[CliLayer]
class CommandResolver:
"""
Small helper responsible for resolving the current command for a repo
and mapping it into a CommandState.
"""
def __init__(self, ctx: RepoContext) -> None:
self._ctx = ctx
def resolve(self) -> CommandState:
"""
Resolve the current command for this repository.
If resolve_command_for_repo raises SystemExit (e.g. Python package
without installed entry point), we treat this as "no command yet"
from the point of view of the installers.
"""
repo = self._ctx.repo
identifier = self._ctx.identifier
repo_dir = self._ctx.repo_dir
try:
cmd = resolve_command_for_repo(
repo=repo,
repo_identifier=identifier,
repo_dir=repo_dir,
)
except SystemExit:
cmd = None
if not cmd:
return CommandState(command=None, layer=None)
layer = classify_command_layer(cmd, repo_dir)
return CommandState(command=cmd, layer=layer)
class InstallationPipeline:
"""
High-level orchestrator that applies a sequence of installers
to a repository based on CLI layer precedence.
"""
def __init__(self, installers: Sequence[BaseInstaller]) -> None:
self._installers = list(installers)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def run(self, ctx: RepoContext) -> None:
"""
Execute the installation pipeline for a single repository.
- Detect initial command & layer.
- Optionally create a symlink.
- Run installers in order, skipping those whose layer is weaker
than an already-loaded CLI.
- After each installer, re-resolve the command and refresh the
symlink if needed.
"""
repo = ctx.repo
repo_dir = ctx.repo_dir
identifier = ctx.identifier
repositories_base_dir = ctx.repositories_base_dir
bin_dir = ctx.bin_dir
all_repos = ctx.all_repos
quiet = ctx.quiet
preview = ctx.preview
resolver = CommandResolver(ctx)
state = resolver.resolve()
# Persist initial command (if any) and create a symlink.
if state.command:
repo["command"] = state.command
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
else:
repo.pop("command", None)
provided_capabilities: Set[str] = set()
# Main installer loop
for installer in self._installers:
layer_name = getattr(installer, "layer", None)
# Installers without a layer participate without precedence logic.
if layer_name is None:
self._run_installer(installer, ctx, identifier, repo_dir, quiet)
continue
try:
installer_layer = CliLayer(layer_name)
except ValueError:
# Unknown layer string → treat as lowest priority.
installer_layer = None
# "Previous/Current layer already loaded?"
if state.layer is not None and installer_layer is not None:
current_prio = layer_priority(state.layer)
installer_prio = layer_priority(installer_layer)
if current_prio < installer_prio:
# Current CLI comes from a higher-priority layer,
# so we skip this installer entirely.
if not quiet:
print(
f"[pkgmgr] Skipping installer "
f"{installer.__class__.__name__} for {identifier} "
f"CLI already provided by layer {state.layer.value!r}."
)
continue
if current_prio == installer_prio:
# Same layer already provides a CLI; usually there is no
# need to run another installer on top of it.
if not quiet:
print(
f"[pkgmgr] Skipping installer "
f"{installer.__class__.__name__} for {identifier} "
f"layer {installer_layer.value!r} is already loaded."
)
continue
# Check if this installer is applicable at all.
if not installer.supports(ctx):
continue
# Capabilities: if everything this installer would provide is already
# covered, we can safely skip it.
caps = installer.discover_capabilities(ctx)
if caps and caps.issubset(provided_capabilities):
if not quiet:
print(
f"Skipping installer {installer.__class__.__name__} "
f"for {identifier} capabilities {caps} already provided."
)
continue
if not quiet:
print(
f"[pkgmgr] Running installer {installer.__class__.__name__} "
f"for {identifier} in '{repo_dir}' "
f"(new capabilities: {caps or set()})..."
)
# Run the installer with error reporting.
self._run_installer(installer, ctx, identifier, repo_dir, quiet)
provided_capabilities.update(caps)
# After running an installer, re-resolve the command and layer.
new_state = resolver.resolve()
if new_state.command:
repo["command"] = new_state.command
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
else:
repo.pop("command", None)
state = new_state
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
@staticmethod
def _run_installer(
installer: BaseInstaller,
ctx: RepoContext,
identifier: str,
repo_dir: str,
quiet: bool,
) -> None:
"""
Execute a single installer with unified error handling.
"""
try:
installer.run(ctx)
except SystemExit as exc:
exit_code = exc.code if isinstance(exc.code, int) else str(exc.code)
print(
f"[ERROR] Installer {installer.__class__.__name__} failed "
f"for repository {identifier} (dir: {repo_dir}) "
f"with exit code {exit_code}."
)
print(
"[ERROR] This usually means an underlying command failed "
"(e.g. 'make install', 'nix build', 'pip install', ...)."
)
print(
"[ERROR] Check the log above for the exact command output. "
"You can also run this repository in isolation via:\n"
f" pkgmgr install {identifier} "
"--clone-mode shallow --no-verification"
)
raise

View File

@@ -2,7 +2,7 @@ import sys
import shutil
from pkgmgr.actions.repository.pull import pull_with_verification
from pkgmgr.actions.repository.install import install_repos
from pkgmgr.actions.install import install_repos
def update_repos(

View File

@@ -34,7 +34,6 @@ dependency formats, including:
\033[1;33mNix:\033[0m flake.nix
\033[1;33mArch Linux:\033[0m PKGBUILD
\033[1;33mAnsible:\033[0m requirements.yml
\033[1;33mpkgmgr-native:\033[0m pkgmgr.yml
This allows pkgmgr to perform installation, updates, verification, dependency
resolution, and synchronization across complex multi-repo environments with a

View File

@@ -7,7 +7,7 @@ import sys
from typing import Any, Dict, List
from pkgmgr.cli.context import CLIContext
from pkgmgr.actions.repository.install import install_repos
from pkgmgr.actions.install import install_repos
from pkgmgr.actions.repository.deinstall import deinstall_repos
from pkgmgr.actions.repository.delete import delete_repos
from pkgmgr.actions.repository.update import update_repos
@@ -16,10 +16,36 @@ from pkgmgr.actions.repository.list import list_repositories
from pkgmgr.core.command.run import run_command
from pkgmgr.actions.repository.create import create_repo
from pkgmgr.core.repository.selected import get_selected_repos
from pkgmgr.core.repository.dir import get_repo_dir
Repository = Dict[str, Any]
def _resolve_repository_directory(repository: Repository, ctx: CLIContext) -> str:
"""
Resolve the local filesystem directory for a repository.
Priority:
1. Use repository["directory"] if present.
2. Fallback to get_repo_dir(...) using the repositories base directory
from the CLI context.
"""
repo_dir = repository.get("directory")
if repo_dir:
return repo_dir
base_dir = (
getattr(ctx, "repositories_base_dir", None)
or getattr(ctx, "repositories_dir", None)
)
if not base_dir:
raise RuntimeError(
"Cannot resolve repositories base directory from context; "
"expected ctx.repositories_base_dir or ctx.repositories_dir."
)
return get_repo_dir(base_dir, repository)
def handle_repos_command(
args,
ctx: CLIContext,
@@ -108,8 +134,25 @@ def handle_repos_command(
# path
# ------------------------------------------------------------
if args.command == "path":
if not selected:
print("[pkgmgr] No repositories selected for path.")
return
for repository in selected:
print(repository["directory"])
try:
repo_dir = _resolve_repository_directory(repository, ctx)
except Exception as exc:
ident = (
f"{repository.get('provider', '?')}/"
f"{repository.get('account', '?')}/"
f"{repository.get('repository', '?')}"
)
print(
f"[WARN] Could not resolve directory for {ident}: {exc}"
)
continue
print(repo_dir)
return
# ------------------------------------------------------------
@@ -119,14 +162,14 @@ def handle_repos_command(
if not args.shell_command:
print("[ERROR] 'shell' requires a command via -c/--command.")
sys.exit(2)
command_to_run = " ".join(args.shell_command)
for repository in selected:
print(
f"Executing in '{repository['directory']}': {command_to_run}"
)
repo_dir = _resolve_repository_directory(repository, ctx)
print(f"Executing in '{repo_dir}': {command_to_run}")
run_command(
command_to_run,
cwd=repository["directory"],
cwd=repo_dir,
preview=args.preview,
)
return

Some files were not shown because too many files have changed in this diff Show More