Compare commits

...

53 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
f3c5460e48 feat(mirror): support SSH MIRRORS, multi-push origin and remote probe
Some checks failed
CI / test-unit (push) Has been cancelled
CI / test-integration (push) Has been cancelled
CI / test-container (push) Has been cancelled
CI / test-e2e (push) Has been cancelled
CI / test-virgin-user (push) Has been cancelled
CI / test-virgin-root (push) Has been cancelled
- Switch MIRRORS to SSH-based URLs including custom ports/domains
  (GitHub, git.veen.world, code.cymais.cloud)
- Extend mirror IO:
  - load_config_mirrors filters empty values
  - read_mirrors_file now supports:
    * "name url" lines
    * "url" lines with auto-generated names from URL host (host[:port])
  - write_mirrors_file prints full preview content
- Enhance git_remote:
  - determine_primary_remote_url used for origin bootstrap
  - ensure_origin_remote keeps existing origin URL and
    adds all mirror URLs as additional push URLs
  - add is_remote_reachable() helper based on `git ls-remote --exit-code`
- Implement non-destructive remote mirror checks in setup_cmd:
  - `_probe_mirror()` wraps `git ls-remote` and returns (ok, message)
  - `pkgmgr mirror setup --remote` now probes each mirror URL and
    prints [OK]/[WARN] with details instead of placeholder text
- Add unit tests for mirror actions:
  - test_git_remote: default SSH URL building and primary URL selection
  - test_io: config + MIRRORS parsing including auto-named URL-only entries
  - test_setup_cmd: probe_mirror success/failure handling

https://chatgpt.com/share/693adee0-aa3c-800f-b72a-98473fdaf760
2025-12-11 17:49:31 +01:00
Kevin Veen-Birkenbach
39b16b87a8 CI: Add debugging instrumentation to identify container build/run anomalies
Some checks failed
CI / test-unit (push) Has been cancelled
CI / test-integration (push) Has been cancelled
CI / test-container (push) Has been cancelled
CI / test-e2e (push) Has been cancelled
CI / test-virgin-user (push) Has been cancelled
CI / test-virgin-root (push) Has been cancelled
- Added `git rev-parse HEAD` to test-container workflow to confirm the exact
  commit SHA used during CI runs.
- Updated Dockerfile to print BASE_IMAGE and OS release information during
  build for better reproducibility diagnostics.
- Extended test-container script to dump the first 40 lines of
  `docker image inspect` output, allowing verification of the image ID,
  creation time, and applied build args.

These additions help trace discrepancies between local builds and GitHub
Actions, ensuring we can detect mismatches in commit SHA, base image,
or container metadata.

https://chatgpt.com/share/693ae07a-8c58-800f-88e6-254cdb00b676
2025-12-11 17:27:57 +01:00
Kevin Veen-Birkenbach
26c9d79814 Added mirrors 2025-12-11 16:47:23 +01:00
Kevin Veen-Birkenbach
2776d18a42 Implemented arch support 2025-12-11 16:31:00 +01:00
Kevin Veen-Birkenbach
7057ccfb95 CI: Always rebuild test images with --no-cache before container and E2E tests
This ensures that GitHub Actions never reuses outdated Docker layers and that
each test run starts from a fully clean environment. The workflows for
test-container and test-e2e now explicitly invoke:

    distro="${{ matrix.distro }}" make build-no-cache

before executing the actual tests.
This aligns the CI behaviour with local testing, eliminates hidden caching
differences, and guarantees deterministic test results across all distros.

https://chatgpt.com/share/693ae07a-8c58-800f-88e6-254cdb00b676
2025-12-11 16:17:10 +01:00
Kevin Veen-Birkenbach
1807949c6f Add mirror management commands and refactor CLI parser into modules
- Implement new mirror actions:
  - list_mirrors: show mirrors from config, MIRRORS file, or merged view
  - diff_mirrors: compare config mirrors with MIRRORS file (ONLY IN CONFIG,
    ONLY IN FILE, URL MISMATCH, OK)
  - merge_mirrors: merge mirrors between config and MIRRORS file in both
    directions, with preview mode and user config writing via save_user_config
  - setup_mirrors: prepare local Git remotes (ensure origin) and print
    provider-URL suggestions for remote repositories
- Introduce mirror utilities:
  - RepoMirrorContext with resolved_mirrors (config + file, file wins)
  - load_config_mirrors supporting dict and list-of-dicts shapes
  - read/write MIRRORS file with simple "name url" format and preview mode
  - helper for building default SSH URLs from provider/account/repository
- Wire mirror commands into CLI:
  - Add handle_mirror_command and integrate "mirror" into dispatch
  - Add dedicated CLI parser modules under pkgmgr.cli.parser:
    * common, install_update, config_cmd, navigation_cmd,
      branch_cmd, release_cmd, version_cmd, changelog_cmd,
      list_cmd, make_cmd, mirror_cmd
  - Replace old flat cli/parser.py with modular parser package and
    SortedSubParsersAction in common.py
- Update TODO.md to mark MIRROR as implemented
- Add E2E tests for mirror commands:
  - test_mirror_help
  - test_mirror_list_preview_all
  - test_mirror_diff_preview_all
  - test_mirror_merge_config_to_file_preview_all
  - test_mirror_setup_preview_all

https://chatgpt.com/share/693adee0-aa3c-800f-b72a-98473fdaf760
2025-12-11 16:10:19 +01:00
Kevin Veen-Birkenbach
d611720b8f Solved bug when volumes don't exist 2025-12-11 15:46:45 +01:00
Kevin Veen-Birkenbach
bf871650a8 Added purge option to makefile 2025-12-11 15:29:51 +01:00
Kevin Veen-Birkenbach
5ca1adda7b Refactor CI distro handling and container build scripts
- Introduce a GitHub Actions matrix for `test-container` and `test-e2e`
  to run against arch, debian, ubuntu, fedora, and centos
- Run unit and integration tests only in the Arch container by passing
  `distro="arch"` via make in the corresponding workflows
- Replace the global DISTROS loop with a single `distro` variable in
  the Makefile, defaulting to `arch`, and export it for all scripts
- Update build scripts (build-image, build-image-no-cache, build-image-missing)
  to build images for the selected distro only
- Simplify test-container script to validate a single distro image using
  the `distro` environment variable
- Simplify E2E, unit, and integration test scripts to run against a
  single distro container instead of iterating over all distros

https://chatgpt.com/share/693acbba-9e30-800f-94fb-fea4489e9078
2025-12-11 14:48:36 +01:00
Kevin Veen-Birkenbach
acb18adf76 test: restore Dockerfile ENTRYPOINT for all test runs (fix Nix TLS on CentOS)
All test scripts (unit, integration, e2e) previously overwrote the Docker
ENTRYPOINT by using `--entrypoint bash`, which bypassed the container’s
startup logic in `docker-entry.sh`.

`docker-entry.sh` performs essential initialization steps such as:

- CA bundle auto-detection (NIX_SSL_CERT_FILE, SSL_CERT_FILE, etc.)
- Nix environment setup
- PATH adjustments and distro logging

By removing the explicit `--entrypoint bash` and invoking:

  bash -lc '...'

directly as the container command, the Dockerfile’s ENTRYPOINT is restored
and runs as intended before executing the test logic.

This fixes TLS issues in CentOS E2E runs where Nix was unable to fetch
flake inputs due to missing CA configuration.

https://chatgpt.com/share/693ac1f3-fb7c-800f-9e5c-b40c351a9f04
2025-12-11 14:06:39 +01:00
Kevin Veen-Birkenbach
c18490f5d3 deb: remove hard dependency on distro-provided Nix
The Debian Nix package causes flake builds to fail inside the test and
container environment due to sandboxing and patched Nix behavior.

To ensure consistent behaviour across all distributions and align
container logic with production logic, pkgmgr now relies on its own
`init-nix.sh` bootstrap script instead of the distro’s `nix` package.

Dropping `Depends: nix` guarantees that both Debian containers and real
Debian systems install and initialize Nix via the upstream installer,
matching the behaviour on Arch, Fedora, and Ubuntu.

https://chatgpt.com/share/693ab9bf-e6ac-800f-83ba-a4abd1bfe407
2025-12-11 13:31:56 +01:00
Kevin Veen-Birkenbach
eeda944b73 ci: migrate tests to reusable workflows and introduce stable-tag pipeline
- convert all test workflows to reusable workflow_call
- add central CI workflow for branches and PRs
- add mark-stable workflow triggered on main pushes
- ensure stable tag updates only after all tests succeed
- remove duplicated triggers from test workflows
`

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 13:04:44 +01:00
Kevin Veen-Birkenbach
52cfbebba4 ci: make mark-stable robust for workflow_run
- fetch workflow_run runs without head_sha filter
- match by workflow name and head_sha in jq
- keep tagging logic and permissions unchanged

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:46:42 +01:00
Kevin Veen-Birkenbach
f4385807f1 e2e: disable Nix sandbox for cross-distro flake build test
- Update test_nix_build_pkgmgr.py to invoke
    nix --option sandbox false build .#pkgmgr -L
  to avoid sandbox/permission issues in Debian and Ubuntu containers.
- Keeps the test logic identical across all distros while ensuring
  consistent flake build behaviour during E2E runs.

https://chatgpt.com/share/693aa33f-4e3c-800f-86ec-99c38a07eacb
2025-12-11 12:45:04 +01:00
Kevin Veen-Birkenbach
e9e083c9dd ci: finalize mark-stable workflow fixes
- use correct GitHub API path (/repos/.../actions/runs)
- resolve repository via workflow_run.repository.full_name
- improve logging and safe no-tag exits
- ensure correct token handling and tag update logic

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:38:12 +01:00
Kevin Veen-Birkenbach
3218b2b39f ci: fix mark-stable workflow for workflow_run events
- use workflow_run.repository.full_name for gh API queries
- expose GITHUB_TOKEN as GH_TOKEN for the GitHub CLI
- improve log messages and keep tag skipped when checks are missing or failing
2025-12-11 12:26:29 +01:00
Kevin Veen-Birkenbach
ba296a79c9 ci: fix mark-stable permissions and ignore Nix result symlink
https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:16:34 +01:00
Kevin Veen-Birkenbach
62e05e2f5b ci: tag commit as stable after full test matrix
- add mark-stable workflow that runs on workflow_run for all test pipelines
- use GitHub API to ensure all required workflows succeeded before moving the 'stable' tag
- add Nix flake.lock to pin nixpkgs for reproducible builds

https://chatgpt.com/share/693aa4a6-7460-800f-ba47-cfc15b1b2236
2025-12-11 12:01:21 +01:00
Kevin Veen-Birkenbach
77d8b68ba5 Add E2E Nix flake build test across all distro containers
- Introduce tests/e2e/test_nix_build_pkgmgr.py to inspect the Nix environment
  and build the pkgmgr flake inside the container started by test-e2e.sh
- Run the same commands in every distro container: nix --version, sandbox
  config, id, and nix build .#pkgmgr -L
- Print stdout/stderr and assert the flake build succeeds for easier
  cross-distro Nix debugging

https://chatgpt.com/share/693aa33f-4e3c-800f-86ec-99c38a07eacb
2025-12-11 11:55:43 +01:00
Kevin Veen-Birkenbach
bb0a801396 Fix Git safe.directory handling in E2E containers
- Mark /src and /src/.git as safe to satisfy newer Git ownership checks
- Add '*' as safe.directory for ephemeral test containers to avoid Nix flake failures

https://chatgpt.com/share/693a9e1f-1cc8-800f-9df4-90813cbb6bd5
2025-12-11 11:33:51 +01:00
Kevin Veen-Birkenbach
ee968efc4b Harden E2E test runner and fix Git safe.directory in containers
- Quote Nix store/cache volumes and distro image name in docker run
- Use strict bash flags (set -euo pipefail) inside test container
- Print distro ID robustly with fallback
- Configure /src as Git safe.directory when git is available

https://chatgpt.com/share/693a9c0e-59ec-800f-83a1-eec31bd76962
2025-12-11 11:25:11 +01:00
Kevin Veen-Birkenbach
644b2b8fa0 Align Nix Python environment and add lazy CLI import
- Switch flake package and dev shell to Python 3.11 to match pyproject
- Ensure the python-with-deps environment is preferred on PATH in nix develop
- Introduce a lightweight pkgmgr __init__ with lazy loading of pkgmgr.cli
- Avoid pulling in CLI/config dependencies on plain `import pkgmgr`, fixing
  unit test imports and PyYAML availability in the Nix test containers

https://chatgpt.com/share/693a9723-27ac-800f-a6c2-c1bcc91b7dff
2025-12-11 11:04:12 +01:00
Kevin Veen-Birkenbach
0f74907f82 flake.nix: switch to generic python3 and remove side-effects from pkgmgr package root
- Replace hardcoded python311 references with generic python3 to avoid minor
  version pinning and ensure consistent interpreter selection across systems.
- Use python.pkgs instead of python311Packages in the build pipeline.
- Update devShell to use python3.withPackages, including pip and pyyaml.
- Add Python version echo in shellHook for improved debugging.
- Remove cli re-export from src/pkgmgr/__init__.py to eliminate heavy
  side-effects during import and prevent premature config loading in tests.
2025-12-11 10:30:19 +01:00
Kevin Veen-Birkenbach
5a8b1b11de arch packaging: exclude assets from PKGBUILD rsync
Exclude the assets/ directory from the PKGBUILD rsync step to avoid
permission issues (e.g. map.png) when building the Arch package in
Docker as aur_builder.

https://chatgpt.com/share/693a8c25-4464-800f-8d5e-5c4579d78b52
2025-12-11 10:17:14 +01:00
Kevin Veen-Birkenbach
389ec40163 Refine Nix dev shell, ensure PyYAML availability, fix Python invocation, and
expose pkgmgr.cli for Python 3.13 compatibility

- Add `.nix-dev-installed` to .gitignore
- Improve flake.nix:
  * unify pkgs/pyPkgs definitions
  * provide python311.withPackages including pip + PyYAML
  * remove unused pkgmgrPkg reference from devShell
  * fix PYTHONPATH export and devShell help message
- Update unit/integration test scripts to use `python3 -m unittest`
- Add top-level pkgmgr.__init__ exposing `cli` attribute for
  pkgutil.resolve_name compatibility under Python 3.13+
2025-12-11 09:33:55 +01:00
Kevin Veen-Birkenbach
1d03055491 Removed ignore files 2025-12-11 09:07:18 +01:00
Kevin Veen-Birkenbach
7775c6d974 Refine packaging layout and Arch build paths
* Move Arch-specific ignore rules into `packaging/arch/.gitignore` and simplify top-level `.gitignore`/`.dockerignore`.
* Update Arch `PKGBUILD` to sync from the project root and drop `packaging/` from the installed tree.
* Fix OS-specific `package.sh` helpers to resolve the new `packaging/*` locations correctly for Arch, Debian/Ubuntu, Fedora, and CentOS.
2025-12-11 09:04:17 +01:00
Kevin Veen-Birkenbach
a24a819511 Restructure repo layout, wiring src/ and packaging for local and distro builds
- Add dev runner main.py that prefers local src/ over installed pkgmgr
- Move Arch/Debian/Fedora packaging files under packaging/* and update build scripts
- Adjust .gitignore/.dockerignore for new packaging paths and src/source/
- Improve config defaults discovery to support src/ layout and installed packages
- Update architecture diagram and add TODO overview for TAGS/MIRROR/SIGNING_KEY

https://chatgpt.com/share/693a76a0-e408-800f-9939-868524cbef4d
2025-12-11 08:45:07 +01:00
Kevin Veen-Birkenbach
0a6c2f2988 Release version 0.9.1 2025-12-10 22:56:04 +01:00
Kevin Veen-Birkenbach
0c90e984ad Refine setup workflows and add architecture map
- Split virgin tests into separate root and user GitHub Actions workflows
  (test-virgin-root, test-virgin-user) and adjust Arch container flows
- Introduce scripts/installation/venv-create.sh and reuse it from
  scripts/installation/main.sh with separate root/system and user/dev paths
- Add PKGMGR architecture & setup map (assets/map.png) and section in README
  with link to the up-to-date master page
- Simplify README by removing outdated Docker quickstart, usage examples,
  and AI footer
- Extend .gitignore to exclude src/source artifacts

https://chatgpt.com/share/6939bbfe-5cb0-800f-8ea8-95628dc911f5
2025-12-10 22:51:40 +01:00
Kevin Veen-Birkenbach
0a0cbbfe6d fix(init-nix): create 'nix' user with a valid shell across all distros
The init-nix.sh script previously hardcoded /usr/bin/bash as the login shell
for the 'nix' user, which exists on Arch but not on Debian. This caused the
Nix single-user installer (run via `su - nix`) to fail silently or break in
unpredictable ways on Debian-based images.

We now resolve the shell dynamically via `command -v bash` and fall back to
/bin/sh on minimal systems. This makes Nix installation deterministic across
Arch, Debian, Ubuntu, Fedora, CentOS and CI containers.

https://chatgpt.com/share/6939e97f-c93c-800f-887b-27c7e67ec46d
2025-12-10 22:43:20 +01:00
Kevin Veen-Birkenbach
15c44cd484 Removed deprecated pkgmgr.yml 2025-12-10 21:34:33 +01:00
Kevin Veen-Birkenbach
6d7ee6fc04 Fix test scripts: ensure default distro and always run via bash
- Remove Makefile inline variable export (distro=arch) and invoke scripts via bash
- Add robust default in test-unit.sh and test-integration.sh:
    : "${distro:=arch}"
- Prevent "unbound variable" errors under `set -u` when no distro is provided
2025-12-10 21:09:18 +01:00
Kevin Veen-Birkenbach
5a022db0db Use dynamic distro selection for UNIT and INTEGRATION tests
- Pass `distro=arch` from Makefile into test scripts
- Replace hardcoded "arch" references with "${distro}"
- Update test-unit.sh and test-integration.sh to use dynamic image names
- Improve log output to reflect selected distro

https://chatgpt.com/share/6939c98a-d428-800f-8bb8-cf72e80ba80c
2025-12-10 20:27:03 +01:00
Kevin Veen-Birkenbach
37ac22e0b4 test: isolate Nix store/cache per distro to fix cross-distro manifest conflicts
- Replace shared Nix volumes with distro-specific volumes
  (pkgmgr_nix_store_<distro>, pkgmgr_nix_cache_<distro>)
- Prevent incompatible profile manifest versions between Ubuntu and Debian
- Update all test scripts (unit, integration, container, e2e)
- Remove unused global Nix volume variables from Makefile
- Improve consistency of test-e2e.sh formatting and environment handling
- Add Git safe.directory configuration for mounted /src to avoid ownership warnings
2025-12-10 20:07:41 +01:00
Kevin Veen-Birkenbach
bcea440e40 Fix path and shell repo directory resolution + add unit/E2E tests
- Introduce `_resolve_repository_directory()` to unify directory lookup
  (explicit `directory` key → fallback to `get_repo_dir()` using base dir)
- Fix `pkgmgr path` to avoid KeyError and behave consistently with
  other commands using lazy directory resolution
- Fix `pkgmgr shell` to use resolved directory and correctly emit cwd
- Add full E2E tests for `pkgmgr path --all` and `pkgmgr path pkgmgr`
- Add unit tests covering:
    * explicit directory usage
    * fallback resolution via get_repo_dir()
    * empty selection behavior
    * shell command cwd resolution
    * missing shell command error handling
2025-12-10 19:47:26 +01:00
Kevin Veen-Birkenbach
6edde2d65b Release version 0.9.0 2025-12-10 18:38:10 +01:00
Kevin Veen-Birkenbach
74189c1e14 Add virgin Nix flake E2E workflow and update .gitignore
- Introduce `test-nix-flake-e2e.yml` workflow to run a full Arch-based virgin
  environment test with Nix flakes enabled and shared Docker caches
- Ensure pkgmgr self-installation and flake-based installer path are exercised
- Update .gitignore with additional build artifacts, Debian packaging files,
  and pkgmgr output directories
2025-12-10 18:37:29 +01:00
Kevin Veen-Birkenbach
b5ddf7402a Release version 0.8.0 2025-12-10 17:32:00 +01:00
Kevin Veen-Birkenbach
900224ed2e Moved installer dir 2025-12-10 17:27:26 +01:00
Kevin Veen-Birkenbach
e290043089 Refine installer capability integration tests and documentation
- Adjust install_repos integration test to patch resolve_command_for_repo
  in the pipeline module and tighten DummyInstaller overrides
- Rewrite recursive capability integration tests to focus on layer
  ordering and capability shadowing across Makefile, Python, Nix
  and OS-package installers
- Extend recursive capabilities markdown with hierarchy diagram,
  capability matrix, scenario matrix and link to the external
  setup controller schema

https://chatgpt.com/share/69399857-4d84-800f-a636-6bcd1ab5e192
2025-12-10 17:23:33 +01:00
Kevin Veen-Birkenbach
a7fd37d646 Add unit tests for install pipeline, Nix flake installer, and command resolution
https://chatgpt.com/share/69399857-4d84-800f-a636-6bcd1ab5e192
2025-12-10 16:57:02 +01:00
Kevin Veen-Birkenbach
d4b00046d3 Refine installer layering and Python/Nix integration
- Introduce explicit CLI layer model (os-packages, nix, python, makefile)
  and central InstallationPipeline to orchestrate installers.
- Move installer orchestration out of install_repos() into
  pkgmgr.actions.repository.install.pipeline, using layer precedence and
  capability tracking.
- Add pkgmgr.actions.repository.install.layers to classify commands into
  layers and compare priorities.
- Rework PythonInstaller to always use isolated environments:
  PKGMGR_PIP override → active venv → per-repo venv under ~/.venvs/<identifier>,
  avoiding system Python and PEP 668 conflicts.
- Adjust NixFlakeInstaller to install flake outputs based on repository
  identity: pkgmgr/package-manager → pkgmgr (mandatory) + default (optional),
  all other repos → default (mandatory).
- Tighten MakefileInstaller behaviour, add global
  PKGMGR_DISABLE_MAKEFILE_INSTALLER switch, and simplify install target
  detection.
- Rewrite resolve_command_for_repo() with explicit Repository typing,
  better Python package detection, Nix/PATH resolution, and a
  library-only fallback instead of raising on missing CLI.
- Update flake.nix devShell to provide python3 with pip and add pip as a
  propagated build input.
- Remove deprecated/wip repository entries from config defaults and drop
  the unused config/wip.yml.

https://chatgpt.com/share/69399157-86d8-800f-9935-1a820893e908
2025-12-10 16:26:23 +01:00
Kevin Veen-Birkenbach
545d345ea4 core(command): implement explicit command=None bypass and add unit tests
This update introduces Variant B behavior in the command resolver:

- If a repository explicitly defines the key \"command\" (even if its value is None),
  resolve_command_for_repo() treats it as authoritative and returns immediately.
  This allows library-only repositories to declare:
      command: null
  which disables CLI resolution entirely.

- As a result, Python package repositories without installed CLI entry points
  no longer trigger SystemExit during update/install flows, as long as they set
  command: null in their repo configuration.

The resolution logic is now bypassed for such repositories, skipping:
  - Python package detection (src/*/__main__.py)
  - PATH/Nix/venv binary lookup
  - main.sh/main.py fallback evaluation

A new unit test suite has been added under
  tests/unit/pkgmgr/core/command/test_resolve.py
covering:

 1) Python package without installed command → SystemExit
 2) Python package with installed command → returned correctly
 3) Script repository fallback to main.py
 4) Explicit command overrides all logic

This commit stabilizes update/install flows and ensures library-only
repositories behave as intended when no CLI command is provided.

https://chatgpt.com/share/69394a53-bc78-800f-995d-21099a68dd60
2025-12-10 11:23:57 +01:00
Kevin Veen-Birkenbach
a29b831e41 Release version 0.7.14 2025-12-10 10:38:36 +01:00
Kevin Veen-Birkenbach
bc9ca140bd fix(e2e): treat SystemExit(0) as successful CLI termination in clone-all test
The pkgmgr proxy layer may intentionally terminate the process via
SystemExit(0). The previous test logic interpreted any SystemExit as a failure,
causing false negatives during `pkgmgr clone --all` E2E runs.

This patch updates `test_clone_all.py` to:
- accept SystemExit(0) as a successful run,
- only fail on non-zero exit codes,
- preserve diagnostic output for real failures.

This stabilizes the clone-all E2E test across proxy-triggered exits.

https://chatgpt.com/share/69393f6b-b854-800f-aabb-25811bbb8c74
2025-12-10 10:37:40 +01:00
Kevin Veen-Birkenbach
ad8e3cd07c Updated CHANGELOG.md 2025-12-10 10:28:20 +01:00
Kevin Veen-Birkenbach
22efe0b32e Release version 0.7.13 2025-12-10 10:27:27 +01:00
Kevin Veen-Birkenbach
d23a0a94d5 Fix tools path resolution and add tests
- Use _resolve_repository_path() for explore, terminal and code commands
  so tools no longer rely on a 'directory' key in the repository dict.
- Fall back to repositories_base_dir/repositories_dir via get_repo_dir()
  when no explicit path-like key is present.
- Make VS Code workspace creation more robust (safe default for
  directories.workspaces and UTF-8 when writing JSON).
- Add unit tests for handle_tools_command (explore, terminal, code) under
  tests/unit/pkgmgr/cli/commands/test_tools.py.
- Add E2E/integration-style tests for the tools subcommands' --help
  output under tests/e2e/test_tools_help.py, treating SystemExit(0) as
  success.

This change fixes the KeyError: 'directory' when running 'pkgmgr code'
and verifies the behavior via unit and integration tests.

https://chatgpt.com/share/69393ca1-b554-800f-9967-abf8c4e3fea3
2025-12-10 10:25:29 +01:00
Kevin Veen-Birkenbach
e42b79c9d8 Add E2E tests for 'clone --all' and 'update --all' using HTTPS mode
This commit introduces two new end-to-end integration tests:

  • tests/e2e/test_clone_all.py
      Runs: pkgmgr clone --all --clone-mode https --no-verification
      Verifies that full HTTPS cloning of all configured repositories
      works inside the test container environment.

  • tests/e2e/test_update_all.py
      Runs: pkgmgr update --all --clone-mode https --no-verification
      Ensures that updating all repositories with HTTPS mode completes
      successfully without raising exceptions.

Both tests:
  - Provide extended diagnostics on SystemExit
  - Reuse nix-profile cleanup helpers for consistent test environments
  - Validate that `pkgmgr --help` works after execution

These tests complement the existing shallow-install integration test
and improve overall reliability of HTTPS clone/update workflows.
2025-12-09 23:47:43 +01:00
Kevin Veen-Birkenbach
3b2c657bfa Release version 0.7.12 2025-12-09 23:36:38 +01:00
Kevin Veen-Birkenbach
e335ab05a1 fix(core/ink): prevent self-referential symlinks + add unit tests
This commit adds a safety guard to create_ink() to prevent creation of
self-referential symlinks when the resolved command already lives at the
intended link target (e.g. ~/.local/bin/package-manager). Such a situation
previously resulted in broken shells with the error:

    "zsh: too many levels of symbolic links"

Key changes:
  - create_ink():
      • Introduce early-abort guard when command == link_path
      • Improve function signature and formatting
      • Enhance alias creation messaging

  - Added comprehensive unit tests under:
        tests/unit/pkgmgr/core/command/test_ink.py
    Tests cover:
      • Self-referential command path → skip symlink creation
      • Standard symlink + alias creation behaviour

This prevents pkgmgr from overwriting user-managed binaries inside ~/.local/bin
and ensures predictable, safe behaviour across all installer layers.

https://chatgpt.com/share/6938a43b-0eb8-800f-9545-6cb555ab406d
2025-12-09 23:35:29 +01:00
Kevin Veen-Birkenbach
75f963d6e2 Removed tests/e2e/test_install_all_shallow.py 2025-12-09 23:18:49 +01:00
177 changed files with 6050 additions and 2134 deletions

View File

@@ -25,7 +25,5 @@ venv/
.DS_Store
Thumbs.db
# Arch pkg artifacts
*.pkg.tar.*
*.log
package-manager-*
# Logs
*.log

26
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,26 @@
name: CI
on:
push:
branches-ignore:
- main
pull_request:
jobs:
test-unit:
uses: ./.github/workflows/test-unit.yml
test-integration:
uses: ./.github/workflows/test-integration.yml
test-container:
uses: ./.github/workflows/test-container.yml
test-e2e:
uses: ./.github/workflows/test-e2e.yml
test-virgin-user:
uses: ./.github/workflows/test-virgin-user.yml
test-virgin-root:
uses: ./.github/workflows/test-virgin-root.yml

64
.github/workflows/mark-stable.yml vendored Normal file
View File

@@ -0,0 +1,64 @@
name: Mark stable commit
on:
push:
branches:
- main
jobs:
test-unit:
uses: ./.github/workflows/test-unit.yml
test-integration:
uses: ./.github/workflows/test-integration.yml
test-container:
uses: ./.github/workflows/test-container.yml
test-e2e:
uses: ./.github/workflows/test-e2e.yml
test-virgin-user:
uses: ./.github/workflows/test-virgin-user.yml
test-virgin-root:
uses: ./.github/workflows/test-virgin-root.yml
mark-stable:
needs:
- test-unit
- test-integration
- test-container
- test-e2e
- test-virgin-user
- test-virgin-root
runs-on: ubuntu-latest
permissions:
contents: write # to move the tag
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Move 'stable' tag to this commit
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
echo "Tagging commit $GITHUB_SHA as stable…"
# delete local tag if exists
git tag -d stable 2>/dev/null || true
# delete remote tag if exists
git push origin :refs/tags/stable || true
# create new tag on this commit
git tag stable "$GITHUB_SHA"
git push origin stable
echo "✅ Stable tag updated."

View File

@@ -1,25 +1,28 @@
name: Test OS Containers
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-container:
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
distro: [arch, debian, ubuntu, fedora, centos]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Show commit SHA
run: git rev-parse HEAD
- name: Show Docker version
run: docker version
- name: Run container tests
run: make test-container
- name: Run container tests (${{ matrix.distro }})
run: |
set -euo pipefail
distro="${{ matrix.distro }}" make test-container

View File

@@ -1,18 +1,16 @@
name: Test End-To-End
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-e2e:
runs-on: ubuntu-latest
timeout-minutes: 60 # E2E + all distros can be heavier
timeout-minutes: 60 # E2E can be heavier
strategy:
fail-fast: false
matrix:
distro: [arch, debian, ubuntu, fedora, centos]
steps:
- name: Checkout repository
@@ -21,5 +19,7 @@ jobs:
- name: Show Docker version
run: docker version
- name: Run E2E tests via make (all distros)
run: make test-e2e
- name: Run E2E tests via make (${{ matrix.distro }})
run: |
set -euo pipefail
distro="${{ matrix.distro }}" make test-e2e

View File

@@ -1,13 +1,7 @@
name: Test Code Integration
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-integration:
@@ -22,4 +16,4 @@ jobs:
run: docker version
- name: Run integration tests via make (Arch container)
run: make test-integration DISTROS="arch"
run: make test-integration distro="arch"

View File

@@ -1,13 +1,7 @@
name: Test Units
on:
push:
branches:
- main
- master
- develop
- "*"
pull_request:
workflow_call:
jobs:
test-unit:
@@ -22,4 +16,4 @@ jobs:
run: docker version
- name: Run unit tests via make (Arch container)
run: make test-unit DISTROS="arch"
run: make test-unit distro="arch"

58
.github/workflows/test-virgin-root.yml vendored Normal file
View File

@@ -0,0 +1,58 @@
name: Test Virgin Root
on:
workflow_call:
jobs:
test-virgin-root:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Show Docker version
run: docker version
- name: Virgin Arch pkgmgr flake test (root)
run: |
set -euo pipefail
echo ">>> Starting virgin ArchLinux container test (root, with shared caches)..."
docker run --rm \
-v "$PWD":/src \
-v pkgmgr_repos:/root/Repositories \
-v pkgmgr_pip_cache:/root/.cache/pip \
-w /src \
archlinux:latest \
bash -lc '
set -euo pipefail
echo ">>> Updating and upgrading Arch system..."
pacman -Syu --noconfirm git python python-pip nix >/dev/null
echo ">>> Creating isolated virtual environment for pkgmgr..."
python -m venv /tmp/pkgmgr-venv
echo ">>> Activating virtual environment..."
source /tmp/pkgmgr-venv/bin/activate
echo ">>> Upgrading pip (cached)..."
python -m pip install --upgrade pip >/dev/null
echo ">>> Installing pkgmgr from current source tree (cached pip)..."
python -m pip install /src >/dev/null
echo ">>> Enabling Nix experimental features..."
export NIX_CONFIG="experimental-features = nix-command flakes"
echo ">>> Running: pkgmgr update pkgmgr --clone-mode shallow --no-verification"
pkgmgr update pkgmgr --clone-mode shallow --no-verification
echo ">>> Running: pkgmgr version pkgmgr"
pkgmgr version pkgmgr
echo ">>> Virgin Arch (root) test completed successfully."
'

73
.github/workflows/test-virgin-user.yml vendored Normal file
View File

@@ -0,0 +1,73 @@
name: Test Virgin User
on:
workflow_call:
jobs:
test-virgin-user:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Show Docker version
run: docker version
- name: Virgin Arch pkgmgr user test (non-root with sudo)
run: |
set -euo pipefail
echo ">>> Starting virgin ArchLinux container test (non-root user with sudo)..."
docker run --rm \
-v "$PWD":/src \
archlinux:latest \
bash -lc '
set -euo pipefail
echo ">>> [root] Updating and upgrading Arch system..."
pacman -Syu --noconfirm git python python-pip sudo base-devel debugedit
echo ">>> [root] Creating non-root user dev..."
useradd -m dev
echo ">>> [root] Allowing passwordless sudo for dev..."
echo "dev ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/dev
chmod 0440 /etc/sudoers.d/dev
echo ">>> [root] Adjusting ownership of /src for dev..."
chown -R dev:dev /src
echo ">>> [root] Running pkgmgr flow as non-root user dev..."
sudo -u dev env PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 bash -lc "
set -euo pipefail
cd /src
echo \">>> [dev] Using user: \$(whoami)\"
echo \">>> [dev] Running scripts/installation/main.sh...\"
bash scripts/installation/main.sh
echo \">>> [dev] Activating venv...\"
. \"\$HOME/.venvs/pkgmgr/bin/activate\"
echo \">>> [dev] Installing pkgmgr into venv via pip...\"
python -m pip install /src >/dev/null
echo \">>> [dev] PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=\$PKGMGR_DISABLE_NIX_FLAKE_INSTALLER\"
echo \">>> [dev] Updating managed repo package-manager via pkgmgr...\"
pkgmgr update pkgmgr --clone-mode shallow --no-verification
echo \">>> [dev] PATH:\"
echo \"\$PATH\"
echo \">>> [dev] which pkgmgr:\"
which pkgmgr || echo \">>> [dev] pkgmgr not found in PATH\"
echo \">>> [dev] Running: pkgmgr version pkgmgr\"
pkgmgr version pkgmgr
"
echo ">>> [root] Container flow finished."
'

13
.gitignore vendored
View File

@@ -1,9 +1,6 @@
# Prevents unwanted files from being committed to version control.
# Custom Config file
config/config.yaml
# Python bytecode
__pycache__/
*.pyc
@@ -17,6 +14,7 @@ venv/
dist/
build/*
*.egg-info/
package-manager-*
# Editor files
.vscode/
@@ -28,14 +26,9 @@ Thumbs.db
# Nix Cache to speed up tests
.nix/
.nix-dev-installed
# Ignore logs
*.log
package-manager-*
# debian
debian/package-manager/
debian/debhelper-build-stamp
debian/files
debian/.debhelper/
debian/package-manager.substvars
result

View File

@@ -1,3 +1,48 @@
## [0.9.1] - 2025-12-10
* * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
## [0.9.0] - 2025-12-10
* Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
## [0.8.0] - 2025-12-10
* **v0.7.15 — Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
## [0.7.14] - 2025-12-10
* Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.
## [0.7.13] - 2025-12-10
### Fix tools path resolution and add tests
- Fixed a crash in `pkgmgr code` caused by missing `directory` metadata by introducing `_resolve_repository_path()` with proper fallbacks to `repositories_base_dir` / `repositories_dir`.
- Updated `explore`, `terminal` and `code` tool commands to use the new resolver.
- Improved VS Code workspace generation and path handling.
- Added unit & E2E tests for tool commands.
## [0.7.12] - 2025-12-09
* Fixed self refering alias during setup
## [0.7.11] - 2025-12-09
* test: fix installer unit tests for OS packages and Nix dev shell

View File

@@ -1,9 +1,12 @@
# ------------------------------------------------------------
# Base image selector — overridden by Makefile
# ------------------------------------------------------------
ARG BASE_IMAGE=archlinux:latest
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
RUN echo "BASE_IMAGE=${BASE_IMAGE}" && \
cat /etc/os-release || true
# ------------------------------------------------------------
# Nix environment defaults
#

3
MIRRORS Normal file
View File

@@ -0,0 +1,3 @@
git@github.com:kevinveenbirkenbach/package-manager.git
ssh://git@git.veen.world:2201/kevinveenbirkenbach/pkgmgr.git
ssh://git@code.cymais.cloud:2201/kevinveenbirkenbach/pkgmgr.git

View File

@@ -2,17 +2,15 @@
test build build-no-cache test-unit test-e2e test-integration \
test-container
# ------------------------------------------------------------
# Local Nix cache directories in the repo
# ------------------------------------------------------------
NIX_STORE_VOLUME := pkgmgr_nix_store
NIX_CACHE_VOLUME := pkgmgr_nix_cache
# Distro
# Options: arch debian ubuntu fedora centos
distro ?= arch
export distro
# ------------------------------------------------------------
# Distro list and base images
# Base images
# (kept for documentation/reference; actual build logic is in scripts/build)
# ------------------------------------------------------------
DISTROS := arch debian ubuntu fedora centos
BASE_IMAGE_ARCH := archlinux:latest
BASE_IMAGE_DEBIAN := debian:stable-slim
BASE_IMAGE_UBUNTU := ubuntu:latest
@@ -20,7 +18,6 @@ BASE_IMAGE_FEDORA := fedora:latest
BASE_IMAGE_CENTOS := quay.io/centos/centos:stream9
# Make them available in scripts
export DISTROS
export BASE_IMAGE_ARCH
export BASE_IMAGE_DEBIAN
export BASE_IMAGE_UBUNTU
@@ -68,8 +65,13 @@ test-container: build-missing
build-missing:
@bash scripts/build/build-image-missing.sh
# Combined test target for local + CI (unit + e2e + integration)
test: test-container test-unit test-e2e test-integration
# Combined test target for local + CI (unit + integration + e2e)
test: test-container test-unit test-integration test-e2e
delete-volumes:
@docker volume rm pkgmgr_nix_store_${distro} pkgmgr_nix_cache_${distro} || true
purge: delete-volumes build-no-cache
# ------------------------------------------------------------
# System install (native packages, calls scripts/installation/run-package.sh)

View File

@@ -24,6 +24,15 @@
- **Custom Aliases:**
Generate and manage custom aliases for easy command invocation.
## Architecture & Setup Map 🗺️
The following diagram provides a full overview of PKGMGRs package structure,
installation layers, and setup controller flow:
![PKGMGR Architecture](assets/map.png)
**Diagram status:** *Stand: 11. Dezember 2025*
**Always-up-to-date version:** https://s.veen.world/pkgmgrmp
## Installation ⚙️
@@ -51,55 +60,6 @@ The `make setup` command will:
- Install required packages from `requirements.txt`.
- Execute `python main.py install` to complete the installation.
## Docker Quickstart 🐳
Alternatively to installing locally, you can use Docker: build the image with
```bash
docker build --no-cache -t pkgmgr .
```
or alternativ pull it via
```bash
docker pull kevinveenbirkenbach/pkgmgr:latest
```
and then run
```bash
docker run --rm pkgmgr --help
```
## Usage 📖
Run the script with different commands. For example:
- **Install all packages:**
```bash
pkgmgr install --all
```
- **Pull updates for a specific repository:**
```bash
pkgmgr pull pkgmgr
```
- **Commit changes with extra Git parameters:**
```bash
pkgmgr commit pkgmgr -- -m "Your commit message"
```
- **List all configured packages:**
```bash
pkgmgr config show
```
- **Manage configuration:**
```bash
pkgmgr config init
pkgmgr config add
pkgmgr config edit
pkgmgr config delete <identifier>
pkgmgr config ignore <identifier> --set true
```
## License 📄
This project is licensed under the MIT License.
@@ -108,9 +68,3 @@ This project is licensed under the MIT License.
Kevin Veen-Birkenbach
[https://www.veen.world](https://www.veen.world)
---
**Repository:** [github.com/kevinveenbirkenbach/package-manager](https://github.com/kevinveenbirkenbach/package-manager)
*Created with AI 🤖 - [View conversation](https://chatgpt.com/share/67c728c4-92d0-800f-8945-003fa9bf27c6)*

6
TODO.md Normal file
View File

@@ -0,0 +1,6 @@
# to-dos
For the following checkout the implementation map:
- Implement TAGS
- Implement SIGNING_KEY

BIN
assets/map.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

View File

@@ -380,17 +380,6 @@ repositories:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
alias: infinito-presentation
description: This repository contains a Infinito.Nexus presentation designed for customers, end-users, investors, developers, and administrators, offering tailored content and insights for each group.
homepage: https://github.com/kevinveenbirkenbach/infinito-presentation
provider: github.com
repository: infinito-presentation
verified:
gpg_keys:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
description: A lightweight Python utility to generate dynamic color schemes from a single base color. Provides HSL-based color transformations for theming, UI design, and CSS variable generation. Optimized for integration in Python projects, Flask applications, and Ansible roles.
homepage: https://github.com/kevinveenbirkenbach/colorscheme-generator
@@ -599,17 +588,6 @@ repositories:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
desciption: Infinito Inventory Builder — a containerized web application that dynamically generates Ansible inventory files from invokable Infinito.Nexus roles through an interactive, browser-based interface.
homepage: https://github.com/kevinveenbirkenbach/infinito-inventory-builder
alias: invbuild
provider: github.com
repository: infinito-inventory-builder
verified:
gpg_keys:
- 44D8F11FD62F878E
- B5690EEEBB952194
- account: kevinveenbirkenbach
desciption: A simple Python CLI tool to safely rename Linux user accounts using usermod — including home directory migration and validation checks.
homepage: https://github.com/kevinveenbirkenbach/user-rename

View File

@@ -1,7 +0,0 @@
- account: kevinveenbirkenbach
alias: gkfdrtdtcntr
provider: github.com
repository: federated-to-central-social-network-bridge
verified:
gpg_keys:
- 44D8F11FD62F878E

27
flake.lock generated Normal file
View File

@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1765186076,
"narHash": "sha256-hM20uyap1a0M9d344I692r+ik4gTMyj60cQWO+hAYP8=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "addf7cf5f383a3101ecfba091b98d0a1263dc9b8",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

View File

@@ -26,12 +26,17 @@
packages = forAllSystems (system:
let
pkgs = nixpkgs.legacyPackages.${system};
# Single source of truth for pkgmgr: Python 3.11
# - Matches pyproject.toml: requires-python = ">=3.11"
# - Uses python311Packages so that PyYAML etc. are available
python = pkgs.python311;
pyPkgs = pkgs.python311Packages;
in
rec {
pkgmgr = pyPkgs.buildPythonApplication {
pname = "package-manager";
version = "0.7.11";
version = "0.9.1";
# Use the git repo as source
src = ./.;
@@ -45,18 +50,17 @@
pyPkgs.wheel
];
# Runtime dependencies (matches [project.dependencies])
# Runtime dependencies (matches [project.dependencies] in pyproject.toml)
propagatedBuildInputs = [
pyPkgs.pyyaml
# Add more here if needed, e.g.:
# pyPkgs.click
# pyPkgs.rich
pyPkgs.pip
];
doCheck = false;
pythonImportsCheck = [ "pkgmgr" ];
};
default = pkgmgr;
}
);
@@ -67,23 +71,42 @@
devShells = forAllSystems (system:
let
pkgs = nixpkgs.legacyPackages.${system};
pkgmgrPkg = self.packages.${system}.pkgmgr;
ansiblePkg =
if pkgs ? ansible-core then pkgs.ansible-core
else pkgs.ansible;
# Use the same Python version as the package (3.11)
python = pkgs.python311;
pythonWithDeps = python.withPackages (ps: [
ps.pip
ps.pyyaml
]);
in
{
default = pkgs.mkShell {
buildInputs = [
pkgmgrPkg
pythonWithDeps
pkgs.git
ansiblePkg
];
shellHook = ''
# Ensure our Python with dependencies is preferred on PATH
export PATH=${pythonWithDeps}/bin:$PATH
# Ensure src/ layout is importable:
# pkgmgr lives in ./src/pkgmgr
export PYTHONPATH="$PWD/src:${PYTHONPATH:-}"
# Also add repo root in case tools/tests rely on it
export PYTHONPATH="$PWD:$PYTHONPATH"
echo "Entered pkgmgr development shell for ${system}"
echo "pkgmgr CLI is available via the flake build"
echo "Python used in this shell:"
python --version
echo "pkgmgr CLI (from source) is available via:"
echo " python -m pkgmgr.cli --help"
'';
};
}

10
main.py
View File

@@ -1,6 +1,14 @@
#!/usr/bin/env python3
import sys
from pathlib import Path
# Ensure local src/ overrides installed package
ROOT = Path(__file__).resolve().parent
SRC = ROOT / "src"
if SRC.is_dir():
sys.path.insert(0, str(SRC))
from pkgmgr.cli import main
if __name__ == "__main__":
main()
main()

6
packaging/arch/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# Arch pkg artifacts
*.pkg.tar.*
*.log
package-manager-*
src/
pkg/

View File

@@ -1,7 +1,7 @@
# Maintainer: Kevin Veen-Birkenbach <info@veen.world>
pkgname=package-manager
pkgver=0.7.11
pkgver=0.9.1
pkgrel=1
pkgdesc="Local-flake wrapper for Kevin's package-manager (Nix-based)."
arch=('any')
@@ -15,7 +15,7 @@ makedepends=('rsync')
install=${pkgname}.install
# Local source checkout — avoids the tarball requirement.
# This assumes you build the package from inside the main project repository.
# We build from the project root (two levels above packaging/arch/).
source=()
sha256sums=()
@@ -24,12 +24,18 @@ _srcdir_name="source"
prepare() {
mkdir -p "$srcdir/$_srcdir_name"
local project_root
project_root="$(cd "$startdir/../.." && pwd)"
rsync -a \
--exclude=".git" \
--exclude=".github" \
--exclude="pkg" \
--exclude="srcpkg" \
"$startdir/" "$srcdir/$_srcdir_name/"
--exclude="packaging" \
--exclude="assets" \
"$project_root/" "$srcdir/$_srcdir_name/"
}
build() {
@@ -62,7 +68,8 @@ package() {
"$pkgdir/usr/lib/package-manager/PKGBUILD" \
"$pkgdir/usr/lib/package-manager/Dockerfile" \
"$pkgdir/usr/lib/package-manager/debian" \
"$pkgdir/usr/lib/package-manager/packaging" \
"$pkgdir/usr/lib/package-manager/.gitignore" \
"$pkgdir/usr/lib/package-manager/__pycache__" \
"$pkgdir/usr/lib/package-manager/.gitkeep"
"$pkgdir/usr/lib/package-manager/.gitkeep" || true
}

6
packaging/debian/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# debian
package-manager/
debhelper-build-stamp
files
.debhelper/
package-manager.substvars

View File

@@ -1,3 +1,49 @@
package-manager (0.9.1-1) unstable; urgency=medium
* * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 22:56:01 +0100
package-manager (0.9.0-1) unstable; urgency=medium
* Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 18:38:07 +0100
package-manager (0.8.0-1) unstable; urgency=medium
* **v0.7.15 — Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 17:31:57 +0100
package-manager (0.7.14-1) unstable; urgency=medium
* Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 10:38:33 +0100
package-manager (0.7.13-1) unstable; urgency=medium
* Automated release.
-- Kevin Veen-Birkenbach <kevin@veen.world> Wed, 10 Dec 2025 10:27:24 +0100
package-manager (0.7.12-1) unstable; urgency=medium
* Fixed self refering alias during setup
-- Kevin Veen-Birkenbach <kevin@veen.world> Tue, 09 Dec 2025 23:36:35 +0100
package-manager (0.7.11-1) unstable; urgency=medium
* test: fix installer unit tests for OS packages and Nix dev shell

View File

@@ -9,7 +9,7 @@ Homepage: https://github.com/kevinveenbirkenbach/package-manager
Package: package-manager
Architecture: any
Depends: nix, ${misc:Depends}
Depends: ${misc:Depends}
Description: Wrapper that runs Kevin's package-manager via Nix flake
This package provides the `pkgmgr` command, which runs Kevin's package
manager via a local Nix flake

View File

@@ -1,5 +1,5 @@
Name: package-manager
Version: 0.7.11
Version: 0.9.1
Release: 1%{?dist}
Summary: Wrapper that runs Kevin's package-manager via Nix flake
@@ -77,6 +77,34 @@ echo ">>> package-manager removed. Nix itself was not removed."
/usr/lib/package-manager/
%changelog
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.9.1-1
- * Refactored installer: new `venv-create.sh`, cleaner root/user setup flow, updated README with architecture map.
* Split virgin tests into root/user workflows; stabilized Nix installer across distros; improved test scripts with dynamic distro selection and isolated Nix stores.
* Fixed repository directory resolution; improved `pkgmgr path` and `pkgmgr shell`; added full unit/E2E coverage.
* Removed deprecated files and updated `.gitignore`.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.9.0-1
- Introduce a virgin Arch-based Nix flake E2E workflow that validates pkgmgrs full flake installation path using shared caches for faster and reproducible CI runs.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.8.0-1
- **v0.7.15 Installer & Command Resolution Improvements**
* Introduced a unified **layer-based installer pipeline** with clear precedence (OS-packages, Nix, Python, Makefile).
* Reworked installer structure and improved Python/Nix/Makefile installers, including isolated Python venvs and refined flake-output handling.
* Fully rewrote **command resolution** with stronger typing, safer fallbacks, and explicit support for `command: null` to mark library-only repositories.
* Added extensive **unit and integration tests** for installer capability ordering, command resolution, and Nix/Python installer behavior.
* Expanded documentation with capability hierarchy diagrams and scenario matrices.
* Removed deprecated repository entries and obsolete configuration files.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.7.14-1
- Fixed the clone-all integration test so that `SystemExit(0)` from the proxy is treated as a successful command instead of a failure.
* Wed Dec 10 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.7.13-1
- Automated release.
* Tue Dec 09 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.7.12-1
- Fixed self refering alias during setup
* Tue Dec 09 2025 Kevin Veen-Birkenbach <kevin@veen.world> - 0.7.11-1
- test: fix installer unit tests for OS packages and Nix dev shell

View File

@@ -1,7 +0,0 @@
version: 1
author: "Kevin Veen-Birkenbach"
url: "https://github.com/kevinveenbirkenbach/package-manager"
description: "A configurable Python-based package manager for managing multiple repositories via Bash."
dependencies: []

View File

@@ -1,294 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Repository installation pipeline for pkgmgr.
This module orchestrates the installation of repositories by:
1. Ensuring the repository directory exists (cloning if necessary).
2. Verifying the repository according to the configured policies.
3. Creating executable links using create_ink(), after resolving the
appropriate command via resolve_command_for_repo().
4. Running a sequence of modular installer components that handle
specific technologies or manifests (PKGBUILD, Nix flakes, Python
via pyproject.toml, Makefile, OS-specific package metadata).
The goal is to keep this file thin and delegate most logic to small,
focused installer classes.
"""
import os
from typing import List, Dict, Any
from pkgmgr.core.repository.identifier import get_repo_identifier
from pkgmgr.core.repository.dir import get_repo_dir
from pkgmgr.core.command.ink import create_ink
from pkgmgr.core.repository.verify import verify_repository
from pkgmgr.actions.repository.clone import clone_repos
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.core.command.resolve import resolve_command_for_repo
# Installer implementations
from pkgmgr.actions.repository.install.installers.os_packages import (
ArchPkgbuildInstaller,
DebianControlInstaller,
RpmSpecInstaller,
)
from pkgmgr.actions.repository.install.installers.nix_flake import NixFlakeInstaller
from pkgmgr.actions.repository.install.installers.python import PythonInstaller
from pkgmgr.actions.repository.install.installers.makefile import MakefileInstaller
# Layering:
# 1) OS packages: PKGBUILD / debian/control / RPM spec → os-deps.*
# 2) Nix flakes (flake.nix) → e.g. python-runtime, make-install
# 3) Python (pyproject.toml) → e.g. python-runtime, make-install
# 4) Makefile fallback → e.g. make-install
INSTALLERS = [
ArchPkgbuildInstaller(), # Arch
DebianControlInstaller(), # Debian/Ubuntu
RpmSpecInstaller(), # Fedora/RHEL/CentOS
NixFlakeInstaller(), # flake.nix (Nix layer)
PythonInstaller(), # pyproject.toml
MakefileInstaller(), # generic 'make install'
]
def _ensure_repo_dir(
repo: Dict[str, Any],
repositories_base_dir: str,
all_repos: List[Dict[str, Any]],
preview: bool,
no_verification: bool,
clone_mode: str,
identifier: str,
) -> str:
"""
Ensure the repository directory exists. If not, attempt to clone it.
Returns the repository directory path or an empty string if cloning failed.
"""
repo_dir = get_repo_dir(repositories_base_dir, repo)
if not os.path.exists(repo_dir):
print(f"Repository directory '{repo_dir}' does not exist. Cloning it now...")
clone_repos(
[repo],
repositories_base_dir,
all_repos,
preview,
no_verification,
clone_mode,
)
if not os.path.exists(repo_dir):
print(f"Cloning failed for repository {identifier}. Skipping installation.")
return ""
return repo_dir
def _verify_repo(
repo: Dict[str, Any],
repo_dir: str,
no_verification: bool,
identifier: str,
) -> bool:
"""
Verify the repository using verify_repository().
Returns True if installation should proceed, False if it should be skipped.
"""
verified_info = repo.get("verified")
verified_ok, errors, commit_hash, signing_key = verify_repository(
repo,
repo_dir,
mode="local",
no_verification=no_verification,
)
if not no_verification and verified_info and not verified_ok:
print(f"Warning: Verification failed for {identifier}:")
for err in errors:
print(f" - {err}")
choice = input("Proceed with installation? (y/N): ").strip().lower()
if choice != "y":
print(f"Skipping installation for {identifier}.")
return False
return True
def _create_context(
repo: Dict[str, Any],
identifier: str,
repo_dir: str,
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Dict[str, Any]],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> RepoContext:
"""
Build a RepoContext for the given repository and parameters.
"""
return RepoContext(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
def install_repos(
selected_repos: List[Dict[str, Any]],
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Dict[str, Any]],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> None:
"""
Install repositories by creating symbolic links and processing standard
manifest files (PKGBUILD, flake.nix, Python manifests, Makefile, etc.)
via dedicated installer components.
Any installer failure (SystemExit) is treated as fatal and will abort
the current installation.
"""
for repo in selected_repos:
identifier = get_repo_identifier(repo, all_repos)
repo_dir = _ensure_repo_dir(
repo=repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
no_verification=no_verification,
clone_mode=clone_mode,
identifier=identifier,
)
if not repo_dir:
continue
if not _verify_repo(
repo=repo,
repo_dir=repo_dir,
no_verification=no_verification,
identifier=identifier,
):
continue
ctx = _create_context(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
# ------------------------------------------------------------
# Resolve the command for this repository before creating the link.
# If no command is resolved, no link will be created.
# ------------------------------------------------------------
resolved_command = resolve_command_for_repo(
repo=repo,
repo_identifier=identifier,
repo_dir=repo_dir,
)
if resolved_command:
repo["command"] = resolved_command
else:
repo.pop("command", None)
# ------------------------------------------------------------
# Create the symlink using create_ink (if a command is set).
# ------------------------------------------------------------
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
# Track which logical capabilities have already been provided by
# earlier installers for this repository. This allows us to skip
# installers that would only duplicate work (e.g. Python runtime
# already provided by Nix flake → skip pyproject/Makefile).
provided_capabilities: set[str] = set()
# Run all installers that support this repository, but only if they
# provide at least one capability that is not yet satisfied.
for installer in INSTALLERS:
if not installer.supports(ctx):
continue
caps = installer.discover_capabilities(ctx)
# If the installer declares capabilities and *all* of them are
# already provided, we can safely skip it.
if caps and caps.issubset(provided_capabilities):
if not quiet:
print(
f"Skipping installer {installer.__class__.__name__} "
f"for {identifier} capabilities {caps} already provided."
)
continue
# ------------------------------------------------------------
# Debug output + clear error if an installer fails
# ------------------------------------------------------------
if not quiet:
print(
f"[pkgmgr] Running installer {installer.__class__.__name__} "
f"for {identifier} in '{repo_dir}' "
f"(new capabilities: {caps or ''})..."
)
try:
installer.run(ctx)
except SystemExit as exc:
exit_code = exc.code if isinstance(exc.code, int) else str(exc.code)
print(
f"[ERROR] Installer {installer.__class__.__name__} failed "
f"for repository {identifier} (dir: {repo_dir}) "
f"with exit code {exit_code}."
)
print(
"[ERROR] This usually means an underlying command failed "
"(e.g. 'make install', 'nix build', 'pip install', ...)."
)
print(
"[ERROR] Check the log above for the exact command output. "
"You can also run this repository in isolation via:\n"
f" pkgmgr install {identifier} --clone-mode shallow --no-verification"
)
# Re-raise so that CLI/tests fail clearly,
# but now with much more context.
raise
# Only merge capabilities if the installer succeeded
provided_capabilities.update(caps)

View File

@@ -1,19 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer package for pkgmgr.
This exposes all installer classes so users can import them directly from
pkgmgr.actions.repository.install.installers.
"""
from pkgmgr.actions.repository.install.installers.base import BaseInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.nix_flake import NixFlakeInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.python import PythonInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.makefile import MakefileInstaller # noqa: F401
# OS-specific installers
from pkgmgr.actions.repository.install.installers.os_packages.arch_pkgbuild import ArchPkgbuildInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.os_packages.debian_control import DebianControlInstaller # noqa: F401
from pkgmgr.actions.repository.install.installers.os_packages.rpm_spec import RpmSpecInstaller # noqa: F401

View File

@@ -1,93 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer that triggers `make install` if a Makefile is present and
the Makefile actually defines an 'install' target.
This is useful for repositories that expose a standard Makefile-based
installation step.
"""
import os
import re
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
class MakefileInstaller(BaseInstaller):
"""Run `make install` if a Makefile with an 'install' target exists."""
# Logical layer name, used by capability matchers.
layer = "makefile"
MAKEFILE_NAME = "Makefile"
def supports(self, ctx: RepoContext) -> bool:
"""Return True if a Makefile exists in the repository directory."""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
return os.path.exists(makefile_path)
def _has_install_target(self, makefile_path: str) -> bool:
"""
Check whether the Makefile defines an 'install' target.
We treat the presence of a real install target as either:
- a line starting with 'install:' (optionally preceded by whitespace), or
- a .PHONY line that lists 'install' as one of the targets.
"""
try:
with open(makefile_path, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
except OSError:
# If we cannot read the Makefile for some reason, assume no target.
return False
# install: ...
if re.search(r"^\s*install\s*:", content, flags=re.MULTILINE):
return True
# .PHONY: ... install ...
if re.search(r"^\s*\.PHONY\s*:\s*.*\binstall\b", content, flags=re.MULTILINE):
return True
return False
def run(self, ctx: RepoContext) -> None:
"""
Execute `make install` in the repository directory, but only if an
'install' target is actually defined in the Makefile.
Any failure in `make install` is treated as a fatal error and will
propagate as SystemExit from run_command().
"""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
if not os.path.exists(makefile_path):
# Should normally not happen if supports() was checked before,
# but keep this guard for robustness.
if not ctx.quiet:
print(
f"[pkgmgr] Makefile '{makefile_path}' not found, "
"skipping make install."
)
return
if not self._has_install_target(makefile_path):
if not ctx.quiet:
print(
"[pkgmgr] Skipping Makefile install: no 'install' target "
f"found in {makefile_path}."
)
return
if not ctx.quiet:
print(
f"[pkgmgr] Running 'make install' in {ctx.repo_dir} "
"(install target detected in Makefile)."
)
cmd = "make install"
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -1,127 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer for Python projects based on pyproject.toml.
Strategy:
- Determine a pip command in this order:
1. $PKGMGR_PIP (explicit override, e.g. ~/.venvs/pkgmgr/bin/pip)
2. sys.executable -m pip (current interpreter)
3. "pip" from PATH as last resort
- If pyproject.toml exists: pip install .
All installation failures are treated as fatal errors (SystemExit),
except when we explicitly skip the installer:
- If IN_NIX_SHELL is set, we assume Python is managed by Nix and
skip this installer entirely.
- If PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set, the installer is
globally disabled (useful for CI or debugging).
"""
from __future__ import annotations
import os
import sys
from typing import TYPE_CHECKING
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install import InstallContext
class PythonInstaller(BaseInstaller):
"""Install Python projects and dependencies via pip."""
# Logical layer name, used by capability matchers.
layer = "python"
def _in_nix_shell(self) -> bool:
"""
Return True if we appear to be running inside a Nix dev shell.
Nix sets IN_NIX_SHELL in `nix develop` environments. In that case
the Python environment is already provided by Nix, so we must not
attempt an additional pip-based installation.
"""
return bool(os.environ.get("IN_NIX_SHELL"))
def supports(self, ctx: "RepoContext") -> bool:
"""
Return True if this installer should handle the given repository.
Only pyproject.toml is supported as the single source of truth
for Python dependencies and packaging metadata.
The installer is *disabled* when:
- IN_NIX_SHELL is set (Python managed by Nix dev shell), or
- PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set.
"""
# 1) Skip in Nix dev shells Python is managed by the flake/devShell.
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected; skipping PythonInstaller. "
"Python runtime is provided by the Nix dev shell."
)
return False
# 2) Optional global kill-switch.
if os.environ.get("PKGMGR_DISABLE_PYTHON_INSTALLER") == "1":
print(
"[INFO] PKGMGR_DISABLE_PYTHON_INSTALLER=1 "
"PythonInstaller is disabled."
)
return False
repo_dir = ctx.repo_dir
return os.path.exists(os.path.join(repo_dir, "pyproject.toml"))
def _pip_cmd(self) -> str:
"""
Resolve the pip command to use.
Order:
1) PKGMGR_PIP (explicit override)
2) sys.executable -m pip
3) plain "pip"
"""
explicit = os.environ.get("PKGMGR_PIP", "").strip()
if explicit:
return explicit
if sys.executable:
return f"{sys.executable} -m pip"
return "pip"
def run(self, ctx: "InstallContext") -> None:
"""
Install Python project defined via pyproject.toml.
Any pip failure is propagated as SystemExit.
"""
# Extra guard in case run() is called directly without supports().
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected in PythonInstaller.run(); "
"skipping pip-based installation."
)
return
if not self.supports(ctx): # type: ignore[arg-type]
return
pip_cmd = self._pip_cmd()
pyproject = os.path.join(ctx.repo_dir, "pyproject.toml")
if os.path.exists(pyproject):
print(
f"pyproject.toml found in {ctx.identifier}, "
f"installing Python project..."
)
cmd = f"{pip_cmd} install ."
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -1,83 +0,0 @@
from __future__ import annotations
import json
import os
from typing import Any, Dict, List
from pkgmgr.cli.context import CLIContext
from pkgmgr.core.command.run import run_command
from pkgmgr.core.repository.identifier import get_repo_identifier
Repository = Dict[str, Any]
def handle_tools_command(
args,
ctx: CLIContext,
selected: List[Repository],
) -> None:
"""
Handle integration commands:
- explore (file manager)
- terminal (GNOME Terminal)
- code (VS Code workspace)
"""
# --------------------------------------------------------
# explore
# --------------------------------------------------------
if args.command == "explore":
for repository in selected:
run_command(
f"nautilus {repository['directory']} & disown"
)
return
# --------------------------------------------------------
# terminal
# --------------------------------------------------------
if args.command == "terminal":
for repository in selected:
run_command(
f'gnome-terminal --tab --working-directory="{repository["directory"]}"'
)
return
# --------------------------------------------------------
# code
# --------------------------------------------------------
if args.command == "code":
if not selected:
print("No repositories selected.")
return
identifiers = [
get_repo_identifier(repo, ctx.all_repositories)
for repo in selected
]
sorted_identifiers = sorted(identifiers)
workspace_name = "_".join(sorted_identifiers) + ".code-workspace"
workspaces_dir = os.path.expanduser(
ctx.config_merged.get("directories").get("workspaces")
)
os.makedirs(workspaces_dir, exist_ok=True)
workspace_file = os.path.join(workspaces_dir, workspace_name)
folders = [{"path": repository["directory"]} for repository in selected]
workspace_data = {
"folders": folders,
"settings": {},
}
if not os.path.exists(workspace_file):
with open(workspace_file, "w") as f:
json.dump(workspace_data, f, indent=4)
print(f"Created workspace file: {workspace_file}")
else:
print(f"Using existing workspace file: {workspace_file}")
run_command(f'code "{workspace_file}"')
return

View File

@@ -1,505 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from __future__ import annotations
import argparse
from pkgmgr.cli.proxy import register_proxy_commands
class SortedSubParsersAction(argparse._SubParsersAction):
"""
Subparsers action that keeps choices sorted alphabetically.
"""
def add_parser(self, name, **kwargs):
parser = super().add_parser(name, **kwargs)
# Sort choices alphabetically by dest (subcommand name)
self._choices_actions.sort(key=lambda a: a.dest)
return parser
def add_identifier_arguments(subparser: argparse.ArgumentParser) -> None:
"""
Common identifier / selection arguments for many subcommands.
Selection modes (mutual intent, not hard-enforced):
- identifiers (positional): select by alias / provider/account/repo
- --all: select all repositories
- --category / --string / --tag: filter-based selection on top
of the full repository set
"""
subparser.add_argument(
"identifiers",
nargs="*",
help=(
"Identifier(s) for repositories. "
"Default: Repository of current folder."
),
)
subparser.add_argument(
"--all",
action="store_true",
default=False,
help=(
"Apply the subcommand to all repositories in the config. "
"Some subcommands ask for confirmation. If you want to give this "
"confirmation for all repositories, pipe 'yes'. E.g: "
"yes | pkgmgr {subcommand} --all"
),
)
subparser.add_argument(
"--category",
nargs="+",
default=[],
help=(
"Filter repositories by category patterns derived from config "
"filenames or repo metadata (use filename without .yml/.yaml, "
"or /regex/ to use a regular expression)."
),
)
subparser.add_argument(
"--string",
default="",
help=(
"Filter repositories whose identifier / name / path contains this "
"substring (case-insensitive). Use /regex/ for regular expressions."
),
)
subparser.add_argument(
"--tag",
action="append",
default=[],
help=(
"Filter repositories by tag. Matches tags from the repository "
"collector and category tags. Use /regex/ for regular expressions."
),
)
subparser.add_argument(
"--preview",
action="store_true",
help="Preview changes without executing commands",
)
subparser.add_argument(
"--list",
action="store_true",
help="List affected repositories (with preview or status)",
)
subparser.add_argument(
"-a",
"--args",
nargs=argparse.REMAINDER,
dest="extra_args",
help="Additional parameters to be attached.",
default=[],
)
def add_install_update_arguments(subparser: argparse.ArgumentParser) -> None:
"""
Common arguments for install/update commands.
"""
add_identifier_arguments(subparser)
subparser.add_argument(
"-q",
"--quiet",
action="store_true",
help="Suppress warnings and info messages",
)
subparser.add_argument(
"--no-verification",
action="store_true",
default=False,
help="Disable verification via commit/gpg",
)
subparser.add_argument(
"--dependencies",
action="store_true",
help="Also pull and update dependencies",
)
subparser.add_argument(
"--clone-mode",
choices=["ssh", "https", "shallow"],
default="ssh",
help=(
"Specify the clone mode: ssh, https, or shallow "
"(HTTPS shallow clone; default: ssh)"
),
)
def create_parser(description_text: str) -> argparse.ArgumentParser:
"""
Create the top-level argument parser for pkgmgr.
"""
parser = argparse.ArgumentParser(
description=description_text,
formatter_class=argparse.RawTextHelpFormatter,
)
subparsers = parser.add_subparsers(
dest="command",
help="Subcommands",
action=SortedSubParsersAction,
)
# ------------------------------------------------------------
# install / update / deinstall / delete
# ------------------------------------------------------------
install_parser = subparsers.add_parser(
"install",
help="Setup repository/repositories alias links to executables",
)
add_install_update_arguments(install_parser)
update_parser = subparsers.add_parser(
"update",
help="Update (pull + install) repository/repositories",
)
add_install_update_arguments(update_parser)
update_parser.add_argument(
"--system",
action="store_true",
help="Include system update commands",
)
deinstall_parser = subparsers.add_parser(
"deinstall",
help="Remove alias links to repository/repositories",
)
add_identifier_arguments(deinstall_parser)
delete_parser = subparsers.add_parser(
"delete",
help="Delete repository/repositories alias links to executables",
)
add_identifier_arguments(delete_parser)
# ------------------------------------------------------------
# create
# ------------------------------------------------------------
create_cmd_parser = subparsers.add_parser(
"create",
help=(
"Create new repository entries: add them to the config if not "
"already present, initialize the local repository, and push "
"remotely if --remote is set."
),
)
add_identifier_arguments(create_cmd_parser)
create_cmd_parser.add_argument(
"--remote",
action="store_true",
help="If set, add the remote and push the initial commit.",
)
# ------------------------------------------------------------
# status
# ------------------------------------------------------------
status_parser = subparsers.add_parser(
"status",
help="Show status for repository/repositories or system",
)
add_identifier_arguments(status_parser)
status_parser.add_argument(
"--system",
action="store_true",
help="Show system status",
)
# ------------------------------------------------------------
# config
# ------------------------------------------------------------
config_parser = subparsers.add_parser(
"config",
help="Manage configuration",
)
config_subparsers = config_parser.add_subparsers(
dest="subcommand",
help="Config subcommands",
required=True,
)
config_show = config_subparsers.add_parser(
"show",
help="Show configuration",
)
add_identifier_arguments(config_show)
config_subparsers.add_parser(
"add",
help="Interactively add a new repository entry",
)
config_subparsers.add_parser(
"edit",
help="Edit configuration file with nano",
)
config_subparsers.add_parser(
"init",
help="Initialize user configuration by scanning the base directory",
)
config_delete = config_subparsers.add_parser(
"delete",
help="Delete repository entry from user config",
)
add_identifier_arguments(config_delete)
config_ignore = config_subparsers.add_parser(
"ignore",
help="Set ignore flag for repository entries in user config",
)
add_identifier_arguments(config_ignore)
config_ignore.add_argument(
"--set",
choices=["true", "false"],
required=True,
help="Set ignore to true or false",
)
config_subparsers.add_parser(
"update",
help=(
"Update default config files in ~/.config/pkgmgr/ from the "
"installed pkgmgr package (does not touch config.yaml)."
),
)
# ------------------------------------------------------------
# path / explore / terminal / code / shell
# ------------------------------------------------------------
path_parser = subparsers.add_parser(
"path",
help="Print the path(s) of repository/repositories",
)
add_identifier_arguments(path_parser)
explore_parser = subparsers.add_parser(
"explore",
help="Open repository in Nautilus file manager",
)
add_identifier_arguments(explore_parser)
terminal_parser = subparsers.add_parser(
"terminal",
help="Open repository in a new GNOME Terminal tab",
)
add_identifier_arguments(terminal_parser)
code_parser = subparsers.add_parser(
"code",
help="Open repository workspace with VS Code",
)
add_identifier_arguments(code_parser)
shell_parser = subparsers.add_parser(
"shell",
help="Execute a shell command in each repository",
)
add_identifier_arguments(shell_parser)
shell_parser.add_argument(
"-c",
"--command",
nargs=argparse.REMAINDER,
dest="shell_command",
help=(
"The shell command (and its arguments) to execute in each "
"repository"
),
default=[],
)
# ------------------------------------------------------------
# branch
# ------------------------------------------------------------
branch_parser = subparsers.add_parser(
"branch",
help="Branch-related utilities (e.g. open/close feature branches)",
)
branch_subparsers = branch_parser.add_subparsers(
dest="subcommand",
help="Branch subcommands",
required=True,
)
branch_open = branch_subparsers.add_parser(
"open",
help="Create and push a new branch on top of a base branch",
)
branch_open.add_argument(
"name",
nargs="?",
help=(
"Name of the new branch (optional; will be asked interactively "
"if omitted)"
),
)
branch_open.add_argument(
"--base",
default="main",
help="Base branch to create the new branch from (default: main)",
)
branch_close = branch_subparsers.add_parser(
"close",
help="Merge a feature branch into base and delete it",
)
branch_close.add_argument(
"name",
nargs="?",
help=(
"Name of the branch to close (optional; current branch is used "
"if omitted)"
),
)
branch_close.add_argument(
"--base",
default="main",
help=(
"Base branch to merge into (default: main; falls back to master "
"internally if main does not exist)"
),
)
# ------------------------------------------------------------
# release
# ------------------------------------------------------------
release_parser = subparsers.add_parser(
"release",
help=(
"Create a release for repository/ies by incrementing version "
"and updating the changelog."
),
)
release_parser.add_argument(
"release_type",
choices=["major", "minor", "patch"],
help="Type of version increment for the release (major, minor, patch).",
)
release_parser.add_argument(
"-m",
"--message",
default=None,
help=(
"Optional release message to add to the changelog and tag."
),
)
# Generic selection / preview / list / extra_args
add_identifier_arguments(release_parser)
# Close current branch after successful release
release_parser.add_argument(
"--close",
action="store_true",
help=(
"Close the current branch after a successful release in each "
"repository, if it is not main/master."
),
)
# Force: skip preview+confirmation and run release directly
release_parser.add_argument(
"-f",
"--force",
action="store_true",
help=(
"Skip the interactive preview+confirmation step and run the "
"release directly."
),
)
# ------------------------------------------------------------
# version
# ------------------------------------------------------------
version_parser = subparsers.add_parser(
"version",
help=(
"Show version information for repository/ies "
"(git tags, pyproject.toml, flake.nix, PKGBUILD, debian, spec, "
"Ansible Galaxy)."
),
)
add_identifier_arguments(version_parser)
# ------------------------------------------------------------
# changelog
# ------------------------------------------------------------
changelog_parser = subparsers.add_parser(
"changelog",
help=(
"Show changelog derived from Git history. "
"By default, shows the changes between the last two SemVer tags."
),
)
changelog_parser.add_argument(
"range",
nargs="?",
default="",
help=(
"Optional tag or range (e.g. v1.2.3 or v1.2.0..v1.2.3). "
"If omitted, the changelog between the last two SemVer "
"tags is shown."
),
)
add_identifier_arguments(changelog_parser)
# ------------------------------------------------------------
# list
# ------------------------------------------------------------
list_parser = subparsers.add_parser(
"list",
help="List all repositories with details and status",
)
# dieselbe Selektionslogik wie bei install/update/etc.:
add_identifier_arguments(list_parser)
list_parser.add_argument(
"--status",
type=str,
default="",
help=(
"Filter repositories by status (case insensitive). "
"Use /regex/ for regular expressions."
),
)
list_parser.add_argument(
"--description",
action="store_true",
help=(
"Show an additional detailed section per repository "
"(description, homepage, tags, categories, paths)."
),
)
# ------------------------------------------------------------
# make
# ------------------------------------------------------------
make_parser = subparsers.add_parser(
"make",
help="Executes make commands",
)
add_identifier_arguments(make_parser)
make_subparsers = make_parser.add_subparsers(
dest="subcommand",
help="Make subcommands",
required=True,
)
make_install = make_subparsers.add_parser(
"install",
help="Executes the make install command",
)
add_identifier_arguments(make_install)
make_deinstall = make_subparsers.add_parser(
"deinstall",
help="Executes the make deinstall command",
)
add_identifier_arguments(make_deinstall)
# ------------------------------------------------------------
# Proxy commands (git, docker, docker compose, ...)
# ------------------------------------------------------------
register_proxy_commands(subparsers)
return parser

View File

@@ -1,113 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Command resolver for repositories.
This module determines the correct command to expose via symlink.
It implements the following priority:
1. Explicit command in repo config → command
2. System package manager binary (/usr/...) → NO LINK (respect OS)
3. Nix profile binary (~/.nix-profile/bin/<id>) → command
4. Python / non-system console script on PATH → command
5. Fallback: repository's main.sh or main.py → command
6. If nothing is available → raise error
The actual symlink creation is handled by create_ink(). This resolver
only decides *what* should be used as the entrypoint, or whether no
link should be created at all.
"""
import os
import shutil
from typing import Optional
def resolve_command_for_repo(repo, repo_identifier: str, repo_dir: str) -> Optional[str]:
"""
Determine the command for this repository.
Returns:
str → path to the command (a symlink should be created)
None → do NOT create a link (e.g. system package already provides it)
On total failure (no suitable command found at any layer), this function
raises SystemExit with a descriptive error message.
"""
# ------------------------------------------------------------
# 1. Explicit command defined by repository config
# ------------------------------------------------------------
explicit = repo.get("command")
if explicit:
return explicit
home = os.path.expanduser("~")
def is_executable(path: str) -> bool:
return os.path.exists(path) and os.access(path, os.X_OK)
# ------------------------------------------------------------
# 2. System package manager binary via PATH
#
# If the binary lives under /usr/, we treat it as a system-managed
# package (e.g. installed via pacman/apt/yum). In that case, pkgmgr
# does NOT create a link at all and defers entirely to the OS.
# ------------------------------------------------------------
path_candidate = shutil.which(repo_identifier)
system_binary: Optional[str] = None
non_system_binary: Optional[str] = None
if path_candidate:
if path_candidate.startswith("/usr/"):
system_binary = path_candidate
else:
non_system_binary = path_candidate
if system_binary:
# Respect system package manager: do not create a link.
if repo.get("debug", False):
print(
f"[pkgmgr] System binary for '{repo_identifier}' found at "
f"{system_binary}; no symlink will be created."
)
return None
# ------------------------------------------------------------
# 3. Nix profile binary (~/.nix-profile/bin/<identifier>)
# ------------------------------------------------------------
nix_candidate = os.path.join(home, ".nix-profile", "bin", repo_identifier)
if is_executable(nix_candidate):
return nix_candidate
# ------------------------------------------------------------
# 4. Python / non-system console script on PATH
#
# Here we reuse the non-system PATH candidate (e.g. from a venv or
# a user-local install like ~/.local/bin). This is treated as a
# valid command target.
# ------------------------------------------------------------
if non_system_binary and is_executable(non_system_binary):
return non_system_binary
# ------------------------------------------------------------
# 5. Fallback: main.sh / main.py inside the repository
# ------------------------------------------------------------
main_sh = os.path.join(repo_dir, "main.sh")
main_py = os.path.join(repo_dir, "main.py")
if is_executable(main_sh):
return main_sh
if is_executable(main_py) or os.path.exists(main_py):
return main_py
# ------------------------------------------------------------
# 6. Nothing found → treat as a hard error
# ------------------------------------------------------------
raise SystemExit(
f"No executable command could be resolved for repository '{repo_identifier}'. "
"No explicit 'command' configured, no system-managed binary under /usr/, "
"no Nix profile binary, no non-system console script on PATH, and no "
"main.sh/main.py found in the repository."
)

View File

@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "package-manager"
version = "0.7.11"
version = "0.9.1"
description = "Kevin's package-manager tool (pkgmgr)"
readme = "README.md"
requires-python = ">=3.11"
@@ -39,13 +39,13 @@ pkgmgr = "pkgmgr.cli:main"
# -----------------------------
# setuptools configuration
# -----------------------------
# We use find_packages(), not a fixed list,
# and explicitly include pkgmgr* and config*
# Source layout: all packages live under "src/"
[tool.setuptools]
package-dir = { "" = "src", "config" = "config" }
[tool.setuptools.packages.find]
where = ["."]
where = ["src", "."]
include = ["pkgmgr*", "config*"]
# Ensure defaults.yaml is shipped inside wheels & nix builds
[tool.setuptools.package-data]
"config" = ["defaults.yaml"]

View File

@@ -4,32 +4,21 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
source "${SCRIPT_DIR}/resolve-base-image.sh"
echo "============================================================"
echo ">>> Building ONLY missing container images"
echo "============================================================"
IMAGE="package-manager-test-$distro"
BASE_IMAGE="$(resolve_base_image "$distro")"
for distro in $DISTROS; do
IMAGE="package-manager-test-$distro"
BASE_IMAGE="$(resolve_base_image "$distro")"
if docker image inspect "$IMAGE" >/dev/null 2>&1; then
echo "[build-missing] Image already exists: $IMAGE (skipping)"
continue
fi
echo
echo "------------------------------------------------------------"
echo "[build-missing] Building missing image: $IMAGE"
echo "BASE_IMAGE = $BASE_IMAGE"
echo "------------------------------------------------------------"
docker build \
--build-arg BASE_IMAGE="$BASE_IMAGE" \
-t "$IMAGE" \
.
done
if docker image inspect "$IMAGE" >/dev/null 2>&1; then
echo "[build-missing] Image already exists: $IMAGE (skipping)"
exit 0
fi
echo
echo "============================================================"
echo ">>> build-missing: Done"
echo "============================================================"
echo "------------------------------------------------------------"
echo "[build-missing] Building missing image: $IMAGE"
echo "BASE_IMAGE = $BASE_IMAGE"
echo "------------------------------------------------------------"
docker build \
--build-arg BASE_IMAGE="$BASE_IMAGE" \
-t "$IMAGE" \
.

View File

@@ -4,14 +4,12 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
source "${SCRIPT_DIR}/resolve-base-image.sh"
for distro in $DISTROS; do
base_image="$(resolve_base_image "$distro")"
base_image="$(resolve_base_image "$distro")"
echo ">>> Building test image for distro '$distro' with NO CACHE (BASE_IMAGE=$base_image)..."
echo ">>> Building test image for distro '$distro' with NO CACHE (BASE_IMAGE=$base_image)..."
docker build \
--no-cache \
--build-arg BASE_IMAGE="$base_image" \
-t "package-manager-test-$distro" \
.
done
docker build \
--no-cache \
--build-arg BASE_IMAGE="$base_image" \
-t "package-manager-test-$distro" \
.

View File

@@ -4,13 +4,11 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
source "${SCRIPT_DIR}/resolve-base-image.sh"
for distro in $DISTROS; do
base_image="$(resolve_base_image "$distro")"
base_image="$(resolve_base_image "$distro")"
echo ">>> Building test image for distro '$distro' (BASE_IMAGE=$base_image)..."
echo ">>> Building test image for distro '$distro' (BASE_IMAGE=$base_image)..."
docker build \
--build-arg BASE_IMAGE="$base_image" \
-t "package-manager-test-$distro" \
.
done
docker build \
--build-arg BASE_IMAGE="$base_image" \
-t "package-manager-test-$distro" \
.

View File

@@ -94,7 +94,15 @@ if [[ "${IN_CONTAINER}" -eq 1 && "${EUID:-0}" -eq 0 ]]; then
# Ensure "nix" user (home at /home/nix)
if ! id nix >/dev/null 2>&1; then
echo "[init-nix] Creating user 'nix'..."
useradd -m -r -g nixbld -s /usr/bin/bash nix
# Resolve a valid shell path across distros:
# - Debian/Ubuntu: /bin/bash
# - Arch: /usr/bin/bash (often symlinked)
# Fall back to /bin/sh on ultra-minimal systems.
BASH_SHELL="$(command -v bash || true)"
if [[ -z "${BASH_SHELL}" ]]; then
BASH_SHELL="/bin/sh"
fi
useradd -m -r -g nixbld -s "${BASH_SHELL}" nix
fi
# Ensure /nix exists and is writable by the "nix" user.

View File

@@ -3,10 +3,21 @@ set -euo pipefail
echo "[arch/package] Building Arch package (makepkg --nodeps)..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
PKG_DIR="${PROJECT_ROOT}/packaging/arch"
if [[ ! -f "${PKG_DIR}/PKGBUILD" ]]; then
echo "[arch/package] ERROR: PKGBUILD not found in ${PKG_DIR}"
exit 1
fi
cd "${PKG_DIR}"
if id aur_builder >/dev/null 2>&1; then
echo "[arch/package] Using 'aur_builder' user for makepkg..."
chown -R aur_builder:aur_builder "$(pwd)"
su aur_builder -c "cd '$(pwd)' && rm -f package-manager-*.pkg.tar.* && makepkg --noconfirm --clean --nodeps"
chown -R aur_builder:aur_builder "${PKG_DIR}"
su aur_builder -c "cd '${PKG_DIR}' && rm -f package-manager-*.pkg.tar.* && makepkg --noconfirm --clean --nodeps"
else
echo "[arch/package] WARNING: user 'aur_builder' not found, running makepkg as current user..."
rm -f package-manager-*.pkg.tar.*

View File

@@ -4,8 +4,17 @@ set -euo pipefail
echo "[centos/package] Setting up rpmbuild directories..."
mkdir -p /root/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
SPEC_PATH="${PROJECT_ROOT}/packaging/fedora/package-manager.spec"
if [[ ! -f "${SPEC_PATH}" ]]; then
echo "[centos/package] ERROR: SPEC file not found: ${SPEC_PATH}"
exit 1
fi
echo "[centos/package] Extracting version from package-manager.spec..."
version="$(grep -E '^Version:' package-manager.spec | awk '{print $2}')"
version="$(grep -E '^Version:' "${SPEC_PATH}" | awk '{print $2}')"
if [[ -z "${version}" ]]; then
echo "ERROR: Version missing!"
exit 1
@@ -15,13 +24,13 @@ srcdir="package-manager-${version}"
echo "[centos/package] Preparing source tree: ${srcdir}"
rm -rf "/tmp/${srcdir}"
mkdir -p "/tmp/${srcdir}"
cp -a . "/tmp/${srcdir}/"
cp -a "${PROJECT_ROOT}/." "/tmp/${srcdir}/"
echo "[centos/package] Creating source tarball..."
tar czf "/root/rpmbuild/SOURCES/${srcdir}.tar.gz" -C /tmp "${srcdir}"
echo "[centos/package] Copying SPEC..."
cp package-manager.spec /root/rpmbuild/SPECS/
cp "${SPEC_PATH}" /root/rpmbuild/SPECS/
echo "[centos/package] Running rpmbuild..."
cd /root/rpmbuild/SPECS

View File

@@ -3,6 +3,25 @@ set -euo pipefail
echo "[debian/package] Building Debian package..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
BUILD_ROOT="/tmp/package-manager-debian-build"
rm -rf "${BUILD_ROOT}"
mkdir -p "${BUILD_ROOT}"
echo "[debian/package] Syncing project sources to ${BUILD_ROOT}..."
rsync -a \
--exclude 'packaging/debian' \
"${PROJECT_ROOT}/" "${BUILD_ROOT}/"
echo "[debian/package] Overlaying debian/ metadata from packaging/debian..."
mkdir -p "${BUILD_ROOT}/debian"
cp -a "${PROJECT_ROOT}/packaging/debian/." "${BUILD_ROOT}/debian/"
cd "${BUILD_ROOT}"
echo "[debian/package] Running dpkg-buildpackage..."
dpkg-buildpackage -us -uc -b
echo "[debian/package] Installing generated DEB package..."

View File

@@ -4,8 +4,17 @@ set -euo pipefail
echo "[fedora/package] Setting up rpmbuild directories..."
mkdir -p /root/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
SPEC_PATH="${PROJECT_ROOT}/packaging/fedora/package-manager.spec"
if [[ ! -f "${SPEC_PATH}" ]]; then
echo "[fedora/package] ERROR: SPEC file not found: ${SPEC_PATH}"
exit 1
fi
echo "[fedora/package] Extracting version from package-manager.spec..."
version="$(grep -E '^Version:' package-manager.spec | awk '{print $2}')"
version="$(grep -E '^Version:' "${SPEC_PATH}" | awk '{print $2}')"
if [[ -z "${version}" ]]; then
echo "ERROR: Version missing!"
exit 1
@@ -15,13 +24,13 @@ srcdir="package-manager-${version}"
echo "[fedora/package] Preparing source tree: ${srcdir}"
rm -rf "/tmp/${srcdir}"
mkdir -p "/tmp/${srcdir}"
cp -a . "/tmp/${srcdir}/"
cp -a "${PROJECT_ROOT}/." "/tmp/${srcdir}/"
echo "[fedora/package] Creating source tarball..."
tar czf "/root/rpmbuild/SOURCES/${srcdir}.tar.gz" -C /tmp "${srcdir}"
echo "[fedora/package] Copying SPEC..."
cp package-manager.spec /root/rpmbuild/SPECS/
cp "${SPEC_PATH}" /root/rpmbuild/SPECS/
echo "[fedora/package] Running rpmbuild..."
cd /root/rpmbuild/SPECS

View File

@@ -4,20 +4,22 @@ set -euo pipefail
# ------------------------------------------------------------
# main.sh
#
# Developer setup entrypoint.
# Developer / system setup entrypoint.
#
# Responsibilities:
# - If inside a Nix shell (IN_NIX_SHELL=1):
# * Skip venv creation and dependency installation
# * Run `python3 main.py install`
# - Otherwise:
# - If running as root (EUID=0):
# * Run system-level installer (run-package.sh)
# - Otherwise (normal user):
# * Create ~/.venvs/pkgmgr virtual environment if missing
# * Install Python dependencies into that venv
# * Append auto-activation to ~/.bashrc and ~/.zshrc
# * Run `main.py install` using the venv Python
# ------------------------------------------------------------
echo "[installation/main] Starting developer setup..."
echo "[installation/main] Starting setup..."
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "${PROJECT_ROOT}"
@@ -26,20 +28,34 @@ VENV_DIR="${HOME}/.venvs/pkgmgr"
RC_LINE='if [ -d "${HOME}/.venvs/pkgmgr" ]; then . "${HOME}/.venvs/pkgmgr/bin/activate"; if [ -n "${PS1:-}" ]; then echo "Global Python virtual environment '\''~/.venvs/pkgmgr'\'' activated."; fi; fi'
# ------------------------------------------------------------
# Nix shell mode: do not touch venv, only run main.py install
# 1) Nix shell mode: do not touch venv, only run main.py install
# ------------------------------------------------------------
if [[ -n "${IN_NIX_SHELL:-}" ]]; then
echo "[installation/main] Nix shell detected (IN_NIX_SHELL=1)."
echo "[installation/main] Skipping virtualenv creation and dependency installation."
echo "[installation/main] Running main.py install via system python3..."
python3 main.py install
echo "[installation/main] Developer setup finished (Nix mode)."
echo "[installation/main] Setup finished (Nix mode)."
exit 0
fi
# ------------------------------------------------------------
# Normal host mode: create/update venv and run main.py install
# 2) Root mode: system / distro-level installation
# ------------------------------------------------------------
if [[ "${EUID:-$(id -u)}" -eq 0 ]]; then
echo "[installation/main] Running as root (EUID=0)."
echo "[installation/main] Skipping user virtualenv and shell RC modifications."
echo "[installation/main] Delegating to scripts/installation/run-package.sh..."
bash scripts/installation/run-package.sh
echo "[installation/main] Root/system setup complete."
exit 0
fi
# ------------------------------------------------------------
# 3) Normal user mode: dev setup with venv
# ------------------------------------------------------------
echo "[installation/main] Running in normal user mode (developer setup)."
echo "[installation/main] Ensuring main.py is executable..."
chmod +x main.py || true
@@ -47,26 +63,8 @@ chmod +x main.py || true
echo "[installation/main] Ensuring global virtualenv root: ${HOME}/.venvs"
mkdir -p "${HOME}/.venvs"
if [[ ! -d "${VENV_DIR}" ]]; then
echo "[installation/main] Creating virtual environment at: ${VENV_DIR}"
python3 -m venv "${VENV_DIR}"
else
echo "[installation/main] Virtual environment already exists at: ${VENV_DIR}"
fi
echo "[installation/main] Installing Python tooling into venv..."
"${VENV_DIR}/bin/python" -m ensurepip --upgrade
"${VENV_DIR}/bin/pip" install --upgrade pip setuptools wheel
if [[ -f "requirements.txt" ]]; then
echo "[installation/main] Installing dependencies from requirements.txt..."
"${VENV_DIR}/bin/pip" install -r requirements.txt
elif [[ -f "_requirements.txt" ]]; then
echo "[installation/main] Installing dependencies from _requirements.txt..."
"${VENV_DIR}/bin/pip" install -r _requirements.txt
else
echo "[installation/main] No requirements.txt or _requirements.txt found. Skipping dependency installation."
fi
echo "[installation/main] Creating/updating virtualenv via helper..."
PKGMGR_VENV_DIR="${VENV_DIR}" bash scripts/installation/venv-create.sh
echo "[installation/main] Ensuring ~/.bashrc and ~/.zshrc exist..."
touch "${HOME}/.bashrc" "${HOME}/.zshrc"

View File

@@ -8,6 +8,12 @@ source "${SCRIPT_DIR}/lib.sh"
OS_ID="$(detect_os_id)"
# Map Manjaro to Arch
if [[ "${OS_ID}" == "manjaro" ]]; then
echo "[run-package] Mapping OS 'manjaro' → 'arch'"
OS_ID="arch"
fi
echo "[run-package] Detected OS: ${OS_ID}"
case "${OS_ID}" in

View File

@@ -3,6 +3,25 @@ set -euo pipefail
echo "[ubuntu/package] Building Ubuntu (Debian-style) package..."
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
BUILD_ROOT="/tmp/package-manager-ubuntu-build"
rm -rf "${BUILD_ROOT}"
mkdir -p "${BUILD_ROOT}"
echo "[ubuntu/package] Syncing project sources to ${BUILD_ROOT}..."
rsync -a \
--exclude 'packaging/debian' \
"${PROJECT_ROOT}/" "${BUILD_ROOT}/"
echo "[ubuntu/package] Overlaying debian/ metadata from packaging/debian..."
mkdir -p "${BUILD_ROOT}/debian"
cp -a "${PROJECT_ROOT}/packaging/debian/." "${BUILD_ROOT}/debian/"
cd "${BUILD_ROOT}"
echo "[ubuntu/package] Running dpkg-buildpackage..."
dpkg-buildpackage -us -uc -b
echo "[ubuntu/package] Installing generated DEB package..."

View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -euo pipefail
# venv-create.sh
#
# Small helper to create/update a Python virtual environment for pkgmgr.
#
# Usage:
# PKGMGR_VENV_DIR=/home/dev/.venvs/pkgmgr bash scripts/installation/venv-create.sh
# or
# bash scripts/installation/venv-create.sh /home/dev/.venvs/pkgmgr
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "${PROJECT_ROOT}"
VENV_DIR="${PKGMGR_VENV_DIR:-${1:-${HOME}/.venvs/pkgmgr}}"
echo "[venv-create] Using VENV_DIR=${VENV_DIR}"
echo "[venv-create] Ensuring virtualenv parent directory exists..."
mkdir -p "$(dirname "${VENV_DIR}")"
if [[ ! -d "${VENV_DIR}" ]]; then
echo "[venv-create] Creating virtual environment at: ${VENV_DIR}"
python3 -m venv "${VENV_DIR}"
else
echo "[venv-create] Virtual environment already exists at: ${VENV_DIR}"
fi
echo "[venv-create] Installing Python tooling into venv..."
"${VENV_DIR}/bin/python" -m ensurepip --upgrade
"${VENV_DIR}/bin/pip" install --upgrade pip setuptools wheel
if [[ -f "requirements.txt" ]]; then
echo "[venv-create] Installing dependencies from requirements.txt..."
"${VENV_DIR}/bin/pip" install -r requirements.txt
elif [[ -f "_requirements.txt" ]]; then
echo "[venv-create] Installing dependencies from _requirements.txt..."
"${VENV_DIR}/bin/pip" install -r _requirements.txt
else
echo "[venv-create] No requirements.txt or _requirements.txt found. Skipping dependency installation."
fi
echo "[venv-create] Done."

View File

@@ -1,41 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
echo "============================================================"
echo ">>> Running sanity test: verifying test containers start"
echo "============================================================"
for distro in $DISTROS; do
IMAGE="package-manager-test-$distro"
echo
echo "------------------------------------------------------------"
echo ">>> Testing container: $IMAGE"
echo "------------------------------------------------------------"
echo "[test-container] Running: docker run --rm --entrypoint pkgmgr $IMAGE --help"
echo
# Run the command and capture the output
if OUTPUT=$(docker run --rm \
-e PKGMGR_DEV=1 \
-v pkgmgr_nix_store:/nix \
-v "$(pwd):/src" \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
"$IMAGE" 2>&1); then
echo "$OUTPUT"
echo
echo "[test-container] SUCCESS: $IMAGE responded to 'pkgmgr --help'"
else
echo "$OUTPUT"
echo
echo "[test-container] ERROR: $IMAGE failed to run 'pkgmgr --help'"
exit 1
fi
done
IMAGE="package-manager-test-$distro"
echo
echo "============================================================"
echo ">>> All containers passed the sanity check"
echo "============================================================"
echo "------------------------------------------------------------"
echo ">>> Testing container: $IMAGE"
echo "------------------------------------------------------------"
echo "[test-container] Inspect image metadata:"
docker image inspect "$IMAGE" | sed -n '1,40p'
echo "[test-container] Running: docker run --rm --entrypoint pkgmgr $IMAGE --help"
echo
# Run the command and capture the output
if OUTPUT=$(docker run --rm \
-e PKGMGR_DEV=1 \
-v pkgmgr_nix_store_${distro}:/nix \
-v "$(pwd):/src" \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
"$IMAGE" 2>&1); then
echo "$OUTPUT"
echo
echo "[test-container] SUCCESS: $IMAGE responded to 'pkgmgr --help'"
else
echo "$OUTPUT"
echo
echo "[test-container] ERROR: $IMAGE failed to run 'pkgmgr --help'"
exit 1
fi

View File

@@ -1,52 +1,60 @@
#!/usr/bin/env bash
set -euo pipefail
echo ">>> Running E2E tests in all distros: $DISTROS"
echo "============================================================"
echo ">>> Running E2E tests: $distro"
echo "============================================================"
for distro in $DISTROS; do
echo "============================================================"
echo ">>> Running E2E tests: $distro"
echo "============================================================"
docker run --rm \
-v "$(pwd):/src" \
-v "pkgmgr_nix_store_${distro}:/nix" \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--workdir /src \
"package-manager-test-${distro}" \
bash -lc '
set -euo pipefail
docker run --rm \
-v "$(pwd):/src" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--workdir /src \
--entrypoint bash \
"package-manager-test-$distro" \
-c '
set -e;
# Load distro info
if [ -f /etc/os-release ]; then
. /etc/os-release
fi
if [ -f /etc/os-release ]; then
. /etc/os-release;
fi;
echo "Running tests inside distro: ${ID:-unknown}"
echo "Running tests inside distro: $ID";
# Load Nix environment if available
if [ -f "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh" ]; then
. "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh"
fi
# Try to load nix environment
if [ -f "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh" ]; then
. "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh";
fi
if [ -f "$HOME/.nix-profile/etc/profile.d/nix.sh" ]; then
. "$HOME/.nix-profile/etc/profile.d/nix.sh"
fi
if [ -f "$HOME/.nix-profile/etc/profile.d/nix.sh" ]; then
. "$HOME/.nix-profile/etc/profile.d/nix.sh";
fi
PATH="/nix/var/nix/profiles/default/bin:$HOME/.nix-profile/bin:$PATH"
PATH="/nix/var/nix/profiles/default/bin:$HOME/.nix-profile/bin:$PATH";
command -v nix >/dev/null || {
echo "ERROR: nix not found."
exit 1
}
command -v nix >/dev/null || {
echo "ERROR: nix not found.";
exit 1;
}
# Mark the mounted repository as safe to avoid Git ownership errors.
# Newer Git (e.g. on Ubuntu) complains about the gitdir (/src/.git),
# older versions about the worktree (/src). Nix turns "." into the
# flake input "git+file:///src", which then uses Git under the hood.
if command -v git >/dev/null 2>&1; then
# Worktree path
git config --global --add safe.directory /src || true
# Gitdir path shown in the "dubious ownership" error
git config --global --add safe.directory /src/.git || true
# Ephemeral CI containers: allow all paths as a last resort
git config --global --add safe.directory '*' || true
fi
git config --global --add safe.directory /src || true;
nix develop .#default --no-write-lock-file -c \
python3 -m unittest discover \
-s /src/tests/e2e \
-p "$TEST_PATTERN";
'
done
# Run the E2E tests inside the Nix development shell
nix develop .#default --no-write-lock-file -c \
python3 -m unittest discover \
-s /src/tests/e2e \
-p "$TEST_PATTERN"
'

View File

@@ -2,23 +2,22 @@
set -euo pipefail
echo "============================================================"
echo ">>> Running INTEGRATION tests in Arch container"
echo ">>> Running INTEGRATION tests in ${distro} container"
echo "============================================================"
docker run --rm \
-v "$(pwd):/src" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v pkgmgr_nix_store_${distro}:/nix \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
--workdir /src \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--entrypoint bash \
"package-manager-test-arch" \
-c '
"package-manager-test-${distro}" \
bash -lc '
set -e;
git config --global --add safe.directory /src || true;
nix develop .#default --no-write-lock-file -c \
python -m unittest discover \
python3 -m unittest discover \
-s tests/integration \
-t /src \
-p "$TEST_PATTERN";

View File

@@ -2,23 +2,22 @@
set -euo pipefail
echo "============================================================"
echo ">>> Running UNIT tests in Arch container"
echo ">>> Running UNIT tests in ${distro} container"
echo "============================================================"
docker run --rm \
-v "$(pwd):/src" \
-v "pkgmgr_nix_cache:/root/.cache/nix" \
-v pkgmgr_nix_store:/nix \
-v "pkgmgr_nix_cache_${distro}:/root/.cache/nix" \
-v pkgmgr_nix_store_${distro}:/nix \
--workdir /src \
-e PKGMGR_DEV=1 \
-e TEST_PATTERN="${TEST_PATTERN}" \
--entrypoint bash \
"package-manager-test-arch" \
-c '
"package-manager-test-${distro}" \
bash -lc '
set -e;
git config --global --add safe.directory /src || true;
nix develop .#default --no-write-lock-file -c \
python -m unittest discover \
python3 -m unittest discover \
-s tests/unit \
-t /src \
-p "$TEST_PATTERN";

36
src/pkgmgr/__init__.py Normal file
View File

@@ -0,0 +1,36 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Top-level pkgmgr package.
We deliberately avoid importing heavy submodules (like the CLI)
on import to keep unit tests fast and to not require optional
dependencies (like PyYAML) unless they are actually used.
Accessing ``pkgmgr.cli`` will load the CLI module lazily via
``__getattr__``. This keeps patterns like
from pkgmgr import cli
working as expected in tests and entry points.
"""
from __future__ import annotations
from importlib import import_module
from typing import Any
__all__ = ["cli"]
def __getattr__(name: str) -> Any:
"""
Lazily expose ``pkgmgr.cli`` as attribute on the top-level package.
This keeps ``import pkgmgr`` lightweight while still allowing
``from pkgmgr import cli`` in tests and entry points.
"""
if name == "cli":
return import_module("pkgmgr.cli")
raise AttributeError(f"module 'pkgmgr' has no attribute {name!r}")

View File

@@ -0,0 +1,218 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
High-level entry point for repository installation.
Responsibilities:
- Ensure the repository directory exists (clone if necessary).
- Verify the repository (GPG / commit checks).
- Build a RepoContext object.
- Delegate the actual installation decision logic to InstallationPipeline.
"""
from __future__ import annotations
import os
from typing import Any, Dict, List
from pkgmgr.core.repository.identifier import get_repo_identifier
from pkgmgr.core.repository.dir import get_repo_dir
from pkgmgr.core.repository.verify import verify_repository
from pkgmgr.actions.repository.clone import clone_repos
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.os_packages import (
ArchPkgbuildInstaller,
DebianControlInstaller,
RpmSpecInstaller,
)
from pkgmgr.actions.install.installers.nix_flake import (
NixFlakeInstaller,
)
from pkgmgr.actions.install.installers.python import PythonInstaller
from pkgmgr.actions.install.installers.makefile import (
MakefileInstaller,
)
from pkgmgr.actions.install.pipeline import InstallationPipeline
Repository = Dict[str, Any]
# All available installers, in the order they should be considered.
INSTALLERS = [
ArchPkgbuildInstaller(),
DebianControlInstaller(),
RpmSpecInstaller(),
NixFlakeInstaller(),
PythonInstaller(),
MakefileInstaller(),
]
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _ensure_repo_dir(
repo: Repository,
repositories_base_dir: str,
all_repos: List[Repository],
preview: bool,
no_verification: bool,
clone_mode: str,
identifier: str,
) -> str | None:
"""
Compute and, if necessary, clone the repository directory.
Returns the absolute repository path or None if cloning ultimately failed.
"""
repo_dir = get_repo_dir(repositories_base_dir, repo)
if not os.path.exists(repo_dir):
print(
f"Repository directory '{repo_dir}' does not exist. "
f"Cloning it now..."
)
clone_repos(
[repo],
repositories_base_dir,
all_repos,
preview,
no_verification,
clone_mode,
)
if not os.path.exists(repo_dir):
print(
f"Cloning failed for repository {identifier}. "
f"Skipping installation."
)
return None
return repo_dir
def _verify_repo(
repo: Repository,
repo_dir: str,
no_verification: bool,
identifier: str,
) -> bool:
"""
Verify a repository using the configured verification data.
Returns True if verification is considered okay and installation may continue.
"""
verified_info = repo.get("verified")
verified_ok, errors, _commit_hash, _signing_key = verify_repository(
repo,
repo_dir,
mode="local",
no_verification=no_verification,
)
if not no_verification and verified_info and not verified_ok:
print(f"Warning: Verification failed for {identifier}:")
for err in errors:
print(f" - {err}")
choice = input("Continue anyway? [y/N]: ").strip().lower()
if choice != "y":
print(f"Skipping installation for {identifier}.")
return False
return True
def _create_context(
repo: Repository,
identifier: str,
repo_dir: str,
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Repository],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> RepoContext:
"""
Build a RepoContext instance for the given repository.
"""
return RepoContext(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def install_repos(
selected_repos: List[Repository],
repositories_base_dir: str,
bin_dir: str,
all_repos: List[Repository],
no_verification: bool,
preview: bool,
quiet: bool,
clone_mode: str,
update_dependencies: bool,
) -> None:
"""
Install one or more repositories according to the configured installers
and the CLI layer precedence rules.
"""
pipeline = InstallationPipeline(INSTALLERS)
for repo in selected_repos:
identifier = get_repo_identifier(repo, all_repos)
repo_dir = _ensure_repo_dir(
repo=repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
no_verification=no_verification,
clone_mode=clone_mode,
identifier=identifier,
)
if not repo_dir:
continue
if not _verify_repo(
repo=repo,
repo_dir=repo_dir,
no_verification=no_verification,
identifier=identifier,
):
continue
ctx = _create_context(
repo=repo,
identifier=identifier,
repo_dir=repo_dir,
repositories_base_dir=repositories_base_dir,
bin_dir=bin_dir,
all_repos=all_repos,
no_verification=no_verification,
preview=preview,
quiet=quiet,
clone_mode=clone_mode,
update_dependencies=update_dependencies,
)
pipeline.run(ctx)

View File

@@ -38,7 +38,7 @@ from abc import ABC, abstractmethod
from typing import Iterable, TYPE_CHECKING
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.install.context import RepoContext
# ---------------------------------------------------------------------------

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installer package for pkgmgr.
This exposes all installer classes so users can import them directly from
pkgmgr.actions.install.installers.
"""
from pkgmgr.actions.install.installers.base import BaseInstaller # noqa: F401
from pkgmgr.actions.install.installers.nix_flake import NixFlakeInstaller # noqa: F401
from pkgmgr.actions.install.installers.python import PythonInstaller # noqa: F401
from pkgmgr.actions.install.installers.makefile import MakefileInstaller # noqa: F401
# OS-specific installers
from pkgmgr.actions.install.installers.os_packages.arch_pkgbuild import ArchPkgbuildInstaller # noqa: F401
from pkgmgr.actions.install.installers.os_packages.debian_control import DebianControlInstaller # noqa: F401
from pkgmgr.actions.install.installers.os_packages.rpm_spec import RpmSpecInstaller # noqa: F401

View File

@@ -8,8 +8,8 @@ Base interface for all installer components in the pkgmgr installation pipeline.
from abc import ABC, abstractmethod
from typing import Set
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.capabilities import CAPABILITY_MATCHERS
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.capabilities import CAPABILITY_MATCHERS
class BaseInstaller(ABC):

View File

@@ -0,0 +1,97 @@
from __future__ import annotations
import os
import re
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
class MakefileInstaller(BaseInstaller):
"""
Generic installer that runs `make install` if a Makefile with an
install target is present.
Safety rules:
- If PKGMGR_DISABLE_MAKEFILE_INSTALLER=1 is set, this installer
is globally disabled.
- The higher-level InstallationPipeline ensures that Makefile
installation does not run if a stronger CLI layer already owns
the command (e.g. Nix or OS packages).
"""
layer = "makefile"
MAKEFILE_NAME = "Makefile"
def supports(self, ctx: RepoContext) -> bool:
"""
Return True if this repository has a Makefile and the installer
is not globally disabled.
"""
# Optional global kill switch.
if os.environ.get("PKGMGR_DISABLE_MAKEFILE_INSTALLER") == "1":
if not ctx.quiet:
print(
"[INFO] MakefileInstaller is disabled via "
"PKGMGR_DISABLE_MAKEFILE_INSTALLER."
)
return False
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
return os.path.exists(makefile_path)
def _has_install_target(self, makefile_path: str) -> bool:
"""
Heuristically check whether the Makefile defines an install target.
We look for:
- a plain 'install:' target, or
- any 'install-*:' style target.
"""
try:
with open(makefile_path, "r", encoding="utf-8", errors="ignore") as f:
content = f.read()
except OSError:
return False
# Simple heuristics: look for "install:" or targets starting with "install-"
if re.search(r"^install\s*:", content, flags=re.MULTILINE):
return True
if re.search(r"^install-[a-zA-Z0-9_-]*\s*:", content, flags=re.MULTILINE):
return True
return False
def run(self, ctx: RepoContext) -> None:
"""
Execute `make install` in the repository directory if an install
target exists.
"""
makefile_path = os.path.join(ctx.repo_dir, self.MAKEFILE_NAME)
if not os.path.exists(makefile_path):
if not ctx.quiet:
print(
f"[pkgmgr] Makefile '{makefile_path}' not found, "
"skipping MakefileInstaller."
)
return
if not self._has_install_target(makefile_path):
if not ctx.quiet:
print(
f"[pkgmgr] No 'install' target found in {makefile_path}."
)
return
if not ctx.quiet:
print(
f"[pkgmgr] Running 'make install' in {ctx.repo_dir} "
f"(MakefileInstaller)"
)
cmd = "make install"
run_command(cmd, cwd=ctx.repo_dir, preview=ctx.preview)

View File

@@ -10,28 +10,31 @@ installer will try to install profile outputs from the flake.
Behavior:
- If flake.nix is present and `nix` exists on PATH:
* First remove any existing `package-manager` profile entry (best-effort).
* Then install the flake outputs (`pkgmgr`, `default`) via `nix profile install`.
- Failure installing `pkgmgr` is treated as fatal.
- Failure installing `default` is logged as an error/warning but does not abort.
* Then install one or more flake outputs via `nix profile install`.
- For the package-manager repo:
* `pkgmgr` is mandatory (CLI), `default` is optional.
- For all other repos:
* `default` is mandatory.
Special handling for dev shells / CI:
- If IN_NIX_SHELL is set (e.g. inside `nix develop`), the installer is
disabled. In that environment the flake outputs are already provided
by the dev shell and we must not touch the user profile.
Special handling:
- If PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 is set, the installer is
globally disabled (useful for CI or debugging).
The higher-level InstallationPipeline and CLI-layer model decide when this
installer is allowed to run, based on where the current CLI comes from
(e.g. Nix, OS packages, Python, Makefile).
"""
import os
import shutil
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, List, Tuple
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install import InstallContext
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install import InstallContext
class NixFlakeInstaller(BaseInstaller):
@@ -43,33 +46,14 @@ class NixFlakeInstaller(BaseInstaller):
FLAKE_FILE = "flake.nix"
PROFILE_NAME = "package-manager"
def _in_nix_shell(self) -> bool:
"""
Return True if we appear to be running inside a Nix dev shell.
Nix sets IN_NIX_SHELL in `nix develop` environments. In that case
the flake outputs are already available, and touching the user
profile (nix profile install/remove) is undesirable.
"""
return bool(os.environ.get("IN_NIX_SHELL"))
def supports(self, ctx: "RepoContext") -> bool:
"""
Only support repositories that:
- Are NOT inside a Nix dev shell (IN_NIX_SHELL unset),
- Are NOT explicitly disabled via PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1,
- Have a flake.nix,
- And have the `nix` command available.
"""
# 1) Skip when running inside a dev shell flake is already active.
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected; skipping NixFlakeInstaller. "
"Flake outputs are provided by the development shell."
)
return False
# 2) Optional global kill-switch for CI or debugging.
# Optional global kill-switch for CI or debugging.
if os.environ.get("PKGMGR_DISABLE_NIX_FLAKE_INSTALLER") == "1":
print(
"[INFO] PKGMGR_DISABLE_NIX_FLAKE_INSTALLER=1 "
@@ -77,11 +61,11 @@ class NixFlakeInstaller(BaseInstaller):
)
return False
# 3) Nix must be available.
# Nix must be available.
if shutil.which("nix") is None:
return False
# 4) flake.nix must exist in the repository.
# flake.nix must exist in the repository.
flake_path = os.path.join(ctx.repo_dir, self.FLAKE_FILE)
return os.path.exists(flake_path)
@@ -107,36 +91,56 @@ class NixFlakeInstaller(BaseInstaller):
# Unit tests explicitly assert this is swallowed
pass
def _profile_outputs(self, ctx: "RepoContext") -> List[Tuple[str, bool]]:
"""
Decide which flake outputs to install and whether failures are fatal.
Returns a list of (output_name, allow_failure) tuples.
Rules:
- For the package-manager repo (identifier 'pkgmgr' or 'package-manager'):
[("pkgmgr", False), ("default", True)]
- For all other repos:
[("default", False)]
"""
ident = ctx.identifier
if ident in {"pkgmgr", "package-manager"}:
# pkgmgr: main CLI output is "pkgmgr" (mandatory),
# "default" is nice-to-have (non-fatal).
return [("pkgmgr", False), ("default", True)]
# Generic repos: we expect a sensible "default" package/app.
# Failure to install it is considered fatal.
return [("default", False)]
def run(self, ctx: "InstallContext") -> None:
"""
Install Nix flake profile outputs (pkgmgr, default).
Install Nix flake profile outputs.
Any failure installing `pkgmgr` is treated as fatal (SystemExit).
A failure installing `default` is logged but does not abort.
For the package-manager repo, failure installing 'pkgmgr' is fatal,
failure installing 'default' is non-fatal.
For other repos, failure installing 'default' is fatal.
"""
# Extra guard in case run() is called directly without supports().
if self._in_nix_shell():
print(
"[INFO] IN_NIX_SHELL detected in run(); "
"skipping Nix flake profile installation."
)
return
# Reuse supports() to keep logic in one place
# Reuse supports() to keep logic in one place.
if not self.supports(ctx): # type: ignore[arg-type]
return
print("Nix flake detected, attempting to install profile outputs...")
outputs = self._profile_outputs(ctx) # list of (name, allow_failure)
# Handle the "already installed" case up-front:
print(
"Nix flake detected in "
f"{ctx.identifier}, attempting to install profile outputs: "
+ ", ".join(name for name, _ in outputs)
)
# Handle the "already installed" case up-front for the shared profile.
self._ensure_old_profile_removed(ctx) # type: ignore[arg-type]
for output in ("pkgmgr", "default"):
for output, allow_failure in outputs:
cmd = f"nix profile install {ctx.repo_dir}#{output}"
try:
# For 'default' we don't want the process to exit on error
allow_failure = output == "default"
run_command(
cmd,
cwd=ctx.repo_dir,
@@ -146,12 +150,11 @@ class NixFlakeInstaller(BaseInstaller):
print(f"Nix flake output '{output}' successfully installed.")
except SystemExit as e:
print(f"[Error] Failed to install Nix flake output '{output}': {e}")
if output == "pkgmgr":
# Broken main CLI install → fatal
if not allow_failure:
# Mandatory output failed → fatal for the pipeline.
raise
# For 'default' we log and continue
# Optional output failed → log and continue.
print(
"[Warning] Continuing despite failure to install 'default' "
"because 'pkgmgr' is already installed."
"[Warning] Continuing despite failure to install "
f"optional output '{output}'."
)
break

View File

@@ -3,8 +3,8 @@
import os
import shutil
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -19,8 +19,8 @@ import os
import shutil
from typing import List
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -21,8 +21,8 @@ import shutil
import tarfile
from typing import List, Optional, Tuple
from pkgmgr.actions.repository.install.context import RepoContext
from pkgmgr.actions.repository.install.installers.base import BaseInstaller
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command

View File

@@ -0,0 +1,139 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
PythonInstaller — install Python projects defined via pyproject.toml.
Installation rules:
1. pip command resolution:
a) If PKGMGR_PIP is set → use it exactly as provided.
b) Else if running inside a virtualenv → use `sys.executable -m pip`.
c) Else → create/use a per-repository virtualenv under ~/.venvs/<repo>/.
2. Installation target:
- Always install into the resolved pip environment.
- Never modify system Python, never rely on --user.
- Nix-immutable systems (PEP 668) are automatically avoided because we
never touch system Python.
3. The installer is skipped when:
- PKGMGR_DISABLE_PYTHON_INSTALLER=1 is set.
- The repository has no pyproject.toml.
All pip failures are treated as fatal.
"""
from __future__ import annotations
import os
import sys
import subprocess
from typing import TYPE_CHECKING
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.core.command.run import run_command
if TYPE_CHECKING:
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install import InstallContext
class PythonInstaller(BaseInstaller):
"""Install Python projects and dependencies via pip using isolated environments."""
layer = "python"
# ----------------------------------------------------------------------
# Installer activation logic
# ----------------------------------------------------------------------
def supports(self, ctx: "RepoContext") -> bool:
"""
Return True if this installer should handle this repository.
The installer is active only when:
- A pyproject.toml exists in the repo, and
- PKGMGR_DISABLE_PYTHON_INSTALLER is not set.
"""
if os.environ.get("PKGMGR_DISABLE_PYTHON_INSTALLER") == "1":
print("[INFO] PythonInstaller disabled via PKGMGR_DISABLE_PYTHON_INSTALLER.")
return False
return os.path.exists(os.path.join(ctx.repo_dir, "pyproject.toml"))
# ----------------------------------------------------------------------
# Virtualenv handling
# ----------------------------------------------------------------------
def _in_virtualenv(self) -> bool:
"""Detect whether the current interpreter is inside a venv."""
if os.environ.get("VIRTUAL_ENV"):
return True
base = getattr(sys, "base_prefix", sys.prefix)
return sys.prefix != base
def _ensure_repo_venv(self, ctx: "InstallContext") -> str:
"""
Ensure that ~/.venvs/<identifier>/ exists and contains a minimal venv.
Returns the venv directory path.
"""
venv_dir = os.path.expanduser(f"~/.venvs/{ctx.identifier}")
python = sys.executable
if not os.path.isdir(venv_dir):
print(f"[python-installer] Creating virtualenv: {venv_dir}")
subprocess.check_call([python, "-m", "venv", venv_dir])
return venv_dir
# ----------------------------------------------------------------------
# pip command resolution
# ----------------------------------------------------------------------
def _pip_cmd(self, ctx: "InstallContext") -> str:
"""
Determine which pip command to use.
Priority:
1. PKGMGR_PIP override given by user or automation.
2. Active virtualenv → use sys.executable -m pip.
3. Per-repository venv → ~/.venvs/<repo>/bin/pip
"""
explicit = os.environ.get("PKGMGR_PIP", "").strip()
if explicit:
return explicit
if self._in_virtualenv():
return f"{sys.executable} -m pip"
venv_dir = self._ensure_repo_venv(ctx)
pip_path = os.path.join(venv_dir, "bin", "pip")
return pip_path
# ----------------------------------------------------------------------
# Execution
# ----------------------------------------------------------------------
def run(self, ctx: "InstallContext") -> None:
"""
Install the project defined by pyproject.toml.
Uses the resolved pip environment. Installation is isolated and never
touches system Python.
"""
if not self.supports(ctx): # type: ignore[arg-type]
return
pyproject = os.path.join(ctx.repo_dir, "pyproject.toml")
if not os.path.exists(pyproject):
return
print(f"[python-installer] Installing Python project for {ctx.identifier}...")
pip_cmd = self._pip_cmd(ctx)
# Final install command: ALWAYS isolated, never system-wide.
install_cmd = f"{pip_cmd} install ."
run_command(install_cmd, cwd=ctx.repo_dir, preview=ctx.preview)
print(f"[python-installer] Installation finished for {ctx.identifier}.")

View File

@@ -0,0 +1,91 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
CLI layer model for the pkgmgr installation pipeline.
We treat CLI entry points as coming from one of four conceptual layers:
- os-packages : system package managers (pacman/apt/dnf/…)
- nix : Nix flake / nix profile
- python : pip / virtualenv / user-local scripts
- makefile : repo-local Makefile / scripts inside the repo
The layer order defines precedence: higher layers "own" the CLI and
lower layers will not be executed once a higher-priority CLI exists.
"""
from __future__ import annotations
import os
from enum import Enum
from typing import Optional
class CliLayer(str, Enum):
OS_PACKAGES = "os-packages"
NIX = "nix"
PYTHON = "python"
MAKEFILE = "makefile"
# Highest priority first
CLI_LAYERS: list[CliLayer] = [
CliLayer.OS_PACKAGES,
CliLayer.NIX,
CliLayer.PYTHON,
CliLayer.MAKEFILE,
]
def layer_priority(layer: Optional[CliLayer]) -> int:
"""
Return a numeric priority index for a given layer.
Lower index → higher priority.
Unknown / None → very low priority.
"""
if layer is None:
return len(CLI_LAYERS)
try:
return CLI_LAYERS.index(layer)
except ValueError:
return len(CLI_LAYERS)
def classify_command_layer(command: str, repo_dir: str) -> CliLayer:
"""
Heuristically classify a resolved command path into a CLI layer.
Rules (best effort):
- /usr/... or /bin/... → os-packages
- /nix/store/... or ~/.nix-profile → nix
- ~/.local/bin/... → python
- inside repo_dir → makefile
- everything else → python (user/venv scripts, etc.)
"""
command_abs = os.path.abspath(os.path.expanduser(command))
repo_abs = os.path.abspath(repo_dir)
home = os.path.expanduser("~")
# OS package managers
if command_abs.startswith("/usr/") or command_abs.startswith("/bin/"):
return CliLayer.OS_PACKAGES
# Nix store / profile
if command_abs.startswith("/nix/store/") or command_abs.startswith(
os.path.join(home, ".nix-profile")
):
return CliLayer.NIX
# User-local bin
if command_abs.startswith(os.path.join(home, ".local", "bin")):
return CliLayer.PYTHON
# Inside the repository → usually a Makefile/script
if command_abs.startswith(repo_abs):
return CliLayer.MAKEFILE
# Fallback: treat as Python-style/user-level script
return CliLayer.PYTHON

View File

@@ -0,0 +1,257 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Installation pipeline orchestration for repositories.
This module implements the "Setup Controller" logic:
1. Detect current CLI command for the repo (if any).
2. Classify it into a layer (os-packages, nix, python, makefile).
3. Iterate over installers in layer order:
- Skip installers whose layer is weaker than an already-loaded one.
- Run only installers that support() the repo and add new capabilities.
- After each installer, re-resolve the command and update the layer.
4. Maintain the repo["command"] field and create/update symlinks via create_ink().
The goal is to prevent conflicting installations and make the layering
behaviour explicit and testable.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Optional, Sequence, Set
from pkgmgr.actions.install.context import RepoContext
from pkgmgr.actions.install.installers.base import BaseInstaller
from pkgmgr.actions.install.layers import (
CliLayer,
classify_command_layer,
layer_priority,
)
from pkgmgr.core.command.ink import create_ink
from pkgmgr.core.command.resolve import resolve_command_for_repo
@dataclass
class CommandState:
"""
Represents the current CLI state for a repository:
- command: absolute or relative path to the CLI entry point
- layer: which conceptual layer this command belongs to
"""
command: Optional[str]
layer: Optional[CliLayer]
class CommandResolver:
"""
Small helper responsible for resolving the current command for a repo
and mapping it into a CommandState.
"""
def __init__(self, ctx: RepoContext) -> None:
self._ctx = ctx
def resolve(self) -> CommandState:
"""
Resolve the current command for this repository.
If resolve_command_for_repo raises SystemExit (e.g. Python package
without installed entry point), we treat this as "no command yet"
from the point of view of the installers.
"""
repo = self._ctx.repo
identifier = self._ctx.identifier
repo_dir = self._ctx.repo_dir
try:
cmd = resolve_command_for_repo(
repo=repo,
repo_identifier=identifier,
repo_dir=repo_dir,
)
except SystemExit:
cmd = None
if not cmd:
return CommandState(command=None, layer=None)
layer = classify_command_layer(cmd, repo_dir)
return CommandState(command=cmd, layer=layer)
class InstallationPipeline:
"""
High-level orchestrator that applies a sequence of installers
to a repository based on CLI layer precedence.
"""
def __init__(self, installers: Sequence[BaseInstaller]) -> None:
self._installers = list(installers)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def run(self, ctx: RepoContext) -> None:
"""
Execute the installation pipeline for a single repository.
- Detect initial command & layer.
- Optionally create a symlink.
- Run installers in order, skipping those whose layer is weaker
than an already-loaded CLI.
- After each installer, re-resolve the command and refresh the
symlink if needed.
"""
repo = ctx.repo
repo_dir = ctx.repo_dir
identifier = ctx.identifier
repositories_base_dir = ctx.repositories_base_dir
bin_dir = ctx.bin_dir
all_repos = ctx.all_repos
quiet = ctx.quiet
preview = ctx.preview
resolver = CommandResolver(ctx)
state = resolver.resolve()
# Persist initial command (if any) and create a symlink.
if state.command:
repo["command"] = state.command
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
else:
repo.pop("command", None)
provided_capabilities: Set[str] = set()
# Main installer loop
for installer in self._installers:
layer_name = getattr(installer, "layer", None)
# Installers without a layer participate without precedence logic.
if layer_name is None:
self._run_installer(installer, ctx, identifier, repo_dir, quiet)
continue
try:
installer_layer = CliLayer(layer_name)
except ValueError:
# Unknown layer string → treat as lowest priority.
installer_layer = None
# "Previous/Current layer already loaded?"
if state.layer is not None and installer_layer is not None:
current_prio = layer_priority(state.layer)
installer_prio = layer_priority(installer_layer)
if current_prio < installer_prio:
# Current CLI comes from a higher-priority layer,
# so we skip this installer entirely.
if not quiet:
print(
f"[pkgmgr] Skipping installer "
f"{installer.__class__.__name__} for {identifier} "
f"CLI already provided by layer {state.layer.value!r}."
)
continue
if current_prio == installer_prio:
# Same layer already provides a CLI; usually there is no
# need to run another installer on top of it.
if not quiet:
print(
f"[pkgmgr] Skipping installer "
f"{installer.__class__.__name__} for {identifier} "
f"layer {installer_layer.value!r} is already loaded."
)
continue
# Check if this installer is applicable at all.
if not installer.supports(ctx):
continue
# Capabilities: if everything this installer would provide is already
# covered, we can safely skip it.
caps = installer.discover_capabilities(ctx)
if caps and caps.issubset(provided_capabilities):
if not quiet:
print(
f"Skipping installer {installer.__class__.__name__} "
f"for {identifier} capabilities {caps} already provided."
)
continue
if not quiet:
print(
f"[pkgmgr] Running installer {installer.__class__.__name__} "
f"for {identifier} in '{repo_dir}' "
f"(new capabilities: {caps or set()})..."
)
# Run the installer with error reporting.
self._run_installer(installer, ctx, identifier, repo_dir, quiet)
provided_capabilities.update(caps)
# After running an installer, re-resolve the command and layer.
new_state = resolver.resolve()
if new_state.command:
repo["command"] = new_state.command
create_ink(
repo,
repositories_base_dir,
bin_dir,
all_repos,
quiet=quiet,
preview=preview,
)
else:
repo.pop("command", None)
state = new_state
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
@staticmethod
def _run_installer(
installer: BaseInstaller,
ctx: RepoContext,
identifier: str,
repo_dir: str,
quiet: bool,
) -> None:
"""
Execute a single installer with unified error handling.
"""
try:
installer.run(ctx)
except SystemExit as exc:
exit_code = exc.code if isinstance(exc.code, int) else str(exc.code)
print(
f"[ERROR] Installer {installer.__class__.__name__} failed "
f"for repository {identifier} (dir: {repo_dir}) "
f"with exit code {exit_code}."
)
print(
"[ERROR] This usually means an underlying command failed "
"(e.g. 'make install', 'nix build', 'pip install', ...)."
)
print(
"[ERROR] Check the log above for the exact command output. "
"You can also run this repository in isolation via:\n"
f" pkgmgr install {identifier} "
"--clone-mode shallow --no-verification"
)
raise

View File

@@ -0,0 +1,26 @@
from __future__ import annotations
"""
High-level mirror actions.
Public API:
- list_mirrors
- diff_mirrors
- merge_mirrors
- setup_mirrors
"""
from .types import Repository, MirrorMap
from .list_cmd import list_mirrors
from .diff_cmd import diff_mirrors
from .merge_cmd import merge_mirrors
from .setup_cmd import setup_mirrors
__all__ = [
"Repository",
"MirrorMap",
"list_mirrors",
"diff_mirrors",
"merge_mirrors",
"setup_mirrors",
]

View File

@@ -0,0 +1,31 @@
from __future__ import annotations
from typing import List
from pkgmgr.core.repository.dir import get_repo_dir
from pkgmgr.core.repository.identifier import get_repo_identifier
from .io import load_config_mirrors, read_mirrors_file
from .types import MirrorMap, RepoMirrorContext, Repository
def build_context(
repo: Repository,
repositories_base_dir: str,
all_repos: List[Repository],
) -> RepoMirrorContext:
"""
Build a RepoMirrorContext for a single repository.
"""
identifier = get_repo_identifier(repo, all_repos)
repo_dir = get_repo_dir(repositories_base_dir, repo)
config_mirrors: MirrorMap = load_config_mirrors(repo)
file_mirrors: MirrorMap = read_mirrors_file(repo_dir)
return RepoMirrorContext(
identifier=identifier,
repo_dir=repo_dir,
config_mirrors=config_mirrors,
file_mirrors=file_mirrors,
)

View File

@@ -0,0 +1,60 @@
from __future__ import annotations
from typing import List
from .context import build_context
from .printing import print_header
from .types import Repository
def diff_mirrors(
selected_repos: List[Repository],
repositories_base_dir: str,
all_repos: List[Repository],
) -> None:
"""
Show differences between config mirrors and MIRRORS file.
- Mirrors present only in config are reported as "ONLY IN CONFIG".
- Mirrors present only in MIRRORS file are reported as "ONLY IN FILE".
- Mirrors with same name but different URLs are reported as "URL MISMATCH".
"""
for repo in selected_repos:
ctx = build_context(repo, repositories_base_dir, all_repos)
print_header("[MIRROR DIFF]", ctx)
config_m = ctx.config_mirrors
file_m = ctx.file_mirrors
if not config_m and not file_m:
print(" No mirrors configured in config or MIRRORS file.")
print()
continue
# Mirrors only in config
for name, url in sorted(config_m.items()):
if name not in file_m:
print(f" [ONLY IN CONFIG] {name}: {url}")
# Mirrors only in MIRRORS file
for name, url in sorted(file_m.items()):
if name not in config_m:
print(f" [ONLY IN FILE] {name}: {url}")
# Mirrors with same name but different URLs
shared = set(config_m) & set(file_m)
for name in sorted(shared):
url_cfg = config_m.get(name)
url_file = file_m.get(name)
if url_cfg != url_file:
print(
f" [URL MISMATCH] {name}:\n"
f" config: {url_cfg}\n"
f" file: {url_file}"
)
if config_m and file_m and config_m == file_m:
print(" [OK] Mirrors in config and MIRRORS file are in sync.")
print()

View File

@@ -0,0 +1,179 @@
from __future__ import annotations
import os
from typing import List, Optional, Set
from pkgmgr.core.command.run import run_command
from pkgmgr.core.git import GitError, run_git
from .types import MirrorMap, RepoMirrorContext, Repository
def build_default_ssh_url(repo: Repository) -> Optional[str]:
"""
Build a simple SSH URL from repo config if no explicit mirror is defined.
Example: git@github.com:account/repository.git
"""
provider = repo.get("provider")
account = repo.get("account")
name = repo.get("repository")
port = repo.get("port")
if not provider or not account or not name:
return None
provider = str(provider)
account = str(account)
name = str(name)
if port:
return f"ssh://git@{provider}:{port}/{account}/{name}.git"
# GitHub-style shorthand
return f"git@{provider}:{account}/{name}.git"
def determine_primary_remote_url(
repo: Repository,
resolved_mirrors: MirrorMap,
) -> Optional[str]:
"""
Determine the primary remote URL in a consistent way:
1. resolved_mirrors["origin"]
2. any resolved mirror (first by name)
3. default SSH URL from provider/account/repository
"""
if "origin" in resolved_mirrors:
return resolved_mirrors["origin"]
if resolved_mirrors:
first_name = sorted(resolved_mirrors.keys())[0]
return resolved_mirrors[first_name]
return build_default_ssh_url(repo)
def _safe_git_output(args: List[str], cwd: str) -> Optional[str]:
"""
Run a Git command via run_git and return its stdout, or None on failure.
"""
try:
return run_git(args, cwd=cwd)
except GitError:
return None
def current_origin_url(repo_dir: str) -> Optional[str]:
"""
Return the current URL for remote 'origin', or None if not present.
"""
output = _safe_git_output(["remote", "get-url", "origin"], cwd=repo_dir)
if not output:
return None
url = output.strip()
return url or None
def has_origin_remote(repo_dir: str) -> bool:
"""
Check whether a remote called 'origin' exists in the repository.
"""
output = _safe_git_output(["remote"], cwd=repo_dir)
if not output:
return False
names = output.split()
return "origin" in names
def _ensure_push_urls_for_origin(
repo_dir: str,
mirrors: MirrorMap,
preview: bool,
) -> None:
"""
Ensure that all mirror URLs are present as push URLs on 'origin'.
"""
desired: Set[str] = {url for url in mirrors.values() if url}
if not desired:
return
existing_output = _safe_git_output(
["remote", "get-url", "--push", "--all", "origin"],
cwd=repo_dir,
)
existing = set(existing_output.splitlines()) if existing_output else set()
missing = sorted(desired - existing)
for url in missing:
cmd = f"git remote set-url --add --push origin {url}"
if preview:
print(f"[PREVIEW] Would run in {repo_dir!r}: {cmd}")
else:
print(f"[INFO] Adding push URL to 'origin': {url}")
run_command(cmd, cwd=repo_dir, preview=False)
def ensure_origin_remote(
repo: Repository,
ctx: RepoMirrorContext,
preview: bool,
) -> None:
"""
Ensure that a usable 'origin' remote exists and has all push URLs.
"""
repo_dir = ctx.repo_dir
resolved_mirrors = ctx.resolved_mirrors
if not os.path.isdir(os.path.join(repo_dir, ".git")):
print(f"[WARN] {repo_dir} is not a Git repository (no .git directory).")
return
url = determine_primary_remote_url(repo, resolved_mirrors)
if not has_origin_remote(repo_dir):
if not url:
print(
"[WARN] Could not determine URL for 'origin' remote. "
"Please configure mirrors or provider/account/repository."
)
return
cmd = f"git remote add origin {url}"
if preview:
print(f"[PREVIEW] Would run in {repo_dir!r}: {cmd}")
else:
print(f"[INFO] Adding 'origin' remote in {repo_dir}: {url}")
run_command(cmd, cwd=repo_dir, preview=False)
else:
current = current_origin_url(repo_dir)
if current == url or not url:
print(
f"[INFO] 'origin' already points to "
f"{current or '<unknown>'} (no change needed)."
)
else:
# We do not auto-change origin here, only log the mismatch.
print(
"[INFO] 'origin' exists with URL "
f"{current or '<unknown>'}; not changing to {url}."
)
# Ensure all mirrors are present as push URLs
_ensure_push_urls_for_origin(repo_dir, resolved_mirrors, preview)
def is_remote_reachable(url: str, cwd: Optional[str] = None) -> bool:
"""
Check whether a remote repository is reachable via `git ls-remote`.
This does NOT modify anything; it only probes the remote.
"""
workdir = cwd or os.getcwd()
try:
# --exit-code → non-zero exit code if the remote does not exist
run_git(["ls-remote", "--exit-code", url], cwd=workdir)
return True
except GitError:
return False

View File

@@ -0,0 +1,98 @@
from __future__ import annotations
import os
from urllib.parse import urlparse
from typing import List, Mapping
from .types import MirrorMap, Repository
def load_config_mirrors(repo: Repository) -> MirrorMap:
mirrors = repo.get("mirrors") or {}
result: MirrorMap = {}
if isinstance(mirrors, dict):
for name, url in mirrors.items():
if url:
result[str(name)] = str(url)
return result
if isinstance(mirrors, list):
for entry in mirrors:
if isinstance(entry, dict):
name = entry.get("name")
url = entry.get("url")
if name and url:
result[str(name)] = str(url)
return result
def read_mirrors_file(repo_dir: str, filename: str = "MIRRORS") -> MirrorMap:
"""
Supports:
NAME URL
URL → auto name = hostname
"""
path = os.path.join(repo_dir, filename)
mirrors: MirrorMap = {}
if not os.path.exists(path):
return mirrors
try:
with open(path, "r", encoding="utf-8") as fh:
for line in fh:
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
parts = stripped.split(None, 1)
# Case 1: "name url"
if len(parts) == 2:
name, url = parts
# Case 2: "url" → auto-generate name
elif len(parts) == 1:
url = parts[0]
parsed = urlparse(url)
host = (parsed.netloc or "").split(":")[0]
base = host or "mirror"
name = base
i = 2
while name in mirrors:
name = f"{base}{i}"
i += 1
else:
continue
mirrors[name] = url
except OSError as exc:
print(f"[WARN] Could not read MIRRORS file at {path}: {exc}")
return mirrors
def write_mirrors_file(
repo_dir: str,
mirrors: Mapping[str, str],
filename: str = "MIRRORS",
preview: bool = False,
) -> None:
path = os.path.join(repo_dir, filename)
lines = [f"{name} {url}" for name, url in sorted(mirrors.items())]
content = "\n".join(lines) + ("\n" if lines else "")
if preview:
print(f"[PREVIEW] Would write MIRRORS file at {path}:")
print(content or "(empty)")
return
try:
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as fh:
fh.write(content)
print(f"[INFO] Wrote MIRRORS file at {path}")
except OSError as exc:
print(f"[ERROR] Failed to write MIRRORS file at {path}: {exc}")

View File

@@ -0,0 +1,46 @@
from __future__ import annotations
from typing import List
from .context import build_context
from .printing import print_header, print_named_mirrors
from .types import Repository
def list_mirrors(
selected_repos: List[Repository],
repositories_base_dir: str,
all_repos: List[Repository],
source: str = "all",
) -> None:
"""
List mirrors for the selected repositories.
source:
- "config" → only mirrors from configuration
- "file" → only mirrors from MIRRORS file
- "resolved" → merged view (config + file, file wins)
- "all" → show config + file + resolved
"""
for repo in selected_repos:
ctx = build_context(repo, repositories_base_dir, all_repos)
resolved_m = ctx.resolved_mirrors
print_header("[MIRROR]", ctx)
if source in ("config", "all"):
print_named_mirrors("config mirrors", ctx.config_mirrors)
if source == "config":
print()
continue # next repo
if source in ("file", "all"):
print_named_mirrors("MIRRORS file", ctx.file_mirrors)
if source == "file":
print()
continue # next repo
if source in ("resolved", "all"):
print_named_mirrors("resolved mirrors", resolved_m)
print()

View File

@@ -0,0 +1,162 @@
from __future__ import annotations
import os
from typing import Dict, List, Tuple, Optional
import yaml
from pkgmgr.core.config.save import save_user_config
from .context import build_context
from .io import write_mirrors_file
from .types import MirrorMap, Repository
# -----------------------------------------------------------------------------
# Helpers
# -----------------------------------------------------------------------------
def _repo_key(repo: Repository) -> Tuple[str, str, str]:
"""
Normalised key for identifying a repository in config files.
"""
return (
str(repo.get("provider", "")),
str(repo.get("account", "")),
str(repo.get("repository", "")),
)
def _load_user_config(path: str) -> Dict[str, object]:
"""
Load a user config YAML file as dict.
Non-dicts yield {}.
"""
if not os.path.exists(path):
return {}
try:
with open(path, "r", encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
return data if isinstance(data, dict) else {}
except Exception:
return {}
# -----------------------------------------------------------------------------
# Main merge command
# -----------------------------------------------------------------------------
def merge_mirrors(
selected_repos: List[Repository],
repositories_base_dir: str,
all_repos: List[Repository],
source: str,
target: str,
preview: bool = False,
user_config_path: Optional[str] = None,
) -> None:
"""
Merge mirrors between config and MIRRORS file.
Rules:
- source, target ∈ {"config", "file"}.
- merged = (target_mirrors overridden by source_mirrors)
- If target == "file" → write MIRRORS file.
- If target == "config":
* update the user config YAML directly
* write it using save_user_config()
The merge strategy is:
dst + src (src wins on same name)
"""
# Load user config once if we intend to write to it.
user_cfg: Optional[Dict[str, object]] = None
user_cfg_path_expanded: Optional[str] = None
if target == "config" and user_config_path and not preview:
user_cfg_path_expanded = os.path.expanduser(user_config_path)
user_cfg = _load_user_config(user_cfg_path_expanded)
if not isinstance(user_cfg.get("repositories"), list):
user_cfg["repositories"] = []
for repo in selected_repos:
ctx = build_context(repo, repositories_base_dir, all_repos)
print("============================================================")
print(f"[MIRROR MERGE] Repository: {ctx.identifier}")
print(f"[MIRROR MERGE] Directory: {ctx.repo_dir}")
print(f"[MIRROR MERGE] {source}{target}")
print("============================================================")
# Pick the correct source/target maps
if source == "config":
src = ctx.config_mirrors
dst = ctx.file_mirrors
else: # source == "file"
src = ctx.file_mirrors
dst = ctx.config_mirrors
# Merge (src overrides dst)
merged: MirrorMap = dict(dst)
merged.update(src)
# ---------------------------------------------------------
# WRITE TO FILE
# ---------------------------------------------------------
if target == "file":
write_mirrors_file(ctx.repo_dir, merged, preview=preview)
print()
continue
# ---------------------------------------------------------
# WRITE TO CONFIG
# ---------------------------------------------------------
if target == "config":
# If preview or no config path → show intended output
if preview or not user_cfg:
print("[INFO] The following mirrors would be written to config:")
if not merged:
print(" (no mirrors)")
else:
for name, url in sorted(merged.items()):
print(f" - {name}: {url}")
print(" (Config not modified due to preview or missing path.)")
print()
continue
repos = user_cfg.get("repositories")
target_key = _repo_key(repo)
existing_repo: Optional[Repository] = None
# Find existing repo entry
for entry in repos:
if isinstance(entry, dict) and _repo_key(entry) == target_key:
existing_repo = entry
break
# Create entry if missing
if existing_repo is None:
existing_repo = {
"provider": repo.get("provider"),
"account": repo.get("account"),
"repository": repo.get("repository"),
}
repos.append(existing_repo)
# Write or delete mirrors
if merged:
existing_repo["mirrors"] = dict(merged)
else:
existing_repo.pop("mirrors", None)
print(" [OK] Updated repo['mirrors'] in user config.")
print()
# -------------------------------------------------------------
# SAVE CONFIG (once at the end)
# -------------------------------------------------------------
if user_cfg is not None and user_cfg_path_expanded is not None and not preview:
save_user_config(user_cfg, user_cfg_path_expanded)
print(f"[OK] Saved updated config: {user_cfg_path_expanded}")

View File

@@ -0,0 +1,35 @@
from __future__ import annotations
from .types import MirrorMap, RepoMirrorContext
def print_header(
title_prefix: str,
ctx: RepoMirrorContext,
) -> None:
"""
Print a standard header for mirror-related output.
title_prefix examples:
- "[MIRROR]"
- "[MIRROR DIFF]"
- "[MIRROR MERGE]"
- "[MIRROR SETUP:LOCAL]"
- "[MIRROR SETUP:REMOTE]"
"""
print("============================================================")
print(f"{title_prefix} Repository: {ctx.identifier}")
print(f"{title_prefix} Directory: {ctx.repo_dir}")
print("============================================================")
def print_named_mirrors(label: str, mirrors: MirrorMap) -> None:
"""
Print a labeled mirror block (e.g. '[config mirrors]').
"""
print(f" [{label}]")
if mirrors:
for name, url in sorted(mirrors.items()):
print(f" - {name}: {url}")
else:
print(" (none)")

View File

@@ -0,0 +1,165 @@
from __future__ import annotations
from typing import List, Tuple
from pkgmgr.core.git import run_git, GitError
from .context import build_context
from .git_remote import determine_primary_remote_url, ensure_origin_remote
from .types import Repository
def _setup_local_mirrors_for_repo(
repo: Repository,
repositories_base_dir: str,
all_repos: List[Repository],
preview: bool,
) -> None:
"""
Ensure local Git state is sane (currently: 'origin' remote).
"""
ctx = build_context(repo, repositories_base_dir, all_repos)
print("------------------------------------------------------------")
print(f"[MIRROR SETUP:LOCAL] {ctx.identifier}")
print(f"[MIRROR SETUP:LOCAL] dir: {ctx.repo_dir}")
print("------------------------------------------------------------")
ensure_origin_remote(repo, ctx, preview=preview)
print()
def _probe_mirror(url: str, repo_dir: str) -> Tuple[bool, str]:
"""
Probe a remote mirror by running `git ls-remote <url>`.
Returns:
(True, "") on success,
(False, error_message) on failure.
Wichtig:
- Wir werten ausschließlich den Exit-Code aus.
- STDERR kann Hinweise/Warnings enthalten und ist NICHT automatisch ein Fehler.
"""
try:
# Wir ignorieren stdout komplett; wichtig ist nur, dass der Befehl ohne
# GitError (also Exit-Code 0) durchläuft.
run_git(["ls-remote", url], cwd=repo_dir)
return True, ""
except GitError as exc:
return False, str(exc)
def _setup_remote_mirrors_for_repo(
repo: Repository,
repositories_base_dir: str,
all_repos: List[Repository],
preview: bool,
) -> None:
"""
Remote-side setup / validation.
Aktuell werden nur **nicht-destruktive Checks** gemacht:
- Für jeden Mirror (aus config + MIRRORS-Datei, file gewinnt):
* `git ls-remote <url>` wird ausgeführt.
* Bei Exit-Code 0 → [OK]
* Bei Fehler → [WARN] + Details aus der GitError-Exception
Es werden **keine** Provider-APIs aufgerufen und keine Repos angelegt.
"""
ctx = build_context(repo, repositories_base_dir, all_repos)
resolved_m = ctx.resolved_mirrors
print("------------------------------------------------------------")
print(f"[MIRROR SETUP:REMOTE] {ctx.identifier}")
print(f"[MIRROR SETUP:REMOTE] dir: {ctx.repo_dir}")
print("------------------------------------------------------------")
if not resolved_m:
# Optional: Fallback auf eine heuristisch bestimmte URL, falls wir
# irgendwann "automatisch anlegen" implementieren wollen.
primary_url = determine_primary_remote_url(repo, resolved_m)
if not primary_url:
print(
"[INFO] No mirrors configured (config or MIRRORS file), and no "
"primary URL could be derived from provider/account/repository."
)
print()
return
ok, error_message = _probe_mirror(primary_url, ctx.repo_dir)
if ok:
print(f"[OK] Remote mirror (primary) is reachable: {primary_url}")
else:
print("[WARN] Primary remote URL is NOT reachable:")
print(f" {primary_url}")
if error_message:
print(" Details:")
for line in error_message.splitlines():
print(f" {line}")
print()
print(
"[INFO] Remote checks are non-destructive and only use `git ls-remote` "
"to probe mirror URLs."
)
print()
return
# Normaler Fall: wir haben benannte Mirrors aus config/MIRRORS
for name, url in sorted(resolved_m.items()):
ok, error_message = _probe_mirror(url, ctx.repo_dir)
if ok:
print(f"[OK] Remote mirror '{name}' is reachable: {url}")
else:
print(f"[WARN] Remote mirror '{name}' is NOT reachable:")
print(f" {url}")
if error_message:
print(" Details:")
for line in error_message.splitlines():
print(f" {line}")
print()
print(
"[INFO] Remote checks are non-destructive and only use `git ls-remote` "
"to probe mirror URLs."
)
print()
def setup_mirrors(
selected_repos: List[Repository],
repositories_base_dir: str,
all_repos: List[Repository],
preview: bool = False,
local: bool = True,
remote: bool = True,
) -> None:
"""
Setup mirrors for the selected repositories.
local:
- Configure local Git remotes (currently: ensure 'origin' is present and
points to a reasonable URL).
remote:
- Non-destructive remote checks using `git ls-remote` for each mirror URL.
Es werden keine Repositories auf dem Provider angelegt.
"""
for repo in selected_repos:
if local:
_setup_local_mirrors_for_repo(
repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
)
if remote:
_setup_remote_mirrors_for_repo(
repo,
repositories_base_dir=repositories_base_dir,
all_repos=all_repos,
preview=preview,
)

View File

@@ -0,0 +1,32 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Dict
Repository = Dict[str, Any]
MirrorMap = Dict[str, str]
@dataclass(frozen=True)
class RepoMirrorContext:
"""
Bundle mirror-related information for a single repository.
"""
identifier: str
repo_dir: str
config_mirrors: MirrorMap
file_mirrors: MirrorMap
@property
def resolved_mirrors(self) -> MirrorMap:
"""
Combined mirrors from config and MIRRORS file.
Strategy:
- Start from config mirrors
- Overlay MIRRORS file (file wins on same name)
"""
merged: MirrorMap = dict(self.config_mirrors)
merged.update(self.file_mirrors)
return merged

Some files were not shown because too many files have changed in this diff Show More