diff --git a/README.md b/README.md index 5f6cd36..83902d0 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,330 @@ # dynapt -**Dyn**amic **APT** repository \ No newline at end of file +**Dynamic APT repository.** + +If you're frustrated that an app you use doesn't ship automated updates via an +APT repository, a PPA or Flatpak, the classic solution is a file server hosting +DEB files that the APT package manager can consume. + +dynapt does exactly that, **but on your local machine**: it downloads DEB files +(and can wrap AppImages into DEBs), turns them into a valid local APT +repository, and serves it over HTTP so that ordinary `apt` commands work as if +the app shipped its own repository. + +> Not a package manager. dynapt interfaces with APT instead. You keep using +> your usual tools — `apt`, `aptitude`, Synaptic or your desktop environment's +> app store. Compare with [deb-get](https://github.com/wimpysworld/deb-get) +> (a package manager for DEB files) and [AM](https://github.com/ivan-hc/AM) +> (a package manager for AppImage files). + +## Features + +- Mirror DEB files from a direct download URL or from GitHub Releases. +- Wrap AppImages into installable DEB packages (with `.desktop` file, icon and + optional `/usr/bin` launcher). +- Apply a [`jq`](https://jqlang.github.io/jq/) filter when a URL returns JSON + that points to the real package file. +- Cache downloads with ETags/GitHub release IDs so unchanged apps are not + reprocessed. +- Serve a standard APT repository (`Packages` and `Release` indices plus the + `data/` package store) over localhost HTTP. +- Optional GitHub access token to avoid GitHub API rate limits. +- Produces per-architecture DEB packages (`amd64` and `arm64`). + +## Installation + +Prebuilt DEBs (as produced by [`build.sh`](./build.sh)) are published on the +[releases page](https://git.kaki87.net/KaKi87/dynapt/releases). + +Download and install automatically (replace the architecture if needed): + +```bash +wget "https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-$(dpkg --print-architecture).deb" \ + && sudo apt install ./dynapt-$(dpkg --print-architecture).deb +``` + +Or grab it manually: + +- [`dynapt-amd64.deb`](https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-amd64.deb) (x86_64) +- [`dynapt-arm64.deb`](https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-arm64.deb) (aarch64) + +The package installs to `/opt/dynapt` and provides the `dynapt` command in +`/usr/bin`. + +## Configuration + +Create `/opt/dynapt/config.json` (when running from source, this is +`./config.json` next to `main.js`): + +```bash +nano /opt/dynapt/config.json +``` + +Start from the minimal template: + +```json +{ + "port": 3000, + "concurrency": null, + "apps": [] +} +``` + +### Top-level options + +| Option | Type | Description | +| --- | --- | --- | +| `port` | integer | Port the repository server listens on. Change it if `3000` is already in use. | +| `concurrency` | integer or `null` | Maximum number of apps downloaded/packaged at the same time. `null` (default) processes all apps in parallel for faster `apt update`; set `1` on low-power or low-bandwidth machines. | +| `apps` | array | List of apps to mirror/wrap, see below. | +| `githubAccessToken` | string | Optional [GitHub personal access token](https://github.com/settings/tokens) to avoid API rate limits when tracking many GitHub-hosted apps. | + +The config is validated on every start with +[Joi](https://joi.dev/api/). App `id` values are optional: dynapt generates a +UUID (`crypto.randomUUID()`) and writes it back into the file automatically, +where it is used as the cache key. + +### App options + +Every item in `apps` supports at least `name` and exactly one download source +(`url` or `github`): + +| Field | Type | Description | +| --- | --- | --- | +| `name` | string | Display name of the app. | +| `url` | string | HTTPS direct download URL to a DEB or AppImage. | +| `github` | object | GitHub Release metadata. | +| `github.repo` | string | Repository in `owner/name` format (required). | +| `github.filter` | string | Optional filename filter ([picomatch](https://github.com/micromatch/picomatch) patterns) when multiple DEB/AppImage assets are available (architecture/variant selection). | +| `github.prerelease` | boolean | When `true`, follow the latest pre-release instead of the latest stable release. | +| `appimage` | boolean or object | `true` (or `{ "bin": "" }`) when the download is an AppImage that must be wrapped into a DEB. `bin` adds a launcher command in `/usr/bin`. Omit it to download a DEB directly. | +| `jq` | string | `jq` filter applied to a JSON `url` response to obtain the actual package file URL. Requires `url`. | +| `id` | string | Auto-generated UUID cache key — leave this out of hand-written configs. | + +## Usage + +Add dynapt as a local APT repository (adjust the port if you changed it): + +```bash +echo "deb [trusted=yes] http://127.0.0.1:3000/ /" | sudo tee /etc/apt/sources.list.d/dynapt.list +``` + +Start the server (it listens on `localhost:` and rebuilds the repository +on demand whenever a client requests `Packages` or `Release`): + +```bash +nohup dynapt & +``` + +Make it start automatically at boot with cron: + +```bash +crontab -l | { cat; echo "@reboot dynapt"; } | crontab - +``` + +Then use APT as normal: `sudo apt update && sudo apt install `. + +### CLI reference + +The `dynapt` command (entry point [`main.js`](./main.js), run with +[Deno](https://deno.com/)) provides: + +| Command | Description | +| --- | --- | +| `dynapt` (no command) or `dynapt serve` | Run the APT repository server. | +| `dynapt list` | Print a table of configured apps, packaged files and their installed/available versions (queries the local `apt` database). | +| `dynapt update` | Download and package all configured apps **without** touching APT — useful to debug fetch problems. | +| `dynapt clean` | Remove the cached `cache.json` and downloaded/generated `data/` files. | +| `dynapt version` (or `-v`, `-V`, `--version`) | Print the dynapt version. | + +### How it works under the hood + +On each repository request dynapt reuses a cached update (for 60 seconds) and, +for every app: + +1. Resolves the download — direct `url`, a GitHub Release asset list (with + optional `jq` transform), honoring ETag/`if-none-match` and GitHub release + IDs so unchanged apps are skipped. +2. Downloads the file into `data/-/`. +3. For AppImages, extracts the embedded `.desktop`/icon files, rewrites the + desktop entry, and builds a DEB with `dpkg-deb`. +4. Indexes the result with `dpkg-scanpackages` into a `.package` stanza and + eventually a `Packages` list. +5. Generates the APT `Release` file with `apt-ftparchive`. + +## App configuration examples + +### FOSS apps + +Tested configurations for open-source apps are documented on the related +issues/discussions: + +- [bat](https://github.com/sharkdp/bat/issues/323#issuecomment-2323572873) +- [code-server](https://github.com/coder/code-server/issues/2600#issuecomment-2323550416) +- [Etcher](https://github.com/balena-io/etcher/issues/4059#issuecomment-2323893199) +- [GitHub Desktop](https://github.com/shiftkey/desktop/issues/105#issuecomment-2323564520) +- [Helium Browser](https://github.com/imputnet/helium-linux/issues/86#issuecomment-3571248678) +- [IPFS Desktop](https://github.com/ipfs/ipfs-desktop/issues/691#issuecomment-2323571447) +- [Jitsi Meet](https://github.com/jitsi/jitsi-meet-electron/issues/576#issuecomment-2848151962) +- [Legcord](https://github.com/Legcord/Legcord/issues/985#issuecomment-3992255292) +- [LosslessCut](https://github.com/mifi/lossless-cut/discussions/1811#discussioncomment-10517191) +- [PDFSam](https://github.com/torakiki/pdfsam/issues/236#issuecomment-2323568526) +- [scrcpy](https://github.com/Genymobile/scrcpy/issues/1010#issuecomment-2848151962) +- [Zen Browser](https://github.com/zen-browser/desktop/discussions/1593#discussioncomment-15064021) + +### Proprietary apps + +#### Discord + +```json +{ + "name": "Discord", + "url": "https://discord.com/api/download?platform=linux&format=deb" +} +``` + +#### Free Download Manager + +```json +{ + "name": "Free Download Manager", + "url": "https://files2.freedownloadmanager.org/6/latest/freedownloadmanager.deb" +} +``` + +#### kDrive + +```json +{ + "name": "kDrive", + "url": "https://www.infomaniak.com/drive/latest", + "jq": ".[\"linux-amd64\"].downloadurl", + "appimage": true +} +``` + +Replace `amd64` with `arm64` where applicable. + +#### Kiro + +```json +{ + "name": "Kiro", + "url": "https://prod.download.desktop.kiro.dev/stable/metadata-linux-x64-deb-stable.json", + "jq": ".releases[].updateTo.url | select(endswith(\".deb\"))" +} +``` + +#### LM Studio + +```json +{ + "name": "LM Studio", + "url": "https://lmstudio.ai/download/latest/linux/x64", + "appimage": true +} +``` + +## Troubleshooting + +### Logs + +In addition to the APT command output, check the latest entries in +`/opt/dynapt/dynapt.log` (next to the executable) for download and packaging +errors. + +### Clean the cache + +Remove previously downloaded and packaged files, then re-download them on the +next `apt update`: + +```bash +dynapt clean +``` + +### Update without APT + +Debug download problems without involving APT: + +```bash +dynapt update +``` + +Update APT **only** for the dynapt repository: + +```bash +sudo apt update \ + -o Dir::Etc::sourcelist="sources.list.d/dynapt.list" \ + -o Dir::Etc::sourceparts="-" \ + -o APT::Get::List-Cleanup="0" +``` + +### Inspect packaged versions + +```bash +dynapt list +``` + +## Building from source + +Requirements: + +- [Deno](https://deno.com/) +- Debian/Ubuntu tooling and utilities: `bash`, `coreutils`, `dpkg`, + `dpkg-dev`, `dpkg-scanpackages`, `apt`, `apt-ftparchive`, `squashfs-tools` + (`unsquashfs`), `file` (AppImage introspection and DEB packaging). + +Build both `amd64` and `arm64` packages (uses `deno compile` per target): + +```bash +./build.sh +``` + +The resulting `.deb` files are written to the `build/` directory. The compile +flags and allowed subprocesses/network imports live in +[`permissions.sh`](./permissions.sh). + +### Running from a source checkout + +```bash +./dynapt # run the server (same calls as the installed package) +deno run --no-prompt --allow-sys=hostname \ + --allow-env --allow-run --allow-import --allow-read \ + --allow-write --allow-net main.js +``` + +The `dynapt` wrapper sources [`permissions.sh`](./permissions.sh) for its +allow-list. + +## Project layout + +| Path | Purpose | +| --- | --- | +| [`main.js`](./main.js) | CLI entry point (commander): server, `list`, `update`, `clean`, `version`. | +| [`dirname.js`](./dirname.js) | Resolves the base directory (executable directory when compiled, source directory otherwise). | +| [`src/getConfig.js`](./src/getConfig.js) | Reads and Joi-validates `config.json`, assigns app UUIDs. | +| [`src/server.js`](./src/server.js) | Fastify server exposing `/Packages`, `/Release` and `data/`. | +| [`src/updatePackages.js`](./src/updatePackages.js) | Download, cache (ETag/ID), AppImage→DEB wrapping, `Packages`/`Release` generation. | +| [`src/list.js`](./src/list.js) | `dynapt list` table. | +| [`lib/createDebFromAppimage.js`](./lib/createDebFromAppimage.js) | Extracts an AppImage and builds a Debian package. | +| [`lib/extractAppimage.js`](./lib/extractAppimage.js) | Locates and extracts bundled `.desktop`/icons via `unsquashfs`. | +| [`lib/getBinArchitecture.js`](./lib/getBinArchitecture.js) | Falls back to `file` to derive architecture. | +| [`lib/buildDeb.js`](./lib/buildDeb.js) | Shells out to `dpkg-deb --build`. | +| [`lib/loxer.js`](./lib/loxer.js) | Logging setup. | + +## Status and roadmap + +dynapt is **experimental**. Once it reaches a stable release it will provide its +own APT self-updates and an optional step-by-step wizard instead of this manual +setup/procedure; a stable CLI/GUI is also planned. Until then it must be updated +manually. Subscribe to +[the tutorial gist](https://gist.github.com/KaKi87/f620788e9901abbfef4978eb7ad358b4) +or watch [this repository](https://git.kaki87.net/KaKi87/dynapt) for release +notifications. + +Questions, bug reports and feature requests are welcome as +[repository issues](https://git.kaki87.net/KaKi87/dynapt/issues). + +--- + +Thank you for using dynapt. :heart: