📝 Add a self-contained README (closes #1) #3

Closed
upfixb02rkd wants to merge 1 commit from upfixb02rkd/dynapt:docs/issue-1-self-contained-readme into master

329
README.md
View file

@ -1,3 +1,330 @@
# dynapt
**Dyn**amic **APT** repository
**Dynamic APT repository.**
If you're frustrated that an app you use doesn't ship automated updates via an
APT repository, a PPA or Flatpak, the classic solution is a file server hosting
DEB files that the APT package manager can consume.
dynapt does exactly that, **but on your local machine**: it downloads DEB files
(and can wrap AppImages into DEBs), turns them into a valid local APT
repository, and serves it over HTTP so that ordinary `apt` commands work as if
the app shipped its own repository.
> Not a package manager. dynapt interfaces with APT instead. You keep using
> your usual tools — `apt`, `aptitude`, Synaptic or your desktop environment's
> app store. Compare with [deb-get](https://github.com/wimpysworld/deb-get)
> (a package manager for DEB files) and [AM](https://github.com/ivan-hc/AM)
> (a package manager for AppImage files).
## Features
- Mirror DEB files from a direct download URL or from GitHub Releases.
- Wrap AppImages into installable DEB packages (with `.desktop` file, icon and
optional `/usr/bin` launcher).
- Apply a [`jq`](https://jqlang.github.io/jq/) filter when a URL returns JSON
that points to the real package file.
- Cache downloads with ETags/GitHub release IDs so unchanged apps are not
reprocessed.
- Serve a standard APT repository (`Packages` and `Release` indices plus the
`data/` package store) over localhost HTTP.
- Optional GitHub access token to avoid GitHub API rate limits.
- Produces per-architecture DEB packages (`amd64` and `arm64`).
## Installation
Prebuilt DEBs (as produced by [`build.sh`](./build.sh)) are published on the
[releases page](https://git.kaki87.net/KaKi87/dynapt/releases).
Download and install automatically (replace the architecture if needed):
```bash
wget "https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-$(dpkg --print-architecture).deb" \
&& sudo apt install ./dynapt-$(dpkg --print-architecture).deb
```
Or grab it manually:
- [`dynapt-amd64.deb`](https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-amd64.deb) (x86_64)
- [`dynapt-arm64.deb`](https://git.kaki87.net/KaKi87/dynapt/releases/download/0.0.0-dev.9b8fa75/dynapt-arm64.deb) (aarch64)
The package installs to `/opt/dynapt` and provides the `dynapt` command in
`/usr/bin`.
## Configuration
Create `/opt/dynapt/config.json` (when running from source, this is
`./config.json` next to `main.js`):
```bash
nano /opt/dynapt/config.json
```
Start from the minimal template:
```json
{
"port": 3000,
"concurrency": null,
"apps": []
}
```
### Top-level options
| Option | Type | Description |
| --- | --- | --- |
| `port` | integer | Port the repository server listens on. Change it if `3000` is already in use. |
| `concurrency` | integer or `null` | Maximum number of apps downloaded/packaged at the same time. `null` (default) processes all apps in parallel for faster `apt update`; set `1` on low-power or low-bandwidth machines. |
| `apps` | array | List of apps to mirror/wrap, see below. |
| `githubAccessToken` | string | Optional [GitHub personal access token](https://github.com/settings/tokens) to avoid API rate limits when tracking many GitHub-hosted apps. |
The config is validated on every start with
[Joi](https://joi.dev/api/). App `id` values are optional: dynapt generates a
UUID (`crypto.randomUUID()`) and writes it back into the file automatically,
where it is used as the cache key.
### App options
Every item in `apps` supports at least `name` and exactly one download source
(`url` or `github`):
| Field | Type | Description |
| --- | --- | --- |
| `name` | string | Display name of the app. |
| `url` | string | HTTPS direct download URL to a DEB or AppImage. |
| `github` | object | GitHub Release metadata. |
| `github.repo` | string | Repository in `owner/name` format (required). |
| `github.filter` | string | Optional filename filter ([picomatch](https://github.com/micromatch/picomatch) patterns) when multiple DEB/AppImage assets are available (architecture/variant selection). |
| `github.prerelease` | boolean | When `true`, follow the latest pre-release instead of the latest stable release. |
| `appimage` | boolean or object | `true` (or `{ "bin": "<command>" }`) when the download is an AppImage that must be wrapped into a DEB. `bin` adds a launcher command in `/usr/bin`. Omit it to download a DEB directly. |
| `jq` | string | `jq` filter applied to a JSON `url` response to obtain the actual package file URL. Requires `url`. |
| `id` | string | Auto-generated UUID cache key — leave this out of hand-written configs. |
## Usage
Add dynapt as a local APT repository (adjust the port if you changed it):
```bash
echo "deb [trusted=yes] http://127.0.0.1:3000/ /" | sudo tee /etc/apt/sources.list.d/dynapt.list
```
Start the server (it listens on `localhost:<port>` and rebuilds the repository
on demand whenever a client requests `Packages` or `Release`):
```bash
nohup dynapt &
```
Make it start automatically at boot with cron:
```bash
crontab -l | { cat; echo "@reboot dynapt"; } | crontab -
```
Then use APT as normal: `sudo apt update && sudo apt install <app>`.
### CLI reference
The `dynapt` command (entry point [`main.js`](./main.js), run with
[Deno](https://deno.com/)) provides:
| Command | Description |
| --- | --- |
| `dynapt` (no command) or `dynapt serve` | Run the APT repository server. |
| `dynapt list` | Print a table of configured apps, packaged files and their installed/available versions (queries the local `apt` database). |
| `dynapt update` | Download and package all configured apps **without** touching APT — useful to debug fetch problems. |
| `dynapt clean` | Remove the cached `cache.json` and downloaded/generated `data/` files. |
| `dynapt version` (or `-v`, `-V`, `--version`) | Print the dynapt version. |
### How it works under the hood
On each repository request dynapt reuses a cached update (for 60 seconds) and,
for every app:
1. Resolves the download — direct `url`, a GitHub Release asset list (with
optional `jq` transform), honoring ETag/`if-none-match` and GitHub release
IDs so unchanged apps are skipped.
2. Downloads the file into `data/<appId>-<index>/`.
3. For AppImages, extracts the embedded `.desktop`/icon files, rewrites the
desktop entry, and builds a DEB with `dpkg-deb`.
4. Indexes the result with `dpkg-scanpackages` into a `.package` stanza and
eventually a `Packages` list.
5. Generates the APT `Release` file with `apt-ftparchive`.
## App configuration examples
### FOSS apps
Tested configurations for open-source apps are documented on the related
issues/discussions:
- [bat](https://github.com/sharkdp/bat/issues/323#issuecomment-2323572873)
- [code-server](https://github.com/coder/code-server/issues/2600#issuecomment-2323550416)
- [Etcher](https://github.com/balena-io/etcher/issues/4059#issuecomment-2323893199)
- [GitHub Desktop](https://github.com/shiftkey/desktop/issues/105#issuecomment-2323564520)
- [Helium Browser](https://github.com/imputnet/helium-linux/issues/86#issuecomment-3571248678)
- [IPFS Desktop](https://github.com/ipfs/ipfs-desktop/issues/691#issuecomment-2323571447)
- [Jitsi Meet](https://github.com/jitsi/jitsi-meet-electron/issues/576#issuecomment-2848151962)
- [Legcord](https://github.com/Legcord/Legcord/issues/985#issuecomment-3992255292)
- [LosslessCut](https://github.com/mifi/lossless-cut/discussions/1811#discussioncomment-10517191)
- [PDFSam](https://github.com/torakiki/pdfsam/issues/236#issuecomment-2323568526)
- [scrcpy](https://github.com/Genymobile/scrcpy/issues/1010#issuecomment-2848151962)
- [Zen Browser](https://github.com/zen-browser/desktop/discussions/1593#discussioncomment-15064021)
### Proprietary apps
#### Discord
```json
{
"name": "Discord",
"url": "https://discord.com/api/download?platform=linux&format=deb"
}
```
#### Free Download Manager
```json
{
"name": "Free Download Manager",
"url": "https://files2.freedownloadmanager.org/6/latest/freedownloadmanager.deb"
}
```
#### kDrive
```json
{
"name": "kDrive",
"url": "https://www.infomaniak.com/drive/latest",
"jq": ".[\"linux-amd64\"].downloadurl",
"appimage": true
}
```
Replace `amd64` with `arm64` where applicable.
#### Kiro
```json
{
"name": "Kiro",
"url": "https://prod.download.desktop.kiro.dev/stable/metadata-linux-x64-deb-stable.json",
"jq": ".releases[].updateTo.url | select(endswith(\".deb\"))"
}
```
#### LM Studio
```json
{
"name": "LM Studio",
"url": "https://lmstudio.ai/download/latest/linux/x64",
"appimage": true
}
```
## Troubleshooting
### Logs
In addition to the APT command output, check the latest entries in
`/opt/dynapt/dynapt.log` (next to the executable) for download and packaging
errors.
### Clean the cache
Remove previously downloaded and packaged files, then re-download them on the
next `apt update`:
```bash
dynapt clean
```
### Update without APT
Debug download problems without involving APT:
```bash
dynapt update
```
Update APT **only** for the dynapt repository:
```bash
sudo apt update \
-o Dir::Etc::sourcelist="sources.list.d/dynapt.list" \
-o Dir::Etc::sourceparts="-" \
-o APT::Get::List-Cleanup="0"
```
### Inspect packaged versions
```bash
dynapt list
```
## Building from source
Requirements:
- [Deno](https://deno.com/)
- Debian/Ubuntu tooling and utilities: `bash`, `coreutils`, `dpkg`,
`dpkg-dev`, `dpkg-scanpackages`, `apt`, `apt-ftparchive`, `squashfs-tools`
(`unsquashfs`), `file` (AppImage introspection and DEB packaging).
Build both `amd64` and `arm64` packages (uses `deno compile` per target):
```bash
./build.sh
```
The resulting `.deb` files are written to the `build/` directory. The compile
flags and allowed subprocesses/network imports live in
[`permissions.sh`](./permissions.sh).
### Running from a source checkout
```bash
./dynapt # run the server (same calls as the installed package)
deno run --no-prompt --allow-sys=hostname \
--allow-env --allow-run --allow-import --allow-read \
--allow-write --allow-net main.js
```
The `dynapt` wrapper sources [`permissions.sh`](./permissions.sh) for its
allow-list.
## Project layout
| Path | Purpose |
| --- | --- |
| [`main.js`](./main.js) | CLI entry point (commander): server, `list`, `update`, `clean`, `version`. |
| [`dirname.js`](./dirname.js) | Resolves the base directory (executable directory when compiled, source directory otherwise). |
| [`src/getConfig.js`](./src/getConfig.js) | Reads and Joi-validates `config.json`, assigns app UUIDs. |
| [`src/server.js`](./src/server.js) | Fastify server exposing `/Packages`, `/Release` and `data/`. |
| [`src/updatePackages.js`](./src/updatePackages.js) | Download, cache (ETag/ID), AppImage→DEB wrapping, `Packages`/`Release` generation. |
| [`src/list.js`](./src/list.js) | `dynapt list` table. |
| [`lib/createDebFromAppimage.js`](./lib/createDebFromAppimage.js) | Extracts an AppImage and builds a Debian package. |
| [`lib/extractAppimage.js`](./lib/extractAppimage.js) | Locates and extracts bundled `.desktop`/icons via `unsquashfs`. |
| [`lib/getBinArchitecture.js`](./lib/getBinArchitecture.js) | Falls back to `file` to derive architecture. |
| [`lib/buildDeb.js`](./lib/buildDeb.js) | Shells out to `dpkg-deb --build`. |
| [`lib/loxer.js`](./lib/loxer.js) | Logging setup. |
## Status and roadmap
dynapt is **experimental**. Once it reaches a stable release it will provide its
own APT self-updates and an optional step-by-step wizard instead of this manual
setup/procedure; a stable CLI/GUI is also planned. Until then it must be updated
manually. Subscribe to
[the tutorial gist](https://gist.github.com/KaKi87/f620788e9901abbfef4978eb7ad358b4)
or watch [this repository](https://git.kaki87.net/KaKi87/dynapt) for release
notifications.
Questions, bug reports and feature requests are welcome as
[repository issues](https://git.kaki87.net/KaKi87/dynapt/issues).
---
Thank you for using dynapt. :heart: