# isda

Generic task queue server with scripts for downloading with different tools.

## Architecture

There is a central task queue server (isdad) and multiple workers that connect
to that server to complete tasks.

Tasks have a _key_ which is a string used to identify it and _data_ which is a
arbitrary JSON object. Keys are in the format `kind:id`; The kind is used by the
queue server to determine which workers are able to process the task.

When workers connect to the queue server they register which tasks kinds they
can process. After that they can accept tasks. Whenever a supported task exists,
it will be assigned to that worker. The worker will then carry out the work and
mark the task as completed. If workers disconnect (crash) while working, all
assigned tasks will be moved back to the queue.

## Installation

Install the queue server with `cargo install --path .`. The worker scripts dont
need installation.

For ArchLinux there exist PKGBUILDs for both
[workers](https://codeberg.org/metamuffin/pkgbuilds/src/branch/master/isdaw-git)
and
[the queue server](https://codeberg.org/metamuffin/pkgbuilds/src/branch/master/isdad-git);
Both also available pre-built on
[pkg.metamuffin.org](https://pkg.metamuffin.org).

## Usage

First start the queue daemon process. It will save its state to JSON files in
its working directory. As its first argument supply the global configuration
file.

```
isdad config.yaml
```

The configuration file contains two keys read be the daemon `bind_port` and
`bind_addr`. All other options are passed as-is to the workers.

Then you can start workers linked to the queue server. Most workers take the URI
to the worker websocket endpoint of the queue server as their first argument.

```
deno run -A scripts/<name>.ts ws://127.0.0.1:44794/worker_ws
```

Each workers usage and configuration is documented in the markdown file next to
the script. These are also linked below.

## Worker Scripts

- [enqueuer](./scripts/enqueue.md)
- [yt-dlp media downloader](./scripts/ytdlp_download.md)
- [yt-dlp playlist flattener](./scripts/ytdlp_flatten.md)
- [yt-dlp channel info generator](./scripts/ytdlp_channel_info.md)
- [complete tasks based on existing files](./scripts/complete_from_files.md)

## License

AGPL-3.0-only; See [COPYING](./COPYING).