document the remaining workers

5 files changed, 84 insertions, 1 deletions

diff --git a/.gitignore b/.gitignore
index c26e47a..aaca451 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,3 @@
 /target
 /*.json
-/*.yaml
\ No newline at end of file
+/*.yaml
diff --git a/readme.md b/readme.md
index 28fedd0..c7d1070 100644
--- a/readme.md
+++ b/readme.md
@@ -2,6 +2,21 @@
 
 Generic task queue server with scripts for downloading with different tools.
 
+## Architecture
+
+There is a central task queue server (isdad) and multiple workers that connect
+to that server to complete tasks.
+
+Tasks have a _key_ which is a string used to identify it and _data_ which is a
+arbitrary JSON object. Keys are in the format `kind:id`; The kind is used by the
+queue server to determine which workers are able to process the task.
+
+When workers connect to the queue server they register which tasks kinds they
+can process. After that they can accept tasks. Whenever a supported task exists,
+it will be assigned to that worker. The worker will then carry out the work and
+mark the task as completed. If workers disconnect (crash) while working, all
+assigned tasks will be moved back to the queue.
+
 ## Usage
 
 First start the queue daemon process. It will save its state to JSON files in
diff --git a/scripts/ytdlp_channel_info.md b/scripts/ytdlp_channel_info.md
index e69de29..b824317 100644
--- a/scripts/ytdlp_channel_info.md
+++ b/scripts/ytdlp_channel_info.md
@@ -0,0 +1,16 @@
+# yt-dlp Channel Info Generator
+
+Saves channel metadata for yt-dlp channels or playlists.
+
+Currently supported task kinds are: `youtube-channel-info` and
+`bilibili-channel-info`.
+
+It will place files in the `output` directory of the task:
+
+- `channel.info.json`: A yt-dlp info.json for the channel
+- `backdrop.jpeg`: The channel's banner picture
+- `poster.jpeg`: The channel icon
+
+## Configuration
+
+None
diff --git a/scripts/ytdlp_download.md b/scripts/ytdlp_download.md
index e69de29..c9f695a 100644
--- a/scripts/ytdlp_download.md
+++ b/scripts/ytdlp_download.md
@@ -0,0 +1,30 @@
+# yt-dlp Media Downloader
+
+Downloads media with yt-dlp.
+
+Currently supported task kinds are `youtube` and `bilibili`.
+
+The output directory is the tasks `output` data attribute which must be present.
+Additional yt-dlp arguments are added based on the task's `profile` attribute.
+
+## Configuration
+
+- `ytdlp_download`
+  - `profiles`: Map from profile name to list of arguments passed to yt-dlp.
+
+Sample configuration:
+
+```yaml
+ytdlp_download:
+    profiles:
+        video:
+            - -f
+            - bestvideo+bestaudio
+            - --embed-metadata
+            - --remux=mkv
+        audio:
+            - -f
+            - bestaudio
+            - --embed-metadata
+            - --remux=mka
+```
diff --git a/scripts/ytdlp_flatten.md b/scripts/ytdlp_flatten.md
index e69de29..97ffb39 100644
--- a/scripts/ytdlp_flatten.md
+++ b/scripts/ytdlp_flatten.md
@@ -0,0 +1,22 @@
+# yt-dlp Playlist Flattener
+
+Transforms a playlist-like object into the media items that make it up.
+
+Currently supported task kinds are `youtube-channel` and `bilibili-channel`.
+Where task ids are the ids assigned by the respective platform.
+
+## Configuration
+
+- `ytdlp_flatten`
+  - `filters`: Map from flag to filter. Flags are taken from the `flag`
+    attribute in task data. Filters are passed straight to yt-dlp's
+    `--match-filter` flag.
+
+Sample configuration:
+
+```yaml
+ytdlp_flatten:
+    filters:
+        notlive: live_status=not_live
+        short: duration<300
+```