diff options
| -rw-r--r-- | .gitignore | 2 | ||||
| -rw-r--r-- | readme.md | 15 | ||||
| -rw-r--r-- | scripts/ytdlp_channel_info.md | 16 | ||||
| -rw-r--r-- | scripts/ytdlp_download.md | 30 | ||||
| -rw-r--r-- | scripts/ytdlp_flatten.md | 22 |
5 files changed, 84 insertions, 1 deletions
@@ -1,3 +1,3 @@ /target /*.json -/*.yaml
\ No newline at end of file +/*.yaml @@ -2,6 +2,21 @@ Generic task queue server with scripts for downloading with different tools. +## Architecture + +There is a central task queue server (isdad) and multiple workers that connect +to that server to complete tasks. + +Tasks have a _key_ which is a string used to identify it and _data_ which is a +arbitrary JSON object. Keys are in the format `kind:id`; The kind is used by the +queue server to determine which workers are able to process the task. + +When workers connect to the queue server they register which tasks kinds they +can process. After that they can accept tasks. Whenever a supported task exists, +it will be assigned to that worker. The worker will then carry out the work and +mark the task as completed. If workers disconnect (crash) while working, all +assigned tasks will be moved back to the queue. + ## Usage First start the queue daemon process. It will save its state to JSON files in diff --git a/scripts/ytdlp_channel_info.md b/scripts/ytdlp_channel_info.md index e69de29..b824317 100644 --- a/scripts/ytdlp_channel_info.md +++ b/scripts/ytdlp_channel_info.md @@ -0,0 +1,16 @@ +# yt-dlp Channel Info Generator + +Saves channel metadata for yt-dlp channels or playlists. + +Currently supported task kinds are: `youtube-channel-info` and +`bilibili-channel-info`. + +It will place files in the `output` directory of the task: + +- `channel.info.json`: A yt-dlp info.json for the channel +- `backdrop.jpeg`: The channel's banner picture +- `poster.jpeg`: The channel icon + +## Configuration + +None diff --git a/scripts/ytdlp_download.md b/scripts/ytdlp_download.md index e69de29..c9f695a 100644 --- a/scripts/ytdlp_download.md +++ b/scripts/ytdlp_download.md @@ -0,0 +1,30 @@ +# yt-dlp Media Downloader + +Downloads media with yt-dlp. + +Currently supported task kinds are `youtube` and `bilibili`. + +The output directory is the tasks `output` data attribute which must be present. +Additional yt-dlp arguments are added based on the task's `profile` attribute. + +## Configuration + +- `ytdlp_download` + - `profiles`: Map from profile name to list of arguments passed to yt-dlp. + +Sample configuration: + +```yaml +ytdlp_download: + profiles: + video: + - -f + - bestvideo+bestaudio + - --embed-metadata + - --remux=mkv + audio: + - -f + - bestaudio + - --embed-metadata + - --remux=mka +``` diff --git a/scripts/ytdlp_flatten.md b/scripts/ytdlp_flatten.md index e69de29..97ffb39 100644 --- a/scripts/ytdlp_flatten.md +++ b/scripts/ytdlp_flatten.md @@ -0,0 +1,22 @@ +# yt-dlp Playlist Flattener + +Transforms a playlist-like object into the media items that make it up. + +Currently supported task kinds are `youtube-channel` and `bilibili-channel`. +Where task ids are the ids assigned by the respective platform. + +## Configuration + +- `ytdlp_flatten` + - `filters`: Map from flag to filter. Flags are taken from the `flag` + attribute in task data. Filters are passed straight to yt-dlp's + `--match-filter` flag. + +Sample configuration: + +```yaml +ytdlp_flatten: + filters: + notlive: live_status=not_live + short: duration<300 +``` |