aboutsummaryrefslogtreecommitdiff

Enqueue

Inserts tasks into the queue with some time interval.

Configuration

  • enqueue
  • list_file: Path to the task list. The format is described below. (optional string)
  • retry_failed: Move failed completed tasks back to the queue instead of adding from a list file.
  • kind: Task kind to insert.
  • interval: Number of seconds to wait in between inserts. (number)
  • filter: Only inserts tasks that have this flag attached. The flag will be removed. This filters for fail reason if retry_failed is used. (optional string)
  • default_flags: List of flags attached to every task except where manually removed. (optional list of string)
  • data: Additional attributes that will be attached to the tasks data.
enqueue:
    - list_file: /home/user/silly_tasks.ini
      kind: webpage-download
      interval: 86400
      filter: "important"
      default_flags: ["novideos"]
      data:
          priority: 100
          output: /home/user/somedir

    - retry_failed: true
      kind: webpage-download
      interval: 86400
      filter: "some_reason"

Task list file format

A file that contains one task per line. Empty lines are ignored. The INI syntax-highlighted works well with this format.

Each line follows the format OUTPUT=ID;FLAGS where

  • ID forms the task key together with the task kind from the config
  • OUTPUT is attached as-is to the task data as output. If output is already specified in data then OUTPUT is path-joined to that.
  • FLAGS is a space seperated list of flags that will be added to the task data's flags attribute together with all default flags. Including -FLAGNAME here, prevents the default flag FLAGNAME from being added.

Sample file:

example=https://example.org;important noimages
sometask=https://example.com;noimages -novideos