Key Concepts
MediaMoth coordinates media-processing work through pipelines, jobs, service metadata, and event-driven projections. This page defines those concepts and how they relate when a pipeline runs.
Media
A media record is metadata about content, not the stored file itself. It includes a title, year, video properties such as resolution and chapter range, and custom tags. Input and output files are addressed separately with file-source URIs.
Pipelines
A pipeline is an event-sourced workflow definition. Its nodes form a directed acyclic graph: independent branches may run in parallel, while a node with predecessors waits until every predecessor finishes.
The workflow module in MediaMoth Core supports creating, updating, and deleting pipelines. A pipeline can also own reusable parameter templates.
Nodes
A node connects a stable node ID, a label, a worker service ID, and the IDs of its next nodes. The workflow module validates graph structure and checks that referenced services exist.
Node IDs matter after execution too. The job module stores node metadata on concrete jobs so search and rerun operations can relate work to the pipeline definition.
Services and schemas
A processing service registers through health events and advertises:
- a service ID and version;
- a parameter JSON Schema;
- a UI Schema;
- its health and worker availability.
The schema is generated from the service's typed Go parameters. Clients can therefore render forms for new worker versions without maintaining a separate parameter model.
Current processing services include:
- Video Service for FFmpeg and HandBrake conversion;
- Chapter Inserter for fixed and ML-detected chapter boundaries;
- VHS Decode for RF-capture decoding;
- TBC Decode for exporting time-base-corrected captures.
Jobs
A job is one execution of a pipeline for selected media. The job module converts pipeline nodes into a job sequence, resolves smart tags, snapshots the resolved tag map, assigns execution IDs, and queues the root nodes.
Workers report start, progress, completion, and failure. When a node finishes, the job module queues downstream nodes whose predecessors are all complete.
The API also supports cancellation, explicit retry/requeue, pipeline reruns, and streamed queue submission.
Events and projections
MediaMoth Core persists state changes as events. Its query APIs and search module read projections derived from those events. Kafka carries module and worker events; the Core PostgreSQL database stores event streams and projections in separate schemas, worker databases store River queues, and Elasticsearch backs job, service, pipeline, and media search.
If a projection is missing or its implementation changes, an administrative command can replay stored events. See Projection rebuilds.
How the pieces connect
- Media metadata supplies built-in and custom smart tags.
- A pipeline selects registered services and orders their nodes.
- A pipeline template may prefill node parameters.
- The Core job module resolves parameters and creates a concrete job sequence.
- Worker services validate their typed parameters and enqueue River work.
- Status events advance the sequence and update projections and search indexes.