SwarmScribe

A backlog of recordings, transcribed by every machine you can spare.

SwarmScribe shares one big transcription job across many computers. A leader keeps the list of recordings and hands them out. Followers, on servers or ordinary PCs, each take a recording, transcribe it and send the text back.

In development. Not available yet.

Archive queue 4 followers connected
  1. Town hall, 12 March 2019 1:42:10 Done
  2. Oral history interview 0417 58:03 Done
  3. Lecture series, week 9 1:15:44 gpu-02
  4. Board meeting, audio only 2:03:29 office-pc-4
  5. Field recording 2231 37:12 Queued
  6. Panel discussion, not cleared 1:08:55 Never sent
An illustration of the queue, not a screenshot.

How it works

  1. You mark what may be transcribed

    Only recordings you have explicitly cleared become jobs. Everything else stays where it is, and no follower can ask for it.

  2. Machines join with one command

    A follower needs Docker or Kubernetes and a join token. It uses a GPU if it finds one and falls back to the CPU if not.

  3. Each follower takes one recording at a time

    Followers call the leader and never accept incoming connections, so a PC behind a home router works as well as a server in a rack.

  4. Results land in your storage

    The leader stores every finished transcript in Azure Blob Storage, Google Cloud Storage or a local disk, whichever you already use.

What you get for every recording

Plain text

The full transcript, ready to read, search or index.

Thank you all for coming tonight.
Before we start, a few notes
on the agenda.

Subtitles

A standard SRT file that drops straight into a video player or editor.

1
00:00:04,120 --> 00:00:06,480
Thank you all for coming tonight.

Word-level timing

Start and end times for every word, with the settings that produced them.

{ "word": "Thank",
  "start": 4.12, "end": 4.38,
  "probability": 0.98 }

Built to be left running

Nothing gets stranded
If a follower dies mid-job, its recording goes back in the queue. Every recording ends up either finished or parked with a reason.
Followers hold no keys
A follower never sees your storage credentials. It gets a short-lived link to one file, and nothing more.
The leader can be restarted any time
All state lives in Postgres, so any copy of the leader can stop without losing work.
Your hardware, your audio
Transcription runs on open Whisper models on machines you choose. No recording is sent to a transcription service.
You can see what is happening
Queue depth, each follower's throughput and every failure are visible without reading logs.

Where it stands

SwarmScribe is being built now. The design is settled and the first pieces are under way. There is nothing to download yet, and no date to announce.

Not in the first release

  • Languages other than English
  • Telling speakers apart
  • Translation
  • Live transcription
  • A web dashboard
  • An installer that does not need Docker