How to bulk transcribe a YouTube playlist or channel

Transcribing one video is a solved problem — paste a URL into almost anything and you get text back. Transcribing all 200 videos on a channel, or an entire playlist, breaks in ways a single-video workflow never surfaces: you get rate limited or IP-blocked partway through, a chunk of videos silently come back empty because they have no captions, long videos time out or get truncated, and if you re-run the job you pay to re-fetch videos you already have.

This is about doing that properly — as a developer building a dataset, repurposing content, or running research over a channel's back catalog — without either getting blocked or ending up with a dataset that's quietly missing a slice of the videos.

Why this breaks at scale when it doesn't for one video

Rate limits and IP blocking. YouTube (and most transcript sources) rate limit by IP, and datacenter IPs get treated more suspiciously than residential ones. A loop that hammers requests with no delay or backoff will get throttled or blocked well before it finishes a 200-video channel.

Videos with no captions. Not every video has a caption track. The uploader may have turned captions off, or YouTube never auto-generated them — common for music, very short clips, and less common languages. A tool that only reads existing caption tracks doesn't error loudly on these; it just returns nothing, or skips them, and your dataset ends up quietly incomplete unless you go check.

Long videos. A two-hour podcast or lecture needs different handling than a five-minute clip — timeouts, chunking, or just a longer wait, depending on what's doing the work.

Paying for the same video twice. If a batch job fails halfway through and you re-run it from the top, a naive loop re-fetches (and re-pays for) videos you already have, instead of skipping ahead.

Step 1: enumerate the videos

Before fetching anything, get the list of video IDs. yt-dlp with --flat-playlist does this without downloading any video or audio — it just lists what's there:

# A playlist
yt-dlp --flat-playlist --print "%(id)s" \
  "https://www.youtube.com/playlist?list=PLxxxxxxxx" > ids.txt

# A channel's uploads
yt-dlp --flat-playlist --print "%(id)s" \
  "https://www.youtube.com/@channelname/videos" > ids.txt

--flat-playlist skips resolving each video individually, so this is fast even for a channel with thousands of uploads. ids.txt is now one video ID per line — the input for everything that follows.

Step 2: fetch transcripts, honestly

The DIY approach

If you're reading captions directly (for example with youtube-transcript-api), a bulk loop needs three things beyond the single-video version: concurrency (bounded — not unlimited), retry with backoff on transient failures, and a cache so a re-run skips videos you already fetched.

import json
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path

from youtube_transcript_api import YouTubeTranscriptApi
from youtube_transcript_api._errors import TranscriptsDisabled, NoTranscriptFound

OUT_DIR = Path("transcripts")
OUT_DIR.mkdir(exist_ok=True)
api = YouTubeTranscriptApi()


def fetch(video_id: str, retries: int = 3) -> None:
    out_path = OUT_DIR / f"{video_id}.json"
    if out_path.exists():
        return  # already fetched — this is the cache

    for attempt in range(retries):
        try:
            transcript = api.fetch(video_id)
            out_path.write_text(json.dumps(transcript.to_raw_data()))
            return
        except (TranscriptsDisabled, NoTranscriptFound):
            # No caption track exists. This is not a transient error —
            # retrying won't help, and a caption-only pipeline has no
            # fallback here. Log it and move on.
            (OUT_DIR / f"{video_id}.no_captions").touch()
            return
        except Exception:
            time.sleep(2 ** attempt)  # backoff, then retry
    (OUT_DIR / f"{video_id}.failed").touch()


ids = Path("ids.txt").read_text().splitlines()
with ThreadPoolExecutor(max_workers=4) as pool:  # bounded concurrency
    futures = [pool.submit(fetch, vid) for vid in ids]
    for f in as_completed(futures):
        f.result()

This is genuinely workable, and worth knowing even if you don't run it long-term. The .no_captions marker files matter more than they look — they're how you'd notice, later, that your dataset is missing videos rather than assuming the loop covered everything.

Why the no-caption slice matters more at bulk

For one video, a caption-only tool failing is obvious — you get an error and you deal with it. At 200 videos, that same failure is silent: the loop keeps going, writes 160 transcripts, and skips 40 without raising anything you'd notice unless you count files afterward. On a real channel, captions-disabled and never-auto-captioned videos are a meaningful slice, not an edge case — so a caption-only pipeline run at bulk produces a dataset with a systematic gap, not a random one.

Using the transcript.land API over a list of IDs

The same enumeration step feeds a loop over the API instead. The request shape is the same one documented in the quickstart and API reference: POST /v1/transcript with a bearer token, and if the video has no captions, the response is a 202 with a job_id you poll instead of the transcript itself — the same call handles both cases, so the loop doesn't need to branch on whether a given video has captions.

#!/usr/bin/env bash
set -euo pipefail

while read -r id; do
  out="transcripts/${id}.json"
  [ -f "$out" ] && continue  # cache: skip videos already fetched

  resp=$(curl -sS -X POST https://api.transcript.land/v1/transcript \
    -H "Authorization: Bearer $TRANSCRIPT_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"url\": \"https://youtu.be/${id}\", \"format\": \"json\"}")

  job_id=$(echo "$resp" | jq -r '.job_id // empty')
  if [ -n "$job_id" ]; then
    # No captions — an ASR job was queued. Poll until it's done.
    while true; do
      sleep 3
      job=$(curl -sS "https://api.transcript.land/v1/jobs/${job_id}" \
        -H "Authorization: Bearer $TRANSCRIPT_API_KEY")
      status=$(echo "$job" | jq -r '.status')
      [ "$status" = "done" ] && { echo "$job" > "$out"; break; }
      [ "$status" = "error" ] && { echo "job failed for $id" >&2; break; }
    done
  else
    echo "$resp" > "$out"
  fi

  sleep 0.5  # keep a modest pace between requests
done < ids.txt

Or the same idea with the CLI, which does the job polling for you:

while read -r id; do
  out="transcripts/${id}.json"
  [ -f "$out" ] && continue
  transcript get "https://youtu.be/${id}" -o json --quiet > "$out"
  sleep 0.5
done < ids.txt

Rate limits scale with your plan — check current limits in dashboard settings — and both the API and CLI apply automatic retry with backoff if you're briefly throttled while polling a job. Fetching a video you've already fetched is served from cache rather than re-run, so re-executing a loop that skips existing output files (as above) doesn't double up on either time or cost.

Formats, for dataset builders

Pick json if you're building a corpus — it's the only format that keeps per-segment timestamps and is trivial to reassemble into whatever schema your pipeline expects.

A note on scraping responsibly

None of this is a reason to hammer YouTube or any API at maximum rate. Space out requests, respect the rate limits your plan gives you, and back off on 429s instead of retrying immediately in a hot loop. A slower job that finishes is better than a fast one that gets your IP or API key throttled halfway through.

Summary

• Enumerate videos first with yt-dlp --flat-playlist, before fetching anything.
• A DIY, caption-only loop needs bounded concurrency, retry with backoff, and a cache — and it will silently skip any video with no caption track, which at bulk is a real, non-trivial slice of a channel, not an edge case.
• Looping the transcript.land API or CLI over the same ID list handles both the caption and no-caption case with the same call, polls the job_id for you (or the CLI does), and skips videos you've already fetched — but it isn't the only tool that transcribes caption-less video this way; several services do AI transcription as a fallback. What matters for a bulk job either way is not treating "no captions" as a special case you have to detect yourself.