Subtitles guide
Fix repeated lines in YouTube auto-subtitles
You downloaded auto-generated captions with yt-dlp --write-auto-subs --skip-download, and every line of the text shows up twice, mixed with <c>tags and cues that last 10 milliseconds. yt-dlp saved what YouTube sent. Auto-captions are "rolling" captions made for a video player, and you have to flatten them before they read as a transcript.
What the file actually looks like
This is the start of a real auto-caption file for a YouTube video:
00:00:01.040 --> 00:00:03.830 align:start position:0%
hi<00:00:01.439><c> my</c><00:00:01.680><c> name</c><00:00:01.839><c> is</c>...
00:00:03.830 --> 00:00:03.840 align:start position:0%
hi my name is han founder and ceo of
00:00:03.840 --> 00:00:04.789 align:start position:0%
hi my name is han founder and ceo of
gene<00:00:04.080><c> ai</c>The player shows two lines at a time. Each new cue repeats the line from before as the top line and types the new one underneath, word by word (the <00:00:01.439> tags are per-word timings). The 10 ms cues in between hold the finished line still while it scrolls up. Read it as plain text and you get every line twice.
Why converting to SRT doesn't fix it
--convert-subs srt changes the container, not the content. It strips the word tags, but every rolling cue survives. Converting the file above with ffmpeg, which is what --convert-subs uses, gives:
2
00:00:03,830 --> 00:00:03,840
hi my name is han founder and ceo of
3
00:00:03,840 --> 00:00:04,789
hi my name is han founder and ceo of
gene aiThat's fine for watching the video in a player, but in a transcript, a search index or an LLM prompt the repeats waste space and confuse things.
Flatten it: keep the new line from each cue
Download just the English auto-captions as VTT:
yt-dlp --write-auto-subs --sub-langs en --sub-format vtt --skip-download \
-o "%(id)s" "https://www.youtube.com/watch?v=VIDEO_ID"Then keep only the last line of each cue, strip the tags, and drop a line when it's the same as the one before:
import re
import sys
TAG = re.compile(r"<[^>]+>")
TIME = re.compile(r"^(\d\d:\d\d:\d\d\.\d\d\d) --> (\d\d:\d\d:\d\d\.\d\d\d)")
def clean_lines(path):
cues, start, text = [], None, []
for line in open(path, encoding="utf-8").read().splitlines() + [""]:
m = TIME.match(line)
if m:
start, text = m.group(1), []
elif line == "" and start:
cues.append((start, text))
start = None
elif start:
text.append(line)
out = []
for start, lines in cues:
# each rolling cue repeats the previous line; the last non-blank line is the new one
kept = [TAG.sub("", l).strip() for l in lines if l.strip()]
last = kept[-1] if kept else ""
if last and (not out or out[-1][1] != last):
out.append((start, last))
return out
for start, text in clean_lines(sys.argv[1]):
print(f"[{start[:8]}] {text}")Run it with python3 dedupe_vtt.py VIDEO_ID.en.vtt. On the file above it turns 467 cues into 234 lines, each once, with the time it starts:
[00:00:01] hi my name is han founder and ceo of
[00:00:03] gene ai
[00:00:04] in this episode i'm going to introduce
[00:00:07] you some basic concepts in ginaKeep the timestamps if you want to link back to a moment in the video (&t=SECONDS). Drop them and join the lines if you only need the text. Auto-captions have no punctuation or speaker labels either way: that's what YouTube generated.
Other options
- Check for real subtitles first.
yt-dlp --list-subs URLlists uploaded subtitles separately from automatic captions. Uploaded ones don't roll, so--write-subsgives clean cues when the creator added them. - Other caption formats. The same list shows the formats YouTube offers for each language (json3, srv3 and others besides vtt). Pick one with
--sub-formatif you'd rather parse YouTube's own JSON than the VTT. - No captions at all. Some videos have none, auto or uploaded. Then you need speech-to-text on the audio; see extract audio with yt-dlp.
More on languages and formats in download subtitles with yt-dlp. If caption downloads fail on a server with a bot check, that's the IP, not the flags: see YouTube transcripts that fail on a server.
For builders
Using yt-dlp inside an app or script?
Paste a link to see what a managed media-import API returns — no install, no signup.
Building this into an app? See the API your backend calls instead