The Codex Reel Pipeline
Turn a folder of phone clips into a posted Reel with one prompt. Every prompt, every config, the bash script.
The Codex Reel Pipeline
Turn a folder of phone clips into a posted Reel with one prompt. The full setup, every prompt, and the bash script that runs it on any folder.
What this is
Last week I dumped 14 phone clips from a hackathon into Codex with one prompt. It gave me back a posted Reel. No timeline, no CapCut, no editing.
This doc has every prompt, every config file, and every fix for the things that broke when I built it. Run it as-is to produce a hackathon-style recap reel. Swap in your own clips for any other reel.
It works in Codex, Claude Code, or VS Code with Cline. The prompts are written to be tool-agnostic — paste them into whichever agentic CLI you use.
Time to first reel: ~45 minutes if you’ve never done this. ~10 minutes for every reel after.
Who this is for: creators making 10+ reels a month, podcasters cutting shorts from episodes, anyone tired of dragging clips onto a CapCut timeline at 11 PM. If you make one reel a week, this isn’t for you. CapCut is fine. Volume is the unlock here.
The 90-second tour
There are three tiers. Each one builds on the last.
Tier 1 — Dump and stitch. You drop a folder of phone clips. Pipeline normalizes orientation, scores each clip, picks the best ones, transcribes speech, burns captions, outputs a 60-second 9:16 reel. This is the “I just need a reel posted” tier.
Tier 2 — Filler removal and pacing. Same as tier 1 plus: cuts filler words, removes dead air over 350ms, snaps cuts to the beat of your music bed, drops in context cards when you mention specific phrases.
Tier 3 — Narrative arc. Same as tier 2 plus: structures the reel into three acts using a one-page outline, places slide overlays at act breaks, ducks music under speech, adds title and end cards.
You can stop at any tier. Tier 1 alone is already better than most CapCut work.
Setup is one-time. Pipeline is reusable. Build it once, run it on any folder forever. That’s the actual unlock.
Setup (one-time, ~15 minutes)
Tested on macOS 14+. Windows and Linux work but commands differ for the install steps.
Install the four tools
# Node 20+
brew install node
# ffmpeg (the audio/video swiss army knife)
brew install ffmpeg
# Remotion (React-for-video framework)
npm create video@latest reel-pipeline
cd reel-pipeline
# Your agentic CLI of choice — install ONE of these:
# Option A: Codex (OpenAI)
npm install -g @openai/codex
# Option B: Claude Code (Anthropic)
npm install -g @anthropic-ai/claude-code
# Option C: VS Code + Cline (extension)
# Install Cline from the VS Code marketplace
Verify everything’s there
node --version # should be v20.x or higher
ffmpeg -version # should print version info
codex --version # or: claude --version
If any of those fail, fix that before moving on. The pipeline assumes all four are working.
Drop these files into your project root
You’ll need three small config files. Working defaults below — they’ll produce a hackathon recap reel out of the box. Swap in your own values once it works once.
brand.json — your colors, font, and handle. Used for slide cards and end cards.
{
"primary": "#1a1a1a",
"accent": "#FF6B35",
"text": "#FFFFFF",
"font": "Inter",
"handle": "@yourhandle"
}
blurbs.json — context cards that pop up when specific phrases are mentioned. Format: trigger phrase, title, body.
[
{"trigger": "hackathon", "title": "Chicago AI Builders", "body": "Monthly meetup, ~50 builders"},
{"trigger": "Codex", "title": "Codex", "body": "OpenAI's agentic coding CLI"},
{"trigger": "Remotion", "title": "Remotion", "body": "React for programmatic video"},
{"trigger": "ffmpeg", "title": "ffmpeg", "body": "Open-source media toolkit"}
]
outline.md — only used in tier 3. The story arc of your reel.
# Hackathon Recap
## Act 1: Arrival
- Anchor: yesterday I went to a hackathon
- Slide title: Chicago AI Builders
- Slide body: April 2026
## Act 2: The Insight
- Anchor: what broke my brain
- Slide title: What I Saw
- Slide body: Agentic editing · One prompt, full reel
## Act 3: Takeaway
- Anchor: try it this weekend
- Slide title: Try It
- Slide body: Repo in bio · Fork and ship
Add your raw clips
Put them in ./raw/. Anything from your phone — talking heads, b-roll, event footage. The pipeline handles the variety.
If you don’t have your own clips yet, the working default for testing is: 4 talking-head clips of you saying “yesterday I went to a hackathon,” “half the creators were doing this,” “what broke my brain,” “try it this weekend” — plus 6-8 b-roll clips of anything (your room, a walk outside, your laptop screen).
That gets you to ~12-14 clips, which is what tier 1 needs.
Optional: drop a music file
Put a royalty-free track at ./music/bg.mp3. The pipeline runs without music, but tier 2 and tier 3 are noticeably better with one.
Free sources: YouTube Audio Library, Pixabay Music. Search “upbeat tech” or “uplifting corporate.” Download, rename to bg.mp3, drop in ./music/.
The pre-bake (run once, before anything else)
This is the single most important step. Skip it and you’ll spend an hour debugging rotation.
iPhone clips store rotation as metadata, not as baked pixels. ffmpeg sometimes honors this metadata, sometimes doesn’t. The agentic pipeline gets confused either way. Solution: bake rotation into pixels once, strip the metadata, never think about it again.
Save this as ./normalize-raw.sh and run it once on your raw folder.
#!/bin/bash
# normalize-raw.sh — bakes rotation into pixels, strips metadata flags.
# Run ONCE on ./raw/ before any pipeline runs.
mkdir -p ./raw_normalized
for f in ./raw/*.{mp4,mov,MP4,MOV}; do
[ -f "$f" ] || continue
filename=$(basename "$f")
# Read rotation from display matrix (most reliable on iPhone)
rotation=$(ffprobe -loglevel error -select_streams v:0 \
-show_entries stream_side_data=rotation \
-of default=nw=1:nk=1 "$f" 2>/dev/null | head -1)
# Fallback to rotate tag
if [ -z "$rotation" ]; then
rotation=$(ffprobe -loglevel error -select_streams v:0 \
-show_entries stream_tags=rotate \
-of default=nw=1:nk=1 "$f" 2>/dev/null | head -1)
fi
rotation=${rotation:-0}
# Build the right transpose filter
case "$rotation" in
90|-270) filter="transpose=1" ;;
180|-180) filter="transpose=2,transpose=2" ;;
270|-90) filter="transpose=2" ;;
*) filter="null" ;;
esac
ffmpeg -y -loglevel error -i "$f" \
-vf "$filter" \
-metadata:s:v:0 rotate=0 \
-c:v libx264 -preset fast -crf 18 \
-c:a copy \
"./raw_normalized/$filename"
echo "Normalized: $filename (rotation: $rotation)"
done
# Swap folders so the pipeline reads the clean ones
mv ./raw ./raw_original
mv ./raw_normalized ./raw
echo "Done. Original clips backed up in ./raw_original/"
Make it executable and run:
chmod +x normalize-raw.sh
./normalize-raw.sh
Verify one clip plays right-side up:
open ./raw/$(ls ./raw | head -1)
If it’s upright, you’re good. If it’s still sideways, see the Troubleshooting section.
Tier 1: Dump and stitch
The agentic pipeline. One prompt. Out comes a reel.
Open your CLI (Codex, Claude Code, or Cline) in the project directory and paste this prompt:
Build tier1.mp4 from ./raw/. The output is a postable 9:16 Reel, max 60 seconds.
INPUT: All clips in ./raw/. Orientation is already normalized — do not reapply rotation.
STEP 1 — TEMPORAL ORDERING
For every clip, extract creation timestamp using this priority:
a. ffprobe creation_time (format tag)
b. ffprobe com.apple.quicktime.creationdate (iPhone metadata)
c. exiftool DateTimeOriginal as fallback
d. filesystem mtime as last resort
Sort all clips strictly by timestamp ascending. Lock this ordering for all later steps.
Print the sorted timeline to ./out/tier1_timeline.txt.
STEP 2 — CLIP SCORING AND SELECTION
For each clip, compute a 0-100 score:
- Face detection confidence on a sampled frame (weight 0.30)
- Audio energy and speech-to-noise ratio (weight 0.30)
- Sharpness via Laplacian variance (weight 0.20)
- Motion stability via mean frame-to-frame pixel difference (weight 0.20)
Drop clips scoring below 40. Log dropped clips with their scores.
SELECTION CONSTRAINTS (HARD):
- Use AT LEAST 8 different clips in the final output
- No clip used more than once
- No clip longer than 8 seconds in the final cut
- Total runtime: 45-60 seconds
STEP 3 — TRIM AND REFRAME
For each kept clip:
- Trim 0.3s from head and tail (camera shake)
- Pick the strongest 3-6 second segment
- Talking-head clips: cleanest speech segment
- B-roll clips: highest motion + best exposure window
- Reframe horizontal clips to 9:16 with face-centered crop where a face exists,
center-crop otherwise. Use mediapipe face detection. Smooth tracking with a
0.6 EMA to avoid jitter.
- Normalize audio to -16 LUFS
STEP 4 — CAPTIONS
Transcribe every speech-bearing clip with Whisper "large-v3" (NOT base, NOT small).
Use word-level timestamps.
Verification pass:
- Re-transcribe each clip at temperature 0.0 AND 0.2
- Where the two passes disagree, flag as low-confidence
- Prefer the temp=0.0 version for low-confidence words
- Apply this domain dictionary, force-correcting near-matches:
["Codex", "Claude", "Claude Code", "Remotion", "ffmpeg", "Whisper",
"OpenAI", "Anthropic", "hackathon", "Reel", "CapCut", "9:16"]
Save the final transcript to ./out/tier1_transcript.txt before rendering captions.
Caption rendering:
- Sans-serif (Inter), 56px, white, 4px black stroke
- Position: y=1720 (lower-third, above mobile UI safe zone)
- Per-word highlight on the active word, max 6 words on screen
- Only render where Whisper confidence > 0.7
STEP 5 — MANIFEST BEFORE RENDER
Write ./out/tier1_manifest.txt containing:
- Sorted timeline with timestamps
- Score per clip and kept/dropped status
- Final edit decision list: clip | start | end | duration
- Verified transcript
- Total runtime and clip count
Render only after manifest is written.
STEP 6 — RENDER AND VERIFY
Render to ./out/tier1.mp4 at 1080x1920, 30fps.
After render, run ffprobe and confirm:
- Duration in 45-60s range
- Resolution exactly 1080x1920
- No rotation flags
- Audio peak below -1 dBFS
If any check fails, fix and re-render. Do not surface failures — fix and re-render.
Do all of this in one run. No interactive checkpoints.
Run it. Walk away for 5-10 minutes. Come back to ./out/tier1.mp4.
Open the manifest before you watch the video:
cat ./out/tier1_manifest.txt
If the manifest looks right (8+ distinct clips, total ~60s, no clip used twice), the render will be right. If the manifest looks wrong, fix the prompt and re-run before wasting a render cycle.
That’s tier 1. Most people stop here. Honestly, this is already a postable reel for most use cases. If you’re building this for the first time, post tier 1 to validate the setup works end-to-end, then come back for tier 2.
Tier 2: Filler cuts, beat-snapping, context cards
Tier 1 with the polish. Cuts filler words. Removes dead air. Snaps to music. Pops in context cards.
Build tier2.mp4 from ./raw/. Tier 2 extends tier 1 with filler removal,
beat-snapping, and context blurbs.
START FROM the tier 1 pipeline (timestamp ordering, scoring, trimming, reframing,
captions). All those rules apply. Then add:
STEP 7 — FILLER AND DEAD AIR
For talking-head clips:
- Remove these filler words: um, uh, like (filler usage only, not as comparison),
you know, kind of, sort of, I mean, basically
- Cut silences > 350ms via ffmpeg silencedetect
- Re-align word-level caption timestamps to the cut timeline
STEP 8 — PACING
Check if ./music/bg.mp3 exists.
IF EXISTS:
- Detect BPM via librosa or aubio
- Snap b-roll cut points to nearest beat (200ms tolerance)
- Spine clips (with speech) cut on natural sentence boundaries — speech rhythm
beats music rhythm
- Mix bg.mp3 under the comp:
- Duck to -22 LUFS under speech
- Restore to -16 LUFS in speech gaps
- 250ms ramp on each duck transition
IF MISSING:
- Skip music entirely
- Snap b-roll to half-second grid for consistency
- Log "no music bed found, using grid-based cutting"
Do not fail the run if music is missing.
STEP 9 — CONTEXT BLURBS
Read ./blurbs.json. For each trigger phrase:
- Find every occurrence in the verified transcript
- Render a chip-style card in the upper-third
- Hold for 2.5s after the trigger word ends
- Fade in 200ms, fade out 200ms
- Use brand.json colors and font
- Cap at 3 blurb appearances total (take the first 3 if more triggers fire)
STEP 10 — UPDATED MANIFEST
Update the manifest to include:
- Filler removal log (which words cut, where)
- Silence cuts (timestamps and durations)
- BPM detected and beats snapped to
- Blurb fire log: trigger word | timestamp | which blurb
Write to ./out/tier2_manifest.txt.
STEP 11 — RENDER
Render to ./out/tier2.mp4 with the same verification as tier 1.
Do all of this in one run. No interactive checkpoints.
Run, check manifest, watch output. The blurbs are the part most viewers notice — they make the reel feel “produced” without you doing any extra work.
Tier 3: Three-act narrative with slides
The big swing. Story arc, slide overlays, title card, end card. This is what no other tool does.
Build tier3.mp4 from ./raw/. Tier 3 extends tier 2 with narrative structure,
slide overlays, and title/end cards.
START FROM the tier 2 pipeline. All rules apply. Then add:
STEP 12 — NARRATIVE STRUCTURE FROM ./outline.md
Read ./outline.md. It defines three acts with anchor phrases for each.
For each act:
- Find the talking-head clip whose verified transcript contains the act's anchor phrase
- That clip is the act's spine
- B-roll clips from the same time window go under the spine clip
(this is critical: never mix time windows across acts — morning b-roll plays
during morning narration, afternoon b-roll plays during afternoon narration)
- Render the act's slide as a 1.5s title card at the act's opening beat
STEP 13 — SLIDE OVERLAYS
For each act in ./outline.md, build a slide:
- Title (3-5 words) and body (2-3 short bullets) from the outline
- Top 32% of the 1080x1920 frame
- Background: brand.json primary color, 92% opacity
- Title: brand.json accent color, 64px Inter Bold
- Bullets: white, 36px Inter Regular
- Drop shadow: 0 8px 24px rgba(0,0,0,0.3)
- Enter: slide-down + fade, 280ms ease-out
- Exit: fade only, 220ms
- Hold: 4.5s per slide
STEP 14 — TITLE AND END CARDS
Title card (0:00 to 0:01.5):
- Full-frame brand.json primary color
- Centered title from outline.md
- Subtitle: "Month YYYY" from earliest clip's timestamp
- Fade to first content clip over 250ms
End card (final 1.8s):
- Full-frame brand.json primary color
- handle from brand.json
- Subtitle: "Repo in bio"
- Slow zoom-in 1.04x over the hold
STEP 15 — COMPOSITION GEOMETRY
When slides are visible:
- Talking-head reframed to bottom 56% of the frame
- Slides occupy top 32%, 80px gap below
- Captions stay at y=1720 (lower-third)
STEP 16 — UPDATED MANIFEST
./out/tier3_manifest.txt includes:
- Act assignments per clip
- Slide schedule with timestamps
- Title and end card timestamps
- Final EDL with all overlays
STEP 17 — RENDER
Render to ./out/tier3.mp4 with the same verification as tier 1.
Target runtime 50-65s (slightly longer to accommodate cards).
Do all of this in one run. No interactive checkpoints.
Run. Manifest. Watch. Tier 3 is the version that makes people ask how you made it.
Running it on volume (the actual unlock)
The whole point. Once tier 3 works on one folder, run it on ten folders overnight.
Save this as ./batch-run.sh:
#!/bin/bash
# batch-run.sh — run the pipeline on every folder in ./batch/
for folder in ./batch/*/; do
echo "=== Processing $folder ==="
# Symlink the folder to ./raw temporarily
rm -rf ./raw
ln -s "$(cd "$folder" && pwd)" ./raw
# Run the agentic pipeline (replace 'codex' with your CLI)
codex run "Build tier3.mp4 from ./raw/ following the pipeline rules in ./PIPELINE.md"
# Move output to a named file
folder_name=$(basename "$folder")
mv ./out/tier3.mp4 "./out/${folder_name}_reel.mp4"
mv ./out/tier3_manifest.txt "./out/${folder_name}_manifest.txt"
echo "=== Done: ${folder_name}_reel.mp4 ==="
done
echo "All folders processed. Outputs in ./out/"
Folder structure:
./batch/
├── hackathon_april/
│ ├── clip001.mov
│ ├── clip002.mov
│ └── ...
├── podcast_ep_47/
│ └── ...
├── pandey_factory/
│ └── ...
└── client_xyz/
└── ...
Each folder gets its own reel. Each reel uses the same pipeline rules but its own clips. If you want different blurbs or outlines per folder, drop a blurbs.json and outline.md inside each folder and update the script to read them per-run.
Kick off before bed. Wake up to a ./out/ folder full of reels.
Troubleshooting
The eight things that will go wrong, ranked by likelihood. Each fix is a one-prompt re-run.
1. Output is sideways or upside-down
You skipped the pre-bake or it didn’t catch your clips. Run ./normalize-raw.sh again. If clips are still wrong:
# Inspect a problem clip
ffprobe -select_streams v:0 -show_entries stream_side_data=rotation -show_entries stream_tags=rotate ./raw/your_clip.mov
If the rotation tag is anything other than 0, the pre-bake didn’t run on it. Re-run normalize-raw.sh on the original folder.
2. Captions have typos on your branded terms
The domain dictionary in the prompt didn’t cover them. Add them to the dictionary in the prompt and re-run:
Apply this domain dictionary, force-correcting near-matches:
["YourBrand", "YourTool", "YourPodcastName", ...add yours here]
Whisper will still misspell things outside the dictionary on first pass. The dictionary is your only reliable fix.
3. Pipeline used the same clip on loop / didn’t stitch
Your CLI ignored the “use at least 8 distinct clips” constraint because earlier rules in the prompt were too vague. The fix is the manifest — force the pipeline to print the EDL before rendering. If the manifest shows one clip used 14 times, the prompt needs to be re-run with stricter language. The prompts in this doc already include the manifest gate.
4. Render finished but the output is empty / black
Two common causes:
The score threshold dropped every clip. Lower the score threshold from 40 to 25 and re-run.
The face-tracked crop math went wrong on b-roll. Add
If no face is detected, fall back to center-crop without smoothingto the reframe rule in the prompt.
5. Audio is way too loud or way too quiet
The LUFS normalization didn’t apply, or your music bed was at a wildly different level. Check the manifest for “audio levels report.” If absent, add this to the prompt:
Audio normalization is mandatory for every clip:
- Speech clips: -16 LUFS integrated, -1 dBFS true peak
- Music bed: -22 LUFS under speech, -16 LUFS in speech gaps
Verify with ffmpeg loudnorm and log the measured values to the manifest.
6. Slides appear at the wrong moment in tier 3
The anchor phrase from outline.md didn’t match anything in the transcript, or matched something unintended. Three fixes:
Make anchor phrases more specific. “yesterday” is too common; “yesterday I went to” is unique.
Check
./out/tier3_transcript.txtfor the exact wording — the agentic CLI is matching against what Whisper transcribed, not what you said.If a phrase matches in multiple places, use the earliest match by adding
take the first match in transcript orderto the slide rule.
7. Music ducking sounds choppy
The duck ramp is too short. Default is 250ms. Increase to 400-500ms in the prompt:
- 450ms ramp on each duck transition (smoother)
8. The whole pipeline takes way too long
Tier 3 on 14 clips should take 5-8 minutes on a modern Mac. If you’re seeing 30+ minutes:
Check if Whisper is running on CPU instead of GPU. Whisper large-v3 on CPU is brutally slow.
Force Whisper to use the smallest model that still gives accurate captions on YOUR audio. If your audio is clean,
mediumis often fine and 3x faster thanlarge-v3. The domain dictionary covers most accuracy gaps from a smaller model.The Remotion render itself is single-threaded by default. Add
--concurrency=4to the render command to parallelize across cores.
The mindset shifts that matter
Ten things I learned the hard way. Skim these before you start your first run.
1. Shoot vertical from the start. Hold the phone how the reel will be watched. Vertical clips skip every rotation problem, every awkward crop. Five seconds of intention while filming saves an hour of pipeline pain.
2. Film more than you think you need. Pipelines drop bad clips automatically. 20 clips in, 8 used. That’s the math. If you film exactly enough, one bad clip kills the reel.
3. Talk in complete thoughts. The pipeline cuts on sentence boundaries. If you ramble and self-interrupt, it has nothing clean to cut on. One thought per clip. Pause before. Pause after.
4. Force a manifest before every render. Don’t trust the output, trust the plan. A manifest is the pipeline showing its work in plain English before it commits. Reads in ten seconds. Saves a five-minute render every time.
5. Build the pipeline once. Run it everywhere. The trap is treating each reel as a fresh project. The whole point is the opposite. One pipeline, many folders. If you find yourself rewriting the prompt for every video, you’re doing it wrong.
6. The first run will suck. Plan for it. No pipeline one-shots on attempt one. Captions typo. Music is too loud. Budget two hours for the first build. Every run after is two minutes of your time.
7. Be specific or be ignored. “Make it punchy” means nothing. “45-60 seconds, 8 clips minimum, no clip over 7 seconds, hard cuts” means something. Vague gets vague output.
8. Keep your domain words in a dictionary. Brand names, tool names, your own name. Force the captions to use those spellings always. Nothing kills a reel faster than your own product name spelled wrong on screen.
9. Don’t fight the pipeline. Fix the input. If the output is bad, 90% of the time the raw folder is the problem, not the prompt. Bad lighting, sideways clips, mumbled audio. Clean inputs first. Tweak prompts second.
10. Volume is the actual unlock. If you’re making one reel, use CapCut. If you’re making fifty, this changes your week. The pipeline pays off when the same logic runs across five folders while you’re at lunch.
What’s next
I’m building three more pipelines. Whichever gets the most votes ships next:
Auto-thumbnail generation — pipeline picks the best frame, adds text overlay, exports a 1080x1920 cover image for the reel
Multi-platform export — same reel, reformatted for TikTok, YouTube Shorts, and LinkedIn with platform-specific caption styles
Voice-cloned translations — reel dubbed into Hindi, Spanish, French, with lip-sync, no re-recording
Reply to the welcome email with your vote. Or comment on the YouTube video.
If you want this customized for your team
If your business makes 50+ videos a month — agency, podcast network, multi-brand creator, content team — I help build pipelines like this customized for your stack. Brand styling, content review workflow, multi-team access, integrated with your existing tools.
Reply to this email with what you’re trying to ship. I’ll tell you whether it’s worth a call.
Subscribe for the next breakdown
This newsletter goes out when I want to send it . BUT! Each one is a complete buildable pipeline like this one — no fluff, all prompts. Topics coming up:
The LinkedIn carousel pipeline (drop screenshots, get a posted carousel)
The podcast-to-shorts pipeline (one episode, ten reels, fully automated)
The Loom-to-tutorial pipeline (record once, edit never)
Built by Naman. Host of Ready Set Do (@ReadySetDoNaman). I run experiments like this and write up what actually works.