Productized AI sports-coaching pipeline — video in, narrated coaching out
Problem solved: Consumer-product founders need an end-to-end video analysis pipeline cheap enough to support $20–30/mo SaaS pricing; this stack ships at ~$2.79/hr to analyze 5fps 1080p video.
What
End-to-end video analysis pipeline on Google Cloud Run (8 vCPU, 32 GB RAM, no GPU): Gemini 2.5 Flash watches the footage and emits JSON segments with timestamps and coaching text; Google Cloud Video Intelligence (Person Detection) extracts 12 skeletal landmarks at 10 FPS; Gemini 2.5 Flash TTS narrates at 24 kHz, speed-adjusted via ffmpeg atempo; ffmpeg composes the final video (footage + skeleton overlay + caption + narration); results land in Firestore. Generalises to any video-analysis vertical (training, fitness, technique coaching).
Why it matters
Processing time runs ~2 min for a 30-second clip. 13 languages supported. The economics ($2.79/hr to analyze 5fps 1080p) clear the bar for consumer-subscription pricing against $250–400/yr competitor pricing. The architecture is replicable for any coach-replacement product (golf, tennis, martial arts, dance, music technique).
End-to-end
- Stand up a Cloud Run service (8 vCPU, 32 GB RAM, no GPU is enough).
- On upload, call Gemini 2.5 Flash with the video; parse output as JSON segments (timestamp ranges + coaching text).
- In parallel, call Google Cloud Video Intelligence Person Detection for 12 skeletal landmarks at 10 FPS.
- Send each coaching segment to Gemini 2.5 Flash TTS at 24 kHz; pad/compress audio length to segment length via ffmpeg
atempo. - Compose final video with ffmpeg: original footage + skeleton overlay + captioned coaching text + narrated audio.
- Write results (rendered video URL, segments, landmarks) to Firestore.
- For 1-hour videos, fan out: split into ~12 chunks, run steps 2–4 in parallel, merge with a single text-merge prompt; wall-time collapses to one chunk's runtime.
Gotchas
Sequential single-pass processing on hour-long video burns wall time even though compute is cheap. The parallel-chunk fan-out (12 chunks → merge) is the unlock; without it the UX is "upload then come back tomorrow", which is fatal at consumer pricing.
Tools
- Google Cloud Run, Gemini 2.5 Flash API (incl. TTS), Cloud Video Intelligence API
- ffmpeg (incl.
atempofilter), OpenCV - Firestore for results
- Parent app (web or mobile) that exposes the finished output