Video transcripts & storyboards
Goal: turn an MP4 into captions, searchable text, and a visual index an agent can read, without running speech-to-text or ffmpeg yourself.
POST /v1/videos/:id/enrich generates plain text, timed WebVTT, timestamped JSON segments, and bounded JPEG storyboard sheets for an owned upload. Videos without audio skip only the transcript. Private videos return expiry-matched signed URLs for every artifact.
Upload, enrich, wait
npm install -g pixelvault-cli@latest
# Upload a local MP4, start enrichment, wait, print typed JSON artifacts
pixelvault enrich-video ./launch-demo.mp4 --json
# Resume an existing asset (a local timeout never cancels the server job)
pixelvault enrich-video vid_abc123 --timeout 1200 --json The call is idempotent: it returns 202 while queued or processing and 200 when complete. Poll GET /v1/videos/:id until enrichment.status is complete or failed; a failed job can be started again and gets a new ven_ job ID. The polling pattern is described once in Async jobs.
{ "data": {
"enrichment": {
"job_id": "ven_abc123",
"status": "complete",
"transcript": {
"status": "complete",
"text_url": "https://img.pixelvault.dev/proj_xyz789/vid_abc123.transcript.txt",
"vtt_url": "https://img.pixelvault.dev/proj_xyz789/vid_abc123.transcript.vtt",
"segments_url": "https://img.pixelvault.dev/proj_xyz789/vid_abc123.transcript.json"
},
"storyboard": {
"status": "complete",
"index_url": "https://img.pixelvault.dev/proj_xyz789/vid_abc123.storyboard.json",
"frame_count": 16,
"sheets": [{ "url": "https://img.pixelvault.dev/proj_xyz789/vid_abc123.storyboard-1.jpg",
"start_seconds": 0, "end_seconds": 42.417, "frame_count": 16 }]
}
}
} } CLI details
enrich-video (CLI 0.23.0+) reports the asset and stable job IDs before polling, backs off for up to 600 seconds by default, and returns explicit queued, running, complete, failed, or timeout states. A local timeout does not cancel the server job. --no-wait returns immediately; --timeout 1200 changes only the local wait bound. For a new private upload use --private --expires 12h; an existing asset keeps its visibility. JSON includes typed artifact URLs with visibility and signed-link expiry, and omits transcript/storyboard URLs until they exist. URL sources are not accepted by this command: import with pixelvault upload <url> --json, then enrich the returned vid_ ID.
Use the artifacts
Attach the WebVTT file as captions:
<video controls src="https://img.pixelvault.dev/proj_xyz789/vid_abc123.mp4" crossorigin="anonymous">
<track kind="captions" srclang="en" label="English"
src="https://img.pixelvault.dev/proj_xyz789/vid_abc123.transcript.vtt" default>
</video> Feed segments_url (timestamped JSON) to an LLM for chaptering or search, and the storyboard sheets to a vision model to see the video without decoding it.
Limits and billing
Enrichment accepts source MP4s up to 100 MB and 10 minutes. It uses the monthly media-enrichment allowance (2 minutes on Free, 30 on Starter, 120 on Pro), shared with audio transcription; the remaining allowance must cover the full source duration (402 enrichment_limit_reached otherwise). Transcripts and storyboards count toward storage. Check what's left in GET /v1/usage (video_enrichment).
Skip polling with webhooks
Subscribe to video.enrichment.completed and video.enrichment.failed. The payload lists only artifacts that exist, with a refresh_url for private links. A runnable receiver is video-enrichment-webhook.mjs; see Webhooks for verification.
Next steps
- Transcribe audio with the same allowance.
- Make a social-ready copy of the same video.
- MCP:
enrich_videofor agents.