How CUTLINE works
You give one sentence describing what you want the video to do-for example, “Explain why coffee makes you feel awake in 30 seconds” or “A short product teaser for our new API.” CUTLINE turns that into a finished short video: script, voiceover, images, motion, and subtitles. No templates, no storyboards; the system acts as both director and editor.
This page walks through the pipeline, the tech behind it, how to pick models, the two video modes, and how jobs are run in the background. Use the side menu to jump to a section.
Pipeline: from sentence to video
The video is built in a single linear pipeline: 12 stages run one after another. Each stage uses the output of the previous one. If any stage fails, the whole job fails (though the system retries a few times for things like temporary API errors). There are no branches or optional paths-every slideshow video goes through the same sequence so behavior is predictable and debuggable.
Input → Your one sentence (e.g. “Explain why coffee makes you feel awake in 30 seconds”)
1. Intent
2. Narrative
3. Shots
4. Script
5. Subtitles (draft)
6. TTS (voice)
7. Subtitle refine
8. Motion
9. Asset analysis (if you uploaded assets)
10. Visuals
11. Image sourcing
12. Remotion render
Output →
/temp/[jobId].mp4What each pipeline stage does
Here’s what happens at each step and why it matters for the final video.
1. Intent
Understanding your sentence
An AI (via OpenRouter) reads your one sentence and turns it into a structured “intent”: who the video is for, what it’s trying to achieve (e.g. explain, persuade, tease), the tone (professional, casual, etc.), how complex the topic is, and how long the video should be. If you don’t say a duration, the system infers one. Everything later in the pipeline uses this intent.
2. Narrative
Planning the story
From the intent, the same AI plans the narrative: a simple arc (e.g. hook → explanation → conclusion), 3-5 beats, and pacing. This is an internal plan, not something you see-it’s used to decide what each shot will be and what the script will say.
3. Shots
Breaking the video into shots
The AI produces a list of 8-12 shots. For each shot it decides: purpose (e.g. establish, reveal, transition), how long it lasts, how much text it carries, and how it connects to the previous and next shot. This is the “edit decision list” that drives both the script and the final cut.
4. Script
Writing what’s spoken
The AI writes the actual words (or silence) for each shot, aligned to the shot list. So you get one script entry per shot, with text that will be spoken by the TTS voice or left silent. This script is what gets turned into audio and subtitles.
5. Subtitles (draft)
Chunking and rough timing
The script is split into subtitle-sized chunks and given an estimated duration per chunk. This is a first pass; the real timing comes after TTS, when we know exactly when each word is spoken.
6. TTS (text-to-speech)
Generating the voiceover
Each script segment is sent to ElevenLabs or PlayHT. They return audio (and, where supported, word-level timings). Segments with no text get silence. The result is one continuous audio track plus timing data used to refine subtitles.
7. Subtitle refine
Syncing subtitles to the voice
Using the word timings from TTS, subtitle chunks are adjusted so they appear and disappear in sync with what’s actually spoken. That way captions match the voice instead of being based only on estimates.
8. Motion
How each shot moves
From the shot list, the system computes a “motion spec” per shot: e.g. scale, pan, zoom. No AI here-it’s rules based on shot purpose and order. This drives how the image is animated in the final render.
9. Asset analysis
Understanding your uploads (optional)
If you uploaded a logo, product photos, or reference media, an AI with vision looks at them and extracts: dominant colors, aspect ratio, suggested placement (e.g. logo as watermark or in outro), and for product photos, suggested shot types. This feeds into the visual style and image sourcing.
10. Visuals
Overall look and layout
From the intent and any analyzed assets, the system decides the visual spec: colors, layout tendencies. Again, this is in-process logic (no extra API call), and it guides how the Remotion composition looks.
11. Image sourcing
Getting an image for every shot
Every shot needs an image. For each shot, the AI (OpenRouter) suggests a search query or image prompt. The system then tries, in order: Unsplash (stock photos), DALL·E 3 (AI-generated), Pexels (another stock source), and if needed a simplified query. If everything fails, a placeholder image is used so the video still renders. If you uploaded product photos, some shots may be assigned those instead.
12. Remotion render
Assembling the final video
Remotion (React-based video framework) takes the script, shot list, subtitles, motion spec, image URLs, and TTS audio and renders a single MP4. Output is written to public/temp/[jobId].mp4 and served at /temp/[jobId].mp4.
What CUTLINE uses (tech stack)
A quick reference for the main technologies and services. All of these are configurable via environment variables or (where noted) per-job options.
Next.js 16
The app and API: App Router, React 19, API routes for generate, assets, and job status. The web UI and the Remotion composition share the same stack.
OpenRouter
One API for many LLMs. Used for Intent, Narrative, Shots, Script, image-query derivation, and asset analysis. You set a default model in .env and can override it per job with the Text model dropdown or the textModel API field.
ElevenLabs / PlayHT
Text-to-speech. You choose one provider via TTS_PROVIDER. They return audio and (where available) word timings so subtitles can be refined. Set the relevant API keys and voice ID in .env.
Unsplash, Pexels, DALL·E 3
Images for each shot. The pipeline tries Unsplash first, then DALL·E, then Pexels, then a simplified query. At least one stock source (Unsplash or Pexels) is required; OpenAI is recommended for a strong fallback.
Remotion 4
Programmatic video. The final MP4 is rendered by Remotion from a React composition: it layers images, motion, subtitles, and audio into one file. Same codebase as the app, so rendering is deterministic and easy to reason about.
BullMQ + Redis
Background jobs. When you submit a video, a job is added to a queue. A separate worker process (npm run worker) runs the pipeline. Redis stores the queue and is also used for rate limiting. You must run Redis (locally or a managed service) for generation to work.
Google Veo (optional)
Used only in “Talking object” mode: AI-generated talking character clips. Requires GEMINI_API_KEY. For videos longer than ~8 seconds, multiple clips are stitched with ffmpeg.
Storage
Uploads can be stored on disk (local) or in S3. There is no database for pipeline state-job status lives in Redis. Rendered videos and temp assets are cleaned up automatically based on retention settings.
How to choose and change AI models
Every AI step in the pipeline (Intent, Narrative, Shots, Script, image-query derivation, and asset analysis) goes through OpenRouter. So you have one place to control which “brain” is used: either a single default for all jobs, or a per-job override.
Default model (all jobs)
Set OPENROUTER_MODEL in .env.local. For example, anthropic/claude-3.5-haiku is a fast default. This model is used for every LLM stage unless you override it for a specific job.
For asset analysis (when you upload images), you can set OPENROUTER_VISION_MODEL to a vision-capable model. If you don’t, the same OPENROUTER_MODEL is used.
Per-job override
On the main page, the “Text model” dropdown lets you pick a different model for that single video. When you start a job, the app sends textModel in the request body. That model is then used for every LLM stage in that job (Intent, Narrative, Shots, Script, image query, asset analysis). Useful when you want to try a more capable model for one video without changing .env.
You can use any model ID that OpenRouter supports (e.g. anthropic/claude-3.5-sonnet, openai/gpt-4o) as long as your OpenRouter API key has access.
Example options in the dropdown (approximate limits are from the providers):
• Default (from env) - uses OPENROUTER_MODEL
• google/gemini-2.0-flash-exp - ~500/day
• google/gemini-2.5-flash-preview-05-20 - ~10K/day
• google/gemini-2.5-flash-image - ~2K/day
• anthropic/claude-3.5-haiku - fast default text model
Slideshow vs Talking object
CUTLINE has two ways to produce a video. “Slideshow” is the full pipeline above: script, voice, stock or generated images, motion, subtitles, all composed in Remotion. “Talking object” skips that and uses Google Veo to generate a talking character that speaks your script-no slideshow, no image sourcing; just AI-generated video.
Slideshow (default)
The full 12-stage pipeline runs. You get a classic short-form video: voiceover, one image per shot (from Unsplash/DALL·E/Pexels or your uploads), motion (pan, zoom, etc.), and subtitles. Duration is either inferred from your sentence or set by you (10-60 seconds) via the duration control or durationSeconds in the API.
Best for: explainers, product teasers, social clips where you want a polished edit with voice and imagery.
Talking object
The pipeline still does Intent → Narrative → Shots → Script so the script matches your intent. But instead of TTS + images + Remotion, the script (or chunks of it) is sent to Google Veo with a prompt like “Cartoon [subject] with a friendly face, talking to camera. Say the following: [script].” You get back a short video of an AI-generated character speaking. No image sourcing or Remotion for this mode.
You need GEMINI_API_KEY in .env.local. Veo produces clips of roughly 8 seconds. For longer videos (e.g. 30 or 60 seconds), the system splits the script into ~8-second chunks, generates one Veo clip per chunk, then concatenates them with ffmpeg. So for talking_object videos longer than 8 seconds, ffmpeg must be installed (on PATH or via FFMPEG_PATH). If the script is short for the requested duration, the system may extend the script so the video fills the length.
Best for: character-driven explainers or fun, talking-avatar style clips when you don’t need a slideshow.
How jobs run: submit, worker, poll
Video generation can take a minute or more, so it runs in the background. The browser never waits for the pipeline to finish. Instead, you get a job ID immediately and the UI polls until the job is done. That requires Redis and a separate worker process-the Next.js app only creates jobs and returns status; it does not run the pipeline itself.
1.
Submit
You enter your sentence (and optionally duration, mode, text model, assets) and hit create. The app sends POST /api/generate with your input. The API validates the input, applies rate limiting, then adds a job to the BullMQ queue and returns a jobId. Nothing has been rendered yet-the job is just queued.
2.
Worker
Somewhere (same machine or another server) you run npm run worker. That process connects to the same Redis, picks up jobs from the queue, and for each job runs the full pipeline (Intent through Remotion render, or the talking_object path). When it finishes, it writes the MP4 to public/temp/[jobId].mp4 and marks the job completed. If something fails, the job is marked failed and the error is stored so the UI can show it.
3.
Poll
The UI calls GET /api/generate/[jobId] every few seconds. The API returns status (pending, processing, completed, or failed) and, when completed, the video URL (e.g. /temp/[jobId].mp4). When the status is completed or failed, the UI stops polling and either plays the video or shows the error.
4.
Cleanup
When the worker is running, a repeatable BullMQ job also runs periodically. It deletes old temp videos, expired uploads, and per-job image caches based on VIDEO_RETENTION_HOURS, UPLOAD_RETENTION_HOURS, and CLEANUP_ENABLED. So you don’t have to manually delete old files.
Optional inputs you can send
Besides the required one-sentence input, you can send these in the body of POST /api/generate (or use the controls on the main page / Generate page).
- durationSeconds - A number between 10 and 60. Overrides the duration that would otherwise be inferred from your sentence. Used for both slideshow and talking_object.
- assetIds - An array of strings: the IDs you got from
POST /api/assets/upload. When present, the pipeline runs asset analysis (logo, product photos, reference media) and uses the results in the visual spec and image sourcing (e.g. assigning your product photos to certain shots). - brandColors - An object
{ primary: "hex", secondary?: "hex" }. Passed into the visual spec so the composition can use your brand colors where appropriate. - mode - Either
"slideshow"or"talking_object". Default is slideshow. - textModel - An OpenRouter model ID (e.g.
google/gemini-2.5-flash-preview-05-20). OverridesOPENROUTER_MODELfor this job only; every LLM stage in that job uses this model.
Environment variables
Copy .env.example to .env.local and fill in the values. The README and docs/IMAGE_API_KEYS.md explain where to get each API key.
# Required for generation
REDIS_URL - Queue and rate limiting (e.g. redis://localhost:6379 or a managed Redis URL).
OPENROUTER_API_KEY - Used for every LLM stage (Intent, Narrative, Shots, Script, image query, asset analysis).
# TTS - pick one provider
ELEVENLABS_API_KEY and TTS_PROVIDER=elevenlabs (and optionally ELEVENLABS_USE_MP3, TTS_VOICE_ID).
Or PLAYHT_API_KEY, PLAYHT_USER_ID, and TTS_PROVIDER=playht.
# Images - at least one stock source; DALL·E recommended as fallback
UNSPLASH_ACCESS_KEY and/or PEXELS_API_KEY; OPENAI_API_KEY for DALL·E 3.
# Optional
OPENROUTER_MODEL, OPENROUTER_VISION_MODEL - default LLM (and vision model for asset analysis).
GEMINI_API_KEY - required only for talking_object mode (Veo).
FFMPEG_PATH - for talking_object videos longer than 8 seconds (concatenation).
CLEANUP_ENABLED, VIDEO_RETENTION_HOURS, UPLOAD_RETENTION_HOURS, RATE_LIMIT_* - tune cleanup and rate limits.