Use reference images to influence video subject, style, or identity without exact frame control
Use this guide when you need to add reference-to-video generation where images influence the output without forcing exact first or last frames.By the end, your implementation should submit a reference-to-video job with input_references.
An OpenRouter API key available as OPENROUTER_API_KEY
Node.js 20 or newer
One or more public HTTPS image URLs, starting with REFERENCE_IMAGE_URL
A model that supports reference-to-video, confirmed from the current OpenRouter video docs or model description
If you haven’t chosen a model yet, read Choose a Video Generation Model first. It helps you pick one based on clip duration, output shape, input type, audio, provider controls, and cost.
Use the API reference pages as the source of truth for exact fields:
Use input_references for visual guidance. Use frame_images only when you need exact frame control.Use stable, directly downloadable image URLs. Some providers cannot fetch image URLs that require cookies, redirects through HTML pages, bot checks, or unusual headers.
Submitting POST /api/v1/videos starts a real video generation job and may spend OpenRouter credits.
The video models endpoint doesn’t expose a dedicated structured reference-image field for every provider. Confirm reference support from the model description or current docs before you submit:
For bytedance/seedance-2.0-fast, the model list can confirm the example duration, resolution, and aspect_ratio; reference-image support may still need confirmation from the model description or docs.
Step 1: Write a prompt that tells the model how to use the references
Reference images work best when the prompt explains what should stay consistent.
Create a 4-second product video of the same backpack from the reference image.Keep the shape, color, and logo placement consistent.Place it on a wet city sidewalk at night with neon reflections.Use a slow orbiting camera move and realistic lighting.
Build the video request with input_references when the images should guide subject, identity, or style. Unlike frame_images, reference images are not exact frame anchors.Example request shape:
const apiKey = process.env.OPENROUTER_API_KEY;const referenceImageUrl = process.env.REFERENCE_IMAGE_URL;if (!apiKey) { throw new Error("Set OPENROUTER_API_KEY first.");}if (!referenceImageUrl) { throw new Error( "Set REFERENCE_IMAGE_URL to a directly downloadable image URL.", );}const response = await fetch("https://openrouter.ai/api/v1/videos", { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ model: "bytedance/seedance-2.0-fast", prompt: "Create a 4-second product video of the same backpack from the reference image. Keep the shape, color, and logo placement consistent. Place it on a wet city sidewalk at night with neon reflections. Use a slow orbiting camera move and realistic lighting.", duration: 4, resolution: "720p", aspect_ratio: "16:9", generate_audio: false, input_references: [ { type: "image_url", image_url: { url: referenceImageUrl, }, }, ], }),});if (!response.ok) { throw new Error(await response.text());}const job = await response.json();console.log(job);
The submit call returns the job fields immediately. In the QA run, the submitted job later completed and downloaded with this final summary:
Step 3: Add more references when consistency matters
Some models can use multiple reference images. Before doing this in production, check the current docs or model description for the selected model. Start with the smallest number of references that gives you enough consistency.
After submission, poll from a server route, worker, or job runner instead of the browser. Poll with a limit, stop on terminal failure, then download the completed video.Example polling and download helper:
import { writeFile } from "node:fs/promises";async function waitForVideo(job) { let current = job; for (let attempt = 1; attempt <= 60; attempt += 1) { if (current.status === "completed") { return current; } if (current.status === "failed") { throw new Error(current.error ?? "Video generation failed."); } if (["cancelled", "expired"].includes(current.status)) { throw new Error(current.error ?? `Video generation ${current.status}.`); } await new Promise((resolve) => setTimeout(resolve, 30_000)); if (!current.polling_url) { throw new Error("Video job did not include a polling_url."); } const pollingUrl = new URL(current.polling_url, "https://openrouter.ai"); const response = await fetch(pollingUrl, { headers: { Authorization: `Bearer ${apiKey}`, }, }); if (!response.ok) { throw new Error(await response.text()); } current = await response.json(); } throw new Error("Video generation did not complete after 60 attempts.");}async function downloadVideo(job) { const videoUrl = job.unsigned_urls?.[0] ?? `https://openrouter.ai/api/v1/videos/${job.id}/content?index=0`; const response = await fetch(videoUrl, { headers: videoUrl.startsWith("https://openrouter.ai/api/") ? { Authorization: `Bearer ${apiKey}` } : undefined, }); if (!response.ok) { throw new Error(await response.text()); } return Buffer.from(await response.arrayBuffer());}const completedJob = await waitForVideo(job);const videoBuffer = await downloadVideo(completedJob);await writeFile("reference-video.mp4", videoBuffer);console.log("Saved reference-video.mp4");
The QA run saved the finished video after polling completed:
The output should borrow subject, style, or identity cues from the reference images while still following the generated scene described in the prompt. The implementation should produce a playable MP4 from the completed job.
Was this page helpful?
Assistant
Responses are generated using AI and may contain mistakes.
