Image-to-video takes one frame and produces several seconds of motion from it. The first frame is yours; everything after it is invented. Understanding that one fact explains most of what these models do well and badly.
What the model is predicting
The model has learned how scenes tend to move. Given a starting frame, it predicts a plausible continuation — not the continuation, just one that is consistent with the first frame and with how things of that kind usually behave.
This is why the results feel natural when the motion is ordinary and strange when it is not. Hair moving, fabric settling, a slow camera drift, a shift of weight: all heavily represented in training data. Anything unusual has far less to draw on, and the model falls back on something generic.
Why the clips are short
Every generated frame is conditioned on the ones before it, so small errors compound. Five seconds in, the model is extrapolating from its own output rather than from your image, and drift becomes visible — faces shifting, backgrounds reorganising, objects changing shape.
Cost is the other half. Video is many images with temporal consistency on top, and the compute scales with length. Short clips are where quality and price are both still reasonable, which is why nearly every tool lands in the same few-second range.
Describing motion usefully
- Name one motion. Several at once produces a clip where none of them reads clearly.
- Say how fast. "Slowly turning" and "turning" give noticeably different results.
- Separate camera movement from subject movement. A slow push in is a different instruction from the subject stepping forward.
- Leave out what should stay still. Mentioning something tends to animate it.
Choosing the starting frame
The first frame matters more than the description. A sharp image with a clear subject and unambiguous depth gives the model something to work with. A soft or cluttered frame gives it room to invent, and it will.
Composition matters too. Leave space in the direction the motion is going. A subject already at the edge has nowhere to move, and the model will either keep them still or push the frame somewhere you did not want.
What to expect
First attempts rarely land. The useful loop is to generate, watch which part failed, and change only the instruction covering that part — keeping the same starting frame, which is the stable half of the equation.
Judge the result at full size and full speed. Video that looks acceptable in a small preview often shows its problems at scale, and problems in the last second are common enough to be worth checking for specifically.
Why the end of a clip is worse than the start
Watch several generated clips and a pattern appears: the opening is close to your image and convincing, and quality falls away towards the end. This is not a flaw in any one model, it is a property of how they generate.
Each frame is conditioned on what came before. Early on, that chain leads back to your actual image. Later, it leads back mostly to the model’s own earlier guesses, and any small error has been fed forward and amplified at every step. By the final second the model is extrapolating from itself.
Two things follow. Judge a clip by its last second, because that is where the model is weakest and where a viewer will notice. And put the moment you care about early, since the beginning is the part you can rely on.
Not wasting money on attempts
Video costs far more per attempt than images, usually by a wide margin, so a scattergun approach gets expensive quickly.
- Settle the starting frame completely before generating any video. It is the cheap half and it determines most of the result.
- Change one thing per attempt. Rewriting the whole instruction means you cannot tell which change helped.
- Ask for the shortest clip that shows what you need. Longer is more expensive and degrades more.
- Keep what nearly worked. A clip with one bad second is often fixable by a small adjustment, and starting over discards what was already right.
Above all, resist describing an entire sequence. These models produce a continuous moment, not a shot list. Asking for someone to turn, then smile, then walk away reliably produces none of the three cleanly.
What these models are not for
It is worth being clear about the ceiling. Image-to-video produces a moment, not a scene: a few seconds of plausible continuation from one frame. It does not produce narrative, it does not hold a character across separate clips, and it does not take direction about what should happen at a particular second.
Anything needing continuity across shots still needs the traditional approach, with these clips as individual elements inside it. Treated as a single take, they are strong. Treated as a shot list, they disappoint, and most frustration with them comes from that mismatch rather than from the quality of any one result.