Abstract
Video diffusion transformers address their tokens by position on the pixel-time grid: an address in the tensor, not in the world. The address we would want, the world point a token depicts, lies on a surface not yet generated, while its camera ray is fixed once the user specifies a trajectory. SCoPE therefore treats the ray as a second positional coordinate, and camera control becomes a property of the coordinate system, not an added module. The ray is added to the pretrained attention's queries and keys, and the score gains a term that reads the two rays alone. Its canonical form, the reciprocal product of line geometry, measures how nearly two lines of sight meet. Normalize-Gate-Inject makes a single encoding trainable across metric and up-to-scale pose sources. The retrofit keeps RoPE bit-exact, starts from the unchanged pretrained DiT, and adds under 0.1% new parameters. On Wan2.2 at 5B and 14B under matched data and budget, SCoPE improves every camera-controllability and fidelity metric, and the margins widen with model size. At 14B, rotation error falls 29% and FVD 43% below the strongest baseline on each metric.
Key Ideas
- A second positional coordinate. The world point a token will depict is unknown before generation, but its camera ray is fixed by the user's trajectory. SCoPE uses that ray as the token's second address.
- Geometry inside attention. Adding ray features to pretrained queries and keys gives the attention score a ray-only term. Its canonical form is the reciprocal product of line geometry, which measures how nearly two lines of sight meet.
- A minimal, stable retrofit. Normalize-Gate-Inject works across metric and up-to-scale poses while preserving RoPE bit-exactly. SCoPE starts from the unchanged pretrained DiT, adds under 0.1% parameters, and improves both 5B and 14B models.
Scenes
One generated scene browsed under six different camera trajectories. The inset visualizes the commanded camera path.
Motions
WASD inputs drive the camera trajectory that moves the third-person subject. Overlay keys reflect the per-frame motion.
Real-World Scenes
Diverse real captures driven along varied camera paths from a single frame.
In-the-Wild Videos
Everyday footage re-rendered under controllable camera motion.
Stylized & Artistic Scenes
Illustrated and painterly inputs animated with 3D-consistent camera control.
Citation
@article{yin2026scope,
title = {SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers},
author = {Yin, Minghao and Lu, Jiahao and Hu, Wenbo and Zhao, Wang and Shan, Ying and Han, Kai},
journal = {arXiv preprint arXiv:2606.27345},
year = {2026}
}