RINO: RGB In, RGB Out

Official PyTorch implementation of "Let RGB Be the Language of Vision".

Authors: Timing Yang, Jinrui Yang, Xinlong Li, Yuhan Wang, Haoran Li, Yanqing Liu, Guoyizhe Wei, Jixuan Ying, Chen Wei, Rama Chellappa, Yuyin Zhou, Cihang Xie, Alan Yuille, Feng Wang*

Johns Hopkins University · UC Santa Cruz · Carnegie Mellon University · Rice University

_{*Project lead}

RINO renders every input and output as RGB, so one frozen image editor handles both estimation and conditioned generation — no task-specific module.

Zero-shot vision benchmarks for open-source image-edit models under a single RGB-to-RGB interface. Every task is expressed as image editing: understanding tasks render the structured output (mask, depth, normals, pose, …) as an RGB image, and conditioned-generation tasks consume an RGB condition. No task-specific head, encoder, decoder, or adapter is added — the editor is used as a frozen black box.

Each task lives under tasks/ and follows the same layout — tasks/<task>/evaluate/ for code, tasks/<task>/output/ for runtime results — and is scored by copied official metric code so the numbers are comparable to the literature. Shared image-edit backends live at the repo root in core/. See each task's README for full details.

Backends

Three open-source image editors, all used as a black-box RGB-to-RGB editor (released weights, no fine-tuning), wired up in core/:

• Qwen-Image-Edit — Qwen/Qwen-Image-Edit-2511
• FireRed-Image-Edit — FireRedTeam/FireRed-Image-Edit-1.0
• LongCat-Image-Edit — meituan-longcat/LongCat-Image-Edit

Every task accepts BACKEND=qwen|firered|longcat with a matching MODEL=<hf-id>.

Demo

Two ready-to-run scripts under demo/ show the RGB-in / RGB-out interface end-to-end — one estimation task, one conditioned-generation task. No dataset, no metrics; defaults chain together so both run with zero args.

pip install -r requirements.txt

# 1) RGB -> grayscale depth
python demo/estimation_depth.py
# writes demo_output/depth/raw.png  (grayscale depth: near=bright, far=dark)
#        demo_output/depth/pred.npy (relative depth)

# 2) depth + caption -> RGB scene (consumes step 1's output)
python demo/generation_depth.py
# writes demo_output/gen_depth/scene.png

The two commands above, run back-to-back: one frozen editor turns an RGB photo into a grayscale depth map (step 1), then re-hallucinates an RGB scene from that depth (step 2).

Point them at your own inputs with --image, --depth, --caption, or override the editor with --backend qwen|firered|longcat + a matching --model <hf-id> (default: Qwen/Qwen-Image-Edit-2511). Weights are pulled from Hugging Face on first run. For the full benchmark on each task see tasks/<task>/README.md.

Citation

@misc{yang2026letrgblanguagevision,
      title={Let RGB Be the Language of Vision}, 
      author={Timing Yang and Jinrui Yang and Xinlong Li and Yuhan Wang and Haoran Li and Yanqing Liu and Guoyizhe Wei and Jixuan Ying and Chen Wei and Rama Chellappa and Yuyin Zhou and Cihang Xie and Alan Yuille and Feng Wang},
      year={2026},
      eprint={2607.12450},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.12450}, 
}