UT Austin LIVEYouTube

HDR-Q: Seeing Beyond 8 Bits

Multimodal LLMs for HDR Video Quality Assessment
with HDR-Aware Policy Optimization

Shreshth Saini

Laboratory for Image & Video Engineering (LIVE) · The University of Texas at Austin

Advised by Prof. Alan C. Bovik

CVPR 2026 · December 2025

Saini et al., Seeing Beyond 8bits, CVPR 2026 (arXiv:2603.00938)

Can AI see and reason about HDR video quality?

Right now the answer is no. Every existing model was built for the 8-bit SDR world.

SDR encoders cannot see what HDR adds

Every modern phone captures 10-bit HDR by default.

  • HDR10 is 10-bit, BT.2020 gamut, PQ (perceptual quantizer) transfer.
  • Peak of 1000 nits or more, against about 100 for SDR.
  • New artifacts such as highlight clipping, near-black banding and color blooming.
  • SigLIP and CLIP read 8-bit sRGB, so they are structurally blind to these.
HDR vs SDR

HDR keeps luminance and color detail that SDR collapses.

HDR-QCVPR 2026

Where current methods break

Method familyHDR inputPerceptual groundingReasoningContinuous MOSInterpretableSROCC
Classical VQA (BRISQUE, VMAF)✗✗✗✓✗0.41
Deep VQA (FastVQA, DOVER)✗partial✗✓✗0.51
HDR-specific (HIDRO-VQA) Saini et al., WACV 2024✓✓✗✓✗0.85
MLLM-VQA (Q-Insight, DeQA)✗✗✓partial✓0.52
HDR-Q (ours)✓✓✓✓✓0.92
VQA is video quality assessment, MLLM a multimodal LLM, MOS the mean opinion score, SROCC the Spearman rank correlation with MOS.

Only HDR-Q has HDR-aware perception, a continuous score and readable reasoning at once.

HDR-QCVPR 2026

Three obstacles, and the third surprised us

O1. SDR-pretrained encoders

Blind to 10-bit PQ.

O2. A continuous score from tokens

The MOS (mean opinion score) must come out of text.

O3. Modality neglect

GRPO (Group Relative Policy Optimization) ignores HDR tokens.

0.891SROCC with SDR input only
0.875standard GRPO, HDR input added

Adding HDR made it worse.

HDR-QCVPR 2026

Beyond8Bits, the data HDR-Q trains on Saini et al., CVPR 2026

44,276HDR-UGC video clips
6,861unique source videos
1.5M+human quality ratings
  • The first large-scale HDR-UGC (user-generated content) quality dataset.
  • About 35 ratings per video, inter-subject SROCC 0.90.
  • 10-bit HEVC, PQ, BT.2020, 360p to 1080p, 0.2 to 5 Mbps.
  • 2,253 crowdsourced videos plus 4,608 from Vimeo.
Beyond8Bits scene categories

Scene categories across the 44K clips. The full figure is on a backup slide.

HDR-QCVPR 2026

HDR-Q has one part per obstacle

Architecture

HDR-aware encoder (O1)

SigLIP-2 on native 10-bit PQ, trained to tell HDR from SDR.

HAPO training (O3)

HDR-Aware Policy Optimization makes the policy use the HDR tokens.

Structured output (O2)

Ovis2.5 (9B), rank-4 LoRA. Reasoning in <think>, the MOS in <answer>.

HDR-QCVPR 2026

The encoder separates HDR from SDR

Encoder training

SigLIP-2 finetuned on matched HDR-SDR pairs, with quality captions written by Qwen2.5-VL-72B.

Lctr = max(0, δ − D(E(xHDR), E(c)) + D(E(xSDR), E(c)))

HDR moves toward the caption, SDR moves away.

Lenc = LSigmoid + λctr · Lctr

Input stays native 10-bit PQ.

HDR-QCVPR 2026

Adding HDR input made standard GRPO worse

0.8914SROCC, SDR input only
0.8753GRPO with HDR added
0.9206HAPO with HDR added

Why

  • GRPO treats every token alike.
  • The model can shortcut through the text context.
  • SDR features are familiar, HDR ones are foreign.

HAPO rewards a different output for HDR than for SDR.

HDR-QCVPR 2026

HAPO pushes HDR and SDR outputs apart

KHDR(θ) = DKL(πθHDR ‖ πθSDR)

Ignore HDR and this KL is 0, so we maximize it.

Iθ*(output; HDR | SDR) ≥ γ · KHDR(θ*) − κ

Mutual information with HDR is bounded below.

HDR-QCVPR 2026

HAPO adds three terms to GRPO

JHAPO(θ) = E[Σ min(ρ · Ã, clip(…) · Ã)] − β DKL(πHDR ‖ πref) + γ KHDR − Hdual
TermWhat it doesSetting
ÃEntropy-weighted advantage (HEW), newλHEW = 0.3
β DKLReference KL from GRPO, keeps training stableβ = 0.02
+ γ KHDRContrastive KL, new. Grounds the output in HDRγ = 0.5
− HdualDual entropy, new. Prevents collapseη1 = 0.01, η2 = 0.05

Training is two RL stages, modality alignment then score calibration, with no supervised fine-tuning (SFT).

HDR-QCVPR 2026

0.921 SROCC on Beyond8Bits, and it holds zero-shot

0.921SROCC on Beyond8Bits
0.908LIVE-HDR, zero-shot
0.725SFV+HDR, zero-shot
MethodSROCC ↑PLCC ↑RMSE ↓
BRISQUE0.4100.46911.70
CONTRIQUE0.6250.60515.02
CONVIQT0.7990.8108.48
HIDRO-VQA Saini et al., WACV 20240.8510.8786.09
Q-Insight (best MLLM)0.5170.56220.78
HDR-Q (SDR only)0.8910.8907.42
HDR-Q (full)0.9210.9125.16
Beyond8Bits test set (Saini et al., CVPR 2026). Bold is best, underline is second best.
Cross-dataset

Zero-shot on LIVE-HDR and SFV+HDR, no fine-tuning on the target datasets.

HDR-QCVPR 2026

The baseline invents artifacts, HDR-Q names real ones

Qualitative responses

Same video for both. HAPO (ours) is HDR-Q. Prompt and frame are on a backup slide.

Both get the same prompt, rate the video 0 to 100 and reason in <think> first.

Ovis2.5 baseline

Reports underexposure, color blooming and unnatural hues that are not in the video.

HDR-Q

Highlights roll off without clipping, hues stay stable, no jitter or smear.

HDR-QCVPR 2026

Contrastive KL is the component that matters most

VariantSROCCRMSEToken entropy
GRPO baseline0.8110.730.20
+ HDR encoder0.838.960.24
HAPO without contrastive KL0.867.100.29
HAPO without HEW0.886.110.27
HAPO without dual entropy0.915.820.26
HDR-Q (full)0.925.150.33
Each HAPO row removes one component from the full model.
  • Without contrastive KL, 0.92 falls to 0.86, the largest drop.
  • Without HEW, 0.88. Without dual entropy, 0.91.
  • The HDR encoder lifts GRPO from 0.81 to 0.83.

Token entropy rises from 0.20 to 0.33.

HDR-QCVPR 2026

What we learned, what is still open

Lessons

  • Modality neglect is the bottleneck, more than encoder capacity.
  • Collapsing token entropy signals text shortcuts.
  • Two RL stages beat SFT then RL.

Open problems

  • Only T = 8 frames, so slow temporal patterns are missed.
  • Little synthetic or gaming HDR in training.
  • Too slow for real time.
HDR-QCVPR 2026

Takeaways

  1. First MLLM for HDR video quality, 0.92 SROCC against 0.52.
  2. HAPO fixes modality neglect, with a mutual-information guarantee.
  3. Zero-shot, 0.908 on LIVE-HDR, 0.725 on SFV+HDR.
  4. The reasoning names real HDR artifacts.

Related work of mine

  • BrightRate, HDR-UGC quality model. WACV 2026 (oral, best student paper)
  • CHUG, crowdsourced HDR-UGC dataset. ICIP 2025
  • HIDRO-VQA, contrastive HDR features. WACV 2024

Thank you.

UT Austin LIVE YouTube

Backup

Backup slides

Training pipeline and reward design

Stage 1, modality alignment

Full HAPO with γ = 0.5 on a curated Beyond8Bits subset of matched HDR-SDR pairs. The goal is to teach the model to attend to HDR-specific information.

Stage 2, quality calibration

The full Beyond8Bits corpus. The score reward is the main signal and γ is reduced. The goal is prediction accuracy while keeping the HDR grounding from stage 1.

Both stages use RL. There is no SFT stage.

Why two stages

Modality alignment and quality calibration pull against each other. Aggressive contrastive training hurts absolute accuracy, and pure quality training lets modality neglect back in. Doing them in sequence resolves the tension.

Composite reward

R = wfmt · Rfmt + wsc · Rscore + wself · Rself
  • Rfmt: binary, 1 when the <think> and <answer> tags are valid.
  • Rscore: Gaussian, exp(−(ŝ − s*)2 / 2σ2) with σ = 3. Smooth and differentiable.
  • Rself: majority-vote consistency across K = 8 completions.

Setup: Ovis2.5 (9B) with rank-4 LoRA, K = 8, ε = 0.1, 4 H200 GPUs, BF16, T = 8 frames, native 10-bit PQ input.

HDR-QCVPR 2026

Training dynamics under HAPO and GRPO

Entropy

Token entropy during training. GRPO collapses, HAPO keeps exploring.

How GRPO fails

Entropy drops, the policy turns deterministic, ignores the visual modality and shortcuts through text context.

How HAPO stabilizes

Dual entropy holds H ≈ 0.33 at quality-critical positions, so exploration survives.

Reasoning gets shorter

Chain of thought drops from 168 to 137 tokens (18% fewer). Boilerplate goes, the quality observations stay.

HDR-QCVPR 2026

Beyond8Bits, the data HDR-Q trains on (full figure) Saini et al., CVPR 2026

44,276HDR-UGC video clips
6,861unique source videos
1.5M+human quality ratings
  • The first large-scale HDR-UGC (user-generated content) quality dataset.
  • About 35 ratings per video, inter-subject SROCC 0.90.
  • 10-bit HEVC, PQ, BT.2020, 360p to 1080p, 0.2 to 5 Mbps.
  • 2,253 crowdsourced videos plus 4,608 from Vimeo.
Beyond8Bits

Dataset diversity, HDR against SDR characteristics, and HDR-Q performance gains.

HDR-QCVPR 2026

The baseline invents artifacts, HDR-Q names real ones (full figure)

Qualitative

Both get the same prompt, rate the video 0 to 100 and reason in <think> first.

Ovis2.5 baseline

Reports underexposure, color blooming and unnatural hues that are not in the video.

HDR-Q

Highlights roll off without clipping, hues stay stable, no jitter or smear.

HDR-QCVPR 2026