The problem
Most pictures are still SDR
HDR displays are widespread, but most images and video remain 8-bit SDR in BT.709. Converting that material to 10-bit HDR in PQ and BT.2020 is not a simple format change. SDR creation compresses luminance and gamut, so the inverse mapping is underdetermined.
A fixed inverse tone curve can brighten an image, but it cannot recover highlight structure that is no longer present in the SDR signal. A learned converter can infer missing structure, but a model trained around one synthetic degradation can become tied to that degradation. Video adds another failure mode: converting each frame independently can make brightness and color flicker over time.
LumaFlux treats SDR to HDR conversion as a constrained generative problem. The model needs enough prior knowledge to reconstruct plausible highlights, but it also needs physical and perceptual controls that keep the output tied to the input.
The idea
Steer a frozen generative model
LumaFlux does not retrain a generative backbone. FLUX.1-dev and its VAE remain frozen. A frozen SigLIP encoder also supplies image features. The trainable parts total 71,315,893 parameters, or 0.57% of the FLUX.1-dev backbone.
The point of freezing the backbone is separation of responsibilities. FLUX.1-dev provides the image prior. Small adapters tell that prior where luminance, edges, color, and image content require HDR-specific changes. The tone head then calibrates the frozen VAE decode into display-referred HDR luminance.
PGA
Physically-Guided Adaptation adds gated low-rank attention residuals. Luminance, gradient, saturation, and spectral-band cues decide where those residuals should act.
PCM
Perceptual Cross-Modulation uses FiLM conditioning from frozen SigLIP image features. It modulates the hidden representation with perceptual context from the SDR frame.
HDR Residual Coupler
The HDR Residual Coupler merges the physical and perceptual paths. Its contribution changes with both the solver timestep and the transformer layer.
RQS tone-field decoder
The RQS tone-field decoder predicts a monotone rational-quadratic spline. The spline expands luminance without reversing tone order and calibrates the frozen VAE output for HDR display.
Inference
Prompt-free in 8 solver steps
The input frame itself supplies all conditioning. LumaFlux uses no text prompt and has no sampling-strength knob. The bridge begins near the encoded SDR frame:
z1 = VAEenc(SDR) + 0.05 · ε. The rectified-flow solver then integrates from t = 1 to t = 0 in K = 8 steps.Starting from the SDR latent makes the task a bridge from the observed image, not open-ended generation. Eight steps are the operating point: 40 steps cost 4.7 times more with no quality gain.
Video
The same image model, with corresponding paths
LumaFlux uses the same image model for video. It adds no temporal layers, optical flow, video fine-tuning, or temporal loss. At inference time, all frames use a shared bridge-noise realization, and an exponential moving average is applied to the RQS tone-curve parameters actually used for each frame.
The shared noise matters because it removes a source of frame-to-frame randomness. Neighbouring SDR frames start with the same noise pattern, so corresponding content is pushed onto corresponding rectified-flow paths instead of being perturbed in unrelated directions. The RQS-parameter EMA then smooths the applied tone mapping without averaging the image pixels.
Together, shared noise and the RQS-parameter EMA reduce signed excess PU21 flicker by 53.3%. Pooled raw temporal energy remains within 3.3% of the reference, so the reduction does not come from simply flattening temporal change. This result uses zero temporal training.
Evaluation
One protocol for every method
Luma-Eval evaluates every method with identical inputs, output encoding, and metric implementations. This removes protocol changes that can make separately reported numbers difficult to compare.
The benchmark has three tracks:
- synthetic tone-mapping and codec degradations;
- expert-graded SDR from LIVE-TMHDR;
- native paired SDR and HDR.
It reports seven metrics: PU21-PSNR, PU21-PSNR-Y, PU21-SSIM, dE-ITP, HDR-LPIPS, HDR-VDP-3, and FR-HIDROVQA. All evaluation samples use 16-bit PQ BT.2020 at a common 1,000-nit mastering convention.
lumaflux-hdrtv1k checkpoint trained on the HDRTV1K training split. The main mixed-corpus lumaflux-main model was never trained on HDRTV1K, and its 27.52 dB result is zero-shot.| HDRTV1K model | PSNR | SSIM | SR-SIM | dE-ITP | HDR-VDP-3 |
|---|---|---|---|---|---|
In-domain (lumaflux-hdrtv1k) |
33.34 | 0.9427 | 0.9941 | 14.69 | 7.962 |
Zero-shot (lumaflux-main) |
27.52 | 0.9307 | 0.9837 | 29.86 | 7.618 |
Results
The comparisons that matter
LumaFlux is best on every Luma-Eval metric. On the 100-frame synthetic track, it reaches 24.23 PU21-PSNR, 0.8294 PU21-SSIM, 56.54 dE-ITP, 0.269 HDR-LPIPS, 5.812 HDR-VDP-3, and 0.631 FR-HIDROVQA. Against the strongest baseline, the margin is +0.49 dB and -5.06 dE-ITP. FR-HIDROVQA falls from 0.703 for HDRTVNet++ to 0.631.
Like-for-like generative comparison
The native-track comparison isolates generative methods. Relative to X2HDR, LumaFlux is better by 4.9 dB, costs about 6 times less per output pixel, uses fewer solver steps, and trains fewer parameters.
| Method | Steps | PU21-PSNR | dE-ITP | s/megapixel | Trainable params |
|---|---|---|---|---|---|
| LEDiff | 50 | 10.99 | 198.58 | 17.0 | 860 M |
| X2HDR | 30 | 18.65 | 92.20 | 16.2 | 149 M |
| LumaFlux | 8 | 23.56 | 47.59 | 2.5 | 71 M |
Compression robustness
Across the HDRTV1K fixed-QP sweep from 27 to 42, the mixed-corpus LumaFlux model loses 0.55 dB. HDRTVNet++ loses 2.78 dB and FMNet loses 2.71 dB across the same sweep.
Cost and limits
Diffusion quality has a runtime cost
On a single GH200, LumaFlux uses 8 steps, takes 5.28 seconds per 1080p frame, and reaches 27.06 GB peak memory. Feed-forward CNN converters take 0.28 to 0.86 seconds per 1080p frame, so LumaFlux is slower even though it is cheaper per output pixel than other diffusion inverse tone-mapping methods.
The current limitations are concrete:
- The frozen VAE is not an HDR codec. The tone head calibrates its output, but it cannot invent detail absent from heavily clipped SDR.
- Temporal stabilization is inference-time only. There is no explicit long-range motion model.
- Qualitative scenes show slight oversaturation: mean ITP chroma is 60.65 for LumaFlux and 49.80 for the references.
- Inference is slower than feed-forward CNN converters, at 5.28 seconds compared with 0.28 to 0.86 seconds per 1080p frame.
- HDRTV4K ground truth could not be obtained through an open channel, so that benchmark is absent.
- A controlled subjective study remains future work.
Try it
Code and adapter weights are available
The code is released under Apache-2.0. The Hugging Face repository provides two 285 MB adapter checkpoints, each with 71.3M parameters:
lumaflux-main.safetensorsis the mixed UGC and PGC checkpoint trained for 100k steps. It is the default for video and Luma-Eval.lumaflux-hdrtv1k.safetensorsis trained for 50k steps on the HDRTV1K training split and is intended for in-domain HDRTV1K comparisons.
Get into the code repository with two commands:
git clone https://github.com/shreshthsaini/LumaFlux
cd LumaFluxThe checkpoints contain adapters only. The frozen FLUX.1-dev backbone and SigLIP are downloaded separately. FLUX.1-dev is gated on Hugging Face, so users must separately accept its license. Because the adapters require FLUX.1-dev, the weights use the FLUX.1-dev Non-Commercial License and are for research use only.