Restoring any degraded image efficiently via just one model has become increasingly significant and impactful, especially with the proliferation of mobile devices. Traditional solutions typically involve training dedicated models per degradation, resulting in inefficiency and redundancy. More recent approaches either introduce additional modules to learn visual prompts—significantly increasing model size—or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism, without scaling up the model or relying on large language models. Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them first in a gated manner. To fuse the intrinsic degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed for enhancing spatial-aware local-global interactions and enriching the restoration details from the frequency perspective. Extensive benchmarking in the all-in-one restoration setting confirms AnyIR’s SOTA performance, reducing model complexity by around 82% in parameters and 85% in FLOPs. Our code will be available.
(a) Dense all-in-one restoration methods often inefficiently allocate parameters when handling multiple degradation types.
(b) While recent Mixture-of-Experts (MoE) approaches address this through sparse computation, their rigid routing mechanisms uniformly distribute inputs across experts without considering the natural relationships between degradations.
(c) To overcome these limitations, we introduce Complexity Experts - adaptive processing blocks with size-varying computational units. Our framework dynamically allocates model capacity using a spring-inspired force mechanism that continuously guides routing decisions toward simpler experts when possible, with the force proportional to the complexity of the input degradation. While initially designed for computational efficiency, this approach naturally emerges as a task-discriminative learning framework, assigning degradations to the most suitable experts. This makes it particularly effective for all-in-one restoration methods, where both task-specific processing and cross-degradation knowledge sharing are crucial.
@misc{ren2025anyir,
title={Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation},
author={Bin Ren and Eduard Zamfir and Zongwei Wu and Yawei Li and Yidi Li and Danda Pani Paudel and Ming-Hsuan Yang and Luc Van Gool and Nicu Sebe},
year={2025},
eprint={2503.xxx},
archivePrefix={arXiv},
primaryClass={cs.CV}
}