NightHaze: Nighttime Image Dehazing via Self-Prior Learning

📄 arXiv: 2403.07408v2 📥 PDF

作者: Beibei Lin, Yeying Jin, Wending Yan, Wei Ye, Yuan Yuan, Robby T. Tan

分类: cs.CV

发布日期: 2024-03-12 (更新: 2024-12-23)

备注: Accepted by AAAI 2025. Project page: https://bb12346.github.io/NightHaze/

💡 一句话要点

提出NightHaze以解决夜间图像去雾问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 夜间图像去雾 自先验学习 数据增强 深度学习 图像恢复 半监督学习 计算机视觉

📋 核心要点

  1. 现有夜间图像去雾方法在处理光效和噪声时表现不佳，导致图像细节丢失和伪影产生。
  2. 本文提出了一种基于自先验学习的去雾方法，通过严重的数据增强来训练模型，使其能够学习到鲁棒的图像先验。
  3. 实验结果显示，NightHaze在MUSIQ和ClipIQA指标上分别提升了15.5%和23.5%，实现了最先进的性能。

📝 摘要（中文）

本文提出了一种新的夜间图像去雾方法NightHaze，基于自先验学习框架，利用严重的数据增强策略来提高模型的鲁棒性。通过将清晰图像与光效和噪声混合进行训练，模型能够学习到清晰背景的先验知识。尽管在某些情况下仍存在过度抑制的伪影，本文通过半监督的教师-学生框架提出了自我精炼模块，以进一步改善图像质量。实验结果表明，NightHaze在夜间图像去雾任务中表现出色，显著超越了现有方法。

🔬 方法详解

问题定义：本文旨在解决夜间图像去雾中的光效和噪声对图像质量的影响，现有方法在处理这些因素时常常导致图像细节丢失和伪影。

核心思路：通过引入严重的数据增强策略，模型在训练过程中将清晰图像与光效和噪声混合，从而学习到清晰背景的先验知识。与传统的掩码自编码器不同，本文利用夜间图像的特性进行增强，提升了模型的鲁棒性。

技术框架：NightHaze的整体架构包括数据增强模块、自先验学习模块和自我精炼模块。数据增强模块负责生成混合图像，自先验学习模块通过恢复清晰图像来学习先验，而自我精炼模块则用于进一步优化输出结果。

关键创新：最重要的创新在于利用光效和噪声作为数据增强的核心因素，使模型能够在训练中学习到更强的图像先验。这种方法与传统的掩码技术有本质区别，能够更好地适应夜间图像的特性。

关键设计：在训练过程中，模型通过调整噪声值来接近光效混合图像的像素强度，从而实现严重的增强。此外，损失函数设计考虑了图像的清晰度和细节恢复，以确保输出质量。网络结构采用了适应性强的深度学习架构，以支持复杂的图像恢复任务。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，NightHaze在夜间图像去雾任务中表现优异，MUSIQ和ClipIQA指标分别提升了15.5%和23.5%。与现有方法相比，NightHaze在图像清晰度和细节恢复方面实现了显著的性能提升，展示了其在实际应用中的潜力。

🎯 应用场景

NightHaze的研究成果在夜间监控、自动驾驶和夜间摄影等领域具有广泛的应用潜力。通过提高夜间图像的可视性，该方法能够帮助改善安全性和用户体验，未来可能在智能交通和安防系统中发挥重要作用。

📄 摘要（原文）

Masked autoencoder (MAE) shows that severe augmentation during training produces robust representations for high-level tasks. This paper brings the MAE-like framework to nighttime image enhancement, demonstrating that severe augmentation during training produces strong network priors that are resilient to real-world night haze degradations. We propose a novel nighttime image dehazing method with self-prior learning. Our main novelty lies in the design of severe augmentation, which allows our model to learn robust priors. Unlike MAE that uses masking, we leverage two key challenging factors of nighttime images as augmentation: light effects and noise. During training, we intentionally degrade clear images by blending them with light effects as well as by adding noise, and subsequently restore the clear images. This enables our model to learn clear background priors. By increasing the noise values to approach as high as the pixel intensity values of the glow and light effect blended images, our augmentation becomes severe, resulting in stronger priors. While our self-prior learning is considerably effective in suppressing glow and revealing details of background scenes, in some cases, there are still some undesired artifacts that remain, particularly in the forms of over-suppression. To address these artifacts, we propose a self-refinement module based on the semi-supervised teacher-student framework. Our NightHaze, especially our MAE-like self-prior learning, shows that models trained with severe augmentation effectively improve the visibility of input haze images, approaching the clarity of clear nighttime images. Extensive experiments demonstrate that our NightHaze achieves state-of-the-art performance, outperforming existing nighttime image dehazing methods by a substantial margin of 15.5% for MUSIQ and 23.5% for ClipIQA.