利用带有缺陷的图谱进行神经滤波的盲视频除闪技术

许多视频都包含闪烁伪影。闪烁的常见原因包括视频处理算法、视频生成算法以及在特定情况下捕捉视频。以往的研究通常需要特定的指导，如闪烁频率、手动注释或额外的一致性视频来消除闪烁。在这项工作中，我们提出了一个通用的闪烁去除框架，它仅接收单个闪烁视频作为输入而不需要额外的指导。由于它对特定的闪烁类型或指导是盲目的，因此我们将其命名为“盲目的消闪”。我们方法的核心是利用神经地图以及神经过滤策略。神经地图是视频中所有帧的统一表示，它提供了时间上的一致性指导，但在许多情况下存在缺陷。为此，我们训练了一个神经网络来模仿过滤器，学习一致的特征（如颜色和亮度），避免在地图中引入伪影。为了验证我们的方法，我们构建了一个包含各种真实世界闪烁视频的数据集。广泛的实验证明，我们的方法实现了令人满意的去除闪烁的表现，甚至在公共基准测试中超越了使用额外指导的基准模型。
Many videos contain flickering artifacts. Common causes of flicker include
video processing algorithms, video generation algorithms, and capturing videos
under specific situations. Prior work usually requires specific guidance such
as the flickering frequency, manual annotations, or extra consistent videos to
remove the flicker. In this work, we propose a general flicker removal
framework that only receives a single flickering video as input without
additional guidance. Since it is blind to a specific flickering type or
guidance, we name this “blind deflickering.” The core of our approach is
utilizing the neural atlas in cooperation with a neural filtering strategy. The
neural atlas is a unified representation for all frames in a video that
provides temporal consistency guidance but is flawed in many cases. To this
end, a neural network is trained to mimic a filter to learn the consistent
features (e.g., color, brightness) and avoid introducing the artifacts in the
atlas. To validate our method, we construct a dataset that contains diverse
real-world flickering videos. Extensive experiments show that our method
achieves satisfying deflickering performance and even outperforms baselines
that use extra guidance on a public benchmark.
论文链接：http://arxiv.org/pdf/2303.08120v1