Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events
2510.03833v1
eess.IV, cs.CV, cs.MM
2025-10-08
Авторы:
Shuoyan Wei, Feng Li, Shengeng Tang, Runmin Cong, Yao Zhao, Meng Wang, Huihui Bai
Abstract
Continuous space-time video super-resolution (C-STVSR) has garnered
increasing interest for its capability to reconstruct high-resolution and
high-frame-rate videos at arbitrary spatial and temporal scales. However,
prevailing methods often generalize poorly, producing unsatisfactory results
when applied to out-of-distribution (OOD) scales. To overcome this limitation,
we present EvEnhancer, a novel approach that marries the unique properties of
high temporal resolution and high dynamic range encapsulated in event streams
to achieve robust and generalizable C-STVSR. Our approach incorporates
event-adapted synthesis that capitalizes on the spatiotemporal correlations
between frames and events to capture long-term motion trajectories, enabling
adaptive interpolation and fusion across space and time. This is then coupled
with a local implicit video transformer that integrates local implicit video
neural function with cross-scale spatiotemporal attention to learn continuous
video representations and generate plausible videos at arbitrary resolutions
and frame rates. We further develop EvEnhancerPlus, which builds a controllable
switching mechanism that dynamically determines the reconstruction difficulty
for each spatiotemporal pixel based on local event statistics. This allows the
model to adaptively route reconstruction along the most suitable pathways at a
fine-grained pixel level, substantially reducing computational overhead while
maintaining excellent performance. Furthermore, we devise a cross-derivative
training strategy that stabilizes the convergence of such a multi-pathway
framework through staged cross-optimization. Extensive experiments demonstrate
that our method achieves state-of-the-art performance on both synthetic and
real-world datasets, while maintaining superior generalizability at OOD scales.
The code is available at https://github.com/W-Shuoyan/EvEnhancerPlus.
Ссылки и действия
Дополнительные ресурсы: