Consistent Kernel Change-Point Detection under m-Dependence for Text Segmentation
2510.03437v1
cs.LG, cs.CL, stat.ML
2025-10-08
Авторы:
Jairo Diaz-Rodriguez, Mumin Jia
Abstract
Kernel change-point detection (KCPD) has become a widely used tool for
identifying structural changes in complex data. While existing theory
establishes consistency under independence assumptions, real-world sequential
data such as text exhibits strong dependencies. We establish new guarantees for
KCPD under $m$-dependent data: specifically, we prove consistency in the number
of detected change points and weak consistency in their locations under mild
additional assumptions. We perform an LLM-based simulation that generates
synthetic $m$-dependent text to validate the asymptotics. To complement these
results, we present the first comprehensive empirical study of KCPD for text
segmentation with modern embeddings. Across diverse text datasets, KCPD with
text embeddings outperforms baselines in standard text segmentation metrics. We
demonstrate through a case study on Taylor Swift's tweets that KCPD not only
provides strong theoretical and simulated reliability but also practical
effectiveness for text segmentation tasks.
Ссылки и действия
Дополнительные ресурсы: