Interpretable Intent Detection in High-Cardinality Scenarios via Dynamical Systems Analysis

Resumen

Trustworthy intent detection is limited by deep learning opacity. While dynamical systems theory has emerged as a powerful tool for interpreting Recurrent Neural Networks (RNNs), its application has been unexplored in high-intent, large scale scenarios common to real-world products. We extend this analytical framework to benchmarks with up to 150 intents. We find RNNs trained on these tasks still converge to an interpretable geometric solution, forming robust, intent-specific clusters in their hidden space. We show this space’s intrinsic dimensionality grows sub-linearly with task complexity. Building on this, we introduce Functional Dimensionality (FD), a novel, task-aware metric that quantifies the minimum dimensionality required to preserve this semantic structure. Our analysis reveals FD is remarkably low, suggesting RNNs solve complex tasks via an efficient, highly organized subspace. We show this subspace is structured for inference, with clusters aligning strongly with their corresponding readout vectors. These findings offer a scalable framework for auditing and interpreting high-intent dialogue systems.