01891nas a2200193 4500008004100000245005900041210005900100260002200159300001400181520124700195653003001442653002501472653001901497653002701516100001701543700002101560700002001581856009601601 2025 eng d00aHidden Neuron Activation Analysis on Labeled Text Data0 aHidden Neuron Activation Analysis on Labeled Text Data aUSAbACMc12/2025 a206 - 2103 a

Understanding the internal mechanisms of deep neural networks remains a central challenge in the field of Explainable Artificial Intelligence (XAI). With the rapid advancement of neural architectures in natural language processing (NLP), analyzing the role of hidden neurons in capturing and processing linguistic features has become increasingly important. This study investigates Hidden Neuron Activation Analysis on labeled text data to reveal how individual neurons contribute to a model’s decision-making process. We propose a model-agnostic explainability framework for text classifiers that identifies concepts activating specific neurons involved in classification. An LSTM-based network is trained on the AG News topic classification dataset, comprising four distinct classes, and the final Dense layer with 64 neurons was analyzed. In addition, statistical analyses like the Mann-Whitney U Test is conducted to assess the robustness and reliability of the system. The statistical analysis shows that, concepts plays important role in the decision making process of neural network. Our findings enhance interpretability in NLP models and offer a foundation for optimizing neural architectures in text classification tasks.

10aConcept-based Explanation10aDense Layer Analysis10aExplainable AI10aHidden Neuron Analysis1 aDas, Avishek1 aDalal, Abhilekha1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/hidden-neuron-activation-analysis-labeled-text-data01220nas a2200169 4500008004100000245006200041210006100103260001400164520066400178653000800842653002200850653001900872100002100891700002000912700002000932856009800952 2024 eng d00aError-margin Analysis for Hidden Neuron Activation Labels0 aErrormargin Analysis for Hidden Neuron Activation Labels bSpringer 3 a

Understanding how high-level concepts are represented with- in artificial neural networks is a fundamental challenge in the field of arti- ficial intelligence. While existing literature in explainable AI emphasizes the importance of labeling neurons with concepts to understand their functioning, they mostly focus on identifying what stimulus activates a neuron in most cases; this corresponds to the notion of recall in informa- tion retrieval. We argue that this is only the first-part of a two-part job; it is imperative to also investigate neuron responses to other stimuli, i.e., their precision. We call this the neuron label’s error margin.

10aCNN10aConcept Induction10aExplainable AI1 aDalal, Abhilekha1 aRayan, Rushrukh1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/error-margin-analysis-hidden-neuron-activation-labels01357nas a2200217 4500008004100000245009200041210006900133260001400202520061900216653000800835653002200843653001900865653000800884100002100892700002000913700001800933700002600951700002400977700002001001856011801021 2024 eng d00aOn the Value of Labeled Data and Symbolic Methods for Hidden Neuron Activation Analysis0 aValue of Labeled Data and Symbolic Methods for Hidden Neuron Act bSpringer 3 a

We introduce a novel model-agnostic post-hoc Explainable AI method that provides meaningful interpretations for hidden neuron activations in a Convolutional Neural Network. Our approach uses a Wikipedia-derived concept hierarchy with approx. 2 million classes as background knowledge, and deductive reasoning based Concept Induc- tion for explanation generation. Additionally, we explore and compare the capabilities of off-the-shelf pre-trained multimodal-based explainable methods. Our evaluation shows that our neurosymbolic method holds a competitive edge in both quantitative and qualitative aspects.

10aCNN10aConcept Induction10aExplainable AI10aLLM1 aDalal, Abhilekha1 aRayan, Rushrukh1 aBarua, Adrita1 aVasserman, Eugene, Y.1 aSarker, Kamruzzaman1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/value-labeled-data-and-symbolic-methods-hidden-neuron-activation-analysis