01698nas a2200169 4500008004100000245007400041210006900115260001300184520108200197100002101279700002401300700001701324700003001341700002601371700002001397856011101417 2026 eng d00aNeurosymbolic Hidden Neuron Analysis in Convolutional Neural Networks0 aNeurosymbolic Hidden Neuron Analysis in Convolutional Neural Net bElsevier3 a

This tutorial introduces a step-by-step, deductive pipeline for making the inner workings of neural networks more transparent by assigning human-understandable concepts to hidden neuron activations. The approach automatically maps neuron behavior to symbolic concepts drawn from structured knowledge sources and attaches an error margin to each label, providing a measure of confidence in its precision. While demonstrated in detail on the ADE20k scene dataset---including single-concept neurons, multiple neurons contributing to the same concept, and multi-concept neurons---the method is also applied to the SUN2012 dataset and adapted for a text classification task, highlighting its generalizability across modalities. The chapter is designed to be practical and educational, focusing on a replicable methodology that readers can adapt to varied applications. Through worked examples, visualizations, and evaluation strategies, the tutorial offers a clear, reusable framework for concept-based neuron analysis in both vision and language models.

1 aDalal, Abhilekha1 aSarma, Moumita, Sen1 aDas, Avishek1 aAkkamahadevi, Samatha, E.1 aVasserman, Eugene, Y.1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/neurosymbolic-hidden-neuron-analysis-convolutional-neural-networks01457nas a2200145 4500008004100000245005500041210005400096260001300150520097800163100001801141700001701159700002401176700002001200856009101220 2026 eng d00aReasoning in Large Language Models: RAG and Beyond0 aReasoning in Large Language Models RAG and Beyond bElsevier3 a

This chapter presents a comprehensive overview of contemporary approaches that integrate neural networks and large language models (LLMs) with classical symbolic reasoning. We review the evolution of methods designed to embed logical inference within neural architectures and explore recent advances in prompting strategies, hybrid reasoning frameworks, retrieval-augmented learning, and reinforcement-based reasoning optimization methods. We discuss the symbolic foundations of logical reasoning and then analyze how neural and LLM-based methods have progressively evolved to emulate, extend, and optimize symbolic reasoning across diverse tasks. Finally, we explore emerging neurosymbolic paradigms that unify neural and symbolic reasoning to achieve interpretable, scalable, and generalizable intelligence. Our analysis underscores the growing importance of neurosymbolic AI as a foundational direction for developing reliable and explainable reasoning systems.

1 aBarua, Adrita1 aDas, Avishek1 aSarma, Moumita, Sen1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/reasoning-large-language-models-rag-and-beyond00681nas a2200193 4500008004100000245007000041210006900111490000600180100002100186700002000207700001800227700003400245700001700279700001700296700002500313700002400338700002000362856010500382 2026 eng d00aToward a Neurosymbolic Understanding of Hidden Neuron Activations0 aToward a Neurosymbolic Understanding of Hidden Neuron Activation0 v21 aDalal, Abhilekha1 aRayan, Rushrukh1 aBarua, Adrita1 aAkkamahadevi, Samatha, Ereshi1 aDas, Avishek1 aWidmer, Cara1 aVasserman, Eugene, Y1 aSarker, Kamruzzaman1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/toward-neurosymbolic-understanding-hidden-neuron-activations01891nas a2200193 4500008004100000245005900041210005900100260002200159300001400181520124700195653003001442653002501472653001901497653002701516100001701543700002101560700002001581856009601601 2025 eng d00aHidden Neuron Activation Analysis on Labeled Text Data0 aHidden Neuron Activation Analysis on Labeled Text Data aUSAbACMc12/2025 a206 - 2103 a

Understanding the internal mechanisms of deep neural networks remains a central challenge in the field of Explainable Artificial Intelligence (XAI). With the rapid advancement of neural architectures in natural language processing (NLP), analyzing the role of hidden neurons in capturing and processing linguistic features has become increasingly important. This study investigates Hidden Neuron Activation Analysis on labeled text data to reveal how individual neurons contribute to a model’s decision-making process. We propose a model-agnostic explainability framework for text classifiers that identifies concepts activating specific neurons involved in classification. An LSTM-based network is trained on the AG News topic classification dataset, comprising four distinct classes, and the final Dense layer with 64 neurons was analyzed. In addition, statistical analyses like the Mann-Whitney U Test is conducted to assess the robustness and reliability of the system. The statistical analysis shows that, concepts plays important role in the decision making process of neural network. Our findings enhance interpretability in NLP models and offer a foundation for optimizing neural architectures in text classification tasks.

10aConcept-based Explanation10aDense Layer Analysis10aExplainable AI10aHidden Neuron Analysis1 aDas, Avishek1 aDalal, Abhilekha1 aHitzler, Pascal uhttps://daselab.cs.ksu.edu/publications/hidden-neuron-activation-analysis-labeled-text-data