01637nas a2200157   4500008004100000245007000041210006900111260002700180520106700207100001401274700002701288700001701315700002001332700002001352856010701372       2017                        eng d00aRelating Input Concepts to Convolutional Neural Network Decisions0 aRelating Input Concepts to Convolutional Neural Network Decision  aCA, USAbNIPSc12/20173 a<p>Many current methods to interpret convolutional neural networks (CNNs) use visualization techniques and words to highlight concepts of the input seemingly relevant to a CNN’s decision. The methods hypothesize that the recognition of these concepts are instrumental in the decision a CNN reaches, but the nature of this relationship has not been well explored. To address this gap, this paper examines the quality of a concept’s recognition by a CNN and the degree to which the recognitions are associated with CNN decisions. The study considers a CNN trained for scene recognition over the ADE20k dataset. It uses a novel approach to find and score the strength of minimally distributed representations of input concepts (defined by objects in scene images) across late stage feature maps. Subsequent analysis finds evidence that concept recognition impacts decision making. Strong recognition of concepts frequently-occurring in few scenes are indicative of correct decisions, but recognizing concepts common to many scenes may mislead the network.</p>
1 aXie, Ning1 aSarker, Md Kamruzzaman1 aDoran, Derek1 aHitzler, Pascal1 aRaymer, Michael  uhttps://daselab.cs.ksu.edu/publications/relating-input-concepts-convolutional-neural-network-decisions