Aligning Neural Networks to Human Semantics