ABSTRACT
In this thesis, we explore a new framework for image classification with an emphasis on generating explainable prediction. Deep neural networks (DNN) have achieved unprecedented accuracy in image classification. However, DNNs are black-box classifiers notoriously hard to interpret. In some application areas, the lack of interpretation has prevented practitioners to embrace the machine learning system. On the other hand, easy to interpret classification methods, e.g., linear discriminant analysis or distance-based approaches, often fall much behind in accuracy. We hereby propose a method to learn the definition of a distance based on commonly used distances for different types of features. The new distance is subject to a so-called positive gradient constraint to ensure interpretability. This new method enables us to interpret the importance of different types of features with respect to particular image class or even individual images. In addition, the method provides insight into why a prediction decision is made. Comparisons have been made with DNN and other widely used classification algorithms. We find that the new approach is competitive in performance when the dataset is of small size.
CHAPTER 1 INTRODUCTION
1.1 MOTIVATION
In this thesis, we explore a new framework for explainable image classification. The motivation behind this is to provide insights into how a prediction decision is made on the features of images. Deep learning (DL) methods have achieved remarkable performance in terms of feature representation learning and image classification. Their black-box nature, however, often makes the representation and decision hard to explain and interpret. It arises from the fact that although they build on top of statistical principles, their complex architecture and large volume of parameters reduced the capability to explicitly represent the knowledge of a given task. People have concerns over how it makes a decision and which feature it uses to reach a decision. The application of deep learning has largely been affected by its lack of interpretation, including the fields like finance [1] and healthcare [2] . Some simpler classification methods such as linear discriminant analysis or decision tree are self-explanatory because their decision boundaries can be visualized in a low dimensional space of their model parameters. Distance or similarity based approaches such as kNN or nearest neighbor algorithm [3] are another class of easy to interpret classification methods. These methods however suffer from inferior performance because they lack the complexity and large amount of parameters as in neural networks that are needed in the classification of high dimensional and complex datasets.
We propose to learn a distance metric which is based on commonly used distances for different types of features. The distance is a weighted combination of individual distances of views. A view is a feature representation of an image, which can be either the feature map of a neural network or classical features such as scatter wavelets. We constrain the weights to be positive (non-negative) and we call the method positive gradient distance (PGD). Each training image will get its own set of weights on its views such that it selectively compares the views with a test image and decides if it is similar or not. By analyzing weights on and distances under individual views, we are able to identify which views are contributing to the distance between two images and how a decision is made on top of the distances. More details can be found in chapter 2.
We would like to introduce two closely related concepts in the next two sections: instance specific distance and attribution based interpretability. Since we calculate a set of non-negative weights for each training image, its distance to other images will be largely controlled by its weights on the views. It coincides with the idea of instance specific distance where each instance has its own manner of measuring distance with other instances. Attributions or heatmaps are a kind of saliency maps that highlight contribution of individual portions of an image to the correct prediction. Though the heatmap looks like some feature maps, the heatmap and the features extracted in this work are of different meaning.
It is described below to avoid any confusions.
1.2 INSTANCE SPECIFIC DISTANCE
Instance specific distance (ISD) is a way to compute distance that captures the characteristics of specific instance when it is compared with other instances. This is desired in certain scenarios such as image query or image search. A fishing net may look similar to images with other kinds of nets from the perspective of texture feature, whereas a sky is close to images with large portion of blue color when color feature is emphasized. Instead of defining an uniform distance metric, we can define ISD such that each instance compares with other instances from its own perspective.
Frome et al. [4] proposed to learn a distance function for a specific image which is a combination of individual distances under local patch-based visual features. The distance function is optimized with constraints applied on pairwise distances to enforce that images in the same class are closer to concerned image than others in a different class. They define a weight vector on the set of local patches and setup the optimization algorithm based on triplets of images: the specific imageIF , a similar image IS in the same class and a dissmilar image ID from a different class. The optimization is then carried out to yield the set of weights with minimum non-negative values which satisfy the constraint D(IF ,IS) < D(IF ,ID), where D(∗) is a weighted summation of patch-based distances. Frome et al. [5] further improved this idea by introducing consistency among the set of distance functions. Zhanet al. [6] proposed metric propagation for ISD that propagates distance metrics from labeled images to unlabeled images with a predefined weight graph corresponding to pairwise distances, so that unlabeled data can also learn their instance specific distances.
The idea of learning instance specific weights as mentioned above is very similar to our work in the thesis where we learn instance specific weights over individual views. The difference is that previous work spread weights over local features of an image where the type of features over local patches should be the same to ensure comparability. Our method however computes different types of features on the entire image and assigns weights to views. The view itself is a complete representation of the image. In fact, the work of learning weights for local features can be integrated into our framework to form a twolayer architecture: the bottom layer learns weights over local patches for each view and the top layer learns weights over the views.
1.3 ATTRIBUTION BASED INTERPRETABILITY
Researchers have explored various methods to explain what a model looks at while making decisions in classification tasks. Depending on the nature of how they approach it, there are several taxonomies to differentiate them [7] . These methods are commonly referred to as explainable artificial intelligence (XAI). If the method is performed before, inside or after a ML/DL model, then it can be viewed as pre-model, in-model or post-model XAI methods. Principle component analysis (PCA) [8] , for example, is a common pre-model method because features are processed before running classification models. Post-model method on the other hand tries to understand what has been learned by a model after it is constructed and trained. In this thesis we focus on in-model XAI methods that incorporate interpretability into the model themselves.
Figure 1.1: Illustration of attributions or heatmaps of VGG-16 (Figure 3 of [9] ).
Attribution or heatmap based method is one of the major XAI methods that assigns attribution values or importance or relevance scores to input features of a neural network. The goal is to unveil the contribution of certain features to the output of a neuron and determine which features lead to correct classification. Note that the features here refer to locality information of an input image rather than a view of features studied in this thesis. Figure 1.1 shows the attributions or heatmaps generated by several XAI methods for the images from Imagenet [10] , which is excerpted from [9] . The attributions are computed as a result of altering inputs and performing backpropagation from the output neuron of VGG-16 [11] . The first attribution column with the title ”Gradient” is the gradients of the output neuron with respect to the input. PatternNet estimates the input signal using an objective function and Pattern Attribution applies Deep Taylor decomposition by searching the rootpoints in the signal direction for each neuron [12] . Deep Taylor [13] regards a neuron as a function on input features and performs Taylor decomposition recursively through layer to layer via backpropagation. The resulted heatmaps from Pattern Attribution, Deep Taylor, or LRP [14] in the figure indicates the contribution of features to the activation of the target neuron, with positive activation marked in red and negative activation marked in blue.
While the attribution serves as an important tool to interpret what a model looks at when it makes prediction, the method is very different from what we introduce in the thesis. The attribution or heatmap is generated from performing backpropagation on a DNN and highlights the places that contribute to the decision making. It is more like a saliency map suggesting salient locations of a visual scene. Our method builds on top of multiple views of features and assigns weights to views. Depending on how a feature is extracted, the feature may look similar to a heatmap, but with different meanings. Our task is to compute weights for the views rather than to generate heatmap-like features.
EXPLAINABLE IMAGE CLASSIFICATION BASED ON POSITIVE GRADIENT DISTANCE