Summary:

In this project we used Shapley values, a method from cooperative game theory, to explain the predictions of machine learning models. Shapley values are a model-agnostic way of attributing the contribution of each input feature to the model output, based on the marginal effect of adding or removing that feature from the input set. Shapley values have some desirable properties, such as:

• Efficiency: The sum of the Shapley values for all features equals the model output.
• Symmetry: If two features have the same impact on the model output, they have the same Shapley value.
• Dummy: If a feature has no impact on the model output, its Shapley value is zero.
• Additivity: The Shapley value of a feature for a complex model is the sum of the Shapley values for that feature for simpler models.

We used shap Python package to compute and visualize Shapley values for a Convolution Neural Network that performs binary classification. For each example, using the Shapley value we can understand which feature or pixel contributed to classifying it to a particular class. The issues of explainability for AI is critical as it ensures that the model is fair and does not have any biases towards any group of data. It is even more important when the model is being used for decision makings that affect well being of one or more persons.