Suggested time: 10 minutes
Researchers and practitioners have made significant advancements in developing tools for better algorithmic transparency. In this course, we survey five of the most popular categories — however, there are many tools that exist beyond what we cover here (and many more that will be developed in the coming years).
Transparency labels and model cards for algorithmic tools are similiar to nutritional labels found in the food industry. Nutritional labels are designed to be transparent and understandable, and their contents are perceived as a highly credible source of information by consumers, who use them to guide decision making.
There are several examples of transparency labels for algorithms that have been designed by researchers, and like nutritional labels, they often contain the “ingredients” that make up an algorithm. For example, the labels may include descriptive information about the factors considered by the algorithm (the ingredients), how they are ranked in terms of their importance in the decision-making (ingredient ranking), and attributes related to fairness, which could be useful for meeting stakeholder goals related to validity, trust, privacy, and fairness.
When is this useful? To provide a global, high-altitude view of the algorithmic system for aspects like (1) what data is being used by the system, (2) how the system weighs that data, (3) metrics on the performance or fairness of the system overall.
Data visualizations can be used to show information about the data used in to create an algorithmic decision system, or facts about the system itself, like how many decisions an algorithm makes per day, and how many people are affected by those decisions.
Visualizations have proved useful for informing users and making complex information more accessible and digestible, and have even been found to have a powerful persuasive effect. Visualizations are often an advisable tool for transparency as they can easily convey lots of information in a simple manner, and organizations commonly staff analysts who specialize in visualizations.
It’s also important that visualizations are designed thoughtfully, as they have the ability to be abused and can successfully misrepresent a message through techniques like exaggeration or understatement.
When is this useful? Useful for presenting complex information in a digestable way, particularly for non-technical users. This could include both internal and external stakeholders, like humans-in-the-loop for the former and affected individauls for the latter.
Some (but not all) algorithms have built in intrinsic transparency mechanisms (also called intrinsic explainability mechanisms) that simply need to be surfaced to offer transparency into how they work.
For example, two common algorithm types are decision trees and rules-lists. For the former it is possible to print out and display the tree diagram for the user. For the latter, one can list out all the rules used to make a decision for. Another type of commonly used algorithm are linear models, which can produce formulas that explain their decision-making. These formulas are sometimes very easy to understand.
Unfortunately, many highly sophisticated algorithms like random forests and neural networks do not have intrinsic transparency mechanisms.
When is this useful? To answer the question ``how does the system work, to the extent that given a new input to the algorithm, I could anticipate the output with a high degree of accuracy?’’ Generally for providing a deeper understanding of how the underlying algorithm in the system functions.
The attribute importance (also called feature importance or factor importance) of an algorithm is a list that shows all the different attributes (sometimes called features or factors) that are considered by an algorithm, and their relative weights. It offers global transparency for an algorithm.
For example, consider an algorithm that makes predictions on whether or ** an individual should receive a loan. The attribute importance could be made up of three attributes: an individual’s income, their credit history, and their education level. The weights for these attributes in the algorithm’s decision-making may be 40% income, 40% credit history, and 20% education level.
There are three benefits of attribute importance:
When is this useful? Provides a global understanding of how an algorithmic system is processing data at a slightly deeper level than what is often found in transparency labels. Useful for learning and support and to some extent recourse. Useful to practitioners for checking the validity of an algorithmic system.
The attribute influence (also called feature influence or SHAP factors)) of an algorithm is similar to the attribute importance, except that it shows how the attributes of a single instance or individual impacted the algorithm’s output. In contrast to attribute importance, the influence shows the local transparency for a particular case. Local transparency refers to understanding how an algorithmic system makes a decision about a single case or instance, and global transparency refers to understanding how an system works overall (a bird’s-eye view).
For example, consider again an algorithm that makes predictions on whether or not an individual should receive a loan. If an individual is rejected for a loan, the attribute influence could tell them your high income and education level were influencing the loan decision from the algorithm positively, but ultimately your low credit score caused the algorithm to reject your loan application.
Like with attribute importance, the attribute influence can be created for any algorithm.
When is this useful? To provide local transparency about a single instance, generally an affected individual. The attribute influence is one of the best ways to answer the question, ``why did the algorithmic system have this output for \emph{this specific person}?’’ Extremely useful for recourse for affected individuals. Very useful for learning and support for humans-in-the-loop.