Suggested time: 10 minutes
Transparency can play a critical role in avoiding the significant risks and harms associated with algorithmic decision systems. These risks include performance risks like algorithmic errors, security risks, control risks like rogue outcomes and unintended consequences, economic risks, ethical risks, and societal risks like unfair outcomes for underprivileged or marginalized communities. For companies, this often means avoiding significant costs and fines.
Transparency may improve your algorithmic systems. Transparency can increase trust among all the stakeholders of an system, which may lead to better outcomes for that system.
Implementing transparency will soon likely be law. Governments around the world have begun to draft and enact legislation regulating the transparency of algorithmic systems, including two major proposals in both the EU and US.
Advancements in AI have made algorithms considerably more complex. These complex, nuanced algorithms that are not easily understood by human users are often referred to as black-box algorithms (because one cannot see inside).
Black-box is a blanket term used to describe any system in which a human cannot see the inner workings of the algorithm. Algorithms commonly designated as black-boxes are nueral networks (used for deep learning), gradient boosted-models, and forest-based models.
Black-box are said to be opaque (i.e. the opposite of transparent). But notably, in the past decade, researchers and practitioners have made significant developments in “opening-up” black-box systems. There are many powerful, free, and easy-to-use tools that can be used to gain insight into how underlying black-box algorithms work. The most popular tool is called SHAP, or SHapley Additive exPlanations (Lundberg and Lee 2017). SHAP is very simple to use and can often be understood by laypersons with just a little training.
Since algorithmic transparency is all about the human understanding of an algorithm, it’s important to identify who it is for. Algorithmic transparency can be important for many people both internally to and externally of an organization. There are five main groups of people impacted by algorithms (Bell, Nov, Stoyanovich 2023):
| Stakeholder | Definition |
|---|---|
| Practitioners | The technical practitioners that are developing, implementing, and maintaining algorithmic systems. They include data scientists, engineers, programmers, developers, and analysts. |
| Managers | The individuals at many different levels in an organization that oversee algorithmic decision-making tools. They include project managers, business developers, and executives. |
| Affected persons | The people who are impacted by the algorithm. For example, if an algorithm is being used to assess job applicants, the job applicants are the affected persons. |
| Humans-in-the-loop | The individuals who are responsible for using the algorithm. Humans-in-the-loop may also be called algorithm managers or users. |
| Compliance officers | Persons who oversee the legal compliance of algorithms, and may include auditors and policymakers. |
Exercise: Recall the examples of algorithmic transparency we have discussed thus far in this course. Can you identify some of the relevant transparency stakeholders for those examples?
There are a diverse set of goals or reasons that a stakeholder may want transparency for an algorithmic system. Below we group goals into six different categories:
| Goal | Definition | Example |
|---|---|---|
| Validity | Making sure the system is constructed correctly, debugging a system | The programmers, engineers, and managers may use transparency to ensure the system is valid and correct |
| Trust | Knowing “how often the system is right” | A policymaker or auditor may use transparency to gain trust in the ADS |
| Learning and Support | Increasing general understanding about how an algorithm reaches a decision | A doctor may use transparency to better understand an algorithms predicted diagnosis of a patient |
| Recourse | Allowing affected persons to take action against a decision | An individual may use transparency about an algorithm to appeal a loan rejection |
| Fairness | Ensuring that an algorithm is making decision biased against a minority group | An auditor may use transparency to make sure that an algorithm is biased |
| Privacy | Ensuring that an algorithm respects the data privacy of individuals | An auditor may use transparency to make sure that an algorithm is violating data privacy laws |
One critical goal for transparency is the idea of algorithmic recourse (sometimes called redress), which is the ability of a person affected by the outcome of an algorithm to see why that decision was made and what they can do to change that outcome. For example, if an algorithm is used to determine whether or not an individual is accepted or rejected for a loan, that individual should be able to see why that decision was made so they can take actions to change the decision in the future (ex. improve credit score).