Suggested time: 15 minutes
The following case study is an example of where algorithmic transparency can go wrong. Again, this case study is based on types of ADS that exist in the real-world, but is a fictional example.
This game is meant to be played in groups of 3 or more, and each group needs 7 materials: 1 copy of the Algorithmic Transparency Checklist, 1 scenario card that describes an algorithmic system being used in the domain, and 5 role cards that represent the different stakeholders of the system. The scenario and role cards can be printed out, or taken directly from this page. Role cards can be distributed at randomly, but at least one player should have the role Vice President of HR at The Important Company.
At the beginning of the game, the role cards will be randomly distributed to each of the participants. Note that the role cards offer suggestions on how the character could be role-played, but these are just guidelines!
The objective of the game is to complete the Algorithmic Transparency Checklist and make sure that all the needs of every role are met.
Scenario Card: Algorithmic Hiring
This scenario takes place at The Important Company, a tech company that prides themselves on doing important technology work! Recently, a programmer at the company developed the first version of an algorithmic tool for processing online resumes called The Ultimate Hiring Tool. The tool is fully operational and will soon be deployed.
The Ultimate Hiring Tool works in two easy steps. First, a human user in HR at The Important Company will define a set of personality characteristics about the candidates they are interested in hiring (ex. motivated, altruistic, works well in teams). Second, whenever a job applicant applies through The Important Company online portal, the resume is run through the tool and a score about how well that candidate matches the personality characteristics is produced. After that, the output is reviewed by the human user.
Role Card: Vice President of HR at The Important Company
It is your job to lead the discussion for going through the Algorithmic Transparency Checklist. To start the game, begin with Step 1 of the Algorithmic Transparency Checklist.
Character notes: you may be excited about the opportunity to use this new tool, but could also be worried about issues like privacy, and protecting the IP of your company.
Role Card: Human user in HR
You are the human-in-the-loop! It is your job to use the tool to try and find candidates that are strong fits for the company.
Characeter notes: you may be a bit distrustful of technology. After all, how much can you really tell about a person’s personality from their resume? Does anything replace a few good-old fashioned job interviews?
Role Card: Job Applicant
You are an individual affected by The Ultimate Hiring Tool. You are a job applicant who will visit the website of The Important Company and submit their resume.
Character notes: You may be a bit of an activist when it comes to technology. You don’t necessarily believe that all tech is good tech (maybe your favorite movie is Terminator) and might be uncomfortable with how pervasive technology seems to be in every aspect of your life (I swear I told my friend about wanting to visit Aruba for the first time yesterday and today Instagram is giving me flight ads!)
Role Card: Compliance Officer
You are in charge of making sure that all technology is up to spec. The Important Company is headquartered in NYC, which just recently passed a law called Local Law 144 of 2021 governing algorithmic tools in hiring. The specific text of the law reads, “NYC Local Law 144 requires employers which are using automated employment decision tools (AEDT) for employment or promotion decisions within NYC to ensure every such tool that substantially assists or replaces a discretionary decision undergoes an annual bias audit by an independent auditor. The scope of the audit shall include but not be limited to the testing of an automated employment decision tool to assess the tool’s disparate impact on persons with regards to sex, race, and ethnicity.
Character notes: You may be a by-the-book, duty-above-all else regulator (after all, the law is the law).
Programmer Behind The Ultimate Hiring Tool
You are the practitioner behind the tool! You are responsible for building and maintaining all aspects related to the tool.
Charcter notes: you may be new to The Important Company and still have some new-hire self-doubt. In fact, it may be the case that you casually started working on the tool because you thought it was cool, but never expected it to be fully adopted by HR. What happens if it doesn’t work right? How can I know?