tldr; We can patch many of the issues with DCT that pose direct ethical problems, but not the deeper misalignment between the technology and the goals we are using it to pursue.
Recently, there has been a huge amount of enthusiasm for adopting digital contact tracing (DCT), like the TraceTogether app used in Singapore, as part of a test and trace strategy to help relax physical distancing regulations and reopen economies that have been shut down to suppress the spread of COVID-19. Discussion in the AI and technology ethics community has largely divided into what can be described as compatibilist and incompatibilist positions on the possibility of implementing DCT ethically. I think the incompatibilists are right, but that there are better ways of stating the position. The primary incompatibility isn't between DCT and specific ethical challenges familiar to AI ethics, such as protecting privacy. Rather, it's a deeper tension between the limits of the technological tool and the requirements DCT would have to meet to function usefully as part of a test and trace strategy.
DCT is being developed rapidly, and we will be highly motivated to make it work - billions of dollars will change hands, much of it from public coffers to private corporations, and lives really do hang in the balance. Moreover, as we learn more about the populations most likely to experience the more severe health and economic outcomes from COVID-19, it is apparent that costs and benefits will not be shared equally.
The AI and technology ethics community has played a prominent role in the development and discussion of the DCT approach, and numerous proposals have been carefully developed in collaborative efforts between technologists and ethicists. Increasingly sophisticated measures to implement contact tracing anonymously such as Decentralized Privacy-Preserving Proximity Tracing have earned the approval of some ethicists, such as Cansu Canca, the Executive Director of AI Ethics Labs, who recently argued that there should be mandatory adoption of these systems, in an approach she calls MPP-DCT (Mandatory Privacy Preserving Digital Contact Tracing).
Can DCT be deployed ethically?
Ethicists have been very effective bringing concerns about surveillance and privacy into the mainstream of the work on digital contact tracing, and views amongst AI and tech ethicists have largely divided into two camps. There's an incompatibilist outlook, which argues that there is no way to implement digital contact tracing that is compatible with privacy rights, and a compatibilist view, that by using technical and regulatory tools we can in fact deploy DCT in an ethical way.
The compatibilists have received significant and much deserved airtime within organizations developing DCT apps, and in the broader policy discussion. This is a welcome acceptance of the essential role that ethical oversight must play in the technology development cycle as we slowly absorb the litany of research showing extensive problems with AI systems with hidden biases, unpredictable behaviour, and unanticipated social and personal and economic costs.
It's not surprising that the incompatibilists receive comparatively less attention and exert less influence. Not only are some of the richest corporations on earth involved in the digital contact tracing project, but they are portrayed as an essential part of plans that could allow governments to relax public health initiatives that have substantially impeded the economic and personal freedom of millions of people.
Discussion of the incompatibilist position on digital contact tracing seems to focus on a particular kind of "slippery slope" argument. If we allow them to track us to cope with the COVID-19 challenge, what will stop them from expanding this in other contexts? Technologists and ethicists have argued for years that the "If you have nothing to hide, you've got nothing to fear" attitude is problematic, and that privacy tradeoffs tend to end up a lot worse than the initial bargain reveals. This has fallen on deaf ears, in part because the problem with these tradeoffs is that there is a deep information asymmetry. We think we know what we are giving up, and it tends to be pretty innocuous, some photos we post, some basic personal info, some anonymized digital fingerprints. What could go wrong? But what we don't know, and sometimes can't know, is how they will ultimately become part of systems that cause harm, because those systems might not yet exist, or the conditions that make them harmful are not yet in place.
Slippery slope arguments are vivid, appealing, and easy to devise. But they tend to be speculative and vague, and invite compelling rebuttals. Countering a slippery slope argument is easy - for any imagined bad landing, one might show that there are regulatory, economic, or technical protections that can be put in place, and then the dialectic tends to head towards a back and forth between potential pitfalls and possible solutions. As this goes on, the ground can shift, towards a higher level objection to the slippery slope, that we have to be realistic, we have to deal with the problems we are facing now, and that the argument is ultimately paranoid, contrarian, or unreasonable.
The compatibilists have excellent answers to many of the slippery slope arguments against DCT. Proposals such as MPP-DCT show that tracking apps can be highly anonymous. The system proposed by Google and Apple also implements technical measures that protect individual privacy, and further innovations will doubtlessly improve these. Emergency public health legislation in effect in most countries with COVID-19 outbreaks have strict limitations that we should, in most jurisdictions, trust to function as intended. Crytopgraphic techniques to create ephemeral device fingerprints can prevent future misuse.
First Wave and Second Wave AI and Technology Ethics
Frank Pasquale, an expert on the law of artificial intelligence, algorithms, and machine learning, recently argued that we can distinguish between what he calls "First Wave" and "Second Wave" algorithmic accountability. First Wave research and policy focussed on fixing the immediate emergencies created by our reliance on systems which don't do what we expect, or don't do it fairly. Second Wave AI ethics asks deeper questions about how we use these systems in the first place - whether a proposed technology can be aligned with the values and norms that matter within the domain in question, whether it should be used at all, rather than whether or not we are doing it fairly. We can fix a lot of First Wave problems for a system which is still fundamentally, morally and pragmatically misaligned. For instance, we might focus on addressing First Wave problems with AI video hiring systems by working to ensure that the data they are trained with is unbiased and used with consent, implementing technical measures to generate interpretable models, and ensuring compliance with labour and data protection regulations. With all of these First Wave problems fixed, we might still find that there is a fundamental mismatch between what the system can measure, and what actually matters when choosing a person for the job. It might just be a technological solution that can't align with our interests in the problem.
The AI ethics debate about digital contact tracing has been overwhelmingly focused on First Wave problems that concern implementation challenges, such as protecting privacy, addressing data protection and ownership issues, and avoiding off-label reuse of the data collected. What are the Second Wave questions we should be asking?
Instead of a slippery slope, where we worry about what might come after digital contact tracing, we should be worried about a trojan horse - what comes along with it that we haven't seen? If we incur significant sunk costs to rapidly launch DCT, what other investments will we have to make to fix it at sea, and what policies, technologies, and behaviours will we need to accept?
It is important to note the economic and epidemiological viability of the test and trace strategy remains in question. For example, researchers at Johns Hopkins estimate that this approach would require 3.6 billion dollars and 100,000 dedicated employees dedicated to maintain the “human-in-the-loop” capacity that is essential to making DCT signals actionable (indeed, TraceTogether was human-led, and the app itself merely augmented conventional contact tracing). Nobel-Laureate Paul Romer estimates that DCT would require almost 22 million tests per day in the United States, a country with limited access to testing and highly variable access to health care resources. As of April 13, the United States has conducted less than 3 million tests in total.
There is uncertainty about the effectiveness of diagnostic and antibody tests, and the significant gaps in our understanding of the conditions of transmission. The basic reproduction number of COVID-19 is thought to be R3, which means that for every person that is infected, there will in turn be three more infected people. We contact a lot more than three people in the timespan where we would potentially transmit the virus. We know the virus can persist on surfaces and be transmitted through indirect contact. Unlike other infectious diseases where contact tracing has been used effectively, COVID-19 is contagious long before symptoms appear, enlarging the pool of contacts that must be searched. Digital contact tracing works by using low energy bluetooth signals from mobile phones, which can reach up to 20 feet, through walls, outside vehicles, and which are an imprecise proxy for contact that could lead to infection. Conventional contact tracing, in contrast, is a process conducted by experts with access to a broad spectrum of data about an individual's activities.
We can look at the broad range of attack patterns familiar in the cybersecurity domain and find analogical threats to DCT, such as the denial of service and wardriving. A digital fingerprint that has been flagged as positive could be deliberately or accidentally associated with many other devices. (For a taste of some of the vulnerabilities we can expect, see https://eprint.iacr.org/2020/399) Cybersecurity has historically been attentive to human factors as well as technical ones, and DCT is vulnerable to social engineering attacks and the tendency of users to quickly discover the dark logics of the systems they are forced to interact with. It won't be long before users learn to manipulate quirks in the system, for instance, it might be beneficial to have been infected so as to be flagged as such in the app. There is now doubt that antibody tests are reliable, and worries that the virus can re-activate after a period of dormancy.
The reliance of DCT on non-causal predictors will drive iteration of the technology that will demand either the investment significant investment in human oversight, or the aggregation of the baseline proximity signals with other data corpora. This will raise substantial ethical and safety issues.
Ethical analysis of technology should anticipate the trajectory that development of the system will take after deployment. Identifying bugs and failure modes that will require remediation within the sociotechnical paradigm of the system is a way to form hypothesis about the direction this development will take.
Even if DCT has flaws, isn’t it good enough to try?
If we launch DCT, these are problems that we will have to debug, and those solutions may have many of the features the incompatibilists are worried about. Will tech platforms need to tie the anonymous bluetooth data to their broader data holdings in order to narrow down contacts that are more likely to correlate with transmission? Will the apps need to have access to other health data in order to ensure that self-reported data is accurate, or make AI assisted medical inferences about users from other proxy signals they collect? Many of our contacts, such as meeting a delivery driver, or pumping gasoline, or running, typically happen without our mobile phones. Perhaps we'll find that we are obligated, morally, legally, or practically, to make sure we are emitting and collecting data at all times.
As the limits of the actual diagnostic capacity are revealed, the apps might still have pragmatic value as a sort of security theatre, acting as a throttle on access to public spaces even when we know the mechanism isn't particularly reliable. At best it will act as quarantine-roulette, at worse it will exhibit biases that reproduce or amplify existing inequities.
This isn't a slippery slope argument, it's an argument about the implication we are committed to when we adopt the logic of this technological perspective on the problem. When you build a piece of software, there's a blurred line between fixing bugs and adding features. If we want to automate the analysis of proxy signals to determine who is eligible for relaxed physical distancing requirements, using the technologies and platforms we have today, these are some of the implications that come along with it.
Digital contact tracing might be a useful source of aggregate data, for macro-scale epidemiological modelling, but there are good reasons to doubt that we can use them as a reliable way of making decisions about individuals. This is not an uncommon problem with the use of big data and artificial intelligence, where we gain the capacity to generate knowledge that we often lack the resources to act on in an ethical or effective way. And yet, if we build the system, we acquire ethical and sometimes legal obligations to take responsible action with the results. It's not clear that we can with DCT, and the solutions available to try to remediate that are problematic.
One might object that this overlooks the positive contribution even imperfect DCT can make. Even though we know that non-medical masks aren't reliable at preventing transmission, we endorse their use because they still help. The difference is of course the downstream commitment - with masks there are few, but with DCT they are substantial. Masks have a measurable impact on transmission, and the commitments that go with them depend on regulatory decisions about mandate and enforcement. It is true that in some places masks have been mandatory where as access to them is poor, and handing out masks would be a better exercise of government power than writing fines. But on the whole, there aren’t a lot of troubling implicated commitments with policies that encourage or mandate mask usage. For DCT, on the other hand, they are significant commitments which will be technically and politically costly to solve, and which pose substantial ethical and practical challenges. Even if one believes that efficacy has presumptive justificatory force, the degree of efficacy would need to be correspondingly high to support the full social and economic costs we would incur to implement even a plausibly effective DCT test and trace strategy.
The problems with DCT are deeper than the sort of privacy issues current proposals focus on solving.
AI and tech ethicists should embrace these Second Wave questions about the appropriate use of technology. Often First Wave issues are more tractable, and easier to solve within the institutional mandates technology ethicists operate with as part of collaborative efforts with diverse stakeholders, often spanning private and public interests. Second Wave perspectives are more likely to cast doubt fundamental assumptions about the viability of projects as a whole and the likelihood of success. We risk ethics-washing when we focus too closely on patching issues on the edges of systems that have deeper tensions with our goals and values. There was controversy this winter when the GermEval 2020 machine learning contest included a task to develop AI that makes inferences about the “intellectual ability” of the authors of text samples. Some ethicists warned that even if we solved ethical problems at the periphery, there are fundamental ethical issues baked into problem definition itself, that should cause us to object to building software for the problem at all. Sometimes we can fix the bugs, but deeper problems with the problem definition or the limits of our tools mean we still can’t perform the task ethically. Second Wave algorithmic accountability challenges us to pay attention to these cases.
The implementation of DCT will require a significant exercise of political will and economic investment that could perhaps be deployed more effectively in other ways with more empirically demonstrable likelihood of having positive impacts, such as improving access to health care services, tackling known social determinants of poor outcomes, addressing the economic costs of the distancing, and putting into action the well-established practices and protocols that will help respond to future pandemics. Of course, it’s a poor argument to claim that we can’t fix these things and also implement DCT, but in practice, political and economic capital for public health initiatives is a precious and limited resource, and we should be cautious about how we spend it. “It won’t work” isn’t just a practical problem when the measures we’ll need to take to make it work have serious ethical challenges, and even more so when some of the bugs carry serious moral risks.
We might still be optimistic that AI and big data could be part of the solutions to these problems, even if we are skeptical about the value of implementing DCT.
This point bears repeating. In the United States evidence is mounting that the heaviest impacts of COVID-19 will be on the most vulnerable populations. "What the vulnerable portion of society looks like varies from country to country, but in America, that vulnerability is highly intersected with race and poverty.". "The rate of COVID-19 infection is eight times higher on the Navajo Nation and the death rate is 16 times higher than the rest of New Mexico."
In Canada, where I live, we've had a heartbreaking series of news stories about terrible conditions in senior care homes with extreme levels of COVID-19 infection, highly limited capacity for treatment and care, and terrible outcomes. I recently had a chance to hear first hand from a RN at senior care facility how dire this situation is, in a part of our society that is in many ways hidden.
In a thoughtful essay on managing scarce health care resources, Dr. Hannah C. McLane reminds us that saving the most lives involves value-laden judgements. Using tools like DCT to accelerate efforts to relax measures that are currently slowing the rate of infection will amplify existing inequities in our allocation of public goods. You might think that's a distal problem, an unfortunate reality, and one that shouldn't weigh too heavily against broadly utilitarian calculations that are inescapable in public health ethics. But we should pay close attention to the extent to which the technological paradigms with which we approach problems constrain these choices, tilting the scale so that making decisions that save the most lives with the least cost just happen to favour one kind of life, and risk another.