By: Dr. Jonathan W. Williams
In the age of algorithmic decision-making, data models shape the way people experience the world in ways that are both seen and unseen. These systems are not neutral. They are designed. And the designers of these systems, data modelers and scientists, wield enormous, and often invisible, power.
In the concluding pages of Cathy O’Neil’s Weapons of Math Destruction (2016), she suggests that the responsibility of data modeling regulation starts with the people (i.e. the modelers) who build the algorithms: “Like doctors, data scientists should pledge a Hippocratic Oath, one that focuses on the possible misuses and misinterpretations of their models” (pg. 173). O’Neil argues that one’s own values, morals, and self-regulation can influence the decisions that are ultimately made and embedded in models, yet those same values, morals, and self-regulation abilities are pre-pressure tested with the on-ground work of model building. Model builders are faced with building models that are more predictive, create stickier patterns for users, circulate information in corporate-approved patterns, and ultimately extract profit and efficiency at scale. These data modelers hold incredible power, and it’s no longer permissible for their individual level ethical codes to guide their workplace behavior.
As modeling has developed as a job, it has also developed as a profession. The ‘professions’ exist as a function of curated expertise. That curation of expertise creates information asymmetry between those who are in the profession and those who are outside of the profession (Susskind & Susskind, 2022). The asymmetry of specialized knowledge comes along with trust from society; society trusts professionals with this novel information so much so that we grant monopolies to those professionals to practice with their knowledge. The asymmetric information thus creates asymmetric power. Not just anyone can profess to be a doctor or lawyer, or architect or engineer; one must train diligently to enter the profession to be awarded the power of information asymmetry. In today’s algorithmically driven world, data modelers and data scientists hold incredible asymmetric power via proprietary models and opaque decision systems. Yet, there is no pathway into this profession. There is no required licensure, no required oath of practice, and no shared conduct obligations in the same way that other professions have developed. With such asymmetric power, perhaps data modelers and scientists should have similar obligations.
As a society, we are building to an inevitable moment where trust is eroded for modelers in their work. Modelers have not had such a reckoning yet, but cultural infrastructure to circumvent that reckoning can start not as regulation, but with ritual. O’Neil provides text for an oath of practice for data scientists by Derman and Wilmott, and others have offered such oaths as well. Medical students commonly recite or acknowledge institution-specific declarations or oaths of practice when they graduate. Data modelers and practitioners can, and should, do the same. It creates rituals for these cohorts of students. As students enter into the domain of professional practice, their entry can reinforce shared responsibility of the profession. This oath at graduation would mark the starting point of professional identity.
Professional associations reinforce and sustain that identity through codes and conditions of membership. For professional organizations like the ACM (Association for Computing Machinery), INFORMS (Institute for Operations Research and the Management Sciences), or ASA (American Statistical Association), ascribing to shared standards of practice is a part of their eligibility requirements or is a function of their memberships' working priorities. The ACM, INFORMS, and ASA have all implemented various ways to address this issue. The ACM offers a Code of Ethics with reporting and enforcement procedures for violations. INFORMS runs the Certified Analytics Professional Program that establishes a code when the certification is conferred. ASA offers Ethical Guidelines for Statistical Practice. These are meaningful steps that signal a ritual and commitment to ethical standards. While these practices are important, they do stop short of holding practitioners accountable; there are no teeth in these types of practices to stop bad actors. A data modeler's right to practice is not suspended if a violation is made.
It is time for the data modeling ‘job’ to graduate to a full profession. It should be aligned with other high-stakes, high-power, imbalanced professions. There should be a governing body that is empowered to investigate misconduct, impose meaningful sanctions, and withdraw the right to practice. The time for this professional evolution is now; voluntary codes or pledges are only stop-gap measures that exist at an interstitial moment for this group of practitioners.
References
O’Neil, C. (2016). Weapons of Math Destruction: How big data increases inequality and threatens democracy. Penguin Books.
Susskind, R., & Susskind, D. (2022). The Future of the Professions How Technology Will Transform the Work of Human Experts, Updated Edition. Oxford University Press USA - OSO.
