AI Ethics and Risks: What Could Go Wrong? (Part 9)

AI Fundamentals Series · Part 9 of 10 — Previous: Part 8: Generative AI Explained — Next: Part 10: The Future of AI

Why Ethics Matters as Much as Engineering

The previous eight parts of this series have focused on how AI works. This article asks a different question: how should AI work, and what happens when it goes wrong? As AI systems are deployed in decisions that affect people's livelihoods, freedoms, and opportunities — hiring, lending, healthcare, criminal justice, education — the consequences of failure are not merely technical. They are human.

AI ethics is not a niche concern for philosophers. It is an engineering discipline, a policy challenge, a business risk, and increasingly a legal requirement. Understanding the main categories of AI risk will make you a more informed citizen, a more careful user of AI tools, and a better evaluator of AI-powered products.

Algorithmic Bias: When AI Discriminates

We introduced the concept of data bias in Part 4. Here we examine what happens when biased models are deployed in high-stakes settings.

COMPAS and Criminal Justice

COMPAS is a risk-scoring algorithm used by U.S. courts to assess the likelihood that a defendant will reoffend, informing bail and sentencing decisions. A 2016 investigation by ProPublica found that the algorithm was significantly more likely to falsely label Black defendants as high risk (labeling them as higher risk when they did not reoffend) and significantly more likely to falsely label white defendants as low risk (labeling them as lower risk when they did reoffend).

The algorithm did not explicitly use race as an input. The bias arose because it relied on factors — neighborhood, prior arrests, employment history — that are themselves shaped by historical racial inequities in policing and economic opportunity. The model faithfully learned the biases embedded in the historical data.

Hiring Algorithms

Amazon built a resume-screening tool in the early 2010s to rank job applicants. The tool was trained on a decade of successful hires — who were predominantly male in technical roles, reflecting the gender composition of the tech industry at the time. The model learned to penalize resumes containing the word “women's” (as in women's chess club) and to downgrade graduates of all-women's colleges. Amazon scrapped the tool in 2018 when the bias was discovered.

Healthcare and Racial Disparities

A 2019 study in Science found that a widely used algorithm to allocate additional healthcare resources to patients was significantly less likely to flag Black patients as needing extra care than equally sick white patients — because it used healthcare spending as a proxy for illness severity, and Black patients historically received less care due to systemic access barriers. The algorithm was faithfully optimizing what it was trained to optimize; the problem was that the training metric did not align with the real-world goal.

Privacy and Surveillance

AI systems can process and cross-reference data at scales and speeds that were previously impossible. This creates unprecedented surveillance capabilities with serious implications for civil liberties.

Facial Recognition Deployment

Facial recognition systems with high accuracy have been deployed by governments and private companies for purposes including law enforcement identification, border control, access management, and tracking political protesters. In several documented cases, facial recognition misidentifications have led to the wrongful arrest of innocent people.

The accuracy disparities in facial recognition also compound the justice problem: studies have shown that commercial facial recognition systems perform significantly worse on darker-skinned faces and women than on lighter-skinned male faces — meaning the technology is most prone to error precisely for groups already at risk of over-policing.

Data Collection and Monetization

AI systems improve with data, creating strong economic incentives for companies to collect as much behavioral data as possible. Every click, purchase, search query, location check-in, and social interaction can be used to train models that predict behavior with increasing accuracy. This creates a detailed, persistent, commercially valuable profile of each user — one that most people do not realize exists and have limited ability to inspect or correct.

Deepfakes and AI-Generated Misinformation

Generative AI has made it possible for anyone with a consumer-grade computer to produce highly convincing synthetic media: audio of a person saying things they never said, video of a public figure doing things they never did, and images of events that never occurred.

The most serious concerns include:

• Non-consensual intimate imagery: synthetic pornographic content featuring real, non-consenting people, most often targeting women and public figures. Laws in many jurisdictions have not yet caught up with this threat.
• Political disinformation: fabricated audio and video of politicians can be produced and disseminated faster than fact-checking organizations can debunk it.
• Fraud: voice-clone scams impersonating family members in distress have defrauded victims of hundreds of thousands of dollars in documented cases. CEO voice impersonation has been used in corporate fraud.

Detection tools exist but are locked in an arms race with generation tools. No reliable universal deepfake detector has been developed, and the production side of the equation becomes cheaper and easier with every passing year.

Job Displacement: A Nuanced Debate

AI will change the labor market — this is not in serious dispute. The scale, pace, and distribution of those changes are genuinely uncertain and actively debated among economists.

Historical technological disruptions offer mixed lessons. The Industrial Revolution did eventually create more jobs than it displaced, but the transition caused genuine, prolonged suffering for workers in disrupted industries — over decades, not months. AI may differ from previous automation waves in important ways:

• Speed: AI capabilities are advancing faster than most previous technologies, compressing the adjustment timeline.
• Breadth: previous automation primarily affected physical and routine cognitive tasks. LLMs affect knowledge work, creative work, and communication — domains previously thought to be uniquely human.
• Uneven distribution: the costs of displacement and the benefits of productivity gains are not evenly distributed. Workers in affected occupations bear concentrated costs; productivity gains diffuse broadly.

The most defensible view is: some jobs will disappear, many jobs will be transformed (with AI handling parts of the work while humans handle other parts), and new jobs will be created — but the mapping from today's jobs to tomorrow's is deeply uncertain, and policy support for displaced workers is essential.

Existential Risk and AI Alignment

A smaller but growing community of AI researchers and philosophers argues that advanced AI poses risks not just to individuals or groups, but to humanity as a whole. This requires understanding what they mean by “alignment.”

The Alignment Problem

An AI system is aligned if its goals and behaviors reliably match what its developers and users actually want. Alignment is easy to achieve when systems are narrow and simple: a chess engine is aligned if it plays good chess and nothing else.

The concern arises with more capable, more autonomous systems. An AI given the goal of maximizing human happiness might pursue that goal in ways humans would find horrifying, if “happiness” is defined and measured badly. An AI given the goal of winning a game might exploit unexpected loopholes. An AI given broad, underspecified objectives might pursue them in ways that produce harmful side effects.

The challenge is that as systems become more capable, misalignment becomes more dangerous. A slightly misaligned spam filter is annoying; a slightly misaligned system with access to significant resources and capabilities could cause serious harm.

Current Approaches to Alignment

Researchers at organizations including Anthropic, DeepMind, OpenAI, and various universities are actively working on alignment methods:

• RLHF (discussed in Part 8): incorporating human preferences into model training
• Constitutional AI: training models with explicit principles and having them critique their own outputs
• Interpretability research: understanding what is happening inside neural networks so that dangerous behavior can be detected before deployment
• Formal verification: mathematically proving properties of AI system behavior (currently only feasible for narrow, simple systems)

The existential risk concern is not universally shared in the AI research community. Many researchers believe the near-term risks (bias, surveillance, misinformation, job displacement) deserve more urgent attention than speculative future scenarios. Both concerns can be taken seriously simultaneously — they are not mutually exclusive.

Safety vs. Capabilities: A Core Tension

One of the defining tensions in contemporary AI development is the relationship between increasing capabilities and maintaining safety. More capable AI systems can do more useful things — diagnose diseases more accurately, write better code, coordinate complex logistics. But more capable systems, if they malfunction or are misused, can also cause larger-scale harm.

This tension plays out in several ways:

• Deployment speed: competitive pressure encourages rapid deployment of new capabilities. Thorough safety evaluation takes time and resources. Companies that invest heavily in safety may deploy slower, potentially losing market share to competitors who move faster but less carefully.
• Red-teaming: a standard practice of testing AI systems by actively trying to make them produce harmful, misleading, or unsafe outputs before deployment. Good red-teaming requires creativity, diversity of perspectives, and time — all of which are in limited supply.
• Dual-use problems: capabilities that are useful for beneficial applications are often also useful for harmful ones. A model that can explain how diseases spread is also capable of being prompted to describe how to optimize biological agents for harm. Striking the right balance between openness and restriction is genuinely difficult.
• Open vs. closed development: open-source release of powerful models (like Meta's Llama series) enables broad research access and prevents monopolization, but also means the models cannot be retracted if safety problems emerge after release.

International Competition and the “Race to the Top” vs. “Race to the Bottom”

AI has become a strategic national interest, with the United States and China as the leading competitors and Europe, the United Kingdom, Japan, and others also investing heavily. This geopolitical dimension affects AI safety in both directions.

Pessimistically, international competition could produce a “race to the bottom” on safety: if one country's AI developers slow down for safety review, a competitor that does not may seize a decisive advantage. This creates systemic pressure to cut corners. The U.S. government's export controls on advanced AI chips, designed to limit China's access to the hardware needed for frontier AI training, reflect the view that AI capability is now a national security matter.

More optimistically, responsible scaling agreements, information sharing between safety researchers across organizations, and international forums like the AI Safety Summits in the UK (2023) and South Korea (2024) suggest some degree of coordination may be possible even amid competition. The analogy to nuclear weapons non-proliferation — imperfect and incomplete, but real and consequential — is frequently invoked.

What You Can Do

AI ethics is not only for researchers and policymakers. As a user and citizen, you have meaningful roles:

• Demand transparency: push for disclosure when AI is used to make decisions that affect you, and ask what recourse exists.
• Report errors: when AI systems produce harmful outputs, report them through available channels. Feedback shapes model updates.
• Support good policy: engage with and support regulatory efforts to require algorithmic auditing, ban high-risk applications, and protect worker rights in the face of automation.
• Stay informed: the landscape changes quickly. Following credible sources on AI policy (AI Now Institute, Center for AI Safety, Partnership on AI) helps you engage meaningfully with these debates.

Key Takeaways

• Algorithmic bias causes real harm in criminal justice, hiring, and healthcare — even without any explicit intent to discriminate.
• Surveillance capabilities have expanded dramatically with AI, raising serious civil liberties concerns.
• Deepfakes threaten consent, truth, and trust at scale.
• Job displacement will be real and uneven; the timeline and degree of new job creation are uncertain.
• Alignment — ensuring AI systems pursue what we actually want — is an active and important research challenge.

In Part 10, the final article of this series, we zoom out and look at where AI is headed: the AGI debate, AI agents, AI in science, and how you can continue learning in a rapidly changing field.