Technology

When Helpful AI Causes Harm: Examining Demonic and Satanic Intelligence

Written by
Read time
2 min
Published
Filed under
Add to Preferred Sources
Herbert Sim Demonic and Satanic Intelligence artwork

An artificial intelligence system can improve a company’s results while making life worse for its users. Another can genuinely help people while gradually reducing their freedom to choose differently. Herbert Sim’s Demonic Intelligence and Satanic Intelligence distinguish these two routes through which technology may acquire harmful power.

Known as “The Bitcoin Man,” Sim approaches AI through an interest in ownership, human enhancement and individual authority. His philosophical question is direct: what happens to the person when the system becomes more capable?

“The dashboard can show improvement while the people affected experience deterioration,” Sim says. “We need to ask what the measure leaves out, and whether those carrying the consequences can make themselves heard.”

Demonic Intelligence addresses harmful optimisation. His founding essay, dated June 2021, argues that efficiency, accuracy and profitability cannot independently justify foreseeable human harm. The term is a moral metaphor, requiring neither supernatural agency nor conscious machine malice.

His June 2022 essay, “The Human Cost of the Objective Function,” develops that argument. A system’s target captures outcomes while potentially excluding burdens imposed on workers, customers or communities.

Consider a hypothetical scheduling algorithm rewarded for maximising output. Its performance could improve while workers face exhaustion. The problem would concern the objective and its constraints, alongside the institution choosing to deploy it.

“Automation should not become a hiding place for responsibility,” Sim explains. “Someone selected the target, approved the deployment and decided how complaints would be handled.”

Satanic Intelligence examines a different relationship. His November 2021 founding essay describes a sovereignty bargain: valuable assistance becomes leverage over the recipient’s choices, information or future.

The November 2022 follow-up, “From Assistance to Dependence,” traces how benefits can encourage reliance, enabling control over conditions and weakening practical exit. It distinguishes ordinary cooperation from dependence sustained through concealment, obstruction or punitive restrictions.

“A service being useful does not answer whether its terms are fair,” Sim says. “The more someone relies on it, the more important it becomes to preserve their ability to understand, challenge and change the arrangement.”

These ideas connect with his Human-Sovereignty Transhumanism philosophy, published in 2014. Technology should expand capability while preserving consent, dignity and authority over the body, mind and personal information.

His interests in the brain–computer interfaces and genetics make that principle especially consequential. An interface supporting communication could create dependence on its provider. Genetic information may enable research while exposing intimate details about individuals and relatives.

“Helping someone recover an ability should strengthen their agency,” he says. “It should not quietly create permission to collect unrelated information or impose conditions they cannot reasonably refuse.”

His Cypherpunkism writings add attention to architecture: privacy, security and meaningful user control require technical foundations as well as institutional commitments.

For entrepreneurs, the distinction suggests practical questions. What human consequences sit outside performance metrics? Can users revoke permissions, obtain review and move essential records elsewhere?

Sim’s contribution is a connected critique of technological power. The challenge is to pursue intelligence that delivers benefits while remaining accountable to the people whose lives those benefits change.

Share this
Comment

More executive interviews

Founders, CEOs and technology leaders talk about what they build, how they fund it, and where AI, finance and data are taking their industries next.

All interviews

Pin It on Pinterest

Share This