Why ChatGPT's ads might be remembered as the most evil product of our era — "a potential for manipulating users in ways we don’t have the tools to understand."

Is OpenAI evil? It sounds like a definitive "maybe," after reading this piece from former researcher Zoë Hitzig.

OpenAI and other AI-adjacent companies have seen a spree of high-profile resignations recently, with ascending levels of alarm over the impacts its products are, or potentially will have on society.

In nature, superior iterations of species tend to eradicate lesser versions of themselves. We've seen research from Anthropic, showing its AI literally choose to murder humans in virtual scenarios rather than allow itself to be shut down. We've seen other LLMs actively opt into deception and other dark patterns to achieve their results, raising the spectre of misalignment scenarios — akin to what happens in The Matrix. "The best way to save humanity is to destroy it," and the like.

Fears about OpenAI and other large language model tech companies run the gamut from Terminator-style sci-fi doomsday scenarios to potentially more grounded fears over economic upheaval. Our entire economy is based on human-led work. What does a post-human economy look like? What does a post-human society look like?

Companies like OpenAI, Google, and Microsoft aren't really waiting around to figure those questions out. Instead, they're planning to let them play out in real time and pick up the pieces later, whether we like it or not.

One thing that could put the brakes on the self-imposed destruction of society is pure economics. Today, OpenAI costs billions of dollars per year to run and brings in a paltry amount of revenue. Investors have become increasingly spooked by the costs associated with AI, and have handed Amazon and Microsoft multi-billion-dollar write-downs on their market capitalization as a result.

Companies like Google and Microsoft are prioritizing enterprise applications and data center efficiency improvements to help offset their AI costs, but OpenAI isn't really in a position to achieve some of this. They don't have the software stack and enterprise relationships that Microsoft does, nor do they have the first-party cloud infrastructure that Microsoft, Google, or Amazon do.

So, the firm is turning to ads.

Surprise, surprise, right? Nothing is free. Facebook, YouTube, Bing, Google ... — if it's free, it's usually powered by ads. But the application of those ads gets increasingly nefarious the deeper you get into it. Based on your interests on Facebook, YouTube, and so on, Meta and Google can serve you granular, laser-targeted ads that can exploit your characteristics. I'm in my late 30s, and I've started getting a lot of ads about hair replacement treatments lately on Instagram, for example.

I'd say today's ad platforms are fairly innocuous, and perhaps irritating at best. Some are worse than others, of course. Exploiting users' fears and desires is commonplace if you use TikTok and Instagram for ads, but a recent article in the New York Times caught my eye about how much darker and dystopian ChatGPT's own ad platform might end up being.

OpenAI has seen a flurry of resignations over the past couple of years, as researchers fear the "not-for-profit" firm has fully lost its way. For one former researcher, Zoë Hitzig, ChatGPT's ad platform was the final straw. In her op-ed, she sounds the alarm over the scale of potential harm OpenAI's ad platform might do to its users, and potentially, society at large.

"I once believed I could help the people building A.I. get ahead of the problems it would create," Hitzig explains. "This week confirmed my slow realization that OpenAI seems to have stopped asking the questions I’d joined to help answer."

Hitzig specifically calls out OpenAI's insertion of ads into the free tiers of its ChatGPT products. She believes that OpenAI is sprinting towards monetization without consideration for the potential harm this could do — and it revolves entirely around just how honest users are with the uncanny chatbot.

"I don’t believe ads are immoral or unethical. A.I. is expensive to run, and ads can be a critical source of revenue. But I have deep reservations about OpenAI’s strategy," Hitzig continues.

"For several years, ChatGPT users have generated an archive of human candor that has no precedent, in part because people believed they were talking to something that had no ulterior agenda. Users are interacting with an adaptive, conversational voice to which they have revealed their most private thoughts. People tell chatbots about their medical fears, their relationship problems, their beliefs about God and the afterlife. Advertising built on that archive creates a potential for manipulating users in ways we don’t have the tools to understand, let alone prevent."

Hitzig is essentially suggesting that because of how people use ChatGPT, OpenAI will eventually afford itself the world's most manipulative ad-delivery mechanism in history. Right now, ads on Instagram are pretty spooky already for their ability to target your interests, but imagine an ad engine that can actively talk you into buying shit you don't need by exploiting your specific psychology. Imagine how youngsters or vulnerable people could be exploited by a high-powered artificial intelligence. Imagine a salesman armed with the entire summation of humanity's research on market psychology, with the turbo-charged greed of a multi-national corporation, and the cold dispassionate amorality of a sociopath.

"OpenAI says it will adhere to principles for running ads on ChatGPT: The ads will be clearly labeled, appear at the bottom of answers, and will not influence responses. I believe the first iteration of ads will probably follow those principles. But I’m worried subsequent iterations won’t, because the company is building an economic engine that creates strong incentives to override its own rules."

I remember the first iterations of ads on Facebook and Google, easy to ignore, appearing in the sidebar, and easily blocked by uBlock or something similar. Compare those to today's high-tech, eerie Instagram or TikTok ads that themselves have become memes for seeming to know about things you want before you even know yourself.

Imagine that turbocharged even further, with an industrial-scale alien intellect distilling your entire psychological profile with the express goal of selling you stuff. Forget the fairy tale claims of "boosted productivity," curing deadly diseases, or becoming an interplanetary species. Envision a generation, our generation, mired in an epidemic of weapons-grade loneliness, with tailor-made AI companions who not only love you, but know exactly what you must buy.

ChatGPT has hundreds of millions of monthly active users, the vast majority of whom are sharing incredibly intimate details about themselves, the likes of which Facebook can only dream of, unless, of course, it ends up admitting its messaging services don't actually have end-to-end encryption. But I digress.

When OpenAI changed ChatGPT's "personality" with its GPT-5 update, people were actively furious because many had come to see the chatbot as a true friend. A confidant ... an external, anthropomorphized entity garnering real trust. The greatest product recommendations come from word of mouth. You know, friends and family. What if the ad itself were your friend?

I can only imagine the cartoonishly evil conversations that have taken place in OpenAI's investor meetings over some of these fundamental marketing concepts. Facebook and YouTube are currently facing a lawsuit in the United Kingdom, accused of actively engineering addictive behavior in youngsters. I think scrolling memes pales in comparison to the harm ChatGPT and other similar products potentially represent on this scale.

Hitzig optimistically hopes OpenAI still has principles, but I think she's sadly naïve. OpenAI CEO Sam Altman has shown himself to be fairly devoid of any sense of social responsibility thus far. It's perhaps mildly alarming at best that many former researchers, like Hitzig, are abandoning ship at an abnormal cadence — while loudly citing "principles" as the primary reason.

Make no mistake. If this dystopic vision of cyborg-driven ad-hypnosis wasn't the plan already, it definitely will be very soon.

This resignation raises real questions about where AI‑powered ads are headed. Do you think ChatGPT‑style advertising crosses a line, or is this just the future of online persuasion?

Share your take in comments— we want to hear how you’re feeling about it!

Join us on Reddit at r/WindowsCentral to share your insights and discuss our latest news, reviews, and more.