People assigned to review Copilot prompts are being shown sexually explicit images and potentially illegal content, according to a report by 404 Media. The report cites Microsoft documents, threads with messages from contractors, and guidelines from a company called Prolific that hires the contractors assigned to review prompts given to Copilot.
Copilot prompts and image uploads can be reviewed by human contractors to improve the tool.
The fact that humans review Copilot conversations should not come as a surprise. Many companies rely on people to assess and audit AI responses. There are, however, some key differences between the recently uncovered situation and other reviews of AI responses.
The people who are looking at these queries are not being asked to flag inappropriate content or help reduce sexual conversations with Copilot. Instead, the contractors are being asked if Copilot did a good job responding to queries.
There are several negative consequences that come as a result of this setup. One is the fact that people are being inundated with sexual questions and images, often without warning.
404 Media saw the contents of an internal message board discussing reviews of Copilot prompts. The thread showed that people were asked to ensure a woman's skirt had been made short enough or if a woman's breasts had been adequately enlarged.
She-Hulk images come up a lot, according to the thread seen by 404 Media. Other content includes suggestive images of young girls and making an image of Ariana Grande with anorexia.
"Who is writing these prompts and who is deciding that basically generating porn is what Copilot is now focused on? It’s hard to take things seriously when my focus has to be what model generated the appropriate bust size or which middle aged woman was put in the appropriate sexually suggestive position," said one contractor in the thread.
Microsoft's guidance to contractors is to trust their intuition, according to documents seen by 404 Media. The company asks people to rely on their gut reaction to determine if an image meets the demands of a prompt.
In a vacuum, asking people to assess the realism of AI-generated images is fine. Humans have an uncanny ability to detect when an image feels off. But the contractors are not being asked to determine if an innocent image looks real.
Contractors are sometimes given tasks that turn out to include explicit, unsafe, or potentially illegal images, according to 404 Media. One contractor discussed being sent a collection of upskirt photos. While those types of images can be taken with consent, they often are taken without permission.
Other prompts reviewed by humans include a request to make fetish images of children's cartoon characters.
Microsoft provided a statement to 404 Media:
"Microsoft uses customer data as described in our terms of use, including to improve our products and enforce our code of conduct."
Prolific did not respond to a request for comment made by 404 Media.
404 Media breaks down the situation extensively, so I suggest reading through all of their coverage as well. The piece asks for prompt reviewers to reach out to 404 Media, so we may see more details emerge in the future.
Join us on Reddit at r/WindowsCentral to share your insights and discuss our latest news, reviews, and more.
