As early as the 18th century, humans have been playing a game of “don’t look behind the curtain” with fake automation. In 1770, inventor Wolfgang von Kempelen unveiled the Mechanical Turk, a machine billed as a clockwork-powered robot that could play chess against human opponents. In reality, the Mechanical Turk hid a person inside a concealed compartment within the machine. The operator could see the chessboard and use a system of levers and pulleys to move the robot’s arm, making it seem as if the machine was thinking for itself.
As artificial intelligence cements itself as the backbone of the tech industry, the real work is still getting quietly outsourced to people in the background.
Amazon borrowed the name for its own crowdsourcing platform, Amazon Mechanical Turk, which it describes as “a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually.” The platform doesn’t pretend to be automated; instead, it’s a nod to the original trick, openly relying on human labor to handle tasks that computers can’t yet do.
In 2014, a New York startup called X.ai set out to build an AI assistant that could schedule meetings over email, acting like a real human assistant. Users would copy the assistant, named “Amy” or “Andrew” on email threads and it would handle the back-and-forth of setting up appointments, sending calendar invites and managing replies. The company raised $44 million chasing this goal. But while some automation existed, almost every email was actually processed by human operators. According to industry estimates, building a truly human-level AI scheduling assistant without hidden workers would have cost $30 million to more than $100 million in research and development, not including scaling, compliance and infrastructure.
Nate, an AI shopping app, went even further. Launched in 2019 and marketed as a cutting-edge fintech solution, it promised a one-tap shopping experience that could automatically fill out checkout forms and complete transactions. Behind the curtain, however, was cheap human labor in the Philippines, where call center wages average about $3 to $5 an hour for contractor roles. According to an April Justice Department indictment, the automation rate was “zero percent.” Albert Saniger, the founder and former CEO, could face up to 40 years in prison. “Saniger allegedly abused the integrity associated with his former position as the CEO to perpetuate a scheme filled with smoke and mirrors,” the Justice Department said.
Perhaps the most high-profile flop in recent years belongs to Amazon. The company’s Just Walk Out system, hyped as a fully AI-powered cashierless checkout, quickly came under fire after a source revealed that “700 out of every 1,000 transactions needed to be verified by workers in India.” In a blog post, Dilip Kumar, vice president of AWS Apps, insisted the service was in line with industry standards and rejected the idea human reviewers were monitoring shoppers live.
“The erroneous reports that Just Walk Out technology relies on human reviewers watching from afar is untrue,” Kumar wrote. “Most AI systems, including the underlying machine learning models behind these technologies, are continuously improved by annotating synthetic and real shopping data. Our associates are responsible for this labeling and annotation step. Associates don’t watch live video of shoppers to generate receipts — that’s taken care of automatically by the computer vision algorithms.”
In plain terms, Indian workers are not watching shoppers live to see what they are buying, but instead label and tag images or video after the fact to help the AI learn to recognize objects. Kumar’s defense focuses on customer satisfaction and revenue gains while sidestepping the report that Just Walk Out required the support of about 1,000 manual workers in India to function.
For context, AWS software engineers in India can earn up to 2.5 million rupees, or about $30,000 a year, compared with $167,000 to $251,000 for the same role in the United States.
Recently, Builder.ai, a London-based startup once valued at $1 billion, filed for bankruptcy after coming under fire for fraudulent claims. The vision was simple, if ambitious: Consumers could plug in the features they wanted in their app and AI would do the rest. For investors, the truth was far from the narrative. Most of the labor came from developers in India, with little actual automation in use. Estimates put development costs at about $1 million per year for all coders combined, compared with tens of millions of dollars for real AI research and development. After reports of exaggerated revenue and a $37 million loan default, the company did not survive.
Faking AI by hiding human labor has real consequences for everyday people. It means more low-wage jobs are quietly sent overseas, often with few protections or benefits for workers, while consumers are misled into thinking they’re interacting with cutting-edge technology. This kind of outsourcing keeps wages low in developing countries, widens global inequality and allows companies to avoid responsibility for the working conditions behind proverbial smoke and mirrors.
Progress should mean technology that makes life better for workers and consumers alike, not just for investors.
Featured image by Tumisu from Pixabay
Edited by James Sutton
