Say Goodbye To Your Highly Skilled Job. It’s Now a “Human Intelligence Task.” (Backchannel)

Harry K. sits at his desk in Vancouver, Canada, scanning sepia-tinted swirls, loops and blobs on his computer screen. Every second or so, he jabs at his mouse and adds a fluorescent dot to the image. After a minute, a new image pops up in front of him.

Harry is tagging images of cells removed from breast cancers. It’s a painstaking job but not a difficult one, he says: “It’s like playing Etch A Sketch or a video game where you color in certain dots.”

Harry found the gig on Crowdflower, a crowdworking platform. Usually that cell-tagging task would be the job of pathologists, who typically start their careers with annual salaries of around $200,000 — an hourly wage of about $80. Harry, on the other hand, earns just four cents for annotating a batch of five images, which takes him between two to eight minutes. His hourly wage is about 60 cents.

Granted, Harry can’t perform most of the tasks in a pathologist’s repertoire. But in 2016 — 11 years after the launch of the ur-platform, Amazon Mechanical Turk — crowdworking (sometimes also called crowdsourcing) is eating into increasingly high-skilled jobs. The engineers who are developing this model of labor have a bold ambition to atomize entire careers into micro-tasks that almost anyone, anywhere in the world, can carry out online. They’re banking on the idea that any technology that can make a complex process 100 times cheaper, as in Harry’s case, will spread like wildfire.

Perhaps it’s inevitable that in a few years, software will swallow up these jobs, too. But as the tech conversation has fixated on how artificial intelligence will affect the job market, crowdwork has quietly grown in impact and scale.

The next jobs to receive the crowd treatment? Doctors, managers and teachers.

***

When Amazon revealed Mechanical Turk in 2005, the service became an overnight hit. It was the first online platform to allow businesses to post small jobs (called ‘HITs,’ short for ‘human intelligence tasks’), and it quickly attracted a global pool of under-employed people eager to tackle these jobs for equally small rewards. Workers, or ‘turkers,’ choose which tasks to accept, and how long to work. They might, for example, check websites for opening hours, categorize images, or answer survey questions.

Isaac Nichols, now Chief Product Officer of the platform zCrowd, was part of the team that developed Mechanical Turk. The original intent was to use crowdworkers to clean up the company’s databases, extract information from photos, and complete listings for CDs and MP3s. “We had a huge need for a workforce to do this work, but managing that was complicated in terms of hiring, staffing, and seasonality of the work,” he says.

Amazon realized that if it was facing these kinds of issues, other companies in the digital ecosystem were likely also suffering. “Mechanical Turk was designed from the ground-up to be an external tool,” says Nichols. It wasn’t long before other crowdworking platforms soon popped up. “If you have a task that almost anyone in the world can do, then there’s a seemingly infinite supply of people willing to do it,” says Lukas Biewald, founder of CrowdFlower, the platform Harry uses. If you needed to pitch the concept to a venture capitalist in an elevator, you could say crowdworking is Uber for brains.

More than a decade later, dozens of crowdworking platforms now serve up tiny units of labor to millions of workers around the world. Last year, the JPMorgan Chase Institute looked at the anonymized accounts of 6.3m of its customers, and found that over 265,000 people had received income from online economy platforms. This includes so-called capital platforms like Uber or AirBnB, which encourage people to monetize their possessions. These have made money for an estimated 3 percent of Americans, compared with only around 1 percent of Americans (about 3 million people) for crowdworking.

But crowdworking is growing faster, having increased more than tenfold in the last three years. The money crowdworkers earned grew even more dramatically between 2012 and 2015 — by a factor of 54. Most crowdworkers initially turn to digital labor to supplement low earnings or fill gaps between traditional jobs, but once they start, they tend to do more of it.

Unlike Uber drivers or many TaskRabbit laborers, who usually have to be physically present wherever they are needed, digital workers can earn money from coffee shops or kitchen tables anywhere in the world. And unlike the design, writing and coding professionals you can find on platforms like Upwork or Elance, most crowdworkers need no particular skills or training. Robert Reich, Secretary of Labor in the Clinton administration, calls these workers “fungible, sought only for their reliability and low cost.” Tech workers sometimes use a more visceral term: meatware.

***

The downside of making something 100 times cheaper means that someone — and probably lots of people — are losing money. A few pathologists needing to retrain might not upset you, but how exactly can people like Harry live on 60 cents an hour? In 2010, researchers at New York University calculated the median wage of Mechanical Turk workers at $1.38 an hour. While a few experienced crowdworkers say they earn between $5 and $12 an hour, many requesters continue to pay far below the federal minimum wage of $7.25 (which does not apply to independent contractors).

For example, a recent job post by the University of California, Los Angeles, required workers to complete a 45-minute survey for $1.13, equivalent to $1.50 an hour, with the risk of forfeiting the entire payment should they answer a single question incorrectly. And some payments are actually falling.

“When I started working originally on the legal HITs, it was 25 cents each,” says Serana Winter, a 34-year old Ohioan specializing in processing legal documents. “We now do them for 15 cents a HIT. Four used to get me to a dollar, now it takes seven.”

She estimates that “I can do 8 hours of work and make $30, if I’m lucky.” Winter uses zCrowd, which adds an efficient workflow management system within Mechanical Turk, and she puts in a full eight-hour shift several days a week. She is also working towards a bachelor’s degree in health and wellness, so the flexible hours make it worth it for her.

Isaac Nichols, zCrowd’s founder, admits that pricing is tricky. “What’s difficult is that you’re dealing with averages, but every job is slightly different. We often have workers who get 7, 8 or 9 bucks an hour,” he says. “I’d love to pay more but [there are] some upper bounds in terms of what our customers want to pay.”

To get a better sense of what these figures mean, I decided to give it a try myself. As it turns out, becoming a crowdworker is something you can literally do between breakfast and lunch.

I signed up for the same breast cancer screening work Harry was doing. Joining CrowdFlower takes a few minutes, then I plough through the terms of service. Like all crowdworking platforms, CrowdFlower insists that I relinquish intellectual property rights for work carried out with them, and has me agree that payments can be withheld (or even clawed back) if requesters don’t like my work. Above all, the terms stress that I am not an employee, partner, franchisee or sales representative of CrowdFlower, but rather an independent contractor responsible for all my own tax and benefits.

The work itself is like being thrown cold into a Pathology 101 mid-term. I walk through a couple of training screens intended to give me a bare bones understanding of the task ahead. Essentially, I have to look at an image, decide if there are brown cells in it, and place a dot in the center of each one. The cells vary in shape and size, and I am told to avoid blue-ish cells and sepia blotches.

It sounds like a low-rent smartphone game, but I am even worse at it than I am at Angry Birds. I misidentify background swirls for cells and have trouble distinguishing between blobs that are close together. In some images, the sheer quantity of cells is bewildering. Surely I can’t be expected to click on dozens and dozens for less than a penny an image? I struggle my way through the quiz mode and qualify, barely, for paid work.

Incredibly, this is even harder. The images are murkier than those in the quiz and I find myself clicking randomly on specks and blots, feeling guilty all the while that I am poisoning the well of medical research. It’s a relief when the system kicks me out for failing to catch a few ‘easy’ test questions mixed in with the genuine work. I have worked for about half an hour and earned precisely nothing.

Harry shares my frustration. “I only did a couple of tasks,” he admits. “They were too tedious and detailed for four pennies.” Harry should know. Since a traumatic divorce in 2010 left him with hefty legal and child support bills, he reckons he’s completed over 25,000 crowdworking tasks — on top of a full-time job at a large Canadian packaging company.

“If I had the opportunity to not do my day job and do crowdworking instead, I would,” he tells me. “These are dynamic jobs, even if they’re dynamic about auction websites and whether an Amazon listing is correct. Now I’ve learned about what ear infections look like, and what breast mastectomy cell analysis is about. I’ve got a holistic attitude — I bring it all in.”

Andrew Beck, the professor who created the breast cancer cell-tagging work, says that he set the payments for his tasks based on recommendations from CrowdFlower. “We were able to use feedback from completed jobs to adjust the compensation to a level that we and the crowd were both happy with.”

About a third of workers who took the initial breast cancer quiz failed it. Of those who passed, another third failed during the work itself, and earned nothing. The remaining crowd of 2216 workers tagged over 2.4m cell nuclei in a total of 472 hours. None were actually asked whether or not they were happy with the pay they received.

***

SoI decided to ask them. I wanted to know why anyone would work for such small change, and whether they felt pressured, exploited — or perhaps even empowered as part of this revolution in labor.

Finding a random, representative selection of anonymous crowdworkers was easy: I simply set up my own HIT on Amazon Mechanical Turk. I would give 25 turkers $1.50 each to answer a dozen or so questions about their crowdworking experiences. Having been on the other side of the keyboard, I was keen to avoid some common requester pitfalls. I did not ask anything that would allow me to identify the workers, and I set the pay (I hoped) well above the federal minimum wage. At the end, I asked them whether I had paid fairly.

Amazon records how long turkers take to complete tasks and computes an average wage: my crowd earned the equivalent of $9.42 an hour. All respondents told me that this was acceptable for the work involved. In fact, it was a few cents over the average of what they thought a fair hourly wage for crowdworking should be: $9.23, from a range of replies from $3 to $15.

The workers, 13 men and 12 women from across the US, are aged from their early 20s to their late 60s, and include high school dropouts, graduates and holders of advanced degrees. A few get less than $50 a week from crowdworking, most earn a few hundred dollars, and two said they earn over $500 a week online. Most have completed over 10,000 HITs.

Their reasons for crowdworking are as diverse as their backgrounds. Some are paying off college loans, others are earning pocket money for restaurant meals and vacations. One woman needs work that allows her to care for her severely disabled son. A farmer crowdworks while watching over her livestock. A retiree calls it “a fun and profitable way to kill three or four hours a day.”

One thing they have in common is their concern over pay. Nearly half say that crowdworking is rarely or never fairly paid, and twice as many think that pay is decreasing rather than increasing. “Even requesters who once paid a fair wage drop that wage when they realize that there is someone out there who will keep working on their HITs for pennies,” says one worker.

(I also ran a larger survey, asking 209 turkers how much they actually earned, day in, day out. The average hourly rate they reported was $3.25, with a third earning less than $3. Fewer than one in ten crowdworkers said they earned $7 an hour or more).

Several crowdworkers feel like they are in a race to the bottom, not just with human rivals, but with machines, too. “Some of [my work] could possibly be as easily done automated,” says a 27-year old worker from Florida. “It is really up to the requester if they want a computer to complete it or have it done by a human being. If they find it cheaper to do automated, they will more than likely go that route.”

Perhaps such mechanical work is better handled by machines anyway. Hundreds of thousands of manufacturing jobs in developed countries have been replaced by assembly line robots. Many of the routine image recognition and translation tasks carried out by human turkers are now being targeted by AI researchers. A startup called Mirador, for example, asked turkers to classify 50,000 images as ‘nude’ or ‘not nude’. Once its deep learning software had been trained on that data, the human workers were no longer needed. (Arguably, this is a great use for automation, as software can automatically sift pornographic, violent or disturbing imagery from social media without a human having to suffer it.)

“Algorithms are going to take a piece of the work,” admits Isaac Nichols. “It’s a slow, powerful steady process,” agrees Lukas Biewald. “What we see is that people will move little pieces to automated systems over time.”

Adam Devine, a vice president at WorkFusion, another crowdworking platform, goes further. “There is absolutely no future where a person is reading information and simply keying in data.” For example, one of WorkFusion’s clients processes payment records between banks in different countries. These documents arrive from global banks in emails, spreadsheet files, PDFs and even faxes. Workers then have to transcribe each one perfectly. According to Devine, a single mistake can cost half a billion dollars.

Devine says that by toggling back and forth between a human worker training an algorithm, and an algorithm that makes a mistake and is corrected by a human, WorkFusion can achieve near-perfect accuracy at a fifth of the cost of using people alone. And as machines get smarter, the illegible writing and scrawled numbers that currently need human input will get fewer and fewer. “The workers don’t even realize that they’re working on an interface that’s ultimately going to remove a lot of the work they’re doing,” says Devine.

***

But that doesn’t mean crowdworking is doomed. Praveen Paritosh, a research scientist in human and machine intelligence at Google, estimates that the myriad mundane tasks posted on Mechanical Turk (currently about 600,000 every day) represent just 5 percent of the potential market for crowdworking.

“The very ephemeral, very shallow kind of work on Mechanical Turk is going to generate some economic activity, but is generally not sustainable in the very long term,” he says. “What is sustainable is moving crowdworking further along the computation spectrum, where there is room for more skill, more education, more training.”

Paritosh envisages a virtual circle where workers get higher pay and more interesting tasks, platforms get larger commissions, businesses save more money — and humans stay one step ahead of the steady advance of automation.

Udacity, for example, offers short ‘nanodegrees’ in high-tech subjects and pays crowdworkers around $50 an hour to grade students’ projects. “To me, it was, what number would I want to work for?” says Oliver Cameron, a vice president at Udacity. “We need to pay better than Mechanical Turk and all these other services because we want to attract the best talent. And when we have a ton of submissions, that $50 might become $60 or $75 bucks an hour.”

The graders, most of whom are themselves Udacity graduates, seem fairly content with their lot. Nicholas Davidson is a freelance musician living in Cleveland who grades introductory programming and web developer projects. “I do find it extremely rewarding,” he says. “An on-demand thing might not be something I could do in the long run, I think I would like something more stable and full-time, but it’s definitely something that’s helping me grow.”

At zCrowd, a peer-review system allows crowdworkers with more experience to oversee the work of newcomers. Senior workers can provide individualized feedback and recommend the best workers to level up to become graders themselves. Isaac Nichols is building a system that has, he says, “the crowd managing itself, creating middle managers by putting management best practices on top of a very unstructured environment.”

The system seems to work. Nichols set up a test job where 500 workers were asked to determine whether one MP3 song was derivative of another. He pitted zCrowd’s peer review system against more traditional practices — for example, inserting test questions in actual jobs (to weed out random clickers) or sending the same work to multiple crowdworkers. Peer review delivered the highest quality results.

Serana Winter, who has been grading and moderating others doing legal work at zCrowd (for the same $3 to $7 an hour) for a couple of years, has mixed feelings about it. “I do find it more satisfying but it should also pay more than the work itself,” she says. “It’s harder than the tasks, making sure every little period and comma is in the right place. It feels like a managerial job when we resolve disputes.”

***

Nichols believes that workers will ultimately prefer an environment where they feel part of the larger business and can acquire new skills. “Look at any company structure and there’s a ladder to climb where people get paid more as they move up. There’s a similar structure to be built on top of the crowd,” he says. “I’d love to see the day where someone can have a career in crowdworking: to work whenever they want, wherever they want, and get paid more for their experience.”

Google’s Paritosh is a big fan of this human-centered approach to crowdworking. “You use the technology and the platform to connect and communicate, but you’re essentially building a human organization,” he says.

Ryo Suzuki of the University of Colorado Boulder tried to do just that. He developed a ‘micro-internship’ platform called Atelier that paid experienced programmers to mentor newcomers on a week-long e-commerce coding project. In its first test, heavier use of Atelier was associated with higher quality work, and the interns reported learning new skills.

Workers might then move on to an online vocational education provider like Udacity. Paritosh thinks that integrating online education with online work could ultimately help bootstrap poorer people, and those in developing countries, into higher-paid jobs.

But this techno-utopian vision of the future of labor could be a tough sell to some of today’s crowdworkers. Years of dull work, low pay and a lack of professional recognition have selected for a workforce that often just wants to clock in, earn a few dollars, and get back to their offline lives. “I normally just do it until I’m bored then quit,” admits a 50-year old crowdworker from Kentucky.

A few people in my survey said that they had learned skills like typing, research and coding on the job, but only a quarter said that they actually wanted to be stretched by their work online. Even those that do aspire to more interesting tasks have low expectations about forging a career from them. “I would like more challenging work,” says a 38-year old worker living in Wisconsin, “But they’d probably not pay enough anyway.”

One 28-year old New Yorker with a master’s degree thinks that crowdworking has expanded her knowledge base slightly, “but I wouldn’t say I’ve gained any real world skills.” Perhaps this is the real danger. Increasingly sophisticated systems are making it ever easier to atomize some daily duties of doctors, lawyers, teachers and managers into discrete tasks, but the skills lost at the top do not seem to be trickling down to the crowd.

We are at a turning point for crowdworking. In one scenario, the crowd continues to grow and expand, but the work remains mundane, repetitious and poorly paid. Eventually, much of it gets automated completely.

In another, as crowdworking expands, the incremental jobs require more responsibility, more social interaction, and more genuine intelligence than today’s mechanized chores. Fairly paid work becomes available to almost anyone, anywhere with access to a computer. This in turn entices some of the 40 percent of Americans who are of working age but not currently employed to rejoin the workforce.

“It’s not about creating some new kind of magical computational data center of the workplace,” says Praveen Paritosh. “It’s more about building a platform in the digital world that follows the model of being humane and being a good co-worker and being pleasant to work with. I know that there will problems as we go along. But the bigger problem would be if crowdworking dies before it gets there.”

Crowdworking’s current woes foreshadow a problem that will only become more acute as AI matures. Do we have the political and social will to transform the way we live and work, to embrace the efficiencies of automation without discarding the tremendous creativity and flexibility of human beings? If we can’t, the world of work will be a much poorer — and probably even more poorly paid — place.

This story was produced in partnership with the McGraw Center for Business Journalism at the CUNY Graduate School of Journalism.