AI – Avoiding the ‘Turing Trap’

N.B. An edited transcript of their discussion follows.

Erik Brynjolfsson: That’s a great question. Let me start with what’s similar, because going back over the decades we see some recurring patterns. When I first started working on this, economist Bob Solow asked me to look at the productivity paradox of some amazing technologies in the ’80s, since computers weren’t translating into real productivity gains.

One of the things we learned was that awesome technology alone is not enough. What you really need is to update your business processes, reskill your workforce, and sometimes even change your business models and organisation in a big way. This can lead to what we call a productivity J-curve, where initially, as you’re doing all those changes, you don’t see productivity gains, and may even experience a productivity loss. Over time, the second part of the J-curve kicks in, and you get these bigger benefits.

I believe that we’re likely to see a similar bit of a lag with generative AI, because it does ultimately require some rethinking of how businesses are run. But what’s different with generative AI is that the lag is shorter; generative AI is happening a lot faster. And that’s partly because generative AI is, frankly, one of the biggest, most effective technologies for changing the way work is done that’s ever been invented. Also, it’s just easier to implement than many earlier technologies, and a lot of it is something regular workers can get going on in a few hours or less.

The dawn of the second machine age

Lareina Yee: Erik, there are two phrases that get thrown around a lot. One is “Fourth Industrial Revolution,” which is a really big concept. And then there’s “knowledge workers.” Can you help us understand how those two concepts come into play with generative AI?

Erik Brynjolfsson: Every time a new technology comes along, you need to rethink how the economy is run. If you simply pave the cow paths and put the same technologies on top of the old way of working, you don’t really get the business benefits. The earlier technological revolutions from the steam engine and electricity each triggered (the four) industrial revolutions viz:

1. 1. 1. Mechanisation/ Steam
      2. Mass production/ Electricity
      3. Automation/ IT
      4. AI/ Connectivity

I actually like to compress them into two bigger, mega revolutions: the first machine age and the second machine age. I wrote a book called ‘The Second Machine Age’ with Andrew McAfee, which focuses on automating and augmenting physical work, and then doing the same for cognitive work.

Whereas the first wave focused on us teaching the machines what to do, step-by-step, we’re now in the second wave of that process. The new wave is based around machines learning how to solve problems on their own, which will probably play out over decades. And it will have a transformative effect at least as big as the First Industrial Revolution. And the reason—to your second point—is that it’s affecting knowledge work. And most workers in the US economy are information workers or knowledge workers. They move around ideas — bits not atoms.

The physical component of the economy remains important. But now that we’re addressing not just the physical but the cognitive, we are likely to have even bigger productivity gains than we saw in the past.

Automating and augmenting jobs, not replacing them

Erik Brynjolfsson: You talked about automating. I like to say both automating and augmenting, because it’s rarely just replacing the entire job or an entire set of tasks. It’s more often allowing people to do them better with more quality and effectiveness, augmenting their ability to do the job.

But as you say, it’s affecting the majority of the tasks for knowledge workers. We pioneered this methodology, and the basic idea is that you can take a company and break it down into occupations, and then break those occupations down into individual tasks. Then, if you look at each of the individual tasks done by, for instance, software engineers, economists, or radiologists — who we learned perform 27 distinct tasks — you can evaluate each of them as to whether or not there’s a gen AI solution that can help them. And by evaluating each of those individual tasks, you get a sense of how much of the economy is likely to be affected.

This is a moving target as technology improves. But one thing that was quite striking was the fact that there was not a single occupation we looked at that was entirely automated, although almost all the jobs had at least some affected tasks. Of course, it will take time for all that to happen, so this represents the potential, not the current reality. And one of the goals is to figure out those optimal places where the technology can be applied first, get it implemented, and convert the potential of technology into the reality of business value.

Lareina Yee: But there are also a lot of concerns in popular media that it could mean some jobs lost and some jobs gained. How should people in business today, at the start of this technological revolution, think about this?

Erik Brynjolfsson: You’re absolutely right. There will be jobs lost, and jobs gained. But let’s keep this in perspective. Technology’s always been destroying jobs, and has always been creating jobs. It’s actually more dangerous to try to freeze in place all of the jobs as they are today, or as they were ten years ago.

The only safe option is to be continually reskilling and relearning and managing this dynamism that has made so many companies successful. That said, it’s going to require a significant amount of retraining and flexibility in the workforce. Over time, it will, I think, tend to disproportionately eliminate some of the most routine, dangerous, boring kinds of jobs, and disproportionately leave more creative, human-centered work alone.

That won’t always be the case, but it’s the general trend. And we’re already seeing it complement a lot of human activity, allowing people to do their jobs more effectively than they could before. So in most cases, it’s really not a replacement but more of an amplification.

Might AI empathise better than humans?

Lareina Yee: If I’m an average person at a company, what are the types of skills that I should develop to be a manager, a leader in this?

Erik Brynjolfsson: I want to be modest here because that’s a tough question, and it’s been evolving rapidly. When Andrew McAfee and I wrote The Second Machine Age, in 2014, we did make a list of skills, and, to be frank, it held up pretty well for five or ten years. But it’s beginning to change.

At the top of the list, we put creative work, interpersonal skills, and physical work. And those continue to be ones that machines have a hard time doing. But all of them are under fire right now. For example, there was a study where medical doctors were asked to answer some questions, and then an AI system was asked to answer the same questions. And most patients found the AI system’s answers were actually somewhat more accurate, or the assessment was that the AI system’s answers were somewhat more factually correct. So AI beat the doctors a bit on the IQ side.

But more surprisingly, patients also found the AI system warmer and more personable. So that was a little scary to see the AI system beating doctors on both dimensions—and even more so on what we might have thought was the human advantage. We have to keep an eye on this and see how it’s evolving.

That said, I think there are some things where humans continue to have a real advantage, like large-scale planning and problem-solving, figuring out what needs to be done and prioritising, and setting the goals that you want to achieve and pointing the system in the right place. Those are jobs for humans.

Another challenge for AI is dealing with exceptions. We looked at call centres, and there are always some questions that come up a lot, and others that are very rare. And the AI system is great when it’s seen the data and knows how to answer the question it’s learned from the data. But when it’s an exception, machine learning has trouble.

Gen AI can train workers and improve performance

Lareina Yee: You mentioned the call centre study, which is something I’ve referenced a lot. I’ve oversimplified it by calling it “the triple win,” because everyone was happy. Could you tell us about what your research found at this call centre?

Erik Brynjolfsson: I love that name, “the triple win.” So let me just describe what we did first. We looked at the implementation of a large language model [LLM] that was to help with their call centre operators. And the great thing for me as a researcher was that they phased it in. Some people had access to it, and some didn’t, so it was a natural experiment where we could really get causal estimates about how the technology performed. The other cool thing was that they used it to augment their workers, rather than try to replace them.

So what did we find? First off, we quickly found very large productivity gains, double-digit gains, sometimes as much as 30 or 35 percent. Within four or five months, the people using the AI system were significantly outperforming their colleagues who’d been on the job for a year or more.

Second, we found that customers were happier and customer satisfaction scores were higher. We also examined customer sentiment by looking at millions and millions of transcripts, using natural language processing to search for happy and angry words. And there were a lot more happy words from customers when the LLM was used, so they seemed to be better off.

Last, but not least, I was a little worried this would turn into some sort of an electronic sweatshop to squeeze the employees. But we found the opposite to be true. The employees were actually happier, and there was significantly less employee turnover as well.

So it is a triple win, like you said. The shareholders gained because the system was creating more productivity and business value. The customers gained more satisfaction and better sentiment. And the employees were happier, with less turnover.

Lareina Yee: It’s pretty remarkable. This was all with humans and machines working together in a way to create a better result. I love that the individual call centre agents were happier and more willing to stay with their organisation. Do you know why that is?

Erik Brynjolfsson: We found that the least-skilled workers, the same people who had previously been doing worse than anyone in the company, actually benefited the most. That’s because they were basically being coached and learning from the system. And the system was identifying the best answers from across the organisation, which often came from the best workers, making them accessible to the least-skilled and least-experienced workers and really bringing them up a lot faster.

So they achieved the biggest gains, up to 35 percent, whereas a lot of the most-skilled workers had basically zero gains. They were already doing everything close to optimal. And so the system is doing something we hadn’t seen before in the history of technology, which is finding a way to capture the tacit organisational and individual knowledge of the best workers and making it accessible to other workers.

And the last thing I’ll mention is that, every once in a while, when the system was down for a couple hours, they still had to answer customer calls. We were worried they’d been de-skilled and they’d kind of used it as a crutch. But the opposite turned out to be true. In fact, the workers who had been using the system continued to give better answers because they had internalised them and continued to pass them on to the customers.

Support, not supplant, humans

Lareina Yee: One piece of research we’re doing at McKinsey involves the concept of experience capital, or, rather, the knowledge gained with first-hand experience. You learn a lot during your formal education, but what you learn on the job is equally important. The idea of being able to accelerate that learning is the essence of reskilling, and CEOs and senior executives are eager to accelerate the spread of best practices so frontline workers reach the expert level faster. We think there’s trillions of value in increased productivity. So if I’m a company trying to capture some of that growth, how do I get started?

Erik Brynjolfsson: Technology alone is not enough. You need to convert it into changes in the way you run your business, and that starts with identifying what the opportunities are.

Erik Brynjolfsson: Alan Turing, the renowned computer scientist and mathematician, came up with this iconic test of intelligence, which came to be known as the ‘Turing test’. The basic idea was to suppose you had a computer in one room, a human in the other, and you interact with each of them without knowing which is which. And if the user can’t tell the difference between the computer and the human, Turing said it has passed the Turing test, and you can consider it intelligent.

I remember reading about it as a kid and thought, “Oh, that’s a great test. It makes a lot of sense to me.” But I’ve since concluded that it’s actually not a good test at all. The sad truth is humans are not that hard to fool, and even very simple AI systems can trick them. So it’s not really a test of intelligence. But the more important thing for me as an economist is that it’s a bad goal.

If engineers, AI researchers, or business executives are trying to replace humans with machines, that creates two really serious problems.

The first one is that it’s actually not ambitious enough. Machines can do so many things better than humans. The point is not to try to mimic exactly what a person does. It’s to lean into the strengths of technology, whether it’s reading millions of books or synthesising lots of texts, since these are things that humans really can’t do.

Second, and more profoundly, it could lead to a really dystopian society. Because if you substitute a machine for human labour, it tends to drive down the value of wages, make human labour less valuable, and concentrate wealth and power among capital or technology owners.

And over time, that can lead to a trap that I’ve called the ‘Turing trap’, where the people who have lost their economic bargaining power also lose their political bargaining power. So the vast majority of people are left behind, or perhaps left to the charity of the people with all the wealth and power, which isn’t a very stable outcome either.

So instead, I propose a different approach, which is making machines that complement and extend human abilities, allowing them to do things they never could have done before. If you have machines that augment humans instead of replacing them, you not only create a lot more value, but you have a higher ceiling, and you’re also more likely to share prosperity. I think that’s the path going forward. We can avoid the Turing trap by complementing instead of substituting human labour.

Lareina Yee: There’s a lot to unpack there, Erik. How should business executives think about the Turing trap and their own strategies and implementation?

Erik Brynjolfsson: I think the easy path and, frankly, the lazy one is to just look at what’s already being done and say, “How can we get a machine to do the same thing as this person?” The greater value lies in asking yourself, “How can I do something new that hasn’t been done before?” or “How can I augment this work to create new kinds of quality?”

If you’re looking to automate the workers, they’re not going to help you.

If you’re looking to augment them and help them do their job better—with higher quality in a faster, more productive way—they’re going to help you with those tough questions of how to be more creative. Ultimately, that’s the much bigger win, and that’s what most of the successful companies are doing.

AI as imperfect critic and collaborator

Lareina Yee: On a lighter note, with all you know about generative AI, how do you use it for your own personal productivity?

Erik Brynjolfsson: I try to use a cluster of different tools: Gemini, Claude, and ChatGPT. They all have different strengths and weaknesses, and they’re all constantly evolving. I even use Pi for a little coaching, but I try to use a lot of different ones.

I do think the technology is a great critic, and a fun thing I discovered is that it can read my papers and then critique them. It’s fun because it doesn’t always get it right, but it often has some good insights.

I think this highlights the fact that you want to keep a human in the loop. Combine the technology with a person to make the final call.

So I encourage people to make it a collaborator, to make it a partner. Don’t try to outsource your work to it. But if you work together with the system, I think you can often improve your work.

N.B. Comments and opinions expressed by interviewees are their own and do not represent or reflect the opinions, policies, or positions of McKinsey & Company or have its endorsement.