AI and improving human decision-making
This is a slightly edited transcript from a talk I gave last year at Prowler.io’s “Decision Summit”
Over six years ago now, I read a book called “Thinking Fast and Slow” by Daniel Kahneman, which is now very well known. This book really grabbed me: it got me thinking about the limitations of human reasoning and how these limitations underpin a lot of problems in the world - from my own personal indecision to big societal problems such as climate change and poverty. I went to do a PhD in behavioural science, to research strategies for overcoming our ‘biases’. And it’s from this perspective that I first got interested in AI: as something that might help us to do better, as humans.
I now work for a research group at Cambridge called the Centre for the Future of Intelligence, thinking about the ethical and policy issues surrounding the use of AI systems in society. AI has been getting increasing amounts of attention over the last year or two: with multiple articles being published on AI across different media outlets every day, governments across the world beginning to develop AI strategies, and new academic research groups like the one I work for cropping up all the time. But what, exactly, makes AI so exciting? I’d like to suggest that the biggest reason is that we hope AI might be able to help us solve really important problems in the world, problems that we as humans alone struggle to solve.
AI is already being used to solve important problems that we couldn’t solve alone as humans. For example, AI is beginning to improve the quality of diagnosis and treatment in healthcare. It could help reduce poverty - last May I was at the UN’s “AI for Global Good Summit” where Stuart Russell discussed the potential of using machine learning and satellite imagery to rapidly and accurately map poverty and wealth across different parts of the world. Even more ambitiously, AI could help us do better scientific research, speeding up progress on all kinds of problems: helping neuroscientists better understand the brain, or physicists better analyze physics data - perhaps ultimately AI systems will be able to come up with better, more creative scientific theories than we can.
We can all agree that the potential for AI to help us ‘make the world a better’ place is exciting. But at the same time, there are two very different models we might have of how AI systems will help us to solve important problems:
The first is to think of AI systems as replacing human capabilities: we solve problems better by increasingly outsourcing tasks and decisions to automated systems which can solve them more quickly, efficiently, and effectively. To give a simple example, Google maps is much better than my brain could ever be at knowing all the different routes in a city and calculating the quickest way to get from A to B: so most of the time I just input where I am and where I want to go to, and then follow what Google tells me to do pretty blindly.
A second, different way to think of AI systems is as complementing human capabilities: AI systems can help us to understand the world in new and important ways, which are complementary to - rather than simply ‘better than’ - the ways that humans understand the world. Returning to the example of Google maps, there may be things I know about my city - which routes are safest at night, or which are most scenic for example - that aren’t captured by the software. Using google maps can help me to identify the quickest route very quickly, which saves me time and energy - but it’s best used in conjunction with things I already know.
I think that a lot of current discourse around AI research and its application in society implicitly assumes this first model - that the aim is to replace human capabilities with better, AI ones. I also think that a lot of the ethical concerns and fears around deploying AI systems in society naturally stem from this assumption. There are serious concerns about the increasing automation of jobs in society, and what this will do to the economy, inequality, and people’s sense of worth and meaning. People are beginning to worry about how certain human skills might atrophy as we have to use them less: perhaps our memories are already worse today than when we didn’t have the ability to look up everything on the internet. And there are concerns about the safety and reliability of AI systems as they replace humans in safety-critical domains such as self-driving cars.
Especially given all of these concerns, a really important question to ask right now is: do we actually want, or need, to build AI systems that can replace human capabilities? In many domains and applications the answer may be ‘yes’, but I think this question needs to be asked and pushed on a bit more. I want to suggest that there is a quite different way of thinking about how AI systems can help us solve problems - by complementing human capabilities - and that thinking more explicitly about the relative strengths of human and machine capabilities, and how they can work together to solve different types of problems, might be really beneficial.
I suspect this idea that we want AI systems to replace human capabilities is influenced in part by the attitude that people are pretty ‘irrational’. This attitude stems from psychology research in recent decades, which has focused a lot on identifying the various biases and irrationalities that people are prone to, leading to quite a pessimistic picture of human capabilities. Our brains have to process a huge amount of information, filter out what’s relevant and ignore what isn’t, make sense of ambiguity, act quickly, all often to solve problems that aren’t even clearly defined.
To illustrate the kinds of challenges we face day-to-day, take the ‘simple’ task of buying a bike, which I had to do when I moved to Cambridge recently. There are thousands of places you could look online and offline, and thousands of different makes. There are also many different things you might care about when buying a bike - should I just buy the cheapest decent one I can find? Do I really want the prettiest one? Or should I just go for the best reviewed one - but according to which website? It quickly becomes totally overwhelming trying to weigh up all your options on all these different variables at the same time.
Because we’re faced with a huge amount of complexity and uncertainty, we can’t possibly optimise every decision. So we use heuristics, shortcuts: in my case, I bought the bike that my sister has, because I’ve ridden it and it seemed pretty good, and it didn’t seem worth spending much more time to find a slightly better one. In this case, I think this was a pretty good heuristic. But “do what my sister does” might not be a great heuristic for other kinds of decisions - for choosing which political candidate to vote for, for example.
To understand both the strengths and limitations of human reasoning, we need to understand these heuristics that we use to make sense of an incredibly complex and uncertain world. These heuristics actually work extraordinarily well a lot of the time - but they go wrong in some systematic ways. I’ll give a few examples which I think are pretty central to the limitations of human reasoning.
Because we’re faced with an overwhelming amount of information in making even the simplest decisions, we have to decide what to pay attention to, and what to filter out. One problem that occurs here is we tend to overweight things that are particularly emotionally compelling or easy to visualise, relative to important pieces of information that might be more abstract and uncertain. We’re much more motivated by immediate rewards - the desire for just one more scoop of ice cream - than longer-term, more probabilistic ones - such as the long-term benefits of eating healthily.
Because we use ‘rules of thumb’ rather than strict and systematic procedures, our judgements are easily influenced by what are called ‘framing effects’ - how a question is asked, or what other things we’ve thinking about recently, for example. This means that consistency is not a strength of human reasoning - ask me the same question twice on different days, and I might well give different answers. One study, for example, found that experienced radiologists rating x-rays as “normal” or “abnormal” contradicted themselves 20% of the time!
We also aren’t particularly good at reasoning clearly about large numbers: above a certain size, our brains tend to see all large numbers as pretty similar. This is a problem because sometimes these differences really matter - one famous study found that when asked how much they thought it was worth spending to save 10,000 or 100,000 birds, people gave roughly similar answers - which seems mad when we think about how big the difference between these two numbers actually is.
Finally, these information processing shortcuts mean we’re prone to learning “illusory correlations” when faced with complex, messy information: that is, convincing ourselves of relationships that don’t really exist. Some have suggested that this tendency to identify illusory correlations underpins how untrue stereotypes form and persist: if you believe that women are less confident than men, for example, then you may start noticing all the cases where this is true and ignore all the cases where it isn’t.
Machines can help us overcome a lot of these biases and problems, because they have very different strengths and limitations:
Because machines can store and process much larger quantities of information in parallel, it’s much easier for them to weigh up lots of different factors in making a decision. In fact, research has shown that even a very simple linear formula (i.e. no complex functions or machine learning involved) can outperform human judgement on a range of tasks which require weighing lots of different factors: including predicting the future grades of students, the longevity of cancer patients and the chances of success for a new business. So it’s not surprising that machine learning models, trained on a huge amount of relevant data, can do even better still.
Part of the reason given for this is that machines are much more consistent in many ways, and not swayed by irrelevant factors in the way that humans are.
Machines are also much better at working with precise numbers and probabilities than we are, and at identifying reliable patterns in large and complex datasets - this is why they have the potential to be so valuable in healthcare.
But while there certainly are ways that machines could help us overcome human limitations, I think it’s a little too easy to take the view that “humans are irrational, and AI systems, once they’re more advanced, will just be so much better than us at anything.” What AI research has shown us over recent years, if anything, is that many aspects of human cognition which we take completely for granted are actually incredibly complex and difficult to replicate in machines.
One interesting comparison here: we see chess as a complex game, requiring quite a lot of human intelligence to be good at. This turned out to be surprisingly easy to brute-force with a computer program. By contrast, certain aspects of vision like the ability to consistently recognise a wide range of different ‘chairs’ as belonging to the same category is something we completely take for granted and don’t associate with intelligence at all in humans - but has turned out to be surprisingly difficult to build into AI systems.
This is just to point out that despite some of the flaws and limitations of human reasoning, and despite the huge amount of progress we’ve been seeing in machine learning, human cognition still has a lot of strengths relative to machines. Human reasoning may often be imprecise and inconsistent - but it’s also amazingly robust and flexible. Infants can learn stable and flexible concepts incredibly quickly, learning to tell the difference between cats and dogs pretty reliably after only seeing a few examples - whereas current machine learning systems generally take thousands of examples and still can’t learn in such a flexible way.
I think we sometimes take for granted the strengths of human cognition because they are precisely those things that we do automatically and without effort, like recognising chairs, navigating our environment, picking up nuance in sentences and recognising emotions on people’s faces. By contrast, the things we associate with ‘intelligence’ in humans are those things we find difficult, like chess - but this gives us a distorted view of what’s difficult and impressive in cognition more generally. As it stands at the moment, humans and AI systems appear to have very different and complementary strengths, and my suggestion is that perhaps we should be trying to understand and leverage those differences more.
Through doing so, we might be able to:
identify better ways for humans and machines to work together to solve important problems,
better prioritise what kinds of AI capabilities we most want to develop,
and identify ways that humans can best learn from AI systems and vice versa.
I think there are a lot of different motivations driving AI progress. In part, AI research is driven by curiosity - I think there’s a deep drive to understand what intelligence is, and a hope that advances in AI might help us to get there. In part, research is driven by commercial or near-term incentives: companies trying to get a competitive advantage or make money - that’s how the world works! But as I said at the beginning, I think this desire to improve our ability to solve important problems in the world is really, fundamentally what makes AI so exciting for most people. And if this is what’s really driving us, I think this question of how we can build AI systems that are complementary to humans is a pretty important one.
At the moment, I don’t see very much of this in how people talk about AI. There are some researchers doing great work exploring the relative strengths of human and machine learning, but their focus is on understanding what we can learn from human strengths about how to build more generally capable machine learning systems. I think this is important, but perhaps equally important is to ask: how can understanding the strengths and limitations of human reasoning help us build AI systems that best complement those abilities: that do the things we do poorly, well?
I want to end by pointing out that there’s a difficult tension here, in how we think about developing AI. On the one hand, we want AI systems that can do things we can’t, that are better than us - otherwise what’s the point? - but we’re also scared about what this will mean for us as humans. In a way, what I’m really suggesting here is that we ask a bit more explicitly: what do we really want from AI, and how do we want it to affect us as humans?