How useful is technical understanding for working in AI policy?

It’s not totally clear what the ideal background or relevant ‘expertise’ for AI policy is. Working in other areas of technology policy seems like the most directly relevant experience, but because this is such a new and fast-growing area, people are coming from all kinds of different backgrounds. One thing I’ve been thinking about is how useful it is for people working in AI policy to have technical experience/understanding in machine learning, or computer science more generally. Should more people with technical expertise be working on AI policy issues? Should people working on AI policy already be focusing more on developing their technical understanding? I think I lean more strongly towards “yes” on these questions than many people, and so I want to try and spell out why I think this.

(Note: here I’m using ‘AI policy’ quite broadly, to encompass all kinds of thinking about how AI will impact society and how those impacts should be managed - not just referring people working directly in policy jobs.)

Why technical understanding matters

1. Thinking clearly about possibilities and risks

First, thinking clearly about how current AI systems will impact society requires a decent understanding of the capabilities and limitations of those systems. I do think that it’s possible to think usefully about societal decision-making around AI with a pretty high-level sense of what “AI” is. But I also worry that the notion of “AI” that underpins many policy discussions is far too vague, in a way that fuels misunderstanding about what the possibilities and risks posed by AI are. 

We’re currently seeing a lot of people repeating the same buzzwords and concerns in vague terms - e.g. privacy, bias, explainability - but there are relatively few people thinking from ‘first principles’ about what specific current capabilities and limitations mean for society. For example, almost all applications of “AI” in society raising concerns today seem to be specifically those using supervised learning training methods, but this is rarely explicitly acknowledged. There’s an interesting question of whether current concerns around AI being applied in society are overly specific to SL, and whether AI systems based on different training methods (e.g. reinforcement learning) raise different concerns or should be treated differently. Thinking clearly about this question doesn’t require deep expertise in ML, but it does require solid intuitions about the differences between these methods and their applications that is non-trivial and most AI policy researchers probably lack.

2. Thinking ahead

Second, it’s important that AI policy can be anticipatory, not just reacting to the problems that have already arisen, but thinking ahead about how society can prepare for advances in AI capabilities and their possible impacts. Of course, we can’t predict any of this with any certainty, but we can think carefully about different ways that AI capabilities might evolve and the implications of different development trajectories. This requires a high-level understanding of what general capabilities AI systems have, what tasks and problems this makes them well-suited to, where the limitations of current systems are and where research appears to be making progress. I think there’s a real tendency to talk about future AI systems as if they’re magic - e.g. suggesting that in future AI will be used to do scientific research or be able to self-improve, without actually thinking through any details of what being able to do these things might involve - which is at least partly due to lack of technical understanding.

3. Working collaboratively

Third, solutions to problems arising from AI need to be a collaborative effort between policy experts, social scientists, and technical researchers/developers (at the very least.) This means these groups of people need to be able to talk to each other! Of course, there’s a responsibility to bridge important divides on all of these groups, but policy practitioners/researchers working to understand what ML research & development looks like will be an important component of these collaborations. One part of this is just being able to speak roughly the same language as technical researchers - e.g. understanding what it means to train a model, the difference between different training methods. Also important is being able to identify when drawing on deeper technical expertise would be useful, and where to look for it. More generally, most problems arising from AI will have solutions that are part ‘technical’, i.e. partly about the kinds of systems and capabilities we develop, and part ‘social’, i.e. partly about how we design aspects of society to respond to and govern these systems. For example, ensuring that medical AI systems are used safely requires thinking about both (a) how to make sure those systems are robust and verifiable/interpretable on the technical side, and (b) what kinds of checks, processes, and governance are needed more broadly to prevent, catch, and respond to important errors. If people working on (a) and (b) are operating entirely independently of one another, this work is going to be hugely inefficient at best - and in particular those on the governance side need to understand the limitations of technical safety approaches so they know where safety checks and regulations are most needed.

What kind of technical understanding?

One thing that is fairly clear to me is that the most useful kind of technical understanding is in most cases not going to be deep expertise in some specific subfield of ML. Much more likely to be useful is having a decent understanding of what ML research involves in practice, enough terminological understanding to be able to talk to ML researchers and skim papers, and a high-level understanding of current capabilities and limitations, and where they might go in future. I’m not actually sure what the best way to acquire this is - especially the “high-level understanding of current capabilities and limitations” part. I think this high-level understanding is probably quite difficult to acquire, and something that many ML experts don’t actually have, if their focus is very narrow. It’s also not necessarily something you get automatically from learning more about how ML works and knowing the difference between different current methods.

Obviously, the level and type of technical understanding that’s useful depends a lot on the type of research or work you’re doing, and I don’t necessarily think all policy researchers should be going away and taking ML courses. Maybe it’s fine for there to be many people thinking about AI policy in very broad strokes, just understanding the very general features and implications of AI - e.g. the fact that AI involves automating tasks previously done by humans, generally requires large amounts of data, and that methods aren’t always fully interpretable to us. But we do need some people in the policy space who are thinking more deeply about what is and what might be technically possible, to ensure current concerns are well-grounded, to ensure solutions will still be relevant as capabilities advance, and to ensure productive collaboration with ML researchers. In particular I worry that there’s a severe lack of technical expertise in government - where decisions about how AI is actually governed will actually get made.

One thing I’d like to think more about is how this works and has been thought about in other areas of science and technology policy: e.g. how well do people thinking about biosecurity or climate policy understand the relevant science and technical capabilities? Climate policy is a bit disanalogous because we’re not talking about an evolving technology, but there’s still a certain level of scientific understanding that seems like an important prerequisite for working in this space. Governance of biotechnology might be more closely analogous. One suspicion I have is that the average level of technical understanding in AI policy is lower than in many other areas, because almost everyone has a high-level impression of what ‘AI’ is (whereas most non-experts are more aware that they have no idea really what current biotechnology looks like.) It could be that the level of technical understanding required to contribute usefully to AI policy is just lower than in other fields, for this reason - but I also worry that it’s not, and it’s just easier to delude ourselves that we really understand what AI is than it is to delude ourselves about technologies where there’s less of a public narrative.

Thoughts on short- vs. long-term AI policy

It’s generally acknowledged that there’s a distinction between “short-term” (or “near-term”) and “long-term” AI policy issues. But these distinctions actually tend to conflate (at least) three things:

  1. When issues arise.

    e.g. Cave and ÓhÉigeartaigh (2019) define ‘near-term’ issues as “immediate or imminent challenges”; the 80k guide to working in AI policy defines ‘short-term’ issues as “issues society is grappling with today”

  2. How advanced the relevant AI capabilities are.

    e.g. Baum (2018) distinguishes between a ‘futurist’ AI claim which says that “attention should go to the potential for radically transformative long-term AI” and a ‘presentist’ AI claim which says that “attention should go to existing and near-term AI.” Similarly 80k talk about ‘long-term’ issues as those “that either only arise at all or arise to a much greater extent when AI is much more advanced than it is today.”

  3. How likely an issue is to have long-term consequences.

    e.g. 80k say that ‘long-term’ issues are those that “will have very long-lasting consequences.”

These three things are generally assumed to go hand-in-hand, or at least not clearly distinguished. This might seem like quibbling with definitions, but I actually think it fuels confusion about which issues are most important to work on.

I think that what most people in the ‘long-term camp’ really care about is (3) - how likely an issue is to have (large and) long-lasting consequences for society. (1) and (2) only matter insofar as they influence this.

If we define ‘long-term’ issues in this way - as the issues most likely to have long-lasting consequences for society - I’m not sure how many people in the ‘short-term’ camp would actually put themselves in opposition to that. I certainly don’t think many people would say that they are explicitly prioritising issues that will only have a short-term impact on society over those with longer-lasting consequences. This distinction between ‘short-term’ and ‘long-term’ starts to feel a lot messier and less clear-cut.

I think there are actually several different ways in which people disagree about which AI policy issues to work on, that don’t come down to a simple short-/long-term distinction. It’s worth trying to pick these apart, because in doing so we might realise that e.g. people disagree less than they seem to, there’s empirical research that could resolve important disagreements, or perhaps even that important issues or areas are being neglected because of the assumptions being made on both ‘sides’. Here are some key disagreements I think are getting mixed up:

  • (a) Disagreement about whether we should work on issues affecting current vs. future people. There are some genuine disagreements about whether it’s more important to work on issues affecting current populations, and to what extent we should also be concerned about future generations. These stem from pretty deep philosophical beliefs: i.e. some people believe we have a greater moral obligation to those alive today whereas others don’t. I think these views contribute somewhat to what AI policy issues people think are most important to work on, but I suspect it’s only a relatively small part.

  • (b) Disagreement about how long-lasting the consequences of ‘nearer-term’ issues are likely to be. I think many people would broadly agree that all else equal, it’s better to prioritise working on issues with longer-lasting consequences for humanity. I imagine many people working on making algorithms fair and accountable today are doing so because they believe that failing to solve these problems could have extremely bad, long-lasting consequences for society (entrenching extreme power structures, leading to extreme inequality, and so on.)

  • (c) Disagreement about the best ways to influence the long-term future. When prioritising which issues to work on, what matters is not just their potential impact but also whether we have any ‘leverage’ to shape the way things go (a point made nicely by Ben Garfinkel here). One criticism of those who focus on issues relating to very advanced AI is that it’s very difficult for us to have any idea what AI will look like in the future - and the implication, there, I think, is that this means we have little leverage to influence it. On the other side, some in the ‘long-term’ camp might criticise work addressing issues arising today on the basis that it’s unlikely to have any long-lasting impact.

I think that the way the “short vs long-term” divide in AI policy is currently, there’s way too much focus on the deep ideological disagreement of (a), and not enough on really understanding the tricky and mostly empirically-based disagreements of (b) and (c). I think it would be really valuable to try and unpick further some of the assumptions underpinning these disagreements, and think about what kinds of research might actually help us think more clearly about the best ways to influence the long-term societal impacts of AI.

In case it’s not clear by this point, I’m pretty firmly in the “we should care about future populations” camp on (a), and I do think that we should be trying to work in those areas where we might have some influence over how AI impacts society in the very long-run. But I’m much less clear on (b) or (c). I think it’s possible that issues arising from current AI systems or more advanced capabilities that fall far short of AGI could have extreme and long-lasting impacts on society - either by leading to extreme scenarios themselves (e.g. automated surveillance leading to global authoritarianism), or by undermining our collective ability to manage other threats (e.g. AI-enabled disinformation undermining collective decision-making/coordination capacity.)

I also think that some of the best ways to influence the long-term future might be by working on what are mostly ‘current’ issues but with the long-term in mind (e.g. ensuring that today’s AI systems are developed in safe and interpretable ways that extend to more advanced systems; creating good research norms and a culture of responsibility within ML research; developing policy processes that are robust to uncertainty about how AI will develop, and so on.)

Mostly, I think we need more thorough thinking on both how ‘near-term’ and emerging issues might have very long-term consequences, and on what kinds of ‘near-term’ work give us the best leverage over the future trajectory and impact of AI.

One thing I haven’t really talked about, but which is important for prioritising areas to work on, is neglectedness: finding important areas to work on that aren’t getting much attention. Neglectedness is a large part of the reason that so far the ‘long-term’ community has mostly focused on more speculative risks from very advanced AI systems - no-one else was thinking about them. But I now think we may be at a point where something like “near-term work from a long-term perspective” is also looking pretty neglected.

AI and improving human decision-making

This is a slightly edited transcript from a talk I gave last year at’s “Decision Summit”

Over six years ago now, I read a book called “Thinking Fast and Slow” by Daniel Kahneman, which is now very well known. This book really grabbed me: it got me thinking about the limitations of human reasoning and how these limitations underpin a lot of problems in the world - from my own personal indecision to big societal problems such as climate change and poverty. I went to do a PhD in behavioural science, to research strategies for overcoming our ‘biases’. And it’s from this perspective that I first got interested in AI: as something that might help us to do better, as humans.

I now work for a research group at Cambridge called the Centre for the Future of Intelligence, thinking about the ethical and policy issues surrounding the use of AI systems in society. AI has been getting increasing amounts of attention over the last year or two: with multiple articles being published on AI across different media outlets every day, governments across the world beginning to develop AI strategies, and new academic research groups like the one I work for cropping up all the time. But what, exactly, makes AI so exciting? I’d like to suggest that the biggest reason is that we hope AI might be able to help us solve really important problems in the world, problems that we as humans alone struggle to solve.

AI is already being used to solve important problems that we couldn’t solve alone as humans. For example, AI is beginning to improve the quality of diagnosis and treatment in healthcare. It could help reduce poverty - last May I was at the UN’s “AI for Global Good Summit” where Stuart Russell discussed the potential of using machine learning and satellite imagery to rapidly and accurately map poverty and wealth across different parts of the world. Even more ambitiously, AI could help us do better scientific research, speeding up progress on all kinds of problems: helping neuroscientists better understand the brain, or physicists better analyze physics data - perhaps ultimately AI systems will be able to come up with better, more creative scientific theories than we can.

We can all agree that the potential for AI to help us ‘make the world a better’ place is exciting. But at the same time, there are two very different models we might have of how AI systems will help us to solve important problems:

  1. The first is to think of AI systems as replacing human capabilities: we solve problems better by increasingly outsourcing tasks and decisions to automated systems which can solve them more quickly, efficiently, and effectively. To give a simple example, Google maps is much better than my brain could ever be at knowing all the different routes in a city and calculating the quickest way to get from A to B: so most of the time I just input where I am and where I want to go to, and then follow what Google tells me to do pretty blindly.

  2. A second, different way to think of AI systems is as complementing human capabilities: AI systems can help us to understand the world in new and important ways, which are complementary to - rather than simply ‘better than’ - the ways that humans understand the world. Returning to the example of Google maps, there may be things I know about my city - which routes are safest at night, or which are most scenic for example - that aren’t captured by the software. Using google maps can help me to identify the quickest route very quickly, which saves me time and energy - but it’s best used in conjunction with things I already know.

I think that a lot of current discourse around AI research and its application in society implicitly assumes this first model - that the aim is to replace human capabilities with better, AI ones. I also think that a lot of the ethical concerns and fears around deploying AI systems in society naturally stem from this assumption. There are serious concerns about the increasing automation of jobs in society, and what this will do to the economy, inequality, and people’s sense of worth and meaning. People are beginning to worry about how certain human skills might atrophy as we have to use them less: perhaps our memories are already worse today than when we didn’t have the ability to look up everything on the internet. And there are concerns about the safety and reliability of AI systems as they replace humans in safety-critical domains such as self-driving cars.

Especially given all of these concerns, a really important question to ask right now is: do we actually want, or need, to build AI systems that can replace human capabilities? In many domains and applications the answer may be ‘yes’, but I think this question needs to be asked and pushed on a bit more. I want to suggest that there is a quite different way of thinking about how AI systems can help us solve problems - by complementing human capabilities - and that thinking more explicitly about the relative strengths of human and machine capabilities, and how they can work together to solve different types of problems, might be really beneficial.

I suspect this idea that we want AI systems to replace human capabilities is influenced in part by the attitude that people are pretty ‘irrational’. This attitude stems from psychology research in recent decades, which has focused a lot on identifying the various biases and irrationalities that people are prone to, leading to quite a pessimistic picture of human capabilities. Our brains have to process a huge amount of information, filter out what’s relevant and ignore what isn’t, make sense of ambiguity, act quickly, all often to solve problems that aren’t even clearly defined.

To illustrate the kinds of challenges we face day-to-day, take the ‘simple’ task of buying a bike, which I had to do when I moved to Cambridge recently. There are thousands of places you could look online and offline, and thousands of different makes. There are also many different things you might care about when buying a bike - should I just buy the cheapest decent one I can find? Do I really want the prettiest one? Or should I just go for the best reviewed one - but according to which website? It quickly becomes totally overwhelming trying to weigh up all your options on all these different variables at the same time.

Because we’re faced with a huge amount of complexity and uncertainty, we can’t possibly optimise every decision. So we use heuristics, shortcuts: in my case, I bought the bike that my sister has, because I’ve ridden it and it seemed pretty good, and it didn’t seem worth spending much more time to find a slightly better one. In this case, I think this was a pretty good heuristic. But “do what my sister does” might not be a great heuristic for other kinds of decisions - for choosing which political candidate to vote for, for example.

To understand both the strengths and limitations of human reasoning, we need to understand these heuristics that we use to make sense of an incredibly complex and uncertain world. These heuristics actually work extraordinarily well a lot of the time - but they go wrong in some systematic ways. I’ll give a few examples which I think are pretty central to the limitations of human reasoning.

Because we’re faced with an overwhelming amount of information in making even the simplest decisions, we have to decide what to pay attention to, and what to filter out. One problem that occurs here is we tend to overweight things that are particularly emotionally compelling or easy to visualise, relative to important pieces of information that might be more abstract and uncertain. We’re much more motivated by immediate rewards - the desire for just one more scoop of ice cream - than longer-term, more probabilistic ones - such as the long-term benefits of eating healthily.

Because we use ‘rules of thumb’ rather than strict and systematic procedures, our judgements are easily influenced by what are called ‘framing effects’ - how a question is asked, or what other things we’ve thinking about recently, for example. This means that consistency is not a strength of human reasoning - ask me the same question twice on different days, and I might well give different answers. One study, for example, found that experienced radiologists rating x-rays as “normal” or “abnormal” contradicted themselves 20% of the time!

We also aren’t particularly good at reasoning clearly about large numbers: above a certain size, our brains tend to see all large numbers as pretty similar. This is a problem because sometimes these differences really matter - one famous study found that when asked how much they thought it was worth spending to save 10,000 or 100,000 birds, people gave roughly similar answers - which seems mad when we think about how big the difference between these two numbers actually is.

Finally, these information processing shortcuts mean we’re prone to learning “illusory correlations” when faced with complex, messy information: that is, convincing ourselves of relationships that don’t really exist. Some have suggested that this tendency to identify illusory correlations underpins how untrue stereotypes form and persist: if you believe that women are less confident than men, for example, then you may start noticing all the cases where this is true and ignore all the cases where it isn’t.

Machines can help us overcome a lot of these biases and problems, because they have very different strengths and limitations:

  • Because machines can store and process much larger quantities of information in parallel, it’s much easier for them to weigh up lots of different factors in making a decision. In fact, research has shown that even a very simple linear formula (i.e. no complex functions or machine learning involved) can outperform human judgement on a range of tasks which require weighing lots of different factors: including predicting the future grades of students, the longevity of cancer patients and the chances of success for a new business. So it’s not surprising that machine learning models, trained on a huge amount of relevant data, can do even better still.

  • Part of the reason given for this is that machines are much more consistent in many ways, and not swayed by irrelevant factors in the way that humans are.

  • Machines are also much better at working with precise numbers and probabilities than we are, and at identifying reliable patterns in large and complex datasets - this is why they have the potential to be so valuable in healthcare.

But while there certainly are ways that machines could help us overcome human limitations, I think it’s a little too easy to take the view that “humans are irrational, and AI systems, once they’re more advanced, will just be so much better than us at anything.” What AI research has shown us over recent years, if anything, is that many aspects of human cognition which we take completely for granted are actually incredibly complex and difficult to replicate in machines.

One interesting comparison here: we see chess as a complex game, requiring quite a lot of human intelligence to be good at. This turned out to be surprisingly easy to brute-force with a computer program. By contrast, certain aspects of vision like the ability to consistently recognise a wide range of different ‘chairs’ as belonging to the same category is something we completely take for granted and don’t associate with intelligence at all in humans - but has turned out to be surprisingly difficult to build into AI systems.

This is just to point out that despite some of the flaws and limitations of human reasoning, and despite the huge amount of progress we’ve been seeing in machine learning, human cognition still has a lot of strengths relative to machines. Human reasoning may often be imprecise and inconsistent - but it’s also amazingly robust and flexible. Infants can learn stable and flexible concepts incredibly quickly, learning to tell the difference between cats and dogs pretty reliably after only seeing a few examples - whereas current machine learning systems generally take thousands of examples and still can’t learn in such a flexible way.

I think we sometimes take for granted the strengths of human cognition because they are precisely those things that we do automatically and without effort, like recognising chairs, navigating our environment, picking up nuance in sentences and recognising emotions on people’s faces. By contrast, the things we associate with ‘intelligence’ in humans are those things we find difficult, like chess - but this gives us a distorted view of what’s difficult and impressive in cognition more generally. As it stands at the moment, humans and AI systems appear to have very different and complementary strengths, and my suggestion is that perhaps we should be trying to understand and leverage those differences more.

Through doing so, we might be able to:

  • identify better ways for humans and machines to work together to solve important problems,

  • better prioritise what kinds of AI capabilities we most want to develop,

  • and identify ways that humans can best learn from AI systems and vice versa.

I think there are a lot of different motivations driving AI progress. In part, AI research is driven by curiosity - I think there’s a deep drive to understand what intelligence is, and a hope that advances in AI might help us to get there. In part, research is driven by commercial or near-term incentives: companies trying to get a competitive advantage or make money - that’s how the world works! But as I said at the beginning, I think this desire to improve our ability to solve important problems in the world is really, fundamentally what makes AI so exciting for most people. And if this is what’s really driving us, I think this question of how we can build AI systems that are complementary to humans is a pretty important one.

At the moment, I don’t see very much of this in how people talk about AI. There are some researchers doing great work exploring the relative strengths of human and machine learning, but their focus is on understanding what we can learn from human strengths about how to build more generally capable machine learning systems. I think this is important, but perhaps equally important is to ask: how can understanding the strengths and limitations of human reasoning help us build AI systems that best complement those abilities: that do the things we do poorly, well?

I want to end by pointing out that there’s a difficult tension here, in how we think about developing AI. On the one hand, we want AI systems that can do things we can’t, that are better than us - otherwise what’s the point? - but we’re also scared about what this will mean for us as humans. In a way, what I’m really suggesting here is that we ask a bit more explicitly: what do we really want from AI, and how do we want it to affect us as humans?

Reflections on AIES/AAAI 19

Last month, I made the arduous trip to Hawaii (half joking - it is a 20hr journey!) to the AI Ethics and Society conference (AIES), co-located with the AAAI/ACM annual conference. I wanted to share some slightly delayed reflections on the trip.

Some general thoughts on AIES

This is only the second year AIES has run. I was particularly interested in going because it is the first mainstream academic conference focused explicitly on the ethical and societal aspects of advances in AI, covering a wide range of disciplines. Because this is a new and emerging area of research that’s incredibly broad, bringing that all together effectively in a single conference isn’t exactly easy - but I think it’s important to enable people across many different disciplines - philosophy, political science, economics, law, literature, history, and so on... - to collaborate and learn from one another.

Quite a large proportion of the accepted papers either (a) focused on solutions to the technical problem of AI alignment/safety, or (b) presented technical work on making ML systems transparent and fair. I found a lot of these presentations really interesting, and think this kind of work is important at a conference like AIES. But they’re also two areas which are now fairly well-established subfields of AI research, and I was a bit disappointed not to see more research that focused on policy or governance approaches to AI in particular. It’s also definitely challenging to bring together people from different disciplines and have them communicate effectively with one another - in some cases it felt like the talks were steeped in the language of a discipline I wasn’t familiar with and so difficult to follow, and someone else commented to me that they felt some presentations coming from one discipline weren’t aware enough of relevant work in existing disciplines.

All that said, AIES still did a much better job of bringing together a wide range of people and perspectives than most conferences I’ve been to! And it was only the second year of the conference, so there’s lots of room for it to develop and provide an even better environment for cross-disciplinary collaboration and conversations. I think it’d be worth thinking about how to get the call for papers out to a wider range of disciplines and groups next year, and how to build opportunities for sharing insights across relevant disciplines into the conference program (maybe a session or workshop on the side explicitly focused on this would be helpful.)

Some notes and highlights

These are some pretty quick thoughts/summaries of some of the talks I found interesting and actually managed to take notes on...

There were a couple of talks on AI safety/alignment research that I thought did a really good job of explaining technical work in an engaging and accessible way. This is something I particularly care about, as I think it’s really important for those working on ethics/policy issues to have a solid a grounding in what’s technically possible in general, and technical approaches to ensuring safe and beneficial AI specifically.

Anca Dragan - Specifying AI Objectives as a Human-AI Collaboration Problem

First, Anca Dragan from UC Berkeley/CHAI gave a great talk on her group’s work on inverse reward design, an approach to building reinforcement learning systems that can learn what we want them to do from what we tell them. The idea here is to avoid us having to specify a reward function precisely, which is really difficult - in large part because we’re only able to consider a relatively small subset of possible situations, and will inevitably always miss some unintended consequences. So rather than us providing the agent with a reward function and the agent taking that literally as what it should do, the idea of inverse reward design is that the agent should take the specified reward function merely as evidence of what we actually want, as evidence of the “true reward function.” Instead of optimising a single reward function, then, the

agent learns a probability distribution over a range of possible reward functions (i.e. a range of things we might have meant), allowing it to then take actions which account for uncertainty, and are robust across a wide range of possibilities. This approach also makes the task of defining a reward function a much less difficult one for humans - in fact, we could define multiple different reward functions for different environments which are then all taken as evidence of what we actually want. Anca showed some nice results demonstrating how this could improve the robustness and reliability of a robotic arm.

Of course, this only works well if we have a good way to define the appropriate space of possible reward functions. We could give this to the agent explicitly, but we might not always know what the best space to consider is, and this somewhat defeats the point of wanting the agent to learn to be sensitive to situations we haven’t thought about. Something which can help with this is to enable the agent to come back and ‘query’ its designers about unknown situations, rather than being overly risk-averse (called ‘active inverse reward design’). This reframes the problem of trying to get an agent to do what we want somewhat: rather than being an optimisation problem, we might instead think of it as a human-AI collaboration problem. But we still face a tradeoff here in how we want this collaboration to work in practice - we’d rather human input didn’t constrain possibilities too much initially, but the more open we leave things the more the agent will likely need to query the human later, which is also costly and may be impractical.

A next step for research here is to try and build models which capture human biases - which would make it easier to model the kinds of mistakes we might make in specifying reward functions. This would make it easier for the agent to anticipate the kinds of mistakes we are likely to make across different scenarios, and so narrow the space of possible reward functions that are likely to be relevant initially.

Alexander Peysakhovich - Reinforcement learning and inverse reinforcement learning with system 1 and system 2

Second, and relatedly, Alex Peysakhovich presented some interesting work on incorporating models of human biases from behavioural economics into reinforcement learning. Alex begins by pointing out that inferring a person’s goals from their behaviour is important to many problems in AI - including for cooperation between humans and AI in general, for products such as recommender systems, and for inverse reinforcement learning (where we aim to train an agent to learn human goals/preferences from behaviour.) Most approaches to learning goals from behaviour in AI assume a rational actor model, which has been challenged by a great deal of research in psychology/behavioural economics in recent years. Peysakhovich’s paper uses the now-popular idea that some of the irrationalities in human cognition can be modelled as “two systems” (a slow, reflective system 2, and a fast, intuitive, associative system 1), formalised as two separate utility functions. He shows that both reinforcement learning and inverse reinforcement learning still work with this distinction between s1 and s2: it’s still possible to compute an optimal policy using the two utility functions, and you can infer what both s1 and s2 want separately using inverse RL.

Peysakhovich ends by with the broader suggestion that we need better models of human irrationalities for both RL and inverse RL. It was interesting to see this conclusion coming out pretty strongly both in this talk and in Anca’s talk described above. I also wonder whether trying to implement different models of human (ir)rationality in AI systems might yield some interesting findings for cognitive/behavioural science in return, as it could potentially provide a means of testing different models of cognition, and might encourage thinking in novel ways about how we model irrationality. For example, in my PhD I ended up suggesting that some of the things we call ‘biases’ in psychology might be better modelled as solutions to trade-offs that are better or worse suited to different environments (rather than as strict deviations from some well-defined normative standard) - I wonder if modelling the different tradeoffs people commonly face and the way they tend to resolve them could be useful here. More generally, if AI systems are using models of rationality to predict human behaviour or cooperate with humans, and we can measure how effectively they are able to do so, this might tell us something interesting about the utility (or even accuracy?) of those different models.

Daniel Susser - Invisible Influence: Artificial Intelligence and the Ethics of Adaptive Choice Architectures

A third talk I really enjoyed focused on how data and machine learning techniques can be used for online manipulation - something I’ve been thinking about myself recently. Susser opened his talk by pointing out that while a lot is being said now about the impact of AI on structural issues (bias, power, etc.), there’s somewhat less discussion about how AI is affecting and might affect individual experience. He focuses on online manipulation, defined as the use of information technologies to impose hidden influences on another person’s decision-making, which has the potential to both undermine autonomy and to diminish welfare. In particular, new forms of online manipulation are being made possible via the use of adaptive choice architectures: highly personalised choice environments, constructed based on data about individuals, that can be used to steer behaviour. Especially as we get used to new technologies, Susser points out, they recede from our conscious attention and awareness, and so we stop noticing their impact on us. (He uses the term “technological transparency” to refer to this fact that we stop noticing technologies and how they impact us, which is somewhat confusing as the term transparency is often now used to mean the goal of making people more aware of applications of AI technologies, almost the opposite!)

I enjoyed this talk and paper, especially as I’ve been thinking about some very related issues, and I think that the kinds of manipulation made possible by even today’s available data and ML techniques is worrying, and something we need to find ways to prevent. Some of the ways that Susser presented these ideas helped me to clarify some of my own half-formed thoughts on these issues, and I think this is a paper I’ll be returning to. One of the high-level points I took away from the talk was that the ‘invisibility’ of many of the technologies that are potentially influencing our choice environments is a big part of the threat to our autonomy - and that there’s therefore a real tradeoff between the benefits of a ‘seamless user experience’, and the costs of not making conscious decisions about how we use our phones, the internet, social media etc.

A few other mentions:

  • Gillian Hadfield presented joint work with Dylan Hadfield-Menell on Incomplete Contracting and AI Alignment, which attempts to draw insights from economics for the AI alignment problem. The idea of misalignment - between individual and societal welfare - is central to ‘principal agent analysis’ in economics, they point out, and this misalignment is governed by contracts. These contracts are generally incomplete i.e. they do not completely specify all behaviour in all situations, due to our limited rationality and the fact that some things are not easily describable or verifiable. These incomplete contracts are supported by the ability to take disputes into external formal or informal enforcement mechanisms, including legal processes. The idea that we might learn something from this about how to align AI systems with our interests seems like an interesting one, and worth exploring the implications for current work on AI alignment (such as the work on inverse reward design Anca Dragan presented where reward functions are specified imprecisely but agents can then query and work with humans to figure out the best behaviour in edge cases.) The talk didn’t really get into this as much as I would have liked, but they did only have 12 minutes - I should instead probably just read the paper properly!

  • Sky Croeser and Peter Eckersley presented a paper on Theories of parenting and their application to AI. I liked this idea simply because it was an angle I’d never thought of. I wasn’t totally convinced by the analogy e.g. there was a claim that “RL agents are like rampant toddlers”, but actually I think there’s a lot of ways they are very unlike toddlers: in particular, toddlers seem to be heavily driven by curiosity, trying out lots of new things, while this is something that is missing from standard RL (there’s no intrinsic desire to explore and try new things), and needs to be explicitly built in. That aside, the parenting perspective still raised some really interesting points. For example, Croeser and Eckersley suggested that if developers thought about building AI agents more from the perspective of parenting they might invest more effort in dataset curation relative to architecture design (we tend to care a lot about ensuring our children have the right kinds of experiences to learn from, but architecture design is currently much more popular in ML - that said, we have a lot more control over architecture design in AI systems than we do in children!) They also suggested that a parenting perspective might make people more open to differences in AI development - not necessarily seeking to create AI agents that are just like us - and reconsider the problem of control - perhaps being more open, within certain constraints, to giving up control once we have achieved a certain level of trust. I think it could be really interesting to think a bit more about how far these analogies between parenting and developing AI systems go and what their limits are, especially when it comes to questions about how “like us” we want AI systems to be and how much control we should be aiming for. Perhaps it makes sense for us to be much more willing to allow for difference and cede control with our own children, because we already have a relatively high baseline for how “like us” they will be, and good mechanisms for understanding and trusting human children, which we may not have with AI systems.

  • Tom Gilbert and Mckane Andrus gave an interesting talk on their paper Towards a Just Theory of Measurement: A Principled Social Measurement Assurance Program, where they made the point that AI ethics shouldn’t just be about making ML tools fair within the constraints of existing institutions, but that we should be going a step further and trying to use these tools to make the institutions and processes themselves more just. I’m not exactly sure how we do this in practice, but I think it’s a great point and one that applies to how we think about AI ethics more broadly: a lot of discussion and writing focuses on how we ensure that AI systems don’t worsen the status quo in certain ways, but we should also be thinking about how they may be able to change that status quo.

  • Inioluwa Deborah Raji and Joy Buolamwini presented a really cool paper on what they call Actionable Auditing - in particular, looking at the impact of publicly naming companies whose products were found to have biased performance (in this case, racial bias in facial recognition models.) They found that those companies which were publicly named significantly reduced the bias in their models (without reducing overall performance) relative to those who were not named. I really liked this as showing that it’s actually possible to change companies’ behaviour in a direction that’s clearly positive, given a relatively small intervention.

  • David Danks gave a great invited talk on “The Value of Trustworthy AI”. I didn’t manage to take notes on this so I’m forgetting some of the details, but he gave a pretty in-depth, very clear analysis of what we mean by “trust” in AI and why it matters, drawing on both philosophy and psychology literature. I enjoyed how clear and precise the talk was, and Danks is an amazing speaker. By the end, though, I wasn’t sure if he’d really reached any new conclusions, or just built a much more solid and rigorous foundation below claims that we mostly all already accept and know to be true (e.g. why interpretability is important and in what contexts.) I don’t think this is necessarily a problem, and given how often these claims about trust and interpretability and made uncritically and ambiguously I think this more rigorous foundation can be incredibly useful - I’m just not quite sure how deep this foundation needs to go, and how useful it is relative to more constructive work. One new-ish thing the talk did make me think about is how we will, almost inevitably, sometimes need to build trust on something other than really understanding how a system works - and how we probably need much more research on what this might look like. This is something I’ve thought about before but seemed much clearer to me after Danks’ analysis.

A bit of AAAI

I also managed to make it to a couple of sessions at AAAI, the bigger AI conference of which AIES was a part.

The first was a panel debate on the “Future of AI”, which was surprisingly entertaining - the moderator and panelists had decided to try and make it light-hearted given it was in the early evening, and I laughed a lot more than I expected to. The proposition was “The AI community today should continue to focus mostly on ML methods.” Of course, this is frustratingly vague in a few ways - what exactly counts as “mostly”? How long past literally “today” does this extend? - but I’ll resist focusing on this. What was somewhat surprising is that a majority of the audience - 64% - voted against the proposition, and the panel seemed to come out stronger in that direction too. I’m not sure whether people wanted to be contrary or somehow ‘interesting’ or progressive in their answers, but I didn’t expect this. I’d be pretty interested to see a comparison between these votes and the proportion of the audience who themselves work mostly on methods they would refer to as ML (and plan to continue to for the considerable future...) I strongly suspect it would be more than 36%...

The main argument on the “for” side (i.e. the AI community should focus mostly on ML) was that ML is where we are suddenly making a lot of progress that shows no sign of slowing, we haven’t been doing this for all that long, and it’s currently the area of AI we understand least - and so here we should continue to focus, for now. But even those arguing this side suggested that “ultimately” we would need a much broader set of approaches, and emphasised the importance of combining ML with symbolic approaches and different kinds of structure. The “against” side began by taking a more, um, humorous approach - comparing the current focus on ML within AI research to the populist movements leading to the election of Trump and Brexit... And perhaps my favourite quote of the conference: “If you have any doubt that an AI winter is coming, just look outside: we’ve all come to Hawaii and today was a disaster!” (It had been an unusually cold and rainy day by Hawaii standards...)

One ambiguity in the proposition which I found a bit frustrating was that there was no clear statement or agreement on what counts as “ML methods” and what was considered “other approaches.” Much of the time it seemed like “ML” was actually being used to mean something more like “learning purely from data using deep neural networks.” One side would claim something like “we need to figure out how to incorporate more innate knowledge and cognitive architectures into ML approaches” as an argument against focusing mostly on ML, and the other side would just respond with “but that’s still ML!” This reminds me of a similar frustration I’ve felt when people talk about whether “current methods” in AI will enable us to solve certain problems, but it’s not really clear where the boundaries of current methods lie. Presumably many kinds of new architectures don’t move us that far away from current methods, and nor does incorporating insights from other disciplines to improve current methods... I think what people are trying to get at here is the possibility of new, deep algorithmic insights of some kind on a similar level to training neural networks using gradient descent - but I don’t think there’s any clear line here between what counts as totally novel and what’s just an adaption of existing approaches. I’d be interested to see more discussion here that explicitly picks apart different types of approach/research in AI, and different kinds of novel insight/progress that might be made, rather than talking in vague terms about “ML” or “current methods.”

I also made it to Ian Goodfellow’s keynote talk on Adversarial Machine Learning. Goodfellow essentially argued that adversarial approaches underpin (or at least can be very useful for) most of the new and important areas that ML is beginning to branch out into, now that we’ve got the basics down. The basic idea of adversarial ML, as I understand it, is to train two different ML systems (normally neural networks) which have connected and ‘adversarial’ goals, such that they continually force each other to improve. The classic example, generative adversarial networks (GANs) involves training one network that aims to classify images correctly, and another network which aims to generate images that will ‘fool’ the first network into misclassifying them. As the discriminator gets better at telling the difference between images, the generator must get better at producing ‘convincing’ images to achieve its objective, which then means the former has to be able to discriminate more finely, and so on.

Goodfellow then spent an hour going through many of the areas where adversarial ML can be useful: including in generative modelling, security, reliability, model-based optimisation, reinforcement learning, domain adaptation, fairness, accountability and transparency (FATML), and neuroscience. There were some really interesting examples here, and it helped me to better understand what adversarial ML is actually doing, and how it has applications beyond the standard panda image adversarial example. I’m naturally a bit sceptical of anything that attempts to claim that a single method, approach, or theory, can be applied to almost all things we think are important (especially if that method happens to be the speaker’s own specialism), and Goodfellow’s talk felt a little bit like that... but at the same time, contained a lot of really interesting ideas and I liked the fact it had a really clear message and cohesive structure.

Sensitivity and Resilience

Sensitivity and resilience are two character traits which often seem to be at odds with one another. When we think of sensitivity, we think of someone who tends to feel strong emotions, who is more likely to be affected by things that happen to them, someone who has a lot of ups and downs. A resilient person, by contrast, is able to go through life relatively unbothered by their circumstances, and bounces back quickly. 

I’ve always assumed there’s a tradeoff between sensitivity and resilience: both have benefits, but each comes at the cost of the other. Maybe sensitive people can appreciate pleasure and beauty more, and have more empathy for others, but this comes at the cost of more ups and downs, of struggling more. Resilient people can do much more, may never go through truly difficult times, but might miss out on some deeper emotional experiences, or find it harder to connect with people. 

I think a lot of the people in my life do broadly fit into these two categories, to varying degrees. And for a long time I think I’ve implicitly accepted that I’m always going to be on the sensitive side of the tradeoff. 

But I've been thinking recently that maybe this is an unnecessary dichotomy. I don’t think it’s easy, but it may be possible to be both highly sensitive and highly resilient. 

Sensitivity describes how much we react to what’s happening to us our around us - how much and how strongly we respond to other people and our circumstances. But resilience is more about how we respond to our feelings, than about what we actually feel. Being resilient isn't about never encountering difficult circumstances, or never feeling strong emotions. A person who never or rarely experiences difficult emotions might appear resilient from the outside - but someone who is able to experience difficult emotions and not be consumed by them, who can bounce back and keep going, shows real resilience. 

Sometimes, when I feel anxious, I also feel totally consumed by that feeling: like it’s totally controlling me, like I can’t see beyond it, and I need to do whatever I possibly can to make it stop. I’m reacting to the feeling, caught up in it, resisting it with all my might. But other times - more often, recently - I can feel just as anxious and yet somehow I have a little more distance from it. I don’t feel overwhelmed by it, and I feel like I can accept it - of course, I’d rather not be anxious given the choice, but it’s okay. I can look at it a bit more objectively, see that it’s not going to last forever, notice how it’s affecting me physically. In these moments, I feel pretty damn resilient. But it’s not really because the emotions I'm feeling are less strong. 

I worry a bit that sometimes, in an attempt to be more “resilient”, people switch off and ignore or push away strong emotions, for fear of being overwhelmed by them. And that people who are naturally sensitive think they cannot also be resilient, think that’s just a price they have to pay. There’s this sense we have to choose - to the extent that it’s under our control - between sensitivity and resilience. I suspect this is partly based on misconceptions of both sensitivity and resilience: that being sensitive means "overreacting" to things, that being resilient means "grin and bear it." But I think there's a type of sensitivity - a kind of emotional responsiveness - that is totally compatible with a certain kind of resilience - the ability to feel things but not be overwhelmed or controlled by them.

I realised this is probably part of why I like meditation so much, because it's essentially teaching you to be both more sensitive (to be more mindful of your experiences) and more resilient (to not get caught up in or resist what you’re feeling.)

Reflections on confirmation bias

Below is the postscript/"final reflections" section from my PhD thesis. I tried to write it so that it would stand fairly well on its own as a high-level summary of the issues I discuss in more detail in the thesis itself: why the evidence for confirmation bias is much weaker than most people think, and how this fits with broader narratives about the importance of open-mindedness and changing one's mind. 

At some point I'd like to write something about this explicitly for a popular audience, but until then...

This PhD has been an interesting exercise for me in changing my own mind, and trying to set aside my preconceptions. I chose to study confirmation bias because I genuinely believed it was pervasive, and at the root of many of society’s problems. I hoped that my research could help find a way to ‘debias’ people against it, to reduce this harmful source of irrationality. More generally, I had the impression that people are too slow and reluctant to change their minds, too ‘closed-minded’, and that pushing in the other direction - helping people to be more open-minded, was clearly a good thing.

However, over the course of my research, I’ve come to question all of these assumptions. As I begun exploring the literature on confirmation bias in more depth, I first realised that there is not just one thing referred to by ‘confirmation bias’, but a whole host of different tendencies, often overlapping but not well connected. I realised that this is because of course a ‘confirmation bias’ can arise at different stages of reasoning: in how we seek out new information, in how we decide what questions to ask, in how we interpret and evaluate information, and in how we actually update our beliefs. I realised that the term ‘confirmation bias’ was much more poorly defined and less well understood than I’d thought, and that the findings often used to justify it were disparate, disconnected, and not always that robust.

Reasoning that it made sense to start at the beginning of the process, I first focused my attention on selective exposure: this idea that people tend to seek out information they expect to confirm what they already believe. Though I knew that this was not all there was to confirmation bias, I thought that it was a good place to start: if people don’t even engage with different viewpoints at all, how are they ever going to be able to change their minds when they should? My focus therefore shifted from ‘fix confirmation bias’ to the only-mildly-less-ambitious ‘fix selective exposure’. But as I began exploring the selective exposure literature further, and conducting my own experiments, this also began to look misguided: it wasn’t clear from either the existing literature, or from the results of my first few studies, that selective exposure was actually a particularly strong or robust phenomenon. Was I trying to fix a problem that didn’t exist?

Unsurprisingly, at this point I found myself feeling quite confused about what I was really trying to do. I spent several months trying to make sense of the mixed findings in the selective exposure literature, and trying to square this with a belief I still struggled to let go of: that outside of the lab, people do genuinely seem to have a hard time engaging with different perspectives. Eventually I realised that the problem was that selective exposure was far too narrow, and that my measures weren’t really capturing the most important aspects of people’s motivation and behaviour. Someone could display no or little selective exposure - reading a balance of arguments from both sides - but still not really be engaging with those arguments in an ‘open-minded’ way. Equally, the arguments a person chose to pay attention to might make them look biased, but actually be chosen for good reason - based on where they genuinely expected to learn more, for example. At this point I felt that further exploring the question of whether and when selective exposure occurs wasn’t really going to help me make progress on the questions I was really interested in: whether people really are biased towards their existing beliefs, and what it really means to be open-minded.

This set me off along two closely related paths that would eventually converge, both involving taking a big step back.

First, I began exploring the broader literature on confirmation bias in more detail, along with the associated normative issues. My investigation of the selective exposure literature had made me realise that if I wanted to understand confirmation bias, I couldn’t look at different aspects of reasoning independently: I needed to understand how bias might arise at all stages of reasoning, and how these stages interacted with one another. It made me wonder whether other findings I’d taken for granted, like selective exposure, might actually be less robust than I’d thought. I also realised that there were a number of normative questions that the selective exposure research did not adequately deal with - whether selective exposure is genuinely a ‘bias’ or ‘irrational’, and what this really means - that other areas of research might address better. I had been interested in this broader debate around what it means to be rational, and whether it is possible to improve human reasoning, since the beginning of my PhD, so I decided to look into this further.

Second, I started delving into the question of what it really means to be ‘open-minded’ and how we might measure it. I was dissatisfied with the way that selective exposure was often implicitly taken to be a measure of ‘open-mindedness’: where open-mindedness seemed to me to be a much broader concept, a concept that selective exposure experiments were far from capturing. I also recognised that open-mindedness was closely related to confirmation bias, but that the term seemed to be somewhat vague, and I wasn’t aware of good ways to measure how ‘open-minded’ someone was being. I therefore wanted to explore the literature on open-mindedness to see if I could get some more clarity on the concept and its relationship to confirmation bias, and to see whether there were better ways to measure open-mindedness than simply what arguments people select to read.

On the first path - exploring the confirmation bias literature and associated normative issues - I realised that most of the findings commonly cited as evidence for confirmation bias were much less convincing than they first seemed. In large part, this was because the complex question of what it really means to say that something is a ‘bias’ or ‘irrational’ is unacknowledged by most studies of confirmation bias. Often these studies don’t even state what standard of rationality they were claiming people were ‘irrational’ with respect to, or what better judgements might look like. I started to come across more and more papers suggesting that findings classically thought of demonstrating a confirmation bias might actually be interpreted as rational under slightly different assumptions - and found often these papers had much more convincing arguments, based on more thorough theories of rationality.

On the second path, I realised that most of the interesting discussion around open- mindedness was taking place in the philosophical, not the psychological, literature. In psychology, discussion of open-mindedness largely took it for granted what it means to be open-minded, and focused on developing measures of open-mindedness as a personality trait based on self-report scales. I was more interested in whether it was possible to measure open-mindedness behaviourally (i.e. how open-minded someone is in their thinking about a given topic), which required pinning down this vague term to something more precise. The philosophical discussion of open-mindedness seemed to be trying harder to elucidate what it means to be open-minded: but in doing so, found itself caught up in this tricky question of whether it’s possible to be too open-minded, and if so, whether it is misguided for us to think we should teach open-mindedness. For a while, I myself got caught up in this elusive quest to define open-mindedness in a way that evades all possible downsides, before realising this was probably neither useful nor necessary.

All of this investigation led me to seriously question the assumptions that I had started with: that confirmation bias was pervasive, ubiquitous, and problematic, and that more open-mindedness was always better. Some of this can be explained as terminological confusion: as I scrutinised the terms I’d been using unquestioningly, I realised that different interpretations led to different conclusions. I have attempted to clarify some of the terminological confusion that arises around these issues: distinguishing between different things we might mean when we say a ‘confirmation bias’ exists (from bias as simply an inclination in one direction, to a systematic deviation from normative standards), and distinguishing between ‘open-mindedness’ as a descriptive, normative, or prescriptive concept. However, some substantive issues remained, leading me to conclusions I would not have expected myself to be sympathetic to a few years ago: that the extent to which our prior beliefs influence reasoning may well be adaptive across a range of scenarios given the various goals we are pursuing, and that it may not always be better to be ‘more open-minded’. It’s easy to say that people should be more willing to consider alternatives and less influenced by what they believe, but much harder to say how one does this. Being a total ‘blank slate’ with no assumptions or preconceptions is not a desirable or realistic starting point, and temporarily ‘setting aside’ one’s beliefs and assumptions whenever it would be useful to consider alternatives is incredibly cognitively demanding, if possible to do at all. There are tradeoffs we have to make, between the benefits of certainty and assumptions, and the benefits of having an ‘open mind’, that I had not acknowledged before.

There’s a nice irony to the fact that over the course of this PhD, I’ve ended up thoroughly questioning my own views about confirmation bias and open-mindedness: questioning my assumptions about the value of making assumptions, as it were. I haven’t changed
my mind completely - I am still concerned that in some situations, and for certain topics, people really are too dogmatic and could do with exploring more. But I’m certainly more open-minded about this than I was. Whether my increased open-mindedness is a good thing, of course, is another question.

Richard Hamming on doing important research

I've heard a lot of people talk about Richard Hamming's advice on how to do valuable research, but I only just got around to properly reading the transcript of his talk "You and Your Research." Here's a few things he talks about I found particularly interesting.

Have the courage to pursue independent thoughts, and to believe you can do important work:

“One of the characteristics you see, and many people have it including great scientists, is that usually when they were young they had independent thoughts and the courage to pursue them. For example, Einstein somewhere around 12 or 14, asked himself the question, “What would a light wave look like if I went with the velocity of light to look at it?”... He could see a contradiction at the age of 12, 14, or somewhere around there, that everything was not right and that the velocity of light had something peculiar.

One of the characteristics of successful scientists is having courage. Once you get your courage up and believe that you can do important problems, then you can. If you think you can’t, almost surely you are not going to. ”

Beware fame:

“When you are famous it is hard to work on small problems. This is what did Shannon in. After information theory, what do you do for an encore? The great scientists often make this error. They fail to continue to plant the little acorns from which the mighty oak trees grow. They try to get the big thing right off. And that isn’t the way things go.”

If you can’t solve a problem, turn it around:

“I think that if you look carefully you will see that often the great scientists, by turning the problem around a bit, changed a defect to an asset. For example, many scientists when they found they couldn’t do a problem finally began to study why not. They then turned it around around the other way and said, “But of course, this is what it is” and got an important result.”

Don’t underestimate the importance of drive and commitment...:

“You observe that most great scientists have tremendous drive. I worked for ten years with John Turkey at Bell Labs. He had tremendous drive... I went storming into Bode’s office and said, “How can anybody my age know as much as John Turkey does?” He leaned back in his chair, put his hands behind his head, grinned slightly, and said, “You would be surprised Hamming, how much you would know if you worked as hard as he did that many years.”... What Bode was saying was this: “Knowledge and productivity are like compound interest.”... The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity.


If you are deeply immersed and committed to a topic, day after day after day, your subconscious has nothing to do but to work on your problem. And so you wake up one morning, or on some afternoon, and there’s the answer. For those who don’t get committed to their current problem, the subconscious goofs off on other things and doesn’t produce the big result. So the way to manage yourself is that when you have a real important problem you don’t let anything else get the center of your attention - you keep your thoughts on the problem. Keep your subconscious starved so it has to work on your problem, so you can sleep peacefully and get the answer in the morning, free.”

...but maybe don’t overestimate it either:

“The misapplication of effort is a very serious matter. Just hard work is not enough - it must be applied sensibly.”

Learn to be comfortable with ambiguity:

“Most people like to believe something is or is not true. Great scientists tolerate ambiguity very well. They believe the theory enough to go ahead; they doubt it enough to notice the errors and faults so they can step forward and create the new replacement theory. If you believe too much you’ll never notice the flaws; if you doubt too much you won’t get started.”

And of course, work on problems you really believe are important:

“If you do not work on an important problem, it’s unlikely you’ll do important work. It’s perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them.... The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn’t believe that they will lead to important problems.”

The value in vagueness

Lately, I’ve noticed that my taste in reading material has changed slightly. It used to be that almost everything I read was a similar kind of non-fiction: the kind of non-fiction that has a very clear thesis, clear structure, and makes very clear points. I couldn’t be doing with anything that wasn’t clear - lack of clarity to me always seemed unnecessary and pretentious. Even examples and stories used to illustrate points in non-fiction books often frustrated me: I just wanted them to get to the point.

Recently, though, I’ve found myself gravitating towards more autobiographies, more fiction, more narrative non-fiction. Rather than reading blogs that attempt to provide a clear answer to a question or a clear explanation of something, I’ve been enjoying reading those that simply grapple with complex and interesting ideas without necessarily reaching any conclusion.

I think there are a couple of reasons for this. One is that I’ve more explicitly recognised that reading can serve different purposes - sometimes that purpose is to learn things, to absorb facts - in which case, clarity and simplicity can be really useful. But another reason we might read is to evoke feelings, to help us think about something on more of a gut level, or to see someone else’s perspective. And lately, for whatever reason, I’ve been more interested in finding ways to feel different things and see different perspectives than to “learn facts” in the strictest sense.

A slightly different perspective on this, though, is that I’ve begun to appreciate that sometimes vagueness has value. If we want everything we write, read, and say to be clear and concise, we’re going to be limited in what we write, read and talk about. If we prioritise clarity, we’re going to miss out on grappling with some of the most interesting ideas out there: those we don’t fully understand yet. We also risk oversimplifying and thinking we understand things much better than we do.

I read something recently about how people who are willing to grapple with and try to express feelings that they don’t quite understand yet apparently do much better in therapy than those who always seem able to express themselves clearly. Those people who feel they can only talk about things that they can express clearly are, perhaps, failing to acknowledge a whole subsection of their feelings and experiences - those they don’t understand yet - which might be the most important. In some ways, it seems obvious when put like this - you’ll only ever improve your understanding of anything (including yourself) if you’re willing to face what’s presently beyond your understanding.

Acknowledging that the world is messy, that our concepts are vague, that we really don’t understand things, is difficult. We all have a strong drive to make sense of the world, to organise things into neat patterns, to put things in boxes, to make things make sense. The feeling that things suddenly fit together and click into place can be incredibly satisfying.

What’s strange though, is that recently I’ve been finding it somehow more rewarding to grapple with complex ideas I don’t yet understand, to read something that’s thought-provoking but doesn’t have any resolution, than to read someone’s neat and simple explanation of how the world works. Given the choice between an article that attempts to lay out a clear model of how something works, and one that explores a number of connected ideas, tries to make sense of them, and raises interesting questions, the latter feels much more appealing to me. This seems to conflict with my model of how the brain works when it comes to processing ideas - that we find chaos and unpredictability deeply unsettling, and a strong drive to organise ideas so that they “make sense.” So how is it that I’m now finding it oddly satisfying, and maybe even more than that - a sense of greater meaning - from thinking about things that don’t yet make sense to me?

Part of the key here might be in that word “yet” - there’s something exciting about encountering an interesting question you don’t know the answer to if you’re anticipating that you might at some point be able to resolve it. In the same way that sometimes the anticipation of a fun event can be more enjoyable than the event itself, maybe my anticipation of understanding something better can feel as good - or better - than actually reaching the understanding. It might also be that, as I become more and more aware of just how damn complicated the world is and how little I understand, those “simple” explanations feel less satisfying, because I’m harbouring some scepticism about whether they actually explain things as well as they seem to.  

But I think there’s even more to it than this. Even though arguably, a great deal of our sense of meaning comes from this making sense of things, seeing patterns, drawing connections, it feels like I get a different - perhaps deeper - sense of meaning from realising how little I understand. This deeper sense of meaning is like a kind of awe - a sudden appreciation of how incredible and incredibly complex the world is, of how little I understand, of how insignificant I am, of how much I will never understand. And somehow, weirdly, this feels good, in a “looking up at the stars and realising how crazy it is that anything exists at all” kind of way. 

I have a sense that this ability - to let go of needing to make sense of everything, to accept uncertainty without struggling with it, to embrace it and actually see it as good - is incredibly important. It’s what allows us to not get too attached to any one perspective, to be willing to reconsider our views, to listen to viewpoints we disagree with. It’s what allows us to venture out into the unknown and discover new things, to try to understand things about the world that make little or no sense to us.

More ways of improving decision-making

I wrote the other day about a distinction between two different ways of decision-making: trying to improve lots of small decisions via ‘nudging’ (e.g. changing the presentation of options in cafeterias so more people make healthy choices) vs. trying to improve a few, particularly high stakes decisions by training people in better decision-making techniques (e.g. training members of the National Security Council to make more accurate forecasts.)

A conversation I had the other made me realise that these examples actually highlight two important distinctions - and therefore, potentially, four different ways of thinking about improving decision-making. There’s one distinction between different methods we might use to try and improve decisions, and another distinction between different kinds of decisions we might target for improvement:

  1. Different methods: improving decisions via “nudges” vs. teaching people better decision-making strategies

  2. Different kinds of decisions: improving lots of small decisions a small amount (thousands of people eat more healthily) vs. improving a few important decisions a larger amount (govt spends important resources on more effective health interventions.)

I only focused on two possible combinations of these two variables - improving lots of small decisions via nudges, and improving a few important decisions by teaching better strategies. But the other combinations are also possible - we could try to improve decisions made by the general public via training (by teaching better decision-making strategies in schools, for example), or we could try to improve a few very important decisions via small “nudges” in key environments (by, um, putting a picture of a pair of eyes over the Prime Minister’s desk?) To make this clearer, let’s put it in a 2x2 matrix, because everyone likes those:

'Nudging' Training
Lots of small decisions Classic “behavioural insights” work - e.g. changing the wording on a letter so more people change their taxes, changing the display of food options so more people make healthy choices Adding “critical thinking” or other “rationality training” into school curricula, running workshops that help people make better decisions in their lives
A few big decisions Make relevant evidence more easily available and understandable to policymakers, create social rewards for using certain procedures Training influential decision makers (e.g. in tech companies or government) to recognise and avoid cognitive biases or other bad thinking habits

The bottom left square - trying to improve a few very important decisions by “nudges” - seems particularly interesting to me, because it’s perhaps the least obvious or least discussed. It’s not totally clear to me what “nudging” influential decision-makers - e.g. policymakers in government - would look like, but it certainly seems plausible that one could find ways to tweak the environments in which important decisions are made (by changing regulations, processes, or salient ideas/information) in ways that would result in a subtle shift of incentives, thereby really improving the quality of decisions made.

I recently wrote about improving institutional decision-making as a high impact cause area for 80,000 Hours, where I focused mostly on trying to get better decision-making techniques implemented - i.e. the bottom right cell of the above matrix. One piece of pushback I got on this was that this is incredibly difficult to do in practice, because policymakers and other influential decision-makers simply don’t have much incentive to use costly/effortful new strategies, and there are plenty of bureaucratic barriers to doing so. I agree that this is a concern, but I also wasn’t sure what on earth “changing incentives” could look like in practice. One thing it might look like, though, is nudging - making subtle changes to the environment in which decisions are made that make ‘better’ decisions easier. A huge advantage of “nudging” approaches over “training” approaches, long recognised by the behavioural science crowd, is that they don’t require much if any effort from the people whose decisions are being improved.

Of course, the fact that “nudging” often doesn’t even require awareness on the part of people whose decisions are being targeted, is also the reason it’s sometimes ethically dubious. However, I don’t think this is as much of a concern as it seems for nudging institutions towards better decisions, for a couple of reasons. First, if we’re going to try and ‘nudge’ important institutions/teams towards better decisions, those institutions/teams will presumably have to be a lot more involved in doing this than the general public are in the kinds of policy nudges currently employed. I think this doing this kind of thing would look a lot more like a few key specialists in an organisation coming up with proposals for how the organisations’ processes and environment could be subtly changed to incentivise better decisions. This would inevitably have to be approved by at least some of those who would be affected, while still not requiring much effort from them beyond that.  Second, ‘nudging’ for better decisions here would probably focus on improving the processes by which decisions are made, and building the capacity of groups to make better decisions, rather than on improving the outcomes. For example, the focus might be on making it easier for policymakers to make use of relevant evidence when making decisions, or to use certain systematic processes for assessments.

Does a good career need to tell a good story?

I’ve written before about the difference between people who like to think of their lives as a story (“narratives”), and those who see their lives more as a series of disconnected episodes (“episodics.”) 

I’ve been thinking recently about this in the context of career choices. I suspect that a lot of people think about their careers in this narrative sense - what step makes sense next, given what I’ve done so far? Where do I want to end up in 5, 10, or 20 years time, what kind of story do I want to be able to tell about what I’ve done, and how I got to where I am? 

I’ve certainly been noticing this kind of thinking in myself. I’m thinking about what to do next after my PhD, and I’ve been finding myself drawn to options that feel like a good next step in my story, while feeling some resistance to make choices that don’t seem to produce such a good narrative. I’m a bit worried about this, because I’m not sure “telling a good story” necessarily tracks that what I care about - having a career that I enjoy and that has an impact in the world. 
The best way for me to do valuable, fulfilling work might well be to do something that makes a bit less sense, that doesn’t tell such a good story. And yet I do still feel this strong pull towards doing whatever makes for a good story.

There are certainly some reasons why optimising for good storytelling could be a good way to think about your career. It helps you to sell yourself to other people, to ensure you’re developing expertise in a specific area rather than just jumping around randomly, to use and build on what you’ve learnt at each step. This all makes sense. But sometimes this can go too far - wanting to tell a good story might encourage you to stick on a path even if you don’t enjoy it anymore, or lead you to do things that will “justify” past decisions in an irrational way. Having just spent the past few years getting a PhD, I now find myself more attracted to options which require a PhD. These options would help me justify (to myself and others) why spending all that time on a PhD was clearly worth it. But this doesn’t actually make any sense. It’s great that I now have options I wouldn’t have had without a PhD, but if the very best option is something I don’t need a PhD for, it seems crazy to turn it down just for that reason. 

I worry a little bit that many people’s career choices are driven too much by the desire to tell a good story, and this prevents them from considering or choosing otherwise great options that don’t fit into such a great narrative. Maybe we can counteract this by finding ways to tell compelling, unconventional stories about not-so-neat career paths - paths that involve exploring a variety of different options and industries, combining unconventional skills, doing lots of small valuable things rather than one hugely influential thing. I’d personally like to hear more stories of people who have had careers that don’t necessarily fit together neatly, people who spent a few years doing something entirely random, but who don’t see that random thing as a mistake or a waste of time.

Relatedly, I’ve also noticed how difficult I’ve found it meeting new people while I’m exploring and figuring out what to do - because I don’t feel like I have a good story I can tell about what I’m doing now and where I’m going. I find myself trying to fit my introduction into some kind of narrative that makes sense depending on who I’m talking to and what background information they have. Maybe the biggest reason that stories are so important to us, especially when it comes to work, is that they form part of our identities. Having a neat, clear story I can tell about my life helps others to make sense of me, and perhaps even helps me to make sense of myself. But none of us fall into neat, clearly-defined identity boxes anyway - maybe it would be better if we stopped trying to.

Two ways of "improving decision-making"

People sometimes talk about “improving decision making” as a way to improve the world - if we could find ways to overcome the various ‘biases’ and ‘irrationalities’ that people are prone to, we’d be better able to solve some of the world’s most important problems. I think that there’s promise here, and I’d like more people to be focusing on this. But I also think that, as stated above, this project is too broad and vague to be tractable. I’d like to be able to say something a bit more concrete about what working on this problem might look like, and to begin with, I’ve found it helpful to distinguish between two different types of “improving decision-making.”

The idea of improving policy-making using “behavioural insights” has been gaining popularity in government over the last few years - largely due to the work of the UK Behavioural Insights Team (BIT), and other smaller groups and organisations doing similar work. (Disclaimer: I worked for BIT for ~1 year during my PhD.) The basic idea here is that we can use an understanding of behavioural science to design policies that “nudge” citizens’ behaviour in better directions: helping people to eat more healthily, save more for retirement, or get back into work quicker. By improving the design of policies that affect thousands or even millions of people, we can make the world better by improving many, many small decisions in people’s lives.

I think this work is clearly valuable, and I’m glad there’s more focus on it (setting aside potential ethical issues with governments deliberately influencing citizens’ behaviour - I think there are some legitimate worries here but in practice most of this work is defensible.) But there’s also a second way to improve decision-making, a second way of applying “behavioural insights” to improve policy, that I think might be even more valuable, and hasn’t gotten as much attention.

In addition to improving the design of specific policies, we could also apply insights from psychology to improve the processes by which policy decisions are made. Rather than trying to improve lots and lots of small decisions, we could focus our efforts on a few very high-stakes decisions, the decisions made by people in powerful positions most likely to affect humanity’s future. This might be more challenging than small nudges, but might also be much more valuable in the long-run. As technology gets more and more advanced, potential worst-case scenarios from conflict are growing in severity - with nuclear weapons, we have the ability to wipe out millions or even billions of humans, and advances in AI and biotechnology may pose new unprecedented threats. This makes the decisions of powerful institutions all the more crucial, and improving their decision-making competence all the more valuable.

These kinds of “high stakes decisions” - deciding how to respond to threats from other countries or terrorist groups, or deciding how to prioritise government’s scarce resources - are of course much more complex than the decisions most individuals make on a day-to-day basis. In the case of improving citizens’ decisions, generally it’s objectively clear what the “better” decision is (and this is part of how we defend the ethics of nudging) - people making healthier food choices, keeping more people in work, or widening participation in higher education, are all pretty uncontroversially good for society. When it comes to complex and high-stakes government decisions, it’s less often the case that people struggle to make what’s clearly the best decision - and more often that it’s incredibly difficult to know what the best decision is at all.

Perhaps it’s helpful here to additionally distinguish between two types of human irrationality. In some cases, we sort-of-know reflectively what the best decision or answer is, but short-term focused heuristics and incentives mean we fail to act accordingly - I know that I’ll feel better in the long-run if I exercise and keep my finances organised, but I often feel more motivated in the moment to spend money on fancy ice cream than to go running. But for other kinds of problems, it’s incredibly difficult for us to know what the right answer to a question or best course of action is, even reflectively, even given a lot of time to think about it. How advanced is North Korea’s nuclear weapons programme? How likely is it that there will be a nuclear attack on the US in the next two years? Part of the difficulty with answering these questions is incomplete information, of course, but there’s also the fact that our brains naturally struggle to combine large amounts of information at once, to think probabilistically, to see the implications and inferences one should draw given various different pieces of data, and so on. Even given a great deal of relevant information, and enough time for reflection, intelligent people will fail to make accurate judgements about complex problems, especially those involving predicting the future. This is largely a problem of limited cognitive ability, and so quite a different type of “irrationality.”

This means the best approaches for improving the ability of powerful institutions to make high-stakes decisions are likely to be very different from the best approaches for improving small decisions people make on a day-to-day basis. Simple nudges - making the obviously best option easier or more attractive - aren’t going to cut it. There might be some low-hanging fruit in terms of removing impediments to better decision-making: using checklists is surprisingly effective at reducing simple errors, for example. And I think continuing to push for more evidence-based policy is likely to be very valuable, which already has a fair amount of traction. But ultimately I think better institutional decision-making will be less straightforward - most of the best-established techniques from improving judgements and decisions (e.g. from the literature on forecasting and improving calibration) seem to require a fair amount of conscious effort and training on the part of decision-makers. There’s also the issue of incentives - arguably we already know a lot about how to make better decisions, and the reason they’re not used is that influential decision-makers face bureaucratic barriers and competing incentives which mean it’s not in their personal interests to do so. This means that if we want to improve decision-making e.g. at high levels of government, it’s not enough to just understand some techniques that have performed well in academic contexts - this research needs to be combined with an in-depth understanding of how bureaucracies work, and we need to find ways to align better decision-making with other incentives. To see better decision-making techniques adopted in government, I think we’ll need to find ways to show decision makers that these techniques will actually help them achieve their more immediate objectives, whatever they are. 

None of this is easy, especially compared to changing the wording on a letter to increase response rates. So it’s not surprising that people interested in improving decision-making have focused much more on simple nudges like changing the wording on a letter. But I think that psychology research actually has a lot to say about this second type of improving decision-making - improving the processes by which people make complex, high-stakes decisions where there’s no obvious “correct” answer - as long as we acknowledge the practical complexities and avoid oversimplifying the problem. I think it would be really valuable if there was more collaboration between social scientists and people actually making important decisions, and more discussion of ways we could improve the quality of the institutional decision-making. And none of this is really meant to criticise the way government currently makes decisions - I certainly don’t know enough about it! - but just to recognise that humans are imperfect, there’s always room for improvement, and focusing our effort on the highest-stakes decisions might be particularly important.

Dangerous drives

I really like this speech by C.S. Lewis. It’s about the tendency to form “Inner Rings” - informal groups and hierarchies, impossible to pin down precisely, but which exist everywhere - in all schools, organisations, and societies. Lewis argues that the drive to be “on the inside” of some Inner Ring is a more fundamental human drive than most people think. He also thinks it’s a dangerous one.

“Unless you take measures to prevent it, this desire is going to be one of the chief motives of your life, from the first day on which you enter your profession until the day when you are too old to care.”

Why does Lewis think this desire is so dangerous? He gives two reasons:

First, that, “of all the passions, the passion for the Inner Ring is most skillful in making a man who is not yet a very bad man do very bad things.” For almost all people, he says, the choice that might lead them down a bad path will not be an obvious or dramatic one. It will be “the hint of something which is not quite in accordance with the technical rules of fair play: something which the public, the ignorant, romantic public, would never understand...” It will be something which, a friend tells you, “we always do.” And even if something feels a bit off, you might ignore that feeling and do it anyway, because if you were to refuse, you’d feel thrown out - no longer part of that “we”, thrust out of that Inner Ring you so desperately want to be part of.

The second reason Lewis gives is subtler - but I think perhaps all the more dangerous. For “as long as you are governed by that desire,” Lewis cautions, “you will never get what you want... until you conquer the fear of being an outsider, an outsider you will remain.” If you seek recognition within a group simply for the sake of the boost you get from being an “insider” - and not because that group provides you with something you value - then you will never be satisfied. Once you’re “in”, the circle will quickly lose the charm it had from the outside. You’ll soon find some new, smaller, more alluring, or higher-status clique to pine after.

So what’s the alternative?

“The quest of the Inner Ring will break your heart unless you break it. But if you break it, a surprising result will follow. If in your working hours you make the work your end, you will presently find yourself all unawares inside the only circle in your profession that really matters. You will be one of the sound craftsmen, and other sound craftsmen will know it.

And if in your spare time you consort simply with the people you like, you will again find that you have come unawares to a real inside: that you are indeed snug and safe at the center of something which, seen from without, would look exactly like an Inner Ring. But the difference is that its secrecy is accidental, and its exclusiveness a by-product, and no one was led thither by the lure of the esoteric: for it is only four or five people who like one another meeting to do things that they like. This is friendship.”

I think, more generally, we have motives that point at things we actually value, and motives that are more illusory - and the latter can be dangerously alluring. We’re driven to impress, achieve, make money, acquire status, be on “the inside” of a particular group. These things feel good, especially in the short-term, but they’re deceptive - they’re never really satisfied, and we’ll just keep chasing them. These drives can be harmful, as Lewis suggests, because they can lead not-bad people to do bad things. But I think the even greater harm of these drives is that they distract us from what actually matters - the drive to be impressive distracts us from actually getting good at something, the drive to be “on the inside” gets in the way of forming genuinely rewarding relationships.

It seriously worries me how much of the time most of us spend chasing things like achievement, status, recognition - and how little we spend thinking about what we actually value: what we’d find rewarding in a job, what kinds of people make us feel good, what we really want in a relationships. It’s easy, in a sense, to just go after the job that makes the most money, the friendship group that makes us look coolest, the partner who is most attractive. Asking ourselves what we actually want and value in life, and then trying to find that, is much harder.


A while back a friend introduced me to Vectors by James Richardson - a nice little book of aphorisms and very short essays. He's a wise guy, I think (both Richardson and my friend :)). Here are a few that I especially liked:

Later you will not want it. Does that mean you should take it now, or let it go?

I trick myself into sins I could not forgive myself for intending. If I could depend on myself for a little mercy, I would perhaps not have grown so expert in the self-deception that makes it so difficult minute to minute to know what I am really doing.

You keep track of your worth on some wildly cyclic stock market that will soar in fantasy, crash at a cold glance. Other people think you never change.

Say too soon what you think and you will say what everyone else thinks.

More dangerous than the worst is the pretty good you can no longer tell from the best.

So many times I've made myself stupid with the fear of being outsmarted.

I am not unambitious. I am just too ambitious for what you call ambitions.

The days are exactly alike in that they repeat nothing exactly.

I'm sitting here bored, trying to understand. No: trying to remember that everything is a mystery.

My favourite books of 2016

Ok, so this is a bit late - but here are three books that really stood out for me last year. Part of the reason I've delayed this post is I wanted to write in more detail about each of these books, but since I haven't gotten round to that, I at least wanted to briefly share these:

1. Impro -- Keith Johnstone

A book about theatre and improvisation that's about so much more than theatre and improvisation - a book about social interaction and identity, how we perceive ourselves and others, and the things that constrain us. I found this incredibly dense with insights and wisdom - I'm not sure I've ever highlighted/commented in a book so much, or so strongly felt I needed to read a book multiple times to even begin to make the most of it. Keith Johnstone's TED talk is also excellent - definitely my favourite TED talk ever.

I don't even really know where to begin explaining what I got from this book , and I hope to write more about it at some point. I think the biggest thing for me was just realising how much I'm holding back all the time out of fear of being judged, how afraid I am to express myself, of doing the wrong thing, of allowing myself to look silly.

Thanks to Uri Bram for buying this book for me years ago, and somehow knowing it was exactly what I needed well before I was able to see it myself. I somehow couldn't get into it until this year - I think I just wasn't quite in the right place. 

2. I Am A Strange Loop -- Douglas Hofstadter

This is the book Hofstadter wrote years after Godel, Escher, Bach - in response to the fact he felt no-one really got what he was trying to say with GEB (despite the fact it was so popular.) I'm actually only reading GEB now, after having read this - I figured that if Strange Loop was supposed to be a clearer exposition of Hofstadter's ideas, it made sense to read it first. I feel like this was a good decision - I enjoyed it even more than I expected to, and I think I'm finding GEB easier to get into now - I tried reading it a few years ago and it somehow just didn't grab me enough.

I think part of the reason I liked this book so much is it really pushes all my nerd buttons. I've always been fascinated by philosophical questions of consciousness and identity, as well as really enjoying mathematical logic and weird semantic paradoxes and self-reference. These had always seemed quite separate to me - so someone coming along and proposing to link them together, to explain consciousness using logic and self-reference, feels so satisfying to me. Hofstadter is also just an excellent, engaging writer - I think he uses stories and analogies in a really skilful way to make tricky concepts accessible and engaging.

I don't necessarily think Hofstadter manages to crack consciousness completely - but he certainly gave me a different way of thinking about it that I find really useful. Again, hope to write more about this at some point...

3. The Center Cannot Hold -- Elyn Saks

I got really into reading autobiographies this year - particularly memoirs of mental illness. This was my favourite: an autobiography of a woman with schizophrenia. Something I really enjoy when reading is feeling like I'm getting to understand a totally different perspective (which I think is why I've enjoyed autobiographies so much.) I particularly enjoy memoirs of mental illness just because they give me insight into the different ways that people struggle - it's not even that they're about mental illness per se, but just some ways in which the author has found life difficult. I like the honesty and openness that comes with this kind of writing, and I find it helpful to hear others acknowledge just how difficult it is to be human sometimes.

I particularly loved this book, I think, because (a) I knew very little about schizophrenia before, so it was really interesting to learn about, and (b) I was surprised just how... identifiable a lot of the author's experiences sounded, despite the fact I've never experienced anything close to psychotic symptoms. Hearing someone describe how they ended up sitting on a floor in a mental hospital just rocking back and forth moaning for days on end, and thinking, I can totally see how she ended up there, made me extra-aware of just how grey the line between "mentally healthy" and "mentally ill" really is. It made me more aware of how fragile the mind is and how fragile our grip on reality can be. Our brains are generally very good at making sense of the world, of filtering out nonsense, of organising our experiences in a coherent way. I think we take for granted how much hard work is going on here, and it's just not that surprising to me that sometimes these things can break down. So much of what Saks describes just sounds to me like these basic abilities failing in small ways - her brain struggling to filter out what's irrelevant, to zone out silly thoughts that make sense, to organise reality in a neat way. And it just doesn't seem that weird or surprising to me that this could happen - which in turn I think makes 'madness' look much less mad.

Thanks to Kate Donovan for the recommendation!

(I'm conscious of not wanting to sound like I'm oversimplifying schizophrenia - I'm aware I still really have no idea what causes psychosis or what it feels like - but this is just the impression I took from this book.)


I've also found it interesting to notice myself drawing connections between these three books, despite the fact they seem really different on the surface. On some level, they're all about how we organise and make sense of the world, and particularly how we conceptualise ourselves. Strange Loop is about where our sense of identity comes from - how this arises from our ability to abstract away from the basic elements of reality, forming higher-level concepts including a concept of ourselves - and how the weird feedback loop this creates (my perception of myself feeds back into the things that I think and do which then feed back into my perception of myself...) may help explain what gives rise to conscious experience.

A lot of what I took from Impro had to do with how this ability to perceive ourselves, and think about how others might perceive us, can constrain us. A lot of what improvisation is aiming to do seems to be teaching people to 'let go' of a maintaining a certain self-image, or the need to organise reality in certain neat ways. 

The image of schizophrenia I took from The Center Cannot Hold seems to say something about what happens when the brain's basic abilities to organise and make sense of the world aren't able to function properly: when the brain struggles to organise everything in a coherent manner, when it struggles to maintain a stable self-image, when it can't quite filter out what's relevant from what isn't. All of this just makes me aware of how crucial and yet fragile these abilities are, and how much we take them for granted.


When I go for a long run, often I find the latter parts much easier than the beginning. This seems strange in a way: you'd think that as my body got more tired, it would become more of a struggle. But it's not really about how my body feels at all - my mindset is much more important. When I've been running for half an hour or so, my mindset shifts. I feel like what I've already done is enough, that I could stop right now and still feel I'd done a decent run. Strangely, this makes it much easier to continue, because anything I do beyond this point is a bonus. With the feeling of having done enough, a lot of pressure and anxiety I didn't even acknowledge I was putting on myself is lifted. I'm no longer worrying that I might get too tired: I'm already tired, but it's ok. I'm not counting the minutes until I can say I've reached a reasonable time. I just run, one stride at a time, and start to enjoy the rhythm of it. 

I think this tendency is pervasive in other areas of my life too, especially work: a feeling of anxiety about whether I'm doing "enough" gets in the way of my natural motivation to do things. It's common for me to worry about whether I'm being productive enough, working enough hours, getting through enough tasks. I'll often have a specific goal in mind I feel I need to meet in order to be satisfied, and until I reach that point, my motivation is coming primarily from a place of anxiety. Fear that if I don't keep going and reach a certain point I'll feel a failure, my day wasted. But once I get to the point of "enough", I can let go - and ironically, it's often then that I do my best work. It's often then that I really enjoy what I'm doing, and feel better able to identify and focus on what's important. 

It’s when I’ve already met all my goals for the day that I feel like I want to get ahead for tomorrow. It’s when I’ve already written a blog post for the day that all these other ideas I want to write about start streaming in. It’s when I’ve already been to the gym in the morning and ticked off my “exercise” goal for the day that I really feel like going for a swim in the evening.

It’s like all of this anxiety about whether I can meet a given standard is getting in the way of my intrinsic motivation to do things. I’ve realised recently how even things I genuinely want to do can end up feeling aversive, like a burden on me – because my brain quickly and naturally develops a feeling of “should” around any goal I set myself. It feels like this stems from a deep, vague, fear that I’m somehow not good enough – not until I’ve worked enough hours, run far enough, achieved enough.

I wonder what it would feel like not to have this – to simply wake up and feel like I’ve already met this standard of ‘enough’, to always feel free to do things because I want to, because they feel important – not because I should

It’s interesting to ask where this bar for what’s “enough” comes from, and what might shift it. I think it's partly influenced by societal norms and culture. When I was working an office job, for example, I started to internalise the idea that as long as I sat at my desk doing vaguely productive work from 9 til 6ish, I was doing enough. That’s what others around me were doing, and what they thought was enough, after all. Doing a PhD, there’s risk that it never feels like I'm doing enough – there’s always something else that needs doing, my incomplete thesis looming in my mind. And the more I spend time around super ambitious and hardworking people, the higher my standards for what’s 'enough' get. I find myself frequently asking: what do I imagine [absurdly-competent-and-productive-person] would do in this situation, how high would their bar be?

My standards also shift as my expectations for myself change, based on what seems 'good' for me at the moment. Having struggled with motivation a bit recently, I got to the point where even managing a couple of good productive hours a day felt ‘enough’ – because that was the best I’d been achieving recently. But as soon as I had a few good days, my standards started to rise – and suddenly what had been good enough a few days ago no longer was. In a sense, the fact that my concept of what’s enough shifts so easily, and is so relative, should be enough to convince me that it’s not really rooted in anything real – nothing beyond my own self-judgement.

I so badly want to live more of my life in a state of ‘enoughness’, where my motivation comes  from things I genuinely care about and want to do, not fear of failing to live up to some standard. The anxiety I feel when I’m scared I might not do enough is what so often gets in the way of achieving more. The anxiety that I might get tired before I’ve run enough is most of what makes the running unpleasant and hard, which is what makes me want to stop. The anxiety that I might not be able to finish a project to a good enough standard, or fast enough, is what makes me procrastinate. I've sometimes said to friends that I know I could achieve so much more if I wasn't doubting myself all the time.

I don’t really know yet how to deal with this, to be honest. The short-term solution is to try and set low standards for what’s ‘enough’, and find ways to make sure I can meet them. For example, I’ve recently been starting work earlier in the day before doing other things, so that I end up feeling like I’ve done ‘enough’ earlier in the day – and sooner get to a place where I’m free from that pressure.

But this really feels like just a bandaid: working effectively within the constraints of feeling not good enough, while continuing to feed the feeling. Maybe it's naive, but I do believe that I could free myself from these constraints entirely: completely lose the anxiety, the self-judgement, the feeling that I’m not good enough until I’ve achieved enough. I don't like waking up every day thinking I need to prove myself. Perhaps the biggest barrier is the fact that part of me is still afraid: afraid that if I let go of the anxiety and pressure, I might just not achieve anything. I'm scared that if I find some way to feel good enough without achieving anything, then, well - I might not amount to anything. And maybe that wouldn't be enough.

Questioning yourself

Sometimes the most useful ‘advice’ someone else can give you isn’t advice at all - it’s asking you the right question. Good questions can help alert you to things you already knew but hadn’t quite seen the implications of, help you to consider a different angle on a problem, or simply help you better structure your thinking.

As I suggested in my last post, often when we’re struggling with a problem or a decision, what we need isn’t other people’s opinions exactly. Instead, what we need is help making sense of everything we already know, help figuring out how to balance our own conflicting intuitions and feelings. And a good question can really help to do this, to re-orient your thinking.

But of course, it’s not only other people who can ask you questions - you can also question yourself! I think it often doesn’t occur to us to explicitly question ourselves, because it feels a bit strange. There’s this sense we have, that if we could ask ourselves a question and answer it, then we should be able to jump straight to the answer. Since all of this is going on within the confines of our own minds, the idea of being able to 'uncover' something we didn't already know can intuitively seem a bit odd.

But our minds aren’t totally transparent to us. This isn’t some deep Freudian-esque claim about unconscious motivations and desires. It’s just a fact that our brains can only hold, and process, a limited amount of information and considerations at once - we can’t immediately see the logical consequences of everything we believe, or see which of our beliefs are inconsistent with one another. We don’t even really know what we’re feeling, or why, a lot of the time. But in some sense, all of this information is there for us to access, and asking the right questions can help us to do this.

Realising this, I’ve started collecting together lists of questions that I might find it helpful to ask myself in different scenarios: when I’m struggling with a decision; when I’ve lost motivation and don’t know why; when I’m setting goals; when I’m feeling low for no clear reason. Part of the reason other people can often ask us better questions than we can ask ourselves is they have a different perspective. Sometimes when you’re in the throes of a tricky problem it’s sometimes hard to think clearly enough about what you need to ask yourself (especially if you’re also experiencing difficult emotions.) So it seems really useful to have a bank of questions prepared in advance that you can turn to in these moments, when you're not really sure where to turn next.

I’ve started collecting some questions like this here - feel free to take a look and suggest additions, I’d love to hear what other people feel are the most useful questions they ask themselves or have been asked by others.

Of course, these questions are fairly general, and often the most useful question for a given problem is something quite specific to the scenario. So perhaps the most useful question you can get into the habit of asking yourself is, “What’s the most useful question I could be asking myself right now?”


Asking for advice

I’m lucky enough to have a lot of friends who I think are very smart, thoughtful and have good judgement (of course, I’m biased.) This means that whenever I have a problem or a decision I’m struggling to make, I instinctively want to ask lots of different people I know for advice. I also sometimes find that my friends think I might be able to help them solve their problems, and so they ask me for advice.

I’ve been thinking about what makes advice helpful - both so I can get the most out of advice from others for my own problems, and so I can be as helpful as possible to other people. When asking for advice from others, I think there are three main things I might be looking for:

  1. Factual information/knowledge. Sometimes the reason I’m struggling to make a decision on something is just that I’m missing a key piece of information, and someone I know might just be able to give me that information because they’re more knowledgeable on a topic than I am.

  2. Help structuring the problem.  Sometimes what I need isn’t information, but simply help thinking through the different considerations and information I already have. Other people can help with this because it can be easier to think clearly about a problem that isn’t your own, or because they might have experience with a similar problem themselves and found a useful way to think about it. A similar thing I’d put in this category is help thinking through the relevant considerations - sometimes someone else will alert me to an important question or factor I wasn’t already thinking about (which is importantly different from giving me a piece of factual information or knowledge I didn’t already have.)

  3. Actual opinions. Sometimes what you want is just to tell a bunch of different people about the dilemma you’re facing, and then ask them what they think: what would they do in this situation, or what do they think you should do?

If you’re asking someone for advice, it can be really important to be clear on which of these three things you’re looking for (of course, you might want two or even three of these things.) This helps you to clarify where your uncertainty lies on the problem - is it that you think you’re lacking specific knowledge, do you need help thinking through the problem more clearly, or do you just want to know what this person’s opinion is? This can also help the other person, and avoid a situation where you really want one thing and they assume you’re looking for something completely different - e.g. you’re looking for help structuring the problem but they’re giving you lots of factual information you actually think is irrelevant, or them giving you their opinion when it’s not helpful/wanted.

But the main  point I want to make is that, when we think of asking for advice, I think we default to thinking of it in terms of 3. - we’re looking for other people’s opinions, to then help us form our own judgement. But I actually think this is the least useful and most difficult to deal with of the above three.

A problem I frequently come up against is this: I have a decision to make, and I talk to several different people about it, asking for their opinions. Inevitably, I get conflicting advice from different people, which puts me in a bit of a bind. If one smart, thoughtful person thinks I should definitely do A, and another smart, thoughtful person thinks I definitely shouldn’t do A, what should I do? Should I somehow try and weigh their opinions based on their relative smart-ness and thoughtful-ness? Should I go and ask a bunch more people for advice on how they deal with conflicting advice from other people...?

It’s hard for me know what to do with someone’s opinion - and how much weight to give it relative to other opinions - if I don’t know the process that led them to it, what assumptions were made, what information they had. I don’t know how to resolve disagreement in opinions between two people, and therefore who to agree with, if I don’t know exactly where their disagreement lies.

So if you want to integrate different people’s opinions, you really need to know (a) what process/structure they’re using for thinking about the problem, and (b) what assumptions they’re making and what information they do and don’t have. But then, you’d be better off focusing directly on these things - asking other people to help you structure your thinking around the problem, challenge the assumptions you’re making, suggest additional considerations, and give you any factual information you might be missing. That is, when asking people for ‘advice’ - to help you make decisions - it seems much more useful to explicitly ask people for help structuring the decision, or to point out considerations or bits of information you might be missing - than to ask them for their ‘opinion.’

This also lessens the pressure on advice-givers. If you think that someone wants you to tell them what they should do with their life, that can feel pretty daunting! If instead what they want is just someone to help structure the problem clearly, or your knowledge on a specific part of the issue, it’s a lot easier to know how to be helpful.

I think sometimes when I start asking people for their opinions - rather than help thinking through the decision myself, or additional information - I’m actually looking for reassurance more than anything else. Dealing with uncertainty can be scary, and sometimes I’d like to outsource the difficulty of it to someone else, so I can absolve myself from the responsibility and pressure of it. But of course, when I get conflicting suggestions from difficult people, this just puts me back in charge and often in a position of even greater uncertainty. I’d be better off taking more responsibility for the decision myself - accepting the uncertainty, doing my best to navigate it myself, getting support from others around me - not trying to pass the uncertainty onto someone else. 

What makes writing hard?

A lot of people say they find writing hard. But what exactly makes writing so difficult? Just putting random words on paper is easy. What’s difficult is turning those words into sentences that really express the ideas you’re trying to convey. When we talk about difficulty writing, we often say that “the words aren’t coming out right” - we’ll write a sentence, and then delete it because we don’t like how it sounds, then sit staring at a blank piece of paper not knowing how to re-write it.

Writing is all about communicating ideas to others. This means that a really key aspect of writing, before you even get to putting pen to paper (or more often, fingers to keyboard), is figuring out exactly the ideas you want to communicate.

I think it’s useful to explicitly distinguish two different stages of the writing process:

  1. Figuring out yourself what ideas you want to express, and making sure those points are clear in your own mind;

  2. Working on communicating those ideas to others - how to do so clearly, how to keep the reader engaged, perhaps how to evoke certain emotions in your reader, etc.

This might sound obvious. And technically speaking, only 2. is ‘writing’. But I think learning to do 1. well is a vital part of being a good writer. If the ideas you want to express aren’t even clear in your own mind, how on earth are you supposed to communicate them to others?

Based on this, I’ve recently tried distinguishing two very different types of writing. The first type is writing I do to clarify my own ideas: I’m putting words onto paper without worrying how they look,  expressing ideas in the order they come to me rather than the order that I’d want someone to read them. This is “stream-of-consciousness-style” writing: the kind of writing I do when I’m writing in a journal, or when writing a quick email explaining something to a friend. The second type of writing is writing I do to express those ideas to others: I’m taking already-clarified ideas and thinking about how to structure and present them in a clear and engaging way. This is “presentation-style” writing. With stream-of-consciousness-writing, I do very little editing or backtracking. Presentation-style writing, by contrast, involves multiple stages of revisions: first, I get the basic ideas down as clearly as possible, then I review it and think about how to edit those words and change the structure to make it clearer and more engaging.

I think that one reason many people find writing so hard is that they miss step 1. entirely. They’re trying to do “presentation-style” writing, when really what they need to be doing is “stream-of-consciousness-style” writing. It’s really hard to focus on both the content of what you’re expressing, and how that content is presented at the same time. So if you’re spending attention thinking about presentation, you’re going to have much less left over for thinking about content. The idea behind stream-of-consciousness-style writing is that you forget about presentation temporarily, so that you can focus all of your attention on the content of what you want to express.

Most writing advice focuses on presentation-style writing: how to communicate ideas through writing clearly and engagingly. But actually, in my experience, the actual process of writing is relatively easy if you do step 1. right - if you’re really clear what ideas you want to communicate. Or at very least, doing step 1. well makes the actual writing part a lot easier. It might not turn you into a literary genius, but it makes it a lot easier to express what you want to say clearly, which you can then build on.

When I had to write essays at university, I’d spend the bulk of my time on “planning” - scribbling ideas down with pen and paper to figure out exactly what points I wanted to make, and then making detailed notes on how I wanted to structure and present those ideas. Once I’d done that, the actual writing was easy - it was just a matter of stringing those ideas together into sentences. People would be impressed when I said I could write a 2,000 word essay in not much over an hour - but what they were missing was how much preparation had gone in before this point.

A lot of the time, I think that when people say they find writing difficult, what they’re struggling with isn’t creating evocative and beautiful prose from simple language. What they’re struggling with is expressing their ideas in simple language in the first place.

Just to get really meta, here’s the process I used to write this blog post:

  1. First, I spent somewhere between half and hour and an hour on stream of consciousness-style writing: literally just writing words onto paper in the order they came into my head to figure out what ideas I wanted to express. If I found myself not knowing what to write at any point, I just wrote that.

  2. I then went over the output of stage 1. and made notes on how I wanted to structure the ideas in order to best communicate them to someone else.

  3. I then sat down to write this post, writing with more awareness of how I was presenting my ideas to others. This stage itself was made up of two stages: a first pass, where I simply took my notes and expanded them into prose, and a second pass, where I went through what I’d written and edited it.

In total, I spent not much over 2 hours on this - which is quicker than I’ve written most of my blog posts. Of course, the amount of time you want to spend on each stage - braindumping ideas, structuring them, turning that structure into prose, and then editing it - is going to depend on the style and importance of what you’re writing. But I think step 1 - figuring out ideas, stream-of-consciousness-style, is pretty much vital in any case. In fact, if you want to write something quickly, I think you’re much better off cutting back on the later stages - worrying less about editing your stream-of-consciousness writing - than skipping stage 1.

If I can get over my perfectionism enough, I’m going to try publishing more posts on my blog that I’ve written almost entirely in stream-of-consciousness-style, with minimal editing. I really want to write more, and the reason I don’t is that it takes too much time. I’d like to have time to edit each post into something beautiful and coherent, but given that’s not realistic, I’d definitely rather share my ideas - even if they’re not formed into perfect prose - than nothing.

Sometimes, the boundary between writing-to-clarify-ideas and writing-as-presentation gets blurry. You might be be doing the former and find you have a new idea while writing - and then it may make sense to switch styles. This sort of happened to me a couple of paragraphs back - I hadn’t planned the point about the idea-clarification stage being the most important, but it made sense to me as I was writing, so I decided to go off-piste a little. This then led me to the idea in this paragraph - that sometimes you want to be able to switch a bit between the two.

Finally, in my experience, by far the hardest part of writing is just getting those first words onto the page. Once you’ve got a first draft down - even if it’s a mess - it’s much easier to edit it into something coherent. There’s something very psychologically daunting about a blank sheet (/word doc.) And so perhaps the most useful thing about breaking writing down into two stages - first clarifying your ideas, and then figuring out how best to present them - is it makes that initial “getting words onto paper” much less daunting. Trying to start from scratch and produce a masterpiece is never going to happen, and accepting that your first draft is likely to be far from perfect is incredibly useful.

I should caveat that I know writing is a different process for different people, so what works well for me might not necessarily work for you - and similarly, what I find hard about writing might not apply to everyone. But it seems likely enough that this idea would help other people that it was worth sharing. If you think your experience of writing is similar, I’d be really interested to hear that - but if you think it isn’t, and there’s something else about writing you find hard which this doesn’t help with, I’d be really interested to hear that too!

My favourite books of 2015

In no particular order - some of the books I most enjoyed reading, and learned the most from, in 2015. I've linked to notes I made on some of these books where I have them (some may be clearer and more concise than others!) 

I'm looking for book recommendations for 2016, so would love to hear what others' favourite books of 2015 were!

  1. Sapiens: A Brief History of Humankind -- Yuval Harari (notes)
    For me reading this book was so densely packed with "oh, that's a really interesting idea/hypothesis/explanation, I should think about that more" moments, more so than any other book I’ve read in a long time. Some of the ideas feel a bit speculative, but are incredibly thought-provoking all the same. I feel like I need to reread it at least another 5 times to even begin to get the most out of it. 

  2. The Power of Less -- Leo Babauta (notes)
    A very clear and concise guide to simplifying your life and your commitments, with lots of actionable advice, by the author of the Zen Habits blog. I ended up using a lot of this and finding it incredibly helpful when I was trying to cut down and simplify things in my life this time next year, and I look back at my notes on it every time I’m feeling a bit overwhelmed or overcommitted.

  3. Moonwalking with Einstein: The Art and Science of Remembering Everything -- Joshua Foer  (notes)
    Written by a journalist who took on the challenge of becoming a memory grandmaster in just a year. This book really convinced me that improving your memory isn’t about just being able to memorise obscure facts - it actually has the potential to improve your life in very profound ways - making the things you’ve learned and experienced more easily accessible and easier to draw connections between. (This was also one of those books I found got the perfect balance between telling me facts/making its point clear, and using good stories and analogies to illustrate these points. I think this is one of the biggest challenges for good nonfiction writing - often I feel like stories and analogies are just there as padding, and I want to skip them and get to the point - but a few really skillful writers, like Foer, manage to use these illustrations to much more clearly drill in the point they’re making.)

  4. The Antidote: Happiness for People Who Can’t Stand Positive Thinking -- Oliver Burkeman
    I really enjoy Oliver Burkeman’s Guardian column - he seems to manage to do popular psychology in a way that’s engaging without being overly sensationalised, which isn't always easy. This book - about embracing negative emotions, failure, uncertainty, and even death - solidified for me a lot of ideas I’d been thinking about previously about happiness. I got that satisfying feeling reading it of someone else expressing my own thoughts more clearly than I could have done myself.

  5. On The Move: A Life -- Oliver Sacks
    I was incredibly sad and moved when I read the famous neurologist and author’s NYT column earlier in the year, sharing that he’d been diagnosed with a terminal cancer. I’ve really enjoyed some of his other writings - a unique combination of sharing scientific insights through personal stories and case studies - and found this memoir engagingly written, thought-provoking and inspiring. 

  6. Do No Harm: Stories of Life, Death, and Brain Surgery -- Henry Marsh
    An incredibly compelling account of what it’s like to be a brain surgeon. Probably the most “un-put-down-able” book I read this year. It’s difficult to read at times, but in a good way - I felt like I really got some, albeit small, insight into the extreme pressure, stresses, and emotions involved in neurosurgery. Thanks Killian Czuba for this recommendation, I loved this book, but probably would have never found it or picked it up myself.

  7. Being Mortal: Illness, Medicine and What Matters in the End -- Atul Gawande
    A really great, if somewhat depressing, discussion of the problematic way in which the medical profession tends to deal with old age - so obsessed with ‘treatment’ and extending life without enough focus on how to maximise quality of life in the final years.

  8. How to Think About Exercise -- Damon Young
    As someone who spends most of her time in “thinking” mode, but who also gets a lot of pleasure out of exercise, I really enjoyed this discussion of the ways in which exercise relates to various intellectual ‘virtues’. In parts I felt the book went into more discussion of specific historical philosophers than I personally cared for, but the overriding themes and insights helped me to think more about why I exercise and how I can use exercise to become a better person, beyond simply staying in shape.

  9. The Code Book: The Secret History of Codes and Code Breaking -- Simon Singh
    I read Simon Singh’s book Fermat’s Last Theorem years ago - before I started university - and absolutely loved it. I’d been meaning to read this - another very popular book of his about cryptography - for years, and finally got round to it this year. I found it similarly fascinating, though I much preferred the later chapters of the book, which get into explaining some of the more recent and complex cryptography methods like public key encryption and quantum cryptography (the chapter on the latter inspired a brief fascination with quantum physics - but even after reading a couple of popular books on the topic I admit I’m still basically clueless.)

  10. The Obstacle is The Way: The Ancient Art of Turning Adversity to Advantage -- Ryan Holiday
    I was unsure whether to include this book in my 'favourites' list, because I didn’t actually finish it - I felt it got a bit repetitive after a while, and I’d sort of got most of what I needed out of it after a few chapters. But I found the core message - the Stoic idea that inside every difficult emotion and challenging situation there is an amazing opportunity to learn and improve - so useful to internalise, I still felt I wanted to recommend this book. Reminding myself of this idea is the single thing that most consistently helps me deal with tough situations and emotions.

Some new year's reflections

  • I really like taking some time around the new year to do step back and review: to reflect on the past year and what I’d like to improve. I highly recommend taking the time to do this if you can - not just setting resolutions, but reflecting more systematically on how different areas of your life are going, what you’ve achieved and learned over the past year, and what you might do differently. I’ve shared it before, but again, Alex Vermeer has a really great guide to doing this here.

  • Thinking about all the ways in which I could improve my life, though, I have to constantly remind myself to prioritise and take one thing at a time. When I do these reviews, I start out really excited about all the things I could change or improve in my life, but then begin to feel overwhelmed with all the possibilities, which can quickly turn into me feeling unmotivated by all the areas of my life I’m not fixing right now. There are too many things I could be doing, and I don’t know where to start - so I end up doing none of them. The solution is sort of obvious - when overwhelmed with possibilities, prioritise - but I find it really hard to do, because it means letting go of things I really want to do but aren't highest-priority-right-now. Really internalising this was perhaps the single thing that resulted in me making way more improvements to my life last year than I’d ever done in previous years. Each month, I chose a single area of my life - e.g. exercise, diet, social life - or one specific goal - e.g. simplify my life and get rid of unnecessary commitments - and focused on just doing that one thing really well. And ironically, the sense of self-efficacy I got from seeing concrete improvements in one area of my life actually meant I often ended up making more other improvements than I’d planned.

  • If you are setting new year’s resolutions, here’s something I wrote last year with some psychology-based tips on how to set resolutions you’ll actually keep. In short: only choose goals you actually care about and genuinely believe you can achieve, use “implementation intentions” to help behaviours occur more automatically, change your environment to make a behaviour easier, think in advance about possible failure modes and how to avoid them, and set a time later to check in on your progress. The main thing I’d add to this now is to set a couple of really important goals rather than lots and lots of smaller ones (see second bullet!), and to be aware that ‘failure’, at least to a certain extent, is inevitable and an important part of the process of improving

  • My favourite insight of 2016 so fardon’t ask yourself what you want out of life, but what you’re willing to struggle for/what struggles you’re willing to undergo. Another way of putting this is that, for anything you think you want in life, it’s important to make sure you actually want to undergo the process involved in achieving that thing - including the costs and struggles. From the linked article: "If you want the beach body, you have to want the sweat, the soreness, the early mornings, and the hunger pangs. If you want the yacht, you have to also want the late nights, the risky business moves, and the possibility of pissing off a person or ten thousand."

  • A question I've found amazingly useful to ask myself: “What would the best version of myself do right now?” I'm always surprised how much this helps me clarify what I should do at any given time, and helps me to live more in line with what I most care about. “What would make me most proud of myself?” has a similar effect. It's surprising to me because it's often immediately obvious what the "best" version of myself would do - but that's often not the thing I would do if just acting based on my immediate inclinations. I think this is related to why it can be easier to give others advice than ourselves - sometimes we just need to step back from the situation and whatever we're feeling in the immediate moment to get some perspective. (Inspiration from Brian Johnson's notion of "the integrity gap")

  • I think I’ve underestimated just how important it is to get the really ‘essential’ elements of my life really sorted, before tackling other challenges. For me, this means getting enough good quality sleep, and getting enough exercise. I think I haven’t prioritised these enough in the past, despite knowing that sleep and exercise are clearly the things that account for most of the variance in my happiness. Asking yourself what those areas are for you that make a big difference to everything else, and making sure you’re doing everything you can in those areas, is one of those pieces of advice that sounds so obvious but very few people actually follow.

  • People spend a lot of time trying to find ways to extend their lives - eating certain kinds of food, taking certain supplements, exercising in ways that are suggested might make you live longer. I'm all for doing these things, but I think it's also worth thinking about ways you can make your subjective experience of your life longer: key things like living more mindfully and deliberately, building more variety into your days, and finding ways to make your experiences more memorable. Though these things won't technically "extend" the amount of time you live for, in my experience focusing on them makes me feel like my life passes more slowly (in a good way!) and that, in retrospect, I've been living it more fully. And in many ways, I think this might be worth more than an extra year of life that passes by in a blur

  • If I’m going to publicly commit to one new year’s resolution, it’s that I want to write more - and not write to get my name in publications, but write to explore ideas, and share those ideas with others who may have other insights. The biggest barrier for me to doing that more is being less perfectionist with my writing, and being willing to leave questions unanswered. Each of the bullets here could have been it’s own blog post, and it’s taking a lot of self-restraint for me to post this without going into more depth and answering more questions - but I know realistically the alternative is me publishing nothing. So watch this space for more imperfect blog posts...