Rationality From AI to Zombies
Rationality: From AI to Zombies by Eliezer Yudkowsky is a huge tome that covers everything from heuristics to Bayes theorem. Its main goal is to give the reader a better/modern understanding of rationality and the tools one needs to have in their toolkit.
It can be found (as the original books and posts) here.
The book was quite the journey and over the coming months I plan to go back to the individual posts to put concepts in Obsidian and make notes here.
—
“A cognitive bias is a systematic error in how we think, as opposed to a random error or one that’s merely caused by our ignorance. Whereas statistical bias skews a sample so that it less closely resembles a larger population, cognitive biases skew our thinking so that it less accurately tracks the truth (or less reliably serves our other goals)… Like statistical biases, cognitive biases can distort our view of reality, they can’t always be fixed by just gathering more data, and their effects can add up over time. But when the miscalibrated measuring instrument you’re trying to fix is you, debiasing is a unique challenge.”
The goal of the text is teaching (tools of) rationality, talking about biases that we have is the first step/part of it.
With biases, you may still experience them, even if you know beforehand that you have them. On the other hand, you can also over correct. So it’s always difficult/challenging to assess correctly.
- base neglect bias: ignoring how many of X (and Y) there are (e.g. a shy person is more likely a sales person than a librarian because there are more of the former)
- sunk cost fallacy: not ignoring the costs that we made before at the moment of evaluation (of future costs/benefits)
“The map is not the territory.”
—
We don’t clearly adjust our spending/giving based on the scope. We have scope insensitivity.
“The usual finding is that exponential increases in scope create linear increases in willingness-to-pay—perhaps corresponding to the linear time for our eyes to glaze over the zeroes; this small amount of affect is added, not multiplied, with the prototype affect. This hypothesis is known as “valuation by prototype.””
“An alternative hypothesis is “purchase of moral satisfaction.” People spend enough money to create a warm glow in themselves, a sense of having done their duty.”
Or in other words, we care about people/animals, but really don’t see that 10X more saved is 10X better. A good lesson for effective altruism (communication). Focus on the prototype in communication, whilst still ruthlessly strive for the best solution.
—
We should have rationality dojos says Yudkowski. There are now some places devoted to this. But like my weightlifting, I like to learn from the best, practice much alone. And yes, I do recognize that you need to test things in the real world and talk to others. But I think that learning from Dawkins, Dennett, and Deutsch, isn’t that bad either.
—
“The availability heuristic is judging the frequency or probability of an event by the ease with which examples of the event come to mind.”
This is how terrorism and fear of flying (vs driving) works.
Related is absurdity bias, if something hasn’t happened (in a long time) we also can’t image it happening now.
—
https://www.lesswrong.com/s/5g5TkQTe9rmPS5vvM/p/jnZbHi873v9vcpGpZ