Book review
Human Compatible
Russell explains how AI can satisfy a stated objective while missing what people want and considers ways to address that problem. A layperson’s overview with doubts that the then-current systems would lead to general intelligence.
These notes began in 2019 and include a March 2020 update about AI as I understood it then.
Human Compatible by Stuart Russell is a great intro to where we are now with AI. It also lays out some of the problems and ways to tackle them.
I liked the book as a lay-person overview of where we are with AI. Stuart Russell does a great job of not looking too far ahead and also foreseeing some problems that we already have. There was no clear distinction between AI today and AGI, and that divide might be too technical/doubtful, but I (and Russell too) think the systems of today won’t get us to AGI.
—
See this good video om 10 reasons to ignore AI safety (and their rebuttals)
—
Update 17 March 2020: I’ve checked out this review on Slate Star Codex
“Russell goes over all the recent debates in AI – Facebook, algorithmic bias, self-driving cars. Then he shows how these are caused by systems doing what we tell them to do (ie optimizing for one easily-described quantity) rather than what we really want them to do (capture the full range of human values). Then he talks about how future superintelligent systems will have the same problem.”
In a way this is an analogy to a 4 year old, they will do what you tell them to do, but here are the first signs that they will do this literally and not per se as you intended them to do it.
“(from the book) The problem comes from confusing two distinct things: reward signals and actual rewards. In the standard approach to reinforcement learning, these are one and the same. That seems to be a mistake. Instead, they should be treated separately…reward signals provide information about the accumulation of actual reward, which is the thing to be maximized. “
This part highlights some of the work that has already been done on solving the ‘fuck-now-the-world-is-a-paperclip-problem’.
The article also discusses ways in which algorithms, then called current-day AI, can go wrong, but concludes that there was little to worry about in general. As I reported its March 2020 view, deepfakes were not widely used, although there were examples, as there are of forged signatures.