I read Darwin’s Dangerous Idea for the psychohistory book club. A couple things stood out.

First, it’s really easy to get nerdsniped by evolution and envision everything through an evolutionary lens. Check out Aneesh Sathe’s field guide to evolution and tell me that ain’t so.

Second, skyhooks versus cranes is a ridiculously useful frame for thinking about shared assumptions and things that are unknowable. There is always a crane, a mechanistic rationale behind the most wondrous phenomena. It’s just that, at a certain point, the cranes become too hard for us to distinguish and we resort to arguing about “deep truths”. Fortunately, these are not skyhooks but unresolved questions about cranes.

Third, the idea of using cranes to build more powerful cranes has altered my base perception of artificial intelligence. It’s softened my stance and led me to see it more as a form of normal technology. Yet it’s also caused me to pay more attention to the moral quandaries associated with lines that demarcate levels of consciousness.

Fourth—and perhaps most importantly, though—it’s caused me to reevaluate the role of intelligence.

Dennett talks about “universal acid”, a liquid that can eat through any container. Intelligence is pretty universal-acidish. Point an intelligence cannon at an arbitrary problem for long enough and that problem gets solved. The issue is that we humans are ourselves integral pieces of the many problems we’re trying to solve.

This may sound epochal, but it’s probably happened before at a smaller scale. The rise of the internet did the same thing with information, making it act like universal acid. There, we humans were also the impediment. And the bet was that with enough disembodied_information_, any problem could be solved. That didn’t turn out to be quite true, and we ended up evolving sophisticated sociocultural mechanisms for handling the abundant information flows of modern life.

A similar step will have to happen for intelligence. We’re in the unexpected abundance phase, and will soon enter the part of the story where we have to develop smart sociocultural mechanisms for dealing with vast levels of arbitrarily utilised intelligence. In the aggregate, those mechanisms will be smart. But per individual, those mechanisms will be extraordinarily dumb.

“Dumb” here is meant in the best sense of the word. Something that is self-evident and obvious to the agent with minimal deep reflective thought and contemplative action, and thus highly likely to be of maximal local value.

This is why Dennett says that Darwin’s idea is so dangerous. Darwin posited that for populations, it is always crunch time. Variations within that population yield advantages, and those advantages are amplified across generations. This dumb, banal algorithmic process called natural selection underpins life as we know it.

And the dumb use of smarts is, I believe, the most intelligent thing we as a species will ever actually do.