This month I want to highlight a video, Is RL + LLMs enough for AGI?, by the Dwarkesh Podcast, featuring Sholto Douglas & Trenton Bricken. This video explores recent advancements in AI from a scientist’s point of view, which I find particularly refreshing given how much AI chatter today is aimed at selling services. One topic that really stood out is the Generator-Verifier gap: it is much easier to validate a solution than create a solution.

The Generator-Verifier gap reminds me of Cunningham’s Law, which amusingly states: “The best way to get the right answer on the internet is not to ask a question; it’s to post the wrong answer.” When I first heard Cunningham’s Law a couple decades ago, I laughed at how fundamentally true it was. Reflecting further through the years, I began to understand that Cunningham’s Law scratches at a core human feature – we (humans) are far better editors than creators.

Hacking this human feature has many applications. Some use the McDonald’s Theory when suggesting a meal in order to inspire their group to come up with a better idea. I have summarized topics in an overly-simple fashion to help uncover the most meaningful nuance (through folks’ corrections). There are plenty of other applications that introduce information with the intent to drive action or drive towards a conclusion.

I imagine when AI is capable of creating affordable, high-volume solutions, we are likely to adapt our human work away from idea generation and towards idea verification. This may be the next fundamental shift in how we work, and frankly I think we are predisposed to be pretty good at it.

This video is long (2.5h) and entirely worth the watch, although I understand long form content does not fit in everyone’s life. The video below links directly to the Generator-Verifier gap topic.

“But it’s very plausible to me, we’ll be at the point where it’s so easy to generate with these agents that the bottleneck is actually, can I as the human verify the answer?”