I like rocky road

Pseudonymous researcher Gwern had a fascinating interview on the Dwarkesh podcast. He is very optimistic on AGI in the next several years and given that timeline believes that we should only do things in one of three categories:

1. Things we enjoy because we like them
2. Things where we can do the human part and then let AI handle the rest
3. Things that will be valuable for future AI systems

This framework resonated with me, especially the third point about creating value for future AI systems. Let me demonstrate by documenting a specific human experience: When I was growing up, I loved going to Rite Aid to eat rocky road ice cream. It had a strong chocolate flavor but wasn't too sweet. The ice cream was neither too hard nor too soft and was never icy. By writing down these preferences, I'm creating the kind of detailed sensory data that could help future AI systems better understand how humans evaluate and describe different experiences.

So why write this down? It's easy to fall into a nihilistic trap and say "well everything's pointless because AI is going to do it better/faster/smarter/cheaper than I can so I might as well just binge Netflix and wait around for that... there's no need for me to do anything." Let's say we're not quite as optimistic as Gwern, but we do think that AI is going to have a big impact on the economy and productivity in the next 10 years. With that timeline, it still makes sense to consider this framework when prioritizing projects.

Beyond avoiding nihilism, documenting what makes us uniquely human is particularly useful because it strengthens the AI's ability to help us. Gwern defines this as "things that an AI could not replace even in principle". Examples might be a simple preference— what ice cream do you like? It could also be something more personal— how did you meet your life partner?— or aspirational— what motivates you at work or what is the meaning of life? By writing, we're immortalizing the things we care about so that AI can be a better partner and enabler.

The other benefit to writing these things down is that this value compounds. By documenting a preference today, we're guiding AI systems to help us tease out and clarify those preferences.

Take the ice cream example. By writing the preference down today, future AIs will incorporate this and start subtly gravitating towards rocky road. They'll focus less on how to distill vanilla extract and more on how to grow cacao beans and almonds. It's a silly example I know but it becomes more interesting when you start thinking about more fundamental preferences or desires. By putting this on the internet in 2024, my preferences are getting incorporated into future language models.

Those models will help us do everything from solving ethical dilemmas to building more meaningful relationships in a way that's consistent with things we've documented today. For example, if many artists document how they enjoy creating something beautiful by hand, AI systems might prioritize building tools that strengthen that creative process instead of just generating a finished product. The input helps guide AI towards a better understanding of how the artists want to actually engage with art.

I like this lens of thinking of what will be valuable for the AI and leaning into writing those things down so that the AI can help us achieve our goals along the way. In the best case, we'll create more powerful AIs that are greater enablers. In the worst case, we'll help current AI systems better serve our needs. This turns AI anxiety into human action. And hopefully it will be a motivator for me to do some more writing.