The thing I can't put my finger on

Today I was thinking about how routing evals could work, and I ended up spending most of the day on reinforcement learning instead.

I hadn't really looked at RL before. I'd been in machine learning and deep learning and RL was just this other thing off to the side. Then I actually read into it and it turns out it's everywhere. LLMs use it. Workflow systems use it. Self-driving cars use it. Online, offline, all sorts. It isn't going away. It's also relevant to the causal AI research I've been doing, which is part of why I started.

I've been going through the Bellman equation, MDPs, POMDPs. It's more interesting than I expected it to be.

Trying to connect it to what I'm building

I'm building Livin AI right now, so I keep trying to work out where RL fits into evaluating or improving that process. Same with evals in general. I was looking at what it would take to build something in that space and there is already so much out there. LangSmith, a pile of open source tools, observability and tracing everywhere. It's a crowded space and I felt that a bit.

But that's sort of my point. There's so much going on in AI right now and it's all related to each other. Observability and tracing connect to causal AI, which connects to alignment, because all of it is about seeing why the agent did what it did. I know these are technically separate things. I can't quite put my finger on the connection, but it's there.

What I want to change

At work I want to move toward the evals and the workflows we're building on our own inference and framework, and away from the UI work I'm doing now. On the side, the causal AI research. Both of those tie back to RL and evals somehow.

Things I want to build

I saw someone build a robot that rides back and forth along a rail with a rod hanging off it, and it learns to balance the rod upright. Like balancing a pole on your finger. That was very cool and I think I could do that. I've been thinking about making YouTube shorts where I just build things with RL and show them.

I also vibe coded a couple of small things today. One on memory, and one that turns things into podcasts so I can learn faster.

And I started a repo called VStack. The idea is a tech stack but for my skills, somewhere to keep the Claude skills and plugins I build. I didn't use AI much before this, but building my own skills and plugins has made a real difference in how fast I can move, and it frees me up to spend time on the things that actually matter.