TinkerQuest - Autonomous bug agents and cost
New day, no sleep, new problems. I am still waiting in the hospital and still nothing else to do.
- My MVP had issues. At some point I just added logic to spawn Codex when an error shows up. It would analyze and fix it. I still review the result before it goes live. Makes the whole bug fix cycle faster. This is not news, but doing it the first time myself and seeing it in action just puts it into a different perspective.
- Codex costs are low. 0.8€ in more than 24h is a bargain.
- What is missing for me is some way to specify the use cases and requirements in a codified way. That would ensure the solution works as intended and I don’t have to sweat not reading the generated code (besides executable specs).
Probably a good time to check in on costs. Yesterday I ran out of tokens for the night. All in all, I spent a whopping 0.8€ across more than 24 h. In that timeframe, I iterated through 2 PoCs and set up tests and used a bunch of tech I never used before. Building something new has never been so cheap.
But I am also not done yet. All my TinkerCAD1 interactions are broken and I don’t know why. Hard to figure out what is wrong without logs. So off we go adding some baseline observability so Sol can apply a fix.
Next interaction, next error. Sending Codex questions from within TinkerQuest also fails. I start another Codex session and do the same bug fix exercise again. Until I realize that I can automate all of that. Every new error log will trigger a Codex session that analyzes and fixes the bug. If an error cannot be fixed because it is triggered by a dependency like TinkerCAD, it becomes a warning and gets ignored in the future.
Now, I have an agent running alongside my app and modifying it when it finds a problem. That is somehow wild.
Previous read: TinkerQuest - Nightshift and an MVP