Of not appreciating terminal rendering enough
Hacking on TinkerQuest made me really not like terminal sessions on my iPhone. What I want is to have a great UX to interact with my Codex workstation.
This is a new project and I already got some insights:
- Rendering a terminal byte by byte from a tmux1 stream is hard even with stuff like xterm.js2. Codex's inline updates with a scrollable session history broke for me.
- But tmux + ttyd3 works out of the box if you can accept desktop proportions.
- Talking to Sol in the app is a great way for me to sketch out a problem and to iterate through some early ideas quickly. Walking around while thinking helps a lot.
- Thinking about a locally running e2e test pipeline has to happen at the very beginning. Before the first line of code is generated, Sol has to know how to test the whole thing. Especially for apps, results will suffer because Sol lacks the signal to know if it is right or not.
Last time, I ended up with a Hetzner4 server running Tailscale5, herdr6, and Codex, and I used WebSSH7 to run terminal sessions on my phone. Now, I want to replace WebSSH with a new web app. I already have herdr for terminal session persistence, but it has a bunch of extra layers I don’t need for this project, so I go for the OG terminal manager tmux. What I still have to figure out is how to get terminals on my phone.
I am cleaning the dishes at this point, so I pop open ChatGPT voice and start discussing options with Sol. It suggests driftty8 but also mentions that ttyd would work. Since the latter is the more mature project, I go with ttyd.
Back on my phone, Codex has everything set up within 5 min. Celebrating myself, I check my browser and it looks more or less like WebSSH. Not exactly mobile-first. Another iteration with Codex and now fonts are a tiny bit bigger. That is not what I want. Checking driftty again, but that also relies on terminal commands. At this point, I believe I can crank something out that is truly mobile-first. Terminal sessions are like WhatsApp chats on a homepage. You click on them and reconnect. Inside you get autocomplete while typing, you can swipe up and down to scroll in history, and so on. How hard can it be?
Apparently very hard. I am back with Sol on voice and we discuss xterm.js and how it can render tmux terminal data streams. I take that info to Codex and we have a first iteration in a couple of minutes. Doesn’t look too shabby.
But wait. Scrolling is super slow. Why? Because it turns out the web app doesn’t buffer terminal history. So we take another turn and keep a buffer in the app. When I hit the end of that buffer, it loads the next chunk from tmux capture-pane. That works reasonably well, but there is still (at least) one problem: Codex overwrites lines continuously. That breaks the UI on my end. Somehow I also lose the prompts I wrote in my history.
Now, Sol and I get into the weeds of how to fix the rendering issue, but because escape sequences can screw up rendering, we probably have to build our own terminal state, which sounds a lot like creating another tmux. Debugging xterm.js also doesn’t sound too enticing at 9pm on a Tuesday.
Mentally, I take a step back. What do I actually need right now? Not a mobile-friendly terminal. I need a mobile-friendly Codex UI and, alas, Codex now also has an experimental server I could use to that end. Next iteration. Next post.
Previous read: TinkerQuest - How to vibe on your phone