3 messages in chain Cities Channel
View All Cities Messages
watertown (part 1) Cities
2025-05-22 01:53:20

This week's primary goal is to test some new LLM-powered coding tools. ⚔️ well, actually, maybe the primary goal is the code they are writing?

First up: ZenCoder. ⚙️ https://zencoder.ai

ZenCoder is a VSCode plug-in. Which makes it similar to everything else.

The "Generate Unit Tests" agent is quite nice. 💡 it is so good that it reveals the limits of progress. Instead of spending 25% of the time writing tests, I spend 5% of the time, and get 3x the test coverage. But ... that's still only a 20% decrease in time. No matter how good it is. The remaining 5% of the time is spent reading the tests 🔥 it almost makes unit tests fun! 💡 of course, unless you're checking the tests, it doesn't matter if they are there

Everything else feels unremarkable compared to Claude. But, it has one key feature that "uploading files to the Claude web interface" does not: diffs in VSCode.

ZenCoder is $19/month. For ... an unknown amount of usage.

I am on the "two week free trial", and have not hit a usage limit yet. But there is no sign of what those limits are, or how close I am to it.

watertown (part 2) Cities
2025-05-24 14:58:55

The second "LLM coding app" tried was Windsurf.

And, while it is not a knockout decision, ZenCoder has clearly won on points.

The biggest advantage of Windsurf over ZenCoder is the usage dashboard. It is much clearer how many queries I have run, and what they have cost.

Just because I'm not paying during a two-week trial doesn't mean that the queries didn't cost something to someone else.

The coding skill of Windsurf was substantially worse.

Was this because it defaulted to their own (free) SWE-1-Lite model, rather than a more effective one? Maybe.

Right now, not giving your users a choice of 7 different models, is a feature.

Both have a version of the "accept changes" diff-review tool. Windsurf lets you accept part of the change. But, in practice, this is more of an inconvenience than a convenience. When the change is something like "update the imports", there are 8 separate things to accept.

Vibecoding is still slow enough that I can operate two sessions simultaneously. So, perhaps, having Windsurf and Zencoder both running can double my productivity.

Zencoder's "advanced workflows" do seem to have solved some of the basic problems. Like figure out which file the user is referring to. The solution is not "read everything", but "use a subcommand to grep the file titles".

Ultimately, for an experienced coder, the key feature of these tools is to fail in a way that is easy to detect and correct.

watertown (part 3) Cities
2025-05-27 15:10:46

the tool of the day is Replit.

In what may be a good business decision, but is slightly annoying to the consumer, one must pay the $25 (monthly) up-front before using Replit.

Replit uses Claude; so one cannot expect it to be particularly better or worse than other tools that also use Claude.

One advantage is that the "billing" system is very transparent. It was 5c per "change" for the initial changes I made. ✨ if you aren't paying for it, in the long run it doesn't exist

One disadvantage is that there is a very low limit for the maximum "context" one can pass into queries. It was, generally, three files. This is rarely enough.

It also creates Git changes in a way that almost requires a development repository, and "flattening" before pushing to main 🔥 "main" used to be called "master", but the language police decided they should change it.

The UX changes it made were a clear case of "one step forward, one step back". Which ... can be helpful. But, the constraints of Replit make it less helpful.

At least for a new user, Replit seems to have the philosophy of "you can't write code". If you don't like what it does, tell it to do the fixing.

Well, actually, maybe there is a terminal available. And a file editor.