5/5
it has been a drudge to get work done.
----
the end-goal is within sight. one more round of prompt-tuning, a $1 gpt-4.1-nano run, and some rounds of "LLM consensus checking".
the goal then becomes applications.
- LLM benchmarks. 🔥 use the machine to feed the machine 💡 "which word has this definition" questions will be possible at some point. but not yet.
- Elementary education. 💡 which words should a 3rd/5th grader know? be studying? ⚙️ so far no useful progress on "how easy/hard is it to spell this word"
- Second-language learning. 💡 a "which is the Chinese for this word in this sentence" app.
- Text difficulty. A smarter metric than Flesch-Kincaid.
----
The "cosine similarity" question of "how similar are these word definitions" is not yet solved. I'm not sure I can solve it.
I can test it; I have several ways of generating embeddings. And (probably) these can include a sentence as context.
----
I also have no solutions for the "group different word-forms with the same meaning". For jump, jumps, jumped, for example.
This would be a much more substantial problem for more highly-conjugated languages. With English, it is almost avoidable.
----
Around word 5000, i am seeing raft, yield, algebra, and pizza.
This seems correct enough? "Algebra" is more common in encyclopedic contexts, and "pizza" doesn't show up in the 19th century corpus at all.
----
But, as far as "exploration" is concerned, I am reaching diminishing returns.
I have one more list of "see if Claude can do this quickly". After that, "glenora" will become an inactive project.