{"chain":[{"channel":"cities","content":" <<< *Lake Andes* is located in southern South Dakota, part of the Yankton Sioux reservation. >>>\r\n\r\nthe goal for today: merge the \"Trakaido wordlist\" and the \"Greenland wordlist\".\r\n\r\n----\r\n\r\nThe Trakaido wordlist is 720 non-verbs (nouns, adjectives, and \"grammatical words\" like << with >> or << how >>), in various categories. ( there are also verbs, but in conjugated forms, not suitable for a \"merge\" yet.)\r\n\r\nThe Greenland wordlist is 4000-10000 << WordToken >>s, from word frequency lists. These do not clarify specific meaning, or whether it is part of a multi-word phrase.\r\n\r\n----\r\n\r\nThe merged wordlist will probably be << lemma >> based, with a list of \"derivative forms\" for each. ( the issue that \"run\" and \"running\" are separate lemmas when they are different parts of speech ... I will call it something else if need be.)\r\n\r\n--MORE--\r\n\r\nThe current outputs, for Trakaido:\r\n\r\n<<< additional_foods.py:\r\nN16011 = {\r\n 'guid': 'N16011',\r\n 'english': 'ketchup',\r\n 'lithuanian': 'ke\u010dupas',\r\n 'alternatives': {\r\n 'english': [],\r\n 'lithuanian': []\r\n },\r\n 'metadata': {\r\n 'difficulty_level': None,\r\n 'frequency_rank': None,\r\n 'tags': [],\r\n 'notes': ''\r\n }\r\n} >>>\r\n\r\nand, for greenland:\r\n\r\n<<< Word: ketchup\r\nRank: None\r\n\r\nDefinitions:\r\n [1] A thick, sweet, and tangy sauce made primarily from tomatoes, often used as a condiment for foods like fries, burgers, and hot dogs.\r\n Confidence: 1.00\r\n Lemma: ketchup\r\n Part of speech: noun\r\n Grammatical form: noun/singular\r\n Subtype: food_drink\r\n IPA: /\u02c8k\u025bt\u0283\u028cp/\r\n Phonetic Pronunciation: KEH-chup\r\n Chinese: \u756a\u8304\u9171\r\n French: ketchup\r\n Korean: \ucf00\ucca9\r\n Swahili: ketchup\r\n Lithuanian: ketchup\r\n Vietnamese: t\u01b0\u01a1ng c\u00e0\r\n Notes: This is the most common and primary meaning of 'ketchup'.\r\n Examples:\r\n - I like to put ketchup on my fries.\r\n - Could you pass the ketchup, please?\r\n - Ketchup is a popular condiment worldwide.\r\n>>>\r\n( the fact that we used a cheap model, asked for multiple definitions, and got << To apply ketchup to food. >>, as in << She will ketchup the fries after they arrive. >> ... is not relevant. ) ( the fact that it mistranslated ketchup into Lithuanian is more of an issue.)","created_at":"2025-07-24T17:04:20.900618","id":635,"is_target":false,"parent_id":null,"processed_content":"

Lake Andes is located in southern South Dakota, part of the Yankton Sioux reservation.

\n

the goal for today: merge the \"Trakaido wordlist\" and the \"Greenland wordlist\".\r

\n
\n

The Trakaido wordlist is 720 non-verbs (nouns, adjectives, and \"grammatical words\" like with or how), in various categories. \ud83d\udca1 there are also verbs, but in conjugated forms, not suitable for a \"merge\" yet.\r

\n

The Greenland wordlist is 4000-10000 WordTokens, from word frequency lists. These do not clarify specific meaning, or whether it is part of a multi-word phrase.\r

\n
\n

The merged wordlist will probably be lemma based, with a list of \"derivative forms\" for each. \ud83d\udca1 the issue that \"run\" and \"running\" are separate lemmas when they are different parts of speech ... I will call it something else if need be.\r

\n
⎯⎯⎯⎯⎯
\n

The current outputs, for Trakaido:\r

\n

additional_foods.py:\r

\n

N16011 = {\r

\n

'guid': 'N16011',\r

\n

'english': 'ketchup',\r

\n

'lithuanian': 'ke\u010dupas',\r

\n

'alternatives': {\r

\n

'english': [],\r

\n

'lithuanian': []\r

\n

},\r

\n

'metadata': {\r

\n

'difficulty_level': None,\r

\n

'frequency_rank': None,\r

\n

'tags': [],\r

\n

'notes': ''\r

\n

}\r

\n

}

\n

and, for greenland:\r

\n

Word: ketchup\r

\n

Rank: None\r

\n

Definitions:\r

\n

[1] A thick, sweet, and tangy sauce made primarily from tomatoes, often used as a condiment for foods like fries, burgers, and hot dogs.\r

\n

Confidence: 1.00\r

\n

Lemma: ketchup\r

\n

Part of speech: noun\r

\n

Grammatical form: noun/singular\r

\n

Subtype: food_drink\r

\n

IPA: /\u02c8k\u025bt\u0283\u028cp/\r

\n

Phonetic Pronunciation: KEH-chup\r

\n

Chinese: \u756a\u8304\u9171\r

\n

French: ketchup\r

\n

Korean: \ucf00\ucca9\r

\n

Swahili: ketchup\r

\n

Lithuanian: ketchup\r

\n

Vietnamese: t\u01b0\u01a1ng c\u00e0\r

\n

Notes: This is the most common and primary meaning of 'ketchup'.\r

\n

Examples:\r

\n

- I like to put ketchup on my fries.\r

\n

- Could you pass the ketchup, please?\r

\n

- Ketchup is a popular condiment worldwide.\r

\n

\u2694\ufe0f the fact that we used a cheap model, asked for multiple definitions, and got To apply ketchup to food., as in She will ketchup the fries after they arrive. ... is not relevant. \ud83d\udca1 the fact that it mistranslated ketchup into Lithuanian is more of an issue.

","subject":"lake andes"},{"channel":"cities","content":"In Lithuanian, the term for \"Lithuanian language\" is << Lietuvi\u0173 kalba >>, for \"Lithuanian man\" is << lietuvis >>, and for \"Lithuanian woman\" is << lietuv\u0117 >>.\r\n\r\nThese are all the same word in English, << Lithuanian >>. Here, we have separate files ( or \"grammatical form\") for \"nationality\" and \"language\".\r\n\r\n with 200 nationalities and 400 languages, it is fine to have a \"special case\" for each. I am expecting 100-200 \"special cases\", including 10-20 that are catch-alls.\r\n\r\n----\r\n\r\nThe thought of having \"Derivative Forms\" shared between languages is, unfortunately, impossible.\r\n\r\nSo, the \"lemma\" form (infinitive, etc.) will have to be translated. This, still, is difficult. Is it << la piscine >> or just << piscine >>? Or, for that matter, \"pool\" or \"swimming pool\" (or << pool (swimming) >>).\r\n\r\nBut, then, there will be separate derivative forms for \"walks\", \"walked\", etc., all of which are mono-lingual. For << marchons >>, << marchez >>, etc. it will be a separate set.\r\n\r\n----\r\n\r\nFor (conjunctions, prepositions, etc.) I want to still call them \"grammatical words\".\r\n\r\nBecause a word-for-word \"translation\" is too perilous to attempt.\r\n\r\n--MORE--\r\n\r\nIn practical terms, for verbs, Greenland output will have a pythondict like:\r\n\r\n<<<\r\nV0023 = {\r\n\"guid\": \"V0023\",\r\n\"english\": \"to eat\",\r\n\"lithuanian\": \"valgyti\",\r\n\"english_forms\" = {\"first-person-singular-present\": \"I eat\", ...}\r\n\"lithuanian_forms\" = {\"first-person-singular-present\": \"A\u0161 valgau\", ...} ...\r\n>>>\r\n\r\nThis is similar enough to the current Trakaido format:\r\n\r\n<<< \r\n\"valgyti\": {\r\n \"english\": \"to eat\",\r\n \"present_tense\": {\r\n \"1s\": {\"english\": \"I eat\", \"lithuanian\": \"a\u0161 valgau\"},\r\n \"2s\": {\"english\": \"you(s.) eat\", \"lithuanian\": \"tu valgai\"},\r\n \"3s-m\": {\"english\": \"he eats\", \"lithuanian\": \"jis valgo\"}, ...\r\n>>>\r\n\r\nFor both, it will request \"valgyti / first-person-singular-present\" as a \"flashcard\".\r\n\r\n----\r\n\r\nThe new approach will simplify some of the \"more animals\" style groups.\r\n\r\nAll the animals will be in a single \"dictionary\" file. The first 12 will be in \"Animals 1\", the next 18 in \"Animals 2\", etc.\r\n\r\n this will *probably* come with a reworking of the \"corpus\" system. We mostly want \"levels\" now anyway ... and \"decoy sets\" can be configured separately.","created_at":"2025-07-24T21:52:37.965355","id":636,"is_target":true,"parent_id":635,"processed_content":"

In Lithuanian, the term for \"Lithuanian language\" is Lietuvi\u0173 kalba, for \"Lithuanian man\" is lietuvis, and for \"Lithuanian woman\" is lietuv\u0117.\r

\n

These are all the same word in English, Lithuanian. Here, we have separate files \ud83d\udd25 or \"grammatical form\" for \"nationality\" and \"language\".\r

\n

\ud83d\udca1 with 200 nationalities and 400 languages, it is fine to have a \"special case\" for each. I am expecting 100-200 \"special cases\", including 10-20 that are catch-alls.\r

\n
\n

The thought of having \"Derivative Forms\" shared between languages is, unfortunately, impossible.\r

\n

So, the \"lemma\" form (infinitive, etc.) will have to be translated. This, still, is difficult. Is it la piscine or just piscine? Or, for that matter, \"pool\" or \"swimming pool\" (or pool (swimming)).\r

\n

But, then, there will be separate derivative forms for \"walks\", \"walked\", etc., all of which are mono-lingual. For marchons, marchez, etc. it will be a separate set.\r

\n
\n

For (conjunctions, prepositions, etc.) I want to still call them \"grammatical words\".\r

\n

Because a word-for-word \"translation\" is too perilous to attempt.\r

\n
⎯⎯⎯⎯⎯
\n

In practical terms, for verbs, Greenland output will have a pythondict like:\r

\n

V0023 = {\r

\n

\"guid\": \"V0023\",\r

\n

\"english\": \"to eat\",\r

\n

\"lithuanian\": \"valgyti\",\r

\n

\"english_forms\" = {\"first-person-singular-present\": \"I eat\", ...}\r

\n

\"lithuanian_forms\" = {\"first-person-singular-present\": \"A\u0161 valgau\", ...} ...\r

\n

This is similar enough to the current Trakaido format:\r

\n

\"valgyti\": {\r

\n

\"english\": \"to eat\",\r

\n

\"present_tense\": {\r

\n

\"1s\": {\"english\": \"I eat\", \"lithuanian\": \"a\u0161 valgau\"},\r

\n

\"2s\": {\"english\": \"you(s.) eat\", \"lithuanian\": \"tu valgai\"},\r

\n

\"3s-m\": {\"english\": \"he eats\", \"lithuanian\": \"jis valgo\"}, ...\r

\n

For both, it will request \"valgyti / first-person-singular-present\" as a \"flashcard\".\r

\n
\n

The new approach will simplify some of the \"more animals\" style groups.\r

\n

All the animals will be in a single \"dictionary\" file. The first 12 will be in \"Animals 1\", the next 18 in \"Animals 2\", etc.\r

\n

\ud83d\udca1 this will probably come with a reworking of the \"corpus\" system. We mostly want \"levels\" now anyway ... and \"decoy sets\" can be configured separately.

","subject":"lake andes, part 2"},{"channel":"cities","content":"tasks for the next 24 hours:\r\n\r\n# come up with a \"cookbook\" corpus ( consisting of recipes and descriptions of foods) for wordfreq\r\n# get a LLM-script to take a list of 2000 words and return the LONDON ( LONDON is a placeholder; it could be << color >> or << positive adjective >> or << words like devil >>) words.\r\n# write a script to print the \"top 2000 words by part-of-speech\". ( well, actually, I already *have* these lists ... from the previous version of the database) ( maybe \"find the list\" is more accurate)\r\n# write a script that will populate a few of the \"sub-dictionaries\" in the new format ( Countries, Nationalities, Numbers, and Colors will be the first 4, as they are fairly easy to check for completeness)\r\n\r\n( will the \"Trakaido sub-dictionaries\" be the canonical source-of-truth for what the GUIDs are? A flat-file is more cumbersome than a database for adding languages, linking to << derivative forms >>, etc. But, it is easier for humans to read, and to put in Git repos.)\r\n\r\n----\r\n\r\ntasks for the 48 hours after that:\r\n# ensure the categories are stored in the << Lemma >> table.\r\n# re-assess the \"GrammaticalForm\" enum, because it doesn't work across languages. Maybe it needs to be \"EnglishGrammaticalForm\", \"LithuanianGrammaticalForm\", etc.\r\n# generate \"verb forms\" - which requires some form of \"aggregation\" of WordToken entries\r\n# generate all the \"Level 1-5\" entries from wordfreq ( right now, colors like << orange >> are excluded because they are recent borrowings in Lithuanian. this is a very language-specific choice.)\r\n# consider how to handle \"phrases\" ( if you learn << Malonu susipa\u017einti >> before << Malonu >>, it's a phrase) and sentences","created_at":"2025-07-25T15:57:07.455679","id":638,"is_target":false,"parent_id":636,"processed_content":"

tasks for the next 24 hours:\r

\n
    \n
  • come up with a \"cookbook\" corpus \u2699\ufe0f consisting of recipes and descriptions of foods for wordfreq\r
    • \n
  • get a LLM-script to take a list of 2000 words and return the LONDON \u2699\ufe0f LONDON is a placeholder; it could be color or positive adjective or words like devil words.\r
    • \n
  • write a script to print the \"top 2000 words by part-of-speech\". \u2694\ufe0f well, actually, I already have these lists ... from the previous version of the database \ud83d\udca1 maybe \"find the list\" is more accurate\r
    • \n
  • write a script that will populate a few of the \"sub-dictionaries\" in the new format \ud83d\udca1 Countries, Nationalities, Numbers, and Colors will be the first 4, as they are fairly easy to check for completeness\r
    • \n
\n

\ud83d\udca1 will the \"Trakaido sub-dictionaries\" be the canonical source-of-truth for what the GUIDs are? A flat-file is more cumbersome than a database for adding languages, linking to derivative forms, etc. But, it is easier for humans to read, and to put in Git repos.\r

\n
\n

tasks for the 48 hours after that:\r

\n
    \n
  • ensure the categories are stored in the Lemma table.\r
    • \n
  • re-assess the \"GrammaticalForm\" enum, because it doesn't work across languages. Maybe it needs to be \"EnglishGrammaticalForm\", \"LithuanianGrammaticalForm\", etc.\r
    • \n
  • generate \"verb forms\" - which requires some form of \"aggregation\" of WordToken entries\r
    • \n
  • generate all the \"Level 1-5\" entries from wordfreq \ud83d\udca1 right now, colors like orange are excluded because they are recent borrowings in Lithuanian. this is a very language-specific choice.\r
    • \n
  • consider how to handle \"phrases\" \ud83d\udca1 if you learn Malonu susipa\u017einti before Malonu, it's a phrase and sentences
    • \n
","subject":"lake andes, part 3"}]}