{"channel":"cities","content":" Jamestown, North Dakota, is located along I-94 in the eastern half of the state.\r\n\r\nToday's focus is on \"word frequency\".\r\n\r\n----\r\n\r\n <<< I started with two corpuses: one of 19th century literature (from Project Gutenberg), one of 20th century \"sci-fi\" literature. I got a rough word-rank for each, and combined them ( using the harmonic mean) to get a combined word-list. >>>\r\n\r\n <<< While many high-frequency function words such as *the*, *and*, and *of* maintain consistent rankings, others like *said*, *her*, *she*, and *me* show substantial divergence, suggesting notable stylistic or thematic shifts between the two periods and genres. >>>\r\n\r\n----\r\n\r\nIt is also a word-list at all. Some of the notes:\r\n\r\n# The word \"whale\" shows up a lot more in the 19th century corpus. This is because one of the books is [[Moby Dick]].\r\n# I am hoping to run an exhaustive listing of a few attributes. These include:\r\n> polysemy. ( I am less concerned with words like << get >> which have so many meanings as-to be indefinable, but instead words like << saw >> (\u770b or \u952f\u5b50) or << face >> (\u9762\u5411 or \u8138))\r\n> by lemma. \"went\" (108th) v. \"go\" (80th).\r\n> by part-of-speech. defined as \"what the LLMs define as part-of-speech\".\r\n> a \"second-level\" of word-type details.","created_at":"2025-04-06T20:39:52.199751","id":343,"llm_annotations":{},"parent_id":null,"processed_content":"

\n \u2699\ufe0f\n Jamestown, North Dakota, is located along I-94 in the eastern half of the state.\r\n

\n

Today's focus is on \"word frequency\".\r

I started with two corpuses: one of 19th century literature (from Project Gutenberg), one of 20th century \"sci-fi\" literature. I got a rough word-rank for each, and combined them \n \u2699\ufe0f\n ( using the harmonic mean)\n to get a combined word-list.

\r

\n

While many high-frequency function words such as the, and, and of maintain consistent rankings, others like said, her, she, and me show substantial divergence, suggesting notable stylistic or thematic shifts between the two periods and genres.

\r

It is also a word-list at all. Some of the notes:\r

\n
    \n
  • The word \"whale\" shows up a lot more in the 19th century corpus. This is because one of the books is Moby Dick.\r
    • \n
  • I am hoping to run an exhaustive listing of a few attributes. These include:\r
    • \n
\n
    \n
  • polysemy. \n \u2699\ufe0f\n ( I am less concerned with words like get which have so many meanings as-to be indefinable, but instead words like saw (\u770b or \u952f\u5b50) or face (\u9762\u5411 or \u8138))\n \r
    • \n
  • by lemma. \"went\" (108th) v. \"go\" (80th).\r
    • \n
  • by part-of-speech. defined as \"what the LLMs define as part-of-speech\".\r
    • \n
  • a \"second-level\" of word-type details.
    • \n
","quotes":[],"subject":"jamestown (part 1)"}