1 |
A Statistical Model of Word Rank Evolution ...
|
|
|
|
Abstract:
The availability of large linguistic data sets enables data-driven approaches to study linguistic change. The Google Books corpus unigram frequency data set is used to investigate the word rank dynamics in eight languages. We observed the rank changes of the unigrams from 1900 to 2008 and compared it to a Wright-Fisher inspired model that we developed for our analysis. The model simulates a neutral evolutionary process with the restriction of having no disappearing and added words. This work explains the mathematical framework of the model - written as a Markov Chain with multinomial transition probabilities - to show how frequencies of words change in time. From our observations in the data and our model, word rank stability shows two types of characteristics: (1) the increase/decrease in ranks are monotonic, or (2) the rank stays the same. Based on our model, high-ranked words tend to be more stable while low-ranked words tend to be more volatile. Some words change in ranks in two ways: (a) by an ... : This manuscript - with 31 pages (main), 10 figures (main), 24 pages (supplementary), and 19 figures (supplementary) - is a manuscript for a journal research article ...
|
|
Keyword:
Applications stat.AP; Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2107.09948 https://dx.doi.org/10.48550/arxiv.2107.09948
|
|
BASE
|
|
Hide details
|
|
2 |
Body synchrony in triadic interaction.
|
|
|
|
In: Royal Society open science, vol 7, iss 9 (2020)
|
|
BASE
|
|
Show details
|
|
3 |
Body synchrony in triadic interaction
|
|
|
|
In: R Soc Open Sci (2020)
|
|
BASE
|
|
Show details
|
|
4 |
Language Origins Viewed in Spontaneous and Interactive Vocal Rates of Human and Bonobo Infants
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Social and configural effects on the cognitive dynamics of perspective-taking ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Sequence Memory Constraints Give Rise to Language-Like Structure through Iterated Learning
|
|
|
|
BASE
|
|
Show details
|
|
13 |
The early emergence and puzzling decline of relational reasoning: Effects of knowledge and search on inferring abstract concepts.
|
|
|
|
BASE
|
|
Show details
|
|
14 |
The early emergence and puzzling decline of relational reasoning: Effects of knowledge and search on inferring abstract concepts.
|
|
|
|
BASE
|
|
Show details
|
|
|
|