1 |
The Negative Binomial Model of Word Usage ...
|
|
|
|
Abstract:
How people make texts is a rarely asked but central question in linguistics. From the language engineering perspective texts are outcomes of stochastic processes. A cognitive approach in linguistics holds that the speaker�s intention is the key to text formation. We propose a biologically inspired statistical formulation of word usage that brings together these views. We have observed that in several multilingual text collections1 word frequency distributions in a majority of non-rare words fit the negative binomial distribution (NBD). However, word counts in artificially randomized (permuted) texts agree with the geometric distribution. We conclude that the parameters of NBD deal with linguistic features of words. Specifically, named entities tend to have similar parameter values. We suggest that the NDB in natural texts accounts for intentionality of the speaker. ...
|
|
Keyword:
negative binomial distribution NBD; text randomization; word frequency
|
|
URL: https://dx.doi.org/10.1285/i20705948v6n1p84 http://siba-ese.unisalento.it/index.php/ejasa/article/view/12119
|
|
BASE
|
|
Hide details
|
|
2 |
The Negative Binomial Model of Word Usage
|
|
|
|
In: Electronic Journal of Applied Statistical Analysis; Vol 6, No 1 (2013): Electronic Journal of Applied Statistical Analysis; 84-96 (2013)
|
|
BASE
|
|
Show details
|
|
4 |
Lexical Diversity in a Literary Genre: A Corpus Study of the Rgveda
|
|
|
|
BASE
|
|
Show details
|
|
|
|