Recent work on large language models relies on the intuition that most natural language processing tasks can be described via natural language instructions and that models trained on these instructions show strong zero-shot performance on several standard datasets. However, these models, even though impressive, still perform poorly on a wide range of tasks outside of their respective training and evaluation sets.
There is wide debate about the degree to which the properties of human cognition affect how languages are structured and how they change over time. This controversy extends to the lexicon. We show that the decline of lexical items can be partially accounted for by biases that have been demonstrated in the cognitive science literature.
Our world is extremely complex, and yet we are able to exchange our thoughts and beliefs about it using a relatively small number of words. What computational principles can explain this extraordinary ability?