Publications

The Architectural Bottleneck Principle

In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into …
Tiago Pimentel*, Josef Valvoda*, Niklas Stoehr, Ryan Cotterell
PDF URL
The Architectural Bottleneck Principle

On Parsing as Tagging

There have been many proposals to reduce constituency parsing to tagging in the literature. To better understand what these approaches …
Afra Amini, Ryan Cotterell
PDF URL
On Parsing as Tagging

Mutual Information and Hallucinations in Abstractive Summarization

Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the …
Liam van der Poel, Ryan Cotterell, Clara Meister
PDF URL
Mutual Information and Hallucinations in Abstractive Summarization

Kernelized Concept Erasure

One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure …
Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, Ryan Cotterell
PDF URL
Kernelized Concept Erasure

Autoregressive Structure Prediction with Language Models

In recent years, NLP has moved towards the application of language models to a more diverse set of tasks. However, applying …
Tianyu Liu, Yuchen Jiang, Nicholas Monath, Ryan Cotterell, Mrinmaya Sachan
PDF URL
Autoregressive Structure Prediction with Language Models

Algorithms for Weighted Pushdown Automata

Weighted pushdown automata (WPDAs) are at the core of many natural language processing tasks, like syntax-based statistical machine …
Alexandra Butoi, Brian DuSell, Tim Vieira, Ryan Cotterell, David Chiang
PDF URL
Algorithms for Weighted Pushdown Automata

Algorithms for Weighted Finite-State Automata with Failure Arcs

Weighted finite-state automata (WSFAs) are commonly used in NLP. Failure transitions are a useful extension for compactly representing …
Anej Svete, Benjamin Dayan, Tim Vieira, Ryan Cotterell, Jason Eisner

Equivariant Transduction through Invariant Alignment

The ability to generalize compositionally is key to understanding the potentially infinite number of sentences that can be constructed …
Jennifer White, Ryan Cotterell
URL

Benchmarking Compositionality with Formal Languages

Recombining known primitive concepts into larger novel combinations is a quintessentially human cognitive capability. Whether large …
Josef Valvoda, Naomi Saphra, Jon Rawski, Adina Williams, Ryan Cotterell
PDF URL
Benchmarking Compositionality with Formal Languages

The SIGTYP 2022 Shared Task on the Prediction of Cognate Reflexes

This study describes the structure and the results of the SIGTYP 2022 shared task on the prediction of cognate reflexes from …
Johann-Mattis List, Ekaterina Vylomova, Robert Forkel, Nathan Hill, Ryan Cotterell

The SIGMORPHON 2022 Shared Task on Morpheme Segmentation

The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes and …
Khuyagbaatar Batsuren, Gábor Bella, Aryaman Arora, Viktor Martinovic, Kyle Gorman, Zdeněk Žabokrtský, Amarsanaa Ganbold, Šárka Dohnalová, Magda Ševčíková, Kateřina Pelegrinová, Fausto Giunchiglia, Ryan Cotterell, Ekaterina Vylomova

SIGMORPHON--UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection

The 2022 SIGMORPHON–UniMorph shared task on large scale morphological inflection generation included a wide range of typologically …
Jordan Kodner, Salam Khalifa, Khuyagbaatar Batsuren, Hossep Dolatian, Ryan Cotterell, Faruk Akkus, Antonios Anastasopoulos, Taras Andrushko, Aryaman Arora, Nona Atanalov, Gábor Bella, Elena Budianskaya, Yustinus Ghanggo Ate, Omer Goldman, David Guriel, Simon Guriel, Silvia Guriel-Agiashvili, Witold Kieraś, Andrew Krizhanovsky, Natalia Krizhanovsky, Igor Marchenko, Magdalena Markowska, Polina Mashkovtseva, Maria Nepomniashchaya, Daria Rodionova, Karina Scheifer, Alexandra Sorova, Anastasia Yemelina, Jeremiah Young, Ekaterina Vylomova

Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models

The success of multilingual pre-trained models is underpinned by their ability to learn representations shared by multiple languages …
Karolina Stańczak, Edoardo Ponti, Lucas Torroba Hennigen, Ryan Cotterell, Isabelle Augenstein
PDF URL
Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models

Probing via Prompting

Probing is a popular approach to understand what linguistic information is contained in the representations of pre-trained language …
Jiaoda Li, Ryan Cotterell, Mrinmaya Sachan
PDF URL
Probing via Prompting

On the Machine Learning of Ethical Judgments from Natural Language

Ethics is one of the longest standing intellectual endeavors of humanity. In recent years, the fields of AI and NLP have attempted to …
Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, Adina Williams
PDF URL

Linear Adversarial Concept Erasure

Modern neural models trained on textual data rely on pre-trained representations that emerge without direct supervision. As these …
Shauli Ravfogel, Michael Twiton, Yoav Goldberg, Ryan Cotterell
PDF URL
Linear Adversarial Concept Erasure

Exact Paired-Permutation Testing for Structured Test Statistics

Significance testing—especially the paired-permutation test—has played a vital role in developing NLP systems to provide confidence …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF URL
Exact Paired-Permutation Testing for Structured Test Statistics

BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation

Standard automatic metrics, e.g. BLEU, are not reliable for document-level MT evaluation. They can neither distinguish document-level …
Yuchen Eleanor Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, Ming Zhou
PDF URL
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation

A Structured Span Selector

Many natural language processing tasks, e.g., coreference resolution and semantic role labeling, require selecting text spans and …
Tianyu Liu, Yuchen Eleanor Jiang, Ryan Cotterell, Mrinmaya Sachan
PDF URL
A Structured Span Selector

Probing for the Usage of Grammatical Number

A central quest of probing is to uncover how pre-trained models encode a linguistic property within their representations. An encoding, …
Karim Lasri, Tiago Pimentel, Alessandro Lenci, Thierry Poibeau, Ryan Cotterell
PDF URL

Probing as Quantifying the Inductive Bias of Pre-trained Representations

Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance …
Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin, Ryan Cotterell
PDF URL
Probing as Quantifying the Inductive Bias of Pre-trained Representations

On the probability-quality paradox in language generation

When generating natural language from neural probabilistic models, high probability does not always coincide with high quality: It has …
On the probability-quality paradox in language generation
PDF URL
On the probability-quality paradox in language generation

Estimating the Entropy of Linguistic Distributions

Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language. However, …
Aryaman Arora, Clara Meister, Ryan Cotterell
PDF URL

Analyzing Wrap-Up Effects through an Information-Theoretic Lens

Numerous analyses of reading time (RT) data have been undertaken in the effort to learn more about the internal processes that occur …
Clara Meister, Tiago Pimentel, Thomas Hikaru Clark, Ryan Cotterell, Roger P. Levy
PDF URL
Analyzing Wrap-Up Effects through an Information-Theoretic Lens

Visual Comparison of Language Model Adaptation

Neural language models are widely used; however, their model parameters often need to be adapted to the specific domains and tasks of …
Rita Sevastjanova, Eren Cakmak, Shauli Ravfogel, Ryan Cotterell, Mennatallah El-Assady

State-of-the-art generalisation research in NLP: a taxonomy and review

Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, Zhijing Jin
URL

On the Role of Negative Precedent in Legal Outcome Prediction

Every legal case sets a precedent by devel- oping the law in one of the following two ways. It either expands its scope, in which case …
Josef Valvoda, Simone Teufel, Ryan Cotterell
PDF
On the Role of Negative Precedent in Legal Outcome Prediction

On the Intersection of Context-Free and Regular Languages

The Bar-Hillel construction is a classic result in formal language theory. It shows, by construction, that the intersection between a …
Clemente Pasti, Andreas Opedal, Tiago Pimentel, Tim Vieira, Jason Eisner, Ryan Cotterell
PDF URL

On Decoding Strategies for Neural Text Generators

When generating text from probabilistic models, the chosen decoding strategy has a profound effect on the resulting text. Yet …
Gian Wiher, Clara Meister, Ryan Cotterell
PDF URL

Naturalistic Causal Probing for Morpho-Syntax

Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing. However, there …
Afra Amini, Tiago Pimentel, Clara Meister, Ryan Cotterell
PDF URL
Naturalistic Causal Probing for Morpho-Syntax

Locally Typical Sampling

Today’s probabilistic language generators fall short when it comes to producing coherent and fluent text, despite the fact that …
Clara Meister, Tiago Pimentel, Gian Wiher, Ryan Cotterell
PDF URL
Locally Typical Sampling

Cluster-based Evaluation of Automatically Generated Text

While probabilistic language generators have improved dramatically over the last few years, the automatic evaluation metrics used to …
Tiago Pimentel, Clara Meister, Ryan Cotterell
PDF URL
Cluster-based Evaluation of Automatically Generated Text

An Ordinal Latent Variable Model of Conflict Intensity

For the quantitative monitoring of international relations, political events are extracted from the news and parsed into …
Niklas Stoehr, Lucas Torroba Hennigen, Josef Valvoda, Robert West, Ryan Cotterell, Aaron Schein
PDF URL

A Word on Machine Ethics: A Response to Jiang et al. (2021)

Ethics is one of the longest standing intellectual endeavors of humanity. In recent years, the fields of AI and NLP have attempted to …
Zeerak Talat$^*$, Hagen Blix$^*$, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, Adina Williams
PDF
A Word on Machine Ethics: A Response to Jiang et al. (2021)

Searching for More Efficient Dynamic Programs

Computational models of human language often involve combinatorial problems. For instance, a probabilistic parser may marginalize over …
Tim Vieira, Ryan Cotterell, Jason Eisner
PDF arXiv
Searching for More Efficient Dynamic Programs

Revisiting the Uniform Information Density Hypothesis

The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that …
Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger, Ryan Cotterell, Roger Levy
PDF Code arXiv
Revisiting the Uniform Information Density Hypothesis

On Homophony and Rényi Entropy

Homophony’s widespread presence in natural languages is a controversial topic. Recent theories of language optimality have tried …
Tiago Pimentel, Clara Meister, Simone Teufel, Ryan Cotterell
PDF Code arXiv
On Homophony and Rényi Entropy

Efficient Sampling of Dependency Structure

Probabilistic distributions over spanning trees in directed graphs are a fundamental model of dependency structure in natural language …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF Code arXiv
Efficient Sampling of Dependency Structure

Conditional Poisson Stochastic Beam Search

Beam search is the default decoding strategy for many sequence generation tasks in NLP. The set of approximate K-best items returned by …
Clara Meister, Afra Amini, Tim Vieira, Ryan Cotterell
PDF Code arXiv
Conditional Poisson Stochastic Beam Search

Classifying Dyads for Militarized Conflict Analysis

Understanding the origins of militarized conflict is a complex, yet important undertaking. Existing research seeks to build this …
Niklas Stoehr, Lucas Torroba Hennigen, Samin Ahbab, Robert West, Ryan Cotterell
PDF Code arXiv
Classifying Dyads for Militarized Conflict Analysis

A surprisal--duration trade-off across and within the world's languages

While there exist scores of natural languages, each with its unique features and idiosyncrasies, they all share a unifying theme: …
Tiago Pimentel, Clara Meister, Elizabeth Salesky, Simone Teufel, Damián Blasi, Ryan Cotterell
PDF Code arXiv
A surprisal--duration trade-off across and within the world's languages

A Plug-and-Play Method for Controlled Text Generation

Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, …
Damian Pascual, Beni Egressy, Clara Meister, Ryan Cotterell, Roger Wattenhofer
PDF Code arXiv
A Plug-and-Play Method for Controlled Text Generation

A Bayesian Framework for Information-Theoretic Probing

Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as …
Tiago Pimentel, Ryan Cotterell
PDF Code arXiv
A Bayesian Framework for Information-Theoretic Probing

On Finding the $K$-best Non-projective Dependency Trees

The connection between the maximum spanning tree in a directed graph and the best dependency tree of a sentence has been exploited by …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
On Finding the $K$-best Non-projective Dependency Trees

Modeling the Unigram Distribution

The unigram distribution is the non-contextual probability of finding a specific word form in a corpus. While of central importance to …
Irene Nikkarinen*, Tiago Pimentel*, Damián Blasi, Ryan Cotterell
PDF Code Anthology arXiv
Modeling the Unigram Distribution

Language Model Evaluation Beyond Perplexity

We propose an alternate approach to quantifying how well language models learn natural language: we ask how well they match the …
Clara Meister, Ryan Cotterell
PDF Anthology arXiv
Language Model Evaluation Beyond Perplexity

Is Sparse Attention more Interpretable?

Sparse attention has been claimed to increase model interpretability under the assumption that it highlights influential inputs. Yet …
Clara Meister, Stefan Lazov, Isabelle Augenstein, Ryan Cotterell
PDF Anthology arXiv
Is Sparse Attention more Interpretable?

Higher-order Derivatives of Weighted Finite-state Machines

Weighted finite-state machines are a fundamental building block of NLP systems. They have withstood the test of time—from their early …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
Higher-order Derivatives of Weighted Finite-state Machines

Examining the Inductive Bias of Neural Language Models with Artificial Languages

Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the …
Jennifer C. White, Ryan Cotterell
PDF Code Anthology arXiv
Examining the Inductive Bias of Neural Language Models with Artificial Languages

Determinantal Beam Search

Beam search is a go-to strategy for decoding neural sequence models. The algorithm can naturally be viewed as a subset optimization …
Clara Meister, Martina Forster, Ryan Cotterell
PDF Code Anthology arXiv
Determinantal Beam Search

A cognitive regularizer for language modeling

The uniform information density (UID) hypothesis, which posits that speakers prefer utterances that distribute information uniformly …
Jason Wei, Clara Meister, Ryan Cotterell
PDF Anthology arXiv
A cognitive regularizer for language modeling

What About the Precedent: An Information-Theoretic Analysis of Common Law

In common law, the outcome of a new case is determined mostly by precedent cases, rather than by existing statutes. However, how …
Josef Valvoda, Tiago Pimentel, Niklas Stoehr, Ryan Cotterell, Simone Teufel
PDF Code Anthology arXiv
What About the Precedent: An Information-Theoretic Analysis of Common Law

How (Non-)Optimal is the Lexicon?

The mapping of lexical meanings to wordforms is a major feature of natural languages. While usage pressures might assign short words to …
Tiago Pimentel*, Irene Nikkarinen*, Kyle Mahowald, Ryan Cotterell, Damián Blasi
PDF Anthology arXiv
How (Non-)Optimal is the Lexicon?

Finding Concept-specific Biases in Form--Meaning Associations

This work presents an information-theoretic operationalisation of cross-linguistic non-arbitrariness. It is not a new idea that there …
Tiago Pimentel, Brian Roark, Søren Wichmann, Ryan Cotterell, Damián Blasi
PDF Code Anthology arXiv
Finding Concept-specific Biases in Form--Meaning Associations

Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing

Analysing whether neural language models encode linguistic information has become popular in NLP. One method of doing so, which is …
Rowan Hall Mauslay, Ryan Cotterell
PDF Code Anthology arXiv
Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing

A Non-Linear Structural Probe

Probes are models devised to investigate the encoding of knowledge—e.g. syntactic structure—in contextual representations. Probes are …
Jennifer C. White, Tiago Pimentel, Naomi Saphra, Ryan Cotterell
PDF Anthology arXiv
A Non-Linear Structural Probe

Searching for Search Errors in Neural Morphological Inflection

Neural sequence-to-sequence models are currently the predominant choice for language generation tasks. Yet, on word-level tasks, exact …
Martina Forster, Clara Meister, Ryan Cotterell
PDF Code Anthology arXiv
Searching for Search Errors in Neural Morphological Inflection

Disambiguatory signals are stronger in word initial positions

Psycholinguistic studies of human word processing and lexical access provide ample evidence of the preferred nature of word-initial …
Tiago Pimentel, Ryan Cotterell, Brian Roark
PDF Code Anthology arXiv
Disambiguatory signals are stronger in word initial positions

Applying the Transformer to Character-level Transduction

The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP …
Shijie Wu, Mans Hulden, Ryan Cotterell
PDF Code Anthology arXiv
Applying the Transformer to Character-level Transduction

Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages

Most combinations of NLP tasks and language varieties lack in-domain examples for supervised training because of the paucity of …
Edoardo M. Ponti, Ivan Vulić, Ryan Cotterell, Marinela Parović, Roi Reichart, Anna Korhonen
PDF Code Anthology arXiv
Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages

On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs

We use large-scale corpora in six different gendered languages, along with tools from NLP and information theory, to test whether there …
Adina Williams, Ryan Cotterell, Lawrence Wolf-Sonkin, Damian Blasi, Hanna Wallach
PDF Anthology arXiv
On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

Large-scale pretraining and task-specific fine-tuning is now the standard methodology for many tasks in computer vision and natural …
Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki, Desmond Elliott
PDF Code Anthology arXiv
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs

Efficient Computation of Expectations under Spanning Tree Distributions

We give a general framework for inference in spanning tree models. We propose unified algorithms for the important cases of first-order …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
Efficient Computation of Expectations under Spanning Tree Distributions

Differentiable Subset Pruning of Transformer Heads

Jiaoda Li, Ryan Cotterell, Mrinmaya Sachan
PDF Code arXiv

Morphologically Aware Word-Level Translation

We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and …
Paula Czarnowska, Sebastian Ruder, Ryan Cotterell, Ann Copestake
PDF Anthology arXiv
Morphologically Aware Word-Level Translation

Speakers Fill Semantic Gaps with Context

Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more …
Tiago Pimentel, Rowan Hall Maudslay, Damián Blasi, Ryan Cotterell
PDF Code Anthology arXiv
Speakers Fill Semantic Gaps with Context

SIGTYP 2020 Shared Task: Prediction of Typological Features

Typological knowledge bases (KBs) such as WALS (Dryer and Haspelmath, 2013) contain information about linguistic properties of the …
Johannes Bjerva, Elizabeth Salesky, Sabrina J. Mielke, Aditi Chaudhary, Giuseppe G. A. Celano, Edoardo M. Ponti, Ekaterina Vylomova, Ryan Cotterell, Isabelle Augenstein
PDF Code Anthology arXiv
SIGTYP 2020 Shared Task: Prediction of Typological Features

Please Mind the Root: Decoding Arborescences for Dependency Parsing

The connection between dependency trees and spanning trees is exploited by the NLP community to train and to decode graph-based …
Ran Zmigrod, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
Please Mind the Root: Decoding Arborescences for Dependency Parsing

Pareto Probing: Trading Off Accuracy for Simplicity

The question of how to probe contextual word representations for linguistic structure in a way that is both principled and useful has …
Tiago Pimentel, Naomi Saphra, Adina Williams, Ryan Cotterell
PDF Code Anthology arXiv
Pareto Probing: Trading Off Accuracy for Simplicity

Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions

A grammatical gender system divides a lexicon into a small number of relatively fixed grammatical categories. How similar are these …
Arya D. McCarthy, Adina Williams, Shijia Liu, David Yarowsky, Ryan Cotterell
PDF Anthology
Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions

Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model

Across languages, multiple consecutive adjectives modifying a noun (e.g. ``the big red dog″) follow certain unmarked ordering rules. …
Jun Yen Leung, Guy Emerson, Ryan Cotterell
PDF Anthology arXiv
Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model

Intrinsic Probing through Dimension Selection

Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of …
Lucas Torroba Hennigen, Adina Williams, Ryan Cotterell
PDF Code Anthology arXiv

Intrinsic Probing through Dimension Selection

Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of …
Lucas Torroba Hennigen, Adina Williams, Ryan Cotterell
PDF Code Anthology arXiv
Intrinsic Probing through Dimension Selection

If Beam Search is the Answer, What was the Question?

Quite surprisingly, exact maximum a posteriori (MAP) decoding of neural language generators frequently leads to low-quality results. …
Clara Meister, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
If Beam Search is the Answer, What was the Question?

Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained …
Francisco Vargas, Ryan Cotterell
PDF Code Anthology arXiv
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

The Paradigm Discovery Problem

This work treats the paradigm discovery problem (PDP)—the task of learning an inflectional morphological system from unannotated …
Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell, Nizar Habash
PDF Code Anthology arXiv
The Paradigm Discovery Problem

SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection

A broad goal in natural language processing (NLP) is to develop a system that has the capacity to process any natural language. Most …
Ekaterina Vylomova, Jennifer White, Elizabeth Salesky, Sabrina J. Mielke, Shijie Wu, Edoardo Maria Ponti, Rowan Hall Maudslay, Ran Zmigrod, Josef Valvoda, Svetlana Toldova, Francis Tyers, Elena Klyachko, Ilya Yegorov, Natalia Krizhanovsky, Paula Czarnowska, Irene Nikkarinen, Andrew Krizhanovsky, Tiago Pimentel, Lucas Torroba Hennigen, Christo Kirov, Garrett Nicolai, Adina Williams, Antonios Anastasopoulos, Hilaria Cruz, Eleanor Chodroff, Ryan Cotterell, Miikka Silfverberg, Mans Hulden
PDF Code Anthology arXiv
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection

Predicting Declension Class from Form and Meaning

The noun lexica of many natural languages are divided into several declension classes with characteristic morphological properties. …
Adina Williams, Tiago Pimentel, Arya McCarthy, Hagen Blix, Eleanor Chodroff, Ryan Cotterell
PDF Code Anthology arXiv
Predicting Declension Class from Form and Meaning

Metaphor Detection Using Context and Concreteness

We report the results of our system on the Metaphor Detection Shared Task at the Second Workshop on Figurative Language Processing …
Rowan Hall Maudslay, Tiago Pimentel, Ryan Cotterell, Simone Teufel
PDF Anthology
Metaphor Detection Using Context and Concreteness

It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information

The performance of neural machine translation systems is commonly evaluated in terms of BLEU. However, due to its reliance on target …
Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos, Ryan Cotterell, Naoaki Okazaki
PDF Code Anthology arXiv
It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information

Information-Theoretic Probing for Linguistic Structure

The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually know …
Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, Ryan Cotterell
PDF Code Anthology arXiv
Information-Theoretic Probing for Linguistic Structure

Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing

Prior work has explored directly regularizing the output distributions of probabilistic models to alleviate peaky (i.e. over-confident) …
Clara Meister, Elizabeth Salesky, Ryan Cotterell
PDF Code Anthology arXiv
Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing

A Tale of a Probe and a Parser

Measuring what linguistic information is encoded in continuous representations of language has become a popular area of research. To do …
Rowan Hall Maudslay, Josef Valvoda, Tiago Pimentel, Adina Williams, Ryan Cotterell
PDF Anthology arXiv
A Tale of a Probe and a Parser

A Corpus for Large-Scale Phonetic Typology

A major hurdle in data-driven research on typology is having sufficient data in many languages to draw meaningful conclusions. We …
Elizabeth Salesky, Eleanor Chodroff, Tiago Pimentel, Matthew Wiesner, Ryan Cotterell, Alan W Black, Jason Eisner
PDF Code Anthology arXiv
A Corpus for Large-Scale Phonetic Typology

UniMorph 3.0: Universal Morphology

The Universal Morphology (UniMorph) project is a collaborative effort providing broad-coverage instantiated normalized morphological …
Arya D. McCarthy, Christo Kirov, Matteo Grella, Amrit Nidhi, Patrick Xia, Kyle Gorman, Ekaterina Vylomova, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, Timofey Arkhangelskiy, Nataly Krizhanovsky, Andrew Krizhanovsky, Elena Klyachko, Alexey Sorokin, John Mansfield, Valts Ernštreits, Yuval Pinter, Cassandra L. Jacobs, Ryan Cotterell, Mans Hulden, David Yarowsky
PDF Anthology
UniMorph 3.0: Universal Morphology

SIGMORPHON 2020 Task 0 System Description: ETH Zürich Team

This paper presents our system for the SIGMORPHON 2020 Shared Task. We build off of the baseline systems, performing exact inference on …
Martina Forster, Clara Meister
PDF Anthology
SIGMORPHON 2020 Task 0 System Description: ETH Zürich Team

Phonotactic Complexity and its Trade-offs

We present methods for calculating a measure of phonotactic complexity—bits per phoneme—that permits a straightforward cross-linguistic …
Tiago Pimentel, Brian Roark, Ryan Cotterell
PDF Code Anthology arXiv
Phonotactic Complexity and its Trade-offs

Best-First Beam Search

Decoding for many NLP tasks requires a heuristic algorithm for approximating exact search since the full search space is often …
Clara Meister, Tim Vieira, Ryan Cotterell
PDF Code Anthology arXiv
Best-First Beam Search

Towards Zero-Shot Language Modeling

Can we construct a neural language model which is inductively biased towards learning human language? Motivated by this question, we …
Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell, Roi Reichart, Anna Korhonen
PDF Anthology arXiv
Towards Zero-Shot Language Modeling

Quantifying the Semantic Core of Gender Systems

Many of the world’s languages employ grammatical gender on the lexeme. For instance, in Spanish, house “casa” is …
Adina Williams, Ryan Cotterell, Lawrence Wolf-Sonkin, Damian Blasi, Hanna Wallach
PDF Anthology arXiv
Quantifying the Semantic Core of Gender Systems

It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as …
Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, Simone Teufel
PDF Anthology arXiv
It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution

Examining Gender Bias in Languages with Grammatical Gender

Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora. However, most studies to date …
Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, Kai-Wei Chang
PDF Anthology arXiv
Examining Gender Bias in Languages with Grammatical Gender

Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Human translators routinely have to translate rare inflections of words–due to the Zipfian distribution of words in a language. …
Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, Ann Copestake
PDF Anthology arXiv
Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of …
Arya D. McCarthy, Ekaterina Vylomova, Shijie Wu, Chaitanya Malaviya, Lawrence Wolf-Sonkin, Garrett Nicolai, Christo Kirov, Miikka Silfverberg, Sabrina Mielke, Jeffrey Heinz, Ryan Cotterell, Mans Hulden
PDF Anthology arXiv
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

What Kind of Language Is Hard to Language-Model?

How language-agnostic are current state-of-the-art NLP tools? Are there some types of language that are easier to model with current …
Sabrina Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, Jason Eisner
PDF Anthology arXiv
What Kind of Language Is Hard to Language-Model?

Unsupervised Discovery of Gendered Language through Latent-Variable Modeling

Studying the ways in which language is gendered has long been an area of interest in sociolinguistics. Studies have explored, for …
Alexander M. Hoyle, Lawrence Wolf-Sonkin, Hanna Wallach, Isabelle Augenstein, Ryan Cotterell
PDF Anthology arXiv
Unsupervised Discovery of Gendered Language through Latent-Variable Modeling

Uncovering Typological Implications with Belief Nets

The study of linguistic typology is rooted in the implications we find between linguistic features, such as the fact that languages …
Johannes Bjerva, Yova Kementchedjhieva, Ryan Cotterell, Isabelle Augenstein
PDF Anthology arXiv
Uncovering Typological Implications with Belief Nets

On the distribution of deep clausal embeddings: A large cross-linguistic study

Embedding a clause inside another (``the girl [who likes cars [that run fast]] has arrived″) is a fundamental resource that has been …
Damian Blasi, Ryan Cotterell, Lawrence Wolf-Sonkin, Sabine Stoll, Balthasar Bickel, Marco Baroni
PDF Anthology
On the distribution of deep clausal embeddings: A large cross-linguistic study

Measuring Morphological Irregularity

We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity …
Shijie Wu, Ryan Cotterell, Timothy J. O'Donnell
PDF Anthology arXiv
Measuring Morphological Irregularity

Meaning to Form: Measuring Systematicity as Information

A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an …
Tiago Pimentel, Arya McCarthy, Damian Blasi, Brian Roark, Ryan Cotterell
PDF Anthology arXiv
Meaning to Form: Measuring Systematicity as Information

Exact Hard Monotonic Attention for Character-Level Transduction

Many common character-level, string-to-string transduction tasks, e.g., grapheme-to-phoneme conversion and morphological inflection, …
Shijie Wu, Ryan Cotterell
PDF Anthology arXiv
Exact Hard Monotonic Attention for Character-Level Transduction

Counterfactual Data Augmentation for Mitigating Gender Bias in Languages with Rich Morphology

Gender stereotypes are manifest in most of the world’s languages and are consequently propagated or amplified by NLP systems. …
Ran Zmigrod, Sabrina Mielke, Hanna Wallach, Ryan Cotterell
PDF Anthology arXiv
Counterfactual Data Augmentation for Mitigating Gender Bias in Languages with Rich Morphology

On the Idiosyncrasies of the Mandarin Chinese Classifier System

While idiosyncrasies of the Chinese classifier system have been a richly studied topic among linguists (Adams and Conklin, 1973; …
Shijia Liu, Adina Williams, Hongyuan Mei, Ryan Cotterell
PDF Anthology arXiv
On the Idiosyncrasies of the Mandarin Chinese Classifier System

Gender Bias in Contextualized Word Embeddings

In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo’s contextualized word vectors. First, we conduct …
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, Kai-Wei Chang
PDF Anthology arXiv
Gender Bias in Contextualized Word Embeddings

Contextualization of Morphological Inflection

Critical to natural language generation is the production of correctly inflected text. In this paper, we isolate the task of predicting …
Ekaterina Vylomova, Ryan Cotterell, Timothy Baldwin, Trevor Cohn, Jason Eisner
PDF Anthology arXiv
Contextualization of Morphological Inflection

Combining Sentiment Lexica with a Multi-View Variational Autoencoder

When assigning quantitative labels to a dataset, different methodologies may rely on different scales. In particular, when assigning …
Alexander Hoyle, Lawrence Wolf-Sonkin, Hanna Wallach, Ryan Cotterell, Isabelle Augenstein
PDF Anthology arXiv
Combining Sentiment Lexica with a Multi-View Variational Autoencoder

A Simple Joint Model for Improved Contextual Neural Lemmatization

English verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP …
Chaitanya Malaviya, Shijie Wu, Ryan Cotterell
PDF Anthology arXiv
A Simple Joint Model for Improved Contextual Neural Lemmatization

A Probabilistic Generative Model of Linguistic Typology

In the principles-and-parameters framework, the structural features of languages depend on parameters that may be toggled on or off, …
Johannes Bjerva, Yova Kementchedjhieva, Ryan Cotterell, Isabelle Augenstein
PDF Anthology arXiv
A Probabilistic Generative Model of Linguistic Typology

On the Complexity and Typology of Inflectional Morphological Systems

We quantify the linguistic complexity of different languages’ morphological systems. We verify that there is a statistically …
Ryan Cotterell, Christo Kirov, Mans Hulden, Jason Eisner
PDF Anthology arXiv
On the Complexity and Typology of Inflectional Morphological Systems

The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

The CoNLL-SIGMORPHON 2018 shared task on supervised learning of morphological generation featured data sets from 103 typologically …
Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D. McCarthy, Katharina Kann, Sabrina Mielke, Garrett Nicolai, Miikka Silfverberg, David Yarowsky, Jason Eisner, Mans Hulden
PDF Anthology arXiv
The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

Marrying Universal Dependencies and Universal Morphology

The Universal Dependencies (UD) and Universal Morphology (UniMorph) projects each present schemata for annotating the morphosyntactic …
Arya D. McCarthy, Miikka Silfverberg, Ryan Cotterell, Mans Hulden, David Yarowsky
PDF Anthology arXiv
Marrying Universal Dependencies and Universal Morphology

Hard Non-Monotonic Attention for Character-Level Transduction

Character-level string-to-string transduction is an important component of various NLP tasks. The goal is to map an input string to an …
Shijie Wu, Pamela Shapiro, Ryan Cotterell
PDF Anthology arXiv
Hard Non-Monotonic Attention for Character-Level Transduction

Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction

Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages. We …
Yova Kementchedjhieva, Sebastian Ruder, Ryan Cotterell, Anders Søgaard
PDF Anthology arXiv
Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction

A Discriminative Latent-Variable Model for Bilingual Lexicon Induction

We introduce a novel discriminative latent-variable model for the task of bilingual lexicon induction. Our model combines the bipartite …
Sebastian Ruder$^*$, Ryan Cotterell$^*$, Yova Kementchedjhieva, Anders Søgaard
PDF Anthology arXiv
A Discriminative Latent-Variable Model for Bilingual Lexicon Induction

A Structured Variational Autoencoder for Contextual Morphological Inflection

Statistical morphological inflectors are typically trained on fully supervised, type-level data. One remaining open research question …
Lawrence Wolf-Sonkin$^*$, Jason Naradowsky$^*$, Sabrina J. Mielke$^*$, Ryan Cotterell$^*$
PDF Anthology arXiv
A Structured Variational Autoencoder for Contextual Morphological Inflection

Unsupervised Disambiguation of Syncretism in Inflected Lexicons

Lexical ambiguity makes it difficult to compute useful statistics of a corpus. A given word form might represent any of several …
Ryan Cotterell, Christo Kirov, Sabrina J. Mielke, Jason Eisner
PDF Anthology arXiv
Unsupervised Disambiguation of Syncretism in Inflected Lexicons

Are All Languages Equally Hard to Language-Model?

For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on …
Ryan Cotterell, Sabrina J. Mielke, Jason Eisner, Brian Roark
PDF Anthology arXiv
Are All Languages Equally Hard to Language-Model?

A Deep Generative Model of Vowel Formant Typology

What makes some types of languages more probable than others? For instance, we know that almost all spoken languages contain the vowel …
Ryan Cotterell, Jason Eisner
PDF Anthology arXiv
A Deep Generative Model of Vowel Formant Typology

UniMorph 2.0: Universal Morphology

The Universal Morphology (UniMorph) project is a collaborative effort to improve how NLP handles complex morphology across the world’s …
Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina Mielke, Arya McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner, Mans Hulden
PDF Anthology arXiv
UniMorph 2.0: Universal Morphology

Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

Can advances in NLP help advance cognitive modeling? We examine the role of artificial neural networks, the current state of the art in …
Christo Kirov, Ryan Cotterell
PDF Anthology arXiv
Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

On the Diachronic Stability of Irregularity in Inflectional Morphology

Ryan Cotterell, Christo Kirov, Mans Hulden, Jason Eisner
PDF Anthology arXiv
On the Diachronic Stability of Irregularity in Inflectional Morphology

Joint Semantic Synthesis and Morphological Analysis of the Derived Word

Much like sentences are composed of words, words themselves are composed of smaller units. For example, the English word questionably …
Ryan Cotterell, Hinrich Schütze
PDF Anthology arXiv
Joint Semantic Synthesis and Morphological Analysis of the Derived Word

Explaining and Generalizing Back-Translation through Wake-Sleep

Ryan Cotterell, Julia Kreutzer
PDF Anthology arXiv
Explaining and Generalizing Back-Translation through Wake-Sleep

Low-Resource Named Entity Recognition with Cross-lingual, Character-Level Neural Conditional Random Fields

Low-resource named entity recognition is still an open problem in NLP. Most state-of-the-art systems require tens of thousands of …
Ryan Cotterell, Kevin Duh.
PDF Anthology
Low-Resource Named Entity Recognition with Cross-lingual, Character-Level Neural Conditional Random Fields

Paradigm Completion for Derivational Morphology

The generation of complex derived word forms has been an overlooked problem in NLP; we fill this gap by applying neural …
Ryan Cotterell, Ekaterina Vylomova, Huda Khayrallah, Christo Kirov, David Yarowsky
PDF Anthology
Paradigm Completion for Derivational Morphology

Cross-lingual, Character-Level Neural Morphological Tagging

Even for common NLP tasks, sufficient supervision is not available in many languages–morphological tagging is no exception. In …
Ryan Cotterell, Georg Heigold
PDF Anthology arXiv
Cross-lingual, Character-Level Neural Morphological Tagging

Probabilistic Typology: Deep Generative Models of Vowel Inventories

Linguistic typology studies the range of structures present in human language. The main goal of the field is to discover which sets of …
Ryan Cotterell, Jason Eisner
PDF Anthology arXiv
Probabilistic Typology: Deep Generative Models of Vowel Inventories

One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a …
Katharina Kann, Ryan Cotterell, Hinrich Schütze
PDF Anthology arXiv
One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles

We study how different frame annotations complement one another when learning continuous lexical semantics. We learn the …
Francis Ferraro, Adam Poliak, Ryan Cotterell, Benjamin Van Durme
PDF Anthology
Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles

CoNLL--SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

The CoNLL-SIGMORPHON 2017 shared task on supervised morphological generation required systems to be trained and tested in each of 52 …
Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, Mans Hulden
PDF Anthology
CoNLL--SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

Neural Multi-Source Morphological Reinflection

We explore the task of multi-source morphological reinflection, which generalizes the standard, single-source version. The input …
Katharina Kann, Ryan Cotterell, Hinrich Schütze
PDF Anthology arXiv
Neural Multi-Source Morphological Reinflection

Neural Graphical Models over Strings for Principal Parts Morphological Paradigm Completion

Many of the world’s languages contain an abundance of inflected forms for each lexeme. A critical task in processing such …
Ryan Cotterell, John Sylak-Glassman, Christo Kirov
PDF Anthology
Neural Graphical Models over Strings for Principal Parts Morphological Paradigm Completion

Morphological Analysis of the Dravidian Language Family

The Dravidian languages are one of the most widely spoken language families in the world, yet there are very few annotated resources …
Arun Kumar, Ryan Cotterell, Lluís Padró, Antoni Oliver
PDF Anthology
Morphological Analysis of the Dravidian Language Family

Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis

The popular skip-gram model induces word embeddings by exploiting the signal from word-context coocurrence. We offer a new …
Ryan Cotterell, Adam Poliak, Ben Van Durme, Jason Eisner
PDF Anthology
Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis

Context-Aware Prediction of Derivational Word-forms

Derivational morphology is a fundamental and complex characteristic of language. In this paper we propose a new task of predicting the …
Ekaterina Vylomova, Ryan Cotterell, Timothy Baldwin, Trevor Cohn
PDF Anthology
Context-Aware Prediction of Derivational Word-forms

A Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax

A traditional claim in linguistics is that all human languages are equally expressive—able to convey the same wide range of …
Christo Kirov, John Sylak-Glassman, Rebecca Knowles, Ryan Cotterell, Matt Post
PDF Anthology
A Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax

Speed-Accuracy Tradeoffs in Tagging with Variable-Order CRFs and Structured Sparsity

We propose a method for learning the structure of variable-order CRFs, a more flexible variant of higher-order linear-chain CRFs. …
Tim Vieira$^*$, Ryan Cotterell$^*$, Jason Eisner
PDF Anthology
Speed-Accuracy Tradeoffs in Tagging with Variable-Order CRFs and Structured Sparsity

Neural Morphological Analysis: Encoding-Decoding Canonical Segments

Canonical morphological segmentation aims to divide words into a sequence of standardized segments. In this work, we propose a …
Katharina Kann, Ryan Cotterell, Hinrich Schütze
PDF Anthology
Neural Morphological Analysis: Encoding-Decoding Canonical Segments

Morphological Segmentation Inside-Out

Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output. In …
Ryan Cotterell, Arun Kumar, Hinrich Schütze
PDF Anthology arXiv
Morphological Segmentation Inside-Out

The SIGMORPHON 2016 Shared Task—Morphological Reinflection

The 2016 SIGMORPHON Shared Task was devoted to the problem of morphological reinflection. It introduced morphological datasets for 10 …
Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, Mans Hulden
PDF Anthology
The SIGMORPHON 2016 Shared Task—Morphological Reinflection

Morphological Smoothing and Extrapolation of Word Embeddings

Languages with rich inflectional morphology exhibit lexical data sparsity, since the word used to express a given concept will vary …
Ryan Cotterell, Hinrich Schütze, Jason Eisner
PDF Anthology
Morphological Smoothing and Extrapolation of Word Embeddings

Weighting Finite-State Transductions With Neural Context

How should one apply deep learning to tasks such as morphological reinflection, which stochastically edit one string to get another? A …
Pushpendre Rastogi, Ryan Cotterell, Jason Eisner
PDF Anthology
Weighting Finite-State Transductions With Neural Context

A Joint Model of Orthography and Morphological Segmentation

We present a model of morphological segmentation that jointly learns to segment and restore orthographic changes, e.g., funniest 7 → …
Ryan Cotterell, Tim Vieira, Hinrich Schütze
PDF Anthology
A Joint Model of Orthography and Morphological Segmentation

Contrastive Morphological Typology and Logical Hierarchies

John Sylak-Glassman, Ryan Cotterell
PDF

Analysis of Morphology in Topic Modeling

Chandler May, Ryan Cotterell, Benjamin Van Durme
PDF Anthology

Joint Lemmatization and Morphological Tagging with Lemming

We present Lemming, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary …
Thomas Müller, Ryan Cotterell, Alexander Fraser, Hinrich Schütze
PDF Anthology
Joint Lemmatization and Morphological Tagging with Lemming

Dual Decomposition Inference for Graphical Models over Strings

We investigate dual decomposition for joint MAP inference of many strings. Given an arbitrary graphical model, we decompose it into …
Nanyun Peng, Ryan Cotterell, Jason Eisner
PDF Anthology
Dual Decomposition Inference for Graphical Models over Strings

Penalized Expectation Propagation for Graphical Models over Strings

We present penalized expectation propagation (PEP), a novel algorithm for approximate inference in graphical models. Expectation …
Ryan Cotterell, Jason Eisner
PDF Anthology
Penalized Expectation Propagation for Graphical Models over Strings

Morphological Word Embeddings

Linguistic similarity is multi-faceted. For instance, two words may be similar with respect to semantics, syntax, or morphology inter …
Ryan Cotterell, Hinrich Schütze
PDF Anthology arXiv
Morphological Word Embeddings

Labeled Morphological Segmentation with Semi-Markov Models

We present labeled morphological segmentation—an alternative view of morphological processing that unifies several tasks. We introduce …
Ryan Cotterell, Thomas Müller, Alexander Fraser, Hinrich Schütze
PDF Anthology
Labeled Morphological Segmentation with Semi-Markov Models

Modeling Word Forms Using Latent Underlying Morphs and Phonology

The observed pronunciations or spellings of words are often explained as arising from the “underlying forms” of their …
Ryan Cotterell, Nanyun Peng, Jason Eisner
PDF Anthology
Modeling Word Forms Using Latent Underlying Morphs and Phonology

Stochastic Contextual Edit Distance and Probabilistic FSTs

String similarity is most often measured by weighted or unweighted edit distance d(x, y). Ristad and Yianilos (1998) defined stochastic …
Ryan Cotterell, Nanyun Peng, Jason Eisner
PDF Anthology
Stochastic Contextual Edit Distance and Probabilistic FSTs

Translation of the CALLHOME Egyptian Arabic Corpus For Conversational Speech Translation

Gaurav Kumar, Yuan Cao, Ryan Cotterell, Chris Callison-Burch, Daniel Povey, Sanjeev Khudanpur
PDF Anthology
Translation of the CALLHOME Egyptian Arabic Corpus For Conversational Speech Translation

An Algerian Arabic-French Code-Switched Corpus

Ryan Cotterell, Adithya Renduchintala, Naomi Saphra, Chris Callison-Burch
PDF Anthology
An Algerian Arabic-French Code-Switched Corpus

A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic

Ryan Cotterell, Chris Callison-Burch
PDF Anthology
A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic