# Publications

### Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

This work treats the paradigm discovery problem (PDP)—the task of learning an inflectional morphological system from unannotated …

### Predicting Declension Class from Form and Meaning

The noun lexica of many natural languages are divided into several declension classes with characteristic morphological properties. …

### It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information

The performance of neural machine translation systems is commonly evaluated in terms of BLEU. However, due to its reliance on target …

### Information-Theoretic Probing for Linguistic Structure

The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually know …

### Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing

Prior work has explored directly regularizing the output distributions of probabilistic models to alleviate peaky (i.e. over-confident) …

### A Tale of a Probe and a Parser

Measuring what linguistic information is encoded in continuous representations of language has become a popular area of research. To do …

### A Corpus for Large-Scale Phonetic Typology

A major hurdle in data-driven research on typology is having sufficient data in many languages to draw meaningful conclusions. We …

### Phonotactic Complexity and its Trade-offs

We present methods for calculating a measure of phonotactic complexity—bits per phoneme—that permits a straightforward cross-linguistic …

### On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs

We use large-scale corpora in six different gendered languages, along with tools from NLP and information theory, to test whether there …

### Towards Zero-Shot Language Modeling

Can we construct a neural language model which is inductively biased towards learning human language? Motivated by this question, we …

### Quantifying the Semantic Core of Gender Systems

Many of the world’s languages employ grammatical gender on the lexeme. For instance, in Spanish, house “casa” is …

### It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as …

### Examining Gender Bias in Languages with Grammatical Gender

Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora. However, most studies to date …

### Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Human translators routinely have to translate rare inflections of words–due to the Zipfian distribution of words in a language. …

### The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection

The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of …

### What Kind of Language Is Hard to Language-Model?

How language-agnostic are current state-of-the-art NLP tools? Are there some types of language that are easier to model with current …

### Unsupervised Discovery of Gendered Language through Latent-Variable Modeling

Studying the ways in which language is gendered has long been an area of interest in sociolinguistics. Studies have explored, for …

### Uncovering Typological Implications with Belief Nets

The study of linguistic typology is rooted in the implications we find between linguistic features, such as the fact that languages …

### On the distribution of deep clausal embeddings: A large cross-linguistic study

Embedding a clause inside another (the girl [who likes cars [that run fast]] has arrived″) is a fundamental resource that has been …

### Measuring Morphological Irregularity

We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity …

### Meaning to Form: Measuring Systematicity as Information

A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an …

### Exact Hard Monotonic Attention for Character-Level Transduction

Many common character-level, string-to-string transduction tasks, e.g., grapheme-to-phoneme conversion and morphological inflection, …

### Counterfactual Data Augmentation for Mitigating Gender Bias in Languages with Rich Morphology

Gender stereotypes are manifest in most of the world’s languages and are consequently propagated or amplified by NLP systems. …

### On the Idiosyncrasies of the Mandarin Chinese Classifier System

While idiosyncrasies of the Chinese classifier system have been a richly studied topic among linguists (Adams and Conklin, 1973; …

### Gender Bias in Contextualized Word Embeddings

In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo’s contextualized word vectors. First, we conduct …

### Contextualization of Morphological Inflection

Critical to natural language generation is the production of correctly inflected text. In this paper, we isolate the task of predicting …

### Combining Sentiment Lexica with a Multi-View Variational Autoencoder

When assigning quantitative labels to a dataset, different methodologies may rely on different scales. In particular, when assigning …

### A Simple Joint Model for Improved Contextual Neural Lemmatization

English verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP …

### A Probabilistic Generative Model of Linguistic Typology

In the principles-and-parameters framework, the structural features of languages depend on parameters that may be toggled on or off, …

### On the Complexity and Typology of Inflectional Morphological Systems

We quantify the linguistic complexity of different languages’ morphological systems. We verify that there is a statistically …

### The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

The CoNLL-SIGMORPHON 2018 shared task on supervised learning of morphological generation featured data sets from 103 typologically …

### Marrying Universal Dependencies and Universal Morphology

The Universal Dependencies (UD) and Universal Morphology (UniMorph) projects each present schemata for annotating the morphosyntactic …

### Hard Non-Monotonic Attention for Character-Level Transduction

Character-level string-to-string transduction is an important component of various NLP tasks. The goal is to map an input string to an …

### Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction

Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages. We …

### A Discriminative Latent-Variable Model for Bilingual Lexicon Induction

We introduce a novel discriminative latent-variable model for the task of bilingual lexicon induction. Our model combines the bipartite …

### A Structured Variational Autoencoder for Contextual Morphological Inflection

Statistical morphological inflectors are typically trained on fully supervised, type-level data. One remaining open research question …

### Unsupervised Disambiguation of Syncretism in Inflected Lexicons

Lexical ambiguity makes it difficult to compute useful statistics of a corpus. A given word form might represent any of several …

### Are All Languages Equally Hard to Language-Model?

For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on …

### A Deep Generative Model of Vowel Formant Typology

What makes some types of languages more probable than others? For instance, we know that almost all spoken languages contain the vowel …

### UniMorph 2.0: Universal Morphology

The Universal Morphology (UniMorph) project is a collaborative effort to improve how NLP handles complex morphology across the world’s …

### Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

Can advances in NLP help advance cognitive modeling? We examine the role of artificial neural networks, the current state of the art in …

### Joint Semantic Synthesis and Morphological Analysis of the Derived Word

Much like sentences are composed of words, words themselves are composed of smaller units. For example, the English word questionably …

### Low-Resource Named Entity Recognition with Cross-lingual, Character-Level Neural Conditional Random Fields

Low-resource named entity recognition is still an open problem in NLP. Most state-of-the-art systems require tens of thousands of …

### Paradigm Completion for Derivational Morphology

The generation of complex derived word forms has been an overlooked problem in NLP; we fill this gap by applying neural …

### Cross-lingual, Character-Level Neural Morphological Tagging

Even for common NLP tasks, sufficient supervision is not available in many languages–morphological tagging is no exception. In …

### Probabilistic Typology: Deep Generative Models of Vowel Inventories

Linguistic typology studies the range of structures present in human language. The main goal of the field is to discover which sets of …

### One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a …

### Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles

We study how different frame annotations complement one another when learning continuous lexical semantics. We learn the …

### CoNLL--SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

The CoNLL-SIGMORPHON 2017 shared task on supervised morphological generation required systems to be trained and tested in each of 52 …

### Neural Multi-Source Morphological Reinflection

We explore the task of multi-source morphological reinflection, which generalizes the standard, single-source version. The input …

### Neural Graphical Models over Strings for Principal Parts Morphological Paradigm Completion

Many of the world’s languages contain an abundance of inflected forms for each lexeme. A critical task in processing such …

### Morphological Analysis of the Dravidian Language Family

The Dravidian languages are one of the most widely spoken language families in the world, yet there are very few annotated resources …

### Explaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis

The popular skip-gram model induces word embeddings by exploiting the signal from word-context coocurrence. We offer a new …

### Context-Aware Prediction of Derivational Word-forms

Derivational morphology is a fundamental and complex characteristic of language. In this paper we propose a new task of predicting the …

### A Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax

A traditional claim in linguistics is that all human languages are equally expressive—able to convey the same wide range of …

### Speed-Accuracy Tradeoffs in Tagging with Variable-Order CRFs and Structured Sparsity

We propose a method for learning the structure of variable-order CRFs, a more flexible variant of higher-order linear-chain CRFs. …

### Neural Morphological Analysis: Encoding-Decoding Canonical Segments

Canonical morphological segmentation aims to divide words into a sequence of standardized segments. In this work, we propose a …

### Morphological Segmentation Inside-Out

Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output. In …

### The SIGMORPHON 2016 Shared Task—Morphological Reinflection

The 2016 SIGMORPHON Shared Task was devoted to the problem of morphological reinflection. It introduced morphological datasets for 10 …

### Morphological Smoothing and Extrapolation of Word Embeddings

Languages with rich inflectional morphology exhibit lexical data sparsity, since the word used to express a given concept will vary …

### Weighting Finite-State Transductions With Neural Context

How should one apply deep learning to tasks such as morphological reinflection, which stochastically edit one string to get another? A …

### A Joint Model of Orthography and Morphological Segmentation

We present a model of morphological segmentation that jointly learns to segment and restore orthographic changes, e.g., funniest 7 → …

### Joint Lemmatization and Morphological Tagging with Lemming

We present Lemming, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary …

### Dual Decomposition Inference for Graphical Models over Strings

We investigate dual decomposition for joint MAP inference of many strings. Given an arbitrary graphical model, we decompose it into …

### Penalized Expectation Propagation for Graphical Models over Strings

We present penalized expectation propagation (PEP), a novel algorithm for approximate inference in graphical models. Expectation …

### Labeled Morphological Segmentation with Semi-Markov Models

We present labeled morphological segmentation—an alternative view of morphological processing that unifies several tasks. We introduce …

### Modeling Word Forms Using Latent Underlying Morphs and Phonology

The observed pronunciations or spellings of words are often explained as arising from the “underlying forms” of their …

### Stochastic Contextual Edit Distance and Probabilistic FSTs

String similarity is most often measured by weighted or unweighted edit distance d(x, y). Ristad and Yianilos (1998) defined stochastic …