Tom McCoy (Yale)- Bridging the divide between linguistics and NLP: From vectors to symbols and back again
Abstract: Capturing the structure of language is a central goal in both linguistics and natural language processing (NLP). Despite this overlap in goals, linguistics and NLP differ substantially in how they handle language. Linguistic theories typically assume a symbolic view, in which discrete units (e.g., words) are combined in structured ways (e.g., in syntax trees). In NLP, however, the most successful systems are neural networks, which encode information in continuous vectors that do not appear to have any of the rich structure posited in linguistics. In this talk, I will discuss two lines of work that show how vectors and symbols may not be as incompatible as they seem. First, I will present mechanistic interpretability analyses of how standard neural networks represent syntactic structure, with the conclusion that, in at least many cases, these models build implicit symbolic representations. Thus, despite appearances, they may not be incompatible with symbolic views of language. Second, I will turn from representations to learning by showing how neural networks can realize strong, structured learning biases of the sort traditionally seen only in symbolic models; the experiments in this part of the talk are made possible by recent advances in meta-learning. Taken together, these results provide some first steps toward bridging the divide between vectors and symbols.
Speakers
Tom McCoy
Tom McCoy is an Assistant Professor in the Department of Linguistics at Yale University, also affiliated with the Department of Computer Science and the Wu Tsai Institute. He studies computational linguistics using techniques from cognitive science and artificial intelligence. His research focuses on the computational principles that underlie human language. He is particularly interested in language learning and linguistic representations. On the side of learning, he investigates how people can acquire language from so little data and how we can replicate this ability in machines. On the side of representations, he investigates what types of machines can represent the structure of language and how they do it. Much of his research involves neural network language models, with an emphasis on connecting such systems to linguistics.
