Dr. Roche: CYBORG-NLP: Patching “Soft” deep learning methods with “Hard” symbolic processing in NLP

Overview

CYBORG-NLP: Patching “Soft” deep learning methods with “Hard” symbolic processing in NLP

Abstract

In this talk, we will explore the apparent opposition, in the field of Natural Language Processing (NLP), between deep learning and symbolic processing as it relates to human-robot communication. Human-robot communication, because it is constrained by the physical world, provides a unique insight into what the word “understanding” means in the context of NLP, a concept that has been elusive to NLP since its inception. We will also discuss how, contrary to the famous assertion “symbolic processing lost, and the good guys [Deep Learning] won” (G. Hinton 2019), both can and should be integrated together. We call this integration of the “Soft” deep learning approach and the “Hard” symbolic processing, a kind of “Cyborg-NLP”. From DL’s point of view, careful symbolic analysis is, we argue, necessary to solve such problems as the verifiability of some of the most general NLP problems, translation for instance. Symbolic analysis is also necessary to reduce the scale of training data; a scale currently incompatible with what human beings are exposed to. More concretely, we will discuss how this approach pushes us to break the syntax/semantic barrier thereby revisiting the century-old question of what kind of logic underpins natural language. We will give some insight into a formalism that makes this type of discussion possible as well as a family of algorithms that makes it implementable.

Brief Biography

Emmanuel Roche is the founder and CEO of Clover.AI, a company designing new natural language processing technologies designed to achieve natural language programming. He is also the founder of Gammakite, a company designing language learning solutions for "difficult" languages, such as Chinese for English speakers. He holds a BS from Ecole Polytechnique in France as well as a MS and Ph.D. from the University of Paris where he studied with Maurice Gross. Emmanuel Roche was also the co-founder of Teragram Corporation, the leading provider of natural language technology until 2008 when it was acquired by SAS Institute. His interest spans large arrays of natural language questions, whether they intersect with technology, cognition, or evolution. He is the author of numerous articles and the author or co-author of more than thirty patents.