Research Interests
- Machine/deep learning, specialized in recurrent network for sequence learning
( application in computer vision, natural language processing) - Reinforcement learning
- Graph Neural Networks
- Convex Optimization
Research Projects
- Memory augmented RNNs for lifelong learning (supported by DARPA and ONR) The Lifelong Learning Machines (L2M) program from DARPA seeks to develop systems that can learn continuously during execution and become increasingly expert while performing tasks, are subject to safety limits, and apply previous skills and knowledge to new situations – without forgetting previous learning. We proposed a systematic approach to analyze and compare underlying memory structures for the popular recurrent neural networks, including vanilla RNN, LSTM, neural stack and neural Turing machine, etc. Accordingly, a taxonomy for these networks and their variants was constructed. Based on this analysis, we proposed an internal and external memory augmented RL system for efficient lifelong learning. This work also leads to a funding from ONR.
- Recognition of underwater image (supported by NOAA) To recognize biota and coral distributions at Pully Ridge HAPC, a mesophotic reef in the Gulf of Mexico, we developed convolutional neural network-based image segmentation and classification methods to recognize the amount and health states of corals with sparse labels. As an extension of this work, we applied our sequential neural network to learn plant-development syntactic patterns and automatically identify plant species.
- Graph neural networks for NLP (supported by Apple) We address the task of false trigger mitigation and intent classification based on analyzing automatic speech recognition lattices using graph neural networks (GNNs), i.e., graph convolutional neural networks and graph attention neural networks.
- Question Answering on Knowledge Graph with RL (supported by Google) An internal memory-augmented RL is proposed to navigate on a knowledge graph conditioned on the input question by defining a Markov decision process on the knowledge graph. An agent travels from a starting vertex to the ending vertex by choosing edges to find predictive paths and answers.