Hao Zhu
PhD Student, Language Technology Institute, Carnegie Mellon University
About
I take an interactionist approach to study how machine learning models can cooperate with humans in the real world. My research is at the intersection of Natural Language Processing, Cognitive Science, and Robotics.
Education
Carnegie Mellon University
2019 - 2024 (expected)
PhD
Social and Embodied Interactions Produce Language Learners
Tsinghua University
2015 - 2019
Bachelor of Computer Science
Symbol Grounding and Referential Games with Knowledge Learning
Experience outside education
Meta Reality LabRedmond, WA
2023 - 2023
Research Scientist Intern
Research on generalist embodied agents.
Allen Institute for Artificial Intelligence (AI2)Seattle, WARemote
2022 - 2022
Research Scientist Intern
An environment for evaluating and training embodied agents. We published EXCALIBUR: Encouraging and Evaluating Embodied Exploration in CVPR 2023.
Microsoft Semantic MachinesRemote
2021 - 2021
Research Scientist Intern
Few-shot data-to-text generation.
CMU and JHUPittsburgh, PABalitmore, MD
2018 - 2018
Research Intern with Prof. Matt Gormley and Prof. Jason Eisner
Neural parameterization for finite state transducers. We published Neural Finite-State Transducers: Beyond Rational Relations in NAACL 2019.
Project Highlight
ICLR 2024
Spotlight
Social
Environment
Evaluation
LLM
ICLR 2024
Embodied
Navigation
Environment
Evaluation
LLM
EXCALIBUR
zhuhao.meexcalibur
Building on top ProcTHOR scenes, we collected a dataset of challenging questions and answers which improve and examine the ability of agents to explore and interact with objects in a 3D environment. We also built a VR interface to collect human trajectories to compare with machine learning models.
CVPR 2023
Embodied
Navigation
Manipulation
VR
Environment
Evaluation
QA
Theory-of-Mind Modeling
The remarkable ability of humans adapting to new interaction partners is due to our ability to reason about the mental states of others -- Theory-of-Mind (ToM). This project is a series of three papers: Learning ToM through interaction with a group of diverse interlocutors, language learning with feedback and communicative goals, and language aquisiton with ToM.
ICML 2021
CogSci 2022
ICLR 2023
ToM
Social
Embodied
Agent
Publications (*:equal contribution)
In submission
1
Michelle Zhao, Hao Zhu, Henny Admoni, Yonatan Bisk
2024
Conference
1
International Conference on Learning Representations (ICLR)
Xuhui Zhou*, Hao Zhu*, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap
Spotlight
2024
2
International Conference on Learning Representations (ICLR)
Shuyan Zhou*, Frank F Xu*, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig
2024
3
Computer Vision and Pattern Recognition (CVPR)
Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs
2023
4
The Annual Meeting of the Association for Computational Linguistics (ACL) Findings
Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D Hwang, Swabha Swayamdipta, Maarten Sap
2023
5
International Conference on Learning Representations (ICLR)
Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk, Graham Neubig
2023
6
Conference on Empirical Methods in Natural Language Processing (EMNLP)
So Yeon Min, Hao Zhu, Ruslan Salakhutdinov, Yonatan Bisk
2022
7
Annual Meeting of the Cognitive Science Society (CogSci)
Hao Zhu, Yonatan Bisk, Graham Neubig
Oral
2022
8
The SIGNLL Conference on Computational Natural Language Learning (CoNLL)
Ruisi Su, Shruti Rijhwani, Hao Zhu, Junxian He, Xinyu Wang, Yonatan Bisk, Graham Neubig
2021
9
The International Conference on Machine Learning (ICML)
Hao Zhu, Graham Neubig, Yonatan Bisk
2021
10
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)
Chu-Cheng Lin, Hao Zhu, Matthew R. Gormley, Jason Eisner
Oral
2019
11
The Conference on Empirical Methods in Natural Language Processing (EMNLP)
Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
2019
12
The Annual Meeting of the Association for Computational Linguistics (ACL)
Weize Chen, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun
Oral
2019
13
The Annual Meeting of the Association for Computational Linguistics (ACL)
Hao Zhu, Yankai Lin, Zhiyuan Liu, Jie Fu, Tat-Seng Chua, Maosong Sun
2019
14
The Annual Meeting of the Association for Computational Linguistics (ACL)
Huiming Jin*, Hao Zhu*, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin
2018
15
The Conference on Empirical Methods in Natural Language Processing (EMNLP)
Ji Xin, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun
2018
16
The Conference on Empirical Methods in Natural Language Processing (EMNLP)
Xu Han*, Hao Zhu*, Pengfei Yu*, Ziyun Wang*, Yuan Yao, Zhiyuan Liu, Maosong Sun
Oral, Citations>500
2018
17
The Conference on Empirical Methods in Natural Language Processing (EMNLP)
Yihong Gu*, Jun Yan*, Hao Zhu*, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, Leyu Lin
2018
18
The Conference on Empirical Methods in Natural Language Processing (EMNLP)
Fanchao Qi, Yankai Lin, Maosong Sun, Hao Zhu, Ruobing Xie, Zhiyuan Liu
2018
19
International Joint Conference on Artificial Intelligence (IJCAI)
Hao Zhu, Ruobing Xie, Zhiyuan Liu, Maosong Sun
Oral
2017
Journal
1
Transactions of the Association for Computational Linguistics (TACL)
Hao Zhu, Yonatan Bisk, Graham Neubig
2020
Workshop
1
Xiaopeng Lu*, Lynnette Ng*, Jared Fernandez*, Hao Zhu*
2021
Teaching
CMU 11-711 Advanced NLP
2021 Fall, 2022 Fall: TA, Homework design, Taught unsupervised parsing
Service
Reviewer
ACL, EMNLP, NAACL, CVPR, ICLR, NeurIPS, ICML, TMLR, AAAI
Organizer
ToM 2023, ToM 2024 (in prep)