Deng Cai (蔡登)

• compute/data-efficient pretraining
• generalist/specialist alignment
• fast and quality decoding
• multimodality/retrieval augmentation

I have a broad interest in natural language processing and machine learning. My work has spanned from fundamental language analysis (e.g., semantic parsing) to real-world NLP applications (e.g., chatbots & translation). From a systematical view, my research is driven by the ultimate goal of building more interpretable and extensible AI systems. To achieve that, my research has revolved around symbolic semantics and reasoning (ACL20, AAAI20, EMNLP21), and explicit and external memory (NAACL19, EMNLP20, ACL21, ICLR23, ICLR24)

• On the Transformations across Reward Model, Parameter Update, and In-Context Prompt [arxiv]
  Deng Cai, Huayang Li, Tingchen Fu, Siheng Li, Weiwen Xu, Shuaiyi Li, Bowen Cao, Zhisong Zhang, Xinting Huang, Leyang Cui, Yan Wang, Lemao Liu, Taro Watanabe, Shuming Shi
  arXiv, 2024.
• A Survey on the Honesty of Large Language Models [arxiv]
  Siheng Li, Cheng Yang, Taiqiang Wu, Chufan Shi, Yuji Zhang, Xinyu Zhu, Zesen Cheng, Deng Cai, Mo Yu, Lemao Liu, Jie Zhou, Yujiu Yang, Ngai Wong, Xixin Wu, Wai Lam
  arXiv, 2024.
• Inferflow: an Efficient and Highly Configurable Inference Engine for Large Language Models [arxiv] [code]
  Shuming Shi, Enbo Zhao, Deng Cai, Leyang Cui, Xinting Huang, Huayang Li
  arXiv, 2023.
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models [arxiv]
  Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, Shuming Shi
  arXiv, 2023.
  
• A Survey on Retrieval-Augmented Text Generation [arxiv]
  Huayang Li^*, Yixuan Su^*, Deng Cai^*, Yan Wang^*, Lemao Liu^*
  arXiv, 2022.
  
• Narrative Incoherence Detection [arxiv]
  Deng Cai, Yizhe Zhang, Yichen Huang, Wai Lam, Bill Dolan
  arXiv, 2021.
  
• Chinese Word Segmentation: Another Decade Review (2007-2017) [arxiv]
  Zhao Hai, Cai Deng, Huang Changning, Kit Chunyu
  ArXiv, 2017.

• On the Worst Prompt Performance of Large Language Models [arxiv]
  Bowen Cao, Deng Cai^☨, Zhisong Zhang, Yuexian Zou, Wai Lam
  NeurIPS 2024
• StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving [arxiv]
  Chang Gao, Haiyun Jiang, Deng Cai, Shuming Shi, Wai Lam
  NeurIPS 2024
• Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast [arxiv]
  Chufan Shi, Cheng Yang, Xinyu Zhu, Jiahao Wang, Taiqiang Wu, Siheng Li, Deng Cai, Yujiu Yang, Yu Meng
  NeurIPS 2024
• GLBench: A Comprehensive Benchmark for Graph with Large Language Models
  Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor Wai Kin Chan, Jia Li
  NeurIPS 2024 (Datasets and Benchmarks)
• A Thorough Examination of Decoding Methods in the Era of LLMs [arxiv]
  Chufan Shi^*, Haoran Yang^*, Deng Cai^☨, Zhisong Zhang, Yifan Wang, Yujiu Yang, Wai Lam
  EMNLP 2024
• Consecutive Batch Model Editing with HooK Layers
  Shuaiyi Li, Yang Deng, Deng Cai, Hongyuan Lu, Liang Chen, Wai Lam
  EMNLP 2024
• Cross-lingual Contextualized Phrase Retrieval
  Huayang Li, Deng Cai^☨, Zhi Qu, Qu Cui, Hidetaka Kamigaito, Lemao Liu, Taro Watanabe
  EMNLP 2024 (Findings)
• Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning
  Sen Yang, Leyang Cui, Deng Cai, Xinting Huang, Shuming Shi, Wai Lam
  EMNLP 2024 (Findings)
• With Greater Text Comes Greater Necessarily: Inference-Time Training Helps Long Text Generation
  Yan Wang^*, DM^*, Deng Cai
  COLM 2024
• GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation [arxiv]
  Zhanyu Wang, Longyue Wang, Zhen Zhao, Minghao Wu, Chenyang Lyu, Huayang Li, Deng Cai, Luping Zhou, Shuming Shi, Zhaopeng Tu
  ACM MM 2024
  Best Paper Nomination (26/4340)
• Reasons to Reject? Aligning Language Models with Judgments [arxiv]
  Weiwen Xu, Deng Cai^☨, Zhisong Zhang, Wai Lam, Shuming Shi
  ACL 2024 (Findings)
• TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild [blog] [demo] [arxiv] [code]
  Huayang Li^*, Siheng Li^*, Deng Cai^*,☨, Longyue Wang, Lemao Liu, Taro Watanabe, Yujiu Yang, Shuming Shi
  ACL 2024 (Findings)
• Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction [arxiv]
  Tingchen Fu, Deng Cai^☨, Lemao Liu, Shuming Shi, Rui Yan
  ACL 2024 (Findings)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [arxiv]
  Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, Kam-fai Wong
  ACL 2024
• A Frustratingly Simple Decoding Method for Neural Text Generation [arxiv]
  Haoran Yang, Deng Cai^☨, Huayang Li, Wei Bi, Wai Lam, Shuming Shi
  COLING 2024
  
• Retrieval is Accurate Generation [paper]
  Bowen Cao, Deng Cai^☨, Leyang Cui, Xuxin Cheng, Wei Bi, Yuexian Zou, Shuming Shi
  ICLR 2024
  
• The Reasonableness Behind Unreasonable Translation Capability of Large Language Model [paper]
  Tingchen Fu, Lemao Liu, Deng Cai, Guoping Huang, Shuming Shi, Rui Yan
  ICLR 2024
  
• Knowledge Fusion of Large Language Models [paper]
  Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, Shuming Shi
  ICLR 2024
  
• Specialist or Generalist? Instruction Tuning for Specific NLP Tasks [paper]
  Chufan Shi, Yixuan Su, Cheng Yang, Yujiu Yang, Deng Cai^☨
  EMNLP 2023
  
• Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters [arxiv] [paper]
  Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, Jia Li
  EMNLP 2023 (Findings)
  
• Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective [arxiv] [paper] [code]
  Huayang Li, Tian Lan, Zihao Fu, Deng Cai, Lemao Liu, Nigel Collier, Taro Watanabe, Yixuan Su
  NeurIPS 2023
  
• PandaGPT: One Model To Instruction-Follow Them All [blog] [demo] [arxiv] [code]
  Yixuan Su^*, Tian Lan^*, Huayang Li^*, Jialu Xu, Yan Wang, Deng Cai^*,☨
  TLLM Workshop 2023
• Effidit: An Assistant for Improving Writing Efficiency [paper] [demo]
  Tencent AI Lab
  ACL 2023 (Demo)
  
• Copy is All You Need [paper]
  Tian Lan^*, Deng Cai^*,☨, Yan Wang, Heyan Huang, Xian-Ling Mao
  ICLR 2023
  
• Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation [arxiv] [paper] [code]
  Deng Cai, Xin Li, Jackie Chun-Sing Ho, Lidong Bing, Wai Lam
  EMNLP 2022
  
• Linearizing Transformer with Key-Value Memory [arxiv] [paper] [code]
  Yizhe Zhang^*, Deng Cai^*
  EMNLP 2022
  
• N-gram Is Back: Residual Learning of Neural Text Generation with n-gram Language Model [arxiv] [paper]
  Huayang Li, Deng Cai, Jin Xu, Taro Watanabe
  EMNLP 2022 (Findings)
  
• Measuring and Reducing Model Update Regression in Structured Prediction for NLP [arxiv] [blog] [paper]
  Deng Cai, Elman Mansimov, Yi-An Lai, Yixuan Su, Lei Shu, Yi Zhang
  NeurIPS 2022
  
• Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation [arxiv]
  Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, Jian Li
  NeurIPS 2022
  
• Recent Advances in Retrieval-Augmented Text Generation
  Deng Cai, Yan Wang, Lemao Liu, Shuming Shi
  SIGIR 2022 (Tutorial)
  
• Recent Advances in Retrieval-Augmented Text Generation
  Deng Cai, Yan Wang, Lemao Liu, Shuming Shi
  IJCAI 2022 (Tutorial)
  
• Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System [arxiv] [paper] [code]
  Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, Yi Zhang
  ACL 2022
  
• Multilingual AMR Parsing with Noisy Knowledge Distillation [arxiv] [paper] [code]
  Deng Cai, Xin Li, Jackie Chun-Sing Ho, Lidong Bing, Wai Lam
  EMNLP 2021 (Findings)
  
• Exploiting Reasoning Chains for Multi-hop Science Question Answering [arxiv] [paper] [code]
  Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, Wai Lam
  EMNLP 2021 (Findings)
  
• Neural Machine Translation with Monolingual Translation Memory [arxiv] [paper] [code] [slides]
  Deng Cai, Yan Wang, Huayang Li, Wai Lam, Lemao Liu
  ACL 2021
  Outstanding Paper Award (6/3350)
• Dialogue Response Selection with Hierarchical Curriculum Learning [arxiv] [paper] [code]
  Yixuan Su^*, Deng Cai^*, Qingyu Zhou, Zibo Lin, Simon Baker, Yunbo Cao, Shuming Shi, Nigel Collier, Yan Wang
  ACL 2021
• Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Question Answering [arxiv] [paper] [code]
  Weiwen Xu, Huihui Zhang, Deng Cai, Wai Lam
  ACL 2021 (Findings)
• Assessing Dialogue Systems with Distribution Distances [arxiv] [paper] [code]
  Jiannan Xiang^*, Yahui Liu^*, Deng Cai, Huayang Li, Defu Lian, Lemao Liu
  ACL 2021 (Findings)
• Non-Autoregressive Text Generation with Pre-trained Language Models [arxiv] [paper]
  Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier
  EACL 2021
• The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection [arxiv] [paper]
  Zibo Lin^*, Deng Cai^*, Yan Wang, Xiaojiang Liu, Hai-Tao Zheng, Shuming Shi
  EMNLP 2020
• Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach [arxiv]
  Yahui Liu, Marco De Nadai, Deng Cai, Huayang Li, Xavier Alameda-Pineda, Nicu Sebe, Bruno Lepri
  ACM MM 2020
• AMR Parsing via Graph-Sequence Iterative Inference [arxiv] [paper] [code] [slides]
  Deng Cai, Wai Lam
  ACL 2020
• Graph Transformer for Graph-to-Sequence Learning [arxiv] [paper] [code] [slides]
  Deng Cai, Wai Lam
  AAAI 2020
• Core Semantic First: A Top-down Approach for AMR Parsing [arxiv] [paper] [code] [slides]
  Deng Cai, Wai Lam
  EMNLP 2019
• Retrieval-guided Dialogue Response Generation via a Matching-to-Generation Framework [paper] [code]
  Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Shuming Shi
  EMNLP 2019
• Charge-Based Prison Term Prediction with Deep Gating Network [arxiv]
  Huajie Chen^*, Deng Cai^*, Wei Dai, Zehui Dai, Yadong Ding
  EMNLP 2019
• Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory [arxiv] [paper] [code] [slides]
  Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, Shuming Shi
  NAACL 2019
• Unsupervised Learning helps Supervised Neural Word Segmentation [preprint]
  Xiaobin Wang, Deng Cai, Guangwei Xu, Hai Zhao, Linlin Li, Luo Si
  AAAI 2019
• Translating a Math Word Problem to a Expression Tree [paper]
  Lei Wang, Yan Wang, Deng Cai, Dongxiang Zhang, Xiaojiang Liu
  EMNLP 2018
• Fast and Accurate Neural Word Segmentation for Chinese [paper] [code]
  Deng Cai, Hai Zhao, Zhisong Zhang, Yuan Xin, Yongjian Wu, Feiyue Huang
  ACL 2017
• Neural Word Segmentation Learning for Chinese [paper] [code] [slides]
  Deng Cai, Hai Zhao
  ACL 2016
• Exploring Dense Retrieval for Dialogue Response Selection
  Tian Lan, Deng Cai^☨, Yan Wang, Yixuan Su, Heyan Huang, Xian-Ling Mao
  ACM Transactions on Information Systems, 2023.
• PROTOTYPE-TO-STYLE: Dialogue Generation With Style-Aware Editing on Retrieval Memory
  Yixuan Su, Yan Wang, Deng Cai, Simon Baker, Anna Korhonen, Nigel Collier
  IEEE Transactions on Audio, Speech and Language Processing, 2021.
• Neural Machine Translation with Noisy Lexical Constraints
  Huayang Li, Guoping Huang, Deng Cai, Lemao Liu
  IEEE Transactions on Audio, Speech and Language Processing, 2020.
• A Hybrid Model for Chinese Spelling Check [paper]
  Deng Cai^*, Hai Zhao^*, Yang Xin, Yuzhu Wang, Zhongye Jia
  ACM Transactions on Asian and Low-Resource Language Information Process, 2017.