Yanda Chen

I am a Member of Technical Staff (Research Scientist) at the Alignment Science team at Anthropic. I work on natural language processing, AI safety, and machine learning.

Previously, I did my PhD in Computer Science at Columbia University, where I was very fortunate to be co-advised by Prof. Kathy McKeown, Prof. He He, and Prof. Zhou Yu. I received my bachelor's degree in Computer Science at Columbia University in April 2021.

Email  /  CV  /  Semantic Scholar  /  Twitter  /  Github

My current research interest lies in two directions: i) Explainability: building explainable deep learning systems and understanding how LLMs behave, and ii) Reliability: improving the calibration and reducing the sensitivity of LLMs. Below are my publications.

We find that ICL ability of language models emerges from parallel structures in the pre-training data—--pairs of phrases following similar templates in the same context window. Specifically, we show that removing parallel structures in the pre-training data reduces LMs' ICL accuracy by 51% (vs 2% from random ablation). This drop persists even when excluding common patterns such as n-gram repetitions and long-range dependency.

We propose explanation-consistency finetuning (EC-finetuning), which adapts LLMs to generate more consistent natural-language explanations on related examples by finetuning them on synthetic data that is carefully constructed to contain consistent explanations. EC-finetuning improves explanation consistency by 10.0% on four finetuning datasets, and by 4.5% on seven out-of-distribution datasets.

We propose to evaluate the counterfactual simulatability of natural language explanations: whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals of the explained input. We implemented two metrics—precision and generality, and found that i) LLM's explanations have low precision, and ii) precision does not correlate with plausibility.

We find that label bias obscures true ICL sensitivity and that ICL sensitivity is strongly and negatively correlated with accuracy. Motivated by our study, we propose SenSel, a few-shot selective prediction method based on ICL sensitivity.

We proposed in-context learning distillation, which transfers in-context learning (ICL) ability from large language models to small language models by augmenting in-context tuning with teacher-student distillation. Experiments on LAMA and CrossFit show that in-context learning distillation improves the ICL ability of small language models.

We propose a novel few-shot meta-learning method called in-context tuning, where training examples are used as prefix in-context demonstrations for task adaptation. We show that in-context tuning out-performs MAML in terms of accuracy and eliminates several well-known oversensitivity artifacts of few-shot language model prompting.

We propose a data augmentation strategy and a rationale training strategy for cross-lingual sentence selection in low-resource settings where no labeled relevance judgment is available for training. Our methods achieve state-of-the-art results on three language pairs.

We propose two novel synthetic training strategies: targeted synthetic pre-training (a method to select useful synthetic examples to target weakness of existing models) and synthetic knowledge distillation. The two techniques, when combined, yield QA models that are simultaneously smaller, faster, and more accurate.

We propose a framework to systematically detect and reduce language bias of deep learning models under the high-stakes context of gang intervention.

Website design from Jon Barron