COS 597G (Fall 2022): Understanding Large Language Models

<table class="table table-bordered table-striped"><tbody><tr><td>Instructor</td><td><a href="https://www.cs.princeton.edu/~danqic/">Danqi Chen</a> (danqic AT cs.princeton.edu)</td></tr><tr><td>Teaching assistant</td><td><a href="https://www.cs.princeton.edu/~awettig/">Alexander Wettig</a> (awettig AT cs.princeton.edu)</td></tr><tr><td>Lectures</td><td>Monday/Wednesday 10:30-11:50am</td></tr><tr><td>Location</td><td><a href="https://classroominfo.princeton.edu/View.aspx?bl_id=0653&amp;fl_id=01&amp;rm_id=0653_01_J15&amp;bc=SHERH&amp;img=0653101.JPG&amp;rn=101">Sherrerd Hall 101</a></td></tr><tr><td>Pre-lecture feedback meetings</td><td>Monday 3:30-4pm for Wednesday lectures, Friday 4:45pm-5:15pm for Monday lectures, COS 412</td></tr><tr><td>Office hours</td><td>Danqi's office hour: Monday 2:30-3:30pm, COS 412 (<a href="https://calendly.com/danqic/cos597g">by appointment</a>)<br>Alex's office hour: Wednesday 3-4pm, Friend Center (student space lobby)</td></tr><tr><td>Feedback form</td><td><a href="https://forms.gle/rkJVxY8fvn7pchv89">https://forms.gle/rkJVxY8fvn7pchv89</a></td></tr><tr></tr></tbody></table>

**We will use a Slack team for most communiations this semester (no Ed!). We will let you get in the Slack team after the first lecture; If you join the class late, just email us and we will add you. As long as you are on Slack, we prefer Slack messages over emails for all logistical questions. We also encourage students to use Slack for discussion of lecture content and projects.**

Large language models (LLMs) have utterly transformed the field of natural language processing (NLP) in the last 3-4 years. They form the basis of state-of-art systems and become ubiquitous in solving a wide range of natural language understanding and generation tasks. With the unprecedented potential and capabilities, these models also give rise to new ethical and scalability challenges. This course aims to cover cutting-edge research topics centering around pre-trained language models. We will discuss their technical foundations (BERT, GPT, T5 models, mixture-of-expert models, retrieval-based models), emerging capabilities (knowledge, reasoning, few-shot learning, in-context learning), fine-tuning and adaptation, system design, as well as security and ethics. We will cover each topic and discuss important papers in depth. Students will be expected to routinely read and present research papers and complete a research project at the end.

This is an advanced graduate course and all the students are expected to have taken machine learning and NLP courses before and are familiar with deep learning models such as Transformers.

### **Learning goals:**

-   This course is intended to prepare you for performing cutting-edge research in natural language processing, especially topics related to pre-trained language models. We will discuss the state-of-the-art, their capabilities and limitations.
-   Practice your research skills, including reading research papers, conducting literature survey, oral presentations, as well as providing constructive feedback.
-   Gain hands-on experience through the final project, from brainstorming ideas to implementation and empirical evaluation and writing the final paper.

### **Course structure**

-   **Class participation (25%)**: In each class, we will cover 1-2 papers. You are required to read these papers in depth and answer around 3 pre-lecture questions (see "pre-lecture questions" in the schedule table) **before 11:59pm prior to the lecture day**. These questions are designed to test your undersatnding and stimulate your thinking on the topic and will count towards class participation (we will not grade the correctness; as long as you do your best to answer these questions, you will be good). In the last 20 minutes of the class, we will review and discuss these questions in small groups.
-   **Presentations (30%)**: For each lecture, we will ask two students to work together and deliver a 60-minute lecture. The goal is to educate the others in the class about the topic, so do think about how to best cover the material, do a good job with slides, and be prepared for lots of questions. The topics and scheduling will be decided at the beginning of the semester. All the students are expected to come to the class regularly and participate in discussion.
    -   1-2 papers have already been chosen for each topic. We also encourage you to include background, or useful materials from "recommended reading" when you see there is a fit.
    -   You are also required to meet with the instructor before the lecture (Monday 3:30-4pm for Wednesday lectures and Friday 4:45-5:15pm for Monday lectures). Please send your draft slides on Slack **before 11:59pm** the day prior to the meeting and we will go over your slides during the meeting.
    -   You are expected to present 1-2 times and you will receive feedback on your presentation from 3-4 classmates.
-   **Lecture feedback (5%)**: In addition to giving lectures, you are also required to provide written feedback to the presenter(s) on their lecture, 1+ pages in length, commenting on the content, delivery, clarity, completeness, etc. No need for complete sentences, bullet points are fine, but should be thorough and constructive. These notes should be sent to the instructor/TA on Slack within a day of the lecture (**a google doc link is preferred**). You are expected to do this 2-3 times throughout the semester.
-   **Final project (40%)**: At the end of the class, everyone is required to do a class project related to LLMs and submit a final paper. You should work as a team of 2 or 3. Two example types of project include:
    -   Train or fine-tune a medium-sized language model (e.g., BERT/RoBERTa, T5, GPT-2) yourself for the task of your interest. You will probably need to access pre-trained models on [HuggingFace's hub](https://huggingface.co/models). If you don't have decent compute resources, we will provide certain compute budget for you to execute your projects.
    -   Prompt and evaluate a very large language model (e.g., GPT-3, Codex) to understand their capabilities, limitations or risks. We will provide certain budget for you to [access](https://openai.com/api/) these large models if needed.
    Everyone is required to submit a proposal **by Oct 14th 11:59pm** and the final paper is due on **Dec 16th** (dean's date). We will schedule in-class project presentations at the end of semester on Dec 5th. More detailed guidelines can be found [here](https://docs.google.com/document/d/1nXcES8iHizKsShQL7iuhTya8jQlOsCMT_hFtDK1zt9g/edit#heading=h.nm87up44cczl).

### **Useful materials:**

### **Schedule**

| Date                                                         | Topic/papers                                                 | Recommended reading                                          | Pre-lecture questions                                   | Presenters | Feedback providers |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------- | ---------- | ------------------ |
| Sep 7 (Wed)                                                  | **Introduction**                                             |                                                              |                                                         |            |                    |
| 1\. [Human Language Understanding & Reasoning](https://www.amacad.org/sites/default/files/publication/downloads/Daedalus_Sp22_09_Manning.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) (Transformers) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Blog Post: The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) |                                                              |                                                              |                                                         |            |                    |
| 4\. [HuggingFace's course on Transformers](https://huggingface.co/course/chapter1/1) | \-                                                           | Danqi Chen                                                   |                                                         |            |                    |
| [\[slides\]](lectures/lec01.pdf)                             | \-                                                           |                                                              |                                                         |            |                    |
| **What are LLMs?**                                           |                                                              |                                                              |                                                         |            |                    |
| Sep 12 (Mon)                                                 | **BERT (encoder-only models)**                               |                                                              |                                                         |            |                    |
| 1\. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/pdf/1810.04805.pdf) |                                                              |                                                              |                                                         |            |                    |
| 1\. [Deep contextualized word representations](https://arxiv.org/pdf/1802.05365.pdf) (ELMo) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Improving Language Understanding by Generative Pre-Training](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf) (OpenAI GPT) |                                                              |                                                              |                                                         |            |                    |
| 3\. [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://arxiv.org/pdf/1907.11692.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://arxiv.org/pdf/2003.10555.pdf) |                                                              |                                                              |                                                         |            |                    |
| [lec2 questions](https://forms.gle/fmcLffVezSqKFXe3A)        | Danqi Chen                                                   |                                                              |                                                         |            |                    |
| [\[slides\]](lectures/lec02.pdf)                             |                                                              |                                                              |                                                         |            |                    |
| Sep 14 (Wed)                                                 | **T5 (encoder-decoder models)**                              |                                                              |                                                         |            |                    |
| 1\. [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) (T5) |                                                              |                                                              |                                                         |            |                    |
| 1\. [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/pdf/1910.13461.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/pdf/2010.11934.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model](https://arxiv.org/pdf/2208.01448.pdf) | [lec3 questions](https://forms.gle/7Kz2eVJn3u92kqV66)        | Abhishek Panigrahi, Victoria Graf                            |                                                         |            |                    |
| [\[slides\]](lectures/lec03.pdf)                             | Edward Tian, Zihan Ding, Jiatong Yu, Anirudh Ajith           |                                                              |                                                         |            |                    |
| Sep 19 (Mon)                                                 | **GPT-3 (decoder-only models)**                              |                                                              |                                                         |            |                    |
| 1\. [Language Models are Few-Shot Learners](https://arxiv.org/pdf/2005.14165.pdf) (GPT-3) |                                                              |                                                              |                                                         |            |                    |
| 1\. [Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) (GPT-2) |                                                              |                                                              |                                                         |            |                    |
| 2\. [PaLM: Scaling Language Modeling with Pathways](https://arxiv.org/pdf/2204.02311.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [OPT: Open Pre-trained Transformer Language Models](https://arxiv.org/pdf/2205.01068.pdf) |                                                              |                                                              |                                                         |            |                    |
| [lec 4 questions](https://forms.gle/DsoQDgbUbt7s8M3p6)       | Sabhya Chhabria, Michael Tang                                |                                                              |                                                         |            |                    |
| [\[slides\]](lectures/lec04.pdf)                             | Anika Maskara, Tianle Cai, Richard Zhu, Andrea Wynn          |                                                              |                                                         |            |                    |
| **How to Use and Adapt LLMs?**                               |                                                              |                                                              |                                                         |            |                    |
| Sep 21 (Wed)                                                 | **Prompting for few-shot learning**                          |                                                              |                                                         |            |                    |
| 1\. [Making Pre-trained Language Models Better Few-shot Learners](https://arxiv.org/pdf/2012.15723.pdf) ([blog post](https://gaotianyu.xyz/prompting/)) |                                                              |                                                              |                                                         |            |                    |
| 2\. [How Many Data Points is a Prompt Worth?](https://arxiv.org/pdf/2103.08493.pdf) | 1\. [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://arxiv.org/pdf/2001.07676.pdf) |                                                              |                                                         |            |                    |
| 2\. [True Few-Shot Learning with Language Models](https://arxiv.org/pdf/2105.11447.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models](https://arxiv.org/pdf/2106.13353.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/pdf/2107.13586.pdf) | [lec 5 questions](https://forms.gle/qLRpZkvT3SndjjG99)       | Kaixuan Huang, Edward Tian                                   |                                                         |            |                    |
| [\[slides\]](lectures/lec05.pdf)                             | Sam Liang, Mengzhou Xia, Victoria Graf, Tianle Cai           |                                                              |                                                         |            |                    |
| Sep 26 (Mon)                                                 | **Prompting as parameter-efficient fine-tuning**             |                                                              |                                                         |            |                    |
| 1\. [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://arxiv.org/pdf/2101.00190.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/pdf/2104.08691.pdf) | 1\. [Factual Probing Is \[MASK\]: Learning vs. Learning to Recall](https://arxiv.org/pdf/2104.05240.pdf) |                                                              |                                                         |            |                    |
| 2\. [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://arxiv.org/pdf/2110.07602.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/pdf/2106.09685.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [Towards a Unified View of Parameter-Efficient Transfer Learning](https://arxiv.org/pdf/2110.04366.pdf) | [lec 6 questions](https://forms.gle/21bexnWAivc5Udgu8)       | Chris Pan, Hongjie Wang                                      |                                                         |            |                    |
| [\[slides\]](lectures/lec06.pdf)                             | Sabhya Chhabria, Andrea Wynn, Sam Liang, Wenhan Xia          |                                                              |                                                         |            |                    |
| Sep 28 (Wed)                                                 | **In-context learning**                                      |                                                              |                                                         |            |                    |
| 1\. [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://arxiv.org/pdf/2202.12837.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [An Explanation of In-context Learning as Implicit Bayesian Inference](https://arxiv.org/pdf/2111.02080.pdf) (we don't expect you to read this paper in depth, you can check out this [blog post](http://ai.stanford.edu/blog/understanding-incontext/) instead) |                                                              |                                                              |                                                         |            |                    |
| 1\. [What Makes Good In-Context Examples for GPT-3?](https://arxiv.org/pdf/2101.06804.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://arxiv.org/pdf/2104.08786.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Data Distributional Properties Drive Emergent In-Context Learning in Transformers](https://arxiv.org/pdf/2205.05055.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://arxiv.org/pdf/2208.01066.pdf) | [lec 7 questions](https://forms.gle/9rm8cPbCfgaqHHry6)       | Sam Liang, Kexin Jin                                         |                                                         |            |                    |
| [\[slides\]](lectures/lec07.pdf)                             | Anika Maskara, Zixu Zhang, Tong Wu, Victoria Graf            |                                                              |                                                         |            |                    |
| Oct 3 (Mon)                                                  | **Calibration of prompting LLMs**                            |                                                              |                                                         |            |                    |
| 1\. [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://arxiv.org/pdf/2102.09690.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right](https://arxiv.org/pdf/2104.08315.pdf) | 1\. [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://arxiv.org/pdf/2108.04106.pdf) |                                                              |                                                         |            |                    |
| 2\. [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://arxiv.org/pdf/2012.00955.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Language Models (Mostly) Know What They Know](https://arxiv.org/pdf/2207.05221.pdf) | [lec 8 questions](https://forms.gle/oBVVD4Gnv98EX9wB8)       | Vishvak Murahari, Howard Yen                                 |                                                         |            |                    |
| [\[slides\]](lectures/lec08.pdf)                             | Jiatong Yu, Howard Chen, Chris Pan, Andre Niyongabo Rubungo, Devon Wood-Thomas |                                                              |                                                         |            |                    |
| Oct 5 (Wed)                                                  | **Reasoning**                                                |                                                              |                                                         |            |                    |
| 1\. [Chain of Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/pdf/2201.11903.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Large Language Models are Zero-Shot Reasoners](https://arxiv.org/pdf/2205.11916.pdf) | 1\. [Explaining Answers with Entailment Trees](https://arxiv.org/pdf/2104.08661.pdf) |                                                              |                                                         |            |                    |
| 2\. [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://arxiv.org/pdf/2203.11171.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Faithful Reasoning Using Large Language Models](https://arxiv.org/pdf/2208.14271.pdf) | [lec 9 questions](https://forms.gle/3smwAauRvG36PDjU8)       | Zihan Ding, Zixu Zhang                                       |                                                         |            |                    |
| [\[slides\]](lectures/lec09.pdf)                             | Vishvak Murahari, Beiqi Zou, Chris Pan, Xiangyu Qi           |                                                              |                                                         |            |                    |
| Oct 10 (Mon)                                                 | **Knowledge**                                                |                                                              |                                                         |            |                    |
| 1\. [Language Models as Knowledge Bases?](https://arxiv.org/pdf/1909.01066.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://arxiv.org/pdf/2002.08910.pdf) | 1\. [Knowledge Neurons in Pretrained Transformers](https://arxiv.org/pdf/2104.08696.pdf) |                                                              |                                                         |            |                    |
| 2\. [Fast Model Editing at Scale](https://arxiv.org/pdf/2110.11309.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://arxiv.org/pdf/2008.02637.pdf) | [lec 10 questions](https://forms.gle/ay2g8R1bEUS9EBHv6)      | Jane Pan, Mengzhou Xia                                       |                                                         |            |                    |
| [\[slides\]](lectures/lec10.pdf)                             | Andre Niyongabo Rubungo, Devon Wood-Thomas, Xiangyu Qi, Howard Chen |                                                              |                                                         |            |                    |
| **Dissecting LLMs: Data, Model Scaling and Risks**           |                                                              |                                                              |                                                         |            |                    |
| Oct 12 (Wed)                                                 | **Data**                                                     |                                                              |                                                         |            |                    |
| 1\. [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://arxiv.org/pdf/2104.08758.pdf) |                                                              |                                                              |                                                         |            |                    |
| 1\. [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://arxiv.org/pdf/2101.00027.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Deduplicating Training Data Makes Language Models Better](https://arxiv.org/pdf/2107.06499.pdf) | [lec 11 questions](https://forms.gle/wN72eva8WmyfHYhx8)      | Andre Niyongabo Rubungo, Tanushree Banerjee                  |                                                         |            |                    |
| [\[slides\]](lectures/lec11.pdf)                             | Arseniy Andreyev, Wenhan Xia, Xindi Wu, Richard Zhu          |                                                              |                                                         |            |                    |
| Oct 14 (Fri)                                                 | Final project proposal due at 11:59pm                        |                                                              |                                                         |            |                    |
| Submit [here](https://forms.gle/YUfE4m7upopdKxU37).          |                                                              |                                                              |                                                         |            |                    |
| Oct 17 (Mon)                                                 | Fall recess (no class)                                       |                                                              |                                                         |            |                    |
| Oct 19 (Wed)                                                 | Fall recess (no class)                                       |                                                              |                                                         |            |                    |
| Oct 24 (Mon)                                                 | **Scaling**                                                  |                                                              |                                                         |            |                    |
| 1\. [Training Compute-Optimal Large Language Models](https://arxiv.org/pdf/2203.15556.pdf) | 1\. [Scaling Laws for Neural Language Models](https://arxiv.org/pdf/2001.08361.pdf) |                                                              |                                                         |            |                    |
| 2\. [Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers](https://arxiv.org/pdf/2109.10686.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Scaling Laws for Autoregressive Generative Modeling](https://arxiv.org/pdf/2010.14701.pdf) | [lec 12 questions](https://forms.gle/hrzbyLjkb484ErYG9)      | Anika Maskara, Simon Park                                    |                                                         |            |                    |
| [\[slides\]](lectures/lec12.pdf)                             | Hongjie Wang, Sabhya Chhabria, Edward Tian, Kaixuan Huang    |                                                              |                                                         |            |                    |
| Oct 26 (Wed)                                                 | **Privacy**                                                  |                                                              |                                                         |            |                    |
| 1\. [Extracting Training Data from Large Language Models](https://arxiv.org/pdf/2012.07805.pdf) |                                                              |                                                              |                                                         |            |                    |
| 1\. [Quantifying Memorization Across Neural Language Models](https://arxiv.org/pdf/2202.07646.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://arxiv.org/pdf/2202.06539.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Large Language Models Can Be Strong Differentially Private Learners](https://arxiv.org/pdf/2110.05679.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [Recovering Private Text in Federated Learning of Language Models](https://arxiv.org/pdf/2205.08514.pdf) | [lec 13 questions](https://forms.gle/SA9TCsNZzrWJifDS6)      | Xiangyu Qi, Tong Wu                                          |                                                         |            |                    |
| [\[slides\]](lectures/lec13.pdf)                             | Anirudh Ajith, Austin Wang, Tanushree Banerjee, Arseniy Andreyev |                                                              |                                                         |            |                    |
| Oct 31 (Mon)                                                 | **Bias & Toxicity I: evaluation**                            |                                                              |                                                         |            |                    |
| 1\. [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://arxiv.org/pdf/2009.11462.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [OPT paper, Section 4](https://arxiv.org/pdf/2205.01068.pdf) |                                                              |                                                              |                                                         |            |                    |
| 1\. [On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Red Teaming Language Models with Language Models](https://storage.googleapis.com/deepmind-media/Red%20Teaming/Red%20Teaming.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection](https://arxiv.org/pdf/2201.10474.pdf) | [lec 14 questions](https://forms.gle/Jjy6MaTzHGYtavJ88)      | Maxine Perroni-Scharf, Richard Zhu                           |                                                         |            |                    |
| [\[slides\]](lectures/lec14.pdf)                             | Tong Wu, Hongjie Wang, Howard Yen, Mengzhou Xia              |                                                              |                                                         |            |                    |
| Nov 2 (Wed)                                                  | **Bias & Toxicity II: mitigation**                           |                                                              |                                                         |            |                    |
| 1\. [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://arxiv.org/pdf/2103.00453.pdf) |                                                              |                                                              |                                                         |            |                    |
| 1\. [Challenges in Detoxifying Language Models](https://arxiv.org/pdf/2109.07445.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Detoxifying Language Models Risks Marginalizing Minority Voices](https://arxiv.org/pdf/2104.06390.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://arxiv.org/pdf/1912.02164.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [GeDi: Generative discriminator guided sequence generation](https://arxiv.org/pdf/2009.06367.pdf) |                                                              |                                                              |                                                         |            |                    |
| [lec 15 questions](https://forms.gle/81irLDEYb4tQNxn5A)      | Anirudh Ajith, Arnab Bhattacharjee                           |                                                              |                                                         |            |                    |
| [\[slides\]](lectures/lec15.pdf)                             | Maxine Perroni-Scharf, Xindi Wu, Jane Pan, Howard Chen       |                                                              |                                                         |            |                    |
| **Beyond Current LLMs: Models and Applications**             |                                                              |                                                              |                                                         |            |                    |
| Nov 7 (Mon)                                                  | **Sparse models**                                            |                                                              |                                                         |            |                    |
| 1\. [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://jmlr.org/papers/volume23/21-0998/21-0998.pdf) | 1\. [Efficient Large Scale Language Modeling with Mixtures of Experts](https://arxiv.org/pdf/2112.10684.pdf) |                                                              |                                                         |            |                    |
| 2\. [Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models](https://arxiv.org/pdf/2208.03306.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [A Review of Sparse Expert Models in Deep Learning](https://arxiv.org/pdf/2209.01667.pdf) | [lec 16 questions](https://forms.gle/JyruLWfWezSJjeuN9)      | Zhou Lu, Wenhan Xia [\[slides\]](lectures/lec16.pdf)         | Michael Tang, Arnab Bhattacharjee, Kexin Jin, Beiqi Zou |            |                    |
| Nov 9 (Wed)                                                  | **Retrieval-based LMs**                                      |                                                              |                                                         |            |                    |
| 1\. [Improving language models by retrieving from trillions of tokens](https://arxiv.org/pdf/2112.04426.pdf) | 1\. [Generalization through Memorization: Nearest Neighbor Language Models](https://arxiv.org/pdf/1911.00172.pdf) |                                                              |                                                         |            |                    |
| 2\. [Training Language Models with Memory Augmentation](https://arxiv.org/pdf/2205.12674.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Few-shot Learning with Retrieval Augmented Language Models](https://arxiv.org/pdf/2208.03299.pdf) | [lec 17 questions](https://forms.gle/72F7eJJSSwRqeenx7)      | Tianle Cai, Beiqi Zou                                        |                                                         |            |                    |
| [\[slides\]](lectures/lec17.pdf)                             | Simon Park, Jane Pan, Maxine Perroni-Scharf, Abhishek Panigrahi |                                                              |                                                         |            |                    |
| Nov 14 (Mon)                                                 | **Training LMs with human feedback**                         |                                                              |                                                         |            |                    |
| 1\. [Training language models to follow instructions with human feedback](https://arxiv.org/pdf/2203.02155.pdf) | 1\. [Learning to summarize from human feedback](https://arxiv.org/pdf/2009.01325.pdf) |                                                              |                                                         |            |                    |
| 2\. [Fine-Tuning Language Models from Human Preferences](https://arxiv.org/pdf/1909.08593.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [MemPrompt: Memory-assisted Prompt Editing with User Feedback](https://arxiv.org/pdf/2201.06009.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [LaMDA: Language Models for Dialog Application](https://arxiv.org/pdf/2201.08239.pdf) |                                                              |                                                              |                                                         |            |                    |
| [lec 18 questions](https://forms.gle/JGDDc7i2HNeJ9QAF9)      | Howard Chen, Austin Wang                                     |                                                              |                                                         |            |                    |
| [\[slides\]](lectures/lec18.pdf)                             | Abhishek Panigrahi, Simon Park, Kaixuan Huang, Arseniy Andreyev |                                                              |                                                         |            |                    |
| Nov 16 (Wed)                                                 | **Code LMs**                                                 |                                                              |                                                         |            |                    |
| 1\. [Evaluating Large Language Models Trained on Code](https://arxiv.org/pdf/2107.03374.pdf) | 1\. [A Conversational Paradigm for Program Synthesis](https://arxiv.org/pdf/2203.13474.pdf) |                                                              |                                                         |            |                    |
| 2\. [InCoder: A Generative Model for Code Infilling and Synthesis](https://arxiv.org/pdf/2204.05999.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [A Systematic Evaluation of Large Language Models of Code](https://arxiv.org/pdf/2202.13169.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [Language Models of Code are Few-Shot Commonsense Learners](https://arxiv.org/pdf/2210.07128.pdf) |                                                              |                                                              |                                                         |            |                    |
| 5\. [Competition-Level Code Generation with AlphaCode](https://arxiv.org/pdf/2203.07814.pdf) | [lec 19 questions](https://forms.gle/jqrVo32SxdhzuSRG6)      | Arseniy Andreyev, Jiatong Yu                                 |                                                         |            |                    |
| [\[slides\]](lectures/lec19.pdf)                             | Howard Yen, Michael Tang, Tanushree Banerjee, Kexin Jin      |                                                              |                                                         |            |                    |
| Nov 21 (Mon)                                                 | **Multimodal LMs**                                           |                                                              |                                                         |            |                    |
| 1\. [Flamingo: a Visual Language Model for Few-Shot Learning](https://arxiv.org/pdf/2204.14198.pdf) | 1\. [Blog post: Generalized Visual Language Models](https://lilianweng.github.io/posts/2022-06-09-vlm/) |                                                              |                                                         |            |                    |
| 2\. [Learning Transferable Visual Models From Natural Language Supervision](https://arxiv.org/pdf/2103.00020.pdf) (CLIP) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Multimodal Few-Shot Learning with Frozen Language Models](https://arxiv.org/pdf/2106.13884.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [CM3: A Causal Masked Multimodal Model of the Internet](https://arxiv.org/pdf/2201.07520.pdf) | [lec 20 questions](https://forms.gle/CWKoE6RGPHWWq7HS9)      | Andrea Wynn, Xindi Wu                                        |                                                         |            |                    |
| [\[slides\]](lectures/lec20.pdf)                             | Arnab Bhattacharjee, Vishvak Murahari, Austin Wang, Zihan Ding |                                                              |                                                         |            |                    |
| Nov 23 (Wed)                                                 | Thanksgiving recess (no class)                               |                                                              |                                                         |            |                    |
| Nov 28 (Mon)                                                 | **Guest lecture: [Alexander Rush](https://rush-nlp.com/) (Cornell/Hugging Face) |                                                              |                                                         |            |                    |
| Multitask Prompted Training for Zero-Shot Models**           |                                                              |                                                              |                                                         |            |                    |
| 1\. [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://arxiv.org/pdf/2110.08207.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts](https://arxiv.org/pdf/2202.01279.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Scaling Instruction-Finetuned Language Models](https://arxiv.org/pdf/2210.11416.pdf) |                                                              |                                                              |                                                         |            |                    |
| 4\. [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://arxiv.org/pdf/2204.07705.pdf) |                                                              |                                                              |                                                         |            |                    |
| Nov 30 (Wed)                                                 | **AI Alignment + open discussion**                           |                                                              |                                                         |            |                    |
| 1\. [A General Language Assistant as a Laboratory for Alignment](https://arxiv.org/pdf/2112.00861.pdf) |                                                              |                                                              |                                                         |            |                    |
| 2\. [Alignment of Language Agents](https://arxiv.org/pdf/2103.14659.pdf) |                                                              |                                                              |                                                         |            |                    |
| 3\. [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://arxiv.org/pdf/2204.05862.pdf) |                                                              | Devon Wood-Thomas (half of the lecture)                      |                                                         |            |                    |
| [\[slides\]](lectures/lec22.pdf)                             | Richard Zhu, Sabhya Chhabria, Andrea Wynn, Anirudh Ajith     |                                                              |                                                         |            |                    |
| Dec 5 (Mon)                                                  | **in-class presentation (extended class)**                   |                                                              |                                                         |            |                    |
|                                                              |                                                              |                                                              |                                                         |            |                    |
| Dec 7 (Wed)                                                  | No class                                                     |                                                              |                                                         |            |                    |
| Dec 16 (Fri)                                                 | Final project due at 11:59pm (dean's date)                   | <table class="table table-bordered table-striped"><tbody><tr><td>Instructor</td><td><a href="https://www.cs.princeton.edu/~danqic/">Danqi Chen</a> (danqic AT cs.princeton.edu)</td></tr><tr><td>Teaching assistant</td><td><a href="https://www.cs.princeton.edu/~awettig/">Alexander Wettig</a> (awettig AT cs.princeton.edu)</td></tr><tr><td>Lectures</td><td>Monday/Wednesday 10:30-11:50am</td></tr><tr><td>Location</td><td><a href="https://classroominfo.princeton.edu/View.aspx?bl_id=0653&amp;fl_id=01&amp;rm_id=0653_01_J15&amp;bc=SHERH&amp;img=0653101.JPG&amp;rn=101">Sherrerd Hall 101</a></td></tr><tr><td>Pre-lecture feedback meetings</td><td>Monday 3:30-4pm for Wednesday lectures, Friday 4:45pm-5:15pm for Monday lectures, COS 412</td></tr><tr><td>Office hours</td><td>Danqi's office hour: Monday 2:30-3:30pm, COS 412 (<a href="https://calendly.com/danqic/cos597g">by appointment</a>)<br>Alex's office hour: Wednesday 3-4pm, Friend Center (student space lobby)</td></tr><tr><td>Feedback form</td><td><a href="https://forms.gle/rkJVxY8fvn7pchv89">https://forms.gle/rkJVxY8fvn7pchv89</a></td></tr><tr></tr></tbody></table> |                                                         |            |                    |

**We will use a Slack team for most communiations this semester (no Ed!). We will let you get in the Slack team after the first lecture; If you join the class late, just email us and we will add you. As long as you are on Slack, we prefer Slack messages over emails for all logistical questions. We also encourage students to use Slack for discussion of lecture content and projects.**

Large language models (LLMs) have utterly transformed the field of natural language processing (NLP) in the last 3-4 years. They form the basis of state-of-art systems and become ubiquitous in solving a wide range of natural language understanding and generation tasks. With the unprecedented potential and capabilities, these models also give rise to new ethical and scalability challenges. This course aims to cover cutting-edge research topics centering around pre-trained language models. We will discuss their technical foundations (BERT, GPT, T5 models, mixture-of-expert models, retrieval-based models), emerging capabilities (knowledge, reasoning, few-shot learning, in-context learning), fine-tuning and adaptation, system design, as well as security and ethics. We will cover each topic and discuss important papers in depth. Students will be expected to routinely read and present research papers and complete a research project at the end.

This is an advanced graduate course and all the students are expected to have taken machine learning and NLP courses before and are familiar with deep learning models such as Transformers.

### **Learning goals:**

-   This course is intended to prepare you for performing cutting-edge research in natural language processing, especially topics related to pre-trained language models. We will discuss the state-of-the-art, their capabilities and limitations.
-   Practice your research skills, including reading research papers, conducting literature survey, oral presentations, as well as providing constructive feedback.
-   Gain hands-on experience through the final project, from brainstorming ideas to implementation and empirical evaluation and writing the final paper.

### **Course structure**

-   **Class participation (25%)**: In each class, we will cover 1-2 papers. You are required to read these papers in depth and answer around 3 pre-lecture questions (see "pre-lecture questions" in the schedule table) **before 11:59pm prior to the lecture day**. These questions are designed to test your undersatnding and stimulate your thinking on the topic and will count towards class participation (we will not grade the correctness; as long as you do your best to answer these questions, you will be good). In the last 20 minutes of the class, we will review and discuss these questions in small groups.
-   **Presentations (30%)**: For each lecture, we will ask two students to work together and deliver a 60-minute lecture. The goal is to educate the others in the class about the topic, so do think about how to best cover the material, do a good job with slides, and be prepared for lots of questions. The topics and scheduling will be decided at the beginning of the semester. All the students are expected to come to the class regularly and participate in discussion.
    -   1-2 papers have already been chosen for each topic. We also encourage you to include background, or useful materials from "recommended reading" when you see there is a fit.
    -   You are also required to meet with the instructor before the lecture (Monday 3:30-4pm for Wednesday lectures and Friday 4:45-5:15pm for Monday lectures). Please send your draft slides on Slack **before 11:59pm** the day prior to the meeting and we will go over your slides during the meeting.
    -   You are expected to present 1-2 times and you will receive feedback on your presentation from 3-4 classmates.
-   **Lecture feedback (5%)**: In addition to giving lectures, you are also required to provide written feedback to the presenter(s) on their lecture, 1+ pages in length, commenting on the content, delivery, clarity, completeness, etc. No need for complete sentences, bullet points are fine, but should be thorough and constructive. These notes should be sent to the instructor/TA on Slack within a day of the lecture (**a google doc link is preferred**). You are expected to do this 2-3 times throughout the semester.
-   **Final project (40%)**: At the end of the class, everyone is required to do a class project related to LLMs and submit a final paper. You should work as a team of 2 or 3. Two example types of project include:
    -   Train or fine-tune a medium-sized language model (e.g., BERT/RoBERTa, T5, GPT-2) yourself for the task of your interest. You will probably need to access pre-trained models on [HuggingFace's hub](https://huggingface.co/models). If you don't have decent compute resources, we will provide certain compute budget for you to execute your projects.
    -   Prompt and evaluate a very large language model (e.g., GPT-3, Codex) to understand their capabilities, limitations or risks. We will provide certain budget for you to [access](https://openai.com/api/) these large models if needed.
    Everyone is required to submit a proposal **by Oct 14th 11:59pm** and the final paper is due on **Dec 16th** (dean's date). We will schedule in-class project presentations at the end of semester on Dec 5th. More detailed guidelines can be found [here](https://docs.google.com/document/d/1nXcES8iHizKsShQL7iuhTya8jQlOsCMT_hFtDK1zt9g/edit#heading=h.nm87up44cczl).

### **Useful materials:**

### **Schedule**

| Date                                                         | Topic/papers                                                 | Recommended reading                                  | Pre-lecture questions                                   | Presenters | Feedback providers |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ---------------------------------------------------- | ------------------------------------------------------- | ---------- | ------------------ |
| Sep 7 (Wed)                                                  | **Introduction**                                             |                                                      |                                                         |            |                    |
| 1\. [Human Language Understanding & Reasoning](https://www.amacad.org/sites/default/files/publication/downloads/Daedalus_Sp22_09_Manning.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) (Transformers) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Blog Post: The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) |                                                              |                                                      |                                                         |            |                    |
| 4\. [HuggingFace's course on Transformers](https://huggingface.co/course/chapter1/1) | \-                                                           | Danqi Chen                                           |                                                         |            |                    |
| [\[slides\]](lectures/lec01.pdf)                             | \-                                                           |                                                      |                                                         |            |                    |
| **What are LLMs?**                                           |                                                              |                                                      |                                                         |            |                    |
| Sep 12 (Mon)                                                 | **BERT (encoder-only models)**                               |                                                      |                                                         |            |                    |
| 1\. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/pdf/1810.04805.pdf) |                                                              |                                                      |                                                         |            |                    |
| 1\. [Deep contextualized word representations](https://arxiv.org/pdf/1802.05365.pdf) (ELMo) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Improving Language Understanding by Generative Pre-Training](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf) (OpenAI GPT) |                                                              |                                                      |                                                         |            |                    |
| 3\. [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://arxiv.org/pdf/1907.11692.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://arxiv.org/pdf/2003.10555.pdf) |                                                              |                                                      |                                                         |            |                    |
| [lec2 questions](https://forms.gle/fmcLffVezSqKFXe3A)        | Danqi Chen                                                   |                                                      |                                                         |            |                    |
| [\[slides\]](lectures/lec02.pdf)                             |                                                              |                                                      |                                                         |            |                    |
| Sep 14 (Wed)                                                 | **T5 (encoder-decoder models)**                              |                                                      |                                                         |            |                    |
| 1\. [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) (T5) |                                                              |                                                      |                                                         |            |                    |
| 1\. [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://arxiv.org/pdf/1910.13461.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/pdf/2010.11934.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model](https://arxiv.org/pdf/2208.01448.pdf) | [lec3 questions](https://forms.gle/7Kz2eVJn3u92kqV66)        | Abhishek Panigrahi, Victoria Graf                    |                                                         |            |                    |
| [\[slides\]](lectures/lec03.pdf)                             | Edward Tian, Zihan Ding, Jiatong Yu, Anirudh Ajith           |                                                      |                                                         |            |                    |
| Sep 19 (Mon)                                                 | **GPT-3 (decoder-only models)**                              |                                                      |                                                         |            |                    |
| 1\. [Language Models are Few-Shot Learners](https://arxiv.org/pdf/2005.14165.pdf) (GPT-3) |                                                              |                                                      |                                                         |            |                    |
| 1\. [Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) (GPT-2) |                                                              |                                                      |                                                         |            |                    |
| 2\. [PaLM: Scaling Language Modeling with Pathways](https://arxiv.org/pdf/2204.02311.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [OPT: Open Pre-trained Transformer Language Models](https://arxiv.org/pdf/2205.01068.pdf) |                                                              |                                                      |                                                         |            |                    |
| [lec 4 questions](https://forms.gle/DsoQDgbUbt7s8M3p6)       | Sabhya Chhabria, Michael Tang                                |                                                      |                                                         |            |                    |
| [\[slides\]](lectures/lec04.pdf)                             | Anika Maskara, Tianle Cai, Richard Zhu, Andrea Wynn          |                                                      |                                                         |            |                    |
| **How to Use and Adapt LLMs?**                               |                                                              |                                                      |                                                         |            |                    |
| Sep 21 (Wed)                                                 | **Prompting for few-shot learning**                          |                                                      |                                                         |            |                    |
| 1\. [Making Pre-trained Language Models Better Few-shot Learners](https://arxiv.org/pdf/2012.15723.pdf) ([blog post](https://gaotianyu.xyz/prompting/)) |                                                              |                                                      |                                                         |            |                    |
| 2\. [How Many Data Points is a Prompt Worth?](https://arxiv.org/pdf/2103.08493.pdf) | 1\. [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://arxiv.org/pdf/2001.07676.pdf) |                                                      |                                                         |            |                    |
| 2\. [True Few-Shot Learning with Language Models](https://arxiv.org/pdf/2105.11447.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models](https://arxiv.org/pdf/2106.13353.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/pdf/2107.13586.pdf) | [lec 5 questions](https://forms.gle/qLRpZkvT3SndjjG99)       | Kaixuan Huang, Edward Tian                           |                                                         |            |                    |
| [\[slides\]](lectures/lec05.pdf)                             | Sam Liang, Mengzhou Xia, Victoria Graf, Tianle Cai           |                                                      |                                                         |            |                    |
| Sep 26 (Mon)                                                 | **Prompting as parameter-efficient fine-tuning**             |                                                      |                                                         |            |                    |
| 1\. [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://arxiv.org/pdf/2101.00190.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/pdf/2104.08691.pdf) | 1\. [Factual Probing Is \[MASK\]: Learning vs. Learning to Recall](https://arxiv.org/pdf/2104.05240.pdf) |                                                      |                                                         |            |                    |
| 2\. [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://arxiv.org/pdf/2110.07602.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/pdf/2106.09685.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [Towards a Unified View of Parameter-Efficient Transfer Learning](https://arxiv.org/pdf/2110.04366.pdf) | [lec 6 questions](https://forms.gle/21bexnWAivc5Udgu8)       | Chris Pan, Hongjie Wang                              |                                                         |            |                    |
| [\[slides\]](lectures/lec06.pdf)                             | Sabhya Chhabria, Andrea Wynn, Sam Liang, Wenhan Xia          |                                                      |                                                         |            |                    |
| Sep 28 (Wed)                                                 | **In-context learning**                                      |                                                      |                                                         |            |                    |
| 1\. [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://arxiv.org/pdf/2202.12837.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [An Explanation of In-context Learning as Implicit Bayesian Inference](https://arxiv.org/pdf/2111.02080.pdf) (we don't expect you to read this paper in depth, you can check out this [blog post](http://ai.stanford.edu/blog/understanding-incontext/) instead) |                                                              |                                                      |                                                         |            |                    |
| 1\. [What Makes Good In-Context Examples for GPT-3?](https://arxiv.org/pdf/2101.06804.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://arxiv.org/pdf/2104.08786.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Data Distributional Properties Drive Emergent In-Context Learning in Transformers](https://arxiv.org/pdf/2205.05055.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://arxiv.org/pdf/2208.01066.pdf) | [lec 7 questions](https://forms.gle/9rm8cPbCfgaqHHry6)       | Sam Liang, Kexin Jin                                 |                                                         |            |                    |
| [\[slides\]](lectures/lec07.pdf)                             | Anika Maskara, Zixu Zhang, Tong Wu, Victoria Graf            |                                                      |                                                         |            |                    |
| Oct 3 (Mon)                                                  | **Calibration of prompting LLMs**                            |                                                      |                                                         |            |                    |
| 1\. [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://arxiv.org/pdf/2102.09690.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right](https://arxiv.org/pdf/2104.08315.pdf) | 1\. [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://arxiv.org/pdf/2108.04106.pdf) |                                                      |                                                         |            |                    |
| 2\. [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://arxiv.org/pdf/2012.00955.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Language Models (Mostly) Know What They Know](https://arxiv.org/pdf/2207.05221.pdf) | [lec 8 questions](https://forms.gle/oBVVD4Gnv98EX9wB8)       | Vishvak Murahari, Howard Yen                         |                                                         |            |                    |
| [\[slides\]](lectures/lec08.pdf)                             | Jiatong Yu, Howard Chen, Chris Pan, Andre Niyongabo Rubungo, Devon Wood-Thomas |                                                      |                                                         |            |                    |
| Oct 5 (Wed)                                                  | **Reasoning**                                                |                                                      |                                                         |            |                    |
| 1\. [Chain of Thought Prompting Elicits Reasoning in Large Language Models](https://arxiv.org/pdf/2201.11903.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Large Language Models are Zero-Shot Reasoners](https://arxiv.org/pdf/2205.11916.pdf) | 1\. [Explaining Answers with Entailment Trees](https://arxiv.org/pdf/2104.08661.pdf) |                                                      |                                                         |            |                    |
| 2\. [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://arxiv.org/pdf/2203.11171.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Faithful Reasoning Using Large Language Models](https://arxiv.org/pdf/2208.14271.pdf) | [lec 9 questions](https://forms.gle/3smwAauRvG36PDjU8)       | Zihan Ding, Zixu Zhang                               |                                                         |            |                    |
| [\[slides\]](lectures/lec09.pdf)                             | Vishvak Murahari, Beiqi Zou, Chris Pan, Xiangyu Qi           |                                                      |                                                         |            |                    |
| Oct 10 (Mon)                                                 | **Knowledge**                                                |                                                      |                                                         |            |                    |
| 1\. [Language Models as Knowledge Bases?](https://arxiv.org/pdf/1909.01066.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://arxiv.org/pdf/2002.08910.pdf) | 1\. [Knowledge Neurons in Pretrained Transformers](https://arxiv.org/pdf/2104.08696.pdf) |                                                      |                                                         |            |                    |
| 2\. [Fast Model Editing at Scale](https://arxiv.org/pdf/2110.11309.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://arxiv.org/pdf/2008.02637.pdf) | [lec 10 questions](https://forms.gle/ay2g8R1bEUS9EBHv6)      | Jane Pan, Mengzhou Xia                               |                                                         |            |                    |
| [\[slides\]](lectures/lec10.pdf)                             | Andre Niyongabo Rubungo, Devon Wood-Thomas, Xiangyu Qi, Howard Chen |                                                      |                                                         |            |                    |
| **Dissecting LLMs: Data, Model Scaling and Risks**           |                                                              |                                                      |                                                         |            |                    |
| Oct 12 (Wed)                                                 | **Data**                                                     |                                                      |                                                         |            |                    |
| 1\. [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://arxiv.org/pdf/2104.08758.pdf) |                                                              |                                                      |                                                         |            |                    |
| 1\. [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://arxiv.org/pdf/2101.00027.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Deduplicating Training Data Makes Language Models Better](https://arxiv.org/pdf/2107.06499.pdf) | [lec 11 questions](https://forms.gle/wN72eva8WmyfHYhx8)      | Andre Niyongabo Rubungo, Tanushree Banerjee          |                                                         |            |                    |
| [\[slides\]](lectures/lec11.pdf)                             | Arseniy Andreyev, Wenhan Xia, Xindi Wu, Richard Zhu          |                                                      |                                                         |            |                    |
| Oct 14 (Fri)                                                 | Final project proposal due at 11:59pm                        |                                                      |                                                         |            |                    |
| Submit [here](https://forms.gle/YUfE4m7upopdKxU37).          |                                                              |                                                      |                                                         |            |                    |
| Oct 17 (Mon)                                                 | Fall recess (no class)                                       |                                                      |                                                         |            |                    |
| Oct 19 (Wed)                                                 | Fall recess (no class)                                       |                                                      |                                                         |            |                    |
| Oct 24 (Mon)                                                 | **Scaling**                                                  |                                                      |                                                         |            |                    |
| 1\. [Training Compute-Optimal Large Language Models](https://arxiv.org/pdf/2203.15556.pdf) | 1\. [Scaling Laws for Neural Language Models](https://arxiv.org/pdf/2001.08361.pdf) |                                                      |                                                         |            |                    |
| 2\. [Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers](https://arxiv.org/pdf/2109.10686.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Scaling Laws for Autoregressive Generative Modeling](https://arxiv.org/pdf/2010.14701.pdf) | [lec 12 questions](https://forms.gle/hrzbyLjkb484ErYG9)      | Anika Maskara, Simon Park                            |                                                         |            |                    |
| [\[slides\]](lectures/lec12.pdf)                             | Hongjie Wang, Sabhya Chhabria, Edward Tian, Kaixuan Huang    |                                                      |                                                         |            |                    |
| Oct 26 (Wed)                                                 | **Privacy**                                                  |                                                      |                                                         |            |                    |
| 1\. [Extracting Training Data from Large Language Models](https://arxiv.org/pdf/2012.07805.pdf) |                                                              |                                                      |                                                         |            |                    |
| 1\. [Quantifying Memorization Across Neural Language Models](https://arxiv.org/pdf/2202.07646.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://arxiv.org/pdf/2202.06539.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Large Language Models Can Be Strong Differentially Private Learners](https://arxiv.org/pdf/2110.05679.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [Recovering Private Text in Federated Learning of Language Models](https://arxiv.org/pdf/2205.08514.pdf) | [lec 13 questions](https://forms.gle/SA9TCsNZzrWJifDS6)      | Xiangyu Qi, Tong Wu                                  |                                                         |            |                    |
| [\[slides\]](lectures/lec13.pdf)                             | Anirudh Ajith, Austin Wang, Tanushree Banerjee, Arseniy Andreyev |                                                      |                                                         |            |                    |
| Oct 31 (Mon)                                                 | **Bias & Toxicity I: evaluation**                            |                                                      |                                                         |            |                    |
| 1\. [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://arxiv.org/pdf/2009.11462.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [OPT paper, Section 4](https://arxiv.org/pdf/2205.01068.pdf) |                                                              |                                                      |                                                         |            |                    |
| 1\. [On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Red Teaming Language Models with Language Models](https://storage.googleapis.com/deepmind-media/Red%20Teaming/Red%20Teaming.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection](https://arxiv.org/pdf/2201.10474.pdf) | [lec 14 questions](https://forms.gle/Jjy6MaTzHGYtavJ88)      | Maxine Perroni-Scharf, Richard Zhu                   |                                                         |            |                    |
| [\[slides\]](lectures/lec14.pdf)                             | Tong Wu, Hongjie Wang, Howard Yen, Mengzhou Xia              |                                                      |                                                         |            |                    |
| Nov 2 (Wed)                                                  | **Bias & Toxicity II: mitigation**                           |                                                      |                                                         |            |                    |
| 1\. [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://arxiv.org/pdf/2103.00453.pdf) |                                                              |                                                      |                                                         |            |                    |
| 1\. [Challenges in Detoxifying Language Models](https://arxiv.org/pdf/2109.07445.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Detoxifying Language Models Risks Marginalizing Minority Voices](https://arxiv.org/pdf/2104.06390.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://arxiv.org/pdf/1912.02164.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [GeDi: Generative discriminator guided sequence generation](https://arxiv.org/pdf/2009.06367.pdf) |                                                              |                                                      |                                                         |            |                    |
| [lec 15 questions](https://forms.gle/81irLDEYb4tQNxn5A)      | Anirudh Ajith, Arnab Bhattacharjee                           |                                                      |                                                         |            |                    |
| [\[slides\]](lectures/lec15.pdf)                             | Maxine Perroni-Scharf, Xindi Wu, Jane Pan, Howard Chen       |                                                      |                                                         |            |                    |
| **Beyond Current LLMs: Models and Applications**             |                                                              |                                                      |                                                         |            |                    |
| Nov 7 (Mon)                                                  | **Sparse models**                                            |                                                      |                                                         |            |                    |
| 1\. [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://jmlr.org/papers/volume23/21-0998/21-0998.pdf) | 1\. [Efficient Large Scale Language Modeling with Mixtures of Experts](https://arxiv.org/pdf/2112.10684.pdf) |                                                      |                                                         |            |                    |
| 2\. [Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models](https://arxiv.org/pdf/2208.03306.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [A Review of Sparse Expert Models in Deep Learning](https://arxiv.org/pdf/2209.01667.pdf) | [lec 16 questions](https://forms.gle/JyruLWfWezSJjeuN9)      | Zhou Lu, Wenhan Xia [\[slides\]](lectures/lec16.pdf) | Michael Tang, Arnab Bhattacharjee, Kexin Jin, Beiqi Zou |            |                    |
| Nov 9 (Wed)                                                  | **Retrieval-based LMs**                                      |                                                      |                                                         |            |                    |
| 1\. [Improving language models by retrieving from trillions of tokens](https://arxiv.org/pdf/2112.04426.pdf) | 1\. [Generalization through Memorization: Nearest Neighbor Language Models](https://arxiv.org/pdf/1911.00172.pdf) |                                                      |                                                         |            |                    |
| 2\. [Training Language Models with Memory Augmentation](https://arxiv.org/pdf/2205.12674.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Few-shot Learning with Retrieval Augmented Language Models](https://arxiv.org/pdf/2208.03299.pdf) | [lec 17 questions](https://forms.gle/72F7eJJSSwRqeenx7)      | Tianle Cai, Beiqi Zou                                |                                                         |            |                    |
| [\[slides\]](lectures/lec17.pdf)                             | Simon Park, Jane Pan, Maxine Perroni-Scharf, Abhishek Panigrahi |                                                      |                                                         |            |                    |
| Nov 14 (Mon)                                                 | **Training LMs with human feedback**                         |                                                      |                                                         |            |                    |
| 1\. [Training language models to follow instructions with human feedback](https://arxiv.org/pdf/2203.02155.pdf) | 1\. [Learning to summarize from human feedback](https://arxiv.org/pdf/2009.01325.pdf) |                                                      |                                                         |            |                    |
| 2\. [Fine-Tuning Language Models from Human Preferences](https://arxiv.org/pdf/1909.08593.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [MemPrompt: Memory-assisted Prompt Editing with User Feedback](https://arxiv.org/pdf/2201.06009.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [LaMDA: Language Models for Dialog Application](https://arxiv.org/pdf/2201.08239.pdf) |                                                              |                                                      |                                                         |            |                    |
| [lec 18 questions](https://forms.gle/JGDDc7i2HNeJ9QAF9)      | Howard Chen, Austin Wang                                     |                                                      |                                                         |            |                    |
| [\[slides\]](lectures/lec18.pdf)                             | Abhishek Panigrahi, Simon Park, Kaixuan Huang, Arseniy Andreyev |                                                      |                                                         |            |                    |
| Nov 16 (Wed)                                                 | **Code LMs**                                                 |                                                      |                                                         |            |                    |
| 1\. [Evaluating Large Language Models Trained on Code](https://arxiv.org/pdf/2107.03374.pdf) | 1\. [A Conversational Paradigm for Program Synthesis](https://arxiv.org/pdf/2203.13474.pdf) |                                                      |                                                         |            |                    |
| 2\. [InCoder: A Generative Model for Code Infilling and Synthesis](https://arxiv.org/pdf/2204.05999.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [A Systematic Evaluation of Large Language Models of Code](https://arxiv.org/pdf/2202.13169.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [Language Models of Code are Few-Shot Commonsense Learners](https://arxiv.org/pdf/2210.07128.pdf) |                                                              |                                                      |                                                         |            |                    |
| 5\. [Competition-Level Code Generation with AlphaCode](https://arxiv.org/pdf/2203.07814.pdf) | [lec 19 questions](https://forms.gle/jqrVo32SxdhzuSRG6)      | Arseniy Andreyev, Jiatong Yu                         |                                                         |            |                    |
| [\[slides\]](lectures/lec19.pdf)                             | Howard Yen, Michael Tang, Tanushree Banerjee, Kexin Jin      |                                                      |                                                         |            |                    |
| Nov 21 (Mon)                                                 | **Multimodal LMs**                                           |                                                      |                                                         |            |                    |
| 1\. [Flamingo: a Visual Language Model for Few-Shot Learning](https://arxiv.org/pdf/2204.14198.pdf) | 1\. [Blog post: Generalized Visual Language Models](https://lilianweng.github.io/posts/2022-06-09-vlm/) |                                                      |                                                         |            |                    |
| 2\. [Learning Transferable Visual Models From Natural Language Supervision](https://arxiv.org/pdf/2103.00020.pdf) (CLIP) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Multimodal Few-Shot Learning with Frozen Language Models](https://arxiv.org/pdf/2106.13884.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [CM3: A Causal Masked Multimodal Model of the Internet](https://arxiv.org/pdf/2201.07520.pdf) | [lec 20 questions](https://forms.gle/CWKoE6RGPHWWq7HS9)      | Andrea Wynn, Xindi Wu                                |                                                         |            |                    |
| [\[slides\]](lectures/lec20.pdf)                             | Arnab Bhattacharjee, Vishvak Murahari, Austin Wang, Zihan Ding |                                                      |                                                         |            |                    |
| Nov 23 (Wed)                                                 | Thanksgiving recess (no class)                               |                                                      |                                                         |            |                    |
| Nov 28 (Mon)                                                 | **Guest lecture: [Alexander Rush](https://rush-nlp.com/) (Cornell/Hugging Face) |                                                      |                                                         |            |                    |
| Multitask Prompted Training for Zero-Shot Models**           |                                                              |                                                      |                                                         |            |                    |
| 1\. [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://arxiv.org/pdf/2110.08207.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts](https://arxiv.org/pdf/2202.01279.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Scaling Instruction-Finetuned Language Models](https://arxiv.org/pdf/2210.11416.pdf) |                                                              |                                                      |                                                         |            |                    |
| 4\. [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://arxiv.org/pdf/2204.07705.pdf) |                                                              |                                                      |                                                         |            |                    |
| Nov 30 (Wed)                                                 | **AI Alignment + open discussion**                           |                                                      |                                                         |            |                    |
| 1\. [A General Language Assistant as a Laboratory for Alignment](https://arxiv.org/pdf/2112.00861.pdf) |                                                              |                                                      |                                                         |            |                    |
| 2\. [Alignment of Language Agents](https://arxiv.org/pdf/2103.14659.pdf) |                                                              |                                                      |                                                         |            |                    |
| 3\. [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://arxiv.org/pdf/2204.05862.pdf) |                                                              | Devon Wood-Thomas (half of the lecture)              |                                                         |            |                    |
| [\[slides\]](lectures/lec22.pdf)                             | Richard Zhu, Sabhya Chhabria, Andrea Wynn, Anirudh Ajith     |                                                      |                                                         |            |                    |
| Dec 5 (Mon)                                                  | **in-class presentation (extended class)**                   |                                                      |                                                         |            |                    |
|                                                              |                                                              |                                                      |                                                         |            |                    |
| Dec 7 (Wed)                                                  | No class                                                     |                                                      |                                                         |            |                    |
| Dec 16 (Fri)                                                 | Final project due at 11:59pm (dean's date)                   |                                                      |                                                         |            |                    |