Prompt Architecture Research
Enhancing LLM-based Test Generation for Hard-to-Cover Branches via Program Analysis
Last Updated: April 13, 2024Yang, C., Chen, J., Lin, B., Zhou, J. and Wang, Z. (2024). Enhancing LLM-based Test Generation for Hard-to-Cover Branches via Program Analysis. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2404.04966 General Annotation The research paper titled “Enhancing LLM-based Test Generation for Hard-to-Cover Branches via Program Analysis” by Chen Yang, Junjie Chen, Bin Lin, Jianyi Zhou, and Ziqi Wang presents a new…
SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Last Updated: April 13, 2024Zhou, P., Pujara, J., Ren, X., Chen, X., Cheng, H.-T., Le, Q.V., Chi, E.H., Zhou, D., Mishra, S. and Zheng, H.S. (2024). Self-Discover: Large Language Models Self-Compose Reasoning Structures. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2402.03620. General Annotation SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures” by Pei Zhou, Jay Pujara, Xiang Ren, Xinyun Chen, Heng-Tze Cheng, Quoc V. Le, Ed…
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
Last Updated: March 31, 2024Nori, H., Lee, Y.T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., Luo, R., McKinney, S.M., Ness, R.O., Poon, H., Qin, T., Usuyama, N., White, C. and Horvitz, E. (2023). Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine. arXiv:2311.16452 [cs]. [online] Available at: https://arxiv.org/abs/2311.16452…
Large Language Models Are Human-Level Prompt Engineers
Last Updated: March 31, 2024Zhou, Y., Muresanu, A.I., Han, Z., Paster, K., Pitis, S., Chan, H. and Ba, J. (2022). Large Language Models Are Human-Level Prompt Engineers. arXiv:2211.01910 [cs]. [online] Available at: https://arxiv.org/abs/2211.01910 General Annotation The paper titled “Large Language Models are Human-Level Prompt Engineers” by Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan,…
Thread of Thought Unraveling Chaotic Contexts
Last Updated: March 31, 2024Zhou, Y., Geng, X., Shen, T., Tao, C., Long, G., Lou, J.-G. and Shen, J. (2023). Thread of Thought Unraveling Chaotic Contexts. [online] arXiv.org. Available at: https://arxiv.org/abs/2311.08734 [Accessed 9 Dec. 2023]. General Annotation “Thread of Thought: Unraveling Chaotic Contexts” by Yucheng Zhou, Xiubo Geng, Tao Shen, Chongyang Tao, Guodong Long, Jian-Guang Lou, and Jianbing Shen…
Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
Last Updated: March 31, 2024Zheng, H.S., Mishra, S., Chen, X., Cheng, H.-T., Chi, E.H., Le, Q.V. and Zhou, D. (2023). Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models. [online] arXiv.org. Available at: https://arxiv.org/abs/2310.06117 General Annotation “Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models” by Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen,…
Automatic Chain of Thought Prompting in Large Language Models
Last Updated: March 31, 2024Zhang, Z., Zhang, A., Li, M. and Smola, A. (2022). Automatic Chain of Thought Prompting in Large Language Models. arXiv:2210.03493 [cs]. [online] Available at: https://arxiv.org/abs/2210.03493 General Annotation The paper titled “Automatic Chain of Thought Prompting in Large Language Models” by Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola introduces a novel methodology for enhancing…
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
Last Updated: March 31, 2024Yu, W., Zhang, H., Pan, X., Ma, K., Wang, H. and Yu, D. (2023). Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models. [online] arXiv.org. Available at: https://arxiv.org/abs/2311.09210 [Accessed 9 Dec. 2023]. General Annotation “Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models” by Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, and Dong Yu introduces Chain-of-Noting…
ReAct: Synergizing Reasoning and Acting in Language Models
Last Updated: March 31, 2024Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. and Cao, Y. (2023b). ReAct: Synergizing Reasoning and Acting in Language Models. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2210.03629 General Annotation “REACT: Synergizing Reasoning and Acting in Language Models” by Shunyu Yao et al. presents an innovative approach that unifies reasoning and action generation within Large Language…
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Last Updated: March 31, 2024Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T.L., Cao, Y. and Narasimhan, K. (2023a). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2305.10601 General Annotation The paper “Tree of Thoughts: Deliberate Problem Solving with Large Language Models” by Shunyu Yao et al. explores the implementation of a new problem-solving…
Dynamic Prompting: A Unified Framework for Prompt Tuning
Last Updated: March 31, 2024Yang, X., Cheng, W., Zhao, X., Yu, W., Petzold, L. and Chen, H. (2023b). Dynamic Prompting: A Unified Framework for Prompt Tuning. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2303.02909 The paper titled “Dynamic Prompting: A Unified Framework for Prompt Tuning” by Xianjun Yang et al. presents a novel methodology for enhancing the efficacy of prompt tuning in pretrained foundation…
Large Language Models as Optimizers
Last Updated: March 31, 2024Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q.V., Zhou, D. and Chen, X. (2023a). Large Language Models as Optimizers. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2309.03409 General Annotation The paper titled “Large Language Models as Optimizers” by Chengrun Yang et al. introduces Optimization by PROmpting (OPRO), an innovative approach that leverages Large Language Models (LLMs) as optimizers…
Procedures as a Representation for Data in a Computer Program for Understanding Natural Language
Last Updated: March 31, 2024Winograd, T. (1971). Procedures as a Representation for Data in a Computer Program for Understanding Natural Language. dspace.mit.edu. [online] Available at: https://dspace.mit.edu/handle/1721.1/7095 Methodology:Winograd proposes formal procedures consisting of prompt-response pairs as a representation for natural language understanding systems. The prompts match against input phrases to trigger reasoning responses. Key Contributions: Introduced idea of prompts triggering…
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
Last Updated: April 27, 2024White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J. and Schmidt, D.C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv:2302.11382 [cs]. [online] Available at: https://arxiv.org/abs/2302.11382 General Annotation Prompt engineering is an essential skill set for conversing effectively with large language models (LLMs) like ChatGPT….
System 2 Attention (is something you might need too)
Last Updated: March 31, 2024Weston, J. and Sukhbaatar, S. (2023). System 2 Attention (is something you might need too). [online] arXiv.org. Available at: https://arxiv.org/abs/2311.11829 [Accessed 9 Dec. 2023]. General Annotation System 2 Attention (S2A) addresses the challenge faced by Transformer-based Large Language Models (LLMs) where soft attention mechanisms may incorporate irrelevant information from the context into their latent representations,…
ELIZA—a computer program for the study of natural language communication between man and machine
Last Updated: March 31, 2024Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, [online] 9(1), pp.36–45. doi:https://doi.org/10.1145/365153.365168 General Annotation Joseph Weizenbaum’s paper introduces ELIZA, a pioneering program in artificial intelligence that simulates conversational interactions with humans through pattern matching and substitution methodologies. Developed in the 1960s at…
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Chain-of-Thought Prompting
Last Updated: March 31, 2024Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi Quoc, E., Le, V. and Zhou, D. (2023). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Chain-of-Thought Prompting. General Annotation This study introduces Chain-of-Thought prompting as a novel approach to improve LLMs’ performance on complex reasoning tasks. By prompting models to generate…
Metacognitive Prompting Improves Understanding in Large Language Models
Last Updated: March 31, 2024Wang, Y. and Zhao, Y. (2023). Metacognitive Prompting Improves Understanding in Large Language Models. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2308.05342 The study “Metacognitive Prompting Improves Understanding in Large Language Models” introduces a novel approach, Metacognitive Prompting (MP), aimed at enhancing the interpretative and reasoning capabilities of LLMs by mimicking human metacognitive processes. This technique represents a significant departure…
Deep Language Networks: Joint Prompt Training of Stacked LLMs using Variational Inference
Last Updated: March 31, 2024Sordoni, A., Yuan, X., Côté, M.-A., Pereira, M., Trischler, A., Xiao, Z., Hosseini, A., Niedtner, F. and Roux, N.L. (2023). Deep Language Networks: Joint Prompt Training of Stacked LLMs using Variational Inference. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2306.12509 General Annotation This paper presents a groundbreaking approach to optimizing the performance of LLMs by structuring them into Deep Language…
Prompting GPT-3 To Be Reliable
Last Updated: March 31, 2024Si, C., Gan, Z., Yang, Z., Wang, S., Wang, J., Boyd-Graber, J. and Wang, L. (2022). Prompting GPT-3 To Be Reliable. arXiv:2210.09150 [cs]. [online] Available at: https://arxiv.org/abs/2210.09150 The document titled “Prompting GPT-3 to Be Reliable” delves into improving the reliability of GPT-3 through specific prompting strategies, addressing challenges in generalizability, social bias and fairness, uncertainty…
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Last Updated: March 31, 2024Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E. and Singh, S. (2020). AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2010.15980 The document “AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts” presents a methodology and its implications for understanding and leveraging the knowledge embedded within pretrained language models…
Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning
Last Updated: March 31, 2024Ruan, C. and Wang, H. (2023). Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning. [online] Available at: https://arxiv.org/pdf/2309.06123.pdf [Accessed 13 Sep. 2023]. The document “Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning” introduces an innovative approach, DVPT, to enhance the adaptability of large pre-trained models to downstream visual tasks through the generation of…
Compositionality and Generalization In Emergent Languages
Last Updated: March 31, 2024Rahma Chaabouni, Kharitonov, E., Bouchacourt, D., Dupoux, E. and Baroni, M. (2020). Compositionality and Generalization In Emergent Languages. doi:https://doi.org/10.18653/v1/2020.acl-main.407 The paper “Compositionality and Generalization in Emergent Languages” by Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni explores the emergence of language in deep multi-agent simulations, focusing on whether these languages can reference…
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Last Updated: March 31, 2024Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P.J. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, [online] 21(140), pp.1–67. Available at: https://jmlr.org/papers/v21/20-074.html General Annotation “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”…
Improving Language Understanding by Generative Pre-Training
Last Updated: March 31, 2024Openai, A., Openai, K., Openai, T. and Openai, I. (2018). Improving Language Understanding by Generative Pre-Training. [online] Available at: https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf The paper “Improving Language Understanding by Generative Pre-Training” by Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever from OpenAI presents a novel approach to enhancing natural language understanding (NLU) tasks through a two-stage process…
Testing Causal Models of Word Meaning in GPT-3 and -4
Last Updated: March 31, 2024Musker, S. and Pavlick, E. (2023). Testing Causal Models of Word Meaning in GPT-3 and -4. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2305.14630 The paper “Testing Causal Models of Word Meaning in GPT-3 and -4” by Sam Musker and Ellie Pavlick explores how large language models (LLMs) like GPT-3 and GPT-4 understand lexical concepts, specifically through the lens of…
GPT Understands, Too
Last Updated: March 31, 2024Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z. and Tang, J. (2021). GPT Understands, Too. arXiv:2103.10385 [cs]. [online] Available at: https://arxiv.org/abs/2103.10385 The paper “GPT Understands, Too” by Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang introduces a novel methodology called P-Tuning for enhancing the…
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
Last Updated: March 31, 2024Lin, B.Y., Zhou, W., Shen, M., Zhou, P., Bhagavatula, C., Choi, Y. and Ren, X. (2020). CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning. arXiv:1911.03705 [cs]. [online] Available at: https://arxiv.org/abs/1911.03705 The paper “CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning” by Bill Yuchen Lin and colleagues introduces CommonGen, a task focused…
User Interface Goals, AI Opportunities
Last Updated: March 31, 2024Lieberman, H. (2009). User Interface Goals, AI Opportunities. AI Magazine, 30(4), p.16. doi:https://doi.org/10.1609/aimag.v30i4.2266 Henry Lieberman’s 2009 article in AI Magazine, “User Interface Goals, AI Opportunities,” delves into the intersection and potential synergy between Human-Computer Interaction (HCI) and Artificial Intelligence (AI). It presents a critical examination of the historical divergence in goals and methodologies between these…
PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations
Last Updated: March 31, 2024Li, R., Patel, T. and Du, X. (2023). PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations. [online] Available at: https://arxiv.org/pdf/2307.02762.pdf [Accessed 7 Jul. 2023]. The paper “PRD: Peer Rank and Discussion to Improve Large Language Model based Evaluations” by Ruosen Li, Teerth Patel, and Xinya Du proposes innovative methods to refine the…
Large Language Models Understand and Can be Enhanced by Emotional Stimuli
Last Updated: March 31, 2024Li, C., Wang, J., Zhang, Y., Zhu, K., Hou, W., Lian, J., Luo, F., Yang, Q. and Xie, X. (2023). Large Language Models Understand and Can be Enhanced by Emotional Stimuli. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2307.11760 The paper “Large Language Models Understand and Can Be Enhanced by Emotional Stimuli” by Cheng Li et al. investigates the impact…
The Power of Scale for Parameter-Efficient Prompt Tuning
Last Updated: March 31, 2024Lester, B., Al-Rfou, R. and Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. arXiv:2104.08691 [cs]. [online] Available at: https://arxiv.org/abs/2104.08691 The paper “The Power of Scale for Parameter-Efficient Prompt Tuning” by Lester, Al-Rfou, and Constant introduces the concept of “prompt tuning,” a technique designed to optimize the conditioning of large, pre-trained language models…
Large Language Models are Zero-Shot Reasoners
Last Updated: March 31, 2024Kojima, T., Gu, S.S., Reid, M., Matsuo, Y. and Iwasawa, Y. (2022). Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916 [cs]. [online] Available at: https://arxiv.org/abs/2205.11916 General Annotation The paper “Large Language Models are Zero-Shot Reasoners” by Takeshi Kojima et al. introduces an innovative approach, Zero-shot Chain of Thought (Zero-shot-CoT), which significantly enhances the reasoning capabilities of…
Decomposed Prompting: A Modular Approach for Solving Complex Tasks
Last Updated: March 31, 2024Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P. and Sabharwal, A. (2023). Decomposed Prompting: A Modular Approach for Solving Complex Tasks. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2210.02406 METHODOLOGY: General Annotation The research proposes a method to break down complex tasks into simpler sub-tasks through a process called Decomposed Prompting (DECOMP). This approach leverages a…
Smart Reply: Automated Response Suggestion for Email
Last Updated: March 31, 2024Kannan, A., Kurach, K., Ravi, S., Kaufmann, T., Tomkins, A., Miklos, B., Corrado, G., Lukacs, L., Ganea, M., Young, P. and Ramavajjala, V. (2016). Smart Reply: Automated Response Suggestion for Email. arXiv:1606.04870 [cs]. [online] Available at: https://arxiv.org/abs/1606.04870 General Annotation Deng, Y., Zhang, W., Chen, Z., and Gu, Q. (2023) introduce an innovative methodology titled “Rephrase…
Logic and conversation
Last Updated: March 31, 2024Grice, H.P. (1975). Logic and conversation. New York: Academic Press. General Annotation H.P. Grice’s research addresses the nuanced ways in which meaning is constructed in conversation beyond the literal interpretation of words. He contrasts formal logic with the subtleties of natural language, arguing that understanding conversational implicatures—what is implied but not explicitly stated—is crucial for…
Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves
Last Updated: March 31, 2024Deng, Y., Zhang, W., Chen, Z. and Gu, Q. (2023). Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves. [online] arXiv.org. Available at: https://arxiv.org/abs/2311.04205 [Accessed 9 Dec. 2023]. The paper “Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves” by Deng et al. (2023) introduces a novel technique aimed…
Train and You’ll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings
Last Updated: March 31, 2024Chen, M.F., Fu, D.Y., Sala, F., Wu, S., Mullapudi, R.T., Poms, F., Fatahalian, K. and Ré, C. (2020). Train and You’ll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings. [online] arXiv.org. doi:https://doi.org/10.48550/arXiv.2006.15168 Methodologies Used:The authors present an approach that leverages weak supervision and pre-trained embeddings for interactive model iteration. This method aims…
Contrastive Chain-of-Thought Prompting
Last Updated: March 31, 2024Chia, Y.K., Chen, G., Tuan, L.A., Poria, S. and Bing, L. (2023). Contrastive Chain-of-Thought Prompting. [online] arXiv.org. Available at: https://arxiv.org/abs/2311.09277 [Accessed 9 Dec. 2023]. The research paper titled “Contrastive Chain-of-Thought Prompting” by Yew Ken Chia, Guizhen Chen, Luu Anh Tuan, Soujanya Poria, and Lidong Bing presents a novel approach to enhancing the reasoning capabilities of…
From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting
Last Updated: March 31, 2024Adams, G., Fabbri, A., Ladhak, F., Lehman, E. and Elhadad, N. (2023). From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting. [online] arXiv.org. Available at: https://arxiv.org/abs/2309.04269 [Accessed 9 Dec. 2023]. General Annotation The research paper introduces a novel prompt-based iterative method called the Chain of Density (CoD) for generating increasingly entity-dense summaries with…
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
Last Updated: March 31, 2024Bender, E., Mcmillan-Major, A., Shmitchell, S., Gebru, T. and Shmitchell, S.-G. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Timnit Gebru * timnit@blackinai.org Black in AI Palo Alto, CA, USA CCS CONCEPTS • Computing Methodologies → Natural Language processing. ACM Reference Format. [online] doi:https://doi.org/10.1145/3442188.3445922 The paper “On the Dangers of…