Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Boosting LLM Reasoning: Push the Limits of Few-shot Learning with Reinforced In-Context Pruning (2023)
Large Language Models are Null-Shot Learners (2024)
Do LLMs Work on Charts? Designing Few-Shot Prompts for Chart Question Answering and Summarization (2023)
CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities (2024)
Supervised Knowledge Makes Large Language Models Better In-context Learners (2023)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2024-02-10T01:21:28.933Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7556670904159546},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[{"reaction":"👍","users":["ShuoChen99","BriHug"],"count":2}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2402.05403","authors":[{"_id":"65c5915184ed215272356bcd","user":{"_id":"6374cd6b6ea8da14f8fef8dc","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6374cd6b6ea8da14f8fef8dc/l13bg0tKDjCnUw3I895QZ.png","isPro":true,"fullname":"Tianjun Zhang","user":"tianjunz","type":"user"},"name":"Tianjun Zhang","status":"extracted_confirmed","statusLastChangedAt":"2024-08-01T05:49:17.162Z","hidden":false},{"_id":"65c5915184ed215272356bce","user":{"_id":"61bbc2b2d96f5d4111901dcb","avatarUrl":"/avatars/83c826d5cad7bab27d33585cb7e64575.svg","isPro":false,"fullname":"Aman Madaan","user":"amanmadaan","type":"user"},"name":"Aman Madaan","status":"admin_assigned","statusLastChangedAt":"2024-02-09T10:32:23.977Z","hidden":false},{"_id":"65c5915184ed215272356bcf","user":{"_id":"6009aa9126e23ab5edb47afd","avatarUrl":"/avatars/f8afa9cf648227ad5817323451ede378.svg","isPro":false,"fullname":"Luyu Gao","user":"Luyu","type":"user"},"name":"Luyu Gao","status":"admin_assigned","statusLastChangedAt":"2024-02-09T10:32:30.523Z","hidden":false},{"_id":"65c5915184ed215272356bd0","name":"Steven Zheng","hidden":false},{"_id":"65c5915184ed215272356bd1","user":{"_id":"61841feb7c380c66d40cad54","avatarUrl":"/avatars/7a79362a24b844860781fb0625527eae.svg","isPro":false,"fullname":"Swaroop Mishra","user":"swaroop7","type":"user"},"name":"Swaroop Mishra","status":"admin_assigned","statusLastChangedAt":"2024-02-09T10:32:56.676Z","hidden":false},{"_id":"65c5915184ed215272356bd2","name":"Yiming Yang","hidden":false},{"_id":"65c5915184ed215272356bd3","user":{"_id":"62e69a709bb6d949274ad32b","avatarUrl":"/avatars/b6a796d12a1c1c708f5b46c9082cdee6.svg","isPro":false,"fullname":"Niket Tandon","user":"nikett","type":"user"},"name":"Niket Tandon","status":"admin_assigned","statusLastChangedAt":"2024-02-09T10:33:20.842Z","hidden":false},{"_id":"65c5915184ed215272356bd4","user":{"_id":"62cd8153248f9e6bc20ab250","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1657635637504-62cd8153248f9e6bc20ab250.jpeg","isPro":false,"fullname":"Uri Alon","user":"urialon","type":"user"},"name":"Uri Alon","status":"admin_assigned","statusLastChangedAt":"2024-02-09T10:33:39.720Z","hidden":false}],"publishedAt":"2024-02-08T04:42:29.000Z","submittedOnDailyAt":"2024-02-09T00:13:30.340Z","title":"In-Context Principle Learning from Mistakes","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"In-context learning (ICL, also known as few-shot prompting) has been the\nstandard method of adapting LLMs to downstream tasks, by learning from a few\ninput-output examples. Nonetheless, all ICL-based approaches only learn from\ncorrect input-output pairs. In this paper, we revisit this paradigm, by\nlearning more from the few given input-output examples. We introduce Learning\nPrinciples (LEAP): First, we intentionally induce the model to make mistakes on\nthese few examples; then we reflect on these mistakes, and learn explicit\ntask-specific \"principles\" from them, which help solve similar problems and\navoid common mistakes; finally, we prompt the model to answer unseen test\nquestions using the original few-shot examples and these learned general\nprinciples. We evaluate LEAP on a wide range of benchmarks, including multi-hop\nquestion answering (Hotpot QA), textual QA (DROP), Big-Bench Hard reasoning,\nand math problems (GSM8K and MATH); in all these benchmarks, LEAP improves the\nstrongest available LLMs such as GPT-3.5-turbo, GPT-4, GPT-4 turbo and\nClaude-2.1. For example, LEAP improves over the standard few-shot prompting\nusing GPT-4 by 7.5% in DROP, and by 3.3% in HotpotQA. Importantly, LEAP does\nnot require any more input or examples than the standard few-shot prompting\nsettings.","upvotes":18,"discussionId":"65c5915284ed215272356bea","ai_summary":"LEAP, a novel few-shot learning approach, enhances LLM performance by inducing and reflecting on mistakes from initial examples to learn task-specific principles, applied across various benchmarks.","ai_keywords":["in-context learning","few-shot prompting","few-shot examples","Learning Principles","task-specific principles","multi-hop question answering","textual QA","Big-Bench Hard reasoning","math problems","GPT-3.5-turbo","GPT-4","GPT-4 turbo","Claude-2.1","DROP","HotpotQA"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6538119803519fddb4a17e10","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6538119803519fddb4a17e10/ffJMkdx-rM7VvLTCM6ri_.jpeg","isPro":false,"fullname":"samusenps","user":"samusenps","type":"user"},{"_id":"63486df1f8f01fcc4b23e97d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63486df1f8f01fcc4b23e97d/RDpX29ibKTJhgisgtvZ6M.png","isPro":false,"fullname":"Satyam","user":"satyamt","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"646719718334813a7ae1f31f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/646719718334813a7ae1f31f/hi88KiBlNtpPqB-JIlVrP.png","isPro":true,"fullname":"Lucas Rose-Winters","user":"paperboygold","type":"user"},{"_id":"652c30c3d78452c4742d73ba","avatarUrl":"/avatars/06f2286dedad41ba6c8d129a97de5178.svg","isPro":false,"fullname":"Ali Dadsetan","user":"dadsetan","type":"user"},{"_id":"64af24e91cefb66e87b6af61","avatarUrl":"/avatars/5476622a4961f4fe5d1c85476badad37.svg","isPro":false,"fullname":"Jian Liao","user":"imjliao","type":"user"},{"_id":"6332099cacb6472115b17421","avatarUrl":"/avatars/222ab1f6f7cf249b6c13b6f1a81ec319.svg","isPro":true,"fullname":"Full Name","user":"Gatozu35","type":"user"},{"_id":"6555125a4f361968f0e3aad7","avatarUrl":"/avatars/e7692d82804338f21ecdc6e731f5c5ea.svg","isPro":false,"fullname":"marinaretikof","user":"marinaretik","type":"user"},{"_id":"659180299e16fa7510840ac4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/659180299e16fa7510840ac4/I66LcNr3i35ehuzw1vR2Q.png","isPro":false,"fullname":"Ji-Ha","user":"Ji-Ha","type":"user"},{"_id":"6032802e1f993496bc14d9e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png","isPro":false,"fullname":"Omar Sanseviero","user":"osanseviero","type":"user"},{"_id":"657217faabb25ed8aedd5e48","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/657217faabb25ed8aedd5e48/UUHAXeGtOnQBXFD3nYtf2.jpeg","isPro":false,"fullname":"Vlad Bogolin","user":"vladbogo","type":"user"},{"_id":"65b9f159c29f995b6deeaaf9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65b9f159c29f995b6deeaaf9/8vI8bNXWI1aqTYlNcYWlC.jpeg","isPro":false,"fullname":"Anshuman Kumar","user":"anshumankmr","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2402.05403

In-Context Principle Learning from Mistakes

Published on Feb 8, 2024

· Submitted by

AK on Feb 9, 2024

Upvote

Authors:

Tianjun Zhang ,

Aman Madaan ,

Luyu Gao ,

Swaroop Mishra ,

Niket Tandon ,

Uri Alon

Abstract

LEAP, a novel few-shot learning approach, enhances LLM performance by inducing and reflecting on mistakes from initial examples to learn task-specific principles, applied across various benchmarks.

AI-generated summary

In-context learning (ICL, also known as few-shot prompting) has been the standard method of adapting LLMs to downstream tasks, by learning from a few input-output examples. Nonetheless, all ICL-based approaches only learn from correct input-output pairs. In this paper, we revisit this paradigm, by learning more from the few given input-output examples. We introduce Learning Principles (LEAP): First, we intentionally induce the model to make mistakes on these few examples; then we reflect on these mistakes, and learn explicit task-specific "principles" from them, which help solve similar problems and avoid common mistakes; finally, we prompt the model to answer unseen test questions using the original few-shot examples and these learned general principles. We evaluate LEAP on a wide range of benchmarks, including multi-hop question answering (Hotpot QA), textual QA (DROP), Big-Bench Hard reasoning, and math problems (GSM8K and MATH); in all these benchmarks, LEAP improves the strongest available LLMs such as GPT-3.5-turbo, GPT-4, GPT-4 turbo and Claude-2.1. For example, LEAP improves over the standard few-shot prompting using GPT-4 by 7.5% in DROP, and by 3.3% in HotpotQA. Importantly, LEAP does not require any more input or examples than the standard few-shot prompting settings.