https://github.com/benchflow-ai/skillsbench
Website: https://skillsbench.ai/\n","updatedAt":"2026-02-18T00:33:00.620Z","author":{"_id":"663fe2d26304d377fc253322","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg","fullname":"Xiangyi Li","name":"xdotli","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.515940248966217},"editors":["xdotli"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg"],"reactions":[{"reaction":"🔥","users":["taesiri","Hudx111"],"count":2},{"reaction":"🤗","users":["taesiri","Hudx111"],"count":2}],"isReport":false}},{"id":"6995bfeff8241c4259341794","author":{"_id":"663fe2d26304d377fc253322","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg","fullname":"Xiangyi Li","name":"xdotli","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-02-18T13:34:39.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"We made the first benchmark that measures the efficacy of agent skills. We collected 86 tasks from 105 domain experts across 11 domains, every task is verifiable, human created and has verified Skills. SOTA model without skills score ~30% without skills. \n\nWe found a few interesting things: \n1. Skills substitute for model scale — Haiku 4.5 with Skills (27.7%) beats Opus 4.5 without (22.0%). The right procedural knowledge can be worth more than a bigger model. \n2. Skills' improvement has nothing to do with LLMs' internal knowledge. We have an ablation where no Skills provided for the agent, but the agent is prompted to generate relevant procedural knowledge before solving the task. This isolates the impact of LLMs' latent domain knowledge. \n\nThe result is: \n* Curated Skills: +16.2pp average improvement across all 7 agent configs \n* Self-generated Skills: -1.3pp: models can't write their own procedural knowledge pre-trajectory feedbacks. This is used to isolate the impact of LLMs' latent domain knowledge.","html":"
Website: https://skillsbench.ai/\n","updatedAt":"2026-02-18T00:33:00.620Z","author":{"_id":"663fe2d26304d377fc253322","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg","fullname":"Xiangyi Li","name":"xdotli","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.515940248966217},"editors":["xdotli"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg"],"reactions":[{"reaction":"🔥","users":["taesiri","Hudx111"],"count":2},{"reaction":"🤗","users":["taesiri","Hudx111"],"count":2}],"isReport":false}},{"id":"6995bfeff8241c4259341794","author":{"_id":"663fe2d26304d377fc253322","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/wuey_nNXSW4GthPYLfFS4.jpeg","fullname":"Xiangyi Li","name":"xdotli","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-02-18T13:34:39.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"We made the first benchmark that measures the efficacy of agent skills. We collected 86 tasks from 105 domain experts across 11 domains, every task is verifiable, human created and has verified Skills. SOTA model without skills score ~30% without skills. \n\nWe found a few interesting things: \n1. Skills substitute for model scale — Haiku 4.5 with Skills (27.7%) beats Opus 4.5 without (22.0%). The right procedural knowledge can be worth more than a bigger model. \n2. Skills' improvement has nothing to do with LLMs' internal knowledge. We have an ablation where no Skills provided for the agent, but the agent is prompted to generate relevant procedural knowledge before solving the task. This isolates the impact of LLMs' latent domain knowledge. \n\nThe result is: \n* Curated Skills: +16.2pp average improvement across all 7 agent configs \n* Self-generated Skills: -1.3pp: models can't write their own procedural knowledge pre-trajectory feedbacks. This is used to isolate the impact of LLMs' latent domain knowledge.","html":"
We made the first benchmark that measures the efficacy of agent skills. We collected 86 tasks from 105 domain experts across 11 domains, every task is verifiable, human created and has verified Skills. SOTA model without skills score ~30% without skills.
\nWe found a few interesting things:
\n- \n
- Skills substitute for model scale — Haiku 4.5 with Skills (27.7%) beats Opus 4.5 without (22.0%). The right procedural knowledge can be worth more than a bigger model. \n
- Skills' improvement has nothing to do with LLMs' internal knowledge. We have an ablation where no Skills provided for the agent, but the agent is prompted to generate relevant procedural knowledge before solving the task. This isolates the impact of LLMs' latent domain knowledge. \n
The result is:
\n- \n
- Curated Skills: +16.2pp average improvement across all 7 agent configs \n
- Self-generated Skills: -1.3pp: models can't write their own procedural knowledge pre-trajectory feedbacks. This is used to isolate the impact of LLMs' latent domain knowledge. \n
arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/skillsbench-benchmarking-how-well-agent-skills-work-across-diverse-tasks-1475-a8815427
\n- \n
- Executive Summary \n
- Detailed Breakdown \n
- Practical Applications \n
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding (2026) \n
- LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026) \n
- Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward (2026) \n
- TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents (2026) \n
- The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environments (2026) \n
- CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion (2026) \n
- AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Papers
arxiv:2602.12670
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Authors:
Abstract
SkillsBench evaluates agent skills across 86 tasks and finds that curated skills improve performance significantly but inconsistently, while self-generated skills offer no benefit, indicating that models struggle to create useful procedural knowledge despite benefiting from curated versions.
AI-generated summary
Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers. Each task is evaluated under three conditions: no Skills, curated Skills, and self-generated Skills. We test 7 agent-model configurations over 7,308 trajectories. Curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain (+4.5pp for Software Engineering to +51.9pp for Healthcare) and 16 of 84 tasks show negative deltas. Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming. Focused Skills with 2--3 modules outperform comprehensive documentation, and smaller models with Skills can match larger models without them.
Community
We made the first benchmark that measures the efficacy of agent skills. We collected 86 tasks from 105 domain experts across 11 domains, every task is verifiable, human created and has verified Skills. SOTA model without skills score ~30% without skills.
We found a few interesting things:
- Skills substitute for model scale — Haiku 4.5 with Skills (27.7%) beats Opus 4.5 without (22.0%). The right procedural knowledge can be worth more than a bigger model.
- Skills' improvement has nothing to do with LLMs' internal knowledge. We have an ablation where no Skills provided for the agent, but the agent is prompted to generate relevant procedural knowledge before solving the task. This isolates the impact of LLMs' latent domain knowledge.
The result is:
- Curated Skills: +16.2pp average improvement across all 7 agent configs
- Self-generated Skills: -1.3pp: models can't write their own procedural knowledge pre-trajectory feedbacks. This is used to isolate the impact of LLMs' latent domain knowledge.
arXivLens breakdown of this paper 👉 https://arxivlens.com/PaperView/Details/skillsbench-benchmarking-how-well-agent-skills-work-across-diverse-tasks-1475-a8815427
- Executive Summary
- Detailed Breakdown
- Practical Applications
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding (2026)
- LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026)
- Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward (2026)
- TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents (2026)
- The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environments (2026)
- CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion (2026)
- AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2602.12670 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2602.12670 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2602.12670 in a Space README.md to link it from this page.