Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report (2026)
DEER: A Comprehensive and Reliable Benchmark for Deep-Research Expert Reports (2025)
DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing (2026)
IDRBench: Interactive Deep Research Benchmark (2026)
Mind2Report: A Cognitive Deep Research Agent for Expert-Level Commercial Report Synthesis (2026)
ToolGym: an Open-world Tool-using Environment for Scalable Agent Testing and Data Curation (2026)
How Far Are We from Genuinely Useful Deep Research Agents? (2025)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2026-01-16T01:36:26.267Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":317,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7130248546600342},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"6969fee34f549f94c95b002e","author":{"_id":"65d9fc2a0e6ad24551d87a1e","avatarUrl":"/avatars/3aedb9522cc3cd08349d654f523fd792.svg","fullname":"Grant Singleton","name":"grantsing","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false},"createdAt":"2026-01-16T09:03:31.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"arXiv explained breakdown of this paper 👉 https://arxivexplained.com/papers/deepresearcheval-an-automated-framework-for-deep-research-task-construction-and-agentic-evaluation\n","html":"

arXiv explained breakdown of this paper 👉 https://arxivexplained.com/papers/deepresearcheval-an-automated-framework-for-deep-research-task-construction-and-agentic-evaluation

\n","updatedAt":"2026-01-16T09:03:31.723Z","author":{"_id":"65d9fc2a0e6ad24551d87a1e","avatarUrl":"/avatars/3aedb9522cc3cd08349d654f523fd792.svg","fullname":"Grant Singleton","name":"grantsing","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.680065393447876},"editors":["grantsing"],"editorAvatarUrls":["/avatars/3aedb9522cc3cd08349d654f523fd792.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2601.09688","authors":[{"_id":"696864c90ac10a06522f6a4a","user":{"_id":"66f16d166f7038039a1e2770","avatarUrl":"/avatars/0a30d3e9af3b109ce4b82396b0e8d685.svg","isPro":false,"fullname":"Yibo Wang","user":"yiboowang","type":"user"},"name":"Yibo Wang","status":"claimed_verified","statusLastChangedAt":"2026-01-16T14:37:10.922Z","hidden":false},{"_id":"696864c90ac10a06522f6a4b","user":{"_id":"646def60df618b303b419323","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/646def60df618b303b419323/JLJGYen4-5M8ivsLsSk0w.jpeg","isPro":false,"fullname":"Lei Wang","user":"demolei","type":"user"},"name":"Lei Wang","status":"claimed_verified","statusLastChangedAt":"2026-01-16T10:32:43.037Z","hidden":false},{"_id":"696864c90ac10a06522f6a4c","name":"Yue Deng","hidden":false},{"_id":"696864c90ac10a06522f6a4d","user":{"_id":"66bf00ca5b4e241fe266059d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66bf00ca5b4e241fe266059d/VoWPC_C4zoeT6dS699t7L.png","isPro":false,"fullname":"Keming Wu","user":"wukeming11","type":"user"},"name":"Keming Wu","status":"claimed_verified","statusLastChangedAt":"2026-01-15T15:02:22.232Z","hidden":false},{"_id":"696864c90ac10a06522f6a4e","name":"Yao Xiao","hidden":false},{"_id":"696864c90ac10a06522f6a4f","name":"Huanjin Yao","hidden":false},{"_id":"696864c90ac10a06522f6a50","name":"Liwei Kang","hidden":false},{"_id":"696864c90ac10a06522f6a51","name":"Hai Ye","hidden":false},{"_id":"696864c90ac10a06522f6a52","name":"Yongcheng Jing","hidden":false},{"_id":"696864c90ac10a06522f6a53","name":"Lidong Bing","hidden":false}],"publishedAt":"2026-01-14T18:38:31.000Z","submittedOnDailyAt":"2026-01-15T01:33:59.520Z","title":"DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation","submittedOnDailyBy":{"_id":"66bf00ca5b4e241fe266059d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66bf00ca5b4e241fe266059d/VoWPC_C4zoeT6dS699t7L.png","isPro":false,"fullname":"Keming Wu","user":"wukeming11","type":"user"},"summary":"Deep research systems are widely used for multi-step web research, analysis, and cross-source synthesis, yet their evaluation remains challenging. Existing benchmarks often require annotation-intensive task construction, rely on static evaluation dimensions, or fail to reliably verify facts when citations are missing. To bridge these gaps, we introduce DeepResearchEval, an automated framework for deep research task construction and agentic evaluation. For task construction, we propose a persona-driven pipeline generating realistic, complex research tasks anchored in diverse user profiles, applying a two-stage filter Task Qualification and Search Necessity to retain only tasks requiring multi-source evidence integration and external retrieval. For evaluation, we propose an agentic pipeline with two components: an Adaptive Point-wise Quality Evaluation that dynamically derives task-specific evaluation dimensions, criteria, and weights conditioned on each generated task, and an Active Fact-Checking that autonomously extracts and verifies report statements via web search, even when citations are missing.","upvotes":126,"discussionId":"696864c90ac10a06522f6a54","projectPage":"https://infinity-ailab.github.io/deep_research_eval/","githubRepo":"https://github.com/Infinity-AILab/DeepResearchEval","githubRepoAddedBy":"auto","ai_summary":"DeepResearchEval presents an automated framework for creating complex research tasks and evaluating them through agent-based methods that adapt to task specifics and verify facts without relying on citations.","ai_keywords":["automated framework","deep research task construction","agentic evaluation","persona-driven pipeline","task qualification","search necessity","adaptive point-wise quality evaluation","active fact-checking","web search","multi-source evidence integration"],"githubStars":127,"organization":{"_id":"6948e6c46d88786b0ec9cf9d","name":"Infinity-AILab","fullname":"Infinity Lab","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6362a77dd3be91534c2e9213/-zILHmHPjnq27MzoESFsG.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66f16d166f7038039a1e2770","avatarUrl":"/avatars/0a30d3e9af3b109ce4b82396b0e8d685.svg","isPro":false,"fullname":"Yibo Wang","user":"yiboowang","type":"user"},{"_id":"66bf00ca5b4e241fe266059d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66bf00ca5b4e241fe266059d/VoWPC_C4zoeT6dS699t7L.png","isPro":false,"fullname":"Keming Wu","user":"wukeming11","type":"user"},{"_id":"6514599ee31c0e2e3dfb5c9c","avatarUrl":"/avatars/3c3ebd14d228c4c439da542cf8ff20a8.svg","isPro":false,"fullname":"ymh233","user":"ymh233","type":"user"},{"_id":"64104b467a15af878ae6695d","avatarUrl":"/avatars/407983918c12411e5ed636bf7435522b.svg","isPro":false,"fullname":"Fangyu Lei","user":"FangyuLei","type":"user"},{"_id":"656010c6e23401f820041e47","avatarUrl":"/avatars/9678121773b22e98d53e14373e874c81.svg","isPro":false,"fullname":"Yilun Kong","user":"Yilun-Kong","type":"user"},{"_id":"65d6a6f7654f85ff0baf161f","avatarUrl":"/avatars/4a46f8b0522fa572c122249a9d6526c4.svg","isPro":false,"fullname":"Yansong NING","user":"yasNing","type":"user"},{"_id":"6590e03454f8826173ed5ee6","avatarUrl":"/avatars/f5e59d3e58c28a99f2ff39267ca51cdb.svg","isPro":false,"fullname":"Huanjin Yao","user":"HuanjinYao","type":"user"},{"_id":"66ed66a9c77b73895d5f45b5","avatarUrl":"/avatars/fe8d098d133e1b5e3025ddbc7f158de9.svg","isPro":false,"fullname":"Zixuan Hu","user":"EggHu","type":"user"},{"_id":"665e6af92d0161db302f5dbb","avatarUrl":"/avatars/3f1044d90057f1e84e1c4bddd3fc1aa1.svg","isPro":false,"fullname":"guozhi liu","user":"Lslland","type":"user"},{"_id":"65b76915866d92dcfd4f9fb7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65b76915866d92dcfd4f9fb7/WghH3wkLLZ66buF-zDyn3.jpeg","isPro":false,"fullname":"wangbingchao","user":"LilCC","type":"user"},{"_id":"6524d665ab1416594149e07e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6524d665ab1416594149e07e/KMsCaAtV0DLC4tqN8f2a7.png","isPro":false,"fullname":"Zuhao Yang","user":"mwxely","type":"user"},{"_id":"6454685a548f22be598414c4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg","isPro":false,"fullname":"Lidong Bing","user":"LidongBing","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":1,"organization":{"_id":"6948e6c46d88786b0ec9cf9d","name":"Infinity-AILab","fullname":"Infinity Lab","avatar":"https://cdn-uploads.huggingface.co/production/uploads/6362a77dd3be91534c2e9213/-zILHmHPjnq27MzoESFsG.png"}}">

Papers

arxiv:2601.09688

DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation

Published on Jan 14

· Submitted by

Keming Wu on Jan 15

#1 Paper of the day

Infinity Lab

Upvote

126

Authors:

Yibo Wang ,

Lei Wang ,

Keming Wu ,

Abstract

DeepResearchEval presents an automated framework for creating complex research tasks and evaluating them through agent-based methods that adapt to task specifics and verify facts without relying on citations.

AI-generated summary

Deep research systems are widely used for multi-step web research, analysis, and cross-source synthesis, yet their evaluation remains challenging. Existing benchmarks often require annotation-intensive task construction, rely on static evaluation dimensions, or fail to reliably verify facts when citations are missing. To bridge these gaps, we introduce DeepResearchEval, an automated framework for deep research task construction and agentic evaluation. For task construction, we propose a persona-driven pipeline generating realistic, complex research tasks anchored in diverse user profiles, applying a two-stage filter Task Qualification and Search Necessity to retain only tasks requiring multi-source evidence integration and external retrieval. For evaluation, we propose an agentic pipeline with two components: an Adaptive Point-wise Quality Evaluation that dynamically derives task-specific evaluation dimensions, criteria, and weights conditioned on each generated task, and an Active Fact-Checking that autonomously extracts and verifies report statements via web search, even when citations are missing.