homepage: https://retrieval-infused-reasoning-sandbox.github.io/\n","updatedAt":"2026-02-06T03:54:42.397Z","author":{"_id":"6377366acc034ef804cf0aef","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6377366acc034ef804cf0aef/JHsET8LJE2w1E7vJD-R2o.png","fullname":"ShawnYing","name":"ShawnYing","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.44112762808799744},"editors":["ShawnYing"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6377366acc034ef804cf0aef/JHsET8LJE2w1E7vJD-R2o.png"],"reactions":[],"isReport":false,"parentCommentId":"698565cf48999b11958cecfa"}}]},{"id":"6986293d29238d0c819eb9a2","author":{"_id":"65243980050781c16f234f1f","avatarUrl":"/avatars/743a009681d5d554c27e04300db9f267.svg","fullname":"Avi","name":"avahal","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false},"createdAt":"2026-02-06T17:47:41.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/retrieval-infused-reasoning-sandbox-a-benchmark-for-decoupling-retrieval-and-reasoning-capabilities-6192-add5f3f6\n- Executive Summary\n- Detailed Breakdown\n- Practical Applications","html":"
arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/retrieval-infused-reasoning-sandbox-a-benchmark-for-decoupling-retrieval-and-reasoning-capabilities-6192-add5f3f6
\n- \n
- Executive Summary \n
- Detailed Breakdown \n
- Practical Applications \n
arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/retrieval-infused-reasoning-sandbox-a-benchmark-for-decoupling-retrieval-and-reasoning-capabilities-6192-add5f3f6
\n- \n
- Executive Summary \n
- Detailed Breakdown \n
- Practical Applications \n
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering (2026) \n
- DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing (2026) \n
- SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature (2026) \n
- SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence (2026) \n
- RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension (2026) \n
- Beyond the Needle's Illusion: Decoupled Evaluation of Evidence Access and Use under Semantic Interference at 326M-Token Scale (2026) \n
- DeepEra: A Deep Evidence Reranking Agent for Scientific Retrieval-Augmented Generated Question Answering (2026) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Retrieval-Infused Reasoning Sandbox: A Benchmark for Decoupling Retrieval and Reasoning Capabilities
Abstract
DeR2 presents a controlled evaluation framework for assessing language models' document-grounded reasoning capabilities by isolating reasoning from retrieval and toolchain decisions.
Despite strong performance on existing benchmarks, it remains unclear whether large language models can reason over genuinely novel scientific information. Most evaluations score end-to-end RAG pipelines, where reasoning is confounded with retrieval and toolchain choices, and the signal is further contaminated by parametric memorization and open-web volatility. We introduce DeR2, a controlled deep-research sandbox that isolates document-grounded reasoning while preserving core difficulties of deep search: multi-step synthesis, denoising, and evidence-based conclusion making. DeR2 decouples evidence access from reasoning via four regimes--Instruction-only, Concepts (gold concepts without documents), Related-only (only relevant documents), and Full-set (relevant documents plus topically related distractors)--yielding interpretable regime gaps that operationalize retrieval loss vs. reasoning loss and enable fine-grained error attribution. To prevent parametric leakage, we apply a two-phase validation that requires parametric failure without evidence while ensuring oracle-concept solvability. To ensure reproducibility, each instance provides a frozen document library (drawn from 2023-2025 theoretical papers) with expert-annotated concepts and validated rationales. Experiments across a diverse set of state-of-the-art foundation models reveal substantial variation and significant headroom: some models exhibit mode-switch fragility, performing worse with the Full-set than with Instruction-only, while others show structural concept misuse, correctly naming concepts but failing to execute them as procedures.
Community
Despite strong performance on existing benchmarks, it remains unclear whether large language models can reason over genuinely novel scientific information. Most evaluations score end-to-end RAG pipelines, where reasoning is confounded with retrieval and toolchain choices, and the signal is further contaminated by parametric memorization and open-web volatility. We introduce DeR2, a controlled deep-research sandbox that isolates document-grounded reasoning while preserving core difficulties of deep search: multi-step synthesis, denoising, and evidence-based conclusion making. DeR2 decouples evidence access from reasoning via four regimes--Instruction-only, Concepts (gold concepts without documents), Related-only (only relevant documents), and Full-set (relevant documents plus topically related distractors)--yielding interpretable regime gaps that operationalize retrieval loss vs. reasoning loss and enable fine-grained error attribution. To prevent parametric leakage, we apply a two-phase validation that requires parametric failure without evidence while ensuring oracle-concept solvability. To ensure reproducibility, each instance provides a frozen document library (drawn from 2023-2025 theoretical papers) with expert-annotated concepts and validated rationales. Experiments across a diverse set of state-of-the-art foundation models reveal substantial variation and significant headroom: some models exhibit mode-switch fragility, performing worse with the Full-set than with Instruction-only, while others show structural concept misuse, correctly naming concepts but failing to execute them as procedures.
arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/retrieval-infused-reasoning-sandbox-a-benchmark-for-decoupling-retrieval-and-reasoning-capabilities-6192-add5f3f6
- Executive Summary
- Detailed Breakdown
- Practical Applications
arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/retrieval-infused-reasoning-sandbox-a-benchmark-for-decoupling-retrieval-and-reasoning-capabilities-6192-add5f3f6
- Executive Summary
- Detailed Breakdown
- Practical Applications
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering (2026)
- DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing (2026)
- SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature (2026)
- SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence (2026)
- RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension (2026)
- Beyond the Needle's Illusion: Decoupled Evaluation of Evidence Access and Use under Semantic Interference at 326M-Token Scale (2026)
- DeepEra: A Deep Evidence Reranking Agent for Scientific Retrieval-Augmented Generated Question Answering (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper