Datasets: https://huggingface.co/ml-jku\n","updatedAt":"2024-10-10T07:13:42.416Z","author":{"_id":"648826f845a9218318e0272c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/648826f845a9218318e0272c/zWrSQblli4PkmiIzWH1RS.jpeg","fullname":"Fabian Paischer","name":"paischer101","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":1,"editors":["paischer101"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/648826f845a9218318e0272c/zWrSQblli4PkmiIzWH1RS.jpeg"],"reactions":[],"isReport":false}},{"id":"670880870e79a8b46f7ff5dd","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2024-10-11T01:33:59.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI](https://huggingface.co/papers/2410.02751) (2024)\n* [Predictive Coding for Decision Transformer](https://huggingface.co/papers/2410.03408) (2024)\n* [In-Context Imitation Learning via Next-Token Prediction](https://huggingface.co/papers/2408.15980) (2024)\n* [Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining](https://huggingface.co/papers/2410.00564) (2024)\n* [RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models](https://huggingface.co/papers/2409.12294) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI (2024) \n
- Predictive Coding for Decision Transformer (2024) \n
- In-Context Imitation Learning via Next-Token Prediction (2024) \n
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (2024) \n
- RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Retrieval-Augmented Decision Transformer: External Memory for In-context RL
Abstract
RA-DT, a retrieval-augmented decision transformer that uses external memory for in-context learning, outperforms baselines in grid-world environments and robotics simulations by leveraging relevant sub-trajectories without full episode training.
In-context learning (ICL) is the ability of a model to learn a new task by observing a few exemplars in its context. While prevalent in NLP, this capability has recently also been observed in Reinforcement Learning (RL) settings. Prior in-context RL methods, however, require entire episodes in the agent's context. Given that complex environments typically lead to long episodes with sparse rewards, these methods are constrained to simple environments with short episodes. To address these challenges, we introduce Retrieval-Augmented Decision Transformer (RA-DT). RA-DT employs an external memory mechanism to store past experiences from which it retrieves only sub-trajectories relevant for the current situation. The retrieval component in RA-DT does not require training and can be entirely domain-agnostic. We evaluate the capabilities of RA-DT on grid-world environments, robotics simulations, and procedurally-generated video games. On grid-worlds, RA-DT outperforms baselines, while using only a fraction of their context length. Furthermore, we illuminate the limitations of current in-context RL methods on complex environments and discuss future directions. To facilitate future research, we release datasets for four of the considered environments.
Community
GitHub: https://github.com/ml-jku/RA-DT
Datasets: https://huggingface.co/ml-jku
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI (2024)
- Predictive Coding for Decision Transformer (2024)
- In-Context Imitation Learning via Next-Token Prediction (2024)
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (2024)
- RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 3
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper