Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

DocLLM: A layout-aware generative language model for multimodal document understanding (2023)
ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science (2023)
An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction (2024)
HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding (2023)
Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs (2023)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

\n","updatedAt":"2024-01-09T09:54:07.089Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6889129281044006},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[{"reaction":"👍","users":["davanstrien","Darth-Sidious"],"count":2}],"isReport":false}},{"id":"659f23ad21c219062cda8371","author":{"_id":"5e6a3d4ea9afd5125d9ec064","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584020801691-noauth.jpeg","fullname":"Stefan Schweter","name":"stefan-it","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3686,"isUserFollowing":false},"createdAt":"2024-01-10T23:09:33.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Repo skeleton is here: https://github.com/dswang2011/DocGraphLM","html":"

Repo skeleton is here: https://github.com/dswang2011/DocGraphLM

\n","updatedAt":"2024-01-10T23:09:33.009Z","author":{"_id":"5e6a3d4ea9afd5125d9ec064","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584020801691-noauth.jpeg","fullname":"Stefan Schweter","name":"stefan-it","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3686,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.47971320152282715},"editors":["stefan-it"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584020801691-noauth.jpeg"],"reactions":[{"reaction":"🤗","users":["davanstrien"],"count":1}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2401.02823","authors":[{"_id":"659b6ac6eff07dcf1ff16a0c","user":{"_id":"64863e4e60b30c3615b92812","avatarUrl":"/avatars/e4a271b53e567f9423bcd9e62cf77889.svg","isPro":false,"fullname":"Dongsheng Wang","user":"dswang2011","type":"user"},"name":"Dongsheng Wang","status":"admin_assigned","statusLastChangedAt":"2024-01-08T11:37:20.833Z","hidden":false},{"_id":"659b6ac6eff07dcf1ff16a0d","user":{"_id":"64b702e8fa7eabaae5f0cbfb","avatarUrl":"/avatars/512690f60997ae81b79b999a634a1861.svg","isPro":false,"fullname":"Zhiqiang Ma","user":"zqmaster","type":"user"},"name":"Zhiqiang Ma","status":"extracted_pending","statusLastChangedAt":"2024-01-08T03:23:50.788Z","hidden":false},{"_id":"659b6ac6eff07dcf1ff16a0e","user":{"_id":"5e79436f30dc073f817a2b91","avatarUrl":"/avatars/5ffac317818689106be1a00ee69cded0.svg","isPro":false,"fullname":"Armi Nouri","user":"arminouri","type":"user"},"name":"Armineh Nourbakhsh","status":"claimed_verified","statusLastChangedAt":"2024-01-08T08:47:23.876Z","hidden":false},{"_id":"659b6ac6eff07dcf1ff16a0f","name":"Kang Gu","hidden":false},{"_id":"659b6ac6eff07dcf1ff16a10","name":"Sameena Shah","hidden":false}],"publishedAt":"2024-01-05T14:15:36.000Z","submittedOnDailyAt":"2024-01-08T00:53:50.888Z","title":"DocGraphLM: Documental Graph Language Model for Information Extraction","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"Advances in Visually Rich Document Understanding (VrDU) have enabled\ninformation extraction and question answering over documents with complex\nlayouts. Two tropes of architectures have emerged -- transformer-based models\ninspired by LLMs, and Graph Neural Networks. In this paper, we introduce\nDocGraphLM, a novel framework that combines pre-trained language models with\ngraph semantics. To achieve this, we propose 1) a joint encoder architecture to\nrepresent documents, and 2) a novel link prediction approach to reconstruct\ndocument graphs. DocGraphLM predicts both directions and distances between\nnodes using a convergent joint loss function that prioritizes neighborhood\nrestoration and downweighs distant node detection. Our experiments on three\nSotA datasets show consistent improvement on IE and QA tasks with the adoption\nof graph features. Moreover, we report that adopting the graph features\naccelerates convergence in the learning process during training, despite being\nsolely constructed through link prediction.","upvotes":36,"discussionId":"659b6ac6eff07dcf1ff16a2d","ai_summary":"A new framework, DocGraphLM, combines pre-trained language models with graph semantics to improve information extraction and question answering on documents with complex layouts.","ai_keywords":["transformer-based models","Graph Neural Networks","DocGraphLM","joint encoder architecture","link prediction","joint loss function","neighborhood restoration","distant node detection"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63f9cf9f6f55c54f43de36f3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63f9cf9f6f55c54f43de36f3/-OP-VEOqA3A9OAXI75CnX.jpeg","isPro":false,"fullname":"Arnav Bathla","user":"arnavbathla","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6221ec04028e7e9fcbf0748c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1663269439812-6221ec04028e7e9fcbf0748c.jpeg","isPro":false,"fullname":"Rahul V R","user":"rahulvramesh","type":"user"},{"_id":"63307392e7efa1d1a88213ef","avatarUrl":"/avatars/d675566e7a57440c89e642c98cc24c2b.svg","isPro":false,"fullname":"allzero","user":"da02","type":"user"},{"_id":"64c699fd5e5bc55a92245aab","avatarUrl":"/avatars/6155401e469f73c6afaad2c2fca76a63.svg","isPro":false,"fullname":"Alex","user":"aslessor","type":"user"},{"_id":"646427889dd8b530a8615fd8","avatarUrl":"/avatars/72a38d297cec02cdad7c8555dd0e759f.svg","isPro":false,"fullname":"Vince","user":"bolerovt","type":"user"},{"_id":"64a601ecca94a6c037eaa625","avatarUrl":"/avatars/1d97950f70cdb0347d5d4f529bfe8af4.svg","isPro":false,"fullname":"Muten Roshi","user":"MutenRoshi","type":"user"},{"_id":"6450b7e0887fb4f6ae348cd8","avatarUrl":"/avatars/5d2a94c41b0560c91494131afa3deb3f.svg","isPro":true,"fullname":"Erick Sgarbi","user":"sgarbi","type":"user"},{"_id":"6249b4058abe179b008ed178","avatarUrl":"/avatars/485520300100e1ae6a4d18f55db949e0.svg","isPro":true,"fullname":"Danny Busch","user":"dannybusch","type":"user"},{"_id":"63501e81cfefce6e577b597c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63501e81cfefce6e577b597c/k6JtSOxJGUZxOcW5PkOKq.png","isPro":false,"fullname":"Yuan He","user":"heyuan","type":"user"},{"_id":"62fbc913a80632fbd47ac32f","avatarUrl":"/avatars/1a11793e9d23706486736b71eb5f271f.svg","isPro":false,"fullname":"Aravind Prem Kumar Prabhakar","user":"Aravindprem","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":2}">

Papers

arxiv:2401.02823

DocGraphLM: Documental Graph Language Model for Information Extraction

Published on Jan 5, 2024

· Submitted by

AK on Jan 8, 2024

#2 Paper of the day

Upvote

Authors:

Dongsheng Wang ,

Zhiqiang Ma ,

Armineh Nourbakhsh ,

Abstract

A new framework, DocGraphLM, combines pre-trained language models with graph semantics to improve information extraction and question answering on documents with complex layouts.

AI-generated summary

Advances in Visually Rich Document Understanding (VrDU) have enabled information extraction and question answering over documents with complex layouts. Two tropes of architectures have emerged -- transformer-based models inspired by LLMs, and Graph Neural Networks. In this paper, we introduce DocGraphLM, a novel framework that combines pre-trained language models with graph semantics. To achieve this, we propose 1) a joint encoder architecture to represent documents, and 2) a novel link prediction approach to reconstruct document graphs. DocGraphLM predicts both directions and distances between nodes using a convergent joint loss function that prioritizes neighborhood restoration and downweighs distant node detection. Our experiments on three SotA datasets show consistent improvement on IE and QA tasks with the adoption of graph features. Moreover, we report that adopting the graph features accelerates convergence in the learning process during training, despite being solely constructed through link prediction.