Write a top 5points
\n","updatedAt":"2023-10-16T16:29:10.727Z","author":{"_id":"650e696ba0f2ffbeca687a3a","avatarUrl":"/avatars/d5365258828e5aecedc3cdf2768c4f60.svg","fullname":"PATHAN ASHRAF ","name":"Pathanashraf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7698227167129517},"editors":["Pathanashraf"],"editorAvatarUrls":["/avatars/d5365258828e5aecedc3cdf2768c4f60.svg"],"reactions":[],"isReport":false}},{"id":"652d84ca956d6a4244e5e4b5","author":{"_id":"64660eae875b1a86a786c04e","avatarUrl":"/avatars/8b4900c358848aafd55194c12aa31bfe.svg","fullname":"Rohit Kumar","name":"Rohit-788","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2023-10-16T18:45:30.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"\n\n","html":"\n","updatedAt":"2023-10-16T18:45:30.764Z","author":{"_id":"64660eae875b1a86a786c04e","avatarUrl":"/avatars/8b4900c358848aafd55194c12aa31bfe.svg","fullname":"Rohit Kumar","name":"Rohit-788","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.3670727014541626},"editors":["Rohit-788"],"editorAvatarUrls":["/avatars/8b4900c358848aafd55194c12aa31bfe.svg"],"reactions":[],"isReport":false}},{"id":"652d84e546e3998e41e94d15","author":{"_id":"64660eae875b1a86a786c04e","avatarUrl":"/avatars/8b4900c358848aafd55194c12aa31bfe.svg","fullname":"Rohit Kumar","name":"Rohit-788","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2023-10-16T18:45:57.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"start discussing about this paper\n","html":"start discussing about this paper
\n","updatedAt":"2023-10-16T18:45:57.720Z","author":{"_id":"64660eae875b1a86a786c04e","avatarUrl":"/avatars/8b4900c358848aafd55194c12aa31bfe.svg","fullname":"Rohit Kumar","name":"Rohit-788","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8430055379867554},"editors":["Rohit-788"],"editorAvatarUrls":["/avatars/8b4900c358848aafd55194c12aa31bfe.svg"],"reactions":[],"isReport":false}},{"id":"652dadbdd838856129ef807a","author":{"_id":"6486638da4cf2081f20c40ec","avatarUrl":"/avatars/0bc16a7447cd71ac18828a678313bd83.svg","fullname":"Mike Young","name":"mikelabs","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":13,"isUserFollowing":false},"createdAt":"2023-10-16T21:40:13.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Wtf is going on in these comments lol? Anyway, here's my summary...\n\nTables are everywhere - reports, databases, webpages. They neatly organize data for humans to parse. But despite strong language skills, AI still struggles with table comprehension.\n\nEven models like GPT-3 fail at basic tasks like finding where a missing value should go. This is because they're trained mostly on free-flowing text, not 2D tabular data. Unlike unstructured text, data in tables derives meaning from its structure and position!\n\nSo researchers at Microsoft tried \"table-tuning\" - extending training with synthesized table task cases. Tasks like \"impute missing value X\" or \"identify outliers in this table\". They did this using a corpus of real-world tables.\n\nThey also augmented the data more by paraphrasing, reordering rows/columns, chaining model responses, and more. This protects against overfitting.\n\nThe resulting Table-GPT models showed big improvements:\n\n* 25%+ better at unseen table tasks like missing value ID\n* Beat GPT-3 on 98% of test cases over 9 different table tasks\n* Stayed strong even after targeted downstream tuning\n\nTable-tuning seems a promising step toward AI that can handle tables. That would unlock automated analysis over the troves of valuable tabular data out there.\n\n**TLDR: Training models on a large and diverse dataset of synthesized table tasks significantly boosts their table skills.**\n\n[Full Summary is here.](https://notes.aimodels.fyi/table-gpt-table-tuned-gpt-for-diverse-table-tasks/)","html":"Wtf is going on in these comments lol? Anyway, here's my summary...
\nTables are everywhere - reports, databases, webpages. They neatly organize data for humans to parse. But despite strong language skills, AI still struggles with table comprehension.
\nEven models like GPT-3 fail at basic tasks like finding where a missing value should go. This is because they're trained mostly on free-flowing text, not 2D tabular data. Unlike unstructured text, data in tables derives meaning from its structure and position!
\nSo researchers at Microsoft tried \"table-tuning\" - extending training with synthesized table task cases. Tasks like \"impute missing value X\" or \"identify outliers in this table\". They did this using a corpus of real-world tables.
\nThey also augmented the data more by paraphrasing, reordering rows/columns, chaining model responses, and more. This protects against overfitting.
\nThe resulting Table-GPT models showed big improvements:
\n- \n
- 25%+ better at unseen table tasks like missing value ID \n
- Beat GPT-3 on 98% of test cases over 9 different table tasks \n
- Stayed strong even after targeted downstream tuning \n
Table-tuning seems a promising step toward AI that can handle tables. That would unlock automated analysis over the troves of valuable tabular data out there.
\nTLDR: Training models on a large and diverse dataset of synthesized table tasks significantly boosts their table skills.
\n\n","updatedAt":"2023-10-16T21:44:29.235Z","author":{"_id":"6486638da4cf2081f20c40ec","avatarUrl":"/avatars/0bc16a7447cd71ac18828a678313bd83.svg","fullname":"Mike Young","name":"mikelabs","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":13,"isUserFollowing":false}},"numEdits":2,"identifiedLanguage":{"language":"en","probability":0.894295871257782},"editors":["mikelabs"],"editorAvatarUrls":["/avatars/0bc16a7447cd71ac18828a678313bd83.svg"],"reactions":[{"reaction":"🤗","users":["mikelabs","KrishnaKaasyap","amberberli","sciafri","jj97"],"count":5},{"reaction":"👍","users":["b08x","tdingman-scale","sciafri"],"count":3},{"reaction":"❤️","users":["sciafri","kkuhlman"],"count":2}],"isReport":false}},{"id":"652dd3db99687726aef79ffa","author":{"_id":"64aed48cfec303c461d06242","avatarUrl":"/avatars/236c771e6c5a25ef6ed5e1bc061e30b8.svg","fullname":"Krishna Kaasyap","name":"KrishnaKaasyap","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false},"createdAt":"2023-10-17T00:22:51.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"> Can you opensource your training datasets of the different table tasks?\n\nNot sure about that but here's a table dataset that is more than 800 billion tokens!\n\nhttps://huggingface.co/datasets/approximatelabs/tablib-v1-full","html":"\n\nCan you opensource your training datasets of the different table tasks?
\n
Not sure about that but here's a table dataset that is more than 800 billion tokens!
\nhttps://huggingface.co/datasets/approximatelabs/tablib-v1-full
\n","updatedAt":"2023-10-17T00:22:51.192Z","author":{"_id":"64aed48cfec303c461d06242","avatarUrl":"/avatars/236c771e6c5a25ef6ed5e1bc061e30b8.svg","fullname":"Krishna Kaasyap","name":"KrishnaKaasyap","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8816072940826416},"editors":["KrishnaKaasyap"],"editorAvatarUrls":["/avatars/236c771e6c5a25ef6ed5e1bc061e30b8.svg"],"reactions":[{"reaction":"👍","users":["jj97","LisaWang0306"],"count":2}],"isReport":false}},{"id":"6531189fba7ae55b6549179f","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2023-10-19T11:53:03.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks](https://huggingface.co/papers/2310.00789) (2023)\n* [LLM-augmented Preference Learning from Natural Language](https://huggingface.co/papers/2310.08523) (2023)\n* [Efficient Finetuning Large Language Models For Vietnamese Chatbot](https://huggingface.co/papers/2309.04646) (2023)\n* [A Systematic Evaluation of Large Language Models on Out-of-Distribution Logical Reasoning Tasks](https://huggingface.co/papers/2310.09430) (2023)\n* [Training Generative Question-Answering on Synthetic Data Obtained from an Instruct-tuned Model](https://huggingface.co/papers/2310.08072) (2023)\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space","html":"This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks (2023) \n
- LLM-augmented Preference Learning from Natural Language (2023) \n
- Efficient Finetuning Large Language Models For Vietnamese Chatbot (2023) \n
- A Systematic Evaluation of Large Language Models on Out-of-Distribution Logical Reasoning Tasks (2023) \n
- Training Generative Question-Answering on Synthetic Data Obtained from an Instruct-tuned Model (2023) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n","updatedAt":"2023-10-19T11:53:03.673Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7445301413536072},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"654ef7110aa8eba4c209cd31","author":{"_id":"6348f06b5b1a5329cc2a3923","avatarUrl":"/avatars/33212f35f87e7bba5294a5750d96ee0f.svg","fullname":"Shashi Prakash Tripathi","name":"ShishuTripathi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2023-11-11T03:37:53.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I am still not very sure with GPT capabilities of tackling arithmetical calculation in any context let it be tables or time series data. Till now the output that I have seen is not even close.\n","html":"I am still not very sure with GPT capabilities of tackling arithmetical calculation in any context let it be tables or time series data. Till now the output that I have seen is not even close.
\n","updatedAt":"2023-11-11T03:37:53.130Z","author":{"_id":"6348f06b5b1a5329cc2a3923","avatarUrl":"/avatars/33212f35f87e7bba5294a5750d96ee0f.svg","fullname":"Shashi Prakash Tripathi","name":"ShishuTripathi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9469090104103088},"editors":["ShishuTripathi"],"editorAvatarUrls":["/avatars/33212f35f87e7bba5294a5750d96ee0f.svg"],"reactions":[],"isReport":false}},{"id":"655bfe8f948930b0fce7d10f","author":{"_id":"63f4afd0cfd7ba6e26f5d0c5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63f4afd0cfd7ba6e26f5d0c5/imFdETtKl94fpQefk0bon.jpeg","fullname":"Damian","name":"QuantumDamian","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2023-11-21T00:49:19.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Key conclusion from my review of v1 of this paper (published on 13th Oct 2023):\n\nThis paper offers good introduction to simple table-tuning tasks, however task T-3 (TQA) should be significantly improved before Table-GPT can be used commercially.\n\nKey points:\n•\tGeneric overview of results indicates very good results of table-tuning, however in my opinion tasks should be divided based on complexity to better understand value of table-tuning. Please see diagram below.\n\n\n\n•\tFor the most of easy tasks (all tasks except T-3) table-tuning offers great improvements in zero-shot comparing to vanilla models (209% improvement for GPT-3.5 and 119% for ChatGPT) \n•\tFor the most of easy tasks (all tasks except T-3) table-tuning offers good results in few-shot comparing to vanilla models (22% improvement for GPT-3.5 and 12% for ChatGPT)\n \n\n\n•\tT-3 TQA is the most complex task (and with biggest business demand) and for this tasks table-tuning offers very small improvements (1-2% for ChatGPT and 5-8% for GPT-3.5), which is probably not worth of the fine-tuning effort\n \n\n\nOpen questions:\n•\tDo you have plans to fine-tune GPT-4?\n•\tCan you share recommendations on improving T-3 (TQA)? Maybe including TQA tasks in training?\n•\tCan you include as well T-12 (NS) in tests?\n•\tCan you specify number of tokens used (both for training and test execution) for each task\n\nOther remarks:\n•\tMarkdown format increases performance of table-tuning by 3% comparing to CSV and by 5% comparing to JSON (table 5)\n•\tFor most of the tasks few-shot offers strong improvement over zero-shot for vanilla GPT 3.5 and ChatGPT (even without table-tuning).\n•\tTypos found in paper: \n-\tp.4 is “toke-by-token” should be “token-by-token”\n-\tp.6 is “few select table-tasks” should be “few selected table-tasks”\n-\tp.7 is “describes the row-augmentation task” should be “describes the column-augmentation task”","html":"Key conclusion from my review of v1 of this paper (published on 13th Oct 2023):
\nThis paper offers good introduction to simple table-tuning tasks, however task T-3 (TQA) should be significantly improved before Table-GPT can be used commercially.
\nKey points:
•\tGeneric overview of results indicates very good results of table-tuning, however in my opinion tasks should be divided based on complexity to better understand value of table-tuning. Please see diagram below.
•\tFor the most of easy tasks (all tasks except T-3) table-tuning offers great improvements in zero-shot comparing to vanilla models (209% improvement for GPT-3.5 and 119% for ChatGPT)
•\tFor the most of easy tasks (all tasks except T-3) table-tuning offers good results in few-shot comparing to vanilla models (22% improvement for GPT-3.5 and 12% for ChatGPT)
•\tT-3 TQA is the most complex task (and with biggest business demand) and for this tasks table-tuning offers very small improvements (1-2% for ChatGPT and 5-8% for GPT-3.5), which is probably not worth of the fine-tuning effort
\n\nOpen questions:
•\tDo you have plans to fine-tune GPT-4?
•\tCan you share recommendations on improving T-3 (TQA)? Maybe including TQA tasks in training?
•\tCan you include as well T-12 (NS) in tests?
•\tCan you specify number of tokens used (both for training and test execution) for each task
Other remarks:
•\tMarkdown format increases performance of table-tuning by 3% comparing to CSV and by 5% comparing to JSON (table 5)
•\tFor most of the tasks few-shot offers strong improvement over zero-shot for vanilla GPT 3.5 and ChatGPT (even without table-tuning).
•\tTypos found in paper:
-\tp.4 is “toke-by-token” should be “token-by-token”
-\tp.6 is “few select table-tasks” should be “few selected table-tasks”
-\tp.7 is “describes the row-augmentation task” should be “describes the column-augmentation task”
\n\t\n\t\t\n\t\n\t\n\t\tRevolutionizing AI: Table-GPT Enhances Language Models for Complex Table Tasks!\n\t\n
\n \n\n\n\t\n\t\t\n\t\n\t\n\t\tLinks đź”—:\n\t\n
\n👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/
Table-GPT: Table-tuned GPT for Diverse Table Tasks
Abstract
A new table-tuning paradigm improves language models' understanding and generalization in table-related tasks by fine-tuning them with synthesized table data, resulting in enhanced performance.
Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on one-dimensional natural-language texts, whereas relational tables are two-dimensional objects. In this work, we propose a new "table-tuning" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better table-understanding capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong generalizability, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.
Community
Can you opensource your training datasets of the different table tasks?
Write a top 5points
start discussing about this paper
Wtf is going on in these comments lol? Anyway, here's my summary...
Tables are everywhere - reports, databases, webpages. They neatly organize data for humans to parse. But despite strong language skills, AI still struggles with table comprehension.
Even models like GPT-3 fail at basic tasks like finding where a missing value should go. This is because they're trained mostly on free-flowing text, not 2D tabular data. Unlike unstructured text, data in tables derives meaning from its structure and position!
So researchers at Microsoft tried "table-tuning" - extending training with synthesized table task cases. Tasks like "impute missing value X" or "identify outliers in this table". They did this using a corpus of real-world tables.
They also augmented the data more by paraphrasing, reordering rows/columns, chaining model responses, and more. This protects against overfitting.
The resulting Table-GPT models showed big improvements:
- 25%+ better at unseen table tasks like missing value ID
- Beat GPT-3 on 98% of test cases over 9 different table tasks
- Stayed strong even after targeted downstream tuning
Table-tuning seems a promising step toward AI that can handle tables. That would unlock automated analysis over the troves of valuable tabular data out there.
TLDR: Training models on a large and diverse dataset of synthesized table tasks significantly boosts their table skills.
Can you opensource your training datasets of the different table tasks?
Not sure about that but here's a table dataset that is more than 800 billion tokens!
https://huggingface.co/datasets/approximatelabs/tablib-v1-full
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Tasks (2023)
- LLM-augmented Preference Learning from Natural Language (2023)
- Efficient Finetuning Large Language Models For Vietnamese Chatbot (2023)
- A Systematic Evaluation of Large Language Models on Out-of-Distribution Logical Reasoning Tasks (2023)
- Training Generative Question-Answering on Synthetic Data Obtained from an Instruct-tuned Model (2023)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
I am still not very sure with GPT capabilities of tackling arithmetical calculation in any context let it be tables or time series data. Till now the output that I have seen is not even close.
Key conclusion from my review of v1 of this paper (published on 13th Oct 2023):
This paper offers good introduction to simple table-tuning tasks, however task T-3 (TQA) should be significantly improved before Table-GPT can be used commercially.
Key points:
• Generic overview of results indicates very good results of table-tuning, however in my opinion tasks should be divided based on complexity to better understand value of table-tuning. Please see diagram below.
• For the most of easy tasks (all tasks except T-3) table-tuning offers great improvements in zero-shot comparing to vanilla models (209% improvement for GPT-3.5 and 119% for ChatGPT)
• For the most of easy tasks (all tasks except T-3) table-tuning offers good results in few-shot comparing to vanilla models (22% improvement for GPT-3.5 and 12% for ChatGPT)
• T-3 TQA is the most complex task (and with biggest business demand) and for this tasks table-tuning offers very small improvements (1-2% for ChatGPT and 5-8% for GPT-3.5), which is probably not worth of the fine-tuning effort
Open questions:
• Do you have plans to fine-tune GPT-4?
• Can you share recommendations on improving T-3 (TQA)? Maybe including TQA tasks in training?
• Can you include as well T-12 (NS) in tests?
• Can you specify number of tokens used (both for training and test execution) for each task
Other remarks:
• Markdown format increases performance of table-tuning by 3% comparing to CSV and by 5% comparing to JSON (table 5)
• For most of the tasks few-shot offers strong improvement over zero-shot for vanilla GPT 3.5 and ChatGPT (even without table-tuning).
• Typos found in paper:
- p.4 is “toke-by-token” should be “token-by-token”
- p.6 is “few select table-tasks” should be “few selected table-tasks”
- p.7 is “describes the row-augmentation task” should be “describes the column-augmentation task”
Revolutionizing AI: Table-GPT Enhances Language Models for Complex Table Tasks!
Links đź”—:
👉 Subscribe: https://www.youtube.com/@Arxflix
👉 Twitter: https://x.com/arxflix
👉 LMNT (Partner): https://lmnt.com/
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper