https://mbzuai-oryx.github.io/GeoPixel/
Code: https://github.com/mbzuai-oryx/GeoPixel

\n","updatedAt":"2025-01-27T13:27:33.075Z","author":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","fullname":"Niels Rogge","name":"nielsr","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":1096,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.4348635971546173},"editors":["nielsr"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg"],"reactions":[],"isReport":false}},{"id":"679833f8b7d310396fe4c227","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false},"createdAt":"2025-01-28T01:33:44.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing](https://huggingface.co/papers/2501.06828) (2025)\n* [RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts](https://huggingface.co/papers/2412.05679) (2024)\n* [PRIMA: Multi-Image Vision-Language Models for Reasoning Segmentation](https://huggingface.co/papers/2412.15209) (2024)\n* [SeaMo: A Multi-Seasonal and Multimodal Remote Sensing Foundation Model](https://huggingface.co/papers/2412.19237) (2024)\n* [Knowledge Transfer and Domain Adaptation for Fine-Grained Remote Sensing Image Segmentation](https://huggingface.co/papers/2412.06664) (2024)\n* [RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models](https://huggingface.co/papers/2501.06809) (2025)\n* [CC-Diff: Enhancing Contextual Coherence in Remote Sensing Image Synthesis](https://huggingface.co/papers/2412.08464) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-01-28T01:33:44.855Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6960110664367676},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2501.13925","authors":[{"_id":"679789af4a10be7109a28675","name":"Akashah Shabbir","hidden":false},{"_id":"679789af4a10be7109a28676","name":"Mohammed Zumri","hidden":false},{"_id":"679789af4a10be7109a28677","name":"Mohammed Bennamoun","hidden":false},{"_id":"679789af4a10be7109a28678","name":"Fahad S. Khan","hidden":false},{"_id":"679789af4a10be7109a28679","name":"Salman Khan","hidden":false}],"publishedAt":"2025-01-23T18:59:30.000Z","submittedOnDailyAt":"2025-01-27T10:57:33.004Z","title":"GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing","submittedOnDailyBy":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","isPro":false,"fullname":"Niels Rogge","user":"nielsr","type":"user"},"summary":"Recent advances in large multimodal models (LMMs) have recognized\nfine-grained grounding as an imperative factor of visual understanding and\ndialogue. However, the benefits of such representation in LMMs are limited to\nthe natural image domain, and these models perform poorly for remote sensing\n(RS). The distinct overhead viewpoint, scale variation, and presence of small\nobjects in high-resolution RS imagery present a unique challenge in\nregion-level comprehension. Moreover, the development of the grounding\nconversation capability of LMMs within RS is hindered by the lack of granular,\nRS domain-specific grounded data. Addressing these limitations, we propose\nGeoPixel - the first end-to-end high resolution RS-LMM that supports\npixel-level grounding. This capability allows fine-grained visual perception by\ngenerating interleaved masks in conversation. GeoPixel supports up to 4K HD\nresolution in any aspect ratio, ideal for high-precision RS image analysis. To\nsupport the grounded conversation generation (GCG) in RS imagery, we curate a\nvisually grounded dataset GeoPixelD through a semi-automated pipeline that\nutilizes set-of-marks prompting and spatial priors tailored for RS data to\nmethodically control the data generation process. GeoPixel demonstrates\nsuperior performance in pixel-level comprehension, surpassing existing LMMs in\nboth single-target and multi-target segmentation tasks. Our methodological\nablation studies validate the effectiveness of each component in the overall\narchitecture. Our code and data will be publicly released.","upvotes":8,"discussionId":"679789b44a10be7109a28733","githubRepo":"https://github.com/mbzuai-oryx/geopixel","githubRepoAddedBy":"auto","ai_summary":"GeoPixel, an end-to-end RS-LMM, achieves superior pixel-level comprehension in remote sensing images through fine-grained grounding and interleaved mask generation in conversations.","ai_keywords":["large multimodal models","fine-grained grounding","visual understanding","dialogue","remote sensing","region-level comprehension","grounded conversation capability","high-resolution RS imagery","pixel-level grounding","interleaved masks","high-precision RS image analysis","GeoPixel","GeoPixelD","set-of-marks prompting","spatial priors","superior performance","single-target segmentation","multi-target segmentation"],"githubStars":142},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6787e48a9e6186bec4112717","avatarUrl":"/avatars/dce578575abbb30cb8e6cfaf4d7875a0.svg","isPro":false,"fullname":"Haruka","user":"Harukauu","type":"user"},{"_id":"641b754d1911d3be6745cce9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/641b754d1911d3be6745cce9/Ydjcjd4VuNUGj5Cd4QHdB.png","isPro":false,"fullname":"atayloraerospace","user":"Taylor658","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"6342796a0875f2c99cfd313b","avatarUrl":"/avatars/98575092404c4197b20c929a6499a015.svg","isPro":false,"fullname":"Yuseung \"Phillip\" Lee","user":"phillipinseoul","type":"user"},{"_id":"6798fe5985310ecdffc5f530","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6798fe5985310ecdffc5f530/9M7u5mVwhmhBdJ6-eUV8g.jpeg","isPro":false,"fullname":"LuoHe","user":"SuKIIII2","type":"user"},{"_id":"67a9b5ea2c8e57acb591c9b1","avatarUrl":"/avatars/390de54355906e93bcdb5d42356b1a3a.svg","isPro":false,"fullname":"Shabbir","user":"AkashahS","type":"user"},{"_id":"663ccbff3a74a20189d4aa2e","avatarUrl":"/avatars/83a54455e0157480f65c498cd9057cf2.svg","isPro":false,"fullname":"Nguyen Van Thanh","user":"NguyenVanThanhHust","type":"user"},{"_id":"66d559103c5bc37ee0dfa61b","avatarUrl":"/avatars/8310fc0b01e6d8873aec37ba9ef27c5b.svg","isPro":false,"fullname":"SaraG","user":"SLMLAH","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2501.13925

GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing

Published on Jan 23, 2025

· Submitted by

Niels Rogge on Jan 27, 2025

Upvote

Authors:

Abstract

GeoPixel, an end-to-end RS-LMM, achieves superior pixel-level comprehension in remote sensing images through fine-grained grounding and interleaved mask generation in conversations.

AI-generated summary

Recent advances in large multimodal models (LMMs) have recognized fine-grained grounding as an imperative factor of visual understanding and dialogue. However, the benefits of such representation in LMMs are limited to the natural image domain, and these models perform poorly for remote sensing (RS). The distinct overhead viewpoint, scale variation, and presence of small objects in high-resolution RS imagery present a unique challenge in region-level comprehension. Moreover, the development of the grounding conversation capability of LMMs within RS is hindered by the lack of granular, RS domain-specific grounded data. Addressing these limitations, we propose GeoPixel - the first end-to-end high resolution RS-LMM that supports pixel-level grounding. This capability allows fine-grained visual perception by generating interleaved masks in conversation. GeoPixel supports up to 4K HD resolution in any aspect ratio, ideal for high-precision RS image analysis. To support the grounded conversation generation (GCG) in RS imagery, we curate a visually grounded dataset GeoPixelD through a semi-automated pipeline that utilizes set-of-marks prompting and spatial priors tailored for RS data to methodically control the data generation process. GeoPixel demonstrates superior performance in pixel-level comprehension, surpassing existing LMMs in both single-target and multi-target segmentation tasks. Our methodological ablation studies validate the effectiveness of each component in the overall architecture. Our code and data will be publicly released.