{"title": "A2-MAE: A Spatial-temporal-spectral Unified Remote Sensing Pre-training Method Based on Anchor-aware Masked Autoencoder", "year": 2024, "model_name": "A2-MAE", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder masked autoencoder", "params_millions": null, "notes": "Built on the MAE framework with a unified single backbone. Adds an Anchor-Aware Masking (AAM) strategy and a Geographic Encoding Module (GEM). Uses patch size 16x16 and selects three images per structured image set during pretraining."}, "pretraining": {"method": "MAE", "method_details": "A unified masked autoencoding approach for remote sensing that jointly exploits spatial, temporal, and spectral complementarity across multi-source image sets. The key novelties are anchor-aware masking, which adapts masking based on a selected anchor image to avoid feature leakage across sources/times, and a geographic encoding module that injects latitude, longitude, and GSD priors.", "objective": "masked patch reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["STSSD"], "num_images": "2.5M images in 510K image sets", "sensors": ["Sentinel-2", "Landsat-8", "Gaofen-1", "Gaofen-2"], "modalities": ["multispectral", "temporal", "geolocation metadata"], "spatial_resolution": "0.8m-30m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "multi-label classification", "semantic segmentation", "change detection"], "downstream_datasets": ["EuroSAT", "BigEarthNet", "Sen1Floods11", "CropSeg", "LEVIR-CD", "OSCD", "DynamicEarthNet"], "code_available": true, "code_url": null, "weights_available": true, "key_contribution": "The paper introduces STSSD, a large global spatial-temporal-spectral structured remote sensing pretraining dataset, and A2-MAE, a unified MAE-based model that can learn from diverse multi-source remote sensing imagery within a single backbone. Its anchor-aware masking and geographic encoding improve representation learning and generalization across classification, segmentation, and change detection tasks.", "_source_pdf": "A2-MAE_A_spatial-temporal-spectral_unified_remote_sensing_pre-training_method_based_on_anchor-aware_masked_autoencoder.pdf", "_extraction_timestamp": "2026-03-20T20:42:45"}
{"title": "A SENTINEL-3 FOUNDATION MODEL FOR OCEAN COLOUR", "year": 2025, "model_name": "Sentinel-3 ocean colour foundation model", "architecture": {"backbone": "Vision Transformer (Prithvi-EO-style ViT)", "type": "encoder-decoder", "params_millions": 50, "notes": "Based on the Prithvi-EO architecture and pretrained with a masked autoencoder. Uses 1×42×42 inputs with 1×2×2 patch size for Sentinel-3 OLCI/SLSTR data. Fine-tuning uses the pretrained encoder plus a U-Net-based decoder and task-specific pixel-wise regression head. The paper mentions 50 million parameters in the methods, though the conclusion appears to contain a likely typo ('50,000 parameters')."}, "pretraining": {"method": "MAE", "method_details": "Self-supervised masked autoencoder pretraining adapted from Prithvi-EO for ocean colour remote sensing. The model reconstructs masked Sentinel-3 OLCI reflectance bands, optionally with SLSTR sea surface temperature, using smaller image sizes and finer patches suited to 300 m ocean data and non-temporal single-image inputs.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-3 A and B L2 OLCI full resolution water reflectances", "Sentinel-3 L2 SLSTR sea surface temperature"], "num_images": "520000", "sensors": ["Sentinel-3 OLCI", "Sentinel-3 SLSTR"], "modalities": ["multispectral", "sea surface temperature"], "spatial_resolution": "300m", "geographic_coverage": "global ocean coverage sampled across Longhurst regions, excluding polar regions (APLR, ANTA) and inland water (LAKE)"}, "downstream_tasks": ["chlorophyll-a concentration quantification", "ocean primary production quantification"], "downstream_datasets": ["A compilation of global bio-optical in situ data for ocean-colour satellite applications - version 3", "Global marine phytoplankton production dataset", "Bermuda Atlantic Time-Series Study (BATS) primary production", "A database of ocean primary productivity from the 14C method", "Combined constraints on global ocean primary production using observations and models", "California Current Ecosystem LTER primary production estimates from 14C uptake"], "code_available": false, "code_url": null, "weights_available": true, "key_contribution": "The paper introduces the first Vision Transformer-based foundation model for ocean colour remote sensing using Sentinel-3 OLCI/SLSTR data, adapting the Prithvi-EO masked autoencoder framework to 300 m marine observations. It shows that self-supervised pretraining improves data efficiency and yields more realistic spatial patterns for downstream chlorophyll-a and primary production estimation, especially under very limited labeled data.", "_source_pdf": "A_Sentinel-3_foundation_model_for_ocean_colour.pdf", "_extraction_timestamp": "2026-03-20T20:42:58"}
{"title": "AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities", "year": 2024, "model_name": "AnySat", "architecture": {"backbone": "Vision Transformer-style transformer with scale-adaptive patch encoder and cross-attention modality combiner", "type": "encoder-decoder", "params_millions": 125, "notes": "Main model has 125M parameters. Architecture includes modality-specific MLP projectors (33M total for 11 projectors), a shared spatial transformer patch encoder (45M), a modality combiner with self-attention and cross-attention blocks (49M), and a JEPA student-side predictor transformer (29M). Uses scale-adaptive sub-patch encoding, LTAE for temporal modalities, and a teacher-student EMA setup during pretraining. More than 75% of learnable parameters are shared across modalities and resolutions."}, "pretraining": {"method": "JEPA with contrastive multimodal alignment", "method_details": "The paper introduces a multimodal JEPA for Earth observation that predicts teacher multimodal patch embeddings for dropped spatial regions instead of reconstructing pixels. It combines this with a contrastive loss enforcing cross-modal consistency for spatially aligned patches, plus scale-adaptive patch encoders so one model can ingest heterogeneous resolutions, scales, and sensor combinations without rescaling.", "objective": "latent feature prediction of dropped patches plus contrastive alignment across modalities for the same spatial patch", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["GeoPlex", "TreeSatAI-TS", "FLAIR", "PLANTED", "S2NAIP-URBAN", "PASTIS-HD"], "num_images": "about 1.9M+ samples/tiles total (including 1.3M PLANTED, 515k S2NAIP-URBAN, 78k FLAIR, 50k TreeSatAI-TS, 2.4k PASTIS-HD)", "sensors": ["Aerial", "NAIP", "SPOT6/7", "Sentinel-1", "Sentinel-2", "ALOS-2", "Landsat-7", "Landsat-8/9", "MODIS"], "modalities": ["RGB", "NIR", "elevation", "multispectral", "SAR", "temporal"], "spatial_resolution": "0.2m to 250m", "geographic_coverage": "global"}, "downstream_tasks": ["land cover mapping", "tree species identification", "crop type classification", "change detection", "climate type classification", "flood segmentation", "burn scar segmentation", "deforestation segmentation", "semantic segmentation", "classification"], "downstream_datasets": ["TreeSatAI-TS", "PASTIS-HD", "FLAIR", "PLANTED", "SICKLE", "BraDD-S1TS", "TimeSen2Crop", "Sen1Floods11", "So2Sat", "HLS Burn Scar"], "code_available": true, "code_url": "https://github.com/gastruc/AnySat", "weights_available": true, "key_contribution": "The paper proposes AnySat, a single self-supervised Earth observation model that can handle heterogeneous resolutions, scales, and sensor modalities using a scale-adaptive transformer architecture and a multimodal JEPA objective. It also introduces GeoPlex, a curated multimodal pretraining collection, and shows that one pretrained model achieves state-of-the-art or near state-of-the-art performance across diverse EO tasks and unseen sensor configurations.", "_source_pdf": "AnySat_An_Earth_Observation_Model_for_Any_Resolutions_Scales_and_Modalities.pdf", "_extraction_timestamp": "2026-03-20T20:43:12"}
{"title": "Bridging Remote Sensors with Multisensor Geospatial Foundation Models", "year": 2024, "model_name": "msGFM", "architecture": {"backbone": "Swin-Base", "type": "encoder-decoder", "params_millions": null, "notes": "Built on SIMMIM with a shared Swin Transformer encoder across sensor modalities, separate trainable patch embedding layers for each sensor, and separate decoders per sensor. Uses sparsely gated Mixture-of-Experts to replace MLP layers in the encoder; experiments use patch size 16x16 and 8 experts."}, "pretraining": {"method": "masked image modeling with cross-sensor reconstruction", "method_details": "The model supports both paired and unpaired multisensor data. For paired samples from the same geolocation, it reconstructs either the same sensor or the corresponding paired sensor, enabling joint cross-sensor representation learning. For unpaired samples, it falls back to standard masked self-reconstruction. It also uses MoE within the encoder to better handle multisensor heterogeneity.", "objective": "masked image reconstruction / cross-sensor masked reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["GeoPile", "MillionAID", "SEN12MS", "MDAS"], "num_images": "2M", "sensors": ["Sentinel-1", "Sentinel-2", "SAR", "DSM", "RGB", "NAIP", "GeoEye", "WorldView", "QuickBird", "IKONOS", "SPOT", "airborne"], "modalities": ["RGB", "multispectral", "SAR", "DSM"], "spatial_resolution": "mixed (0.1m-153m)", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "semantic segmentation", "cloud removal", "pan-sharpening"], "downstream_datasets": ["BigEarthNet", "Vaihingen", "SEN12MS-CR", "SpaceNet2"], "code_available": true, "code_url": "https://github.com/boranhan/Geospatial_Foundation_Models", "weights_available": true, "key_contribution": "The paper proposes msGFM, a multisensor geospatial foundation model that unifies four remote sensing modalities through a shared encoder and cross-sensor masked image modeling. Its main novelty is learning from both paired and unpaired multisensor data, enabling stronger joint representations and improved performance across multiple single-sensor and multisensor downstream tasks.", "_source_pdf": "Bridging_Remote_Sensors_with_Multisensor_Geospatial_Foundation_Models.pdf", "_extraction_timestamp": "2026-03-20T20:43:23"}
{"title": "Change-Aware Sampling and Contrastive Learning for Satellite Images", "year": 2023, "model_name": "CACo", "architecture": {"backbone": "ResNet-50", "type": "encoder-only", "params_millions": null, "notes": "Built on a contrastive learning framework using MoCo v2; experiments use ResNet-18 and ResNet-50 backbones, with ResNet-50 as the primary/larger CNN variant. The method modifies SeCo by adding long-term temporal negatives and a change-aware conditional contrastive loss. Also shown to generalize to SimCLR."}, "pretraining": {"method": "contrastive learning (MoCo v2) with change-aware contrastive loss", "method_details": "The proposed Change-Aware Contrastive (CACo) loss leverages both short-term and long-term temporal pairs from the same location. Short-term temporal differences are treated as seasonal augmentations, while long-term pairs are pushed apart only when an estimated meaningful change has occurred. Change is estimated during training by comparing long-term feature differences to short-term seasonal variation using a ratio, smoothed over epochs and clustered with a GMM. The paper also introduces improved geography-aware sampling near cities and rejects all-ocean samples.", "objective": "contrastive instance discrimination with seasonal invariance and conditional sensitivity to long-term change", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom Sentinel-2 pretraining dataset"], "num_images": "100k and 1M", "sensors": ["Sentinel-2"], "modalities": ["RGB", "temporal"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["landcover classification", "multi-label land-cover classification", "change detection", "semantic segmentation", "change event retrieval"], "downstream_datasets": ["EuroSat", "BigEarthNet", "OSCD", "DynamicEarthNet", "CaiRoad", "CalFire"], "code_available": null, "code_url": "https://github.com/utkarshmall13/CACo", "weights_available": true, "key_contribution": "The paper proposes CACo, a self-supervised contrastive learning method for satellite imagery that uses long-term temporal information to distinguish meaningful permanent change from short-term seasonal variation. It also introduces change-aware conditional negatives and improved geographic sampling, yielding stronger representations across classification, segmentation, change detection, and change-event retrieval.", "_source_pdf": "CACo_Change-Aware_Sampling_and_Contrastive_Learning_for_Satellite_Images.pdf", "_extraction_timestamp": "2026-03-20T20:43:32"}
{"title": "CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders", "year": 2023, "model_name": "CROMA", "architecture": {"backbone": "ViT-L", "type": "multimodal encoder-decoder with unimodal encoders", "params_millions": null, "notes": "Primary model uses three transformers: a unimodal radar ViT encoder, a unimodal optical ViT encoder, and a multimodal radar-optical transformer encoder with self-attention and cross-attention. Largest variant is CROMA-L with ViT-L backbone; optical encoder has 24 layers, radar encoder and multimodal encoder each use half-depth. Uses 8x8 patches, default 120x120 inputs, and a lightweight 1-layer 512-d transformer decoder during pretraining only. Introduces 2D-ALiBi for self-attention and X-ALiBi for cross-attention instead of standard positional embeddings."}, "pretraining": {"method": "hybrid contrastive + masked autoencoding", "method_details": "CROMA jointly combines radar-optical cross-modal InfoNCE contrastive learning on masked unimodal representations with multimodal masked autoencoding that reconstructs masked radar and optical patches from fused representations. The novelty is designing a CoCa-inspired but remote-sensing-specific framework for spatially aligned Sentinel-1/Sentinel-2 data, plus new 2D-ALiBi and X-ALiBi positional biasing for self- and cross-attention.", "objective": "cross-modal radar↔optical contrastive alignment and masked patch reconstruction of radar and optical data", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO"], "num_images": "1M", "sensors": ["Sentinel-1 GRD", "Sentinel-2 L2A"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["scene/classification", "multi-label classification", "semantic segmentation", "kNN classification", "K-means clustering"], "downstream_datasets": ["BigEarthNet", "fMoW-Sentinel", "EuroSAT", "Canadian Cropland", "DFC2020", "DW-Expert", "MARIDA"], "code_available": true, "code_url": "https://github.com/antofuller/CROMA", "weights_available": true, "key_contribution": "The paper proposes CROMA, a self-supervised multimodal remote sensing foundation model that jointly learns from aligned Sentinel-1 SAR and Sentinel-2 multispectral imagery using complementary contrastive and masked autoencoding objectives. It also introduces 2D-ALiBi and X-ALiBi positional biases, improving multimodal fusion and enabling strong extrapolation to much larger test-time image sizes.", "_source_pdf": "CROMA_Remote_Sensing_Representations_with_Contrastive_Radar-Optical_Masked_Autoencoders.pdf", "_extraction_timestamp": "2026-03-20T20:43:43"}
{"title": "CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations", "year": 2023, "model_name": "CSP", "architecture": {"backbone": "Dual-encoder with Space2Vec grid location encoder and image encoder based on InceptionV3 (iNat2018) or ResNet-50 (fMoW); primary image backbone for remote sensing experiments is Geo-SSL pretrained ResNet-50", "type": "CLIP-style dual-encoder", "params_millions": 42.2, "notes": "Separate location encoder e() and image encoder f(). The image encoder uses an off-the-shelf backbone plus a projection MLP; the location encoder is primarily Space2Vec grid. The paper states InceptionV3 has 41.8M trainable parameters and the Space2Vec location encoder has 0.4M, for about 42.2M total in that setup. During CSP pretraining, the image backbone is locked and only the projection layer is trained alongside the location encoder."}, "pretraining": {"method": "contrastive", "method_details": "CSP aligns geo-locations and images using a dual-encoder and three contrastive pair construction strategies: in-batch negative sampling, random negative location sampling, and SimCSE-style dropout-based positive/negative sampling for locations. It freezes a pretrained image encoder and trains the location encoder (plus image projection head) to match image embeddings.", "objective": "contrastive alignment between location embeddings and image embeddings", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["iNat2018", "fMoW"], "num_images": "approximately 436K geo-tagged training images for iNat2018 and 363,570 location-image pairs for fMoW", "sensors": ["ground-level photography", "satellite"], "modalities": ["image", "geolocation"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["fine-grained species recognition", "satellite image classification"], "downstream_datasets": ["iNat2018", "fMoW"], "code_available": true, "code_url": "https://gengchenmai.github.io/csp-website/", "weights_available": true, "key_contribution": "The paper proposes CSP, a self-supervised contrastive spatial pretraining framework that explicitly aligns geo-locations with images using a dual-encoder, enabling geospatial information to be used during pretraining, finetuning, and inference. It shows that learned location representations improve few-shot and fully supervised classification on iNat2018 and fMoW.", "_source_pdf": "CSP_Self-Supervised_Contrastive_Spatial_Pre-Training_for_Geospatial-Visual_Representations.pdf", "_extraction_timestamp": "2026-03-20T20:43:56"}
{"title": "Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model", "year": 2024, "model_name": "Changen2", "architecture": {"backbone": "RS-DiT (resolution-scalable diffusion transformer), based on DiT-B/2", "type": "conditional latent diffusion transformer / encoder-decoder generative model", "params_millions": null, "notes": "Changen2 implements the Generative Probabilistic Change Model (GPCM) with a latent diffusion model. Its noise predictor is RS-DiT, which removes absolute positional embeddings, adds 3x3 depthwise convolutions in FFN blocks, uses mostly local window attention instead of global self-attention, and includes a dense embedding network for dense conditional inputs. It uses a frozen VAE encoder/decoder and masked change diffusion at inference to preserve unchanged regions while generating changed content."}, "pretraining": {"method": "latent diffusion with self-supervised synthetic change data generation", "method_details": "The paper proposes generative change modeling via GPCM, factorizing change generation into condition-level change event simulation and image-level semantic change synthesis. Changen2 can be trained on labeled single-temporal images, or self-supervised on unlabeled single-temporal images using SAM-derived object contours as editable conditions; it then synthesizes multi-temporal image pairs and change labels to pre-train task-specific change detection models.", "objective": "conditional denoising / masked image generation for semantic change synthesis; downstream synthetic change supervision for change detection", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["xView2 pre-disaster", "OpenEarthMap", "fMoW"], "num_images": "about 1.2M unlabeled fMoW crops for self-supervised training; synthetic datasets generated include Changen2-S1-15k, Changen2-S9-27k, and Changen2-S0-1.2M", "sensors": ["satellite"], "modalities": ["optical imagery", "RGB", "semantic masks", "object contours"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["object change detection", "semantic change detection", "building change detection", "off-nadir building change detection", "building damage assessment", "zero-shot change detection"], "downstream_datasets": ["LEVIR-CD", "WHU-CD", "S2Looking", "SECOND", "xView2"], "code_available": null, "code_url": "https://huggingface.co/EVER-Z", "weights_available": true, "key_contribution": "The paper introduces Changen2, a generative change foundation model that synthesizes multi-temporal remote sensing images and change labels from single-temporal data using a resolution-scalable diffusion transformer and a probabilistic change model. It enables scalable self-supervised generation of synthetic change supervision from unlabeled imagery, leading to strong transferability and zero-shot change detection performance across multiple real-world benchmarks.", "_source_pdf": "Changen2_Multi-Temporal_Remote_Sensing_Generative_Change_Foundation_Model.pdf", "_extraction_timestamp": "2026-03-20T20:44:10"}
{"title": "Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensing", "year": 2023, "model_name": "Cross-Scale MAE", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": null, "notes": "Built on Masked Autoencoder (MAE). The encoder uses a ViT backbone (paper describes ViT-Base with 12 self-attention blocks in the method figure, but experiments state pre-training uses ViT-Large unless otherwise noted). The decoder is a lightweight ViT with 8 self-attention blocks. Uses shared-weight multi-branch processing of scale-augmented views, a projection head for contrastive learning, and an MLP predictor for cross-scale decoder prediction."}, "pretraining": {"method": "MAE + contrastive + cross-scale generative prediction", "method_details": "Introduces scale augmentation to synthesize multiple scales from a single image without requiring aligned multi-scale imagery, and enforces cross-scale consistency at both encoder and decoder levels. Encoder uses InfoNCE contrastive loss between different-scale views; decoder combines masked reconstruction with cross-scale prediction of decoder features.", "objective": "masked image reconstruction, cross-scale contrastive alignment, and cross-scale decoder feature prediction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Functional Map of the World (fMoW) RGB training set"], "num_images": "363.6k", "sensors": null, "modalities": ["RGB"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation", "KNN classification evaluation"], "downstream_datasets": ["RESISC45", "WHU-RS19", "UC Merced", "EuroSAT", "fMoW-RGB", "Potsdam", "Vaihingen"], "code_available": null, "code_url": "https://github.com/aicip/Cross-Scale-MAE", "weights_available": true, "key_contribution": "The paper proposes Cross-Scale MAE, a self-supervised remote sensing representation learning framework that learns scale-consistent features without requiring aligned multi-scale image pairs. It combines scale augmentation, encoder-level contrastive consistency, and decoder-level cross-scale prediction with masked reconstruction, yielding stronger multi-scale representations than SatMAE and Scale-MAE on classification and segmentation benchmarks.", "_source_pdf": "Cross-Scale_MAE_A_Tale_of_Multiscale_Exploitation_in_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:44:19"}
{"title": "Cross-sensor self-supervised training and alignment for remote sensing", "year": 2024, "model_name": "X-STARS", "architecture": {"backbone": "ViT-L/16", "type": "encoder-only", "params_millions": null, "notes": "Primary experiments use a Vision Transformer with DINO-style student-teacher training; also evaluated with ResNet-50 and smaller ViT variants. Uses sensor-specific linear projection heads for the MSAD alignment loss. In continual pretraining, a frozen pretrained domain teacher backbone is used for cross-sensor distillation."}, "pretraining": {"method": "contrastive self-supervised learning with cross-sensor alignment", "method_details": "X-STARS combines an off-the-shelf SSL objective (DINO in this paper) with a new Multi-Sensor Alignment Dense loss (MSAD). MSAD aligns representations from paired images across sensors using patch-wise contrastive matching and knowledge distillation, with label smoothing to handle imperfect temporal/geographic correspondence and differing resolutions.", "objective": "DINO self-distillation plus patch-wise cross-sensor contrastive alignment of paired acquisitions", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MSC-France", "Multi-View Ile-de-France"], "num_images": "4496 triplets in MSC-France; 45,200 pairs in Multi-View Ile-de-France", "sensors": ["Sentinel-2", "Landsat-8", "SPOT-6", "BDORTHO"], "modalities": ["RGB", "multi-sensor", "paired cross-resolution imagery"], "spatial_resolution": "mixed (0.2 m, 1.5 m, 10 m, 30 m)", "geographic_coverage": "France; MSC-France covers 12 major French cities and suburbs, Multi-View Ile-de-France covers Ile-de-France"}, "downstream_tasks": ["scene classification", "semantic segmentation"], "downstream_datasets": ["EuroSAT", "Worldstrat", "UC-Merced", "CV-BrCT", "SpaceNet8", "Chesapeake Land Cover", "FLAIR"], "code_available": false, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes X-STARS, a self-supervised cross-sensor pretraining framework for remote sensing that learns sensor-agnostic representations by combining DINO with a new patch-wise cross-sensor alignment loss, MSAD. It also introduces the MSC-France dataset and shows that X-STARS can outperform prior SSL baselines while using substantially less data and can adapt pretrained models to new sensors via continual pretraining.", "_source_pdf": "Cross-Sensor_Self-Supervised_Training_and_Alignment_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:44:28"}
{"title": "CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentation", "year": 2024, "model_name": "CrossEarth", "architecture": {"backbone": "ViT-L (DINOv2)", "type": "encoder-decoder multi-task model", "params_millions": null, "notes": "Uses a frozen shared DINOv2 backbone with two training branches: a semantic segmentation branch and a masked image modeling branch. The segmentation branch includes a Geospatial Semantic Extractor (5-layer CNN), cross-attention Injectors, and a Mask2Former decoder. The MIM branch uses an ASPP decoder over multi-stage backbone features. Also includes an Earth-Style Injection augmentation pipeline with a frozen Style Predictor and Style Transfer module."}, "pretraining": {"method": "multi-task masked image modeling with style-based augmentation", "method_details": "CrossEarth is built on a pretrained DINOv2 backbone and trained for domain generalization using two key ideas: Earth-Style Injection, which injects remote-sensing style embeddings from Million-AID into training images to broaden domain coverage, and Multi-Task Training, which jointly optimizes semantic segmentation and masked image modeling to learn more domain-invariant geospatial features.", "objective": "semantic segmentation plus masked image reconstruction", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["Million-AID"], "num_images": "over 1M", "sensors": ["mixed remote sensing imagery"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["semantic segmentation", "land cover classification", "disaster assessment", "road extraction", "building extraction", "building detection"], "downstream_datasets": ["ISPRS Potsdam", "ISPRS Vaihingen", "LoveDA", "DeepGlobe", "Massachusetts", "RescueNet", "OpenEarthMap", "GlobalRoadNet", "WHU Building", "CASID"], "code_available": true, "code_url": "https://github.com/VisionXLab/CrossEarth", "weights_available": true, "key_contribution": "The paper introduces CrossEarth, the first vision foundation model tailored to remote sensing domain generalization for semantic segmentation. Its main contribution is combining Earth-Style Injection and multi-task segmentation+MIM training to improve cross-domain generalization across region, spectral band, platform, and climate shifts, together with a new 32-scenario RSDG benchmark.", "_source_pdf": "CrossEarth_Geospatial_Vision_Foundation_Model_for_Domain_Generalizable_Remote_Sensing_Semantic_Segmentation.pdf", "_extraction_timestamp": "2026-03-20T20:44:40"}
{"title": "CtxMIM: Context-Enhanced Masked Image Modeling for Remote Sensing Image Understanding", "year": 2023, "model_name": "CtxMIM", "architecture": {"backbone": "Swin-B", "type": "encoder-decoder Siamese masked image modeling", "params_millions": null, "notes": "Uses an asymmetric Siamese framework with a reconstructive branch and a context-enhanced generative branch. The encoder is a Swin Transformer shared across both branches, and the decoder is a lightweight MLP prediction head following SimMIM-style reconstruction."}, "pretraining": {"method": "masked image modeling with context-enhanced Siamese reconstruction", "method_details": "CtxMIM treats original image patches as a reconstructive template and adds a context-enhanced generative branch that provides contextual information during reconstruction. A context consistency loss constrains the reconstructive branch to match predictions from the context-enhanced branch, aiming to mitigate missing contextual information caused by high object density in remote sensing imagery.", "objective": "masked image reconstruction with reconstruction loss, predictive reconstruction loss, and context consistency loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom large-scale unlabeled remote sensing dataset collected using Google Earth Engine"], "num_images": "1.28M", "sensors": ["WorldView-3"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": "Asia"}, "downstream_tasks": ["land cover classification", "semantic segmentation", "object detection", "instance segmentation"], "downstream_datasets": ["EuroSAT", "NWPU-RESISC45", "SpaceNet (Rio)", "DOTA", "SpaceNet (LasVegas)"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes CtxMIM, a context-enhanced masked image modeling framework tailored to remote sensing images, where high object density makes standard MIM less effective. By adding a Siamese context-enhanced generative branch and a context consistency constraint, the method learns more transferable object- and pixel-level representations and achieves state-of-the-art results across classification, segmentation, detection, and instance segmentation.", "_source_pdf": "CtxMIM_Context-Enhanced_Masked_Image_Modeling_for_Remote_Sensing_Image_Understanding.pdf", "_extraction_timestamp": "2026-03-20T20:44:50"}
{"title": "Extending global-local view alignment for self-supervised learning with remote sensing imagery", "year": 2023, "model_name": "DINO-MC", "architecture": {"backbone": "WRN-50-2", "type": "teacher-student encoder-only self-distillation", "params_millions": 69, "notes": "Built on DINO with student and teacher networks of the same backbone updated via EMA. The paper evaluates ViT-small, Swin-tiny, ResNet-50, and WRN-50-2; the primary/largest reported variant is WRN-50-2. DINO-MC modifies DINO by using multi-sized local crops instead of fixed-size local crops, with global views for the teacher and both global/local views for the student."}, "pretraining": {"method": "DINO-style self-distillation with global-local view alignment", "method_details": "The proposed DINO-MC extends DINO by introducing multi-sized local crops to better reflect limited object-scale variation in remote sensing imagery, plus different color transformations for global and local crops. The paper also studies DINO-TP using temporal positives, but DINO-MC is the main proposed method.", "objective": "alignment of global and local view representations via self-distillation / knowledge distillation", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SeCo-100K"], "num_images": "100K", "sensors": ["Sentinel-2"], "modalities": ["multispectral", "temporal"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["land use classification", "multi-label land cover classification", "change detection"], "downstream_datasets": ["EuroSAT", "BigEarthNet-S2", "OSCD"], "code_available": true, "code_url": "https://github.com/WennyXY/DINO-MC", "weights_available": true, "key_contribution": "The paper extends DINO for remote sensing self-supervised learning by proposing DINO-MC, which replaces fixed-size local crops with multi-sized local crops to create a more effective global-local alignment pretext task. Pretrained on only SeCo-100K, DINO-MC achieves competitive or state-of-the-art results on EuroSAT, BigEarthNet-S2, and OSCD while using less data and compute than prior SSL remote sensing methods.", "_source_pdf": "DINO-MC_Self-supervised_Contrastive_Learning_for_Remote_Sensing_Imagery_with_Multi-sized_Local_Crops.pdf", "_extraction_timestamp": "2026-03-20T20:44:59"}
{"title": "Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation", "year": 2024, "model_name": "DOFA", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": null, "notes": "Unified Transformer with a wavelength-conditioned dynamic hypernetwork that generates modality-specific patch embedding and decoder weights from band central wavelengths. Uses a shared ViT backbone across modalities and a dynamic decoder for masked image modeling. Primary reported variants are ViT-Base and ViT-Large; largest/primary variant is ViT-Large. DOFA+ initializes from DINOv2 and adds hierarchical feature distillation."}, "pretraining": {"method": "masked image modeling with continual pretraining and knowledge distillation", "method_details": "Novelty lies in wavelength-conditioned dynamic weight generation: a hypernetwork maps spectral-band wavelengths to patch embedding/decoder weights, allowing one model to flexibly handle varying numbers of bands and unseen sensors. Continual pretraining uses proxy RGB-based distillation from an ImageNet/DINOv2 teacher; DOFA+ combines MIM with multi-layer hierarchical distillation from DINOv2 for efficient EO adaptation.", "objective": "masked image reconstruction plus feature distillation/alignment", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SatlasPretrain Sentinel-1 subset", "SatlasPretrain Sentinel-2 subset", "NAIP subset from SatlasPretrain", "Gaofen dataset", "EnMAP data from HyspecNet"], "num_images": "11.5M for DOFA; 410K for DOFA+ compact pretraining", "sensors": ["Sentinel-1", "Sentinel-2", "NAIP", "Gaofen", "EnMAP"], "modalities": ["SAR", "RGB", "multispectral", "hyperspectral"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["image classification", "semantic segmentation", "object detection", "change detection"], "downstream_datasets": ["m-bigearthnet", "m-so2sat", "m-brick-kiln", "m-forestnet", "m-eurosat", "m-pv4ger", "m-pv4ger-seg", "m-chesapeake-landcover", "m-cashew-plantation", "m-SA-crop-type", "m-nz-cattle", "m-NeonTree", "BurnSr", "MADOS", "PASTIS", "Sen1Floods11", "DEN", "CTM-SS", "SN7", "AI4Farms", "RESISC-45", "DIOR"], "code_available": true, "code_url": "https://github.com/zhu-xlab/DOFA", "weights_available": true, "key_contribution": "The paper proposes DOFA, a unified multimodal EO foundation model that uses a wavelength-conditioned dynamic hypernetwork to adapt a single Transformer to heterogeneous sensors, spectral bands, and even unseen modalities. It also introduces DOFA+, a more compute-efficient variant that combines masked image modeling with hierarchical DINOv2 distillation, achieving strong cross-modal transfer and state-of-the-art results on diverse EO benchmarks.", "_source_pdf": "DOFA_Neural_Plasticity-Inspired_Foundation_Model_for_Observing_the_Earth_Crossing_Modalities.pdf", "_extraction_timestamp": "2026-03-20T20:45:13"}
{"title": "Decoupling Common and Unique Representations for Multimodal Self-supervised Learning", "year": 2023, "model_name": "DeCUR", "architecture": {"backbone": "ResNet-50", "type": "dual-encoder with modality-specific encoders and projectors", "params_millions": null, "notes": "Primary setup uses separate modality-specific ResNet-50 encoders plus 3-layer MLP projectors with output dimension 8192. Embedding dimensions are split into common and unique parts for cross-modal learning. Optional residual deformable attention is added to the last two stages of ConvNet backbones. Additional experiments also use MiT-B2/B5 and ViT-S/16."}, "pretraining": {"method": "redundancy reduction / Barlow Twins-style multimodal self-supervised learning", "method_details": "DeCUR extends Barlow Twins to multimodal learning by explicitly decoupling embedding dimensions into cross-modal common and modality-unique subspaces. It aligns common dimensions across modalities, decorrelates unique dimensions across modalities, and adds intra-modal redundancy reduction on full embeddings to prevent collapse and strengthen modality-specific learning.", "objective": "cross-modal common representation alignment, cross-modal unique representation decorrelation, and intra-modal redundancy reduction between augmented views", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12", "GeoNRW", "SUN-RGBD"], "num_images": "251k image pairs (SSL4EO-S12), 111k patches from 6,942 training scenes (GeoNRW), 10,335 RGB-D pairs (SUN-RGBD)", "sensors": ["Sentinel-1 GRD", "Sentinel-2 L1C", "aerial orthophoto", "lidar-derived digital elevation model", "RGB-D indoor camera"], "modalities": ["SAR", "multispectral", "RGB", "DEM", "depth"], "spatial_resolution": "10 m for SSL4EO-S12, 1 m for GeoNRW, mixed for SUN-RGBD", "geographic_coverage": "global for SSL4EO-S12; North Rhine-Westphalia, Germany for GeoNRW; indoor scenes for SUN-RGBD"}, "downstream_tasks": ["scene classification", "semantic segmentation"], "downstream_datasets": ["BigEarthNet-MM", "BigEarthNet-S2", "GeoNRW", "SUN-RGBD", "NYU-Depth v2"], "code_available": true, "code_url": "https://github.com/zhu-xlab/DeCUR", "weights_available": true, "key_contribution": "The paper proposes DeCUR, a multimodal self-supervised learning method that explicitly separates shared cross-modal information from modality-specific information in the embedding space. By combining cross-modal decoupling with intra-modal redundancy reduction, it improves multimodal and missing-modality transfer performance across SAR-optical, RGB-DEM, and RGB-depth settings.", "_source_pdf": "DeCUR_Decoupling_Common_and_Unique_Representations_for_Multimodal_Self-supervised_Learning.pdf", "_extraction_timestamp": "2026-03-20T20:45:22"}
{"title": "DeepAndes: A Self-Supervised Vision Foundation Model for Multi-Spectral Remote Sensing Imagery of the Andes", "year": 2025, "model_name": "DeepAndes", "architecture": {"backbone": "ViT-L/14", "type": "encoder-only teacher-student self-distillation transformer", "params_millions": 307, "notes": "DINOv2-based vision transformer adapted for 8-band multispectral imagery; student-teacher networks share ViT architecture with EMA teacher updates; input patch embedding modified to accept 8 channels via 2D convolution; 24 transformer layers, hidden size 1024, patch size 14, outputs 1 CLS token + 256 patch tokens."}, "pretraining": {"method": "DINOv2 with iBOT", "method_details": "Customized DINOv2 self-supervised pipeline for 8-band WorldView multispectral imagery, including adapted preprocessing/augmentation for multi-channel input and modified global-local crop scales ((0.2, 0.5) local and (0.5, 1) global) to better match sparse archaeological and architectural features.", "objective": "self-distillation contrastive image-level alignment via DINO loss plus masked patch-level prediction via iBOT loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom Central Andes pretraining dataset"], "num_images": "3M", "sensors": ["WorldView-2", "WorldView-3"], "modalities": ["multispectral"], "spatial_resolution": "0.31m to 0.46m native, processed to 0.31m", "geographic_coverage": "Central Andes / Peruvian Andes, including western Peru and parts of northern Chile and northwestern Bolivia"}, "downstream_tasks": ["imbalanced image classification", "image instance retrieval", "semantic segmentation"], "downstream_datasets": ["custom loci classification dataset", "Active Buildings dataset", "Active Corrals dataset", "Archaeological Corrals dataset"], "code_available": true, "code_url": "https://github.com/geopacha/DeepAndes", "weights_available": true, "key_contribution": "The paper introduces DeepAndes, the first vision foundation model specifically designed for 8-band multispectral remote sensing imagery in the Andes, pretrained self-supervised on 3 million WorldView image patches. By adapting DINOv2 to multispectral archaeological remote sensing, it shows strong few-shot gains on imbalanced classification, image retrieval, and semantic segmentation.", "_source_pdf": "DeepAndes_A_Self-Supervised_Vision_Foundation_Model_for_Multispectral_Remote_Sensing_Imagery_of_the_Andes.pdf", "_extraction_timestamp": "2026-03-20T20:45:40"}
{"title": "DiffusionSat: A Generative Foundation Model for Satellite Imagery", "year": 2023, "model_name": "DiffusionSat", "architecture": {"backbone": "Stable Diffusion 2.1 latent diffusion U-Net with CLIP text encoder and VAE; extended with a 3D ControlNet for conditional tasks", "type": "latent diffusion encoder-decoder", "params_millions": null, "notes": "Initialized from Stable Diffusion 2.1. Uses SD VAE encoder/decoder and CLIP text encoder. For single-image generation, only the denoising U-Net and metadata/timestep embeddings are updated. Numerical metadata (latitude, longitude, GSD, cloud cover, year, month, day) is embedded via sinusoidal projection plus per-field MLPs and combined with timestep embeddings. For downstream conditional generation, adds a novel 3D ControlNet with 3D zero convolutions, temporal transformer/attention layers, and per-image metadata conditioning for temporal sequences, super-resolution, and inpainting."}, "pretraining": {"method": "diffusion", "method_details": "Pretrains a latent diffusion model for satellite image generation conditioned on text and freely available numerical metadata rather than relying only on captions. The key novelty is metadata-aware conditioning and a subsequent 3D conditioning extension for inverse problems such as temporal prediction, multispectral super-resolution, and inpainting.", "objective": "denoising diffusion noise prediction for conditional image generation", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["fMoW", "Satlas", "SpaceNet"], "num_images": null, "sensors": ["DigitalGlobe", "NAIP", "Sentinel-2"], "modalities": ["RGB", "multispectral", "text", "metadata"], "spatial_resolution": "mixed (about 0.3m-1.5m for fMoW; Sentinel-2 10m-60m also used in the broader framework)", "geographic_coverage": "global"}, "downstream_tasks": ["single-image generation", "multispectral super-resolution", "temporal generation", "temporal prediction", "temporal inpainting", "image inpainting"], "downstream_datasets": ["fMoW", "fMoW-Sentinel-fMoW-RGB", "Texas Housing", "fMoW-temporal", "xBD"], "code_available": true, "code_url": "https://samar-khanna.github.io/DiffusionSat/", "weights_available": null, "key_contribution": "The paper introduces DiffusionSat, the first large-scale generative foundation model for satellite imagery built on latent diffusion and conditioned on both text and numerical satellite metadata. It also proposes a novel 3D ControlNet-style conditioning module that extends the pretrained model to remote-sensing inverse problems including temporal prediction, multispectral super-resolution, and disaster inpainting.", "_source_pdf": "DiffusionSat_A_Generative_Foundation_Model_for_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:45:51"}
{"title": "EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues", "year": 2024, "model_name": "EarthDial", "architecture": {"backbone": "InternViT-300M + Phi-3-mini", "type": "vision-language multimodal LLM with vision encoder + MLP projector + autoregressive decoder LLM", "params_millions": 4000, "notes": "Built on InternVL with modifications for Earth observation. Uses InternViT-300M as visual encoder, Phi-3-mini as LLM, and an MLP projector. Includes Adaptive High Resolution tiling for multi-resolution imagery and a Data Fusion module for multispectral, SAR, RGBI, hyperspectral, and multi-temporal inputs. Three-stage training progressively adds RGB, temporal, multispectral, and SAR capabilities."}, "pretraining": {"method": "autoregressive image-text instruction tuning / conversational pretraining", "method_details": "Stage 1 aligns remote sensing imagery and text by training the vision encoder, MLP projector, and LLM to generate text autoregressively from EO image-text pairs. Novelty lies in adapting a VLM to multi-resolution, multi-sensor EO data and later extending it through staged finetuning to temporal, multispectral, and SAR modalities with a dedicated data fusion module.", "objective": "autoregressive text generation from image-text instruction pairs", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["SatlasPretrain", "SkyScript", "NAIP", "Sentinel-2", "Landsat", "Sentinel-1"], "num_images": "7.6M image-text pairs for Stage 1 pretraining; 11.11M instruction pairs overall in EarthDial-Instruct", "sensors": ["Sentinel-2", "Sentinel-1", "Landsat 8", "NAIP", "aerial"], "modalities": ["RGB", "multispectral", "SAR", "NIR", "infrared", "hyperspectral", "temporal", "text"], "spatial_resolution": "mixed (e.g. 0.5m, 10m, 30m)", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "multi-label scene classification", "temporal scene classification", "object detection", "referred object detection", "visual grounding", "region captioning", "image captioning", "visual question answering", "change detection", "disaster assessment", "methane plume detection", "tree species classification", "urban heat island classification", "local climate zone classification", "earthquake prediction"], "downstream_datasets": ["AID", "UCMerced-LandUse", "WHU-RS19", "EuroSat", "BigEarthNet", "NWPU-RESISC45", "PatternNet", "RS-CD", "RSI-CD256", "fMoW", "FGSCR-42", "TreeSatAI-Time-Series", "SoSAT-LCZ42", "DOTA", "DIOR", "FAIR-1M", "HIT-UAV", "UCAS-AOD", "NWPU-VHR10", "SAR-Ship", "SRSDD-v1.0", "Swimming Pool & Car", "Urban Tree Crown Detection", "DIOR-RSVG", "RSVG", "FloodNet", "MQVQA", "RSVQA-LRBEN", "RSVQA-HRBEN", "Sydney Caption", "UCM Caption", "RSITMD", "RSCID", "NWPU-Captions", "LEVIR-MCI", "SYSU-CC", "Dubai-CC", "MUDS", "STARCOP", "UHI-AD", "QuakeSet", "xBD"], "code_available": true, "code_url": "https://github.com/hiyamdebary/EarthDial", "weights_available": true, "key_contribution": "EarthDial is a domain-specific conversational vision-language model for Earth observation that unifies multi-resolution, multi-spectral, SAR, and multi-temporal remote sensing understanding in a single assistant. The paper also introduces EarthDial-Instruct, a 11.11M-pair instruction-tuning dataset, and shows strong generalization across 44 downstream EO tasks and datasets.", "_source_pdf": "EarthDial_Turning_Multi-sensory_Earth_Observations_to_Interactive_Dialogues.pdf", "_extraction_timestamp": "2026-03-20T20:46:04"}
{"title": "EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain", "year": 2024, "model_name": "EarthGPT", "architecture": {"backbone": "DINOv2 ViT-L/14 + CLIP ConvNeXt-L + LLaMA-2", "type": "instruction-tuned multimodal LLM with hybrid visual encoder and autoregressive language model", "params_millions": null, "notes": "Uses a hybrid visual-enhanced perception module combining a frozen ViT encoder for multi-layer global features and a frozen CNN encoder for multi-scale local features; concatenated visual tokens are projected and fused with language tokens. The language model is LLaMA-2. During cross-modal mutual comprehension, self-attention and RMSNorm layers are unfrozen for alignment; during RS instruction tuning, linear layers are adapted with bias and shift parameters inspired by LLaMA-Adapter V2."}, "pretraining": {"method": "multi-stage multimodal instruction tuning", "method_details": "EarthGPT first performs cross-modal mutual comprehension by concatenating projected visual tokens with text tokens and partially unfreezing LLaMA-2 self-attention and RMSNorm layers for vision-language alignment on natural image-text datasets. It then performs RS-domain unified instruction tuning using bias/shift adaptation of linear layers on the MMRS-1M multi-sensor remote sensing instruction-following dataset.", "objective": "autoregressive next-token prediction for image-text alignment and instruction-following response generation", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["LAION-400M", "COCO Caption", "MMRS-1M"], "num_images": "over 1M image-text pairs for MMRS-1M", "sensors": ["optical", "SAR", "infrared"], "modalities": ["image", "text", "optical", "SAR", "infrared"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "image captioning", "region-level captioning", "visual question answering", "visual grounding", "object detection", "horizontal bounding box detection", "oriented bounding box detection"], "downstream_datasets": ["NWPU-RESISC45", "CLRS", "NaSC-TG2", "NWPU-Captions", "CRSVQA", "RSVQA-HR", "DIOR-RSVG", "MAR20"], "code_available": true, "code_url": "https://github.com/wivizhang/EarthGPT", "weights_available": true, "key_contribution": "The paper proposes EarthGPT, a universal remote sensing multimodal large language model that unifies multi-sensor image understanding and diverse RS tasks within a single instruction-following framework. It also introduces MMRS-1M, a large-scale multi-sensor remote sensing instruction dataset with over 1M image-text pairs spanning optical, SAR, and infrared imagery.", "_source_pdf": "EarthGPT_A_Universal_Multi-modal_Large_Language_Model_for_Multi-sensor_Image_Comprehension_in_Remote_Sensing_Domain.pdf", "_extraction_timestamp": "2026-03-20T20:46:15"}
{"title": "EarthPT: a time series foundation model for Earth Observation", "year": 2023, "model_name": "EarthPT", "architecture": {"backbone": "autoregressive decoder-only Transformer", "type": "autoregressive", "params_millions": 700, "notes": "Primary model is EarthPT-700M with 36 layers, 20 heads, embedding dimension 1280, block size 256. Based on the GPT/Radford et al. decoding transformer. Uses an MLP instead of word embeddings to embed multispectral time-series tokens, and includes date embeddings for the current and next observation."}, "pretraining": {"method": "autoregressive next-step prediction", "method_details": "The model is pretrained on Earth observation pixel time series, treating each temporal observation as a token and predicting the next multispectral observation. It is designed as a large observation model for EO, using multispectral reflectance values plus seasonal/date embeddings.", "objective": "next-observation prediction of pixel-level surface reflectance time series", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["ClearSky dataset"], "num_images": "14B tokens used for EarthPT-700M training; dataset described as just over 100 million individual pixels", "sensors": ["Sentinel-1", "Sentinel-2 equivalent"], "modalities": ["SAR", "multispectral", "temporal"], "spatial_resolution": "10 m", "geographic_coverage": "UK-only (100 x 100 km region in the TL square of the British National Grid for the experiments)"}, "downstream_tasks": ["time-series forecasting", "reflectance prediction", "remote sensing index forecasting", "land use classification"], "downstream_datasets": ["ClearSky dataset"], "code_available": true, "code_url": "https://github.com/aspiaspace/EarthPT", "weights_available": true, "key_contribution": "The paper introduces EarthPT, a 700M-parameter autoregressive transformer foundation model for Earth observation time series, pretrained self-supervised on large-scale multispectral pixel sequences. It shows strong long-horizon forecasting of future surface reflectance and vegetation indices, while also learning semantically meaningful embeddings that could support downstream tasks such as dynamic land use classification.", "_source_pdf": "EarthPT_a_time_series_foundation_model_for_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T20:46:23"}
{"title": "EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision", "year": 2025, "model_name": "EarthMAE", "architecture": {"backbone": "ViT", "type": "encoder-decoder", "params_millions": null, "notes": "Masked Autoencoder with asymmetric ViT encoder-decoder. Uses distinct tokenizers for each source/modalities, resizes all inputs to 224x224, and adds positional, source, and temporal encodings. Temporal encoding is built from year/month/day/hour embeddings; source embeddings distinguish modalities such as multispectral, RGB, hyperspectral, etc."}, "pretraining": {"method": "MAE", "method_details": "EarthMAE extends MAE/SpectralMAE-style pretraining to heterogeneous remote sensing data by combining multiple sources, modalities, and timestamps. It introduces source-specific tokenizers, temporal/source encodings, and explores masking strategies, with tube masking across time and sources performing best.", "objective": "masked image reconstruction with MSE on masked patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["EarthView", "Satellogic", "Sentinel-1", "Sentinel-2", "NEON"], "num_images": "about 15 terapixels total; Satellogic: 2,967,663 patches / 6,165,992 images including revisits; Sentinel: over 1M unique regions; NEON: 35,501 patches", "sensors": ["Satellogic", "Sentinel-1", "Sentinel-2", "NEON"], "modalities": ["RGB", "near-infrared", "multispectral", "SAR", "hyperspectral", "elevation", "segmentation maps", "temporal"], "spatial_resolution": "mixed (0.1m to 60m; key resolutions include 0.1m, 1m, 10m, 20m, 60m)", "geographic_coverage": "global for Satellogic and Sentinel; US forests for NEON"}, "downstream_tasks": ["scene classification", "multi-label classification", "land-use / land-cover classification"], "downstream_datasets": ["m-BigEarthNet", "m-Brick-Kiln", "m-EuroSAT", "m-ForestNet", "m-PV4GER", "m-SO2SAT"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper introduces EarthView, a large-scale multimodal remote sensing dataset for self-supervised learning that combines Satellogic, Sentinel, and NEON data across space, time, and sensing modalities. It also proposes EarthMAE, a masked autoencoder adapted to heterogeneous Earth observation data with source-aware tokenization and temporal/source encodings, showing that pretraining with high-resolution Satellogic imagery improves downstream performance.", "_source_pdf": "EarthView_A_Large_Scale_Remote_Sensing_Dataset_for_Self-Supervision.pdf", "_extraction_timestamp": "2026-03-20T20:46:36"}
{"title": "Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification", "year": 2022, "model_name": "Embedding Earth", "architecture": {"backbone": "UNet3Df", "type": "encoder-decoder", "params_millions": null, "notes": "Uses UNet3Df, a variation of UNet3D, as the dense feature extractor backbone. During pretraining, a projection head with two 1x1 convolution layers and ReLU is added. The method uses a momentum contrast setup with separate query and key encoders and a queue of key embeddings."}, "pretraining": {"method": "contrastive", "method_details": "Dense self-supervised instance discrimination at the pixel level for satellite image time series. Positive pairs are pixels corresponding to the same geographic location across two augmented spatiotemporal views; negatives include other pixels from the same sample and a momentum-encoder queue. The method explicitly tracks augmentations to compute pixel correspondences and extends dense contrastive learning to image time series.", "objective": "pixel-level contrastive alignment of corresponding locations across augmented spatiotemporal views", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["FRA-base", "DEU-base", "GHA-base", "SSU-base", "FRA-space", "FRA-time", "DEU-time", "SSU-time"], "num_images": "about 12k-352k samples depending on dataset; largest mentioned is 351,947 samples", "sensors": ["Sentinel-2"], "modalities": ["multispectral", "temporal"], "spatial_resolution": null, "geographic_coverage": "France, Germany, Ghana, South Sudan"}, "downstream_tasks": ["land cover semantic segmentation", "dense land cover classification"], "downstream_datasets": ["France crop type dataset from RPG", "Germany crop type dataset from Rußwurm and Körner", "Ghana dataset from Radiant Earth Foundation", "South Sudan dataset from Radiant Earth Foundation"], "code_available": true, "code_url": "https://github.com/michaeltrs/DeepSatModels", "weights_available": true, "key_contribution": "The paper proposes Embedding Earth, a self-supervised dense contrastive pretraining method for satellite image time series that learns pixel-level representations by matching corresponding locations across space-time augmentations. It significantly improves downstream land cover semantic segmentation over random initialization and supervised pretraining, with gains up to 25% absolute mIoU and good transfer across regions.", "_source_pdf": "Embedding_Earth_Self-supervised_contrastive_pre-training_for_dense_land_cover_classification.pdf", "_extraction_timestamp": "2026-03-20T20:46:46"}
{"title": "FLAVARS: A Multimodal Foundational Language and Vision Alignment Model for Remote Sensing", "year": 2025, "model_name": "FLAVARS", "architecture": {"backbone": "ViT-B/16", "type": "encoder-decoder multimodal encoder with contrastive alignment and masked modeling", "params_millions": null, "notes": "Built on the FLAVA framework with image and text encoders, masked-image modeling, masked-language modeling, multimodal image-text matching, global image-text contrastive loss, and an additional image-location contrastive alignment loss. The location encoder is initialized from SatCLIP weights. For segmentation evaluation, the pretrained vision encoder is paired with a UperNet decoder."}, "pretraining": {"method": "multi-task masked modeling + contrastive vision-language-location pretraining", "method_details": "FLAVARS adapts FLAVA to remote sensing by combining masked-image modeling, masked-language modeling, image-text matching, and global image-text contrastive learning, while adding geospatial awareness through a contrastive image-location alignment objective. This is designed to improve dense visual representation quality without losing zero-shot vision-language capability.", "objective": "masked image reconstruction, masked language modeling, image-text matching, image-text contrastive alignment, and image-location contrastive alignment", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["SkyScript", "SkyScript-Grounded"], "num_images": "5M", "sensors": ["multi-sensor"], "modalities": ["image", "text", "geospatial coordinates", "grounded bounding boxes"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["KNN scene classification", "zero-shot image classification", "semantic segmentation", "text-to-image retrieval"], "downstream_datasets": ["EuroSAT", "RESISC45", "PatternNet", "RSI-CB-256", "AiRound", "CvBrCT", "MLRSNet", "WHU-RS19", "OPTIMAL-31", "AID", "UCMerced", "fMoW", "SpaceNet1"], "code_available": true, "code_url": "https://huggingface.co/datasets/isaaccorley/FLAVARS", "weights_available": null, "key_contribution": "The paper proposes FLAVARS, a remote sensing multimodal pretraining framework that extends FLAVA with geospatial image-location contrastive alignment. It shows improved vision-only representations and dense prediction performance over SkyCLIP while retaining zero-shot vision-language capability.", "_source_pdf": "FLAVARS_A_Multimodal_Foundational_Language_and_Vision_Alignment_Model_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:46:56"}
{"title": "Falcon: A Remote Sensing Vision-Language Foundation Model", "year": 2025, "model_name": "Falcon", "architecture": {"backbone": "Transformer-based encoder-decoder with an image encoder", "type": "encoder-decoder vision-language model", "params_millions": 700, "notes": "Unified sequence-to-sequence architecture for image-, region-, and pixel-level tasks. Combines an image encoder with a multimodality transformer encoder-decoder and a visual adapter. Initialized from Florence-2 pretrained weights; output token length increased to 4096."}, "pretraining": {"method": "multi-task instruction tuning", "method_details": "Falcon is trained in a unified prompt-based seq2seq manner over 14 remote sensing tasks using the Falcon SFT dataset. A key novelty is dynamic prompt training, where multiple differently phrased instructions with the same semantics are sampled from a prompt pool to improve instruction understanding and reduce reliance on task-specific tokens.", "objective": "instruction-conditioned sequence generation with cross-entropy over textual outputs encoding labels, captions, boxes, polygons, and change masks", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["Falcon SFT", "AID", "ASD", "AiRound", "airplane det", "BHP Watertanks", "CDD", "CLRS", "DIOR", "DIOR-RSVG", "DLRSD", "DOTA2.0", "DSIFN", "EGY BCD", "EuroSAT", "FAIR1M1.0", "GEONRW", "GID15", "Globe230k", "Hefei", "RSVQA HR", "HRSC2016", "HRSCD", "Hurricane Damage", "Inria", "iSAID", "LEVIR-CD", "LEVIR-CD+", "LoveDA", "RSVQA LR", "MAR20", "million-AID", "MSBC", "MSOSCD", "MultiScene", "NaSC TG2", "NJDS", "NWPU RESISC45", "NWPU-VHR-10", "OPTIMAL31", "PatternNet", "RS C11", "RSD46 WHU", "RSI CB", "RSICap", "RSICD", "RSITMD", "RSOD", "RSSCN7", "RSVG", "S2Looking", "S2-SHIPS", "SAMRS", "ship det", "ShipRSImagerNet", "SIRI WHU", "SODA-A", "Sydney Captions", "SYSU-CD", "SZTAKI", "UCAS-AOD", "UCM-Captions", "UCM-Classification", "VHRShips", "WHU GID", "WHU-CD", "WHU RS19", "xView"], "num_images": "5.6M images, 78.2M samples", "sensors": ["satellites", "airplanes", "drones", "aerial", "SAR"], "modalities": ["RGB", "multispectral", "SAR", "text", "image pairs"], "spatial_resolution": "multi-spatial resolution", "geographic_coverage": null}, "downstream_tasks": ["image classification", "visual question answering", "counting", "image captioning", "detailed image captioning", "region classification (HBB)", "region classification (OBB)", "region detection (HBB)", "region detection (OBB)", "visual grounding", "region captioning", "pixel classification", "segmentation", "change detection"], "downstream_datasets": ["BHP Watertanks", "CLRS", "DIOR", "DOTA2.0", "FAIR1M1.0", "GEONRW", "Globe230k", "Hefei", "Hurricane Damage", "LoveDA", "MultiScene", "NWPU RESISC45", "NaSC TG2", "OPTIMAL31", "AiRound", "PatternNet", "RSD46 WHU", "RSITMD", "RSI CB", "RSOD", "RSSCN7", "RS C11", "SIRI WHU", "SODA-A", "UCAS-AOD", "WHU GID", "iSAID", "million-AID", "xView", "ASD", "RSVQA HR", "HRSC2016", "S2-SHIPS", "ShipRS", "VHRShips", "airplane det", "UCM-Captions", "MAR20", "NWPU-VHR-10", "GID15", "CDD", "WHU-CD", "CCD"], "code_available": true, "code_url": "https://github.com/TianHuiLab/Falcon", "weights_available": true, "key_contribution": "The paper introduces Falcon, a unified remote sensing vision-language foundation model that handles 14 tasks spanning image, region, and pixel levels with only 0.7B parameters. It also contributes Falcon SFT, a large-scale 78M-sample multi-task instruction-tuning dataset built from 67 remote sensing datasets to enable broad instruction-following and strong zero-shot performance.", "_source_pdf": "Falcon_A_Remote_Sensing_Vision-Language_Foundation_Model.pdf", "_extraction_timestamp": "2026-03-20T20:47:12"}
{"title": "Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing", "year": 2023, "model_name": "FG-MAE", "architecture": {"backbone": "ViT-S/16", "type": "encoder-decoder", "params_millions": 22, "notes": "Based on MAE with an asymmetric ViT encoder and lightweight ViT decoder. Default model is ViT-S/16 for experiments; scalability is demonstrated up to ViT-H/14 with 658M parameters. FGMAE-MS uses two parallel decoder heads to reconstruct HOG and NDI; FGMAE-SAR reconstructs HOG only."}, "pretraining": {"method": "MAE-style masked image modeling with feature reconstruction", "method_details": "Instead of reconstructing raw pixels like MAE, FG-MAE reconstructs remote sensing features as targets. For multispectral imagery it reconstructs a combination of HOG and NDI, while for SAR imagery it reconstructs HOG to reduce sensitivity to speckle noise and encourage more semantic representations.", "objective": "masked feature reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12"], "num_images": "1M", "sensors": ["Sentinel-1 GRD", "Sentinel-2 L1C"], "modalities": ["SAR", "multispectral", "temporal"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation"], "downstream_datasets": ["BigEarthNet-MM", "EuroSAT", "EuroSAT-SAR", "DFC2020"], "code_available": true, "code_url": "https://github.com/zhu-xlab/FGMAE", "weights_available": true, "key_contribution": "The paper proposes FG-MAE, a simple MAE variant that replaces pixel reconstruction with remote sensing feature reconstruction, using HOG+NDI for multispectral imagery and HOG for SAR. It shows that feature-guided masking improves transfer performance across classification and segmentation tasks, with especially strong gains for SAR, and scales to large ViTs up to 0.7B parameters.", "_source_pdf": "Feature_Guided_Masked_Autoencoder_for_Self-supervised_Learning_in_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:47:20"}
{"title": "FlexiMo: A Flexible Remote Sensing Foundation Model", "year": 2025, "model_name": "FlexiMo", "architecture": {"backbone": "ViT-Base", "type": "encoder-only", "params_millions": null, "notes": "Built on top of the DOFA ViT-Base pretrained model. Adds a lightweight Transformer-based dynamic weight generator for wavelength-guided channel adaptation and a plug-and-play parameter-free spatial resolution-aware module (PI-Resize) to adapt patch embeddings to arbitrary image sizes, resolutions, and patch sizes. Uses a classification head for image-level tasks and UperNet for pixel-level tasks."}, "pretraining": {"method": "foundation model adaptation / fine-tuning of pretrained DOFA", "method_details": "FlexiMo itself is presented primarily as a flexible adaptation framework over pretrained DOFA weights rather than a new standalone pretraining regime. Its novelty lies in combining wavelength-guided dynamic channel adaptation with a parameter-free spatial resolution-aware patch embedding alignment module so the pretrained model can handle arbitrary channel counts, spatial resolutions, and patch sizes.", "objective": "not a new pretraining objective; adapts a pretrained remote sensing foundation model for downstream tasks", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": null, "num_images": null, "sensors": null, "modalities": ["RGB", "multispectral", "SAR"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["scene classification", "land cover classification", "building extraction", "cloud detection"], "downstream_datasets": ["EuroSAT", "EuroSAT-SAR", "SegMunich", "WHU-Building", "GF12MS-WHU"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes FlexiMo, a flexible remote sensing foundation model adaptation framework that enables pretrained ViT-based RSFMs to handle arbitrary spatial resolutions, image sizes, patch sizes, and channel configurations. Its main contribution is the combination of a parameter-free spatial resolution-aware PI-Resize module and a wavelength-guided dynamic channel adaptation mechanism, yielding strong generalization across RGB, multispectral, and SAR downstream tasks.", "_source_pdf": "FlexiMo_A_Flexible_Remote_Sensing_Foundation_Model.pdf", "_extraction_timestamp": "2026-03-20T20:47:30"}
{"title": "FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring", "year": 2023, "model_name": "FoMo-Net", "architecture": {"backbone": "ViT-Base-like Vision Transformer", "type": "encoder-decoder", "params_millions": 67.27, "notes": "Primary model is FoMo-Net1, which uses a single shared linear projection for all spectral bands. Backbone uses a ViT encoder with 12 layers and 12 attention heads, trained with an MAE-style decoder for reconstruction. Spectral bands are tokenized independently with trainable spectral and positional embeddings; maximum of 4 sampled spectral bands during pretraining; input resolution 64x64. FoMo-Netm is an alternative variant with separate per-band projections, but FoMo-Net1 is the better-performing primary variant."}, "pretraining": {"method": "MAE", "method_details": "FoMo-Net proposes a sensor-agnostic masked autoencoding framework that can process arbitrary combinations of remote sensing modalities/bands, including unaligned paired and unpaired data. Its novelty is random dataset and spectral-band sampling during training, treating each spectral band as a modality, plus trainable spectral and positional embeddings and gradient accumulation to stabilize optimization over heterogeneous batches.", "objective": "masked image reconstruction over randomly sampled spectral bands/modalities", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["RapidAI4EO", "TalloS", "SSL4EO-Landsat", "FiveBillionPixels", "FoMo-Bench UAV datasets", "FLAIR #1"], "num_images": null, "sensors": ["Sentinel-1", "Sentinel-2", "Landsat 8", "Landsat 9", "Planet", "Gaofen-2", "UAV", "DEM"], "modalities": ["RGB", "multispectral", "SAR", "DEM"], "spatial_resolution": "mixed (from a few centimeters to 60m)", "geographic_coverage": "global"}, "downstream_tasks": ["multi-label classification", "semantic segmentation", "object detection", "point cloud semantic segmentation"], "downstream_datasets": ["BigEarthNet-MM", "Sen12MS", "RapidAI4EO", "TreeSatAI", "TalloS", "Waititu", "FLAIR #1", "FLAIR #2", "Spekboom", "FiveBillionPixels", "ForestNet", "Woody", "NeonTree", "ReforesTree", "FORinstance"], "code_available": true, "code_url": "https://github.com/RolnickLab/FoMo-Bench", "weights_available": null, "key_contribution": "The paper introduces FoMo-Bench, the first unified global benchmark for forest monitoring spanning diverse remote sensing modalities, scales, and tasks, and adds TalloS, a new global multimodal tree-genus classification dataset. It also proposes FoMo-Net, a flexible self-supervised pretraining framework that learns a single sensor-agnostic backbone capable of handling arbitrary combinations of common remote sensing bands and resolutions.", "_source_pdf": "FoMo-Bench_a_multi-modal_multi-scale_and_multi-task_Forest_Monitoring_Benchmark_for_remote_sensing_foundation_models.pdf", "_extraction_timestamp": "2026-03-20T20:47:43"}
{"title": "FUS-MAE: A CROSS-ATTENTION-BASED DATA FUSION APPROACH FOR MASKED AUTOENCODERS IN REMOTE SENSING", "year": 2024, "model_name": "Fus-MAE", "architecture": {"backbone": "ViT", "type": "encoder-decoder", "params_millions": null, "notes": "MAE-based multimodal transformer for SAR-optical fusion. Uses a ViT encoder with a cross-attended patch projection module (XAttnEncoder) for early fusion, and lightweight modality-specific decoders with optional cross-attention in decoding (XAttnDecoder) for feature-level fusion. Two variants are studied: Fus-MAE XAE (early fusion in encoder) and Fus-MAE XAD (feature-level fusion in decoder); the paper indicates XAE as the stronger primary variant."}, "pretraining": {"method": "MAE", "method_details": "A self-supervised masked image modeling framework for multimodal SAR-optical fusion that introduces cross-attention between modality-specific token streams to better model the large domain gap between SAR and multispectral optical imagery. It studies both early fusion via cross-attended patch projection and feature-level fusion via cross-attention in the decoder, with independent and consistent masking strategies.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet-MM"], "num_images": "354,196", "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["SAR", "multispectral"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["multilabel classification", "land-cover classification"], "downstream_datasets": ["BigEarthNet-MM", "SEN12MS"], "code_available": true, "code_url": null, "weights_available": true, "key_contribution": "The paper proposes Fus-MAE, a multimodal masked autoencoder for SAR-optical remote sensing that uses cross-attention to fuse modality-specific token streams, especially at an early stage through cross-attended patch projection. It shows that this early-fusion MAE strategy can outperform prior masked-autoencoding approaches and compete with specialized contrastive SAR-optical fusion methods on BigEarthNet-MM and SEN12MS.", "_source_pdf": "Fus-MAE_A_cross-attention-based_data_fusion_approach_for_Masked_Autoencoders_in_remote_sensing.pdf", "_extraction_timestamp": "2026-03-20T20:47:52"}
{"title": "GAIR: Improving Multimodal Geo-Foundation Model with Geo-Aligned Implicit Representations", "year": 2025, "model_name": "GAIR", "architecture": {"backbone": "ViT-B", "type": "multimodal contrastive tri-encoder with implicit neural representation module", "params_millions": null, "notes": "Uses three factorized encoders: a ViT-based remote sensing image encoder, a ViT-based street view image encoder, and a geolocation encoder based on Random Fourier Features from GeoCLIP. Adds a linear projection head for image encoders. Includes a novel INR module over RS patch features with feature unfolding (3x3 neighborhood concatenation) and local ensemble interpolation via a shared single-layer perceptron to extract geolocated RS embeddings."}, "pretraining": {"method": "contrastive multi-object pretraining with INR-based geo-alignment", "method_details": "GAIR aligns three modalities—remote sensing imagery, street view imagery, and geolocation—by first converting RS patch features into a continuous spatial representation using an implicit neural representation module, then querying a localized RS embedding at the exact street-view geolocation. Training combines Implicit Neural Contrastive Learning between localized RS and street-view embeddings with Spatially Explicit Contrastive Learning that also aligns both with location embeddings via a memory bank.", "objective": "contrastive alignment of geolocated remote sensing, street view, and location embeddings", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Streetscape1M", "Global Streetscapes dataset"], "num_images": "1M", "sensors": ["Sentinel-2", "street view"], "modalities": ["multispectral", "street-view imagery", "geolocation", "temporal"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["socio-economic indicator regression", "human perception regression", "view direction classification", "imaging platform classification", "burn scars segmentation", "crop type mapping", "cropland polygon segmentation", "geo-aware image regression", "geo-aware species recognition", "geo-aware Flickr image classification"], "downstream_datasets": ["Global Streetscapes dataset", "HLS Burn Scars dataset", "Crop Type Mapping dataset (South Sudan)", "AI4SmallFarms", "MOSAIKS", "BirdSnap", "NABirds", "iNat2017", "iNat2018", "YFCC100M-GEO100"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes GAIR, a multimodal geo-foundation model that jointly learns from overhead remote sensing imagery, street-view imagery, and geolocation metadata. Its main novelty is an INR-based geo-alignment mechanism that extracts location-specific RS embeddings for contrastive training, enabling strong transfer across 10 geospatial tasks spanning image, remote sensing, and location benchmarks.", "_source_pdf": "GAIR_Improving_Multimodal_Geo-Foundation_Model_with_Geo-Aligned_Implicit_Representations.pdf", "_extraction_timestamp": "2026-03-20T20:48:03"}
{"title": "Galileo: Learning Global & Local Features of Many Remote Sensing Modalities", "year": 2025, "model_name": "Galileo", "architecture": {"backbone": "ViT-Base", "type": "encoder-decoder masked latent prediction transformer", "params_millions": 85.0, "notes": "Primary model is Galileo-ViT-Base with 12 transformer blocks, 768 hidden dimension, and 12 heads. Uses a ViT-based online encoder, an EMA target encoder, and a predictor transformer with cross-attention. Supports flexible tokenization over space, time, and channel groups, with sinusoidal spatial/temporal positional embeddings, learnable channel embeddings, and month embeddings. Designed for multimodal inputs including image time series, pixel time series, and single-timestep imagery."}, "pretraining": {"method": "masked latent modeling with dual contrastive objectives", "method_details": "Galileo introduces a dual global-local self-supervised pretraining scheme. The global objective uses structured space-time masking and deep EMA-encoder targets to learn global features, while the local objective uses unstructured masking and shallow linear-projection targets to learn local features. Both objectives use token-level contrastive losses, with the global branch using a batch-aware PatchDiscB loss and the local branch using PatchDisc. The novelty is jointly learning representations at different latent depths and spatial-temporal scales in a single multimodal model.", "objective": "masked token prediction in latent space with contrastive alignment of predicted and target tokens", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom globally sampled pretraining dataset", "Sentinel-1", "Sentinel-2", "Shuttle Radar Topography Mission", "Dynamic World", "World Cereal", "ERA5", "TerraClimate", "VIIRS nighttime lights", "LandScan"], "num_images": "127,155 instances", "sensors": ["Sentinel-1", "Sentinel-2", "SRTM", "ERA5", "TerraClimate", "VIIRS", "LandScan"], "modalities": ["multispectral", "SAR", "elevation", "slope", "weather", "climate", "nightlights", "population", "land cover pseudo-labels", "location", "temporal"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation", "image time series segmentation", "pixel time series classification", "multimodal pixel time series classification", "flood detection", "marine debris segmentation", "crop type classification", "agricultural land cover mapping"], "downstream_datasets": ["m-EuroSat", "m-BigEarthNet", "m-So2Sat", "m-Brick-Kiln", "m-Cashew-Plant", "m-SA-Crop-Type", "MADOS", "Sen1Floods11", "PASTIS", "Breizhcrops", "CropHarvest"], "code_available": true, "code_url": "https://github.com/nasaharvest/galileo", "weights_available": true, "key_contribution": "The paper proposes Galileo, a single highly multimodal remote sensing foundation model that flexibly handles many modalities, spatial-temporal input shapes, and scales. Its main innovation is a dual self-supervised global-local masked latent modeling objective that learns both global semantic structure and fine local detail, leading to state-of-the-art performance across 11 remote sensing benchmarks.", "_source_pdf": "Galileo_Learning_Global_Local_Features_of_Many_Remote_Sensing_Modalities.pdf", "_extraction_timestamp": "2026-03-20T20:48:36"}
{"title": "Generic Knowledge Boosted Pre-training For Remote Sensing Images", "year": 2024, "model_name": "GeRSP", "architecture": {"backbone": "ResNet-50", "type": "teacher-student dual-network pretraining framework with supervised and self-supervised branches", "params_millions": null, "notes": "Uses a student network and an EMA-updated teacher network. Remote sensing contrastive learning (RSCL) is performed with MoCo-style contrastive learning, dynamic queue, and InfoNCE loss; natural image auxiliary learning (NIAL) is supervised on labeled natural images using the student only. Features are pooled with GAP; RSCL uses a 2-layer MLP projector (2048 hidden, 128 output), and NIAL uses a linear predictor/classifier."}, "pretraining": {"method": "hybrid supervised + contrastive self-supervised pretraining", "method_details": "GeRSP jointly learns from unlabeled remote sensing images and labeled natural images in a unified teacher-student framework. It combines RS-domain knowledge from self-supervised contrastive learning with generic visual knowledge from supervised ImageNet training, rather than sequentially pretraining and risking forgetting.", "objective": "InfoNCE contrastive alignment on remote sensing image augmentations plus cross-entropy classification on natural images", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["Million-AID", "ImageNet"], "num_images": "about 1M remote sensing images + ImageNet", "sensors": ["Google Earth", "natural image web photographs"], "modalities": ["RGB"], "spatial_resolution": "mixed; Million-AID ranges from 0.5 to 153 m per pixel", "geographic_coverage": "global for Million-AID"}, "downstream_tasks": ["scene classification", "object detection", "semantic segmentation"], "downstream_datasets": ["EuroSAT", "NWPU-RESISC45", "DIOR", "DOTA", "LoveDA"], "code_available": true, "code_url": "https://github.com/floatingstarZ/GeRSP", "weights_available": true, "key_contribution": "The paper proposes GeRSP, a unified remote sensing pretraining framework that combines supervised learning on natural images with self-supervised contrastive learning on remote sensing images using a teacher-student design. This enables simultaneous acquisition of generic visual knowledge and remote sensing-specific knowledge, improving transfer on scene classification, object detection, and semantic segmentation.", "_source_pdf": "Generic_Knowledge_Boosted_Pre-training_For_Remote_Sensing_Images.pdf", "_extraction_timestamp": "2026-03-20T20:48:45"}
{"title": "GeoChat: Grounded Large Vision-Language Model for Remote Sensing", "year": 2023, "model_name": "GeoChat", "architecture": {"backbone": "CLIP-ViT-L/14", "type": "instruction-tuned vision-language model with image encoder + MLP projector + LLM; grounded conversational VLM", "params_millions": null, "notes": "Built on LLaVA-v1.5. Uses frozen CLIP-ViT(L-14) visual encoder and frozen 2-layer MLP adaptor, with Vicuna-v1.5-7B as the LLM. Positional embeddings are interpolated to support 504x504 inputs (1296 patches instead of 576 at 336x336). LoRA is applied to the LLM. Adds task-specific tokens ([grounding], [identify], [refer]) and textual spatial box representations for region input/output grounding."}, "pretraining": {"method": "vision-language instruction tuning / domain adaptation via LoRA fine-tuning", "method_details": "GeoChat does not pretrain from scratch; it adapts pretrained LLaVA-v1.5 to remote sensing using a newly constructed 318k remote-sensing multimodal instruction-following dataset. The novelty lies in unifying image-level conversation, region-level reasoning, and visual grounding through task tokens and textual spatial coordinates.", "objective": "multitask instruction following, conversational response generation, region-aware reasoning, and grounded text generation", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["DOTA", "DIOR", "FAIR1M", "SAMRS", "RSVQA-LRBEN", "NWPU-RESISC-45", "FloodNet", "LoveDA", "LAION-CC-SBU"], "num_images": "approximately 318k image-instruction pairs", "sensors": ["aerial", "satellite"], "modalities": ["image", "text", "bounding boxes", "segmentation masks"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "visual question answering", "image captioning", "region captioning", "multi-turn conversation", "grounded image description", "visual grounding", "referring expression comprehension", "referring object detection", "region-level conversation"], "downstream_datasets": ["AID", "UCMerced", "RSVQA-LRBEN", "RSVQA-HRBEN", "SAMRS"], "code_available": true, "code_url": "https://github.com/mbzuai-oryx/geochat", "weights_available": true, "key_contribution": "The paper introduces GeoChat, the first remote sensing vision-language model that supports unified multitask conversations with both image-level and region-level reasoning, including visual grounding via spatial coordinates. It also creates a new 318k-sample remote-sensing multimodal instruction-following dataset and benchmark to adapt LLaVA-style models to remote sensing.", "_source_pdf": "GeoChat_Grounded_Large_Vision-Language_Model_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:49:08"}
{"title": "DOFA-CLIP: Multimodal Vision–Language Foundation Models for Earth Observation", "year": 2025, "model_name": "DOFA-CLIP", "architecture": {"backbone": "SoViT-400m (large primary variant), dynamic DOFA ViT", "type": "CLIP-style dual-encoder", "params_millions": 400, "notes": "Uses a single unified wavelength-aware dynamic vision transformer adapted from DOFA, with dynamic patch embedding generated from input channel wavelengths. Base variant uses ViT-Base/16; large variant uses SoViT-400m with patch size 14 and input size 384. Includes a Modality-aware Knowledge Agglomeration (MaKA) module for modality-conditioned feature distillation. Vision encoder initialized from SigLIP weights; paired with a CLIP text encoder."}, "pretraining": {"method": "CLIP/SigLIP-style contrastive vision-language pretraining with multi-teacher distillation", "method_details": "Introduces VECT (Vision-models Enhanced Contrastive Text-image pretraining), which combines pairwise sigmoid contrastive loss with feature-matching distillation from multiple frozen vision foundation models (SigLIP, DINOv2, ImageNet ViT) to improve spatial awareness. Also introduces MaKA for modality-aware distillation and progressive weight merging across RGB and non-RGB modality subsets.", "objective": "image-text contrastive alignment with pairwise sigmoid loss plus intermediate feature-matching distillation", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["Seg4", "Det10", "Flair2-RGB-caption", "VRS-train", "SkyScript dataset", "MMflood-caption", "SAR-ship-caption", "ChatEarthNet-SAR-caption", "ChatEarthNet-S2-caption", "NLCD-hyper-caption", "Flair2-Elevation-caption", "IR-ship-caption"], "num_images": "2,050,983", "sensors": ["Sentinel-2", "Sentinel-1", "EnMAP", "aerial imagery", "elevation maps", "infrared imagery", "SAR"], "modalities": ["RGB", "multispectral", "SAR", "hyperspectral", "infrared", "elevation", "text"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["zero-shot scene classification", "fine-grained attribute classification", "cross-modal retrieval", "semantic segmentation"], "downstream_datasets": ["SkyScript classification dataset", "AID", "EuroSAT", "fMoW", "Million-AID", "PatternNet", "NWPU-RESISC45", "RSI-CB", "m-bigearthnet", "m-so2sat", "m-forestnet", "RSICD", "RSITMD", "UCM-caption", "MADOS", "m-nz-cattle", "m-NeonTree"], "code_available": true, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes DOFA-CLIP, a unified multimodal EO vision-language foundation model that handles heterogeneous modalities and varying spectral bands with a single wavelength-aware transformer. It also contributes the GeoLangBind-2M dataset, the VECT multi-teacher distillation strategy for spatially aware CLIP features, and the MaKA module for modality-aware knowledge transfer, achieving strong zero-shot performance across seen and unseen EO modalities.", "_source_pdf": "GeoLangBind_Unifying_Earth_Observation_with_Agglomerative_Vision-Language_Foundation_Models.pdf", "_extraction_timestamp": "2026-03-20T20:49:19"}
{"title": "GeoViT: A Versatile Vision Transformer Architecture for Geospatial Image Analysis", "year": 2023, "model_name": "GeoViT", "architecture": {"backbone": "Vision Transformer (ViT)", "type": "encoder with task-specific heads; multimodal multi-task model", "params_millions": null, "notes": "GeoViT uses a ViT-based backbone that patchifies inputs, linearly projects patches with positional embeddings, and processes them with multi-head self-attention and feed-forward layers. The CO2 variant uses Sentinel-2 imagery plus a weather vector and adds a SegFormer-based segmentation head with additional fully connected classification and regression heads. The NO2 variant uses Sentinel-2 and Sentinel-5P inputs with a cross-attention fusion mechanism and a regression head for scalar NO2 concentration prediction."}, "pretraining": {"method": null, "method_details": "The paper explicitly states that for the NO2 estimation task they do not pre-train on BigEarthNet and compare against a non-pretrained baseline. No dedicated pretraining strategy for GeoViT is proposed.", "objective": null, "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet"], "num_images": null, "sensors": ["Sentinel-2"], "modalities": ["multispectral"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["segmentation", "classification", "regression", "CO2 plume segmentation", "fuel type classification", "power generation rate estimation", "surface-level NO2 concentration regression"], "downstream_datasets": [], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes GeoViT, a compact ViT-based geospatial model for greenhouse gas monitoring from satellite imagery, with variants for multimodal CO2 analysis and NO2 regression. It shows improved performance over prior CNN-based approaches while using substantially smaller model sizes, aided by transformer-based global context modeling and cross-attention fusion for Sentinel-2/Sentinel-5P data.", "_source_pdf": "GeoViT_A_Versatile_Vision_Transformer_Architecture_for_Geospatial_Image_Analysis.pdf", "_extraction_timestamp": "2026-03-20T20:49:29"}
{"title": "Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images", "year": 2021, "model_name": "GLCNet", "architecture": {"backbone": "ResNet-50", "type": "encoder-decoder", "params_millions": null, "notes": "Built on DeepLabV3+ for semantic segmentation. The encoder is pretrained with a global style contrastive module, and the full encoder-decoder is additionally trained with a local matching contrastive module. Uses MLP projection heads for global and local contrastive learning."}, "pretraining": {"method": "contrastive self-supervised learning", "method_details": "Proposes Global style and Local matching Contrastive learning Network (GLCNet), combining a global style contrastive module using channel-wise mean and variance as style features instead of only global average pooled features, and a local matching contrastive module that aligns corresponding local regions across augmented views to learn pixel/local-region-discriminative representations for semantic segmentation.", "objective": "global image-level contrastive alignment plus local region matching contrastive alignment", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["ISPRS Potsdam Dataset", "DeepGlobe Land Cover Classification Dataset", "Hubei Dataset", "Xiangtan Dataset"], "num_images": "varies by dataset: 13,824 (Potsdam), 18,248 (DGLC), 66,471 (Hubei), 16,051 (Xiangtan)", "sensors": ["aerial", "Gaofen-2", "satellite"], "modalities": ["RGB", "NIR", "multispectral"], "spatial_resolution": "mixed: 0.05m, 0.5m, 2m", "geographic_coverage": "Germany (Potsdam) and China (Hubei, Xiangtan); DGLC geographic coverage unclear"}, "downstream_tasks": ["semantic segmentation"], "downstream_datasets": ["ISPRS Potsdam Dataset", "DeepGlobe Land Cover Classification Dataset", "Hubei Dataset", "Xiangtan Dataset"], "code_available": true, "code_url": "https://github.com/GeoX-Lab/G-RSIM", "weights_available": null, "key_contribution": "The paper introduces GLCNet, a self-supervised contrastive pretraining framework tailored for remote sensing semantic segmentation by jointly learning global style-aware and local region-matching representations. It shows that this design improves segmentation performance under limited labels and often surpasses ImageNet pretraining and prior self-supervised baselines.", "_source_pdf": "Global_and_Local_Contrastive_Self-Supervised_Learning_for_Semantic_Segmentation_of_HR_Remote_Sensing_Images.pdf", "_extraction_timestamp": "2026-03-20T20:49:49"}
{"title": "HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery", "year": 2025, "model_name": "HyperFree", "architecture": {"backbone": "ViT-Base", "type": "promptable encoder-decoder segmentation model based on SAM", "params_millions": null, "notes": "Built on the meta-architecture of Segment Anything Model (SAM) with two custom components: (1) a channel-adaptive embedding layer using a learnable full-spectrum wavelength weight dictionary to dynamically construct the embedding layer for varying hyperspectral channels, and (2) prompt-mask-feature (PMF) interaction for task-specific tuning-free inference. Uses SAM-style prompt encoder and mask decoder."}, "pretraining": {"method": "promptable segmentation pretraining", "method_details": "Instead of MAE or contrastive pretraining, HyperFree is trained in a SAM-like prompt engineering paradigm on a large annotated hyperspectral segmentation dataset. During training, random subsets of channels from the full spectrum are selected so the learned wavelength dictionary becomes wavelength-aware and can generalize to unseen channel configurations. Point prompts are randomly sampled and the model is trained to segment corresponding masks.", "objective": "prompt-conditioned mask segmentation with focal loss and Dice loss", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["Hyper-Seg", "fMoW", "SpaceNet"], "num_images": "nearly 150k images", "sensors": ["AVIRIS", "multispectral satellite imagery from fMoW", "multispectral satellite imagery from SpaceNet"], "modalities": ["hyperspectral", "multispectral", "segmentation masks"], "spatial_resolution": "0.6 to 5.0 m", "geographic_coverage": null}, "downstream_tasks": ["multi-class classification", "one-class classification", "target detection", "anomaly detection", "change detection", "hyperspectral denoising", "hyperspectral unmixing", "hyperspectral object tracking"], "downstream_datasets": ["LongKou", "HanChuan", "HongHu", "XiongAn", "Airport", "Cri", "Beach-1", "Beach-2", "Hermiston", "River", "Washington D.C.", "Urban", "HOTC 2023"], "code_available": true, "code_url": "https://rsidea.whu.edu.cn/hyperfree.htm", "weights_available": null, "key_contribution": "The paper introduces HyperFree, a tuning-free hyperspectral foundation model that can directly handle unseen hyperspectral images with varying channel numbers using a learned full-spectrum wavelength dictionary and SAM-style promptable segmentation. It also proposes PMF interaction to turn one prompt into multiple semantic-aware masks across several hyperspectral tasks, supported by a large new annotated training dataset, Hyper-Seg.", "_source_pdf": "HyperFree_A_Channel-adaptive_and_Tuning-free_Foundation_Model_for_Hyperspectral_Remote_Sensing_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:49:59"}
{"title": "Multi-Modal Multi-Objective Contrastive Learning for Sentinel-1/2 Imagery", "year": 2023, "model_name": "IaI-SimCLR", "architecture": {"backbone": "ResNet-18", "type": "multi-modal dual-encoder contrastive framework", "params_millions": null, "notes": "Uses separate ResNet-18 encoders for Sentinel-1 and Sentinel-2. Includes four projection heads total: inter-modality heads and intra-modality heads for each modality. Encoder features are 512-D and projected to 128-D latent space."}, "pretraining": {"method": "multi-objective contrastive learning (SimCLR-style / NT-Xent)", "method_details": "Proposes Intra-and Inter-modality SimCLR (IaI-SimCLR), combining inter-modality contrastive alignment between co-located Sentinel-1 and Sentinel-2 patches with intra-modality contrastive learning on augmented views for each modality. Also introduces local batch sampling using geocoded proximity to create harder negatives.", "objective": "contrastive alignment of co-located cross-modal pairs plus intra-modal augmentation-invariant representation learning", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SEN12MS"], "num_images": null, "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["SAR", "multispectral"], "spatial_resolution": "10 m and 20 m GSD bands", "geographic_coverage": null}, "downstream_tasks": ["land-cover classification", "crop type mapping", "biomass estimation"], "downstream_datasets": ["SEN12MS-DFC", "SEN12MS-TP", "Canadian-Crops", "GEDI-L4A"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper introduces IaI-SimCLR, a multi-modal multi-objective contrastive learning method for Sentinel-1/2 imagery that preserves both inter-modality and intra-modality information during self-supervised pretraining. It also proposes local batch sampling based on geographic proximity to create harder negatives and shows benefits across multiple remote sensing downstream tasks.", "_source_pdf": "IaI-SimCLR_Multi-Modal_Multi-Objective_Contrastive_Learning_for_Sentinel-12_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:50:21"}
{"title": "IN-DOMAIN REPRESENTATION LEARNING FOR REMOTE SENSING", "year": 2019, "model_name": "In-domain representation learning baseline", "architecture": {"backbone": "ResNet50 v2", "type": "encoder-only", "params_millions": null, "notes": "Single-backbone CNN used for all experiments. Representations are obtained by supervised upstream training/fine-tuning on one remote sensing dataset and then transferred to other remote sensing datasets. Input preprocessing uses resize or crop-and-rotate augmentation."}, "pretraining": {"method": "supervised in-domain pretraining", "method_details": "The paper studies generic remote sensing representations learned by supervised training on one remote sensing dataset and transferring them to other remote sensing tasks, comparing against ImageNet pretraining and training from scratch. It is not a novel SSL/VLM foundation model, but a benchmark and transfer-learning study establishing in-domain baselines.", "objective": "image classification / multi-label classification supervision on upstream remote sensing datasets", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet", "EuroSAT", "NWPU RESISC-45", "So2Sat LCZ-42", "UC Merced Land-Use Dataset"], "num_images": "approximately 1.0M total across the 5 datasets", "sensors": ["Sentinel-2", "Sentinel-1", "aerial imagery"], "modalities": ["RGB", "multispectral", "SAR"], "spatial_resolution": "mixed (0.3 m to 60 m+ depending on dataset/band)", "geographic_coverage": "mixed/global and regional; includes Europe, multiple continents, and US-only subsets depending on dataset"}, "downstream_tasks": ["scene classification", "land-use/land-cover classification", "multi-label land-cover classification", "local climate zone classification"], "downstream_datasets": ["BigEarthNet", "EuroSAT", "NWPU RESISC-45", "So2Sat LCZ-42", "UC Merced Land-Use Dataset"], "code_available": null, "code_url": null, "weights_available": true, "key_contribution": "The paper establishes a standardized benchmark for remote sensing representation learning using five diverse datasets and a common evaluation protocol. It shows that supervised in-domain pretraining generally transfers better than ImageNet pretraining, especially in low-label regimes, and releases standardized datasets plus trained representations via TensorFlow Datasets and TensorFlow Hub.", "_source_pdf": "In-domain_representation_learning_for_remote_sensing.pdf", "_extraction_timestamp": "2026-03-20T20:50:32"}
{"title": "LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Model", "year": 2024, "model_name": "LHRS-Bot", "architecture": {"backbone": "CLIP ViT-L/14 + LLaMA2-7B", "type": "vision-language multimodal large language model with vision encoder + perceiver + LLM", "params_millions": null, "notes": "Uses a frozen pre-trained CLIP ViT-L/14 vision encoder (224x224 input), a trainable vision perceiver with stacked cross-attention and MLP layers, and LLaMA2-7B as the language model. It retains multi-level hidden states from the vision encoder (layers L/3, 2L/3, L-1) and summarizes them with learnable queries using a descending query allocation strategy [64, 48, 32]. LoRA is applied to the LLM during later training stages."}, "pretraining": {"method": "multi-stage vision-language alignment and instruction tuning", "method_details": "Three-stage curriculum learning: (1) caption-generation pretraining on LHRS-Align to align remote sensing images and language using image-caption pairs; (2) multi-task pretraining with instruction-formatted RS datasets for classification, VQA, visual grounding, and captioning, with LoRA on the LLM; (3) supervised fine-tuning on RS-specific instruction-following and reasoning data plus sampled LLaVA reasoning data. Novelty lies in VGI-enhanced data creation and a multi-level vision-language alignment strategy that summarizes features from multiple encoder depths.", "objective": "autoregressive next-token prediction for caption generation and instruction-following conditioned on visual tokens", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["LHRS-Align", "LHRS-Instruct", "RSITMD", "NWPU", "RSVQA-HR", "RSVQA-LR", "UCM", "RSVG", "DIOR-RSVG", "METER-ML", "fMoW", "RSICD", "LLaVA complex reasoning dataset"], "num_images": "LHRS-Align: 1.15M image-text pairs; LHRS-Instruct includes ~39.8K samples; multi-task instruction dataset: 42,322 samples; plus 20K sampled from LLaVA complex reasoning", "sensors": ["Google Earth imagery"], "modalities": ["RGB", "text", "bounding boxes", "VGI/OSM attributes"], "spatial_resolution": "1 m for LHRS-Align; mixed overall", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "visual question answering", "visual grounding", "multimodal instruction following", "remote sensing reasoning"], "downstream_datasets": ["AID", "WHU-RS19", "NWPU", "SIRI-WHU", "EuroSAT", "METER-ML", "fMoW", "RSVQA-LR", "RSVQA-HR", "RSVG", "DIOR-RSVG", "LHRS-Bench"], "code_available": true, "code_url": "https://github.com/NJU-LHRS/LHRS-Bot", "weights_available": true, "key_contribution": "The paper introduces LHRS-Bot, a remote sensing multimodal LLM trained with a novel multi-level vision-language alignment strategy and curriculum learning. It also contributes large-scale RS-specific resources—LHRS-Align, LHRS-Instruct, and LHRS-Bench—built using global Google Earth imagery and OpenStreetMap VGI to improve RS image understanding and reasoning.", "_source_pdf": "LHRS-Bot_Empowering_Remote_Sensing_with_VGI-Enhanced_Large_Multimodal_Language_Model.pdf", "_extraction_timestamp": "2026-03-20T20:50:46"}
{"title": "Masked Angle-Aware Autoencoder for Remote Sensing Images", "year": 2024, "model_name": "MA3E", "architecture": {"backbone": "ViT-B", "type": "encoder-decoder", "params_millions": null, "notes": "Asymmetric MAE-style architecture. Uses a plain ViT-B encoder and an 8-block ViT decoder with 512-dimensional embeddings. Adds learnable angle embeddings to rotated-crop patches; separately masks rotated crop and background patches."}, "pretraining": {"method": "MAE-style masked image modeling with angle-aware reconstruction and optimal transport", "method_details": "MA3E introduces explicit angle variation by inserting a randomly rotated crop created via a scaling center crop operation into the input image, then reconstructs the original image. It adds angle embeddings for rotated regions and uses an optimal transport loss to match rotated-crop predictions with similar original patches, encouraging angle perception and rotation-invariant representation learning.", "objective": "masked image reconstruction plus angle restoration of rotated crops", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID"], "num_images": "990,848", "sensors": ["various sensors"], "modalities": ["RGB"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "rotated object detection", "semantic segmentation"], "downstream_datasets": ["NWPU-RESISC45", "AID", "UC Merced", "DOTA1.0", "DIOR-R", "iSAID", "Potsdam"], "code_available": true, "code_url": "https://github.com/benesakitam/MA3E", "weights_available": null, "key_contribution": "The paper proposes MA3E, a masked autoencoder tailored for remote sensing that explicitly models object orientation during self-supervised pretraining. By reconstructing original images from inputs containing randomly rotated crops and using an optimal transport loss for rotated-region reconstruction, it learns stronger rotation-invariant representations and improves performance across classification, rotated detection, and segmentation benchmarks.", "_source_pdf": "MA3E_Masked_Angle-Aware_Autoencoder_for_Remote_Sensing_Images.pdf", "_extraction_timestamp": "2026-03-20T20:51:05"}
{"title": "Towards Knowledge Guided Pretraining Approaches for Multimodal Foundation Models: Applications in Remote Sensing", "year": 2024, "model_name": "KG-VSF", "architecture": {"backbone": "ViT + unidirectional LSTM + causal transformer", "type": "encoder-decoder", "params_millions": null, "notes": "Heavy encoder and lightweight decoder. A shared Vision Transformer encodes multi-temporal Sentinel-2 image patches; a sequence-to-sequence unidirectional LSTM encodes weather time series; day-of-year and delta-time embeddings are added; a multimodal sequence encoder uses forward-only (causal) attention across time; a forecaster predicts future image embeddings conditioned on weather up to the forecast date; a lightweight shared MLP decoder reconstructs/forecasts spectral imagery. Hidden size 256, patch size 8."}, "pretraining": {"method": "Knowledge-guided variable-step forecasting", "method_details": "Proposes forecasting as a conditional generation task that explicitly encodes causal driver-response structure: weather variables act as drivers and satellite imagery as response. Pretraining is done in two phases: (1) masked reconstruction to learn modality-specific and multimodal representations, then (2) forecasting to inject causal cross-modal dynamics into the embedding space.", "objective": "Forecast future satellite imagery from past satellite imagery plus weather up to the future date", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Global Dataset", "US Cropland Dataset"], "num_images": "Global: 50k train / 10k val / 10k test samples; US Cropland: 60k train / 20k val / 20k test samples", "sensors": ["Sentinel-2", "ERA5-Land"], "modalities": ["multispectral", "weather", "temporal"], "spatial_resolution": "Sentinel-2 10m; ERA5-Land 11km", "geographic_coverage": "global and contiguous US croplands"}, "downstream_tasks": ["pixel-wise crop type mapping", "soil moisture estimation", "soil moisture forecasting", "missing image prediction", "future image forecasting"], "downstream_datasets": ["USDA Cropland Data Layer (CDL)", "SMAP"], "code_available": true, "code_url": null, "weights_available": true, "key_contribution": "The paper introduces KG-VSF, a self-supervised multimodal pretraining objective that embeds domain knowledge of causal relationships by conditioning future satellite-image prediction on weather drivers. It shows that this knowledge-guided forecasting pretraining yields better embeddings than masked reconstruction or standard forecasting for remote sensing tasks where weather-driven land-surface dynamics matter.", "_source_pdf": "MM-VSF_Towards_Knowledge_Guided_Pretraining_Approaches_for_Multimodal_Foundation_Models_in_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:51:27"}
{"title": "MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining", "year": 2024, "model_name": "MTP", "architecture": {"backbone": "ViT-L + RVSA", "type": "encoder-decoder", "params_millions": 305, "notes": "Shared backbone encoder with task-specific decoders for semantic segmentation, instance segmentation, and rotated object detection. Evaluated with RVSA-based vision transformers and InternImage-XL CNNs; primary large variant is ViT-L + RVSA. Uses UperNet, Mask R-CNN, and Oriented R-CNN as task heads during multi-task pretraining, and keeps only the backbone for downstream finetuning."}, "pretraining": {"method": "multi-task supervised pretraining", "method_details": "A stage-wise pretraining paradigm applied on top of existing pretrained backbones (e.g., MAE or ImageNet-pretrained models) to reduce task discrepancy between pretraining and downstream remote sensing tasks. It jointly optimizes semantic segmentation, instance segmentation, and rotated object detection using a shared encoder and multiple task-specific decoders.", "objective": "multi-task supervised learning with semantic segmentation, instance segmentation, and rotated object detection losses", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["SAMRS", "SOTA", "SIOR", "FAST"], "num_images": "105,090", "sensors": null, "modalities": ["optical imagery", "segmentation labels", "instance masks", "bounding boxes", "rotated bounding boxes"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "horizontal object detection", "rotated object detection", "semantic segmentation", "change detection"], "downstream_datasets": ["EuroSAT", "RESISC-45", "Xview", "DIOR", "DIOR-R", "FAIR1M-2.0", "DOTA-V1.0", "DOTA-V2.0", "SpaceNetv1", "LoveDA", "OSCD", "WHU", "LEVIR", "SVCD/CDD"], "code_available": true, "code_url": "https://github.com/ViTAE-Transformer/MTP", "weights_available": true, "key_contribution": "The paper proposes Multi-Task Pretraining (MTP), a stage-wise supervised pretraining framework for remote sensing foundation models that jointly learns semantic segmentation, instance segmentation, and rotated object detection on SAMRS. It shows that this multi-task pretraining substantially improves transfer across 14 remote sensing benchmarks and supports both large vision transformers and CNN backbones.", "_source_pdf": "MTP_Advancing_Remote_Sensing_Foundation_Model_via_Multi-Task_Pretraining.pdf", "_extraction_timestamp": "2026-03-20T20:51:52"}
{"title": "MetaEarth: A Generative Foundation Model for Global-scale Remote Sensing Image Generation", "year": 2024, "model_name": "MetaEarth", "architecture": {"backbone": "U-Net-like diffusion backbone with RRDBNet conditional image encoder", "type": "conditional diffusion, cascaded/self-cascading generative model", "params_millions": 600, "notes": "A unified diffusion model is shared across cascading stages for multi-resolution generation. The denoising network is a U-Net-like architecture with 5 encoder and 5 decoder blocks, 4 rounds of 2x down/up-sampling, channel multipliers [1,2,4,8,8], 3 ResBlocks per block, and AttentionBlocks at 8x channel multiplier stages. Conditional low-resolution images are encoded by RRDBNet and fused with noisy inputs; spatial resolution and timestep are embedded via frequency encoding plus MLP."}, "pretraining": {"method": "diffusion", "method_details": "MetaEarth introduces a resolution-guided self-cascading framework that recursively generates images from low to high resolution using a single shared model across stages, plus a novel noise sampling strategy for continuous unbounded generation. Training also uses high-order degradation to build robust low-/high-resolution pairs.", "objective": "conditional denoising diffusion noise prediction for remote sensing image generation", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["Custom Google Earth global multi-resolution dataset"], "num_images": "about 3.1M", "sensors": ["Google Earth"], "modalities": ["optical", "RGB", "geolocation metadata", "spatial resolution metadata"], "spatial_resolution": "4m, 16m, 64m", "geographic_coverage": "global"}, "downstream_tasks": ["remote sensing image classification", "image generation", "data augmentation"], "downstream_datasets": ["Custom 7-class remote sensing scene classification dataset"], "code_available": true, "code_url": "https://jiupinjia.github.io/metaearth/", "weights_available": null, "key_contribution": "The paper proposes MetaEarth, a 600M-parameter generative foundation model that enables global-scale remote sensing image generation with controllable multi-resolution and arbitrary-size unbounded outputs. Its main innovations are a resolution-guided self-cascading diffusion framework and a noise sampling strategy that makes large tiled generations spatially continuous, while also serving as a useful data engine for downstream tasks.", "_source_pdf": "MetaEarth_A_Generative_Foundation_Model_for_Global-Scale_Remote_Sensing_Image_Generation.pdf", "_extraction_timestamp": "2026-03-20T20:52:11"}
{"title": "OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery", "year": 2024, "model_name": "OReole-FM", "architecture": {"backbone": "ViT-e (3B)", "type": "encoder-decoder", "params_millions": 3067, "notes": "Family of vanilla Vision Transformers paired with an MAE decoder. Primary/largest variant is ViT-e with width 2816, depth 32, MLP 11264, 32 attention heads, using 14x14 patches; decoder follows original MAE. Variants include OReole-MR and OReole-HR."}, "pretraining": {"method": "MAE", "method_details": "Masked autoencoding with vanilla ViTs, used to systematically study billion-scale model scaling for remote sensing. The paper also introduces a transfer initialization strategy for 4-band models by inflating pretrained 3-band RGB weights with a randomly initialized 4th band, and studies the interaction of model scaling with data scaling.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID", "ORBITaL-Net", "TIU"], "num_images": "MillionAID: 1M+; ORBITaL-Net: ~126k; TIU: ~170k", "sensors": ["GeoEye-1", "QuickBird-2", "WorldView-2", "WorldView-3"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "mixed; MillionAID: 0.5-153 m/px, ORBITaL-Net: ~0.47 m/px, TIU: 0.3-0.8 m/px", "geographic_coverage": "global for MillionAID; ORBITaL-Net spans North America, Asia, Africa, South America, and the Middle East; TIU covers Taiwan, India, and Ukraine"}, "downstream_tasks": ["scene classification", "object detection", "oriented object detection", "semantic segmentation", "building footprint extraction"], "downstream_datasets": ["UCM", "AID", "NWPU-RESISC45", "DIOR", "DIOR-R", "Potsdam", "LoveDA", "ORBITaL-Net"], "code_available": true, "code_url": "https://geoai.ornl.gov", "weights_available": true, "key_contribution": "The paper introduces OReole, a family of billion-parameter remote sensing foundation models based on vanilla ViTs pretrained with MAE, and studies their scaling behavior on high-resolution satellite imagery using HPC resources including Frontier. It also contributes the TIU pretraining dataset and shows that effective model scaling in remote sensing requires corresponding data scaling, alongside a practical 3-band-to-4-band weight inflation strategy for multispectral initialization.", "_source_pdf": "OReole-FM_successes_and_challenges_toward_billion-parameter_foundation_models_for_high-resolution_satellite_imagery.pdf", "_extraction_timestamp": "2026-03-20T20:52:21"}
{"title": "OlmoEarth: StableLatentImageModeling for Multimodal Earth Observation", "year": 2025, "model_name": "OlmoEarth", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": 300, "notes": "Vision Transformer-based spatio-temporal multimodal encoder-decoder. Uses a FlexiViT-style projection layer with variable patch size, 2D sinusoidal positional embeddings, sinusoidal temporal embeddings, and learnable modality embeddings. The primary/largest variant uses a 24-layer encoder with hidden dim 1024 and 16 heads; decoder has matching feature dimension and heads but depth 4, with cross-attention from masked tokens to visible encoder tokens."}, "pretraining": {"method": "Latent MIM Lite with contrastive objectives", "method_details": "A simplified latent masked image modeling approach where target tokens are produced by randomly initialized, frozen linear projections from image patches instead of a learned target encoder, improving training stability and avoiding representation collapse. It also introduces modality-aware masking, modality-restricted patch discrimination, and an instance contrastive loss across two differently masked views of the same sample. Supervised map layers and observational modalities are unified by projecting both through the same frozen random projection into token space.", "objective": "masked latent token prediction with modality-aware patch discrimination contrastive loss plus instance-level contrastive alignment", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-1", "Sentinel-2", "Landsat-8", "WorldCereal", "OpenStreetMap", "WorldCover", "Cropland Data Layer", "SRTM", "Canopy Height Map", "olmoearth_pretrain_dataset"], "num_images": "285,288 samples", "sensors": ["Sentinel-1", "Sentinel-2", "Landsat-8", "SRTM"], "modalities": ["SAR", "multispectral", "time series", "maps", "DEM", "canopy height", "land cover", "cropland"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "land cover classification", "crop type classification", "semantic segmentation", "flood segmentation", "marine debris segmentation", "ecosystem classification", "forest loss driver classification", "live fuel moisture content regression", "mangrove classification", "marine infrastructure classification", "object detection", "vessel type classification", "vessel length regression", "solar farm segmentation"], "downstream_datasets": ["m-bigearthnet", "m-so2sat", "m-brick-kiln", "m-forestnet", "m-eurosat", "m-cashewplant", "m-SA-crop-type", "BreizhCrops", "CropHarvest-PRC", "CropHarvest-Togo", "PASTIS", "MADOS", "Sen1Floods11", "AWF", "Nandi", "GEA North Africa", "Forest Loss Driver", "Mangrove", "Marine Infrastructure", "Vessel Detection", "Vessel Length", "Vessel Type", "Solar Farm Detection", "Live Fuel Moisture Content"], "code_available": true, "code_url": "https://github.com/allenai/olmoearth_pretrain", "weights_available": true, "key_contribution": "The paper introduces OlmoEarth, a multimodal spatio-temporal Earth observation foundation model trained with a new stable self-supervised formulation called Latent MIM Lite. Its main innovations are frozen random latent targets, modality-aware masking, and modality-aware contrastive losses, yielding strong transfer and fine-tuning performance across a broad set of remote sensing benchmarks and real-world partner tasks.", "_source_pdf": "OlmoEarth_Stable_Latent_Image_Modeling_for_Multimodal_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T20:52:35"}
{"title": "OmniSat: Self-Supervised Modality Fusion for Earth Observation", "year": 2024, "model_name": "OmniSat", "architecture": {"backbone": "Modality-specific CNN image encoders + LTAE temporal encoders + transformer-based modality combiner", "type": "encoder-decoder", "params_millions": 10, "notes": "Uses dedicated encoders/decoders per modality: CNN-based encoder-decoder for images with max-pool index bypass to decoder (SegNet-style unpooling), and Lightweight Temporal Attention Encoder (LTAE) for optical/SAR time series. Modalities are fused with a transformer combiner composed of 6 residual self-attention blocks followed by a cross-attention block using learned modality-combining tokens. Embedding dimension d=256. Reported parameter counts: image encoder 3.6M, image decoder 1.8M, optical TS encoder 0.403M, optical TS decoder 0.096M, radar TS encoder 0.402M, radar TS decoder 0.095M, combiner 3.6M."}, "pretraining": {"method": "hybrid contrastive + masked autoencoding", "method_details": "OmniSat jointly applies patch-level cross-modal contrastive learning and multimodal masked reconstruction on aligned EO modalities. Its novelty is to exploit georeferenced alignment to contrast corresponding patches across modalities, exclude nearby same-tile negatives to avoid false negatives in spatially regular EO data, and reconstruct masked modality-specific inputs from fused multimodal patch representations. It also filters optical time-series reconstruction to the top 25% most informative dates based on LTAE attention to mitigate cloud contamination.", "objective": "cross-modal patch alignment with InfoNCE-style contrastive loss plus masked multimodal reconstruction of modality-specific inputs", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["TreeSatAI", "TreeSatAI-TS", "PASTIS-HD", "FLAIR"], "num_images": "approximately 50k TreeSatAI-TS tiles, 2.4k PASTIS-HD time series tiles, and 77k FLAIR tiles", "sensors": ["aerial imagery", "SPOT 6-7", "Sentinel-2", "Sentinel-1"], "modalities": ["RGB", "NIR", "multispectral", "SAR", "time series", "very high resolution imagery", "nDSM"], "spatial_resolution": "mixed (0.2 m, 1 m, 10 m)", "geographic_coverage": "France and Germany"}, "downstream_tasks": ["tree species classification", "crop type classification", "land cover classification", "forestry", "crop mapping"], "downstream_datasets": ["TreeSatAI", "TreeSatAI-TS", "PASTIS-HD", "FLAIR"], "code_available": true, "code_url": "https://github.com/gastruc/OmniSat", "weights_available": null, "key_contribution": "The paper proposes OmniSat, a self-supervised multimodal EO fusion architecture that combines heterogeneous aligned modalities such as VHR imagery, optical time series, and SAR time series through joint cross-modal contrastive learning and masked reconstruction. It also introduces two augmented multimodal EO datasets, TreeSatAI-TS and PASTIS-HD, and shows state-of-the-art results in semi- and fully supervised classification while improving even unimodal inference through multimodal pretraining.", "_source_pdf": "OmniSat_Self-Supervised_Modality_Fusion_for_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T20:52:46"}
{"title": "Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation", "year": 2025, "model_name": "Panopticon", "architecture": {"backbone": "ViT-Base", "type": "encoder-only self-distillation transformer", "params_millions": 98.1, "notes": "Built on DINOv2 with a ViT-B backbone (embedding dim 768). Replaces the standard patch embedding with a flexible channel-wise patch embedding: shared per-channel patchifier followed by cross-attention over channels using a single learned query. Uses spectral embeddings for optical wavelengths and learned SAR embeddings for transmit/receive polarization and orbit. Channel-attention dimension is 2304; patch embedding module has 12.9M parameters."}, "pretraining": {"method": "DINOv2-style self-supervised distillation with iBOT", "method_details": "Extends DINOv2 by treating co-registered images of the same geographic footprint from different sensors as natural augmented views, adding spectral channel subsampling as augmentation, and using a cross-attention-over-channels patch embedding to support arbitrary channel combinations across multispectral, hyperspectral, and SAR sensors. Also uses two-stage spectral progressive pretraining: first lower spectral diversity, then all sensors/datasets.", "objective": "self-distillation between multi-view crops of the same footprint plus patch-wise iBOT objective", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW", "fMoW-Sentinel", "MMEarth", "SatlasPretrain", "SpectralEarth"], "num_images": "approx. 2.5M unique footprints/samples", "sensors": ["QuickBird-1", "GeoEye-2", "WorldView-2", "WorldView-3", "Sentinel-2", "Sentinel-1", "Landsat 9", "EnMAP"], "modalities": ["RGB", "multispectral", "hyperspectral", "SAR", "thermal"], "spatial_resolution": "mixed (0.3m to 100m)", "geographic_coverage": "global"}, "downstream_tasks": ["scene/classification", "semantic segmentation", "multi-label classification", "regression", "cross-sensor generalization", "spectral invariance evaluation", "scale invariance evaluation"], "downstream_datasets": ["SpaceNet 1", "fMoW", "EuroSAT", "BrickKiln", "EnMAP-Corine", "RESISC45", "reBEN-S2", "reBEN-S1", "ForestNet", "So2Sat", "PV4Ger", "PV4Ger (segmentation)", "Chesapeake Landcover", "Cashew Plantation", "SA Crop Type", "NZ Cattle", "NEON Tree", "Hyperview", "TropicalCyclone", "DigitalTyphoon", "Corine [SuperDove]", "Corine [MODIS]", "Hyperview [SuperDove]", "Hyperview [MODIS]"], "code_available": true, "code_url": "https://github.com/Panopticon-FM/panopticon", "weights_available": true, "key_contribution": "The paper introduces Panopticon, an any-sensor Earth observation foundation model that can ingest arbitrary multispectral, hyperspectral, and SAR channel configurations without sensor-specific adaptation at inference. By extending DINOv2 with multi-sensor footprint-based augmentations, spectral channel subsampling, and channel cross-attention with sensor-aware embeddings, it achieves strong state-of-the-art results and superior generalization to unseen sensors and resolutions.", "_source_pdf": "Panopticon_Advancing_Any-Sensor_Foundation_Models_for_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T20:53:11"}
{"title": "Lightweight, Pre-trained Transformers for Remote Sensing Timeseries", "year": 2023, "model_name": "Presto", "architecture": {"backbone": "Transformer", "type": "encoder-decoder", "params_millions": 0.81, "notes": "Primary pretrained model uses a lightweight masked autoencoder with identical encoder and decoder configurations: 2 attention layers, 8 heads, embedding size 128, MLP ratio 4. The full encoder-decoder has 0.81M parameters, while the encoder alone used downstream is about 0.40M parameters. Operates on remote-sensing pixel-timeseries tokens grouped by sensor/channel type rather than image patches."}, "pretraining": {"method": "MAE", "method_details": "Presto is a masked autoencoding approach specialized for multi-sensor remote sensing pixel-timeseries. It uses structured masking over random values, channel groups, contiguous timesteps, and timesteps to reconstruct missing observations across time and modalities, making it robust to missing sensors/timestamps and suitable for diverse remote sensing inputs.", "objective": "masked reconstruction of continuous and categorical remote sensing timeseries tokens across sensors and time", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom global pretraining dataset constructed following the Dynamic World sampling strategy", "Sentinel-1", "Sentinel-2", "ERA5", "Dynamic World", "SRTM digital elevation model"], "num_images": "21.5M pixel samples", "sensors": ["Sentinel-1", "Sentinel-2", "ERA5", "SRTM"], "modalities": ["SAR", "multispectral", "climate/reanalysis", "categorical land cover", "topography/DEM", "coordinates", "derived indices", "temporal"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["crop type segmentation", "scene classification", "multi-label tree species classification", "regression", "crop type classification"], "downstream_datasets": ["CropHarvest", "S2-Agri100", "TreeSatAI", "EuroSAT", "live fuel moisture dataset", "Tick Tick Bloom harmful algal bloom detection challenge dataset"], "code_available": true, "code_url": "https://github.com/nasaharvest/presto", "weights_available": true, "key_contribution": "The paper introduces Presto, a lightweight self-supervised transformer tailored to remote sensing pixel-timeseries that jointly leverages temporal structure, multiple sensors, static metadata, and structured masking. It achieves strong performance across diverse globally distributed downstream tasks while using far fewer parameters and FLOPs than prior large vision-based remote sensing foundation models.", "_source_pdf": "Presto_Lightweight_Pre-trained_Transformers_for_Remote_Sensing_Timeseries.pdf", "_extraction_timestamp": "2026-03-20T20:53:29"}
{"title": "Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications", "year": 2024, "model_name": "Prithvi-EO-2.0", "architecture": {"backbone": "ViT-H", "type": "encoder-decoder", "params_millions": 600, "notes": "Masked autoencoder with asymmetric ViT encoder-decoder; primary model uses 3D patch embeddings via Conv3D over spatiotemporal inputs, 3D sinusoidal positional embeddings, and optional temporal/location metadata embeddings (year, day-of-year, latitude, longitude) added as learned weighted biases with metadata dropout. Largest variant is based on ViT-H; also includes smaller ViT-L 300M variant."}, "pretraining": {"method": "MAE", "method_details": "Extends masked autoencoding to multi-temporal Earth observation by jointly modeling spatial and temporal dimensions with 3D patch/position embeddings and incorporating optional time and geolocation metadata embeddings. Uses sequences of four images sampled months apart over a decade to emphasize seasonal and long-term dynamics.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["NASA Harmonized Landsat Sentinel-2 (HLS)"], "num_images": "4.2M", "sensors": ["Landsat 8", "Landsat 9", "Sentinel-2A", "Sentinel-2B"], "modalities": ["multispectral", "temporal", "geolocation metadata", "time metadata"], "spatial_resolution": "30m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation", "flood mapping", "wildfire scar mapping", "burn intensity mapping", "landslide detection", "crop segmentation", "land cover classification", "crop type classification", "above-ground biomass estimation", "gross primary productivity estimation"], "downstream_datasets": ["GEO-Bench", "m-bigearthnet", "m-so2sat", "m-brick-kiln", "m-forestnet", "m-eurosat", "m-pv4ger", "m-pv4ger-seg", "m-chesapeake", "m-cashew-plant", "m-SA-crop-type", "m-nz-cattle", "m-NeonTree", "Sen1Floods11", "hls_burn_scars", "Burned Area Emergency Response (BAER)", "Landslide4Sense", "Sen4Map", "PASTIS", "BioMassters", "FLUXNET", "AmeriFlux", "ICOS"], "code_available": true, "code_url": "https://github.com/NASA-IMPACT/Prithvi-EO-2.0", "weights_available": true, "key_contribution": "The paper introduces Prithvi-EO-2.0, a large multi-temporal Earth observation foundation model pretrained with masked autoencoding on 4.2M global HLS time-series samples using explicit spatiotemporal modeling and optional temporal/geolocation embeddings. It demonstrates strong transfer across a wide range of remote sensing tasks and resolutions, outperforming prior Prithvi and several competing geospatial foundation models in GEO-Bench and SME-led applications.", "_source_pdf": "Prithvi-EO-20_A_Versatile_Multi-Temporal_Foundation_Model_for_Earth_Observation_Applications.pdf", "_extraction_timestamp": "2026-03-20T20:53:42"}
{"title": "Foundation Models for Generalist Geospatial Artificial Intelligence", "year": 2023, "model_name": "Prithvi", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": 100, "notes": "Transformer-based masked autoencoder adapted for multi-temporal, multispectral satellite imagery. Uses asymmetric MAE with ViT backbone, 3D patch embedding via 3D convolutions, and 3D sine-cosine positional encoding. Input size 224x224 with patch size 1x16x16 (time x x x y). Largest/primary open-sourced model is Prithvi-100M."}, "pretraining": {"method": "MAE", "method_details": "Self-supervised masked autoencoding on spatiotemporal multispectral satellite data. Novel aspects include 3D positional embeddings and 3D patch embeddings to handle two spatial dimensions plus time for irregular, low-frequency satellite observations.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Harmonized Landsat Sentinel-2 (HLS)"], "num_images": "more than 1TB of imagery", "sensors": ["Landsat 8", "Landsat 9", "Sentinel-2A", "Sentinel-2B"], "modalities": ["multispectral", "temporal"], "spatial_resolution": "30m", "geographic_coverage": "contiguous United States"}, "downstream_tasks": ["multi-temporal cloud gap imputation", "flood mapping", "wildfire scar segmentation", "multi-temporal crop segmentation"], "downstream_datasets": ["Harmonized Landsat Sentinel-2 (HLS) cloud gap imputation dataset", "Sen1Floods11", "Monitoring Trends in Burn Severity (MTBS)", "USDA Crop Data Layer (CDL)", "multi-temporal crop segmentation dataset"], "code_available": true, "code_url": "https://github.com/NASA-IMPACT/hls-foundation-os", "weights_available": true, "key_contribution": "The paper introduces Prithvi, a 100M-parameter geospatial foundation model and end-to-end framework for scalable pretraining and fine-tuning on raw multispectral, multi-temporal satellite imagery. It shows that MAE-style self-supervised pretraining on HLS data improves fine-tuning speed, data efficiency, and competitive performance across diverse Earth observation tasks such as cloud gap imputation, flood mapping, wildfire scar mapping, and crop segmentation.", "_source_pdf": "Prithvi_Foundation_Models_for_Generalist_Geospatial_Artificial_Intelligence.pdf", "_extraction_timestamp": "2026-03-20T20:53:52"}
{"title": "Prithvi WxC: Foundation Model for Weather and Climate", "year": 2024, "model_name": "Prithvi WxC", "architecture": {"backbone": "Transformer-based vision transformer", "type": "encoder-decoder", "params_millions": 2300, "notes": "2D scalable/flexible ViT that alternates local attention within windows and global attention across windows, inspired by Hiera and MaxViT. Final model uses 25 encoder blocks and 5 decoder blocks, hidden size 2560, 16 attention heads, MLP ratio 4. Uses 2x2 token size, 30x32 pixel windows, Fourier positional encoding, learned lead-time/input-time embeddings, and Swin-shift in decoder during pretraining; Swin-shift also added to encoder in forecasting phase."}, "pretraining": {"method": "mixed masked reconstruction + forecasting", "method_details": "Pretraining combines MAE-style masking with forecasting on reanalysis data, using variable lead times and input time deltas. The target is anomaly relative to climatology rather than raw state or tendency, enabling unified handling of zero-lead and non-zero-lead tasks. Training uses both local and global masking strategies and a two-phase protocol, with a second phase focused on forecasting via dense autoregressive rollout tuning.", "objective": "masked reconstruction and forecasting of climatology-normalized atmospheric state anomalies", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MERRA-2"], "num_images": null, "sensors": ["MERRA-2 reanalysis"], "modalities": ["multivariate gridded atmospheric fields", "temporal", "static geophysical variables", "climatology"], "spatial_resolution": "0.5 x 0.625 degrees", "geographic_coverage": "global"}, "downstream_tasks": ["autoregressive rollout forecasting", "downscaling", "gravity wave flux parameterization", "extreme events estimation", "hurricane track forecasting", "masked reconstruction"], "downstream_datasets": ["MERRA-2", "HURDAT", "EURO-CORDEX", "CORDEX-CMIP5-RCP8.5", "ERA5", "WeatherBench2"], "code_available": true, "code_url": "https://github.com/NASA-IMPACT/Prithvi-WxC", "weights_available": true, "key_contribution": "The paper introduces Prithvi WxC, a 2.3B-parameter foundation model for weather and climate built on MERRA-2 and pretrained with a novel mixed objective that unifies masked reconstruction and forecasting through climatology-normalized anomaly prediction. It shows that a single pretrained model can be adapted across diverse weather/climate tasks including forecasting, downscaling, hurricane tracking, and gravity-wave parameterization, with released code and checkpoints.", "_source_pdf": "Prithvi_WxC_Foundation_Model_for_Weather_and_Climate.pdf", "_extraction_timestamp": "2026-03-20T20:54:05"}
{"title": "RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing", "year": 2023, "model_name": "GeoRSCLIP", "architecture": {"backbone": "CLIP ViT-H/14", "type": "CLIP-style dual-encoder", "params_millions": null, "notes": "Primary model is a CLIP model fine-tuned on RS5M for remote sensing. Experiments include CLIP ViT-B/32, ViT-B/16, ViT-L/14, and ViT-H/14, with the largest/primary reported variant being ViT-H/14. They also evaluate parameter-efficient tuning variants (Pfeiffer, LoRA, Prefix-tuning, UniPELT), but full fine-tuning gives the best main results."}, "pretraining": {"method": "CLIP contrastive fine-tuning", "method_details": "The paper proposes a Domain Vision-Language Model (DVLM) framework that adapts a general VLM (CLIP) to the remote sensing domain using the newly constructed RS5M image-text dataset. The novelty is mainly in domain adaptation at scale plus the RS5M dataset construction pipeline, including filtering web image-text pairs and generating captions for label-only remote sensing datasets.", "objective": "contrastive image-text alignment (InfoNCE)", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["RS5M", "LAION2B-en", "LAION400M", "LAIONCOCO", "COYO700M", "CC3M", "CC12M", "YFCC15M", "WIT", "Redcaps", "SBU", "Visual Genome", "BigEarthNet", "Functional Map of the World (FMoW)", "MillionAID"], "num_images": "5M", "sensors": ["Sentinel-1", "Sentinel-2", "MODIS", "Landsat", "WorldView", "Planet Dove", "aerial"], "modalities": ["RGB", "multispectral", "text", "geospatial metadata"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["zero-shot classification", "remote sensing cross-modal text-image retrieval", "semantic localization"], "downstream_datasets": ["AID", "RESISC45", "EuroSAT", "RSICD", "RSITMD", "AIR-SLT"], "code_available": true, "code_url": "https://github.com/om-ai-lab/RS5M", "weights_available": true, "key_contribution": "The paper introduces RS5M, the first large-scale remote sensing image-text paired dataset with about 5 million pairs, built by filtering web-scale image-text corpora and captioning large remote sensing image datasets. It also presents GeoRSCLIP, a CLIP-based domain vision-language model adapted to remote sensing, showing strong gains on zero-shot classification, cross-modal retrieval, and semantic localization.", "_source_pdf": "RS5M_and_GeoRSCLIP_A_Large_Scale_Vision-Language_Dataset_and_A_Large_Vision-Language_Model_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:54:17"}
{"title": "RSGPT: A Remote Sensing Vision Language Model and Benchmark", "year": 2023, "model_name": "RSGPT", "architecture": {"backbone": "EVA-G + Vicuna-13B", "type": "vision-language encoder-bridge-decoder with frozen image encoder and LLM connected by instruction-aware Q-Former", "params_millions": null, "notes": "Built on InstructBLIP. Uses a frozen EVA-G image encoder and frozen Vicuna LLM (vicuna7b/vicuna13b mentioned), with an instruction-aware Q-Former and a linear projection layer as trainable components. The primary/largest variant mentioned uses Vicuna-13B."}, "pretraining": {"method": "instruction tuning / domain adaptation on top of InstructBLIP", "method_details": "RSGPT is initialized from InstructBLIP pretrained weights and adapted to remote sensing by fine-tuning only the Q-Former and linear projection layer on the human-annotated RSICap dataset, keeping the image encoder and LLM frozen for data-efficient alignment.", "objective": "image-text alignment and instruction-following generation for remote sensing image captioning and visual question answering", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["RSICap", "DOTA-v1.5"], "num_images": "2,585", "sensors": ["GF-2", "JL-1", "Google Earth", "aerial"], "modalities": ["image", "text", "RGB", "panchromatic"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["image captioning", "visual question answering"], "downstream_datasets": ["RSIEval", "UCM-Captions", "Sydney-Captions", "RSICD", "RSVQA-HR", "RSVQA-LR"], "code_available": true, "code_url": "https://github.com/Lavender105/RSGPT", "weights_available": null, "key_contribution": "The paper introduces RSGPT, a remote sensing vision-language model adapted from InstructBLIP using parameter-efficient fine-tuning on a new high-quality human-annotated caption dataset, RSICap. It also releases RSIEval, a benchmark for remote sensing image captioning and VQA, and shows that RSGPT outperforms prior VLM baselines and achieves strong results on several remote sensing benchmarks.", "_source_pdf": "RSGPT_A_Remote_Sensing_Vision_Language_Model_and_Benchmark.pdf", "_extraction_timestamp": "2026-03-20T20:54:26"}
{"title": "RemoteCLIP: A Vision Language Foundation Model for Remote Sensing", "year": 2023, "model_name": "RemoteCLIP", "architecture": {"backbone": "ViT-Large-14", "type": "CLIP-style dual-encoder", "params_millions": 428, "notes": "Primary/largest model uses a ViT-L-14 image encoder (304M params, 24 layers, 16 heads, 14x14 patches) and a Transformer text encoder (124M params, 12 layers, 8 heads, max length 77). The model is initialized from CLIP and continually pretrained for remote sensing."}, "pretraining": {"method": "CLIP-style contrastive pretraining", "method_details": "The paper performs continual pretraining of CLIP on a newly scaled remote sensing image-text corpus created by unifying heterogeneous annotations. It introduces Box-to-Caption (B2C) to convert detection boxes into captions and Mask-to-Box (M2B) to convert segmentation masks into boxes before caption generation, enabling much larger in-domain image-text pretraining.", "objective": "bidirectional image-text contrastive alignment with InfoNCE loss", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["RSICD", "RSITMD", "UCMerced", "AUAIR", "CARPK", "DIOR", "DOTA", "HRRSD", "HRSC", "LEVIR", "RSOD", "Stanford Drone", "Visdrone", "iSAID", "LoveDA", "Potsdam", "Vaihingen"], "num_images": "165,745 images (828,725 image-text pairs)", "sensors": ["satellite", "UAV"], "modalities": ["RGB", "text"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["zero-shot image classification", "linear probing", "k-NN classification", "few-shot classification", "image-to-text retrieval", "text-to-image retrieval", "object counting"], "downstream_datasets": ["RSITMD", "RSICD", "UCMerced", "RemoteCount", "PatternNet", "EuroSAT", "OPTIMAL-31", "RSC11", "AID", "MLRSNet", "RSI-CB128", "RSI-CB256", "RESISC45", "WHU-earth", "WHU-RS19", "RS2800"], "code_available": true, "code_url": "https://github.com/ChenDelong1999/RemoteCLIP", "weights_available": true, "key_contribution": "The paper introduces RemoteCLIP, the first general-purpose vision-language foundation model for remote sensing, built by continual pretraining of CLIP on a large in-domain image-text corpus. Its main novelty is a data scaling pipeline that converts detection and segmentation annotations into captions via B2C and M2B, enabling strong zero-shot, retrieval, few-shot, and counting performance across 16 datasets.", "_source_pdf": "RemoteCLIP_A_Vision_Language_Foundation_Model_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:54:47"}
{"title": "Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach", "year": 2021, "model_name": "Contrastive Sensor Fusion (CSF)", "architecture": {"backbone": "ResNet-50", "type": "encoder-only Siamese contrastive model", "params_millions": null, "notes": "Shared-weight Siamese ResNet encoder trained on multi-sensor inputs with random channel/sensor dropout. Contrastive loss is applied at multiple layers, specifically the last layers of the final two residual stacks. Inputs from three optical sensors are upsampled to a common 0.5 m resolution and concatenated depth-wise into a 12-channel input."}, "pretraining": {"method": "contrastive", "method_details": "CSF creates two augmented views of the same scene by randomly dropping channels/sensors and applying spatial and photometric augmentations, then trains a single shared encoder so representations remain similar across any subset of available sensor channels. It extends Contrastive Multiview Coding by using arbitrary channel dropout instead of fixed view partitions, weight sharing across views, and multi-layer contrastive loss.", "objective": "contrastive alignment of representations from two views of the same coterminous multi-sensor scene using InfoNCE", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom coterminous image triplets from Airbus SPOT", "NAIP", "Airbus Pléiades"], "num_images": "47M", "sensors": ["Airbus SPOT", "NAIP", "Airbus Pléiades"], "modalities": ["RGB", "near-infrared"], "spatial_resolution": "mixed (0.5 m, 1.0 m, 1.5 m; resampled to 0.5 m)", "geographic_coverage": "contiguous United States over the intersection of sensor coverage"}, "downstream_tasks": ["scene classification", "representation quality evaluation via k-nearest neighbors classification", "unsupervised clustering/semantic grouping"], "downstream_datasets": ["custom OpenStreetMap-based classification dataset"], "code_available": true, "code_url": "https://storage.cloud.google.com/public-published-datasets/csf_code.zip", "weights_available": null, "key_contribution": "The paper introduces Contrastive Sensor Fusion, a self-supervised contrastive learning method that learns sensor-invariant representations from any subset of co-registered remote sensing channels using random channel dropout and a shared encoder. Trained on 47 million unlabeled multi-sensor scenes, CSF produces semantically meaningful fused representations that outperform ImageNet-supervised features on a remote sensing classification benchmark.", "_source_pdf": "Representation_Learning_for_Remote_Sensing_An_Unsupervised_Sensor_Fusion_Approach.pdf", "_extraction_timestamp": "2026-03-20T20:54:56"}
{"title": "RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing", "year": 2025, "model_name": "RoMA", "architecture": {"backbone": "Mamba-B", "type": "encoder-decoder autoregressive", "params_millions": 85, "notes": "Uses a standard Mamba architecture following ARM without modifying the backbone. The primary model is Mamba-B with 12 layers, width 768, and Mamba+MLP blocks. Pretraining uses a Mamba encoder over all image patches plus an autoregressive decoder/cross-attention style next-token prediction with KV-cache-based prediction. Larger variant mentioned is Mamba-L (297M), but the main reported model is Mamba-B."}, "pretraining": {"method": "autoregressive self-supervised pretraining", "method_details": "RoMA introduces Rotation-aware Multi-scale Autoregressive learning for remote sensing. It combines an adaptive rotation encoding strategy using feature-based adaptive crop selection and learnable angle embeddings, with a multi-scale token prediction objective to better model arbitrary object orientations, sparse foregrounds, and large scale variation in remote sensing imagery.", "objective": "next-token prediction with multi-scale reconstruction loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["OpticalRS-4M"], "num_images": "4M", "sensors": ["optical"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["scene classification", "change detection", "semantic segmentation"], "downstream_datasets": ["AID", "UCM", "OSCD", "SpaceNetv1", "MillionAID", "iSAID"], "code_available": true, "code_url": "RoMA", "weights_available": true, "key_contribution": "The paper proposes RoMA, the first self-supervised autoregressive pretraining framework for Mamba-based remote sensing foundation models, designed to scale efficiently to high-resolution imagery. Its key innovations are adaptive rotation-aware encoding and multi-scale prediction, which improve representation learning for arbitrarily oriented, sparsely distributed, and scale-varying objects while delivering better accuracy and efficiency than ViT-based counterparts.", "_source_pdf": "RoMA_Scaling_up_Mamba-based_Foundation_Models_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T20:55:05"}
{"title": "S2MAE: A Spatial-Spectral Pretraining Foundation Model for Spectral Remote Sensing Data", "year": 2024, "model_name": "S2MAE", "architecture": {"backbone": "ViT-Huge", "type": "encoder-decoder", "params_millions": 632, "notes": "Masked autoencoder with a plain ViT encoder and lightweight ViT decoder using 3D spatial-spectral patching and random 3D masking. Uses non-overlapping 3D cube tokens (primary setting: patch size 8x8x3), separable learnable spectral-spatial positional embeddings, and a high masking ratio of 90%. Multiple scales are studied (ViT-Base, ViT-Large, ViT-Huge); primary/largest variant reported is ViT-Huge."}, "pretraining": {"method": "MAE", "method_details": "Extends masked autoencoding to spectral remote sensing imagery with 3D random masking over spatial and spectral dimensions, designed to capture local spectral continuity and spatial invariance while handling arbitrary band counts. Also uses progressive pretraining across multiple Sentinel-2 datasets with different image sizes and geographic coverage.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW-S2", "BigEarthNet"], "num_images": "over 1M", "sensors": ["Sentinel-2"], "modalities": ["multispectral"], "spatial_resolution": "mixed", "geographic_coverage": "global + Europe"}, "downstream_tasks": ["single-label classification", "multi-label classification", "change detection"], "downstream_datasets": ["EuroSAT", "BigEarthNet", "OSCD"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes S2MAE, a spectral remote sensing foundation model that adapts masked autoencoding to multispectral imagery using 3D spatial-spectral masking and learnable spectral-spatial embeddings. It shows that high-mask-ratio 3D MAE pretraining, combined with progressive pretraining on large Sentinel-2 datasets, improves performance on classification and change detection tasks.", "_source_pdf": "S2MAE_A_Spatial-Spectral_Pretraining_Foundation_Model_for_Spectral_Remote_Sensing_Data.pdf", "_extraction_timestamp": "2026-03-20T20:55:13"}
{"title": "S4: Self-Supervised Sensing Across the Spectrum", "year": 2024, "model_name": "S4", "architecture": {"backbone": "3D U-Net", "type": "encoder-decoder", "params_millions": null, "notes": "Uses modality-specific first two convolutional layers for radar and optical encoders, followed by two shared convolutional layers. Encoders fr and fo are 3D U-Net backbones with convolution, batch normalization, max-pooling, and leaky ReLU operating jointly over temporal and spatial dimensions. A decoder h is used for cross-modal reconstruction and later adapted for segmentation. A small projection head g with two 1x1x1 3D convolutions is used only during contrastive pretraining."}, "pretraining": {"method": "multi-task self-supervised learning with pixel-wise contrastive learning and cross-modal reconstruction", "method_details": "S4 jointly optimizes a novel multi-modal spatio-temporal (MMST) pixel-wise contrastive loss and a cross-modal reconstruction loss on spatially aligned radar-optical satellite image time series. Positive pairs are corresponding space-time pixels across modalities, while negatives are mismatched pixels across space or time. It also reconstructs one modality from the other, leveraging georegistered alignment and temporal interpolation across modalities.", "objective": "pixel-wise cross-modal spatio-temporal alignment via InfoNCE plus cross-modal image time-series reconstruction via L1 loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["m2s2-SITS"], "num_images": "PASTIS-R region: 5314 time series, 731k Sentinel-1 images, 90k Sentinel-2 images; Africa region: 5941 time series, 193k Sentinel-1 images, 70k Sentinel-2 images", "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["SAR", "multispectral", "RGB", "temporal"], "spatial_resolution": null, "geographic_coverage": "France (PASTIS-R region) and South Sudan/Africa crop type mapping region"}, "downstream_tasks": ["semantic segmentation", "satellite image time series segmentation", "crop type mapping"], "downstream_datasets": ["PASTIS-R", "Africa Crop Type Mapping"], "code_available": true, "code_url": "https://github.com/microsoft/farmvibes-ai", "weights_available": null, "key_contribution": "The paper introduces S4, a self-supervised pretraining framework for multimodal satellite image time series that exploits cross-spectrum spatial alignment between radar and optical imagery using a joint pixel-wise spatio-temporal contrastive loss and cross-modal reconstruction. It also curates m2s2-SITS, a large aligned unlabeled pretraining dataset, and shows improved SITS semantic segmentation under limited-label settings with single-modality inference.", "_source_pdf": "S4_Self-Supervised_Sensing_Across_the_Spectrum.pdf", "_extraction_timestamp": "2026-03-20T20:55:25"}
{"title": "SARATR-X: Toward Building A Foundation Model for SAR Target Recognition", "year": 2024, "model_name": "SARATR-X", "architecture": {"backbone": "HiViT-B", "type": "encoder-decoder", "params_millions": 66.4, "notes": "Uses a hierarchical vision transformer (HiViT) as the backbone within a masked autoencoder-style masked image modeling framework. HiViT was chosen for high spatial resolution and better handling of small SAR targets. The paper also compares against ViT and ConvNeXt-V2, but the primary model is HiViT-Base."}, "pretraining": {"method": "MAE-style masked image modeling with two-step self-supervised pretraining", "method_details": "The method first pretrains on ImageNet with masked image modeling to obtain initialization weights, then continues pretraining on SAR data. During SAR pretraining, it reconstructs multi-scale gradient features instead of raw pixels to suppress speckle noise and improve feature diversity. The two-step strategy is designed to improve attention diversity and model scalability under noisy SAR imagery.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["ImageNet-1K", "MSTAR", "Sandia MiniSAR", "SARSim", "SAMPLE", "SIVED", "OpenSARShip", "SAR-Ship", "AIR-SARShip", "HRSID", "SSDD", "SADD", "SAR-AIRcraft", "MSAR", "OGSOD"], "num_images": "about 1.4M ImageNet images plus 186,600 SAR samples (SARDet-180K); abstract also states 0.18M unlabeled SAR target samples", "sensors": ["ImageNet natural images", "X-band SAR", "Ku-band SAR", "C-band SAR", "Ka-band SAR", "Sentinel-1", "Gaofen-3", "HISEA-1", "TerraSAR-X", "TanDEM-X", "RadarSat-2", "airborne SAR", "satellite SAR", "simulated SAR"], "modalities": ["RGB", "SAR"], "spatial_resolution": "mixed (about 0.1 m to 25 m for SAR; ImageNet mixed natural image resolution)", "geographic_coverage": null}, "downstream_tasks": ["few-shot target classification", "robust target classification", "object detection", "ship detection", "aircraft detection"], "downstream_datasets": ["SAR-VSA", "MSTAR", "SARDet-100K", "OGSOD", "SSDD", "SAR-AIRcraft"], "code_available": true, "code_url": "https://github.com/waterdisappear/SARATR-X", "weights_available": true, "key_contribution": "The paper presents SARATR-X, described as the first foundation model for SAR automatic target recognition, trained with self-supervised learning on a large curated SAR target corpus. Its main novelty is a SAR-tailored two-step masked image modeling scheme that transfers from ImageNet pretraining and reconstructs multi-scale gradient features to better handle SAR speckle noise and support label-efficient classification and detection.", "_source_pdf": "SARATR-X_Toward_Building_a_Foundation_Model_for_SAR_Target_Recognition.pdf", "_extraction_timestamp": "2026-03-20T20:55:51"}
{"title": "SARVLM: A Vision Language Foundation Model for Semantic Understanding and Target Recognition in SAR Imagery", "year": 2025, "model_name": "SARVLM", "architecture": {"backbone": "ViT-L-14", "type": "CLIP-style dual-encoder", "params_millions": 304, "notes": "SARVLM comprises SARCLIP and SARCap. SARCLIP is built on the OpenCLIP framework with image backbones including ResNet-50, ViT-B-32, and ViT-L-14; the primary/largest reported SARCLIP variant uses ViT-L-14. SARCap uses a CoCa encoder-decoder style image captioning architecture with a ViT-L-14 image encoder."}, "pretraining": {"method": "vision-language contrastive pretraining with domain transfer", "method_details": "The model is trained in two stages: first on optical remote sensing vision-language data to learn remote sensing semantics, then fine-tuned on the newly constructed SARVLM-1M SAR image-text corpus to transfer knowledge from optical remote sensing to SAR. SARCap further uses CoCa-style progressive caption training.", "objective": "image-text contrastive alignment (symmetric InfoNCE); for SARCap additionally image caption generation", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["SARVLM-1M", "MSTAR", "SARSim", "OpenSARShip", "SAMPLE", "ATRNet-STAR", "SARDet-100K", "FAIR-CSAR", "SARLANG-1M-Captions", "LoveDA", "DOTA", "RSCID", "RS5M", "SkyScript", "R3+D10+S4", "LAION", "HQRS-210K"], "num_images": "1.7M image-text pairs in SARVLM-1M; 279,996 training images and 1,385,993 training captions", "sensors": ["SAR", "Sentinel-1", "Gaofen-3", "X-band HH radar"], "modalities": ["SAR", "text"], "spatial_resolution": "mixed; includes 0.3 m and 1.124 × 1.728 m", "geographic_coverage": null}, "downstream_tasks": ["image-text retrieval", "target recognition / classification", "zero-shot classification", "semantic localization", "image captioning"], "downstream_datasets": ["SARVLM-1M test", "MSTAR-SOC", "FUSAR-Ship", "SAR-VSA", "AIR-SLT"], "code_available": true, "code_url": null, "weights_available": null, "key_contribution": "The paper introduces SARVLM, presented as the first vision-language foundation model specialized for SAR imagery, along with SARVLM-1M, a large-scale SAR image-text dataset containing about 1.7 million pairs. Its main novelty is a two-stage domain transfer training strategy that transfers semantic alignment from optical remote sensing data to SAR, improving retrieval, recognition, zero-shot classification, localization, and captioning.", "_source_pdf": "SARVLM_A_Vision_Language_Foundation_Model_for_Semantic_Understanding_and_Target_Recognition_in_SAR_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:56:02"}
{"title": "SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classification", "year": 2023, "model_name": "SS-MAE", "architecture": {"backbone": "Dual-attention Transformer + lightweight CNNs (IRB-based hybrid)", "type": "encoder-decoder", "params_millions": 4.4, "notes": "Two-branch masked autoencoder with a spatial-wise branch and a spectral-wise branch. Each branch uses a Transformer encoder-decoder with spatial or spectral attention. The training stage adds two lightweight CNNs with inverted residual blocks; for HSI the CNN uses 3D convolution, and for LiDAR/SAR it uses depth-wise convolution. Best settings reported include 2 Transformer blocks and token dimension 256."}, "pretraining": {"method": "MAE-style masked image modeling", "method_details": "Proposes joint spatial-wise and spectral-wise masking for multi-source remote sensing data. One branch masks spatial pixels/patches and reconstructs missing pixels, while the other masks entire spectral channels/bands and reconstructs missing channels. This is combined with a Transformer-CNN hybrid design to capture both global and local features.", "objective": "masked reconstruction of spatial pixels and spectral channels using MSE loss on masked regions", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Berlin", "Augsburg", "Houston 2018"], "num_images": "up to 200000 sampled patches/pixels for pretraining depending on dataset", "sensors": ["EnMAP simulated from HyMap", "HySpex", "Sentinel-1", "LiDAR", "multispectral LiDAR"], "modalities": ["hyperspectral", "SAR", "LiDAR"], "spatial_resolution": "30 m for Augsburg; mixed/matched by interpolation for Berlin; null for Houston 2018", "geographic_coverage": "Berlin, Germany; Augsburg, Germany; Houston, USA"}, "downstream_tasks": ["land cover classification", "multi-source image classification"], "downstream_datasets": ["Berlin", "Augsburg", "Houston 2018"], "code_available": true, "code_url": "https://github.com/summitgao/SS-MAE", "weights_available": true, "key_contribution": "The paper introduces SS-MAE, a spatial-spectral masked autoencoder for multi-source remote sensing classification that explicitly reconstructs both masked spatial regions and masked spectral bands. It further combines dual-attention Transformers with lightweight CNNs to jointly model global and local features, achieving strong results on HSI+SAR and HSI+LiDAR benchmarks.", "_source_pdf": "SS-MAE_Spatial-Spectral_Masked_Autoencoder_for_Multisource_Remote_Sensing_Image_Classification.pdf", "_extraction_timestamp": "2026-03-20T20:56:11"}
{"title": "SSL4EO-L: Datasets and Foundation Models for Landsat Imagery", "year": 2023, "model_name": "SSL4EO-L", "architecture": {"backbone": "ViT-S16", "type": "encoder-only", "params_millions": 22.46, "notes": "The paper pre-trains multiple backbones (ResNet-18, ResNet-50, ViT-S16) using contrastive SSL on separate Landsat sensor/product splits. The largest/primary backbone reported among these is ResNet-50 by parameter count (23.56M), but ViT-S16 is the main transformer backbone and is used prominently in results. Encoders are initialized from ImageNet weights adapted to multispectral inputs by repeating/scaling RGB weights. Downstream evaluation uses a frozen encoder with a U-Net decoder."}, "pretraining": {"method": "contrastive (MoCo v2 / SimCLR v1)", "method_details": "The paper extends Seasonal Contrast-style pretraining to Landsat by constructing a large global 4-season dataset across multiple Landsat sensors and product levels. Positive views are formed using two randomly sampled seasonal images from the same location ('random season contrast'), enabling multispectral, multitemporal contrastive pretraining for Landsat imagery.", "objective": "contrastive representation learning / instance discrimination with multiseason positive pairs", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-L"], "num_images": "5M", "sensors": ["Landsat 4-5 TM", "Landsat 7 ETM+", "Landsat 8-9 OLI/TIRS"], "modalities": ["multispectral", "thermal", "multitemporal"], "spatial_resolution": "30 m", "geographic_coverage": "global"}, "downstream_tasks": ["semantic segmentation", "cloud detection", "land cover classification", "crop classification"], "downstream_datasets": ["L7 Irish", "L8 Biome", "NLCD", "CDL"], "code_available": true, "code_url": "https://github.com/microsoft/torchgeo", "weights_available": true, "key_contribution": "The paper introduces SSL4EO-L, the first large-scale self-supervised pretraining dataset for Landsat imagery, comprising 5 million image patches across multiple sensors and product levels. It also releases the first Landsat foundation models, modernizes legacy benchmark datasets, and creates new benchmark datasets for TM and ETM+ SR to evaluate transfer on semantic segmentation tasks.", "_source_pdf": "SSL4EO-L_Datasets_and_Foundation_Models_for_Landsat_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:56:21"}
{"title": "SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation", "year": 2022, "model_name": "SSL4EO-S12", "architecture": {"backbone": "ResNet-50 / ViT-S/16", "type": "dataset benchmark for self-supervised pretraining", "params_millions": null, "notes": "The paper does not propose a single new neural architecture; it introduces a pretraining dataset and benchmarks existing SSL methods (MoCo-v2/v3, DINO, MAE, data2vec) using ResNet-50 and ViT-S/16 backbones."}, "pretraining": {"method": "self-supervised pretraining benchmark (contrastive, distillation, masked image modeling, feature prediction)", "method_details": "The novelty is not a new SSL loss but a large-scale EO-specific pretraining corpus with global coverage, multimodal Sentinel-1/2 data, four seasonal timestamps, overlap filtering, and inclusion of both Sentinel-2 L1C and L2A products. The paper benchmarks MoCo-v2/v3, DINO, MAE, and data2vec on this dataset, including RS-specific augmentations such as RandomSeasonContrast and multimodal training with RandomSensorDrop.", "objective": "masked image reconstruction / feature-level prediction / contrastive representation learning / self-distillation, depending on the benchmarked SSL method", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12"], "num_images": "about 3M images; about 1M Sentinel-1/2 triplets across 251,079 locations with 4 seasonal snapshots", "sensors": ["Sentinel-1 GRD", "Sentinel-2 L1C", "Sentinel-2 L2A"], "modalities": ["SAR", "multispectral", "temporal"], "spatial_resolution": "10m to 60m bands; patches are 2640m x 2640m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "land cover classification", "local climate zone classification", "semantic segmentation", "land cover segmentation", "change detection"], "downstream_datasets": ["EuroSAT", "BigEarthNet", "So2Sat-LCZ42", "DFC2020", "OSCD"], "code_available": true, "code_url": "https://github.com/zhu-xlab/SSL4EO-S12", "weights_available": true, "key_contribution": "The paper introduces SSL4EO-S12, a large-scale global Earth observation pretraining dataset with multimodal Sentinel-1/2 data and multi-seasonal coverage, designed for self-supervised learning. It shows that pretraining on this dataset consistently improves transfer performance across classification, semantic segmentation, and change detection compared with existing EO and ImageNet-based pretraining datasets.", "_source_pdf": "SSL4EO-S12_A_Large-Scale_Multi-Modal_Multi-Temporal_Dataset_for_Self-Supervised_Learning_in_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T20:56:33"}
{"title": "SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery", "year": 2022, "model_name": "SatMAE", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": null, "notes": "MAE-style asymmetric transformer with a ViT encoder and transformer decoder. Extended to temporal and multi-spectral satellite imagery via temporal/spectral positional encodings, independent masking across time or spectral groups, and separate patch embeddings for grouped spectral bands in the multi-spectral variant."}, "pretraining": {"method": "MAE", "method_details": "Adapts masked autoencoding to satellite imagery by adding temporal encodings from timestamp components (year, month, hour), spectral/group encodings for band groups, and masking strategies tailored to temporal sequences and multi-spectral bands. For multi-spectral data, bands can be grouped with separate patch embeddings per group.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW RGB", "fMoW Sentinel"], "num_images": "fMoW Sentinel: about 713k training images (712,874 train; 84,939 val; 84,966 test)", "sensors": ["high-resolution satellite imagery", "Sentinel-2"], "modalities": ["RGB", "multispectral", "temporal"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "temporal image-sequence classification", "land cover classification", "multi-label classification", "semantic segmentation", "building segmentation"], "downstream_datasets": ["fMoW RGB", "fMoW Sentinel", "NAIP", "EuroSAT", "BigEarthNet", "SpaceNet v1"], "code_available": true, "code_url": "https://sustainlab-group.github.io/SatMAE/", "weights_available": true, "key_contribution": "The paper introduces SatMAE, a masked autoencoder pretraining framework tailored for temporal and multi-spectral satellite imagery using temporal/spectral encodings and specialized masking strategies. It also releases fMoW-Sentinel and shows strong gains over prior remote sensing SSL methods across classification and segmentation benchmarks.", "_source_pdf": "SatMAE_Pre-training_Transformers_for_Temporal_and_Multi-Spectral_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:57:09"}
{"title": "Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery", "year": 2024, "model_name": "SatMAE++", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": null, "notes": "Built on the MAE/SatMAE framework with a ViT encoder-decoder and additional convolution-based upsampling blocks for multi-scale reconstruction. Uses standard sinusoidal positional encodings; for multispectral Sentinel-2 data it follows SatMAE channel grouping by GSD. Primary RGB setup uses 224x224 input with patch size 16; multispectral setup uses 96x96 input with patch size 8 and reconstructs at up to three scales."}, "pretraining": {"method": "MAE with multi-scale reconstruction", "method_details": "SatMAE++ revisits masked autoencoding for remote sensing by reconstructing the image not only at the base input scale but also at higher spatial scales using transpose-convolution upsampling blocks and residual convolution blocks. Unlike ScaleMAE, it does not rely on GSD-based positional encoding or a Laplacian pyramid decoder, making it applicable to both RGB and multispectral imagery.", "objective": "masked image reconstruction at the base scale with MSE plus higher-scale reconstruction with L1 loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW-RGB", "fMoW-Sentinel"], "num_images": "fMoW-RGB: ~363k train + 53k test; fMoW-Sentinel: 712874 train + 84939 val + 84966 test", "sensors": ["high-resolution RGB satellite imagery", "Sentinel-2"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "mixed; Sentinel-2 bands at 10m, 20m, and 60m GSD", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "land cover classification", "land use and land cover classification", "multi-label classification"], "downstream_datasets": ["fMoW-RGB", "fMoW-Sentinel", "EuroSAT", "RESISC-45", "UC-Merced", "BigEarthNet"], "code_available": true, "code_url": "https://github.com/techmn/satmae_pp", "weights_available": true, "key_contribution": "The paper proposes SatMAE++, a simple multi-scale masked autoencoder for remote sensing that augments SatMAE with convolutional upsampling-based reconstruction at higher resolutions. It shows that standard positional encodings plus multi-scale reconstruction are sufficient to learn stronger representations across both RGB and multispectral satellite imagery, achieving state-of-the-art results on six datasets.", "_source_pdf": "SatMAE_Rethinking_Transformers_Pre-training_for_Multi-Spectral_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:57:21"}
{"title": "SatMamba: Development of Foundation Models for Remote Sensing Imagery Using State Space Models", "year": 2025, "model_name": "SatMamba", "architecture": {"backbone": "Mamba 2", "type": "encoder-decoder", "params_millions": 229.86, "notes": "Masked autoencoder framework with Mamba-based encoder and decoder. Primary model is SatMamba-B with 12-layer encoder, 8-layer decoder, encoder embedding dim 768, decoder embedding dim 512, state dimension 64, and head dimension 96. Uses multi-way Mamba blocks with up to four scanning directions: row-major forward/backward and column-major forward/backward. Maintains natural patch order and optionally uses fixed positional encodings; best results reported without positional encodings."}, "pretraining": {"method": "MAE", "method_details": "Integrates State Space Models (Mamba 2) into a masked autoencoder for remote sensing. Unlike standard MAE with ViT, masking is applied before flattening, patch order is preserved rather than shuffled, and the encoder/decoder use multi-directional scanning Mamba blocks to model images as sequences with linear scaling in input length.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW"], "num_images": "416,614", "sensors": ["high-resolution satellite imagery"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["semantic segmentation", "building damage assessment", "building localization", "damage classification", "change detection"], "downstream_datasets": ["OpenEarthMap", "xBD"], "code_available": true, "code_url": "https://github.com/mdchuc/HRSFM", "weights_available": null, "key_contribution": "The paper proposes SatMamba, a remote sensing foundation model pretraining framework that replaces the usual ViT backbone in MAE with Mamba-based state space models for linear scaling with sequence length. It shows competitive or better performance than ViT-based MAE baselines on high-resolution image-to-image tasks such as semantic segmentation and building damage assessment.", "_source_pdf": "SatMamba_Development_of_Foundation_Models_for_Remote_Sensing_Imagery_Using_State_Space_Models.pdf", "_extraction_timestamp": "2026-03-20T20:57:32"}
{"title": "SatSwinMAE: Efficient Autoencoding for Multiscale Time-series Satellite Imagery", "year": 2024, "model_name": "SatSwinMAE", "architecture": {"backbone": "Swin-Base / Video Swin Transformer", "type": "encoder-decoder", "params_millions": null, "notes": "Hierarchical 3D masked autoencoder for multi-temporal satellite imagery. Uses 3D patch partitioning, Video Swin Transformer blocks, spatial-only patch merging/expanding, and a SwinUNet-like fine-tuning architecture with skip connections. A temporal modulation CNN head is added for downstream multi-temporal tasks. Pretraining uses both encoder and decoder weights for transfer."}, "pretraining": {"method": "MAE", "method_details": "Extends SwinMAE to satellite time series with a hierarchical 3D masked autoencoder using Video Swin Transformer blocks. Applies window masking on each spatial slice, replaces masked tokens with a learnable vector instead of removing them, preserves temporal dimension through the hierarchy, and transfers both encoder and decoder pretrained weights into a SwinUNet-like downstream model with skip connections.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12"], "num_images": "250,000 locations", "sensors": ["Sentinel-2 L1C"], "modalities": ["multispectral", "temporal"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["land cover segmentation", "building density prediction", "flood mapping", "wildfire scar mapping", "multi-temporal crop segmentation"], "downstream_datasets": ["PhilEO Bench", "Sen1Floods11", "MTBS historical fire database", "USDA Crop Data Layer (CDL)"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes SatSwinMAE, a Swin-based 3D masked autoencoder for multi-scale, multi-temporal satellite imagery that captures spatio-temporal dependencies more efficiently than ViT-based MAE models. By combining Video Swin pretraining with SwinUNet-style skip connections and temporal modulation for fine-tuning, it achieves state-of-the-art or better results across several geospatial dense prediction tasks.", "_source_pdf": "SatSwinMAE_Efficient_Autoencoding_for_Multiscale_Time-series_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:57:43"}
{"title": "SATVISION-TOA: A GEOSPATIAL FOUNDATION MODEL FOR COARSE-RESOLUTION ALL-SKY REMOTE SENSING IMAGERY", "year": 2024, "model_name": "SatVision-TOA", "architecture": {"backbone": "SwinV2 Giant", "type": "encoder-only", "params_millions": 3000, "notes": "Masked-image-modeling pretraining on 128x128x14 MODIS TOA chips using 8x8 masked patches. The primary model is SVTOA-Giant; SwinV2 Huge (658M) is also studied. The paper describes hierarchical windowed attention in SwinV2 and uses the pretrained backbone for downstream fine-tuning with an FCN decoder for 3D cloud retrieval."}, "pretraining": {"method": "masked image modeling", "method_details": "Pretrained with a SimMIM-style masked-image-modeling setup on 14-band MODIS Level 1B Top-Of-Atmosphere all-sky imagery, explicitly including clouds and atmospheric variability rather than cloud-free imagery. This targets moderate/coarse-resolution multispectral Earth observation and cross-sensor transfer to MODIS-like instruments.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MOD021KM MODIS/Terra Calibrated Radiances 5-Min L1B Swath 1km", "custom global daily composites of TERRA-MODIS TOA image chips"], "num_images": "100M", "sensors": ["MODIS Terra"], "modalities": ["multispectral", "TOA radiance", "reflectance", "brightness temperature"], "spatial_resolution": "1 km", "geographic_coverage": "global"}, "downstream_tasks": ["image reconstruction", "3D cloud retrieval"], "downstream_datasets": ["MODIS TOA held-out test chips", "GOES-ABI 14-channel image chips", "CloudSat/CALIPSO vertical cloud masks"], "code_available": true, "code_url": "https://github.com/nasa-nccs-hpda/pytorch-caney", "weights_available": true, "key_contribution": "The paper introduces SatVision-TOA, a 3B-parameter SwinV2-based self-supervised foundation model pretrained on 100 million global 14-band MODIS all-sky TOA image chips for moderate/coarse-resolution remote sensing. By learning from cloud-inclusive multispectral imagery, it establishes a large-scale atmospheric-aware geospatial foundation model and substantially improves downstream 3D cloud retrieval over an FCN baseline.", "_source_pdf": "SatVision-TOA_A_Geospatial_Foundation_Model_for_Coarse-Resolution_All-Sky_Remote_Sensing_Imagery.pdf", "_extraction_timestamp": "2026-03-20T20:57:55"}
{"title": "Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning", "year": 2023, "model_name": "Scale-MAE", "architecture": {"backbone": "ViT-Large", "type": "encoder-decoder", "params_millions": 322.9, "notes": "MAE-style ViT encoder with Ground Sample Distance Positional Encoding (GSDPE) and a progressive Laplacian-pyramid decoder. The decoder uses fewer transformer layers than vanilla MAE (3 instead of 8 in the main setting), followed by deconvolution-based upsampling and Laplacian Blocks to reconstruct separate low-frequency and high-frequency outputs at different scales."}, "pretraining": {"method": "MAE", "method_details": "A scale-aware masked autoencoding approach that injects absolute scale via GSD-based positional encoding and uses a Laplacian-pyramid decoder to reconstruct low-frequency and high-frequency targets at different scales. Unlike standard MAE, it explicitly learns cross-scale relationships during pretraining.", "objective": "masked image reconstruction with multiscale low-frequency and high-frequency reconstruction losses", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Functional Map of the World (FMoW)"], "num_images": "363.6k", "sensors": ["satellite", "aerial"], "modalities": ["RGB"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation"], "downstream_datasets": ["RESISC-45", "UC Merced Land Use Dataset", "AiRound", "CV-BrCT", "MLRSNet", "EuroSAT", "Optimal-31", "WHU-RS19", "FMoW-RGB", "SpaceNet v1", "SpaceNet v2", "INRIA Aerial Image", "GID-15"], "code_available": null, "code_url": "https://github.com/bair-climate-initiative/scale-mae", "weights_available": true, "key_contribution": "The paper proposes Scale-MAE, a self-supervised geospatial pretraining method that makes MAE scale-aware through GSD-based positional encoding and a Laplacian-pyramid decoder for multiscale reconstruction. This yields more robust representations across varying ground sample distances and improves remote sensing classification and building segmentation over prior MAE-based methods such as SatMAE and ConvMAE.", "_source_pdf": "Scale-MAE_A_Scale-Aware_Masked_Autoencoder_for_Multiscale_Geospatial_Representation_Learning.pdf", "_extraction_timestamp": "2026-03-20T20:58:16"}
{"title": "SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing", "year": 2024, "model_name": "SeaMo", "architecture": {"backbone": "ViT-Base", "type": "encoder-decoder", "params_millions": null, "notes": "Masked autoencoder-style architecture with modality-specific patch embedding modules for Sentinel-2 optical and Sentinel-1 SAR, a unified multimodal ViT encoder, Temporal-Multimodal (TM) fusion blocks using cross-attention across seasons and modalities, and modality-specific reconstruction decoders. After pretraining, only the encoder is used for downstream tasks."}, "pretraining": {"method": "MAE", "method_details": "SeaMo uses a progressive masked image modeling strategy: first single-time-point multimodal learning, then multi-time-flow multimodal learning with Temporal-Multimodal fusion blocks. Novel elements include unaligned/partially overlapping spatial region selection, unified multimodal encoding of optical+SAR, and season-aware temporal-multimodal fusion via cross-attention.", "objective": "masked image reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12"], "num_images": "3M", "sensors": ["Sentinel-2", "Sentinel-1"], "modalities": ["multispectral", "SAR", "temporal"], "spatial_resolution": null, "geographic_coverage": "global"}, "downstream_tasks": ["single-label classification", "multi-label classification", "semantic segmentation", "change detection"], "downstream_datasets": ["EuroSAT", "fMoW-Sentinel", "BigEarthNet", "DFC2020", "SegMunich", "OSCD", "EuroSAT-SAR", "BigEarthNet-SAR", "DFC2020-SAR", "GLH-Water", "S1S2-Water"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes SeaMo, a season-aware multimodal remote sensing foundation model that jointly exploits spatial, spectral, temporal, and cross-sensor information using masked image modeling. Its main innovations are progressive multimodal-to-seasonal pretraining, partially overlapping geospatial crop selection, and Temporal-Multimodal fusion blocks that explicitly model season-dependent multimodal relationships for improved transfer across optical and SAR tasks.", "_source_pdf": "SeaMo_A_Multi-Seasonal_and_Multimodal_Remote_Sensing_Foundation_Model.pdf", "_extraction_timestamp": "2026-03-20T20:58:37"}
{"title": "Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data", "year": 2021, "model_name": "SeCo", "architecture": {"backbone": "ResNet-50", "type": "encoder with multiple contrastive projection heads (multi-head contrastive SSL)", "params_millions": null, "notes": "Primary evaluations use ResNet-18 and ResNet-50 backbones with a MoCo-v2 style momentum encoder setup. The encoder is followed by 3 separate 2-layer MLP projection heads corresponding to embedding sub-spaces that are invariant to all augmentations, invariant to seasonal changes only, or invariant to synthetic augmentations only. The transfer representation is taken from the shared backbone embedding."}, "pretraining": {"method": "contrastive self-supervised learning (MoCo-v2-based, multi-embedding seasonal contrast)", "method_details": "SeCo exploits temporally aligned satellite images from the same location across seasons as natural positive pairs, while using multiple embedding sub-spaces to learn both season-invariant and season-variant features. Unlike standard contrastive learning that enforces a single invariance structure, SeCo disentangles temporal and synthetic augmentation invariances through separate projection heads and losses.", "objective": "contrastive alignment with InfoNCE over seasonal and artificial views", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SeCo dataset (custom uncurated Sentinel-2 dataset collected with Google Earth Engine)"], "num_images": "1M", "sensors": ["Sentinel-2"], "modalities": ["RGB", "multispectral", "temporal"], "spatial_resolution": "10m, 20m, 60m", "geographic_coverage": "global"}, "downstream_tasks": ["land-cover classification", "change detection"], "downstream_datasets": ["BigEarthNet", "EuroSAT", "OSCD"], "code_available": true, "code_url": "https://github.com/ElementAI/seasonal-contrast", "weights_available": true, "key_contribution": "The paper introduces SeCo, a self-supervised remote sensing pretraining pipeline that combines automatic collection of large-scale uncurated multi-temporal Sentinel-2 imagery with a multi-subspace contrastive objective. Its main novelty is learning representations that preserve both seasonal invariances and temporal variations, improving transfer on land-cover classification and change detection over ImageNet and prior SSL baselines.", "_source_pdf": "Seasonal_Contrast_Unsupervised_Pre-Training_From_Uncurated_Remote_Sensing_Data.pdf", "_extraction_timestamp": "2026-03-20T20:58:49"}
{"title": "Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding", "year": 2021, "model_name": "CMC-RSSR", "architecture": {"backbone": "ResNet-50", "type": "dual-encoder contrastive model", "params_millions": null, "notes": "Uses Contrastive Multiview Coding (CMC) with two ResNet-50 encoder networks processing two channel-split views of the same image, followed by linear projection heads and L2 normalization. For RGB, views are L* and ab* in Lab color space; for multispectral, views are defined either by spectral-band grouping or PCA-based channel decorrelation."}, "pretraining": {"method": "contrastive (CMC / Noise Contrastive Estimation)", "method_details": "Applies Contrastive Multiview Coding to remote sensing imagery by splitting channels into two complementary views and maximizing agreement between views from the same image while contrasting against views from other images. The paper also extends CMC to multispectral Sentinel-2 imagery using two view-construction strategies: wavelength-based band grouping and PCA-based decorrelated views.", "objective": "contrastive alignment between two channel-based views of the same image", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["ImageNet", "NWPU-RESISC45", "DOTA", "NWPU VHR-10", "NWPU-DOTA", "BigEarthNet"], "num_images": "1.2M ImageNet; 196,215 NWPU-DOTA; 269,695 BigEarthNet", "sensors": ["Sentinel-2", "aerial"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "mixed (0.08-60 m across datasets)", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "multi-label scene classification", "land use / land cover classification", "local climate zone classification"], "downstream_datasets": ["AID", "MLRSNet", "BigEarthNet", "So2Sat"], "code_available": true, "code_url": "https://github.com/vladan-stojnic/CMC-RSSR", "weights_available": true, "key_contribution": "The paper systematically evaluates contrastive self-supervised pretraining for remote sensing scene representation learning and shows that in-domain remote sensing pretraining can outperform supervised ImageNet pretraining when used as a feature extractor. It also extends CMC to multispectral Sentinel-2 data, demonstrating additional gains over RGB-only self-supervised pretraining.", "_source_pdf": "Self-Supervised_Learning_of_Remote_Sensing_Scene_Representations_Using_Contrastive_Multiview_Coding.pdf", "_extraction_timestamp": "2026-03-20T20:58:59"}
{"title": "Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks", "year": 2021, "model_name": "MATTER", "architecture": {"backbone": "ResNet-34", "type": "encoder-only", "params_millions": null, "notes": "ResNet-34 backbone with a parameter-free Texture Refinement Network (TeRN) inserted after the first layer, followed by a patch-wise Surface Residual Encoder adapted from Deep-TEN. Uses densely sampled small image crops/patches and 64 learned residual clusters."}, "pretraining": {"method": "contrastive self-supervised learning with residual cluster encoding", "method_details": "MATTER learns material and texture representations from multi-temporal, spatially aligned imagery over unchanged regions. It combines a Texture Refinement Network to amplify low-level texture features with a Surface Residual Encoder that encodes soft affinity to multiple learned surface clusters, and contrastively enforces consistency between residual representations of temporally paired unchanged patches.", "objective": "Noise Contrastive Estimation on patch-wise surface residual representations to make unchanged multi-temporal patches similar and patches from different regions dissimilar", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Custom curated multi-temporal Sentinel-2 unchanged-regions dataset"], "num_images": "14,857 tiles", "sensors": ["Sentinel-2"], "modalities": ["multispectral", "temporal"], "spatial_resolution": null, "geographic_coverage": "27 regions of interest spanning 1217 km2 across a variety of climates"}, "downstream_tasks": ["change detection", "land cover classification", "semantic segmentation", "building segmentation"], "downstream_datasets": ["Onera Satellite Change Detection (OSCD)", "BigEarthNet", "SpaceNet"], "code_available": true, "code_url": "https://github.com/periakiva/MATTER", "weights_available": null, "key_contribution": "The paper introduces MATTER, a self-supervised remote sensing pretraining method that explicitly learns material and texture representations using multi-temporal unchanged Sentinel-2 imagery. Its key novelty is combining low-level texture refinement with patch-wise residual cluster encoding to produce features that transfer strongly to change detection, land cover classification, and segmentation while converging much faster than prior self-supervised baselines.", "_source_pdf": "Self-Supervised_Material_and_Texture_Representation_Learning_for_Remote_Sensing_Tasks.pdf", "_extraction_timestamp": "2026-03-20T20:59:07"}
{"title": "Semantic Segmentation of Remote Sensing Images With Self-Supervised Multitask Representation Learning", "year": 2021, "model_name": "Self-Supervised Multitask Representation Learning", "architecture": {"backbone": "VGG16", "type": "encoder with multitask pretraining heads; triplet Siamese network for self-supervised pretraining", "params_millions": null, "notes": "Primary method uses a shared CNN backbone (experiments with VGG16 and ResNet50) with three branches for inpainting, augmentation transform prediction (ATP), and contrastive learning. The pretraining architecture is a triplet Siamese network with shared weights across input branches. For downstream semantic segmentation, five transposed convolution layers are added on top of the backbone."}, "pretraining": {"method": "multi-task self-supervised learning", "method_details": "The method jointly optimizes three pretext tasks to capture both low-level and high-level remote sensing features: image inpainting for low-level detail recovery, augmentation transform prediction (ATP) for recognizing applied transforms, and contrastive learning for invariant high-level representation learning. A multitask loss combines all three objectives in a shared triplet Siamese framework.", "objective": "masked/occluded image reconstruction, transformation classification, and contrastive instance discrimination", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["DIOR", "DOTA", "Levir"], "num_images": "186486", "sensors": ["aerial", "optical remote sensing"], "modalities": ["RGB"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["semantic segmentation", "cloud detection", "snow detection"], "downstream_datasets": ["Potsdam", "Vaihingen", "Levir_CS"], "code_available": true, "code_url": "https://github.com/flyakon/SSLRemoteSensing", "weights_available": null, "key_contribution": "The paper proposes a self-supervised multitask pretraining framework tailored for remote sensing semantic segmentation that jointly learns low-level and high-level representations using inpainting, augmentation transform prediction, and contrastive learning. It shows stronger segmentation performance than random initialization, ImageNet pretraining, and several prior self-supervised baselines, especially in low-label regimes.", "_source_pdf": "Semantic_Segmentation_of_Remote_Sensing_Images_With_Self-Supervised_Multitask_Representation_Learning.pdf", "_extraction_timestamp": "2026-03-20T20:59:18"}
{"title": "SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model", "year": 2024, "model_name": "SkyEyeGPT", "architecture": {"backbone": "EVA-CLIP ViT-G + LLaMA2-chat 7B", "type": "vision-language decoder-only MLLM with visual encoder + linear alignment layer + LLM decoder", "params_millions": 7000, "notes": "Uses frozen EVA-CLIP as visual encoder, a single linear alignment layer after concatenating every 4 adjacent visual tokens, and an LLaMA2-chat decoder. Fine-tuned with LoRA (rank 64) on the alignment layer and LLM. Input images are resized to 448x448. UAV video features are formed by concatenating features from multiple frame images. Initialized from MiniGPT-v2 checkpoint."}, "pretraining": {"method": "instruction tuning", "method_details": "Rather than proposing a new foundation pretraining objective, the paper builds a remote-sensing multimodal instruction-following dataset (SkyEye-968k) and performs two-stage tuning: first on single-task image-text instructions for RS image-text alignment, then on multi-task conversational instructions to improve multi-round, multi-task dialogue. Task-specific identifiers such as [caption], [vqa], [refer], and [detection] are used to unify diverse RS tasks.", "objective": "autoregressive next-token prediction for instruction-following over multimodal inputs", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["SkyEye-968k", "RSICD", "RSITMD", "UCM-Captions", "Sydney-Captions", "NWPU-Captions", "CapERA", "ERA-VQA", "RSIVQA", "RSVQA-LR", "RSVQA-HR", "RSVG", "DIOR-RSVG", "RSPG", "DOTA-Conversa", "DIOR-Conversa", "UCM-Conversa", "Sydney-Conversa", "DIOR", "DOTA", "event recognition in aerial videos (ERA)"], "num_images": "968k samples", "sensors": ["aerial", "satellite", "UAV video"], "modalities": ["RGB", "text", "video/temporal", "bounding boxes"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["image captioning", "UAV video captioning", "visual question answering", "visual grounding", "phrase grounding", "referring expression generation", "scene classification", "multi-task visual dialogue"], "downstream_datasets": ["UCM-Captions", "Sydney-Captions", "RSICD", "CapERA", "RSVQA-LR", "RSVQA-HR", "RSVG", "DIOR-RSVG", "AID"], "code_available": true, "code_url": null, "weights_available": true, "key_contribution": "The paper introduces SkyEyeGPT, a unified remote sensing multimodal large language model that handles diverse open-ended RS vision-language tasks within a single decoder-only framework using a simple linear visual-language connector. It also contributes SkyEye-968k, a manually verified 968k-sample remote sensing instruction-tuning dataset, and a two-stage tuning strategy that improves both task performance and multi-turn conversational ability.", "_source_pdf": "SkyEyeGPT_Unifying_Remote_Sensing_Vision-Language_Tasks_via_Instruction_Tuning_with_Large_Language_Model.pdf", "_extraction_timestamp": "2026-03-20T20:59:31"}
{"title": "SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing", "year": 2023, "model_name": "SkyCLIP", "architecture": {"backbone": "ViT-L/14", "type": "CLIP-style dual-encoder", "params_millions": null, "notes": "Continual pre-training of CLIP on SkyScript. Primary reported model is SkyCLIP-50 with a ViT-L/14 image encoder initialized from OpenAI CLIP; the paper also evaluates a ViT-B/32 variant. Uses image-text contrastive learning."}, "pretraining": {"method": "CLIP contrastive continual pre-training", "method_details": "The paper's main novelty is constructing SkyScript, a 2.6M-pair remote sensing vision-language dataset by linking Google Earth Engine imagery with OpenStreetMap semantics via geo-coordinates, then continually pre-training CLIP on this dataset. It uses automatically assembled single-object and multi-object captions and filters noisy pairs using CLIP similarity.", "objective": "image-text contrastive alignment", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["SkyScript"], "num_images": "2.6M image-text pairs", "sensors": ["SWISSIMAGE 10 cm RGB imagery", "Spain RGB orthophotos", "Brandenburg (Germany) RGBN orthophotos", "Finland RGB NLS orthophotos", "National Agriculture Imagery Program", "Planet SkySat Public Ortho Imagery, RGB", "Planet SkySat Public Ortho Imagery, MS", "Harmonized Sentinel-2 MSI, Level-2A", "Landsat 8 Collection 2 Tier 1 TOA Reflectance", "Landsat 9 Collection 2 Tier 1 TOA Reflectance"], "modalities": ["RGB", "text"], "spatial_resolution": "0.1 m to 30 m", "geographic_coverage": "global"}, "downstream_tasks": ["zero-shot scene classification", "zero-shot fine-grained attribute classification", "cross-modal retrieval"], "downstream_datasets": ["SkyScript-classification", "AID", "EuroSAT", "fMoW", "Million-AID", "PatternNet", "NWPU-RESISC45", "RSI-CB256", "SkyScript-retrieval", "RSICD", "RSITMD", "UCM-Captions"], "code_available": true, "code_url": "https://github.com/wangzhecheng/SkyScript", "weights_available": true, "key_contribution": "The paper introduces SkyScript, a large-scale and semantically diverse remote sensing vision-language dataset built by geospatially aligning open remote sensing imagery with OpenStreetMap tags. Using SkyScript for continual CLIP pre-training yields SkyCLIP, which improves zero-shot scene classification, fine-grained attribute classification, and cross-modal retrieval in remote sensing.", "_source_pdf": "SkyScript_A_Large_and_Semantically_Diverse_Vision-Language_Dataset_for_Remote_Sensing.pdf", "_extraction_timestamp": "2026-03-20T21:39:25"}
{"title": "Multi-label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining", "year": 2024, "model_name": "SoftCon", "architecture": {"backbone": "ViT-Base/14", "type": "Siamese momentum encoder with encoder-only backbone for contrastive pretraining", "params_millions": 86, "notes": "Primary model uses MoCo-v3-style ViT-B/14 with two separate projectors for contrastive loss and soft contrastive loss. Continual pretraining initializes from DINOv2 weights; for non-RGB EO inputs the input embedding layer is randomly initialized. A frozen momentum encoder is updated by EMA, and Siamese masking masks 20% of patches on the trainable branch. The paper also studies ResNet-50 and ViT-S/14 variants, and both multispectral and SAR models."}, "pretraining": {"method": "soft contrastive learning with continual pretraining", "method_details": "The paper extends supervised contrastive learning to multi-label EO scenes by computing soft pairwise similarities from multi-hot land-cover labels and matching them with feature similarities via a soft contrastive loss. This is combined with standard contrastive loss, plus cross-domain continual pretraining from strong vision foundation models (DINO/DINOv2) and Siamese masking for efficient ViT training.", "objective": "contrastive instance discrimination plus soft similarity alignment based on multi-label land-cover supervision", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12-ML", "SSL4EO-S12", "Dynamic World"], "num_images": "780,371", "sensors": ["Sentinel-2", "Sentinel-1"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["multi-label scene classification", "scene classification", "semantic segmentation", "change detection"], "downstream_datasets": ["BigEarthNet", "BigEarthNet-SAR", "EuroSAT", "EuroSAT-SAR", "fMoW-sentinel", "DFC2020", "OSCD", "m-so2sat", "m-brick-kiln", "m-cashew-plantation", "m-SA-crop-type"], "code_available": true, "code_url": "https://github.com/zhu-xlab/softcon", "weights_available": true, "key_contribution": "The paper proposes SoftCon, a multi-label guided soft contrastive learning framework that uses land-cover-derived scene-level multi-label annotations to model soft semantic similarity between EO images. Combined with simple continual pretraining from strong vision foundation models and Siamese masking, it produces efficient multispectral and SAR EO foundation models that achieve state-of-the-art results on 10 of 11 downstream tasks.", "_source_pdf": "SoftCon_Multi-Label_Guided_Soft_Contrastive_Learning_for_Efficient_Earth_Observation_Pretraining.pdf", "_extraction_timestamp": "2026-03-20T21:39:49"}
{"title": "SpectralEarth: Training Hyperspectral Foundation Models at Scale", "year": 2024, "model_name": "SpectralEarth", "architecture": {"backbone": "Spectral ViT-L", "type": "encoder-only", "params_millions": null, "notes": "Primary architecture family consists of Spectral ResNet-50 and Spectral ViT variants (ViT-S/B/L/H/g) augmented with a spectral adapter. The spectral adapter uses three 1D Conv+BN+ReLU layers (kernel sizes 7, 7, 5; strides 5, 5, 3) to process hyperspectral bands and produce 128-channel features. For ViT, the adapter is placed before patch embedding and uses 4x4 patches to preserve fine spatial detail. Largest primary variant discussed is Spectral ViT-L pretrained with MAE."}, "pretraining": {"method": "multi-method SSL: MAE, DINO, and MoCo-V2", "method_details": "The paper pretrains hyperspectral foundation models on the new SpectralEarth dataset using standard SSL methods adapted to HSI via a lightweight spectral adapter inserted into classical 2D backbones. Temporal overlapping EnMAP acquisitions are additionally used as positive pairs for joint-embedding methods (MoCo-V2, DINO).", "objective": "masked image reconstruction for MAE; self-distillation/invariant representation learning for DINO; contrastive representation learning for MoCo-V2", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SpectralEarth"], "num_images": "538,974 patches", "sensors": ["EnMAP"], "modalities": ["hyperspectral", "temporal"], "spatial_resolution": "30m", "geographic_coverage": "global"}, "downstream_tasks": ["multi-label land-cover classification", "crop-type segmentation", "land-cover segmentation", "tree-species segmentation", "soil parameter regression", "above-ground biomass regression"], "downstream_datasets": ["EnMAP-CORINE", "EnMAP-CDL", "EnMAP-NLCD", "EnMAP-TreeMap", "EnMAP-BDForet", "EnMAP-EuroCrops", "EnMAP-BNETD", "DESIS-CDL", "EO1-CDL", "Hyperview", "HyBiomass"], "code_available": true, "code_url": "https://github.com/AABNassim/spectral_earth", "weights_available": true, "key_contribution": "The paper introduces SpectralEarth, a large-scale globally distributed EnMAP hyperspectral pretraining dataset with 538,974 patches, including multi-temporal coverage, and uses it to train hyperspectral foundation models with SSL. It also proposes a simple spectral adapter for adapting standard ResNet/ViT backbones to HSI and provides extensive benchmarks across nine hyperspectral downstream datasets, showing improved transferability and efficient fine-tuning.", "_source_pdf": "SpectralEarth_Training_Hyperspectral_Foundation_Models_at_Scale.pdf", "_extraction_timestamp": "2026-03-20T21:40:01"}
{"title": "TOV: The Original Vision Model for Optical Remote Sensing Image Understanding via Self-supervised Learning", "year": 2022, "model_name": "TOV", "architecture": {"backbone": "ResNet-50", "type": "encoder-only", "params_millions": null, "notes": "Contrastive self-supervised pretraining with a 2-layer MLP projection head; downstream adapters are task-specific: fully connected layer for scene classification, Faster R-CNN components (RPN + ROIHead) for object detection, and FCN decoder for semantic segmentation. Uses layer freezing in stage 2 as a memory retention strategy."}, "pretraining": {"method": "contrastive self-supervised learning", "method_details": "A human-like two-stage SSL learning path: first pretrain on a large natural image dataset to learn general visual knowledge, then continue pretraining on remote sensing imagery to learn domain-specialized knowledge. To reduce catastrophic forgetting, shallow and middle layers learned in stage 1 are fixed during stage 2.", "objective": "contrastive alignment of augmented views of the same image versus views from different images", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["TOV-NI", "TOV-RS-imbalanced", "TOV-RS-balanced"], "num_images": "1M natural images + 0.5M remote sensing images for the main balanced setting", "sensors": ["Google Earth", "web natural images"], "modalities": ["RGB"], "spatial_resolution": "1-20 m for TOV-RS-balanced; mixed overall", "geographic_coverage": "global, covering more than 100 countries"}, "downstream_tasks": ["scene classification", "object detection", "semantic segmentation"], "downstream_datasets": ["AID", "NWPU-RESISC45", "RSD46", "PatternNet", "UC Merced", "EuroSAT", "NaSC-TG2", "DOTA", "Levir", "DLRSD", "DGLCC", "Potsdam"], "code_available": true, "code_url": "https://github.com/GeoX-Lab/G-RSIM/tree/main/TOV_v1", "weights_available": true, "key_contribution": "The paper introduces TOV, a general optical remote sensing vision model trained in a label-free, task-independent way using contrastive self-supervised learning. Its main novelty is a human-like two-stage pretraining path from natural images to remote sensing images, combined with automated geographically guided dataset construction, yielding strong transfer across scene classification, object detection, and semantic segmentation.", "_source_pdf": "TOV_The_Original_Vision_Model_for_Optical_Remote_Sensing_Image_Understanding_via_Self-supervised_Learning.pdf", "_extraction_timestamp": "2026-03-20T21:40:24"}
{"title": "TerraMind: Large-Scale Generative Multimodality for Earth Observation", "year": 2025, "model_name": "TerraMind", "architecture": {"backbone": "Transformer-based encoder-decoder", "type": "encoder-decoder", "params_millions": null, "notes": "Symmetric Transformer-based encoder-decoder inspired by 4M; supports dual-scale early fusion of token-level and pixel-level inputs. Uses modality-specific tokenizers for image-like modalities with ViT-B encoders, finite-scalar quantization (FSQ), and diffusion decoders; captions and coordinates are tokenized with a modified WordPiece tokenizer. Primary large variant is TerraMindv1-L."}, "pretraining": {"method": "masked multimodal modeling", "method_details": "Two-stage pretraining: first train unimodal modality-specific tokenizers, then pretrain TerraMind on both token-level and pixel-level inputs using dual-scale early fusion. The model reconstructs randomly sampled masked target tokens across modalities, framing pretraining as cross-modal patch-level classification. Novel aspects are any-to-any multimodal generation for EO and joint learning from token abstractions plus raw pixel patches.", "objective": "masked target token reconstruction with cross-entropy loss; autoregressive decoding for sequence-like generation", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["TerraMesh", "SSL4EO-S12", "MajorTOM-Core", "Copernicus DEM 30m", "ESRI Land Use Land Cover", "SEnSeI v2"], "num_images": "9M", "sensors": ["Sentinel-2", "Sentinel-1", "Copernicus DEM"], "modalities": ["multispectral", "SAR", "RGB", "DEM", "NDVI", "LULC", "text", "geolocation"], "spatial_resolution": "10m", "geographic_coverage": "global"}, "downstream_tasks": ["semantic segmentation", "water body mapping", "zero-shot segmentation", "geo-localization", "few-shot classification", "scene classification", "multimodal generation"], "downstream_datasets": ["PANGAEA", "BurnScars", "MADOS", "PASTIS", "Sen1Floods11", "FiveBillionPixels", "DynamicEarthNet", "CTM-SS", "SpaceNet7", "AI4Farms", "EuroSAT", "METER-ML", "South Africa Crop Type Mapping dataset"], "code_available": true, "code_url": "https://github.com/ibm/terramind", "weights_available": true, "key_contribution": "The paper introduces TerraMind, the first any-to-any generative multimodal foundation model for Earth observation, trained with a dual-scale token-level and pixel-level masked modeling strategy on a global multimodal dataset. It shows strong generation, zero-shot, few-shot, and fine-tuning performance, and proposes Thinking-in-Modalities (TiM), where generated modalities are fed back during fine-tuning or inference to improve downstream results.", "_source_pdf": "TerraMind_Large-Scale_Generative_Multimodality_for_Earth_Observation.pdf", "_extraction_timestamp": "2026-03-20T21:40:35"}
{"title": "The color out of space: learning self-supervised representations for Earth Observation imagery", "year": 2021, "model_name": "Colorization pre-training for Earth Observation imagery", "architecture": {"backbone": "ResNet-18", "type": "encoder-decoder", "params_millions": null, "notes": "Uses a ResNet-18 encoder over non-RGB spectral bands and a mirrored decoder with transposed convolutions to predict the a*b* chromatic channels in CIE Lab space. For downstream tasks, the encoder is reused as a feature extractor with a linear classification head. An inference-time ensemble combines a spectral branch pretrained by colorization with an RGB branch pretrained on ImageNet."}, "pretraining": {"method": "colorization", "method_details": "The model is pretrained self-supervised by reconstructing visible color information from non-RGB Sentinel-2 spectral bands rather than from grayscale. It predicts chromatic a*b* channels in Lab space from spectral inputs, encouraging the encoder to learn semantically meaningful remote sensing representations.", "objective": "masked-free image-to-image reconstruction of color channels from spectral bands", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet"], "num_images": "590326", "sensors": ["Sentinel-2A", "Sentinel-2B"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "mixed (10m, 20m, 60m)", "geographic_coverage": null}, "downstream_tasks": ["land-cover classification", "West Nile Virus detection"], "downstream_datasets": ["BigEarthNet", "West Nile Disease Dataset"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes a self-supervised remote sensing pretraining strategy that learns representations by colorizing RGB channels from non-RGB spectral Sentinel-2 bands. The learned encoder transfers well to downstream classification, especially in low-label regimes, and an ensemble with an ImageNet-pretrained RGB branch further improves performance.", "_source_pdf": "The_color_out_of_space_learning_self-supervised_representations_for_Earth_Observation_imagery.pdf", "_extraction_timestamp": "2026-03-20T21:40:47"}
{"title": "Tile2Vec: Unsupervised representation learning for spatially distributed data", "year": 2019, "model_name": "Tile2Vec", "architecture": {"backbone": "ResNet-18", "type": "encoder-only", "params_millions": null, "notes": "Uses a CNN encoder trained on tile triplets with triplet loss. For the main remote sensing experiments, the paper uses a ResNet-18 modified for 28x28 CIFAR-10 images, with an additional residual block to handle larger 50x50 multispectral inputs and with the final classification layer removed."}, "pretraining": {"method": "triplet-based metric learning using spatial context", "method_details": "The method extends the distributional hypothesis to geospatial data: geographically nearby tiles are treated as semantically similar, while distant tiles are treated as dissimilar. Anchor-neighbor-distant triplets are sampled based on spatial neighborhoods, and the encoder is trained to pull nearby tiles together and push faraway tiles apart in embedding space.", "objective": "triplet loss on spatially sampled tiles", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["National Agriculture Imagery Program (NAIP)", "Landsat 8", "Landsat 7", "CIA World Factbook", "DigitalGlobe"], "num_images": "100k triplets for the main NAIP experiment; 300k NAIP tiles available; additional experiments use 100k triplets on Landsat data and 11,564 DigitalGlobe images", "sensors": ["NAIP aerial imagery", "Landsat 8", "Landsat 7", "DigitalGlobe"], "modalities": ["RGB", "multispectral", "tabular"], "spatial_resolution": "mixed (0.6 m NAIP/DigitalGlobe, 30 m Landsat)", "geographic_coverage": "Central Valley, California; San Francisco; New York City; Boston; Uganda; Addis Ababa, Ethiopia; global countries"}, "downstream_tasks": ["land cover classification", "poverty prediction", "visual analogy", "latent space interpolation", "visual query by example", "country health index prediction"], "downstream_datasets": ["Cropland Data Layer (CDL)", "Uganda Living Standards Measurement Study (LSMS)", "CIA World Factbook"], "code_available": true, "code_url": "https://github.com/ermongroup/tile2vec", "weights_available": true, "key_contribution": "The paper proposes Tile2Vec, a self-supervised representation learning method for spatially distributed data that uses geographic proximity as weak supervision via a triplet loss. It shows that spatial context alone can produce semantically meaningful embeddings that improve remote sensing classification and regression performance, even outperforming some supervised baselines with limited labels.", "_source_pdf": "Tile2Vec_Unsupervised_Representation_Learning_for_Spatially_Distributed_Data.pdf", "_extraction_timestamp": "2026-03-20T21:40:58"}
{"title": "Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data", "year": 2025, "model_name": "LESS ViT", "architecture": {"backbone": "LESS ViT-Base", "type": "encoder-decoder", "params_millions": 83.2, "notes": "ViT-Base-like model with 12 LESS attention blocks in the encoder and 8 blocks in the decoder for Hyper-MAE pretraining; embedding dimensions 768/512 for encoder/decoder; patch size 16; uses Tied Patch Embedding, Continuous Positional-Channel Embedding, Perception Field Mask, and low-rank spatial-spectral attention via Kronecker-product approximation. Supports modality-specific tied patch embeddings for optical and SAR, then concatenates tokens."}, "pretraining": {"method": "MAE", "method_details": "Introduces Hyper-MAE, a hyperspectral masked autoencoder that decouples spatial and spectral masking. It masks 75% of spatial patches and 50% of spectral channels, applies identical spatial masks across unmasked channels, and reconstructs masked regions with separate spatial and spectral reconstruction losses to better learn inter-channel and spatial-spectral correlations.", "objective": "masked image reconstruction with decoupled spatial and spectral reconstruction losses", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SSL4EO-S12"], "num_images": "250K locations", "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "mixed", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "multi-label classification", "semantic segmentation", "cross-satellite generalization"], "downstream_datasets": ["EuroSAT", "BigEarthNet", "So2Sat", "SegMunich", "DFC2020", "MARIDA", "NLCD-L"], "code_available": true, "code_url": "https://uiuctml.github.io/GeospatialFM/", "weights_available": null, "key_contribution": "The paper proposes LESS ViT, a scalable and physics-informed transformer architecture for multi-modal and hyperspectral geospatial data that efficiently models spatial-spectral relationships through low-rank attention, continuous positional-channel embeddings, and locality-aware masking. It also introduces Hyper-MAE for hyperspectral self-supervised pretraining and GFM-Bench for standardized evaluation, showing competitive results and improved cross-satellite generalization.", "_source_pdf": "Towards_Scalable_Foundation_Model_for_Multi-modal_and_Hyperspectral_Geospatial_Data.pdf", "_extraction_timestamp": "2026-03-20T21:41:08"}
{"title": "Towards a Unified Copernicus Foundation Model for Earth Vision", "year": 2025, "model_name": "Copernicus-FM", "architecture": {"backbone": "ViT-Base", "type": "encoder-decoder", "params_millions": null, "notes": "Vision Transformer encoder with dynamic hypernetwork-based patch embedding for arbitrary spectral and non-spectral modalities; uses a spectral hypernetwork conditioned on wavelength and bandwidth, a variable hypernetwork conditioned on LLM-encoded variable names, metadata encoding via Fourier features, and a lightweight Transformer decoder for MAE-style masked reconstruction. Primary model is pretrained with ViT-Base; patch sizes vary by modality (16 for Sentinel-1/2, 8 for Sentinel-3, 4 for Sentinel-5P, 64 for DEM)."}, "pretraining": {"method": "masked image modeling with continual distillation", "method_details": "The model unifies all major Copernicus Sentinel modalities plus DEM using dynamic hypernetworks that generate modality-specific patch embeddings for both spectral and non-spectral inputs. It additionally injects metadata (geolocation, area, time) through Fourier encodings and uses auxiliary distillation from frozen DINOv2 and SoftCon teachers to improve representation quality and convergence.", "objective": "MAE-style masked image reconstruction plus representation distillation", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Copernicus-Pretrain", "Copernicus DEM GLO-30"], "num_images": "18.7M", "sensors": ["Sentinel-1 GRD", "Sentinel-2 TOA", "Sentinel-3 OLCI", "Sentinel-5P", "Copernicus DEM GLO-30"], "modalities": ["SAR", "multispectral", "atmospheric variables", "elevation", "metadata"], "spatial_resolution": "10 m to 1 km (mixed)", "geographic_coverage": "global land surface and near-land ocean, including polar regions"}, "downstream_tasks": ["cloud detection", "scene classification", "multi-label land use/land cover classification", "semantic segmentation", "change detection", "flood mapping", "biomass regression", "air quality estimation"], "downstream_datasets": ["Cloud-S2", "Cloud-S3", "EuroSAT-S1", "EuroSAT-S2", "BigEarthNet-S1", "BigEarthNet-S2", "LC100Cls-S3", "DFC2020-S1", "DFC2020-S2", "LC100Seg-S3", "Flood-S1", "LCZ-S2", "Biomass-S3", "AQ-NO2-S5P", "AQ-O3-S5P"], "code_available": true, "code_url": "https://github.com/zhu-xlab/Copernicus-FM", "weights_available": true, "key_contribution": "The paper introduces a unified Earth observation foundation model ecosystem consisting of Copernicus-Pretrain, Copernicus-FM, and Copernicus-Bench. Its main contribution is showing that dynamic hypernetworks and metadata-aware masked pretraining can jointly handle heterogeneous Copernicus modalities—from SAR and multispectral imagery to atmospheric variables and DEM—and improve both surface and atmospheric downstream tasks.", "_source_pdf": "Towards_a_Unified_Copernicus_Foundation_Model_for_Earth_Vision.pdf", "_extraction_timestamp": "2026-03-20T21:41:20"}
{"title": "USat: A Unified Self-Supervised Encoder for Multi-Sensor Satellite Imagery", "year": 2023, "model_name": "USatMAE", "architecture": {"backbone": "ViT-L", "type": "encoder-decoder", "params_millions": null, "notes": "USat is a Vision Transformer encoder modified for multi-sensor, multi-spectral imagery with varying ground sampling distances. It uses separate patch projection layers per spectral band, spectral group pooling, superpositional encodings, and optional spectral-group/sensor encodings. The MAE decoder is an 8-layer Transformer."}, "pretraining": {"method": "MAE", "method_details": "Masked autoencoding over jointly paired multi-sensor imagery (NAIP + Sentinel-2) using a unified encoder that supports arbitrary subsets of spectral bands and varying spatial resolutions/GSDs. Novel components include per-band patch projection, spectral group pooling to reduce sequence length, and superpositional encodings to preserve geospatial relationships across bands/sensors with different GSDs.", "objective": "masked image reconstruction with mean squared error", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Satlas", "USatlas"], "num_images": "3,615,184 training image pairs + 231,626 validation pairs", "sensors": ["NAIP", "Sentinel-2"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "mixed (NAIP 0.6-1m / processed at 1m; Sentinel-2 10m and 20m)", "geographic_coverage": "contiguous U.S."}, "downstream_tasks": ["multi-label classification", "scene classification", "land-cover classification", "infrastructure/methane source mapping classification"], "downstream_datasets": ["USatlas", "EuroSAT", "BigEarthNet", "METER-ML"], "code_available": true, "code_url": "https://github.com/stanfordmlgroup/USat", "weights_available": true, "key_contribution": "The paper introduces USat, a unified self-supervised encoder for multi-sensor satellite imagery, and USatMAE, a masked autoencoding pretraining framework that jointly learns from imagery with different spectral bands and spatial resolutions. Its key innovation is handling arbitrary subsets of bands across sensors while preserving cross-scale geospatial alignment, yielding improved performance over prior remote-sensing MAE baselines on several benchmarks.", "_source_pdf": "USat_A_Unified_Self-Supervised_Encoder_for_Multi-Sensor_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T21:41:30"}
{"title": "UrbanCLIP: Learning Text-Enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web", "year": 2023, "model_name": "UrbanCLIP", "architecture": {"backbone": "Vision Transformer (ViT) with Transformer decoder", "type": "encoder-decoder", "params_millions": null, "notes": "Uses a ViT-based unimodal image encoder and a causal masking Transformer decoder for text; the first half of decoder layers acts as a unimodal text encoder and the remaining decoder layers perform multimodal interaction via cross-attention. Includes learnable [CLS] tokens, task-specific attention poolers, and a decoupled decoder mechanism for joint contrastive and generative training."}, "pretraining": {"method": "joint contrastive language-image pretraining + language modeling", "method_details": "Satellite images are first paired with LLM-generated and refined location descriptions. UrbanCLIP jointly learns image-text alignment with a bidirectional contrastive loss and deep multimodal fusion with an autoregressive language modeling loss. A key novelty is using LLM-generated urban descriptions to inject textual knowledge into urban visual representation learning, plus a decoupled decoder that supports both unimodal text encoding and multimodal cross-attention efficiently.", "objective": "image-text contrastive alignment and autoregressive next-token language modeling", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["custom dataset from Baidu Map API", "LLaMA-Adapter V2 generated location descriptions"], "num_images": "17,562 images", "sensors": ["Baidu Map aerial/satellite imagery"], "modalities": ["RGB", "text"], "spatial_resolution": "around 13 meters per pixel", "geographic_coverage": "four cities in China: Beijing, Shanghai, Guangzhou, Shenzhen"}, "downstream_tasks": ["urban indicator prediction", "population prediction", "GDP prediction", "carbon emission prediction"], "downstream_datasets": ["Beijing", "Shanghai", "Guangzhou", "Shenzhen", "WorldPop", "ODIAC"], "code_available": true, "code_url": "https://github.com/StupidBuluchacha/UrbanCLIP", "weights_available": true, "key_contribution": "The paper introduces UrbanCLIP, the first framework to incorporate LLM-generated text descriptions into urban region profiling from satellite imagery. By jointly optimizing image-text contrastive alignment and language modeling, it learns text-enhanced visual representations that improve prediction of urban indicators such as population, GDP, and carbon emissions across multiple cities.", "_source_pdf": "UrbanCLIP_Learning_Text-Enhanced_Urban_Region_Profiling_with_Contrastive_Language-Image_Pretraining_from_the_Web.pdf", "_extraction_timestamp": "2026-03-20T21:41:42"}
{"title": "WV-Net: A foundation model for SAR WV-mode satellite imagery trained using contrastive self-supervised learning on 10 million images", "year": 2024, "model_name": "WV-Net", "architecture": {"backbone": "ResNet50", "type": "encoder-only", "params_millions": null, "notes": "Trained with the SimCLR framework. The encoder consists of a ResNet50 backbone followed by a smaller projection head during pretraining; only the backbone is used for transfer learning. Input images are grayscale Sentinel-1 WV SAR images repeated to 3 channels for augmentations/model input."}, "pretraining": {"method": "contrastive self-supervised learning (SimCLR)", "method_details": "WV-Net is pretrained on nearly 10 million Sentinel-1 Wave Mode SAR images using SimCLR, with an augmentation set adapted for SAR WV imagery. The final model extends baseline SimCLR augmentations with mixup, random rotation, random color inversion, and sharpness transforms after an augmentation study.", "objective": "contrastive alignment of augmented views of the same image while separating views from different images", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-1 Wave Mode (WV) image archive"], "num_images": "9.9M", "sensors": ["Sentinel-1"], "modalities": ["SAR", "grayscale"], "spatial_resolution": "5 m", "geographic_coverage": "global open ocean"}, "downstream_tasks": ["multilabel classification", "wave height regression", "air temperature regression", "image retrieval"], "downstream_datasets": ["GOALI", "Wang et al. [61] labelled ocean SAR imagery dataset", "Quach et al. [43] wave height regression dataset", "Stopa et al. [52] air temperature regression dataset", "ERA5"], "code_available": true, "code_url": "https://github.com/hawaii-ai/WVNet/", "weights_available": true, "key_contribution": "The paper introduces WV-Net, the first foundation model for Sentinel-1 Wave Mode open-ocean SAR imagery, pretrained with SimCLR on 9.9 million unlabeled images. It shows that domain-specific self-supervised pretraining on SAR WV data yields better embeddings than ImageNet pretraining across classification, regression, and retrieval tasks, with improved data efficiency and strong performance even without full finetuning.", "_source_pdf": "WV-Net_A_foundation_model_for_SAR_WV-mode_satellite_imagery_trained_using_contrastive_self-supervised_learning_on_10_mil.pdf", "_extraction_timestamp": "2026-03-20T21:41:51"}
{"title": "WildSAT: Learning Satellite Image Representations from Wildlife Observations", "year": 2024, "model_name": "WildSAT", "architecture": {"backbone": "ViT-L/16", "type": "CLIP-style multi-modal contrastive fine-tuning framework with image encoder, text encoder, and location encoder", "params_millions": null, "notes": "WildSAT is a training framework rather than a single fixed backbone. It can be applied to ResNet50, Swin, and ViT backbones. The paper evaluates multiple variants; for the largest/primary evaluated variant, a ViT-L/16 backbone is used in some experiments (e.g. SatMAE baseline). The framework aligns satellite images with Wikipedia text and location embeddings from SINR using contrastive losses, plus an image-image temporal/geometric augmentation contrastive term."}, "pretraining": {"method": "contrastive multi-modal pretraining / continual pretraining", "method_details": "WildSAT jointly aligns satellite images, species occurrence locations, environmental covariates, and Wikipedia text in a shared embedding space. It uses wildlife observation locations from citizen science as supervision and adds an image-image contrastive term across temporally distinct and geometrically augmented satellite views of the same location. It can train from scratch or continually pretrain existing satellite and non-satellite models.", "objective": "contrastive alignment of image-text-location embeddings plus image-image temporal/geometric contrastive learning", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["iNaturalist", "Wikipedia", "WorldClim2", "Sentinel-2", "SINR dataset"], "num_images": "305,689 satellite images; 980,376 aligned training samples", "sensors": ["Sentinel-2"], "modalities": ["RGB", "text", "location", "environmental covariates", "temporal"], "spatial_resolution": "10 m", "geographic_coverage": "global, but overrepresented in the United States and Europe"}, "downstream_tasks": ["scene classification", "multi-label land-cover classification", "semantic segmentation", "bird species encounter rate prediction", "zero-shot text-to-image retrieval"], "downstream_datasets": ["UCM", "AID", "RESISC45", "FMoW", "EuroSAT", "So2Sat20k", "BigEarthNet20k", "Cashew1k", "SAcrop3k", "SatBird"], "code_available": true, "code_url": "https://github.com/cvl-umass/wildsat", "weights_available": true, "key_contribution": "The paper introduces WildSAT, a multi-modal contrastive framework that learns satellite image representations from geo-tagged wildlife observations, species-related Wikipedia text, and location/environment context. It shows that wildlife-derived supervision improves both randomly initialized and strong pretrained remote sensing models across classification, segmentation, species encounter prediction, and enables zero-shot text-based satellite image retrieval.", "_source_pdf": "WildSAT_Learning_Satellite_Image_Representations_from_Wildlife_Observations.pdf", "_extraction_timestamp": "2026-03-20T21:42:06"}
{"title": "A generalizable and accessible approach to machine learning with global satellite imagery", "year": 2020, "model_name": "MOSAIKS", "architecture": {"backbone": "Random convolutional features (random kitchen sinks) with ridge regression", "type": "fixed feature extractor + linear regression", "params_millions": null, "notes": "Uses random convolutional features (RCFs) generated by convolving randomly sampled 3x3 RGB patches over images, followed by ReLU and average pooling; main experiments use K=8,192 features per image. Predictions are made with ridge regression on the precomputed features. The paper also interprets MOSAIKS as related to a two-layer CNN with untrained filters or an approximation to kernel ridge regression with a convolutional kernel."}, "pretraining": {"method": "unsupervised random feature featurization", "method_details": "A one-time, task-agnostic featurization based on random kitchen sinks adapted to satellite imagery via random convolutional features. Patches are randomly sampled from training images before labels are used, convolved over each image, passed through ReLU, and average pooled to form fixed embeddings shared across tasks.", "objective": "task-agnostic image encoding for downstream linear prediction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Google Static Maps API imagery"], "num_images": "100,000 in the main US experiment; 423,476 realized samples in the global experiment", "sensors": ["Google Static Maps daytime satellite imagery"], "modalities": ["RGB"], "spatial_resolution": "~1 km × 1 km tiles, 256×256 pixels", "geographic_coverage": "continental US and global"}, "downstream_tasks": ["forest cover prediction", "elevation prediction", "population density prediction", "nighttime lights prediction", "average income prediction", "road length prediction", "average house price prediction", "survey variable estimation", "label super-resolution prediction"], "downstream_datasets": ["Hansen Global Forest Change", "AWS Terrain Tiles", "Gridded Population of the World, Version 4", "VIIRS Day/Night Band Nighttime Lights", "2015 American Community Survey", "TIGER/Line Geodatabases", "ZTRAX", "Rwanda development survey tasks from Head et al. 2017", "Haiti development survey tasks from Head et al. 2017", "Nepal development survey tasks from Head et al. 2017"], "code_available": true, "code_url": "https://github.com/Global-Policy-Lab/mosaiks-paper", "weights_available": null, "key_contribution": "The paper introduces MOSAIKS, a simple unsupervised satellite image featurization method based on random convolutional features that supports many downstream geospatial prediction tasks using only linear regression. It shows that a single shared embedding of global satellite imagery can achieve performance competitive with task-specific deep CNNs while being much faster, more accessible, and scalable across users and tasks.", "_source_pdf": "A_generalizable_and_accessible_approach_to_machine_learning_with_global_satellite_imagery.pdf", "_extraction_timestamp": "2026-03-20T21:49:18"}
{"title": "CONTRASTIVE SELF-SUPERVISED DATA FUSION FOR SATELLITE IMAGERY", "year": 2022, "model_name": "D-SimCLR", "architecture": {"backbone": "ResNet-18", "type": "contrastive dual-encoder", "params_millions": null, "notes": "Primary proposed method uses two separate ResNet-18 encoders for Sentinel-1 and Sentinel-2, replacing SimCLR weight sharing with modality-specific encoders. Each branch has a 2-layer MLP projection head with ReLU, producing 128-dimensional latent vectors; downstream classification concatenates encoder features from both modalities and applies a linear layer."}, "pretraining": {"method": "contrastive", "method_details": "Augmentation-free contrastive self-supervised data fusion using spatially aligned multi-modal satellite image pairs from the same location as positives. Unlike standard SimCLR, it uses dedicated encoders and projection heads for each modality (Sentinel-1 SAR and Sentinel-2 multispectral) and contrasts matched cross-sensor pairs against samples from other locations.", "objective": "contrastive alignment of cross-modal representations for the same geo-location", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SEN12MS"], "num_images": "180,662 paired observations", "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["SAR", "multispectral"], "spatial_resolution": "10m", "geographic_coverage": null}, "downstream_tasks": ["single-label land-cover classification", "multi-label land-cover classification"], "downstream_datasets": ["DFC2020", "EuroSAT"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes augmentation-free contrastive self-supervised data fusion for remote sensing by aligning Sentinel-1 and Sentinel-2 representations from the same location. Its main method, D-SimCLR, learns strong multimodal encoders from unlabeled data and outperforms supervised fusion baselines, especially when labeled data is scarce.", "_source_pdf": "Contrastive_Self-Supervised_Data_Fusion_for_Satellite_Imagery.pdf", "_extraction_timestamp": "2026-03-20T21:49:52"}
{"title": "Geographical Supervision Correction for Remote Sensing Representation Learning", "year": 2022, "model_name": "GeCo", "architecture": {"backbone": "ResNet-50", "type": "encoder-only mean-teacher framework with projection head", "params_millions": null, "notes": "Uses a student-teacher (mean-teacher) setup. Both student and teacher share the same backbone structure; a single fully connected layer is used as the projection head during pretraining. A learnable 8x8 correction matrix adaptively corrects geographical supervision during joint optimization."}, "pretraining": {"method": "geographical supervision-based pretraining with adaptive correction", "method_details": "Pretrains on supervision derived from global land cover (GLC) products, then adaptively corrects noisy/deviated supervision caused by time/resolution mismatch using a learnable correction matrix. Jointly optimizes network parameters and correction matrix, with regularization from two priors: 'minor changes' and 'spatial aggregation'.", "objective": "predict corrected geographical supervision (land-cover proportion vector) with MSE, plus student-teacher consistency and regularization losses", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Levir-KR", "GlobeLand30 2020", "GLCFCS30 2020", "FromGLC30 2017"], "num_images": null, "sensors": ["Gaofen-1 PMS", "Gaofen-2 PMS", "Gaofen-1 WFV", "Gaofen-6 WFV", "Landsat", "HJ-1"], "modalities": ["RGB", "land cover supervision"], "spatial_resolution": "mixed (remote sensing images at 0.8m, 2m, 16m; GLC products at 30m)", "geographic_coverage": "global for GLC products; Levir-KR coverage unclear"}, "downstream_tasks": ["scene classification", "semantic segmentation", "cloud detection", "snow detection", "object detection"], "downstream_datasets": ["UCMerced", "RSSCN7", "Vaihingen", "Levir CS", "Levir"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes GeCo, a remote sensing representation learning method that corrects noisy geographical supervision derived from global land cover products during pretraining. Its main novelty is a jointly optimized correction matrix constrained by land-cover change and spatial aggregation priors, which improves transfer performance across classification, segmentation, cloud/snow detection, and object detection.", "_source_pdf": "Geographical_Supervision_Correction_for_Remote_Sensing_Representation_Learning.pdf", "_extraction_timestamp": "2026-03-20T21:50:15"}
{"title": "Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Images", "year": 2022, "model_name": "RS-BYOL", "architecture": {"backbone": "ResNet-50", "type": "student-teacher distillation / Siamese encoder with BYOL-style predictor-projector", "params_millions": null, "notes": "Based on BYOL. Uses identical student and teacher encoders, with 2-layer MLP projection head and 2-layer MLP prediction head on the student side. Hidden projection size is 256 with batch normalization and ReLU. Teacher weights are updated by exponential moving average with decay 0.9. Two variants are trained: single-channel and three-channel."}, "pretraining": {"method": "BYOL-style self-supervised distillation", "method_details": "The key novelty is treating multi-spectral Sentinel-2 and SAR Sentinel-1 images as two implicit augmented views of the same location, instead of relying only on standard image augmentations. The paper studies both multi-modal MS-SAR pretraining and single-modality MS-only pretraining, and compares single-band versus three-band feature learning.", "objective": "Alignment of positive pairs via BYOL loss between student prediction and teacher projection across MS and SAR views; effectively invariant representation learning across modalities", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sen12MS"], "num_images": "90K pairs (50% of 180,662 triplets); also a 150K-pair variant", "sensors": ["Sentinel-2", "Sentinel-1"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "10m-20m", "geographic_coverage": null}, "downstream_tasks": ["scene classification", "land cover classification", "semantic segmentation", "land cover mapping"], "downstream_datasets": ["EuroSAT", "RESISC45", "Sen12MS", "2020 IEEE Data Fusion Contest (DFC)"], "code_available": true, "code_url": "https://github.com/yaox12/BYOL-PyTorch", "weights_available": null, "key_contribution": "The paper proposes RS-BYOL, a BYOL-based self-supervised remote sensing model that learns invariant representations by aligning multi-spectral and SAR views of the same location. It shows that remote-sensing-specific SSL pretraining, especially with joint MS-SAR data, outperforms ImageNet-based transfer and that even single-band pretraining can yield strong downstream performance.", "_source_pdf": "RS-BYOL_Self-Supervised_Learning_for_Invariant_Representations_From_Multi-Spectral_and_SAR_Images.pdf", "_extraction_timestamp": "2026-03-20T21:50:26"}
{"title": "RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning", "year": 2024, "model_name": "RingMo-Aerial", "architecture": {"backbone": "Swin-Base", "type": "encoder-only", "params_millions": null, "notes": "Built on RingMo with a Swin Transformer backbone. Introduces Frequency-Enhanced Multi-Head Self-Attention (FE-MSA), which adds a residual depthwise 7x7 convolution before window/self-attention to enhance small-object and high-frequency feature modeling. Uses an ARS-Adapter for parameter-efficient downstream fine-tuning; adapter tuning updates about 0.41% of parameters."}, "pretraining": {"method": "MIM + contrastive learning", "method_details": "Stage 1 uses SimMIM-style masked image modeling for pixel-level semantic association learning. Stage 2 adds an affine transformation-based contrastive learning framework (CLAF), where perspective/affine transforms create positive pairs that magnify distant small objects and encourage view/scale-invariant aerial representations.", "objective": "masked image reconstruction and contrastive alignment of original vs affine-transformed aerial images", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom aerial remote sensing pretraining dataset compiled from multiple collected open-source airborne remote sensing datasets"], "num_images": "5.4M", "sensors": ["aerial optical cameras"], "modalities": ["RGB"], "spatial_resolution": "mixed", "geographic_coverage": null}, "downstream_tasks": ["semantic segmentation", "object detection", "change detection", "scene classification", "3D reconstruction", "object tracking"], "downstream_datasets": ["UAVID", "UDD6", "FloodNet", "VisDrone-DET", "UAVDT", "ShipDataset", "DroneVehicle", "AerialSARData", "LEVIR", "CDD"], "code_available": null, "code_url": null, "weights_available": null, "key_contribution": "The paper proposes RingMo-Aerial, presented as the first aerial remote sensing foundation model, tailored to aerial imagery with oblique views, scale variation, and dense small objects. Its main innovations are FE-MSA for frequency-enhanced small-object representation, affine transformation contrastive pretraining (CLAF) for aerial viewpoint adaptation, and ARS-Adapter for efficient fine-tuning across diverse downstream ARS tasks.", "_source_pdf": "RingMo-Aerial_An_Aerial_Remote_Sensing_Foundation_Model_With_Affine_Transformation_Contrastive_Learning.pdf", "_extraction_timestamp": "2026-03-20T21:50:37"}
{"title": "RingMo-lite: A Remote Sensing Multi-task Lightweight Network with CNN-Transformer Hybrid Framework", "year": 2023, "model_name": "RingMo-lite", "architecture": {"backbone": "Swin-Tiny-based CNN-Transformer hybrid backbone", "type": "encoder-decoder", "params_millions": 28.294, "notes": "The encoder uses a dual-branch High-Low Frequency Information Fusion Block (FIFB) with a Swin Transformer low-frequency branch and a CNN high-frequency branch. It follows a 4-stage hierarchical design with block configuration (2, 2, 6, 2), patch size 4x4, window size 7, embed dim 96, and stage dimensions 96/192/384/768. Different task-specific decoder heads are used for classification, object detection, semantic segmentation, and change detection."}, "pretraining": {"method": "masked image modeling", "method_details": "The paper proposes Frequency-Domain Masked Image Modeling (FD-MIM), which analyzes randomly selected image patches in the frequency domain using DFT, separates them into high-frequency and low-frequency patches, applies high-pass/low-pass filtering accordingly, and then performs random pixel masking to improve reconstruction of remote sensing details and match the hybrid CNN-Transformer frequency-aware encoder.", "objective": "masked image reconstruction with L1 regression loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["custom visible remote sensing image dataset"], "num_images": "150000", "sensors": ["optical"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["scene classification", "object detection", "semantic segmentation", "change detection"], "downstream_datasets": ["AID", "NWPU-RESISC45", "UCM", "DIOR", "FAIR1M", "iSAID", "ISPRS Potsdam", "LEVIR-CD"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "The paper introduces RingMo-lite, a lightweight remote sensing foundation model that combines CNN and Swin Transformer branches to explicitly model high-frequency local details and low-frequency global context. It also proposes FD-MIM, a frequency-domain masked image modeling pretraining strategy, enabling strong multi-task performance across classification, detection, segmentation, and change detection while reducing parameters by over 60% relative to RingMo.", "_source_pdf": "RingMo-Lite_A_Remote_Sensing_Lightweight_Network_With_CNN-Transformer_Hybrid_Framework.pdf", "_extraction_timestamp": "2026-03-20T21:50:49"}
{"title": "SELF-SUPERVISED VISION TRANSFORMERS FOR JOINT SAR-OPTICAL REPRESENTATION LEARNING", "year": 2022, "model_name": "DINO-MM", "architecture": {"backbone": "ViT-S/8", "type": "encoder-only student-teacher Siamese/self-distillation architecture", "params_millions": null, "notes": "Based on DINO with a ViT encoder backbone and a 3-layer MLP projection head (hidden dimension 2048), followed by L2 normalization and a weight-normalized fully connected layer of dimension 65,536. Input is formed by concatenating SAR and optical channels."}, "pretraining": {"method": "DINO-based self-distillation contrastive SSL", "method_details": "Extends DINO to multimodal SAR-optical learning by concatenating Sentinel-1 and Sentinel-2 channels and applying a proposed RandomSensorDrop augmentation that randomly masks SAR channels, optical channels, or neither. This lets the model learn from SAR-only, optical-only, and joint SAR-optical views, enabling both intra- and inter-modality representation learning.", "objective": "Contrastive/self-distillation alignment of augmented views using cross-entropy between teacher and student outputs", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet-MM"], "num_images": "590,326 pairs", "sensors": ["Sentinel-1", "Sentinel-2"], "modalities": ["SAR", "multispectral"], "spatial_resolution": "10m", "geographic_coverage": null}, "downstream_tasks": ["multi-label scene classification", "linear classification evaluation"], "downstream_datasets": ["BigEarthNet-MM"], "code_available": true, "code_url": "https://github.com/zhu-xlab/DINO-MM", "weights_available": true, "key_contribution": "The paper proposes DINO-MM, a multimodal self-supervised learning method for joint SAR-optical representation learning using vision transformers. Its key innovation is RandomSensorDrop, which enables a single pretrained model to learn robust representations from SAR-only, optical-only, or fused inputs and improves transfer even to single-modality downstream tasks.", "_source_pdf": "Self-Supervised_Vision_Transformers_for_Joint_SAR-Optical_Representation_Learning.pdf", "_extraction_timestamp": "2026-03-20T21:51:10"}
{"title": "Self-supervised Audiovisual Representation Learning for Remote Sensing Data", "year": 2021, "model_name": "Self-supervised audiovisual representation learning with Batch Triplet Loss", "architecture": {"backbone": "ResNet-50", "type": "CLIP-style dual-encoder", "params_millions": null, "notes": "Modality-specific dual encoders map aerial images and audio log-mel spectrograms into a shared embedding space. Visual encoder evaluated with ResNet-18 and ResNet-50; audio encoder uses ResNet-18. Global average pooling and a final fully connected layer produce single-vector embeddings."}, "pretraining": {"method": "cross-modal contrastive / triplet-based self-supervised learning", "method_details": "The method uses paired geo-tagged overhead imagery and co-located field recordings, training image and audio encoders to align corresponding pairs in a shared embedding space. Its main novelty is the use of remote-sensing audiovisual correspondence and a proposed batch-wise triplet loss that uses all pairings within a batch.", "objective": "image-audio embedding alignment with Batch Triplet Loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SoundingEarth", "Radio Aporee ::: Maps", "Google Earth"], "num_images": "50,545", "sensors": ["Google Earth aerial imagery", "field audio recordings"], "modalities": ["RGB", "audio"], "spatial_resolution": "approximately 0.2 m", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "semantic segmentation", "audiovisual scene classification", "cross-modal retrieval"], "downstream_datasets": ["UC Merced Land Use", "NWPU-RESISC45", "AID", "DeepGlobe Land Cover Classification", "ADVANCE", "SoundingEarth"], "code_available": true, "code_url": "https://github.com/khdlr/SoundingEarth", "weights_available": true, "key_contribution": "The paper introduces SoundingEarth, a large-scale dataset of 50K+ co-located aerial image and audio pairs, and proposes a self-supervised audiovisual pretraining framework for remote sensing. By aligning image and audio embeddings with a batch triplet loss, the learned visual backbones outperform common initialization and prior self-supervised baselines on multiple remote sensing downstream tasks.", "_source_pdf": "Self-supervised_Audiovisual_Representation_Learning_for_Remote_Sensing_Data.pdf", "_extraction_timestamp": "2026-03-20T21:51:19"}
{"title": "SpectralGPT: Spectral Remote Sensing Foundation Model", "year": 2023, "model_name": "SpectralGPT", "architecture": {"backbone": "ViT-Huge", "type": "encoder-decoder", "params_millions": 632, "notes": "MAE-style asymmetric transformer with a 3D generative pretrained transformer design for spectral remote sensing. Uses 3D tensor tokens (e.g. 8x8x3), 3D masking over spatial-spectral cubes, separate spatial and spectral positional embeddings, a transformer encoder over visible tokens, and a lightweight transformer decoder with multi-target reconstruction. Model variants include Base (~86M), Large (307M), and Huge (632M); primary/largest variant is Huge."}, "pretraining": {"method": "MAE-style generative masked modeling", "method_details": "Novel spectral foundation pretraining with 3D spatial-spectral masking, 3D token generation for spatial-spectral coupling, multi-target reconstruction to preserve spectrally sequential patterns, and progressive pretraining across datasets with varying image sizes, regions, and characteristics.", "objective": "masked image reconstruction with token-to-token and spectral-to-spectral reconstruction losses", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW-S2", "BigEarthNet-S2"], "num_images": "about 1.47M total available; pretraining uses 712,874 fMoW-S2 images and 354,196 BigEarthNet-S2 images", "sensors": ["Sentinel-2"], "modalities": ["multispectral", "temporal"], "spatial_resolution": "mixed (Sentinel-2 bands at 10m/20m/60m)", "geographic_coverage": "global for fMoW-S2 and Europe for BigEarthNet-S2"}, "downstream_tasks": ["single-label scene classification", "multi-label scene classification", "semantic segmentation", "change detection"], "downstream_datasets": ["EuroSAT", "BigEarthNet-S2", "SegMunich", "OSCD"], "code_available": true, "code_url": "https://doi.org/10.5281/zenodo.10533809", "weights_available": true, "key_contribution": "The paper introduces SpectralGPT, described as the first remote sensing foundation model purpose-built for spectral data, using a 3D MAE-like generative transformer that explicitly models spatial-spectral coupling and spectral sequentiality. Trained progressively on over one million Sentinel-2 spectral images, it achieves strong gains across scene classification, semantic segmentation, and change detection.", "_source_pdf": "SpectralGPT_Spectral_Remote_Sensing_Foundation_Model.pdf", "_extraction_timestamp": "2026-03-20T21:51:29"}
{"title": "Geographical Knowledge-driven Representation Learning for Remote Sensing Images", "year": 2021, "model_name": "GeoKR", "architecture": {"backbone": "ResNet-50 / VGG16", "type": "encoder-only", "params_millions": null, "notes": "Mean-teacher framework with two identical networks (student and teacher) using ResNet50 or VGG16 as encoder, plus a single fully-connected projection head. Teacher weights updated via moving average from student."}, "pretraining": {"method": "knowledge distillation / geographical knowledge supervision", "method_details": "Uses GlobeLand30 global land cover product and image geolocation as supervision. For each remote sensing image, the matching land cover map is generated from GlobeLand30 and converted into a knowledge representation vector (proportions of land cover classes) used as supervision. A mean-teacher framework with student and teacher networks is trained by aligning network outputs with the knowledge representation via KL divergence loss, which mitigates label noise from time/resolution discrepancies between images and the land cover product.", "objective": "KL divergence between predicted distribution and land-cover knowledge representation, plus KL between student and teacher outputs", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Levir-KR", "GlobeLand30"], "num_images": "1,431,950 images", "sensors": ["Gaofen-1", "Gaofen-2", "Gaofen-6"], "modalities": ["RGB", "multispectral"], "spatial_resolution": "0.8m / 2m / 16m", "geographic_coverage": "global (Gaofen series acquisitions, GlobeLand30 global land cover)"}, "downstream_tasks": ["scene classification", "semantic segmentation", "object detection", "cloud detection", "snow detection"], "downstream_datasets": ["UCMerced", "RSSCN7", "Vaihingen", "Potsdam", "Levir", "Levir_CS"], "code_available": true, "code_url": "https://github.com/flyakon/Geographical-Knowledge-driven-Representaion-Learning", "weights_available": false, "key_contribution": "Proposes GeoKR, a novel pre-training paradigm for remote sensing that uses global land cover products (GlobeLand30) and image geolocation as weak geographical-knowledge supervision instead of manual labels. Introduces a mean-teacher framework to suppress noise from time/resolution discrepancies between images and land cover maps, and constructs Levir-KR, a 1.43M image multi-resolution Gaofen pre-training dataset. Demonstrates improvements over ImageNet pre-training and self-supervised baselines (MoCo/SimCLR/BYOL) on multiple downstream RS tasks.", "_source_pdf": "GeoKR.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "An Empirical Study of Remote Sensing Pretraining", "year": 2023, "model_name": "RSP", "architecture": {"backbone": "ViTAEv2-S (also ResNet-50, Swin-T)", "type": "CNN and Vision Transformer", "params_millions": 18.8, "notes": "Evaluates ResNet-50 (CNN), Swin-T (transformer), and ViTAEv2-S (hybrid CNN+transformer with WMHSA); ViTAEv2-S is the primary/best model with ~18.8M params."}, "pretraining": {"method": "Supervised pretraining", "method_details": "Supervised classification pretraining on MillionAID scene labels, mirroring conventional ImageNet pretraining. Models trained from scratch for 40/100/120/300 epochs with AdamW, cosine LR schedule, AutoAugment, Random Erasing, Mixup, CutMix, color jitter.", "objective": "Scene classification (cross-entropy)", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID"], "num_images": "~1,000,848 (949,848 train / 51,000 val)", "sensors": ["SPOT", "IKONOS", "WorldView", "Landsat series"], "modalities": ["RGB aerial imagery"], "spatial_resolution": "0.5m to 153m", "geographic_coverage": "Global (Google Earth aerial imagery)"}, "downstream_tasks": ["Scene recognition", "Semantic segmentation", "Object detection", "Change detection"], "downstream_datasets": ["UC Merced Land Use (UCM)", "AID", "NWPU-RESISC", "ISPRS Potsdam", "iSAID", "DOTA", "DIOR", "LEVIR-CD", "CDD"], "code_available": true, "code_url": "https://github.com/ViTAE-Transformer/RSP", "weights_available": true, "key_contribution": "First systematic empirical study of supervised remote sensing pretraining on the large-scale MillionAID dataset across CNN (ResNet-50), transformer (Swin-T), and hybrid (ViTAEv2-S) backbones, demonstrating RSP outperforms ImageNet pretraining on aerial scene tasks and that ViTAEv2-S achieves the best results across recognition, segmentation, detection, and change detection.", "_source_pdf": "RSP.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model", "year": 2022, "model_name": "RVSA (ViT-B + RVSA / ViTAE-B + RVSA)", "architecture": {"backbone": "Plain ViT-B and ViTAE-B with Rotated Varied-Size Attention (RVSA)", "type": "Vision Transformer", "params_millions": 100, "notes": "Two backbones (~100M params each): plain ViT-B and ViTAE-B. During finetuning, MHSA layers are replaced by RVSA modules (full attention retained at 1/4 depth layers, similar to ViTDet). Patch size 16, embedding dim 768, 12 heads, depth 12."}, "pretraining": {"method": "MAE (Masked Autoencoder)", "method_details": "Unsupervised MIM pretraining following original MAE recipe with normalized pixel loss. Mask ratio 0.75, 1600 epochs, batch size 2048 on 8 A100 GPUs, AdamW optimizer. Pretrained on MillionAID without using labels.", "objective": "Masked image reconstruction (pixel reconstruction in normalized pixel space)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID"], "num_images": "~1M (subsets S1: 949,848 and S2: 51,000 used)", "sensors": ["Google Earth (multi-source aerial/satellite RGB)"], "modalities": ["RGB optical"], "spatial_resolution": "varied (multi-resolution from Google Earth imagery)", "geographic_coverage": "global (MillionAID, Google Earth)"}, "downstream_tasks": ["object detection (oriented bounding box)", "scene classification", "semantic segmentation"], "downstream_datasets": ["DOTA-V1.0", "DIOR-R", "UCM", "AID", "NWPU-RESISC45", "iSAID", "Potsdam", "LoveDA"], "code_available": true, "code_url": "https://github.com/ViTAE-Transformer/Remote-Sensing-RVSA", "weights_available": true, "key_contribution": "First work to pretrain plain ViT (~100M params) as an RS foundation model via MAE on MillionAID, and proposes Rotated Varied-Size Attention (RVSA) which adds a learnable rotation mechanism to varied-size window attention to better handle arbitrarily oriented objects in remote sensing imagery, achieving SOTA 81.24% mAP on DOTA-V1.0.", "_source_pdf": "RVSA.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain", "year": 2022, "model_name": "CSPT", "architecture": {"backbone": "ViT-B/L", "type": "Vision Transformer (encoder-decoder)", "params_millions": null, "notes": "ViT-B/L encoder with lightweight transformer decoder (depth=8) for MIM pretraining; encoder used as backbone for downstream fine-tuning"}, "pretraining": {"method": "MIM (Masked Image Modeling)", "method_details": "Consecutive self-supervised pretraining: (1) MIM-based MAE pretraining on large-scale natural scene dataset (ImageNet-1K), then (2) further MIM-based self-supervised pretraining on relevant unlabeled remote sensing data to bridge domain gap, then fine-tune on downstream tasks. 75% random masking, 16x16 patches, L1 reconstruction loss on masked pixels.", "objective": "Masked image reconstruction (L1 pixel loss on masked patches)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["ImageNet-1K", "AID", "NWPU-RESISC45", "Million-AID", "Sentinel-2"], "num_images": "1.28M (ImageNet-1K) + remote sensing data for further pretraining", "sensors": ["Optical RGB", "SAR", "Panchromatic", "Infrared"], "modalities": ["optical", "SAR", "panchromatic"], "spatial_resolution": "varies (0.05m to 30m across RS datasets)", "geographic_coverage": "global"}, "downstream_tasks": ["scene classification", "object detection", "land cover classification (semantic segmentation)"], "downstream_datasets": ["AID", "NWPU-RESISC45", "ISPRS Potsdam", "ISPRS Vaihingen", "GID", "DIOR", "NWPUVHR-10", "UCAS-AOD", "HRSC2016", "MSTAR", "HRSID", "SSDD"], "code_available": true, "code_url": "https://github.com/ZhAnGToNG1/transfer_learning_cspt", "weights_available": true, "key_contribution": "Proposes ConSecutive PreTraining (CSPT), a knowledge transfer learning strategy that does not stop pretraining: first MIM-based self-supervised pretraining on large-scale natural scene data (ImageNet), then continued self-supervised pretraining on relevant unlabeled remote sensing data to gradually bridge the domain gap, mitigate task-aware discrepancy, and unlock the potential of unlabeled RS data without expensive labeling.", "_source_pdf": "CSPT.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Towards Geospatial Foundation Models via Continual Pretraining", "year": 2023, "model_name": "GFM", "architecture": {"backbone": "Swin-B", "type": "transformer", "params_millions": 88, "notes": "Swin Transformer Base initialized via continual pretraining from ImageNet-22k weights; teacher-student framework with frozen ImageNet-22k teacher and randomly initialized student"}, "pretraining": {"method": "Continual pretraining (multi-objective: MIM + feature distillation)", "method_details": "Teacher-student paradigm: frozen ImageNet-22k pretrained Swin-B teacher provides intermediate features (at stage 3) to guide a randomly initialized student via cosine similarity feature distillation, while the student is also trained with masked image modeling (MIM) on GeoPile. Total loss = L_MIM + L_feat. Pretrained on 8 V100 GPUs, batch size 2048, image size 192x192, 100 epochs.", "objective": "Masked image modeling (L1 pixel reconstruction) + feature distillation (cosine similarity to teacher features)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["GeoPile (NAIP, RSD46-WHU, MLRSNet, RESISC45, PatternNet)"], "num_images": "~600k", "sensors": ["aerial", "satellite (multi-source RGB)"], "modalities": ["RGB"], "spatial_resolution": "0.1m - 30m GSD", "geographic_coverage": "Global (multi-source remote sensing scenes)"}, "downstream_tasks": ["change detection", "classification", "multi-label classification", "semantic segmentation", "super-resolution"], "downstream_datasets": ["OSCD", "DSIFN", "UC Merced", "BigEarthNet", "WHU Aerial", "Vaihingen", "SpaceNet2"], "code_available": true, "code_url": "https://github.com/mmendiet/GFM", "weights_available": true, "key_contribution": "Proposes a multi-objective continual pretraining paradigm that leverages ImageNet-22k representations as a teacher (via feature distillation) combined with masked image modeling on a curated diverse remote sensing dataset (GeoPile, ~600k images). Achieves SOTA across 7 geospatial downstream tasks while requiring substantially less compute and carbon footprint (8x reduction vs SatMAE).", "_source_pdf": "GFM.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "SwiMDiff: Scene-wide Matching Contrastive Learning with Diffusion Constraint for Remote Sensing Image", "year": 2024, "model_name": "SwiMDiff", "architecture": {"backbone": "ResNet-18", "type": "CNN", "params_millions": 11.7, "notes": "Dual-branch momentum-updated CL encoders (MoCo-v2 style) with 2-layer MLP projection head, plus a U-Net diffusion network (DDPM) with encoder f_n and decoder g_n as auxiliary branch"}, "pretraining": {"method": "Contrastive Learning + Diffusion", "method_details": "Combines scene-wide matching contrastive learning (built on MoCo-v2) with adaptive soft relabeling of false negatives from the same scene, jointly trained with an auxiliary DDPM diffusion constraint that predicts pixel-level noise to capture fine-grained features", "objective": "Joint loss L = lambda_C * L_C (scene-wide matching contrastive loss) + lambda_D * L_D (diffusion noise prediction L2 loss), with lambda_C=1 and lambda_D=10", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sen12MS"], "num_images": "10,000 Sentinel-2 RGB tiles (subset of Sen12MS)", "sensors": ["Sentinel-2"], "modalities": ["optical RGB"], "spatial_resolution": "10 m", "geographic_coverage": "global (Sen12MS subset, 256x256 tiles sampled from each season)"}, "downstream_tasks": ["change detection", "land-cover classification"], "downstream_datasets": ["OSCD", "LEVIR-CD", "BigEarthNet", "EuroSAT"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First framework to integrate a diffusion model as an auxiliary pixel-level constraint with contrastive learning for RS SSL, combined with a scene-wide matching strategy that adaptively relabels false negatives from the same geographic scene to improve both global semantic and fine-grained representation learning.", "_source_pdf": "SwiMDiff.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture", "year": 2024, "model_name": "SAR-JEPA", "architecture": {"backbone": "ViT-Base", "type": "Vision Transformer", "params_millions": 86, "notes": "ViT-B encoder with DNN predictor (LoMaR-SAR variant using 8-layer MAE decoder); target encoder uses multi-scale gradient-by-ratio (GR) features with convolutional kernels of sizes r=5,9,13,17"}, "pretraining": {"method": "Joint-Embedding Predictive Architecture (JEPA) with Masked Image Modeling", "method_details": "Local masked patches (LoMaR-style) predict multi-scale SAR gradient-by-ratio (GR) features of unseen patches in feature space. Combines MIM and physics-guided contrastive architecture (PGCA) ideas. ViT-B encoder with mask tokens and relative positional encoding; 8-layer ViT decoder/predictor. Target features computed via ratio-of-average (ROA) gradient at 4 scales.", "objective": "MSE between predicted DNN features and multi-scale SAR gradient features for masked local patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MSAR", "SAR-Ship-Dataset", "SARSim", "SAMPLE"], "num_images": "~100,000 (28,499 + 39,729 + 21,168 + 5,380)", "sensors": ["HISEA-1 (C-band)", "Gaofen-3 (C-band)", "Sentinel-1 (C-band)", "Simulated X-band SAR", "Airborne X-band SAR"], "modalities": ["SAR amplitude (single-channel magnitude)"], "spatial_resolution": "0.3 m to 25 m", "geographic_coverage": "Global / multiple scenes (ground, ocean, airports, harbors, nearshore, islands, distant seas, urban) plus simulated"}, "downstream_tasks": ["Few-shot fine-grained SAR target classification (vehicle, ship, aircraft)"], "downstream_datasets": ["MSTAR", "FUSAR-Ship", "SAR-ACD"], "code_available": true, "code_url": "https://github.com/waterdisappear/SAR-JEPA", "weights_available": true, "key_contribution": "First joint-embedding predictive architecture for SAR ATR that uses multi-scale SAR gradient-by-ratio features as high-quality target signals to suppress speckle noise, combined with local masking (LoMaR) for small targets in remote sensing; demonstrates SSL scalability for building a SAR foundation model unified across targets, scenes, and sensors.", "_source_pdf": "SAR-JEPA.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery", "year": 2024, "model_name": "SkySense", "architecture": {"backbone": "Swin-H (HSROI) + ViT-L (TMsI) + ViT-L (TSARI) spatial encoders with Multi-Modal Temporal Fusion Transformer", "type": "Hybrid (Swin Transformer + ViT) factorized multi-modal spatiotemporal encoder", "params_millions": 2060, "notes": "Billion-scale model with 2.06B parameters; factorized multi-modal spatiotemporal encoder performing spatial feature extraction independently per modality, then multi-modal temporal fusion; modular design allowing flexible single- or multi-modal use"}, "pretraining": {"method": "Multi-Granularity Contrastive Learning (MGCL) + Cross-Modal Alignment (CMA) + Unsupervised Geo-Context Prototype Learning (GCPL)", "method_details": "Teacher-student framework (BYOL-style with EMA updates) using only positive pairs. Multi-Granularity Contrastive Learning at pixel-, object-, and image-level across HSROI, TMsI, SAR modalities and fused features. Cross-Modal Alignment via multi-modal contrastive loss between modalities of same geo-location. Unsupervised Geo-Context Prototype Learning divides globe into 4096 regions with 100 prototypes each, learned via Sinkhorn-Knopp clustering on multi-modal features. Trained on 80 A100-80GB GPUs with batch size 240.", "objective": "Combined loss L = alpha * L_MGCL + beta * L_align (multi-granularity contrastive + cross-modal alignment), with geo-context prototype learning incorporated implicitly via EMA updates", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Curated multi-modal RSI dataset (HSROIs from WorldView-3, TMsI from Sentinel-2, TSARI from Sentinel-1)"], "num_images": "21.5 million temporal sequences (each sample: 1 static HSROI + 20 Sentinel-2 multispectral images + 10 Sentinel-1 SAR images)", "sensors": ["WorldView-3", "Sentinel-2", "Sentinel-1"], "modalities": ["High-resolution optical RGB (HSROI)", "Multispectral optical time series (TMsI; B2-8, B8A, B11-12)", "SAR time series (TSARI; VV, VH polarization)"], "spatial_resolution": "High-resolution (WorldView-3, sub-meter) for HSROI; ~10m for Sentinel-2 TMsI; ~10m for Sentinel-1 TSARI", "geographic_coverage": "Global; globe partitioned into 4096 regions covering diverse scenarios across resolution, spectrum, and imaging mechanism"}, "downstream_tasks": ["Semantic segmentation", "Horizontal object detection", "Oriented object detection", "Change detection", "Scene classification (single-label)", "Scene classification (multi-label)", "Multi-modal segmentation (land cover mapping)", "Multi-modal segmentation (crop mapping)", "Multi-modal scene classification"], "downstream_datasets": ["Dyna.-Pla.", "iSAID", "Potsdam", "Dyna.-S2", "DIOR", "DIOR-R", "FAIR1M", "LEVIR-CD", "OSCD", "AID", "RESISC-45", "BEN-S2", "fMoW-S2", "Dyna.-MM", "PASTIS-MM", "BEN-MM"], "code_available": false, "code_url": null, "weights_available": true, "key_contribution": "Largest (2.06B params) multi-modal remote sensing foundation model to date, featuring a factorized multi-modal spatiotemporal encoder, Multi-Granularity Contrastive Learning across pixel/object/image levels, and unsupervised Geo-Context Prototype Learning for region-aware representations. Achieves SOTA on 16 datasets across 7 tasks, outperforming 18 prior RSFMs.", "_source_pdf": "SkySense.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection", "year": 2022, "model_name": "SaDL", "architecture": {"backbone": "ResNet-18 with FPN", "type": "CNN (Siamese)", "params_millions": 15.09, "notes": "ResNet-18 backbone (without FC layers and global pooling) equipped with Feature Pyramid Network; output feature map at 1/4 spatial resolution with channel dimension C=256. 2-layer MLP projector (hidden 2048, output 1024) and 2-layer MLP predictor (hidden 256, output 1024)."}, "pretraining": {"method": "Semantic-aware Dense Representation Learning (SaDL)", "method_details": "SimSiam-style Siamese self-supervised framework with semantic supervision. Generates three views per image (two via color/geometry augmentations, one via background swap with color transfer + context-aware blending). Performs class-balanced point sampling within the overlapped region using semantic masks, then enforces point-level cross-view similarity (negative cosine similarity with stop-gradient) between views 1/2 and views 1/3 plus a semantic dissimilarity loss pushing foreground vs. background point embeddings apart.", "objective": "Combined loss: semantic dissimilar loss (L_sd) + cross-view similarity losses (L_s1 between view 1/2, L_s2 between view 1/3 for foreground)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Inria Aerial Image Labeling (building segmentation)"], "num_images": "~45k 256x256 patches cut from 180 images of size 5000x5000 (building/non-building masks)", "sensors": ["Aerial RGB"], "modalities": ["RGB optical imagery", "building semantic masks"], "spatial_resolution": "0.3 m", "geographic_coverage": "Cities covered by Inria dataset (e.g., multiple urban regions across US and Europe)"}, "downstream_tasks": ["Change detection (building)"], "downstream_datasets": ["LEVIR-CD", "WHU-CD", "Guangzhou-CD"], "code_available": true, "code_url": "https://github.com/justchenhao/SaDL_CD", "weights_available": false, "key_contribution": "Proposes SaDL, a semantic-aware dense self-supervised pre-training method for remote sensing change detection that combines class-balanced point sampling with semantic masks, point-level cross-view consistency, semantic dissimilarity loss, and a background-swap third view, yielding spatially-sensitive and foreground-discriminative features that significantly improve data efficiency on LEVIR-CD, WHU-CD, and Guangzhou-CD.", "_source_pdf": "SemanticAware.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model", "year": 2025, "model_name": "HyperSIGMA", "architecture": {"backbone": "ViT (Vision Transformer) with dual subnetworks (SpatViT spatial + SpecViT spectral) using Sparse Sampling Attention (SSA) and Spectral Enhancement Module (SEM)", "type": "Vision Transformer", "params_millions": 1000, "notes": "Two-branch architecture: spatial subnetwork (SpatViT) processes spatial patches and spectral subnetwork (SpecViT) processes spectral channel tokens. Full SA replaced with Sparse Sampling Attention (SSA) in most layers. Features fused via Spectral Enhancement Module (SEM). Scaled to over 1 billion parameters using ViT-Huge backbone for both subnetworks. Base/Large/Huge variants available."}, "pretraining": {"method": "Masked Image Modeling (MAE)", "method_details": "MAE-based self-supervised pre-training applied separately to both spatial and spectral subnetworks. Spatial mask ratio R_spat=0.75 and spectral mask ratio R_spec=0.75. Patch size P=8 for spatial ViT (64 tokens). Random channel selection used during preprocessing to handle varying channel counts. Pre-trained for 1,600 epochs on NVIDIA V100 GPUs.", "objective": "Reconstruction of masked patches/spectral tokens", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["HyperGlobal-450K"], "num_images": "~450K hyperspectral images (447,072 HSI patches: 247,072 EO-1 + 200,000 GF-5), equivalent to over 20 million trispectral images", "sensors": ["EO-1 (Earth Observing One)", "GF-5 (Gaofen-5)"], "modalities": ["hyperspectral"], "spatial_resolution": "64x64 patches", "geographic_coverage": "Global (EO-1 acquired 2011-2017 worldwide; GF-5 covering China)"}, "downstream_tasks": ["hyperspectral image classification", "hyperspectral target detection", "hyperspectral anomaly detection", "hyperspectral change detection", "hyperspectral unmixing", "hyperspectral image denoising", "hyperspectral image super-resolution", "multispectral change detection", "hyperspectral-multispectral collaborative classification", "oil spill detection"], "downstream_datasets": ["Indian Pines", "Pavia University", "HanChuan", "HongHu", "Houston (HU)", "ZY1-02D Yellow River Delta", "Xiongan", "Mosaic", "AVIRIS", "Renourban", "Pavia (HAD)", "Cri", "Viareggio", "Hermiston", "Farmland", "Bay Area", "Santa Barbara", "Urban", "Washington DC Mall (WDC Mall)", "Houston (super-resolution)", "YRE_2", "OSCD", "Gulf of Mexico oil spill"], "code_available": true, "code_url": "https://github.com/WHU-Sigma/HyperSIGMA", "weights_available": true, "key_contribution": "First billion-parameter foundation model specifically designed for hyperspectral image (HSI) interpretation, introducing Sparse Sampling Attention (SSA) and Spectral Enhancement Module (SEM) within a dual spatial-spectral ViT architecture, pre-trained on the newly constructed HyperGlobal-450K dataset (largest global HSI dataset). Provides a unified solution for both high-level and low-level hyperspectral tasks.", "_source_pdf": "HyperSIGMA.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation", "year": 2025, "model_name": "RingMoE", "architecture": {"backbone": "Swin Transformer v2", "type": "Hierarchical Mixture-of-Experts (MoE) Transformer", "params_millions": 14700, "notes": "Sparse MoE architecture with modal-specialized, collaborative, and shared experts (RMoE layer replaces FFN in each Swin v2 block). Supports dynamic expert pruning to produce sub-models: RingMoE-Opt (4.3B), RingMoE-EP (4.3B), and dense 1B variants (RingMoE-KS, RingMoE-KA, RingMoE-KC). Largest RSFM to date at 14.7B parameters."}, "pretraining": {"method": "Masked Image Modeling (MIM) with physics-informed self-supervised learning", "method_details": "Random masking of multi-modal patches; modal-specific decoders reconstruct original targets. Pixel reconstruction for Opt/MS/SAR-L2 and power reconstruction (physics-informed) for complex-valued SAR-L1. Load-balance loss for collaborative and modal-specialized experts.", "objective": "Masked reconstruction loss (pixel reconstruction + power reconstruction for SAR-L1) combined with expert load-balance loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["RingMOSS"], "num_images": "400 million", "sensors": ["Optical", "Multi-spectral (incl. infrared)", "SAR-L1 (complex-valued, GF-3)", "SAR-L2 (amplitude)"], "modalities": ["Optical", "Multi-spectral", "SAR-L1", "SAR-L2"], "spatial_resolution": "Varied (multi-platform, nine satellites)", "geographic_coverage": "Global, six continents and 57 countries"}, "downstream_tasks": ["Scene classification", "Object detection", "Semantic segmentation", "Object tracking", "Change detection", "Depth estimation", "Few-shot classification", "Few-shot segmentation"], "downstream_datasets": ["NWPU-RESISC45", "AID", "UCM", "WHU-RS19", "iSAID", "LoveDA", "Potsdam", "Vaihingen", "Dyna.-pla", "SARSeg-L1", "AIR-POLSAR-SEG", "WHU-OPT-SAR", "DFC23", "DIOR", "DIOR-R", "HRSC2016", "SAR-AIRcraft-1.0", "AIR-SARShip", "AIR-MOT", "AIR-HSAO", "LEVIR-CD", "CDD", "ISPRS Vaihingen (Depth)", "ISPRS Potsdam (Depth)"], "code_available": true, "code_url": "https://github.com/HanboBizl/RingMoEDatasets", "weights_available": false, "key_contribution": "First sparse Mixture-of-Experts foundation model for remote sensing, scaled to 14.7B parameters across four modalities (Opt, MS, SAR-L1, SAR-L2). Introduces modal-specialized, collaborative, and shared experts (RMoE), physics-informed SSL with power reconstruction for complex-valued SAR, and dynamic expert pruning for efficient deployment (down to 1B). Achieves 23 SOTAs across 25 benchmarks on six RS tasks.", "_source_pdf": "RingMoE.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoning", "year": 2025, "model_name": "RingMo-Agent", "architecture": {"backbone": "SigLIP-SO400M-384 visual encoder + DeepSeekMoE (3B) LLM with MLP projector and trajectory decoder", "type": "Vision-Language Model (MLLM)", "params_millions": 3000, "notes": "Visual encoder based on SigLIP with three modality-specific patch embedding layers (optical, SAR, IR) projecting to 1152-dim space; frozen MLP projector (DeepSeek-VL2 style with 2x2 shuffling); LLM is 3B DeepSeekMoE variant from DeepSeek-VL2 fine-tuned with LoRA (rank 64, scaling 16) on Q/V projections; trainable GRU-based trajectory decoder for long-horizon spatial tasks via <trajectory> token."}, "pretraining": {"method": "Two-stage training: vision-language generation stage then instruction-tuning stage", "method_details": "Stage 1 (vision-language generation): adapts model to RS domain using paired image-text data from classification, captioning, and detection datasets, optimized via generative LLM loss; GPT-4 used to expand annotations. Stage 2 (instruction tuning): optimized to follow instructions across 8 RS tasks. Training on 8x A100 80GB GPUs with AdamW; loss combines cross-entropy text loss and MSE trajectory loss (lambda_txt=1, lambda_mse=5).", "objective": "Generative language modeling (cross-entropy) + MSE trajectory regression", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["RS-VL3M"], "num_images": "over 3 million image-text pairs", "sensors": ["optical", "SAR", "infrared (IR)"], "modalities": ["optical", "SAR", "infrared", "text"], "spatial_resolution": "variable; images dynamically tiled to multiples of 384", "geographic_coverage": "global; satellite and UAV platforms over diverse urban and natural scenes"}, "downstream_tasks": ["image captioning", "object detection (including visual grounding)", "scene classification", "visual question answering (VQA)", "relation reasoning", "instruction decomposition", "action decision", "task scheduling (trajectory/navigation planning)"], "downstream_datasets": ["CityNav", "SkyAgent-Plan3k", "ReCon1M-DEC", "FIT-RS", "ReCon1M-REL", "DIOR", "DOTA", "NWPU VHR-10", "SARDet-100k", "IR-DET", "UCM-Captions", "Sydney-Captions", "RSICD", "NWPU-Captions", "SAR-CAP", "IR-CAP", "AID", "NWPU-RESISC45", "WHU-RS19", "UCMerced-LandUse", "SAR-CLA", "IR-CLA", "RSVQA-HR", "RSVQA-LR"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First unified RS vision-language foundation model handling multi-platform (satellite + UAV) and multi-modal (optical, SAR, IR) data across 8 perception and reasoning tasks, supported by the new 3M-pair RS-VL3M dataset, modality-specific embedding layers to mitigate cross-modal interference, and a token-based high-dimensional hidden state trajectory decoder enabling long-horizon spatial reasoning such as aerial navigation.", "_source_pdf": "RingMo-Agent.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "RS-DFM: A Remote Sensing Distributed Foundation Model for Diverse Downstream Tasks", "year": 2024, "model_name": "RS-DFM", "architecture": {"backbone": "Swin-Tiny", "type": "Transformer", "params_millions": null, "notes": "Distributed foundation model with Swin-Tiny encoder, Generalized BEV Generation (GBG) module, and High-Low Frequency Decoupled Collaboration (HLFDC) module for multi-platform collaborative perception"}, "pretraining": {"method": "Supervised multi-task collaborative training", "method_details": "Trained with AdamW optimizer (lr 2e-4), batch size 8, 20 epochs on four NVIDIA A40 GPUs. Uses BEVDET for 3D detection, Fiery for BEV instance segmentation, PowerBEV for trajectory prediction as downstream heads.", "objective": "Multi-task collaborative perception loss across detection, segmentation, and trajectory prediction", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["AirCo-MultiTasks"], "num_images": "38,000 synchronous images (170 training scenes, 30 validation scenes)", "sensors": ["RGB optical camera"], "modalities": ["RGB"], "spatial_resolution": "1600x900 pixels per image; collaborative region ~150m x 150m", "geographic_coverage": "CARLA simulated urban scenes (33 locations from high-quality simulation maps)"}, "downstream_tasks": ["3D object detection", "BEV instance segmentation", "trajectory prediction", "multi-object tracking"], "downstream_datasets": ["AirCo-MultiTasks"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "A distributed foundation model for multi-platform (multi-UAV) collaborative remote sensing perception, introducing a Generalized BEV Generation module that uses geometric priors with relative depth estimation to unify features across altitudes/perspectives, and a High-Low Frequency Decoupled Collaboration module for bandwidth-efficient inter-platform feature transmission, along with the AirCo-MultiTasks simulation dataset.", "_source_pdf": "RS-DFM.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "HSIGene: A Foundation Model for Hyperspectral Image Generation", "year": 2024, "model_name": "HSIGene", "architecture": {"backbone": "Latent Diffusion Model (UNet) with ControlNet encoder and VAE", "type": "Latent Diffusion Model (Controllable Generative Model)", "params_millions": 1500, "notes": "Based on CRS-Diff framework; consists of a pre-trained VAE encoder, UNet diffusion backbone, and ControlNet encoder for multi-condition control. Also includes a two-stage super-resolution framework: DSRNet (diffusion-based RGB super-resolution) and RGAN (Rectangular Guided Attention Network) for HSI super-resolution. UNet initialized from CRS-Diff pretrained weights."}, "pretraining": {"method": "Latent Diffusion (DDPM training with DDIM sampling)", "method_details": "Trained for 100k iterations using AdamW optimizer with learning rate 1e-5 and batch size 16. Forward diffusion process gradually adds noise to latent representations; model trained to predict noise via MSE loss. Supports six control conditions: HED (holistically-nested edge detection), segmentation, sketch, MLSD (multiscale line segment detection), content (CLIP image embeddings), and text. Conditions integrated via ControlNet encoder with zero-convolution modules.", "objective": "Noise prediction (MSE between predicted and added Gaussian noise) in latent space for controllable HSI generation", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["Xiongan", "Chikusei", "DFC2013", "DFC2018", "Heihe"], "num_images": "~7k training patches (256x256, stride 128) plus ~40k augmented patches via super-resolution; ~55k high-resolution RGB images from Google Earth Engine for DSRNet training", "sensors": ["Airborne Multi-Modality Imaging Spectrometer", "Headwall Hyperspec-VNIR-C", "ITRES CASI-1500", "CASI"], "modalities": ["Hyperspectral (HSI)", "RGB"], "spatial_resolution": "0.5m to 2.5m GSD (Xiongan 0.5m, Chikusei 2.5m, DFC2013 2.5m, DFC2018 1m, Heihe 1m)", "geographic_coverage": "China (Xiongan, Heihe), Japan (Chikusei), and DFC2013/DFC2018 benchmark scenes; spectral range 380-1018nm aligned to 400-1000nm with 48 bands"}, "downstream_tasks": ["Hyperspectral image denoising", "Hyperspectral image super-resolution", "Controllable HSI synthesis"], "downstream_datasets": ["Heihe", "Chikusei", "AID (farmland scenes for evaluation)"], "code_available": true, "code_url": "https://github.com/LiPang/HSIGene", "weights_available": true, "key_contribution": "First and largest HSI foundation model for hyperspectral image generation based on latent diffusion, supporting multi-condition composable control (HED, segmentation, sketch, MLSD, content, text). Introduces a novel two-stage super-resolution data augmentation framework (DSRNet + RGAN with rectangular guided attention) to alleviate HSI scarcity while preserving spectral fidelity, enabling generation of high-quality synthetic HSIs that improve downstream denoising and super-resolution performance.", "_source_pdf": "HSIGene.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Model", "year": 2025, "model_name": "RS-vHeat", "architecture": {"backbone": "vHeat-B with Heat Conduction Operator (HCO) and Spatial Correction Embeddings (SCEs)", "type": "Heat-conduction-based visual encoder (Swin-B-style 4-stage hierarchy with 2,2,18,2 blocks)", "params_millions": 148, "notes": "Multi-modal (optical + SAR) encoder using HCO with O(N^1.5) complexity and global receptive field; learnable Frequency Value Embeddings (FVEs) predict heat diffusion coefficients; Spatial Correction Embeddings adjust object boundaries"}, "pretraining": {"method": "Self-supervised frequency domain hierarchical masking with multi-domain reconstruction and contrastive learning", "method_details": "Inputs split via DCT into high/low-frequency components with sector-region masking (rate 20-30%); dual-stream optical and SAR pipelines feed heat-conduction-based encoder; multi-domain decoders reconstruct in spatial and frequency domains; contrastive loss across modalities", "objective": "Combined loss: spatial domain reconstruction (L1) + frequency domain reconstruction (L1) + contrastive loss (L_total = L_Con + L_Spa + L_Fre)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["RingMo-style multi-modal dataset (450k matched optical-SAR pairs, ~3M total entries)"], "num_images": "3,000,000", "sensors": ["optical", "SAR"], "modalities": ["optical RGB", "SAR"], "spatial_resolution": "224x224 image patches (high-resolution RS imagery)", "geographic_coverage": "not specified"}, "downstream_tasks": ["semantic segmentation", "object detection", "scene classification", "change detection"], "downstream_datasets": ["Potsdam", "iSAID", "AIR-PolSAR-Seg", "WHU-OPT-SAR", "DIOR", "FAIR1M-2.0", "SAR-AIRcraft-1.0", "LEVIR-CD", "AID", "NWPU-RESISC45"], "code_available": true, "code_url": null, "weights_available": false, "key_contribution": "First remote sensing foundation model based on heat conduction theory, using a Heat Conduction Operator with O(N^1.5) complexity and global receptive field; combined with frequency-domain hierarchical masking and multi-domain reconstruction self-supervised pretraining on multi-modal optical+SAR data, achieving 84% memory reduction, 24% FLOPs reduction, and 2.7x throughput improvement over attention-based RSFMs while matching or exceeding SOTA accuracy across 4 tasks and 10 datasets", "_source_pdf": "RS-vHeat.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?", "year": 2025, "model_name": "SatDiFuser", "architecture": {"backbone": "DiffusionSat (Stable Diffusion v2-1 based Latent Diffusion Model with U-Net)", "type": "Latent Diffusion Model (U-Net with ResNet, self-attention, and cross-attention blocks)", "params_millions": null, "notes": "Built on DiffusionSat, a satellite-adapted LDM based on Stable Diffusion v2-1. Extracts multi-scale, multi-timestep features from the U-Net decoder (ResNet and self-attention outputs at timesteps {1, 100, 200}) and aggregates them via three proposed fusion strategies: Global Weighted Fusion, Localized Weighted Fusion, and Mixture-of-Experts (MoE) Joint Fusion (E=8 experts, top-k=2)."}, "pretraining": {"method": "Diffusion-based generative self-supervised learning (repurposed as SSL feature extractor)", "method_details": "Leverages a pretrained generative diffusion model (DiffusionSat) as a frozen backbone; only SatDiFuser fusion components and task-specific decoders (linear head for classification, UPerNet for segmentation) are trained downstream. Multi-scale, multi-timestep features are obtained via DDIM inversion from clean latents.", "objective": "Denoising diffusion (noise prediction) objective inherited from DiffusionSat / Stable Diffusion v2-1", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["DiffusionSat pretraining data (large-scale satellite imagery)", "Stable Diffusion v2-1 web-scraped data (for SD v2-1 backbone variant)"], "num_images": "not specified", "sensors": ["multi-spectral satellite (RGB bands used in SatDiFuser experiments)"], "modalities": ["optical RGB"], "spatial_resolution": "not specified", "geographic_coverage": "global"}, "downstream_tasks": ["semantic segmentation", "image classification"], "downstream_datasets": ["GEO-Bench", "m-pv4ger-seg", "m-nz-cattle", "m-NeonTree", "m-cashew-plantation", "m-SA-crop-type", "m-chesapeake-landcover", "m-bigearthnet", "m-brick-kiln", "m-eurosat", "m-forestnet", "m-pv4ger", "m-so2sat"], "code_available": true, "code_url": "https://github.com/yurujaja/SatDiFuser", "weights_available": false, "key_contribution": "First comprehensive adaptation of a large-scale diffusion-based generative model as a self-supervised geospatial foundation model for discriminative remote sensing tasks, introducing three multi-stage feature fusion strategies (global weighted, localized weighted, and mixture-of-experts) that enable diffusion features to rival or exceed state-of-the-art GFMs on GEO-Bench (gains up to +5.7% mIoU in segmentation and +7.9% F1 in classification).", "_source_pdf": "SatDiFuser.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding", "year": 2023, "model_name": "CMID", "architecture": {"backbone": "ResNet-50 / Swin-B", "type": "CNN and ViT (architecture-agnostic)", "params_millions": 88, "notes": "Teacher-student self-distillation with three branches (MIM, global CL, local CL). Reported with ResNet-50 (23.55M) and Swin-B (87.77M) backbones; ViT-B/ViTAE-B variants also discussed."}, "pretraining": {"method": "Contrastive Mask Image Distillation (CMID)", "method_details": "Unified SSL framework combining contrastive learning (CL) with masked image modeling (MIM) via teacher-student self-distillation. Three branches: MIM branch (SimMIM-style with mean spectral value masking and learnable [MASK] token, using spatial L1 loss and focal frequency loss), global CL branch (MoCo-style InfoNCE on globally pooled features), and local CL branch (position-matched local prototype assignment consistency). Teacher updated via EMA of student.", "objective": "L_spat (L1 spatial reconstruction) + L_freq (focal frequency loss) + L_NCE (global contrastive InfoNCE) + L_local (local prototype assignment consistency)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID"], "num_images": "~1 million", "sensors": ["Optical (RGB aerial/satellite from Google Earth)"], "modalities": ["Optical RGB"], "spatial_resolution": "variable (sub-meter to several meters)", "geographic_coverage": "global"}, "downstream_tasks": ["Scene classification", "Semantic segmentation", "Object detection (oriented bounding box)", "Change detection"], "downstream_datasets": ["UC Merced (UCM)", "Potsdam", "Vaihingen", "DOTA", "CDD"], "code_available": true, "code_url": "https://github.com/NJU-LHRS/official-CMID", "weights_available": true, "key_contribution": "First unified SSL framework for remote sensing that combines contrastive learning (CL) with masked image modeling (MIM) via teacher-student self-distillation, learning representations that are simultaneously globally semantic separable and locally spatially perceptible. Architecture-agnostic (works with both CNNs and ViTs), enabling strong transfer across classification, segmentation, detection, and change detection.", "_source_pdf": "CMID.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment", "year": 2023, "model_name": "GRAFT", "architecture": {"backbone": "ViT-B/16 (also ViT-B/32)", "type": "Vision Transformer", "params_millions": 86, "notes": "Satellite image encoder initialized from CLIP image encoder; trained to align with frozen CLIP. Two variants: image-level VLM and pixel-level VLM (patch-based ViT producing per-patch features)."}, "pretraining": {"method": "Ground Remote Alignment for Training (GRAFT) - contrastive alignment between satellite images and co-located internet ground images via frozen CLIP encoder", "method_details": "Trains a satellite image encoder to align satellite embeddings with CLIP embeddings of co-located internet ground images, using ground images as an intermediary to transitively align satellite imagery with text. Uses a multi-positive contrastive loss accounting for multiple ground images per satellite image. Pixel-level model uses geotag-to-pixel mapping for sparse patch-level supervision.", "objective": "Multi-positive contrastive loss (CLIP-style with multiple positives per satellite image)", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["NAIP-Flickr ground-satellite pairs (collected)", "Sentinel-2-Flickr ground-satellite pairs (collected)"], "num_images": "10.2 million NAIP pairs and 8.7 million Sentinel-2 pairs", "sensors": ["NAIP aerial imagery", "Sentinel-2"], "modalities": ["RGB satellite/aerial imagery", "internet ground-level images (via Flickr)"], "spatial_resolution": "1 m (NAIP), 10 m (Sentinel-2)", "geographic_coverage": "United States (NAIP); global (Sentinel-2)"}, "downstream_tasks": ["zero-shot image classification", "text-to-image retrieval", "zero-shot semantic segmentation", "zero-shot visual question answering (VQA)"], "downstream_datasets": ["EuroSAT", "BigEarthNet", "SAT-4", "SAT-6", "NAIP-OSM", "Satlas segmentation (NAIP and Sentinel-2)", "RSVQA (high-resolution)"], "code_available": false, "code_url": null, "weights_available": true, "key_contribution": "First large-scale vision-language foundation model for remote sensing trained without any textual annotations, by using co-located internet ground images as an intermediary between satellite imagery and language; achieves SOTA zero-shot classification, retrieval, segmentation, and VQA on satellite imagery, outperforming supervised VLMs by up to 20% on classification and 80% on segmentation.", "_source_pdf": "GRAFT.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "DynamicVis: Dynamic Visual Perception for Efficient Remote Sensing Foundation Models", "year": 2025, "model_name": "DynamicVis", "architecture": {"backbone": "Dynamic Region-Aware State Space Model (SSM) with hierarchical multi-scale extractor and Feature Pyramid Network", "type": "State Space Model (Mamba-based) with adaptive token routing", "params_millions": 132, "notes": "Two configurations: DynamicVis-Base (~186M) and DynamicVis-Large (~998M) per Table 12. Uses progressive small-stride patch merger (patch sizes [7,3,3,3], strides [4,2,2,2]), 4 hierarchical stages with Sparse Mixer blocks containing ATRIM (Adaptive Token Routing & Incremental Modeling) units with dual-path bidirectional Mamba scanning. Stage-wise token reduction ratios [7/8, 3/4, 1/2, 0]. FPN produces 5 multi-scale feature maps at d=256 channels."}, "pretraining": {"method": "Region-Level Meta-Embedding Multi-Instance Learning (MIL)", "method_details": "Treats each image as a 'bag' of instances with bounding box and categorical labels. Aligns regional visual embeddings (extracted via Generic RoI Extractor across all hierarchical scales) with categorical meta-embeddings initialized from pre-trained OpenCLIP text encoder using prompt templates (e.g., 'An aerial image of a [CLASS]'). Trained via MIL-NCE contrastive loss combined with classification cross-entropy. Gumbel noise injected into routing affinity scores, progressively annealed.", "objective": "MIL-NCE contrastive loss + classification cross-entropy", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["fMoW (Functional Map of the World) - RGB version"], "num_images": "1,027,691 training images (with 20,000 hold-out test images)", "sensors": ["DigitalGlobe satellite constellation (WorldView)"], "modalities": ["RGB optical"], "spatial_resolution": "sub-meter high-resolution (resampled to 512x512 for pre-training)", "geographic_coverage": "Over 200 countries worldwide"}, "downstream_tasks": ["tiny ship detection", "instance segmentation", "zero-shot image retrieval", "road extraction (semantic segmentation)", "building extraction (semantic segmentation)", "bi-temporal change detection", "scene classification", "region classification"], "downstream_datasets": ["LEVIR-Ship", "NWPU VHR-10", "SSDD", "BigEarthNet", "ForestNet", "Massachusetts Roads", "WHU Building", "LEVIR-CD", "WHU-CD", "OSCD", "UC-Merced", "AID", "fMoW"], "code_available": true, "code_url": "https://github.com/KyanChen/DynamicVis", "weights_available": true, "key_contribution": "A remote sensing foundation model explicitly tailored to spatial sparsity of RS imagery via a Dynamic Region-Aware SSM backbone that adaptively routes only task-relevant high-salience tokens through dual-path Mamba scanning while preserving background context via parameter-free integration. Combined with a novel Region-Level Meta-Embedding MIL pre-training paradigm, it processes 2048x2048 imagery with only 97 ms latency and 833 MB memory (~6% and 3% of a base ViT) while achieving SOTA on sparse-target tasks.", "_source_pdf": "DynamicVis.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data", "year": 2025, "model_name": "AlphaEarth Foundations (AEF)", "architecture": {"backbone": "Space Time Precision (STP) transformer with interleaved space (ViT-like spatial self-attention), time (time-axial self-attention), and precision (3x3 conv) operators; trio of teacher video embedding model with implicit decoders, student video embedding model, and text alignment model", "type": "Transformer (custom STP video encoder with implicit decoders)", "params_millions": 480, "notes": "Trained ~1B and ~480M parameter variants; deployed the smaller 480M variant for inference efficiency. Outputs 64-byte embeddings (64-D) constrained to unit sphere S^63 via von Mises-Fisher distribution; quantized to 8 bits for release."}, "pretraining": {"method": "Self-supervised contrastive video teacher-student with implicit decoder reconstruction, batch uniformity objective on S^63, and CLIP-style text alignment", "method_details": "Teacher-student video embeddings share architecture; embeddings treated as mean direction of von Mises-Fisher distribution and decoded via implicit decoders conditioned on sensor metadata and timecode. Batch uniformity loss prevents collapse. Text alignment via CLIP-style contrastive loss between video student and text encoder using Wikipedia articles joined to GBIF species occurrences.", "objective": "Implicit decoder reconstruction losses per source + batch uniformity (uniform distribution on S^63) + contrastive text-video alignment (f_CLIP)", "is_self_supervised": true, "is_vision_language": true}, "pretraining_data": {"datasets": ["Sentinel-2", "Landsat 8/9", "Sentinel-1", "PALSAR-2 (ALOS-2)", "GEDI", "ERA5-Land", "GRACE", "GLO-30", "NLCD", "Wikipedia + GBIF species occurrences"], "num_images": "~3 billion observations across nine gridded data sources and one unstructured text source, covering ~1.1% of Earth's land surface", "sensors": ["Sentinel-2 MSI (optical)", "Landsat 8/9 OLI (optical)", "Sentinel-1 SAR (radar)", "PALSAR-2 (radar)", "GEDI (LiDAR)", "GLO-30 (DEM)", "ERA5-Land (climate reanalysis)", "GRACE (gravimetry)", "NLCD (land cover labels)"], "modalities": ["optical multispectral", "SAR/radar", "LiDAR", "DEM/elevation", "climate/environmental reanalysis", "land cover annotations", "text"], "spatial_resolution": "10 m (resampled to common grid)", "geographic_coverage": "Global terrestrial surface (approximately +/-82 degrees latitude)"}, "downstream_tasks": ["Thematic mapping (land cover/land use classification)", "Crop type classification", "Tree genera classification", "Species/plantation distribution mapping", "Biophysical variable regression (evapotranspiration, surface emissivity)", "Change detection (land cover change, land use change)"], "downstream_datasets": ["LCMAP land cover", "LCMAP land use", "LCMAP land cover change", "LCMAP land use change", "LUCAS land cover", "LUCAS land use", "GLaNCE land cover", "Africa crop mask", "Canada crops (fine)", "Canada crops (coarse)", "Ethiopia crops", "US trees (GBIF)", "Descals oil palm", "OpenET ensemble (evapotranspiration)", "ASTER GED (surface emissivity)"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First EO featurization model with continuous-time conditioning that fuses multi-source/multi-modal spatial-temporal data via a Space Time Precision (STP) architecture, producing compact 64-byte 10 m global embedding fields. AEF consistently outperforms designed and learned featurization baselines (~23.9% mean error reduction) across 15 sparse-label evaluations and is released as annual global embedding layers (2017-2024) on Google Earth Engine.", "_source_pdf": "AlphaEarth.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series", "year": 2025, "model_name": "TiMo", "architecture": {"backbone": "Hierarchical Vision Transformer (Swin-style, 4 stages) with Differential Spatiotemporal Gyroscope Attention (D-STGA) in early stages and MHSA in deeper stages", "type": "Hierarchical Vision Transformer", "params_millions": 298, "notes": "Three variants: TiMo-Base (91M), TiMo-Large (298M), TiMo-Huge (675M). Replaces MHSA with novel Spatiotemporal Gyroscope Attention (STGA / D-STGA) in the first two stages to capture spatiotemporal correlations among tokens sharing spatial or temporal positions."}, "pretraining": {"method": "Masked Image Modeling (Hiera-style MAE adapted to hierarchical ViT)", "method_details": "Follows Hiera's MAE-like approach for hierarchical architectures with mask unit of 1x32x32 pixels (sparser temporal masking suited for SITS). Uses MHSA in all blocks during pre-training. Random temporal sampling of 3 frames per iteration from 10-timestamp sequences.", "objective": "Reconstruction of masked spatiotemporal patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionST"], "num_images": "~1,000,000 Sentinel-2 SITS samples from 100,000 geographic locations across 1,317 cities, each with 10 temporal phases over 5 years (2017-2022)", "sensors": ["Sentinel-2"], "modalities": ["multispectral optical"], "spatial_resolution": "10 m (Sentinel-2)", "geographic_coverage": "Global, primarily Europe, North Africa, and West Asia (1,317 cities ranked among the 10,000 most populous globally)"}, "downstream_tasks": ["deforestation monitoring", "multi-class land cover segmentation", "crop type classification", "flood disaster assessment (classification)", "flood segmentation"], "downstream_datasets": ["MultiEarth", "MultiSenGE", "MTLCC", "Sen12Flood", "KuroSiwo"], "code_available": true, "code_url": "https://github.com/MiliLab/TiMo", "weights_available": true, "key_contribution": "A hierarchical vision transformer foundation model for satellite image time series, introducing Spatiotemporal Gyroscope Attention (STGA) and its differential variant (D-STGA) that compute attention only among tokens sharing the same spatial or temporal coordinate to efficiently capture multiscale spatiotemporal relationships, pretrained via masked image modeling on the new MillionST dataset (1M Sentinel-2 SITS from 100k locations x 10 timestamps).", "_source_pdf": "TiMo.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "CGEarthEye: A High-Resolution Remote Sensing Vision Foundation Model Based on the Jilin-1 Satellite Constellation", "year": 2025, "model_name": "CGEarthEye", "architecture": {"backbone": "ViT (S/B/L/H/G)", "type": "Vision Transformer", "params_millions": 1100, "notes": "Five backbone variants (ViT-S 22M, ViT-B 86M, ViT-L 307M, ViT-H 632M, ViT-G 1100M), totaling 2.1B parameters across all variants. Teacher-student framework with EMA momentum update."}, "pretraining": {"method": "Self-supervised multi-granularity contrastive learning", "method_details": "Multi-granularity SSL framework integrating augmentation-aware contrastive learning, seasonal alignment contrastive learning, and masked patch token contrastive learning. Teacher-student architecture inspired by DINOv2; 2 global crops, 8 local crops, 3 seasonal views, plus masked variants (15 augmented variants per image). EMA momentum 0.992. Trained 150 days on 16 NVIDIA A800 GPUs.", "objective": "L = L_classtoken + L_classtoken_season + L_patch (cross-entropy contrastive losses)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["JLSSD (Jilin-1 Self-supervised Seasonal Dataset)"], "num_images": "15 million", "sensors": ["Jilin-1 satellite constellation"], "modalities": ["optical RGB"], "spatial_resolution": "0.75m (sub-meter)", "geographic_coverage": "Global (8.06M quarterly samples from 2.015M Chinese locations + 7.985M annual mosaic samples from non-Chinese locations, 2023)"}, "downstream_tasks": ["scene classification", "object detection", "semantic segmentation", "change detection"], "downstream_datasets": ["RESISC-45", "AID", "fMoW", "DIOR", "DIOR-R", "LoveDA", "iSAID", "Potsdam", "LEVIR-CD", "SYSU-CD", "CDD"], "code_available": true, "code_url": "https://github.com/1921134176/CGEarthEye", "weights_available": true, "key_contribution": "First high-resolution (sub-meter) remote sensing vision foundation model trained on the Jilin-1 satellite constellation, introducing JLSSD—the first 15-million-scale globally distributed multi-temporal SSL dataset with quarterly sampling—and a multi-granularity SSL framework combining augmentation-aware, seasonal, and masked patch contrastive learning, achieving SOTA across 10 high-resolution benchmarks under frozen-backbone fine-tuning.", "_source_pdf": "CGEarthEye.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models", "year": 2025, "model_name": "MAPEX", "architecture": {"backbone": "ViT-Base", "type": "Mixture-of-Modality Experts (MoE) ViT", "params_millions": 360, "notes": "Pre-trained mixture-of-modality experts encoder with 6 experts (~360M params); after modality-aware pruning yields specialized models (~130M with 2 experts per modality, ~200M with 2 experts + 1 shared). 8-block transformer decoder for reconstruction. Includes learnable [MODALITY] tokens, modality-conditioned token routing, and load-balancing loss."}, "pretraining": {"method": "Masked Image Modeling (MAE) with Mixture-of-Modality Experts", "method_details": "Multi-modal masked autoencoding with 75% masking per modality, modality-wise decoding (separate decoder forward passes per modality), modality-conditioned token routing in MoE layers, modality dropout (50%), and a differentiable load-balancing loss. Trained 50 epochs with AdamW.", "objective": "Mean-squared error pixel reconstruction of masked patches per modality", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet-GeoEnvironment (ben-ge)"], "num_images": "590,326 colocated samples", "sensors": ["Sentinel-1 SAR", "Sentinel-2 MSI", "Elevation/DEM", "Environmental data"], "modalities": ["RGB (S2 B04,B03,B02)", "Red Edge (S2 B05,B06,B07)", "SWIR (S2 B11,B12,B8A)", "NIR (S2 B08)", "SAR (S1 VV,VH)", "Elevation (ELE)"], "spatial_resolution": "10m (Sentinel-2 RGB/NIR), 20m (Red Edge/SWIR), 10m (Sentinel-1 SAR)", "geographic_coverage": "Europe (BigEarthNet coverage)"}, "downstream_tasks": ["Land-cover classification (multi-label)", "Flood detection (binary classification)", "Wildfire detection (binary classification)", "Semantic segmentation"], "downstream_datasets": ["ben-ge-8k", "SEN12-FLOOD", "California Wildfire (Sentinel-2 + CAL FIRE perimeters)"], "code_available": true, "code_url": "https://github.com/HSG-AIML/MAPEX", "weights_available": false, "key_contribution": "Introduces a multi-modal pre-training approach using a mixture-of-modality experts ViT with modality-conditioned token routing, plus a modality-aware expert pruning technique that turns a large multi-modal foundation model into smaller, efficient modality-specific models tailored to downstream tasks.", "_source_pdf": "MAPEX.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensing", "year": 2025, "model_name": "SkySense-V2", "architecture": {"backbone": "Unified transformer encoder with Swin Transformer V2 Blocks (SwinV2B) in stages 1-2 and vanilla Transformer Blocks in stages 3-4, with Adaptive Patch Merging (APM) modules and Mixture of Experts (MoE)", "type": "Hierarchical transformer (unified multi-modal)", "params_millions": 665, "notes": "Single shared backbone across HR optical, MS, and SAR modalities (except distinct tokenizers); integrates APM for resolution harmonization, modality-specific learnable prompt tokens, and MoE (L=6, M=8, k=1) in last L transformer layers. Token dimension C=352, window size 8 in first two stages."}, "pretraining": {"method": "Self-supervised multi-modal contrastive learning with teacher-student EMA framework", "method_details": "Teacher-student architecture (EMA) with Multi-Granularity Contrastive Learning (MGCL), Geo-Context Prototype Learning (GCPL), Image-Text Alignment (ITA) using OpenStreetMap labels, and novel Query-based Semantic Aggregation Contrastive Learning (QSACL). Trained for 600k iterations on 128 H20 GPUs with batch size 1024, AdamW optimizer, lr 2e-4 cosine annealed to 1e-6.", "objective": "Weighted sum of MGCL, ITA, and QSACL losses", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SkySense pretraining dataset (same as SkySense)"], "num_images": "~21 million multi-modal RS imagery sets", "sensors": ["High-resolution optical RGB", "Sentinel-2 (MS, 10 bands)", "Sentinel-1 (SAR, 2 bands)"], "modalities": ["HR optical RGB", "Multi-spectral", "SAR", "Temporal time series", "Text (OSM labels for ITA)"], "spatial_resolution": "HR optical 2048x2048; Sentinel-2 and Sentinel-1 64x64 (avg 65 S2 images and SAR sequences per set)", "geographic_coverage": "Global (same as SkySense)"}, "downstream_tasks": ["Scene classification", "Semantic segmentation", "Horizontal object detection", "Oriented object detection", "Change detection", "Multi-modal segmentation", "Multi-modal scene classification"], "downstream_datasets": ["AID", "RESISC-45", "BEN-S2", "fMoW-S2", "iSAID", "Potsdam", "Dyna-Pla", "Dyna-S2", "DIOR", "DIOR-R", "FAIR1M", "LEVIR-CD", "OSCD", "Dyna.-S2 (change)", "Dyna-MM", "PASTIS-MM", "BEN-MM"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "A unified MM-RSFM that uses a single transformer backbone (665M params) to process HR optical, MS, and SAR modalities simultaneously, with novel Adaptive Patch Merging for resolution harmonization, modality-specific learnable prompt tokens, MoE scaling, and a new Query-based Semantic Aggregation Contrastive Learning (QSACL) SSL strategy tailored to RS images' multi-subject semantic distribution; outperforms SkySense by 1.8 points on average across 16 datasets and 7 tasks.", "_source_pdf": "SkySense-V2.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis", "year": 2025, "model_name": "TESSERA", "architecture": {"backbone": "Dual-branch Transformer encoder (4-layer multi-head self-attention) with GRU pooling and MLP fusion", "type": "Transformer", "params_millions": 45.7, "notes": "Dual-branch encoder processes Sentinel-1 and Sentinel-2 d-pixels separately; each branch uses linear projection + Day-of-Year positional encoding + 4-layer Transformer + GRU pooling to 128-D; outputs concatenated and fused via 2-layer MLP to 128-D fused embedding; quantization-aware training compresses to int8; deep projector MLP expands to 16,384-D during training (discarded at inference)"}, "pretraining": {"method": "Self-supervised redundancy reduction (Barlow Twins) with mix-up regularization and global shuffling", "method_details": "Two augmented views generated per d-pixel by independently sampling 40 valid timesteps; trained with Barlow Twins loss plus mix-up regularization loss (lambda_mix=1.0, lambda_BT=5e-3); uses global data shuffling to decorrelate spatial neighborhoods; one epoch with global batch size 32,768; AdamW optimizer (lr=0.002, weight decay 1e-6) with linear warmup and cosine decay; trained on 16 AMD MI300X GPUs (192GB each)", "objective": "Barlow Twins cross-correlation loss (invariance + redundancy reduction) + mix-up cross-correlation loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-1", "Sentinel-2"], "num_images": "~800 million d-pixels from 3,012 global MGRS tiles (2017-2024)", "sensors": ["Sentinel-1 SAR", "Sentinel-2 MSI"], "modalities": ["SAR", "optical/multispectral"], "spatial_resolution": "10 m", "geographic_coverage": "Global (3,012 MGRS tiles worldwide)"}, "downstream_tasks": ["classification", "segmentation", "regression"], "downstream_datasets": ["TreeSatAI-TS", "PASTIS-R", "Austrian Crop (classification)", "Austrian Crop (segmentation)", "Biomassters", "Borneo Canopy Height"], "code_available": true, "code_url": "https://github.com/ucam-eo/tessera", "weights_available": true, "key_contribution": "A pixel-wise self-supervised foundation model for multi-modal Sentinel-1/2 time series that learns label-efficient embeddings via temporal sampling invariance, introducing the d-pixel representation, global shuffling, and mix-up regularization; releases FAIR global, annual, 10 m pixel-wise int8 embeddings as analysis-ready 'embeddings-as-data' through the GeoTessera Python library, achieving state-of-the-art accuracy across classification, segmentation, and regression benchmarks with minimal computation", "_source_pdf": "TESSERA.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model", "year": 2024, "model_name": "CRS-Diff", "architecture": {"backbone": "Stable Diffusion 1.5 U-Net + ControlNet", "type": "Latent Diffusion Model", "params_millions": 900, "notes": "Two-stage training: fine-tunes SD 1.5 backbone (~0.9B params) for RS text-to-image, then stacks ControlNet-style conditional branch with multi-scale Attention Feature Fusion (AFF) for image/metadata conditions. CLIP ViT-L-14 (123M) text encoder fine-tuned on RSICD."}, "pretraining": {"method": "Diffusion model fine-tuning with multi-conditional control", "method_details": "Stage 1: fine-tune SD 1.5 U-Net on RSICD text-image pairs (10 epochs, AdamW, lr=1e-5, 512x512). Stage 2: train ControlNet-style conditional branch on 200k image-text pairs from fMoW + Million-AID with decoupled conditions (HED, MLSD, Depthmap, Sketch, Seg mask, Road map, Content, Caption, Metadata) using AFF for multi-scale fusion (5 epochs, lr=1e-4). Classifier-free guidance with random condition dropout.", "objective": "Denoising score matching (noise prediction MSE) with classifier-free guidance", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["RSICD", "fMoW", "Million-AID"], "num_images": "~210,921 (10,921 RSICD + 200,000 image-text pairs from fMoW and Million-AID)", "sensors": ["aerial RGB", "satellite RGB"], "modalities": ["RGB imagery", "text captions", "metadata (year, month, day, GSD, latitude, longitude, cloud cover)", "HED edges", "MLSD line segments", "depth maps", "sketches", "semantic segmentation masks", "road maps", "content embeddings"], "spatial_resolution": "224x224 (RSICD, fMoW) and varying (Million-AID); resized to 512x512 for training", "geographic_coverage": "global (fMoW, Million-AID)"}, "downstream_tasks": ["text-to-image generation", "controllable image generation (single-condition)", "multi-condition image generation", "data augmentation for downstream tasks (e.g., road extraction)"], "downstream_datasets": ["RSICD"], "code_available": true, "code_url": "https://github.com/Sonettoo/CRS-Diff", "weights_available": false, "key_contribution": "First multiple-condition controllable diffusion-based generative model for remote sensing imagery, supporting simultaneous text, metadata, and image (HED, MLSD, depth, sketch, segmentation, roadmap, content) conditions via a multi-scale Attention Feature Fusion mechanism over a ControlNet-augmented Stable Diffusion backbone.", "_source_pdf": "CRS-Diff.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised Pretraining", "year": 2024, "model_name": "SenPa-MAE", "architecture": {"backbone": "ViT-Base (12 encoder layers, 12 heads; 3 decoder layers, 12 heads)", "type": "Transformer (Masked Autoencoder)", "params_millions": null, "notes": "ViT-Base MAE-style encoder-decoder with embedding dim 768, patch generated per-patch and per-channel; includes a Sensor Parameter Encoding module (two 5-layer MLPs g^SRF and g^GSD) injecting spectral response function and GSD embeddings"}, "pretraining": {"method": "Masked Autoencoder (MAE) with Sensor Parameter Encoding", "method_details": "Per-channel patch tokenization (N = C x HW/P^2 tokens), 66% masking ratio, sensor parameter encoding module added to positional embeddings before encoder (and optionally before decoder), Spectral Superposition Augmentation (SSA) and resolution augmentation; trained 400 epochs, Adam optimizer, lr 1e-4 cosine decay, weight decay 0.05, batch size 128, 144x144 inputs with 4 randomly selected channels", "objective": "Masked mean absolute error reconstruction of unmasked input", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Custom multi-sensor dataset (Sentinel-2, Planet SuperDove, Landsat-8/9)"], "num_images": "~116k patches of 256x256 px from 26 training locations (50k S2, 50k SD, 25k LS)", "sensors": ["Sentinel-2", "Planet SuperDove", "Landsat-8/9"], "modalities": ["multispectral optical"], "spatial_resolution": "~3.5 m (SD), 10 m and 20 m (S2), 30 m (LS); resampled to 5 m pixel spacing", "geographic_coverage": "Global, sampled from 29 locations worldwide"}, "downstream_tasks": ["land cover semantic segmentation"], "downstream_datasets": ["ESA WorldCover v200"], "code_available": true, "code_url": "https://github.com/JonathanPrexl/SenPa-MAE", "weights_available": true, "key_contribution": "Introduces a sensor-parameter-aware MAE that injects per-channel spectral response functions and GSD into ViT embeddings via a versatile encoding module, plus a Spectral Superposition Augmentation strategy, enabling multi-satellite self-supervised pretraining and sensor-independent inference across Sentinel-2, Planet SuperDove and Landsat.", "_source_pdf": "SenPa-MAE.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "One for All: Toward Unified Foundation Models for Earth Vision", "year": 2024, "model_name": "OFA-Net", "architecture": {"backbone": "Vision Transformer (ViT)", "type": "Transformer", "params_millions": null, "notes": "Single shared Transformer backbone with modality-specific patch embedding layers for five modalities; inputs resized to 224x224"}, "pretraining": {"method": "Masked Image Modeling", "method_details": "MAE-style masked image modeling with modality-specific decoders to reconstruct masked patches; does not require spatially aligned multi-modal data; pre-trained for 100 epochs on 50,000 samples (10,000 per sub-dataset)", "objective": "Masked patch reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-1 (SatlasPretrain)", "Sentinel-2 (SatlasPretrain)", "Gaofen", "NAIP (SatlasPretrain)", "EnMAP (HySpecNet-11k)"], "num_images": "~8.08M total curated (4,642,353 S1 + 977,774 S2 + 117,450 Gaofen + 2,332,351 NAIP + 11,483 EnMAP); 50,000 used for pretraining", "sensors": ["Sentinel-1 SAR", "Sentinel-2 MSI", "Gaofen", "NAIP aerial", "EnMAP hyperspectral"], "modalities": ["SAR", "multispectral", "RGB+NIR", "RGB aerial", "hyperspectral"], "spatial_resolution": "1 m (NAIP), 4 m (Gaofen), 10 m (Sentinel-2), 5x20 m (Sentinel-1), 30 m (EnMAP)", "geographic_coverage": "Global (Sentinel-1/2, EnMAP); USA (NAIP); China (Gaofen)"}, "downstream_tasks": ["image classification", "semantic segmentation"], "downstream_datasets": ["m-bigearthnet", "m-forestnet", "m-brick-kiln", "m-pv4ger", "m-so2sat", "m-eurosat", "m-pv4ger-seg", "m-nz-cattle", "m-NeonTree", "m-cashew-plantation", "m-SA-crop-type", "m-chesapeake-landcover"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "A unified foundation model using a single shared Transformer backbone with modality-specific patch embeddings to handle multiple remote sensing modalities and spatial resolutions simultaneously, pre-trained via masked image modeling without requiring spatially aligned multi-modal data.", "_source_pdf": "OFA-Net.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation", "year": 2024, "model_name": "LeMeViT", "architecture": {"backbone": "Hierarchical Vision Transformer with Cross Attention, Dual Cross-Attention (DCA), and Standard Attention blocks; learnable meta tokens (M=16)", "type": "Vision Transformer", "params_millions": 53.1, "notes": "Three variants: Tiny (8.64M), Small (16.04M), Base (53.10M). Uses learnable meta tokens initialized via cross-attention, then DCA blocks for efficient information exchange between image and meta tokens (linear complexity O(2MN) vs O(N^2))."}, "pretraining": {"method": "Supervised pretraining", "method_details": "Pretrained on MillionAID for 100 epochs following RSP setting; ImageNet-1K trained from scratch for 300 epochs with AdamW and cosine decay, following DeiT recipe.", "objective": "Supervised classification", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["ImageNet-1K", "MillionAID"], "num_images": "~1.3M (ImageNet-1K) + ~1M (MillionAID, 1,000,848 scenes)", "sensors": ["RGB aerial imagery"], "modalities": ["RGB"], "spatial_resolution": "Variable (MillionAID image sizes 110x110 to 31,672x31,672)", "geographic_coverage": "Global (MillionAID)"}, "downstream_tasks": ["Image Classification", "Scene Recognition", "Object Detection", "Semantic Segmentation", "Change Detection"], "downstream_datasets": ["ImageNet-1K", "MillionAID", "DOTA", "ISPRS Potsdam", "CDD"], "code_available": true, "code_url": "https://github.com/ViTAE-Transformer/LeMeViT", "weights_available": true, "key_contribution": "Proposes a hardware-friendly hierarchical Vision Transformer using a small number of learnable meta tokens and a Dual Cross-Attention (DCA) block to reduce self-attention complexity from quadratic to linear, achieving ~1.7x speedup with competitive accuracy on remote sensing tasks.", "_source_pdf": "LeMeViT.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images", "year": 2025, "model_name": "SMARTIES", "architecture": {"backbone": "ViT-B / ViT-L", "type": "Transformer encoder-decoder (MAE-style)", "params_millions": null, "notes": "Vanilla MAE-style ViT encoder-decoder with sensor-agnostic spectrum-aware projection layers and reprojection layers; spectrum-aware embedding size D=768 (ViT-B) / 1024 (ViT-L)"}, "pretraining": {"method": "Masked Image Modeling with cross-sensor token mixup", "method_details": "Spectrum-aware projection layers tokenize bands per wavelength range into a shared spectrum-aware space; pairs of co-located images from different sensors are mixed via cross-sensor token mixup; masked tokens (75% ratio) are reconstructed via spectrum-aware reprojection layers using MSE loss", "objective": "Masked patch reconstruction (MSE) in spectrum-aware space", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW-RGB", "fMoW-S2", "BigEarthNet-MM"], "num_images": "~496K (60K fMoW pairs + 188K BigEarthNet pairs)", "sensors": ["Maxar (submeter RGB)", "Sentinel-2 (multispectral)", "Sentinel-1 (SAR)"], "modalities": ["RGB", "Multispectral", "SAR"], "spatial_resolution": "submeter to 60m (multi-resolution)", "geographic_coverage": "Global"}, "downstream_tasks": ["Multi-label scene classification", "Scene classification", "Semantic segmentation", "Crop-type segmentation", "Burn scar detection", "Land-use/land-cover mapping", "Building detection"], "downstream_datasets": ["BigEarthNet-S1", "BigEarthNet-S2", "BigEarthNet-MM", "EuroSAT", "RESISC-45", "WHU-RS19", "UCMerced", "BurnScars", "DynamicEarthNet", "SpaceNet7", "SICKLE"], "code_available": true, "code_url": "https://gsumbul.github.io/SMARTIES", "weights_available": true, "key_contribution": "A single sensor-agnostic foundation model that projects heterogeneous RS sensors into a shared spectrum-aware space, enabling unified pretraining across diverse sensors, scalable training without sensor-specific backbones, and generalization to unseen sensors via projection-layer interpolation.", "_source_pdf": "SMARTIES.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation", "year": 2025, "model_name": "TerraFM", "architecture": {"backbone": "ViT-Base / ViT-Large (16x16 patch, 224x224 input)", "type": "Vision Transformer", "params_millions": null, "notes": "Modality-specific patch embeddings with cross-attention fusion module aggregating modality-specific tokens via learnable spatial queries; shared transformer encoder; teacher-student framework with EMA-updated teacher and three-layer projection head."}, "pretraining": {"method": "Self-supervised contrastive learning (DINO-style student-teacher with multi-crop, modality augmentation, cross-attention fusion, dual centering)", "method_details": "Builds on DINO with teacher-student EMA distillation; multi-crop learning (2 global + 6 local crops); modalities treated as natural augmentations stochastically assigned to student/teacher; cross-attention fusion with N_q=5 learnable spatial queries; dual-centering scheme combining global center with class-frequency-aware center using WorldCover-derived LULC statistics to address long-tailed distributions.", "objective": "Cross-entropy consistency between teacher and student softmax outputs across views (contrastive self-distillation)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Major-TOM (filtered subset enriched with ESA WorldCover, Köppen-Geiger climate zones, ESRI World Regions metadata)"], "num_images": "18.7M modality-specific training tiles (from 1.5M curated 10.68km x 10.68km grid cells, each split into 534x534 tiles)", "sensors": ["Sentinel-1 SAR (RTC)", "Sentinel-2 L1C", "Sentinel-2 L2A"], "modalities": ["SAR", "Multispectral optical (TOA)", "Multispectral optical (BOA)"], "spatial_resolution": "10-60 m", "geographic_coverage": "Globally distributed terrestrial regions sampled with land-cover, climate-zone, and world-region priors (98% of ocean tiles removed)"}, "downstream_tasks": ["Scene classification", "Multi-label classification", "Semantic segmentation", "Cloud/cloud-shadow segmentation", "Landslide detection"], "downstream_datasets": ["m-EuroSat", "m-BigEarthNet", "m-So2Sat", "m-Brick-Kiln", "m-Cashew-Plant", "m-SA-Crop-Type", "Cloud-S2 (Copernicus-Bench)", "EuroSAT-S1", "EuroSAT-S2", "BigEarthNet-S1", "BigEarthNet-S2", "DFC2020-S1", "DFC2020-S2", "LCZ-S2", "Landslide4Sense"], "code_available": true, "code_url": "https://github.com/mbzuai-oryx/TerraFM", "weights_available": true, "key_contribution": "A scalable multisensor EO foundation model that unifies Sentinel-1 SAR and Sentinel-2 L1C/L2A optical inputs via modality-specific patch embeddings and cross-attention fusion, treats sensing modalities as natural augmentations within a DINO-style contrastive teacher-student framework, and introduces a dual-centering mechanism with class-frequency-aware regularization (using WorldCover priors) to mitigate long-tailed LULC distributions, achieving SoTA on GEO-Bench and Copernicus-Bench.", "_source_pdf": "TerraFM.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Harnessing Massive Satellite Imagery with Efficient Masked Image Modeling", "year": 2024, "model_name": "SelectiveMAE", "architecture": {"backbone": "ViT-B / ViT-L", "type": "Vision Transformer", "params_millions": 307, "notes": "Evaluated with both ViT-B (86M) and ViT-L (307M) backbones; uses asymmetric encoder-decoder with lightweight cross-attention decoder following CrossMAE."}, "pretraining": {"method": "Masked Image Modeling (MIM)", "method_details": "SelectiveMAE: efficient MIM with Progressive Semantic Token Selection (PSTS) module. Uses HOG (Histogram of Oriented Gradients) features to quantify patch semantic richness, encoding only ~15% of semantically rich tokens and reconstructing only ~25% of patches. Progressively shifts from semantically similar (easy) to complementary (hard) tokens during training via curriculum-style selection. Achieves 2x+ pre-training speedup over MAE.", "objective": "Pixel reconstruction (MSE loss) on selected patch tokens", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["OpticalRS-13M"], "num_images": "13.2M (OpticalRS-13M); also OpticalRS-4M lightweight version with 3.92M images", "sensors": ["WorldView", "QuickBird", "GeoEye", "other high-resolution optical satellite platforms"], "modalities": ["optical RGB (visible light)"], "spatial_resolution": "high-resolution visible light imagery (avg ~199,214 pixels per image in OpticalRS-13M)", "geographic_coverage": "Diverse global coverage curated from multiple publicly available RS datasets via exclusion, slicing, and deduplication; 12 main scene/object/event categories"}, "downstream_tasks": ["scene classification", "horizontal object detection", "oriented object detection", "semantic segmentation"], "downstream_datasets": ["AID", "RESISC-45", "DIOR", "DIOR-R", "LoveDA", "SpaceNetv1"], "code_available": true, "code_url": "https://github.com/Fengxiang23/SelectiveMAE", "weights_available": true, "key_contribution": "Introduces (1) OpticalRS-13M, a large-scale (13M images) curated optical RS pre-training dataset 4x larger than prior RS datasets, and (2) SelectiveMAE, an efficient MIM method with Progressive Semantic Token Selection using HOG features that achieves >2x pre-training speedup over MAE while improving downstream RS task performance.", "_source_pdf": "SelectiveMAE.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Pattern Integration and Enhancement Vision Transformer for Self-supervised Learning in Remote Sensing", "year": 2024, "model_name": "PIEViT", "architecture": {"backbone": "ViT-B/16", "type": "Vision Transformer", "params_millions": 88.2, "notes": "Dual-stream teacher-student architecture with Geospatial Pattern Cohesion (GPC) module and Feature Integration Projection (FIP) module. Teacher updated via EMA."}, "pretraining": {"method": "Self-distillation with masked image modeling", "method_details": "Teacher-student framework where teacher network is updated via Exponential Moving Average (EMA, decay 0.99) of student. GPC module aggregates geospatially clustered patches based on cohesion scores; FIP module uses cross-attention blocks to refine masked token reconstruction. Combines image-level (CLS) and patch-level (MIM) objectives. Mask ratio 0.1-0.5.", "objective": "Combined CLS-level cross-entropy loss and patch-level MIM loss between student and teacher predictions", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Million-AID"], "num_images": "~1,000,000", "sensors": ["Optical (RGB aerial/satellite)"], "modalities": ["RGB"], "spatial_resolution": "Variable (high-resolution remote sensing scene imagery)", "geographic_coverage": "Global"}, "downstream_tasks": ["Object Detection", "Land Cover Classification (Semantic Segmentation)", "Change Detection"], "downstream_datasets": ["DIOR", "Potsdam", "LEVIR-CD"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Introduces PIEViT, a self-supervised ViT for remote sensing that combines a Geospatial Pattern Cohesion (GPC) module to aggregate geospatially clustered patches and a Feature Integration Projection (FIP) module to refine masked token reconstruction, enabling discrimination of internal patch features in scenographic remote sensing imagery and achieving SOTA on object detection, land cover classification, and change detection.", "_source_pdf": "PIEViT.pdf", "_extraction_timestamp": "2026-04-17T00:00:00"}
{"title": "Self-Supervised Spatio-Temporal Representation Learning Of Satellite Image Time Series", "year": 2023, "model_name": "U-BARN", "architecture": {"backbone": "Unet (spectro-spatial encoder) + Transformer", "type": "Hybrid CNN-Transformer (Unet-BERT spAtio-temporal Representation eNcoder)", "params_millions": null, "notes": "Patch embedding via Unet-based spectro-spatial encoder (SSE) with 4 down/up-sampling levels; Transformer block with 3 layers, 4 heads, d_model=64, d_hidden=128; positional encoding based on day-of-year (DOY). Temporal attention applied at full spatial resolution on pixel-level time series."}, "pretraining": {"method": "BERT-style masked reconstruction", "method_details": "Generative pretext task inspired by BERT: a percentage (M_rate=60%) of SSE-encoded patch embeddings across timestamps are masked/corrupted by replacing with embeddings from another date, another pixel in the batch, or another patch. A shallow MLP decoder reconstructs the original S2 patch time series from the latent representation. Loss computed only on corrupted patches and only on valid (non-cloudy/in-swath) pixels using a validity mask.", "objective": "Masked patch reconstruction with MSE loss on valid pixels of corrupted patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Sentinel-2 L2A unlabeled dataset over France (13 S2 tiles, 2018-2020)"], "num_images": "13 S2 tiles x 10 ROIs of 1024x1024 over 2018-2020 (annual SITS, ~64x64 patch time series used during training)", "sensors": ["Sentinel-2"], "modalities": ["optical multispectral"], "spatial_resolution": "10 m (20 m bands resampled to 10 m); 4 bands at 10 m + 6 bands at 20 m", "geographic_coverage": "France"}, "downstream_tasks": ["semantic segmentation (crop type classification)", "semantic segmentation (dense land cover classification)"], "downstream_datasets": ["PASTIS", "MultiSenGE"], "code_available": true, "code_url": "https://src.koda.cnrs.fr/iris.dumeur/ssl_ubarn", "weights_available": false, "key_contribution": "A novel Unet+Transformer spatio-temporal architecture (U-BARN) for irregularly sampled, multi-variate Sentinel-2 SITS, combined with a BERT-inspired generative self-supervised pre-training strategy that masks SSE-encoded patch embeddings and reconstructs valid pixels, enabling rich spatio-temporal representations that improve crop and land cover semantic segmentation, especially under label scarcity.", "_source_pdf": "U-BARN.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data", "year": 2025, "model_name": "MAESTRO", "architecture": {"backbone": "Vision Transformer (ViT-B)", "type": "Masked Autoencoder (MAE)", "params_millions": null, "notes": "MAE-derived architecture with modality-specific tokenizers (shared across time steps within a modality, distinct across modalities), 2D spatial positional encodings based on ground sampling distance (as in ScaleMAE), and sine-cosine temporal encodings (as in SatMAE). Uses inter-group token-based fusion mode: early fusion across time steps and within groups of similar modalities, with the last three encoder blocks replaced by fusion blocks enabling cross-group token interactions."}, "pretraining": {"method": "Masked Autoencoding (MAE) with token-based early fusion across time steps and similar modalities, late fusion across dissimilar modalities, and joint-token early fusion for multispectrality with patch-group-wise target normalization", "method_details": "Two-stage masking: structured masking per modality, spatial, and temporal dimensions, plus unstructured adjustment to reach 75% overall masking ratio. Reconstruction targets are normalized patch-group-wise, where spectral bands are partitioned into groups of highly correlated bands and normalized independently per patch and band group, injecting a useful spectral prior. Three-phase workflow: SSL pre-training, probing, and fine-tuning.", "objective": "Masked patch reconstruction with patch-group-wise normalized targets (L1 loss)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["FLAIR-HUB", "S2-NAIP urban", "TreeSatAI-TS (intra-dataset)", "PASTIS-HD (intra-dataset)", "FLAIR#2 (intra-dataset)"], "num_images": "FLAIR-HUB: 63 GPixels; S2-NAIP urban: 64 GPixels", "sensors": ["Aerial (VHR RGB+NIR, ~0.2 m)", "SPOT 6-7 (VHR, resampled to 1 m)", "Sentinel-1 SAR (ascending and descending)", "Sentinel-2 multispectral", "DEM/DSM elevation"], "modalities": ["VHR optical (aerial/SPOT)", "Sentinel-1 SAR time series", "Sentinel-2 multispectral time series", "Elevation (DEM/DSM)"], "spatial_resolution": "0.2 m (aerial), 1 m (SPOT 6-7 resampled), 1.25 m (NAIP), 10 m (Sentinel-2), variable (Sentinel-1)", "geographic_coverage": "France (FLAIR-HUB, FLAIR#2, PASTIS-HD), Germany (TreeSatAI-TS), United States urban areas (S2-NAIP urban)"}, "downstream_tasks": ["Tree species classification (multi-label)", "Crop type semantic segmentation", "Land cover semantic segmentation"], "downstream_datasets": ["TreeSatAI-TS", "PASTIS-HD", "FLAIR#2", "FLAIR-HUB"], "code_available": true, "code_url": "https://github.com/ignf/maestro", "weights_available": true, "key_contribution": "A tailored MAE adaptation for EO data that orchestrates multimodal, multitemporal, and multispectral inputs via inter-group token-based fusion (early fusion across time steps and similar modalities, late fusion across dissimilar ones) combined with joint-token multispectral fusion using a novel patch-group-wise target normalization that injects a spectral prior at negligible computational cost; achieves SOTA on tasks relying on multitemporal dynamics while remaining competitive on others.", "_source_pdf": "MAESTRO.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Geography-Aware Self-Supervised Learning", "year": 2021, "model_name": "GASSL", "architecture": {"backbone": "ResNet-50", "type": "CNN", "params_millions": 25.6, "notes": "MoCo-v2 framework with ResNet-50 query and key encoders, plus a linear geo-location classifier head."}, "pretraining": {"method": "Contrastive learning + geo-location pretext", "method_details": "Extends MoCo-v2 with TemporalInfoNCE using spatially aligned temporal positive pairs, combined with a geo-location classification pretext task on K=100 k-Means clusters of (lat, lon). Joint loss L = alpha*L_temporal + beta*L_geo (alpha=beta=1). 200 epochs, batch size 256, SGD, lr 1e-3, dictionary queue 65536, temperature 0.2.", "objective": "TemporalInfoNCE contrastive loss + cross-entropy geo-location classification loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["fMoW (Functional Map of the World)", "GeoImageNet (geo-tagged ImageNet subset via FLICKR API)"], "num_images": "~363,571 fMoW training images; 543,435 GeoImageNet images", "sensors": ["Multi-source high-resolution satellite (fMoW)", "RGB natural images (GeoImageNet)"], "modalities": ["RGB"], "spatial_resolution": "sub-meter to a few meters (fMoW high-resolution satellite)", "geographic_coverage": "Global (seven continents for fMoW; worldwide geo-tagged for GeoImageNet)"}, "downstream_tasks": ["image classification", "object detection", "semantic segmentation", "land cover classification"], "downstream_datasets": ["fMoW", "xView", "SpaceNet", "NAIP (California Central Valley land cover)", "GeoImageNet"], "code_available": true, "code_url": "https://geography-aware-ssl.github.io", "weights_available": true, "key_contribution": "Introduces geography-aware self-supervised learning that exploits the spatio-temporal structure of remote sensing data via temporal positive pairs (TemporalInfoNCE) and a geo-location classification pretext task, closing the gap between contrastive and supervised learning on remote sensing benchmarks and improving transfer to detection, segmentation, and land cover tasks.", "_source_pdf": "GASSL.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data", "year": 2025, "model_name": "TEOChat", "architecture": {"backbone": "CLIP ViT-L/14 image encoder + 2-layer MLP vision-language connector + LLaMA-2 (Vicuna) LLM decoder", "type": "Vision-Language Model (LLaVA-1.5/Video-LLaVA-style)", "params_millions": null, "notes": "Temporally-shared CLIP ViT-L/14 image encoder, Video-LLaVA initialization, LLM fine-tuned with LoRA and 8-bit quantization; supports temporal sequences up to 8 EO images"}, "pretraining": {"method": "Visual instruction tuning", "method_details": "Initialized from Video-LLaVA (which uses CLIP-pretrained ViT-L/14 and Vicuna/LLaMA-2). Vision encoder frozen; LLM fine-tuned with LoRA; vision-language projector fully fine-tuned. Trained for 2 epochs (~14k iters) on TEOChatlas using next-token cross-entropy loss on response tokens.", "objective": "Next-token cross-entropy on instruction responses (visual instruction tuning)", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["TEOChatlas (245,210 temporal EO instruction examples + 308,861 single-image GeoChat_Instruct examples = 554,071 total)", "fMoW", "fMoW-Sentinel", "xBD", "S2Looking", "QFabric", "GeoChat_Instruct"], "num_images": "554,071 instruction-following examples (1,244,393 images)", "sensors": ["8 sensors spanning the EO datasets including Sentinel-2", "WorldView", "GeoEye", "QuickBird", "and other high-resolution satellite sensors"], "modalities": ["RGB optical", "multispectral (Sentinel-2)"], "spatial_resolution": "mixed (sub-meter high-resolution to 10m Sentinel-2)", "geographic_coverage": "global (considerable geographic diversity across fMoW, xBD, S2Looking, QFabric)"}, "downstream_tasks": ["temporal scene classification", "change detection (building damage assessment, building change detection)", "spatial change referring expression", "change question answering", "region-based change question answering", "temporal referring expression", "region-based temporal question answering", "single-image scene classification", "visual question answering", "image captioning"], "downstream_datasets": ["fMoW (RGB and Sentinel)", "xBD", "S2Looking", "QFabric", "GeoChat single-image benchmarks"], "code_available": true, "code_url": "https://github.com/ermongroup/TEOChat", "weights_available": true, "key_contribution": "First vision-language model for temporal sequences of Earth observation imagery, along with TEOChatlas, the first temporal EO instruction-following dataset; outperforms prior EO VLMs (GeoChat) and video VLMs (Video-LLaVA), rivals specialist models, and surpasses GPT-4o and Gemini-1.5 Pro on multiple temporal EO tasks.", "_source_pdf": "TEOChat.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing", "year": 2024, "model_name": "EarthMarker", "architecture": {"backbone": "Mixture of Visual Experts (MoV: DINOv2-ViT-L/14 + CLIP-ConvNeXt-L) + Llama 2 13B LLM decoder", "type": "Multi-modal Large Language Model (MLLM) with visual prompting", "params_millions": 13000, "notes": "Sharing visual encoding mechanism processes visual prompts (boxes/points) as RGB images. Modality-align projection layer connects vision to LLM. LoRA adapters added to LLM in final tuning stage. Disjoint parameters strategy across training phases."}, "pretraining": {"method": "Three-phase cross-domain progressive learning: (1) multi-domain image-text alignment, (2) spatial perception tuning, (3) RS visual prompting tuning with LoRA", "method_details": "Phase 1 trains only projection layer on COCO + RSVP image-level data; Phase 2 unfreezes LLM attention layers using RefCOCO/RefCOCO+; Phase 3 applies LoRA adapters and trains on RSVP region/point-text pairings. MoV visual encoders kept frozen throughout.", "objective": "Visual instruction tuning / next-token prediction conditioned on visual prompts and text instructions", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["COCO Captions", "RefCOCO", "RefCOCO+", "RSVP (constructed, ~3.65M image-point-text and image-region-text pairings)", "NWPU-Captions", "RSITMD", "Sydney-Captions", "DIOR-RSVG", "NWPU-RESISC45", "OPTIMAL-31", "RSD46", "WHU-RS19", "DOTA V2", "FAIR1M", "NWPUVHR10", "RSOD", "UCAS-AOD", "VisDrone", "MAR20", "DOSR", "LEVIR", "HRSC2016", "HRRSD", "Vaihingen", "Potsdam", "Hi-UCD", "isAID", "UAVid", "SOTA", "FAST", "WHU", "GeoChat instruction data"], "num_images": "~3.65M image-point-text and image-region-text pairings in RSVP", "sensors": ["optical (aerial/satellite, including Google Earth imagery)", "VHR optical", "UAV/drone"], "modalities": ["RGB imagery", "text instructions", "visual prompts (boxes and points)"], "spatial_resolution": "Mixed: from 0.05m (Potsdam) to 30m, including VHR (<0.5m) and medium-to-low resolution", "geographic_coverage": "Broad global coverage (most raw data sourced from Google Earth)"}, "downstream_tasks": ["scene classification", "image captioning", "region captioning", "referring object classification", "relationship analyses", "grounded captioning", "detailed captioning", "point-level referring object classification"], "downstream_datasets": ["AID", "UCMerced", "NWPU-Captions", "DIOR-RSVG"], "code_available": true, "code_url": "https://github.com/wivizhang/EarthMarker", "weights_available": true, "key_contribution": "First visual prompting multi-modal large language model for remote sensing, enabling multi-granularity (image/region/point-level) RS imagery interpretation via box and point visual prompts. Introduces a sharing visual encoding mechanism (visual prompts as images), a cross-domain three-phase training strategy with disjoint parameters, and the RSVP dataset (~3.65M visual prompting instruction pairs) — the first RS visual prompting instruction dataset.", "_source_pdf": "EarthMarker.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding", "year": 2024, "model_name": "SkySenseGPT", "architecture": {"backbone": "CLIP-ViT-L/14 (336x336) vision encoder + MLP projector + Vicuna-v1.5 LLM", "type": "Vision-Language Multi-Modal Model (LMM)", "params_millions": 7000, "notes": "Follows mainstream LMM design (LLaVA-v1.5 style); input images resized to 504x504; pre-trained MLP projector from LLaVA-v1.5; LoRA used to fine-tune the LLM, vision encoder frozen, projector fine-tuned"}, "pretraining": {"method": "Instruction tuning (LoRA fine-tuning of LLM on FIT-RS plus public RS datasets)", "method_details": "Vision encoder (CLIP-ViT-L/14) frozen; MLP projector fine-tuned; Vicuna-v1.5 LLM fine-tuned via LoRA on combined instruction tuning data (1415k from FIT-RS train/val + 365k from public datasets). Trained on 4x NVIDIA A100 (40G).", "objective": "Autoregressive instruction-following (next-token prediction) for fine-grained remote sensing vision-language understanding", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["FIT-RS (built on STAR scene graph dataset)", "NWPU", "UCM", "RSITMD", "EarthVQA", "FloodNet-VQA", "RSVQA-LR", "DOTA-v2.0", "DIOR", "FAIR1M"], "num_images": "FIT-RS contains 1,800,851 instruction samples derived from 82,532 image patches (cropped from 1,273 large-size VHR RSIs); ~1415k FIT-RS train/val instruction samples + 365k from public datasets used for instruction tuning", "sensors": ["Optical (very-high-resolution aerial/satellite RGB)"], "modalities": ["RGB imagery", "Text (instructions/QA/scene graphs)"], "spatial_resolution": "Very-high-resolution (VHR); large-size RSIs ranging from 512x768 to 27,860x31,096 pixels, cropped via 512x512 sliding window with 100-pixel overlap", "geographic_coverage": "Worldwide VHR remote sensing imagery covering complex semantic scenarios (airports, dams, ports, etc.)"}, "downstream_tasks": ["Detailed Image Captioning", "Detailed Region Captioning", "Visual Question Answering", "Multi-Label Scene Classification", "Scene Classification", "Relation Detection", "Relation Reasoning", "Object Detection", "Object Reasoning", "Region-Level Scene Graph Generation", "Image-Level Scene Graph Generation", "Multi-Turn/Task Conversation", "Visual Grounding"], "downstream_datasets": ["FIT-RS (FIT-RSFG benchmark)", "FIT-RSRC", "SIRI-WHU", "AID", "WHU-RS19", "AID-Multilabel", "RSVQA-HR", "RSVQA-LR"], "code_available": true, "code_url": "https://github.com/Luo-Z13/SkySenseGPT", "weights_available": true, "key_contribution": "Introduces FIT-RS, a large-scale (1.8M samples) fine-grained instruction tuning dataset for remote sensing built from scene graph annotations, plus the FIT-RSRC benchmark for relation comprehension; proposes SkySenseGPT, an RSLMM that excels at fine-grained relation reasoning and scene graph generation, surpassing prior RSLMMs on public benchmarks.", "_source_pdf": "SkySenseGPT.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding", "year": 2023, "model_name": "SatlasPretrain", "architecture": {"backbone": "Swin Transformer (Swin-Base)", "type": "Transformer", "params_millions": 88, "notes": "SatlasNet uses Swin-Base backbone with temporal max pooling across image time series, multi-scale features at 4 scales, and seven task-specific output heads (U-Net for segmentation/regression, Faster R-CNN for points, Mask R-CNN for polygons, U-Net + post-processing for polylines, pooling+linear for properties/classification)."}, "pretraining": {"method": "Supervised multi-task pretraining", "method_details": "Jointly trains a single Swin-Base backbone with seven task-specific heads on 137 categories across 7 label types (segmentation, regression, points, polygons, polylines, properties, classification). Initialized with ImageNet weights, Adam optimizer, lr 1e-4 decaying to 1e-6, batch size 32 for 100K batches.", "objective": "Multi-task supervised loss (cross-entropy for segmentation/classification, L1 for regression, Faster/Mask R-CNN losses for detection/instance segmentation)", "is_self_supervised": false, "is_vision_language": false}, "pretraining_data": {"datasets": ["SatlasPretrain (Sentinel-2 + NAIP imagery, labels from OpenStreetMap, NOAA lidar scans, WorldCover, Microsoft Buildings, C2S, plus expert and AMT annotations)"], "num_images": "856K Web-Mercator tiles (zoom 13); 828K train + 28K test low-res tiles with 8-12 Sentinel-2 images each; 46K high-res NAIP tiles (8192x8192) with 3-5 images; 17M image megapixels; 302M labels under 137 categories", "sensors": ["Sentinel-2", "NAIP (aerial)"], "modalities": ["multispectral optical (Sentinel-2, 9 bands)", "high-resolution RGB aerial (NAIP)"], "spatial_resolution": "10 m/pixel (Sentinel-2) and 1 m/pixel (NAIP)", "geographic_coverage": "Global for Sentinel-2 (all continents except Antarctica, 21,320,000 sq km covered); United States only for NAIP high-resolution"}, "downstream_tasks": ["scene classification", "semantic segmentation", "instance segmentation", "object detection"], "downstream_datasets": ["UCM", "RESISC45", "AID", "FMoW", "Massachusetts Roads", "Massachusetts Buildings", "Airbus Ship Detection"], "code_available": true, "code_url": "https://satlas-pretrain.allen.ai/", "weights_available": true, "key_contribution": "A very-large-scale multi-task remote sensing dataset (40x more pixels and 150x more labels than prior datasets) with 137 categories across 7 label types and spatio-temporal image series, plus SatlasNet, a unified multi-head model. Pretraining on SatlasPretrain improves downstream accuracy by 18% over ImageNet and 6% over the next best remote sensing baseline.", "_source_pdf": "SatlasPretrain.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation", "year": 2024, "model_name": "LHRS-Bot-Nova", "architecture": {"backbone": "SigLIP-L/14 vision encoder + LLaMA3-8B LLM with MoE-based vision perceiver bridge", "type": "Multimodal Large Language Model (MLLM)", "params_millions": 8000, "notes": "Vision encoder: SigLIP-L/14 at 336x336 resolution. Vision perceiver uses learnable queries with cross-attention and MoE-FFN (4 experts, top-2 routed) over 6 layers. LLM: LLaMA3-8B with LoRA (rank 128, alpha 256). Multi-level feature extraction at layers {N_L/3, 2N_L/3, N_L-1} with query allocation {112, 96, 64}."}, "pretraining": {"method": "Three-stage curriculum learning: vision-language pretraining, multi-task instruction fine-tuning, supervised fine-tuning", "method_details": "Stage 1 pretrains vision encoder and vision perceiver on LHRS-Align-Recap image-caption pairs. Stage 2 unfreezes LLM with LoRA and trains on multi-task instruction dataset. Stage 3 fine-tunes vision perceiver and LoRA on LHRS-Instruct, LHRS-Instruct-Plus, and LRV-Instruct.", "objective": "Vision-language alignment via image captioning and instruction-following (autoregressive language modeling)", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["LHRS-Align-Recap (feature-guided recaptioned via Share-Captioner using OSM features)", "LHRS-Instruct", "LHRS-Instruct-Plus (generated via GPT-4V from DOTAv2 and FAIR1M)", "LRV-Instruct", "Multi-task dataset combining RSVQA-HR/LR, UCM, fMoW, METER-ML, RSVG, DIOR-RSVG, RSICD", "NWPU, RSITMD"], "num_images": "~1.1M+ (LHRS-Align-Recap caption set; instruction-tuning includes 67K DOTAv2 + 253K FAIR1M conversations + 140K LRV-Instruct + others)", "sensors": ["RGB aerial/satellite imagery"], "modalities": ["RGB optical imagery", "text/language", "OpenStreetMap (OSM) features"], "spatial_resolution": "high-resolution aerial/satellite RGB (varies)", "geographic_coverage": "global (derived from public RS datasets paired with OSM)"}, "downstream_tasks": ["scene classification", "visual question answering", "visual grounding", "image captioning", "object detection (referring/grounding)", "multimodal dialogue/conversation", "multiple-choice question answering"], "downstream_datasets": ["AID", "WHU-RS19", "SIRI-WHU", "EuroSAT", "NWPU", "METER-ML", "fMoW", "RSVQA-HR", "RSVQA-LR", "RSVG", "DIOR-RSVG", "LHRS-Bench"], "code_available": true, "code_url": "https://github.com/NJU-LHRS/LHRS-Bot", "weights_available": true, "key_contribution": "An RS-specialized MLLM combining a scaled-up SigLIP vision encoder with a novel MoE-based vision perceiver bridge layer, trained on a new feature-guided recaptioned image-caption dataset (LHRS-Align-Recap) and an extended spatial-recognition instruction dataset (LHRS-Instruct-Plus), achieving state-of-the-art performance across RS scene classification, VQA, visual grounding, and the LHRS-Bench MCQ benchmark.", "_source_pdf": "LHRS-Bot-Nova.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "A Billion-scale Foundation Model for Remote Sensing Images", "year": 2024, "model_name": "ViT-G12x4", "architecture": {"backbone": "ViT-G12x4 (parallel layer Vision Transformer)", "type": "Vision Transformer", "params_millions": 2420, "notes": "Plain ViT scaled via parallel MHSA/FFN blocks; 12 layers, 4 parallelism, hidden size 2048, MLP size 8192, 32 heads. Also trained ViT-B12x1 (86M), ViT-L12x4 (605.26M), ViT-H12x4 (1.36B). Backbone converted to ViTDET with local/global attention for downstream object localization."}, "pretraining": {"method": "Masked Autoencoder (MAE)", "method_details": "MAE with 75% masking ratio, 224x224 input, 16x16 patches, 400 epochs (1600 for ViT-B), AdamW, batch size 2048, fp16 + activation checkpointing. Effective LR = batch_size * BaseLR / 256.", "objective": "Masked image reconstruction (MSE on masked patches)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MillionAID"], "num_images": "~1,000,848", "sensors": ["Optical (Google Earth)"], "modalities": ["RGB optical"], "spatial_resolution": "0.5m to 153m GSD", "geographic_coverage": "Global"}, "downstream_tasks": ["Rotated object detection", "Semantic segmentation"], "downstream_datasets": ["DOTA v2.0", "DIOR-R", "Potsdam", "LoveDA"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First billion-scale (2.4B parameter) foundation model for remote sensing, demonstrating that scaling vision transformers via parallel MHSA/FFN layers (rather than depth) improves performance and sample efficiency on rotated object detection and semantic segmentation downstream tasks.", "_source_pdf": "BillionPFM.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing", "year": 2024, "model_name": "CSMAE", "architecture": {"backbone": "ViT-B/12 (also evaluated ViT-Ti/12, ViT-S/12, ViT-L/24)", "type": "Vision Transformer (multi-sensor encoder + cross-sensor encoder + multi-sensor decoder)", "params_millions": 114.15, "notes": "Four CSMAE variants: CECD (sensor-common encoder & decoder), CESD, SECD, SESD; embedding dim 512; 8 transformer blocks, 16 heads in decoder; patch size 15x15; 50% masking ratio."}, "pretraining": {"method": "Cross-Sensor Masked Autoencoders (CSMAE)", "method_details": "Adaptation of MAE for multi-modal/multi-sensor RS image archives. Includes uni-modal reconstruction loss (L_UMR) and cross-modal reconstruction loss (L_CMR), optionally combined with inter-modal latent similarity preservation losses (L_MDE or L_MIM). Three multi-modal masking correspondences explored: identical, random, disjoint.", "objective": "Masked image modeling with uni-modal + cross-modal reconstruction (MSE) plus optional inter-modal discrepancy elimination (MDE) or mutual information maximization (MIM)", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["BigEarthNet (BEN-270K subset over 10 European countries; BEN-14K subset over Serbia)"], "num_images": "270,470 multi-modal pairs (BEN-270K) and 14,832 pairs (BEN-14K)", "sensors": ["Sentinel-1 SAR", "Sentinel-2 multispectral"], "modalities": ["SAR (VV, VH)", "multispectral optical"], "spatial_resolution": "10m and 20m (S2 20m bands bicubic-interpolated to 10m)", "geographic_coverage": "10 European countries (BEN-270K) and Serbia (BEN-14K)"}, "downstream_tasks": ["Content-based image retrieval (CBIR)", "Uni-modal CBIR (S1->S1, S2->S2)", "Cross-sensor / cross-modal CBIR (S1->S2, S2->S1)"], "downstream_datasets": ["BigEarthNet (BEN-14K)"], "code_available": true, "code_url": "https://github.com/jakhac/CSMAE", "weights_available": true, "key_contribution": "First systematic study of masked autoencoders for sensor-agnostic (cross-sensor) content-based image retrieval in remote sensing. Introduces four CSMAE variants exploring sensor-common vs sensor-specific encoders/decoders, three multi-modal masking correspondences, and combined uni-modal + cross-modal reconstruction objectives with optional inter-modal latent similarity preservation, enabling MIM-based representation learning across SAR and multispectral RS modalities.", "_source_pdf": "CSMAE.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing", "year": 2025, "model_name": "GeoPixel", "architecture": {"backbone": "InternLM-XComposer-2.5 (IXC-2.5) 7B LLM (InternLM2 7B) with CLIP ViT-L/14 vision encoder (560x560), SAM-2 (Hiera MAE) grounding vision encoder, and pixel decoder", "type": "Vision-Language Multimodal Model with pixel grounding (LMM)", "params_millions": 7000, "notes": "5 components: Adaptive Image Divider, Vision Encoder (CLIP ViT-L/14), LLM (InternLM2 7B), Grounding Vision Encoder (SAM-2 Hiera), and Pixel Decoder. Uses Partial LoRA (alpha=8), two-layer MLP vision projector, and text projection MLP. Supports dynamic resolution up to 4K HD via adaptive image partitioning into local patches plus global view."}, "pretraining": {"method": "Instruction tuning / fine-tuning of pretrained LMM with LoRA on grounded conversation generation data", "method_details": "Initialized from pretrained IXC-2.5 (7B) LLM, frozen CLIP ViT-L vision encoder, and SAM-2 pretrained grounding vision encoder (frozen). Trainable components: pixel decoder, LoRA params (alpha=8), vision projector, language projector. Trained with effective batch size 20 over 10 epochs, lr warmup to 3e-4 over 100 steps then cosine decay, on 2x NVIDIA A6000-48GB GPUs for ~3 days on the GeoPixelD dataset.", "objective": "Grounded conversation generation (GCG) with multi-target pixel-level segmentation mask prediction and language modeling; <SEG> token embeddings decoded into segmentation masks", "is_self_supervised": false, "is_vision_language": true}, "pretraining_data": {"datasets": ["GeoPixelD (curated from iSAID training subset)"], "num_images": "16,795 holistic image-level annotations; 36,793 instance-specific annotations; 17,023 group annotations; 600,817 objects; 53,816 grounded phrases", "sensors": ["aerial/satellite RGB imagery"], "modalities": ["RGB optical imagery", "text"], "spatial_resolution": "high-resolution aerial imagery cropped to 800x800 patches; supports dynamic input resolution up to 4K", "geographic_coverage": "derived from iSAID aerial imagery dataset (global aerial scenes)"}, "downstream_tasks": ["Remote Sensing Grounded Conversation Generation (RS-GCG)", "Referring Remote Sensing Image Segmentation (RRSIS)", "pixel-level grounding", "multi-target segmentation", "image captioning"], "downstream_datasets": ["GeoPixelD (RS-GCG benchmark, test set from iSAID validation subset, 5,427 referring expression-segmentation pairs, 61,384 annotated objects)", "RRSIS-D"], "code_available": true, "code_url": "https://github.com/mbzuai-oryx/GeoPixel", "weights_available": true, "key_contribution": "First end-to-end high-resolution RS large multimodal model (LMM) supporting pixel-level grounding with multi-target segmentation masks at up to 4K resolution via adaptive image partitioning; introduces GeoPixelD, a semi-automated visually grounded RS dataset for grounded conversation generation, and a benchmark for evaluating RS LMMs on fine-grained visual understanding.", "_source_pdf": "GeoPixel.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning", "year": 2024, "model_name": "MMEarth", "architecture": {"backbone": "ConvNeXt V2-Atto", "type": "CNN", "params_millions": 3.7, "notes": "Fully Convolutional Masked Autoencoder (FCMAE) with sparse convolutions; adapted patch size 16x16 and modified first layer (kernel 3, stride 1 + depth-wise conv) for 112x112 Sentinel-2 inputs; shallow per-modality decoders"}, "pretraining": {"method": "Multi-Pretext Masked Autoencoder (MP-MAE)", "method_details": "Extends ConvNeXt V2 FCMAE with multi-modal reconstruction across 12 aligned pixel- and image-level modalities; shared encoder with per-task decoders; multi-task loss with task-uncertainty weighting; trained 200 epochs", "objective": "Multi-task masked reconstruction (MSE for continuous, cross-entropy for categorical) over pixel-level and image-level pretext targets", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["MMEarth"], "num_images": "1.2M locations", "sensors": ["Sentinel-2", "Sentinel-1 SAR", "Aster GDEM", "ETH-GCHM", "Dynamic World", "ESA World Cover", "ERA5"], "modalities": ["optical multi-spectral", "SAR", "elevation/slope (DEM)", "canopy height", "landcover (Dynamic World)", "landcover (ESA World Cover)", "biome", "ecoregion", "ERA5 temperature", "ERA5 precipitation", "geolocation (lat/lon)", "date"], "spatial_resolution": "10 m (Sentinel-2 grid; tiles 128x128 = 1.28x1.28 km)", "geographic_coverage": "Global, 1.2M locations sampled across 14 biomes, years 2017-2020"}, "downstream_tasks": ["multi-label classification", "multi-class classification", "semantic segmentation"], "downstream_datasets": ["BigEarthNet (BigEarth20k)", "So2Sat (So2Sat20k)", "EuroSAT (EuroSat2k)", "Cashew Plantation (Cashew1k)", "SA-Crop-Type (SAcrop3k)"], "code_available": true, "code_url": "https://github.com/vishalned/MMEarth-train", "weights_available": true, "key_contribution": "Introduces MMEarth, a global 1.2M-location multi-modal Sentinel-2-aligned pretraining dataset with 12 modalities, and proposes MP-MAE, a ConvNeXt V2-based fully convolutional multi-pretext masked autoencoder that improves fine-tuning and especially linear-probing performance over ImageNet and Sentinel-2-only MAE baselines", "_source_pdf": "MMEarth.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "S2-CGAN: Self-Supervised Adversarial Representation Learning for Binary Change Detection in Multispectral Images", "year": 2020, "model_name": "S2-cGAN", "architecture": {"backbone": "U-Net", "type": "conditional GAN with U-Net generator and patch-based discriminator", "params_millions": null, "notes": "Generator G is a U-Net autoencoder with six convolutional layers; discriminator D is a pixel-based network with two convolutional layers. Operates on 128x128 patches."}, "pretraining": {"method": "Adversarial", "method_details": "A self-supervised conditional GAN trained only on pairs of unchanged co-registered patches. The generator learns to reconstruct the unchanged distribution while the discriminator learns to distinguish unchanged samples; the discriminator is then directly used as the change classifier (changes as outliers).", "objective": "adversarial training on unchanged sample reconstruction", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Worldview-2 Shenzhen bitemporal scene"], "num_images": "7646 patches of 128x128", "sensors": ["WorldView-2"], "modalities": ["multispectral"], "spatial_resolution": "2m", "geographic_coverage": "Shenzhen, China"}, "downstream_tasks": ["change detection"], "downstream_datasets": ["WorldView-2 Shenzhen"], "code_available": true, "code_url": "https://gitlab.tubit.tu-berlin.de/rsim/S2-cGAN", "weights_available": false, "key_contribution": "Proposes S2-cGAN, the first self-supervised conditional GAN that directly exploits the discriminator's likelihood to formulate binary change detection. Trained only on unchanged sample pairs with no labels for changed samples, the method achieves competitive performance against fully-supervised methods.", "_source_pdf": "S2-CGAN_SSL_Adversarial_Repr_for_Binary_CD.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Self-supervised pre-training enhances change detection in Sentinel-2 imagery", "year": 2021, "model_name": null, "architecture": {"backbone": "Siamese CNN", "type": "shallow Siamese CNN with three convolutional layers per branch and a fusion layer", "params_millions": null, "notes": "Pretext task 1 uses Siamese with fusion layer + classifier; pretext task 2 uses Siamese without fusion. 32-channel feature maps."}, "pretraining": {"method": "Contrastive", "method_details": "Two pretext tasks for change detection pretraining: (1) binary classification of whether two patches are spatially overlapping or not, and (2) triplet margin loss minimizing feature distance between overlapping patches and pushing apart non-overlapping ones.", "objective": "overlap discrimination and triplet contrastive learning between patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["S2MTCP"], "num_images": "1520 Sentinel-2 image pairs (~600x600 px)", "sensors": ["Sentinel-2"], "modalities": ["multispectral"], "spatial_resolution": "10m", "geographic_coverage": "global (1520 cities worldwide)"}, "downstream_tasks": ["change detection"], "downstream_datasets": ["OSCD"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Introduces the Sentinel-2 Multitemporal Cities Pairs (S2MTCP) dataset of 1520 global urban image pairs and shows that self-supervised pretraining on this dataset (using overlap discrimination or triplet contrastive pretext tasks) enables state-of-the-art change detection on OSCD with minimal labeled data.", "_source_pdf": "SSL_pretraining_enhances_CD_in_Sentinel2.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Self-Supervised Multisensor Change Detection", "year": 2022, "model_name": null, "architecture": {"backbone": "two-branch CNN", "type": "two-branch (optical/SAR) CNN with shared common layer; non-shared projection modules and shared prediction module", "params_millions": null, "notes": "L=5 conv layers (L1=4 separate per modality, L2=1 common). Kernel size 3x3, K clusters as final-layer output channels, K=4 used in experiments."}, "pretraining": {"method": "Contrastive", "method_details": "Self-supervised training combining deep clustering, temporal consistency loss, and contrastive loss on a single bi-temporal optical-SAR scene. Projection modules are modality-specific, predictors are shared. Trained on 64x64 patches from the target scene only.", "objective": "deep clustering + temporal consistency + contrastive loss", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["target bi-temporal scenes only (Las Vegas, Chongqing, Abu Dhabi, Montpellier)"], "num_images": "single bi-temporal scene per experiment (few hundred px)", "sensors": ["Sentinel-1", "Sentinel-2", "various optical-SAR pairs"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "10m", "geographic_coverage": "four scenes (Las Vegas USA, Chongqing China, Abu Dhabi UAE, Montpellier France)"}, "downstream_tasks": ["change detection"], "downstream_datasets": ["Las Vegas", "Chongqing", "Abu Dhabi", "Montpellier"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Proposes a self-supervised method for multi-sensor (optical-SAR) change detection that requires only the bi-temporal target scene with no labels or additional unlabeled images. Combines deep clustering, temporal consistency loss, and contrastive loss in a Siamese architecture, outperforming prior unsupervised multisensor CD methods.", "_source_pdf": "SSL_Multisensor_Change_Detection.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Self-supervised Change Detection in Multi-view Remote Sensing Images", "year": 2021, "model_name": null, "architecture": {"backbone": "ResNet-34", "type": "pseudo-Siamese network (two branches with same architecture but unshared weights)", "params_millions": null, "notes": "Strider parameters changed from 2 to 1 in third and fourth layers to adapt to small input patches. Patch sizes p=8,16,24 for homogeneous; p=8,16 for heterogeneous."}, "pretraining": {"method": "Contrastive", "method_details": "Combines a multi-view contrastive loss (heterogeneous images, with InfoNCE-style negatives from a memory bank) and BYOL-style implicit contrastive learning (homogeneous images via mean teacher with EMA). Trained on archived multi-temporal/multi-sensor image pairs.", "objective": "multi-view contrastive (InfoNCE) + BYOL regression", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["augmented OSCD subsets (S2S2, S1S1, L8S2, S1S2)", "California flood scene"], "num_images": null, "sensors": ["Sentinel-1", "Sentinel-2", "Landsat-8"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "10m-100m", "geographic_coverage": "global (OSCD cities)"}, "downstream_tasks": ["change detection"], "downstream_datasets": ["OSCD_S2S2", "OSCD_S1S1", "OSCD_L8S2", "OSCD_S1S2", "California flood"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Proposes a self-supervised change detection approach for multi-view remote sensing images using a pseudo-Siamese ResNet-34 trained with multi-view contrastive loss (heterogeneous case) and BYOL (homogeneous case). Introduces a multi-scale change scoring strategy and validates on five homogeneous and heterogeneous datasets.", "_source_pdf": "SSL_CD_in_Multiview_RS_Images.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Task-Related Self-Supervised Learning for Remote Sensing Image Change Detection", "year": 2021, "model_name": "TSLCD", "architecture": {"backbone": "shallow CNN (4 convolutional layers, 3x3 kernels)", "type": "Siamese-style backbone with two FC layers for change classification", "params_millions": null, "notes": "Backbone composed of four conv layers + ReLU; uses contrastive Euclidean loss with margin m for SSL."}, "pretraining": {"method": "Contrastive", "method_details": "Task-related self-supervised module generates pair-wised similar/dissimilar samples by unsupervised image segmentation (felzenszwalb-style) on a single bi-temporal scene, then trains the backbone with contrastive loss. Hard-sample-mining loss reweights hard samples; smooth mechanism removes pseudo-changes.", "objective": "pairwise contrastive learning on segmented patches", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["target bi-temporal scenes (WV3 Site 1, WV3 Site 2, ZY3, QB)"], "num_images": "1431x1431, 458x559, 1154x740 scenes", "sensors": ["WorldView-3", "ZY-3", "QuickBird"], "modalities": ["RGB"], "spatial_resolution": null, "geographic_coverage": null}, "downstream_tasks": ["change detection"], "downstream_datasets": ["WV3 Site 1", "WV3 Site 2", "ZY3", "QB"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Introduces TSLCD, an unsupervised change detection framework combining a task-related self-supervised module (contrastive pretraining on segmentation-derived pseudo-labels), a hard-sample-mining loss, and a smooth mechanism. Achieves state-of-the-art unsupervised CD across four datasets from different sensors.", "_source_pdf": "TaskRelated_SSL_for_RS_Image_CD.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Self-supervised Remote Sensing Images Change Detection at Pixel-level", "year": 2021, "model_name": "PixSSL", "architecture": {"backbone": "Siamese ResUnet", "type": "Siamese encoder-decoder with vector quantization module", "params_millions": 4.216, "notes": "ResUnet encoder has 3 residual blocks + Conv-BN-ReLU-Pool, decoder mirrors with skip connections. Vector quantization with codebook size 1024 using Gumbel-Softmax."}, "pretraining": {"method": "Contrastive", "method_details": "Pixel-wise contrastive learning via shift-equivariance pretext task: cropped image pair with relative shift fed through Siamese ResUnet, with reverse shift applied to outputs to align features. Vector quantization with codebook loss augments learned representations. Optional teacher-student uncertainty-aware feature learning step using KL loss.", "objective": "pixel-wise contrastive learning with shift equivariance + codebook loss + uncertainty estimation", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["OSCD_S2S2 (extended)", "MUDS_S2S2 (extended)", "California flood"], "num_images": null, "sensors": ["Sentinel-2", "Sentinel-1", "Landsat-8", "Planet"], "modalities": ["multispectral", "SAR"], "spatial_resolution": "4m-30m", "geographic_coverage": "global"}, "downstream_tasks": ["change detection"], "downstream_datasets": ["OSCD_S2S2", "MUDS_S2S2", "California flood"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First pixel-wise contrastive self-supervised approach for change detection in remote sensing. Combines a shift-equivariance contrastive pretext on a Siamese ResUnet with vector quantization and an uncertainty-aware teacher-student paradigm to suppress regular seasonal changes, improving over patch-based SSL CD.", "_source_pdf": "SSL_RS_Images_CD_at_Pixel_level.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation", "year": 2021, "model_name": null, "architecture": {"backbone": "ResNet-50", "type": "contrastive multiview coding encoder (also evaluated with ResNeXt-50 and ResNeSt-50)", "params_millions": null, "notes": "Base encoder with projection head (linear -> BN -> ReLU -> linear, output dim 2048). Tested with U-Net, BiSeNet, DeepLabV3 segmentation heads."}, "pretraining": {"method": "CMC", "method_details": "Contrastive Multiview Coding using SAR, EO, and segmentation map as multiple modalities of the same scene. Full-graph optimization considers all modality pairs; positive samples are images of the same location across modalities and augmentations.", "objective": "contrastive multiview coding across SAR, EO, and segmentation map", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["SpaceNet 6"], "num_images": "scenes patched into 300x300 with 150 strides, resized to 448x448", "sensors": ["Capella Space SAR (X-band quad-pol)", "WorldView-2"], "modalities": ["SAR", "RGB", "segmentation map"], "spatial_resolution": "0.5m", "geographic_coverage": "Rotterdam, Netherlands"}, "downstream_tasks": ["semantic segmentation"], "downstream_datasets": ["SpaceNet 6"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "First multi-modal contrastive multiview coding approach in remote sensing that uses an auxiliary EO sensor (and optionally segmentation maps) to pretrain SAR semantic segmentation models. Demonstrates that more modality views improve SAR building segmentation performance, sample efficiency, and convergence speed across multiple backbones and segmentation heads.", "_source_pdf": "CMC_with_EO_for_SAR_Semantic_Segmentation.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation", "year": 2022, "model_name": "CBT", "architecture": {"backbone": "ResNet-50", "type": "Barlow Twins encoder + projector with EWC regularization for continual learning; UNet++ for downstream segmentation", "params_millions": null, "notes": "ResNet-50 backbone trained with Barlow Twins loss; downstream UNet++ with Squeeze-and-Excitation."}, "pretraining": {"method": "Barlow Twins", "method_details": "Combines Barlow Twins (cross-correlation matrix close to identity) with Elastic Weight Consolidation (EWC) regularization to enable incremental self-supervised pretraining across heterogeneous EO datasets without catastrophic forgetting.", "objective": "Barlow Twins redundancy reduction + EWC regularization", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["US3D", "UAVid", "Potsdam"], "num_images": "~5000 (Potsdam) + ~7500 (UAVid) + ~11000 (US3D)", "sensors": ["WorldView-3", "UAV", "aerial"], "modalities": ["RGB"], "spatial_resolution": "0.05m-0.3m", "geographic_coverage": "US (US3D), various (UAVid), Potsdam Germany"}, "downstream_tasks": ["semantic segmentation"], "downstream_datasets": ["US3D", "UAVid", "Potsdam"], "code_available": true, "code_url": "https://github.com/VMarsocci/CBT", "weights_available": false, "key_contribution": "First combination of self-supervised learning and continual learning for remote sensing. Continual Barlow Twins integrates Barlow Twins with Elastic Weight Consolidation, enabling incremental pretraining on heterogeneous EO datasets (drone, aerial, satellite) without catastrophic forgetting, and proposes a new mixed-domain segmentation benchmark.", "_source_pdf": "Continual_Barlow_Twins_for_RS_Semantic_Segmentation.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Self Supervised Learning for Few Shot Hyperspectral Image Classification", "year": 2022, "model_name": null, "architecture": {"backbone": "custom 2D/1D CNN (2 residual blocks)", "type": "shared encoder + MLP projector for Barlow Twins", "params_millions": null, "notes": "2D CNN uses 9x9 patches; 1D CNN uses single-pixel inputs in 5x5 neighborhood for pair sampling."}, "pretraining": {"method": "Barlow Twins", "method_details": "Pretrain encoder on unlabeled HSI pixels using Barlow-Twins via redundancy reduction. Views generated by spatial transforms (flip, rotation, crop) and spectral transforms (scaling, Gaussian noise, band dropping, pixel removal, band swapping, translation), plus partially overlapping patch pair sampling.", "objective": "redundancy reduction (Barlow Twins) on HSI patches/pixels", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Pavia University", "Houston University"], "num_images": "single HSI scenes (610x340 with 9 classes, 349x1905 with 144 bands and 15 classes)", "sensors": ["airborne hyperspectral"], "modalities": ["hyperspectral"], "spatial_resolution": null, "geographic_coverage": "Pavia Italy, Houston USA"}, "downstream_tasks": ["scene classification", "pixel classification"], "downstream_datasets": ["Pavia University", "Houston University"], "code_available": false, "code_url": null, "weights_available": false, "key_contribution": "Demonstrates that Barlow-Twins-based self-supervised pretraining on unlabeled hyperspectral pixels significantly outperforms supervised baselines for few-shot HSI classification. Introduces patch/pixel pair sampling strategies that exploit spatial regularity and analyzes the impact of spatial vs. spectral data augmentations.", "_source_pdf": "SSL_for_Few_Shot_Hyperspectral_Classification.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
{"title": "Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery", "year": 2024, "model_name": "CRISP", "architecture": {"backbone": "ResNet-50 (ground) + ResNet-50 (aerial)", "type": "dual-encoder CLIP-style image-image contrastive model", "params_millions": null, "notes": "Replaces CLIP text encoder with another image encoder. Four variants: standard CRISP, CRISP-Aug (with RS data augmentation), CRISP-M2o (many-to-one positive pooling), CRISP-Par (parameterized loss weighting)."}, "pretraining": {"method": "Contrastive", "method_details": "ContRastive Image-remote Sensing Pre-training (CRISP) replaces CLIP text encoder with an aerial image encoder and learns joint representation between ground-level and aerial image pairs via symmetric InfoNCE loss. Variants include data augmentation, many-to-one positives, and parameterized loss weighting.", "objective": "image-image contrastive (InfoNCE) between ground-level and aerial views", "is_self_supervised": true, "is_vision_language": false}, "pretraining_data": {"datasets": ["Nature Multi-View (NMV)"], "num_images": ">3 million ground-aerial image pairs from 1.75M observations", "sensors": ["NAIP aerial", "ground-level (iNaturalist)"], "modalities": ["RGB", "near-infrared"], "spatial_resolution": "0.6m (NAIP)", "geographic_coverage": "California, USA"}, "downstream_tasks": ["fine-grained species recognition", "species distribution mapping", "cropland mapping", "tree genus identification"], "downstream_datasets": ["NMV", "iNat", "cropland mapping benchmarks"], "code_available": true, "code_url": "https://hf.co/datasets/andyvhuynh/NatureMultiView", "weights_available": true, "key_contribution": "Introduces CRISP, an image-image contrastive pre-training approach pairing ground-level and aerial imagery for natural world representation learning, and the Nature Multi-View dataset (>3M pairs across 6,000+ California plant taxa). CRISP outperforms ImageNet, MAE, and MoCo baselines on fine-grained species recognition, especially in low-data regimes.", "_source_pdf": "ContrastCLIP_RS_groundlevel_satellite_pretraining.pdf", "_extraction_timestamp": "2026-04-18T00:00:00"}
