T3 Reported numbers disagree across papers
We grouped the 11,488 reported results by model, benchmark, metric, evaluation protocol, and training regime. After excluding generic baselines and detection benchmarks, 252 groups are reported by at least two papers. The reported values differ by at least 5 points in 47 of them (18.7%), by at least 10 in 18, and by at least 20 in 6. For example, CROMA's mIoU on m-Cashew-Plantation is 30.1 in DOFA and 62.2 in SatDiFuser, and the papers do not give enough detail to tell whether the decoders and evaluation settings were the same. The median spread is 0.1 points and the 90th percentile is 8.3.