Innovation Portfolio
A shared, community-governed system for evaluating geospatial AI models — measuring real-world performance, failure modes, compute costs, and generalization across geographies and data types.
A rigorous, open benchmarking system gives the geospatial AI community — researchers, governments, and industry alike — a trusted, shared basis for comparing models and making informed decisions about which tools to use for which problems. It raises the bar for transparency and reproducibility, reduces duplicated effort across organizations, and helps ensure that advances in the field translate into reliable, real-world impact.
BotW creates simple accuracy scores to assess real-world performance, failure modes, compute costs, and generalization across geographies and data types.
Evaluation beyond standard metrics — showing how models behave at deployment scale, where they fail, and how performance varies across geographic contexts.
New tools allow organizations to assess model strengths and blind spots without running full evaluations on every task — lowering the barrier to rigorous testing.
Methods for predicting model performance on a given task before running it — providing an early, low-cost signal for model selection.
Cost and compute requirements are tracked and reported as standard outputs alongside accuracy, making efficiency a first-class evaluation criterion.