Climate change threatens global food systems, but the extent to which adaptation will reduce losses remains unknown and controversial. Even within the well-studied context of US agriculture, some analyses argue that adaptation will be widespread and climate damages small, whereas others conclude that adaptation will be limited and losses severe. Scenario-based analyses indicate that adaptation should have notable consequences on global agricultural productivity but there has been no systematic study of how extensively real-world producers actually adapt at the global scale. Here we empirically estimate the impact of global producer adaptations using longitudinal data on six staple crops spanning 12,658 regions, capturing two-thirds of global crop calories. We estimate that global production declines 5.5 × 1014 kcal annually per 1 °C global mean surface temperature (GMST) rise (120 kcal per person per day or 4.4% of recommended consumption per 1 °C; P < 0.001). We project that adaptation and income growth alleviate 23% of global losses in 2050 and 34% at the end of the century (6% and 12%, respectively; moderate-emissions scenario), but substantial residual losses remain for all staples except rice. In contrast to analyses of other outcomes that project the greatest damages to the global poor, we find that global impacts are dominated by losses to modern-day breadbaskets with favourable climates and limited present adaptation, although losses in low-income regions losses are also substantial. These results indicate a scale of innovation, cropland expansion or further adaptation that might be necessary to ensure food security in a changing climate.