Many experiments have shown that biodiversity enhances ecosystem functioning. However, we have little understanding of how environmental heterogeneity shapes the effect of diversity on ecosystem functioning and to what extent this diversity effect is mediated by variation in species richness or species turnover. This knowledge is crucial to scaling up the results of experiments from local to regional scales. Here we quantify the diversity effect and its components—that is, the contributions of variation in species richness and species turnover—for 22 ecosystem functions of microorganisms, plants and animals across 13 major ecosystem types on Mt Kilimanjaro, Tanzania. Environmental heterogeneity across ecosystem types on average increased the diversity effect from explaining 49% to 72% of the variation in ecosystem functions. In contrast to our expectation, the diversity effect was more strongly mediated by variation in species richness than by species turnover. Our findings reveal that environmental heterogeneity strengthens the relationship between biodiversity and ecosystem functioning and that species richness is a stronger driver of ecosystem functioning than species turnover. Based on a broad range of taxa and ecosystem functions in a non-experimental system, these results are in line with predictions from biodiversity experiments and emphasize that conserving biodiversity is essential for maintaining ecosystem functioning.