The climate analog approach is often considered a valuable tool for climate change impact projection and adaptation planning, especially for complex systems that cannot be modeled reliably. Important examples are smallholder farming systems using agroforestry or other mixed-cropping approaches. For the projected climate at a particular site of interest, the analog approach identifies locations where the current climate is similar to these projected conditions. By comparing baseline-analog site pairs, information on climate impacts and opportunities for adaptation can be obtained. However, the climate analog approach is only meaningful, if climate is a dominant driver of differences between baseline and analog site pairs. For a smallholder farming setting on Mt. Elgon in Kenya, we tested this requirement by comparing yield potentials of maize and coffee (obtained from the IIASA Global Agro-ecological Zones dataset) among 50 close analog sites for different future climate scenarios and models, and by comparing local ecological knowledge and farm characteristics for one baseline-analog pair. Yield potentials among the 50 closest analog locations varied strongly within all climate scenarios, hinting at factors other than climate as major drivers of what the analog approach might interpret as climate effects. However, on average future climatic conditions seemed more favorable to maize and coffee cultivation than current conditions. The detailed site comparison revealed substantial differences between farms in important characteristics, such as farm size and presence of cash crops, casting doubt on the usefulness of the comparison for climate change analysis. Climatic constraints were similar between sites, so that no apparent lessons for adaptation could be derived. Pests and diseases were also similar, indicating that climate change may not lead to strong changes in biotic constraints at the baseline site in the near future. From both analyses, it appeared that differences between baseline and analog sites were mostly explained by non-climatic factors. This does not bode well for using the analog approach for impact projection and adaptation planning for future climatic conditions in agricultural contexts.