TY - JOUR

T1 - Making the most of imperfect data: A critical evaluation of standard information collected in farm household surveys

AU - Pagella, Tim

AU - Hammond, James

AU - Fraval, Simon

AU - Wichern, Jannike

AU - Oosting, Simon J

AU - de Boer, Imke J

AU - Teufal, Nils

AU - Lannerstad, Mats

AU - Waha, Katharina

AU - Rosenstock, Todd S

AU - Giller, Ken E

AU - Herrero, Mario

AU - Harris, David

AU - Van Wijk, Mark T

PY - 2019/4

Y1 - 2019/4

N2 - Household surveys are one of the most commonly used tools for generating insight into rural communities. Despite their prevalence, few studies comprehensively evaluate the quality of data derived from farm household surveys. We critically evaluated a series of standard reported values and indicators that are captured in multiple farm household surveys, and then quantified their credibility, consistency and, thus, their reliability. Surprisingly, even variables which might be considered ‘easy to estimate’ had instances of non-credible observations. In addition, measurements of maize yields and land owned were found to be less reliable than other stationary variables. This lack of reliability has implications for monitoring food security status, poverty status and the land productivity of households. Despite this rather bleak picture, our analysis also shows that if the same farm households are followed over time, the sample sizes needed to detect substantial changes are in the order of hundreds of surveys, and not in the thousands. Our research highlights the value of targeted and systematised household surveys and the importance of ongoing efforts to improve data quality. Improvements must be based on the foundations of robust survey design, transparency of experimental design and effective training. The quality and usability of such data can be further enhanced by improving coordination between agencies, incorporating mixed modes of data collection and continuing systematic validation programmes.

AB - Household surveys are one of the most commonly used tools for generating insight into rural communities. Despite their prevalence, few studies comprehensively evaluate the quality of data derived from farm household surveys. We critically evaluated a series of standard reported values and indicators that are captured in multiple farm household surveys, and then quantified their credibility, consistency and, thus, their reliability. Surprisingly, even variables which might be considered ‘easy to estimate’ had instances of non-credible observations. In addition, measurements of maize yields and land owned were found to be less reliable than other stationary variables. This lack of reliability has implications for monitoring food security status, poverty status and the land productivity of households. Despite this rather bleak picture, our analysis also shows that if the same farm households are followed over time, the sample sizes needed to detect substantial changes are in the order of hundreds of surveys, and not in the thousands. Our research highlights the value of targeted and systematised household surveys and the importance of ongoing efforts to improve data quality. Improvements must be based on the foundations of robust survey design, transparency of experimental design and effective training. The quality and usability of such data can be further enhanced by improving coordination between agencies, incorporating mixed modes of data collection and continuing systematic validation programmes.

U2 - 10.1017/S0014479718000388

DO - 10.1017/S0014479718000388

M3 - Article

VL - 55

SP - 230

EP - 250

JO - Experimental Agriculture

JF - Experimental Agriculture

SN - 0014-4797

IS - Special Issue 2

ER -