Correo Electrónico
DNINFOA - SIA
Bibliotecas
Convocatorias
Identidad U.N.
Published
Comparison of statistical indices for the evaluation of crop models performance
Comparación de índices estadísticos para la evaluación de modelos de cultivos
DOI:
https://doi.org/10.15446/rfnam.v74n3.93562Keywords:
Deviation statistics; crop simulation model; model evaluation; RMSE; index of agreement; efficiency coefficient. (en)Estadísticas de desviación; modelo de simulación de cultivos; evaluación del modelo; RMSE; índice de concordancia; coeficiente de eficiencia. (es)
This study presents a comparison of the usual statistical methods used for crop model assessment. A case study was conducted using a data set from observations of the total dry weight in diploid potato crop, and six simulated data sets derived from the observationsaimed to predict the measured data. Statistical indices such as the coefficient of determination, the root mean squared error, the relative root mean squared error, mean error, index of agreement, modified index of agreement, revised index of agreement, modeling efficiency, and revised modeling efficiency were compared. The results showed that the coefficient of determination is not a useful statistical index for model evaluation. The root mean squared error together with the relative root mean squared error offer an excellent notion of how deviated the simulations are in the same unit of the variable and percentage terms, and they leave no doubt when evaluating the quality of the simulations of a model.
Este artículo presenta una comparación de los métodos estadísticos habituales que se utilizan para la evaluación de modelos de cultivos. Se realizó un estudio de caso utilizando un conjunto de datos observados del peso seco total en un cultivo de papa diploide y seis conjuntos de datos simulados destinados a predecir las observaciones. Los parámetros estadísticos evaluados fueron el coeficiente de determinación, la raíz cuadrada del cuadrado medio del error, la raíz cuadrada del cuadrado medio del error relativo, el error medio, el índice de concordancia, el índice de concordancia modificado, el índice de concordancia revisado, el índice de eficiencia y el índice de eficiencia revisado. Los resultados mostraron que el coeficiente de determinación no es un índice estadístico útil para la evaluación de modelos de cultivo. La raíz cuadrada del cuadrado medio del error junto a la raíz cuadrada del cuadrado medio del error relativo, ofrecen una excelente idea de cuánto están desviadas las simulaciones en la misma unidad de medida de la variable y en términos porcentuales. La raíz cuadrada del cuadrado medio del error y la raíz cuadrada del cuadrado medio del error relativo no dejan dudas al evaluar la calidad de las simulaciones de un modelo respecto a las observaciones.
References
Abedinpour M, Sarangi A, Rajput T, Singh M, Pathak H and Ahmad T. 2012. Performance evaluation of AquaCrop model for maize crop in a semi-arid environment. Agricultural Water Management 110: 55–66. https://doi.org/10.1016/j.agwat.2012.04.001
Addiscott T and Whitmore A. 1987. Computer simulation of changes in soil mineral nitrogen and crop nitrogen during autumn, winter and spring. The Journal of Agricultural Science 109(1): 141–157. https://doi.org/10.1017/S0021859600081089
Ali M and Abustan I. 2014. A new novel index for evaluating model performance. Journal of Natural Resources and Development 2002: 1–9. https://doi.org/10.5027/jnrd.v4i0.01
Asseng S. 2013. Uncertainty in simulating wheat yields. Nature Climate Change 3(9): 627–632. https://doi.org/10.1038/nclimate1916
Bassu S, Brisson N, Durand J, Boote K, Lizaso J, Jones J, Rosenzweig C, Ruane A, Adam M, Baron C, Basso B, Biernath C, Boogaard H, Conijn S, Corbeels M, Deryng D, De Sanctis G, Gayler S, Grassini P, … Waha K, 2014. How do various maize crop models vary in their responses to climate change factors? Global Change Biology 20(7): 2301–2320. https://doi.org/10.1111/gcb.12520
Cao HX, Hanan JS, Liu Y, Liu YX, Yue YB, Zhu DW, Lu JF, Sun JY, Shi CL, Ge DK, Wei XF, Yao AQ, Tian PP, and Bao TL. 2012. Comparison of crop model validation methods. Journal of Integrative Agriculture, 11(8): 1274–1285. https://doi.org/10.1016/S2095-3119(12)60124-5
Ewert F. 2008. The (increasing) complexity of plant systems research and the use of models. pp. 21-24. In: Van Keulen H, Van Laar H and Rabbinge R (eds.). 40 Years Theory and Model at Wageningen. 66 p.
Fleisher D, Condori B, Quiroz R, Alva A, Asseng S, Barreda C, Bindi M, Boote K, Ferrise R, Franke A, Govindakrishnan P, Harahagazwe D, Hoogenboom G, Naresh Kumar S, Merante P, Nendel C, Olesen J, Parker P, Raes D, … Woli P. 2017. A potato model intercomparisonacross varying climates and productivity levels. Global Change Biology 23(3): 1258–1281. https://doi.org/10.1111/gcb.13411
Hoogenboom G, Porter C, Boote K, Shelia V, Wilkens P, Singh U, White J, Asseng S, Lizaso J, Moreno L, Pavan W, Ogoshi R, Hunt L, Tsuji G and Jones J. 2019. The DSSAT crop modeling ecosystem.pp. 173-216. In: Boote K. (ed.) Advances in Crop Modeling for a Sustainable Agriculture. Burleigh Dodds Science Publishing, Cambridge. 543 p.
Hunt R and Parsons I. 2011. A computer program for deriving growth-functions in plant growth analysis. Journal of Applied Ecology 11(1): 297–307. https://doi.org/10.2307/2402022
Iqbal M, Shen Y, Stricevic R, Pei H, Sun H, Amiri E, Penas A and del Rio S. 2014. Evaluation of the FAO AquaCrop model for winter wheat on the North China Plain under deficit irrigation from field experiment to regional yield simulation. Agricultural Water Management 135: 61–72. https://doi.org/10.1016/j.agwat.2013.12.012
Kobayashi K and Salam M. 2000. Comparing simulated and measured values using mean squared deviation and its components. Agronomy Journal 92: 345–352. https://doi.org/10.2134/agronj2000.922345x
Krause P, Boyle D, P and Bäse F. 2005. Comparison of different efficiency criteria for hydrological model assessment. Advances in Geosciences 5: 89–97. https://doi.org/10.5194/adgeo-5-89-2005
Kumudini S, Andrade F, Boote K, Brown G, Dzotsi K, Edmeades G, Gocken T, Goodwin M, Halter A, Hammer G, Hatfield J, Jones J, Kemanian A, Kim S, Kiniry J, Lizaso J, Nendel C, Nielsen R, Parent B, … Tollenaar M. 2014. Predicting maize phenology: Intercomparison of functions for developmental response to temperature. Agronomy Journal 106(6): 2087–2097. https://doi.org/10.2134/agronj14.0200
Legates D and McCabe Jr G. 1999. Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resources Research 35(1): 233–241. https://doi.org/10.1029/1998WR900018
Li T, Hasegawa T, Yin X, Zhu Y, Boote K, Adam M, Bregaglio S, Buis S, Confalonieri R, Fumoto T, Gaydon D, Marcaida M, Nakagawa H, Oriol P, Ruane A, Ruget F, Singh B, Singh U, Tang L, … Bouman B. 2015. Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Global Change Biology 21(3): 1328–1341. https://doi.org/10.1111/gcb.12758
Lin L, Hedayat A, Sinha B, Yang M, Sinha B, Yang M, Lin L and Hedayat A. 2014. Statistical methods in assessing agreement: Models issues and tools. Journal of the American Statistical Association 97(457): 257–270. https://doi.org/10.1198/016214502753479392
McCuen R and Snyder W. 1975. A proposed index for comparing hydrographs. Water Resources Research 11(6): 1021–1024. https://doi.org/10.1029/WR011i006p01021
Montoya F, Camargo D, Ortega J, Córcoles J and Domínguez A. 2016. Evaluation of Aquacrop model for a potato crop under different irrigation conditions. Agricultural Water Management 164: 267–280. https://doi.org/10.1016/j.agwat.2015.10.019
Moreno-Cadena L, Hoogenboom G, Fisher M, Ramirez-Villegas J, Prager S, Becerra Lopez-Lavalle L, Pypers P, Mejia de Tafur M, Wallach D, Muñoz-Carpena R and Asseng S. 2020. Importance of genetic parameters and uncertainty of MANIHOT a new mechanistic cassava simulation model. European Journal of Agronomy 115(2020): 126031. https://doi.org/10.1016/j.eja.2020.126031
Moriasi DN, Arnold JG, Liew MW, Van Bingner RL, Harmel RD and Veith TL. 2007. Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations. Transactions of ASABE 50(3): 885-900. https://doi.org/10.13031/2013.23153
Nash J and Sutcliffe I. 1970. River flow forecasting through conceptual models part I. A discussion of principles. Journal of Hydrology 10: 282–290. https://doi.org/10.1016/0022-1694(70)90255-6
Priesack E, Gayler S and Hartmann H. 2006. The impact of crop growth sub-model choice on simulated water and nitrogen balances. Nutrient Cycling in Agroecosystems 75: 1–13. https://doi.org/10.1007/s10705-006-9006-1
R Core Team. 2020. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.r-project.org/
Ramos H, Meschiatti M, de Matos R and Blain G. 2018. On the performance of three indices of agreement: An easy-to-use R-code for calculating the Willmott indices. Bragantia 77(2): 394–403. https://doi.org/10.1590/1678-4499.2017054
Reckhow K, Clements J and Dodd R. 1990. Statistical evaluation of mechanistic water-quality models. Journal of Environmental Engineering 116(2): 250–268. https://doi.org/10.1061/(ASCE)0733-9372(1990)116:2(250)
Saldaña-Villota T and Cotes-Torres J. 2020. Functional growth analysis of diploid potato cultivars (Solanum phureja Juz. et Buk.). Revista Colombiana de Ciencias Hortícolas 14(3): 402–415. https://doi.org/10.17584/rcch.2020v14i3.10870
Saqib M and Anjum M. 2021. Applications of decision support system: A case study of solanaceous vegetables. Phyton - International Journal of Experimental Botany 90(2): 331–352. https://doi.org/10.32604/phyton.2021.011685
Thornley J. 2011. Plant growth and respiration re-visited: Maintenance respiration defined – it is an emergent property of not a separate process within the system – and why the respiration: Photosynthesis ratio is conservative. Annals of Botany 108(7): 1365–1380. https://doi.org/10.1093/aob/mcr238
Willmott C. 1981. On the validation of models. Physical Geography 2(2): 184–194. https://doi.org/10.1080/02723646.1981.10642213
Willmott C. 1982. Some comments on the evaluation of model performance. Bulletin American Meteorological Society 63(11): 1309–1313. https://doi.org/10.1175/1520-0477(1982)063%3C1309:SCOTEO%3E2.0.CO;2
Willmott C, Ackleson S, Davis R, Feddema J, Klink K, Legates D, O’donnell J and Rowe C. 1985. Statistics for the evaluation and comparison of models. Journal of Geophysical Research 90(5): 8995–9005. https://doi.org/10.1029/JC090iC05p08995
Willmott C, Robeson S and Matsuura K. 2012. A refined index of model performance. International Journal of Climatology 32(13): 2088–2094. https://doi.org/10.1002/joc.2419
Yang J, Greenwood D, Rowell D, Wadsworth G and Burns I. 2000. Statistical methods for evaluating a crop nitrogen simulation model N_ABLE. Agricultural Systems 64(1): 37–53. https://doi.org/10.1016/S0308-521X(00)00010-X
Yang J, Yang J, Dou S, Yang X and Hoogenboom G. 2013. An overview of the crop model STICS. Nutrient Cycling in Agroecosystems 95(3): 309–332. https://doi.org/10.1016/S1161-0301(02)00110-7
Yang J, Yang J, Liu S and Hoogenboom G. 2014. An evaluation of the statistical methods for testing the performance of crop models with observed data. Agricultural Systems 127: 81–89. https://doi.org/10.1016/j.agsy.2014.01.008
How to Cite
APA
ACM
ACS
ABNT
Chicago
Harvard
IEEE
MLA
Turabian
Vancouver
Download Citation
CrossRef Cited-by
1. Amir Rastgoo, Hamed Khajavi. (2023). A novel study on forecasting the airfoil self-noise, using a hybrid model based on the combination of CatBoost and Arithmetic Optimization Algorithm. Expert Systems with Applications, 229, p.120576. https://doi.org/10.1016/j.eswa.2023.120576.
2. J. Vásquez-Jiménez, D.P. Bartholomew, C.J. Wilkerson, B. Vargas-Leitón, G. Hoogenboom. (2024). Predicting growth and development of the pineapple cultivar ‘MD-2’ with the DSSAT Aloha Pineapple Model. Fruits, 79(1), p.1. https://doi.org/10.17660/th2024/002.
3. Angela Patricia Romero-Vergel. (2023). TURION: A physiological crop model for yield prediction of asparagus using sentinel-1 data. European Journal of Agronomy, 143, p.126690. https://doi.org/10.1016/j.eja.2022.126690.
4. Francesco Saretto, Bishwajit Roy, Ricardo Encarnação Coelho, Alfredo Reder, Giusy Fedele, Robert Oakes, Luigia Brandimarte, Tiago Capela Lourenço. (2024). Impacts of Climate Change and Adaptation Strategies for Rainfed Barley Production in the Almería Province, Spain. Atmosphere, 15(5), p.606. https://doi.org/10.3390/atmos15050606.
5. Narubodin Yindi, Nakorn Jongrungklang, Patcharin Songsri, Phanupong Phoncharoen, Peeraya Klomsa-ard, Poramate Banterng. (2025). Growth Analysis and Simulation Approach for Sugarcane Genotypes Grown Under Different Growing Seasons. Sugar Tech, 27(4), p.1335. https://doi.org/10.1007/s12355-025-01585-3.
6. Dinghao Li, Zhi Wang, Yulong Liu, Haobo Shi, Yanbo Wan, Qingdong Shi. (2025). Impact of surface water and groundwater changes on the growth condition of desert riparian forests. Agricultural Water Management, 319, p.109783. https://doi.org/10.1016/j.agwat.2025.109783.
7. Qiang Wu, Xiaoyu Song, Jie Zhang, Yuanyuan Ma, Chunkai Zheng, Tuo Wang, Guijun Yang. (2025). Winter Wheat-Yield Estimation in the Huang-Huai-Hai Region Based on KNN-Ward Phenological Zoning and Multi-Source Data. Remote Sensing, 17(22), p.3686. https://doi.org/10.3390/rs17223686.
Dimensions
PlumX
Article abstract page views
Downloads
License
Copyright (c) 2021 Tatiana María Saldaña Villota
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The journal allows the author(s) to maintain the exploitation rights (copyright) of their articles without restrictions. The author(s) accept the distribution of their articles on the web and in paper support (25 copies per issue) under open access at local, regional, and international levels. The full paper will be included and disseminated through the Portal of Journals and Institutional Repository of the Universidad Nacional de Colombia, and in all the specialized databases that the journal considers pertinent for its indexation, to provide visibility and positioning to the article. All articles must comply with Colombian and international legislation, related to copyright.
Author Commitments
The author(s) undertake to assign the rights of printing and reprinting of the material published to the journal Revista Facultad Nacional de Agronomía Medellín. Any quotation of the articles published in the journal should be made given the respective credits to the journal and its content. In case content duplication of the journal or its partial or total publication in another language, there must be written permission of the Director.
Content Responsibility
The Faculty of Agricultural Sciences and the journal are not necessarily responsible or in solidarity with the concepts issued in the published articles, whose responsibility will be entirely the author or the authors.
