
Published
A Semi-Supervised Deep Learning Model for Defective lime Classification
Modelo semisupervisado de aprendizaje profundo para la clasificación de limones
DOI:
https://doi.org/10.15446/ing.investig.112835Keywords:
semi-supervised learning, citrus fruit classification, anomaly detection, precision agriculture applications (en)aprendizaje semisupervisado, clasificación de frutos cítricos, detección de anomalías, aplicaciones para agricultura (es)
Downloads
For consumers, the predominant fruit selection criterion is visual quality, a factor that classification models emulate when employing images as input data. Most classification paradigms presuppose a balance across classes. In the field of defective fruit detection, databases commonly exhibit a pronounced imbalance between healthy and defective fruit counts. Such disparity can compromise the robustness of classification models or introduce biases stemming from insufficient data. This study introduces a semi-supervised classification framework based on anomaly detection to identify defective lime fruits (Citrus aurantifolia). The framework employs the reconstruction error obtained from an autoencoder neural network and a calculated anomaly probability to locate samples within a two-dimensional space designed for such purpose. Based on the defined parameter ranges, the limes are categorized as either healthy or defective. The proposed classification model underwent training utilizing the publicly accessible Fruits360 database and was tested with a set of 118 new and unlabeled lime images. The classification model attained a precision of 94%, a recall of 0.88, and an F1-score of 0.91 across the test set. These results corroborate that models based on anomaly detection constitute a promising solution to the inherent challenges of unbalanced classification tasks. They offer the advantage of requiring minimal training data and reduced training times while maintaining efficacy, even when the evaluation dataset diverges substantially from the training set. Thus, the proposed model can serve as a decision support tool for farmers, producers, and consumers.
Para los consumidores, el criterio predominante en la selección de frutas es la calidad visual, un factor que los modelos de clasificación emulan cuando emplean imágenes como datos de entrada. La mayoría de los paradigmas de clasificación presuponen un equilibrio entre las clases. En el ámbito de la detección de frutas defectuosas, las bases de datos suelen presentar un desequilibrio pronunciado entre el recuento de frutas sanas y defectuosas. Esta disparidad puede comprometer la solidez de los modelos de clasificación o introducir sesgos derivados de la insuficiencia de datos. En este estudio se introduce un marco de clasificación semisupervisada basado en la detección de anomalías para identificar frutos defectuosos de limón (Citrus aurantifolia). El modelo emplea el error de reconstrucción de una red neuronal autoencoder y una probabilidad de anomalía calculada para localizar muestras dentro de un espacio bidimensional diseñado para tal propósito. A partir de los rangos de parámetros definidos, los limones se clasifican como sanos o defectuosos. El modelo de clasificación propuesto fue entrenado mediante la base de datos de acceso público Fruits360 y evaluado con un conjunto de 118 imágenes de limones nuevas y sin etiquetar. El modelo de clasificación obtuvo una precisión del 94 %, una recuperación del 0,88 y un valor F1 0,91 en el conjunto de pruebas. Estos resultados corroboran que los modelos basados en la detección de anomalías constituyen una solución prometedora a los retos inherentes de las tareas de clasificación no equilibradas; ofrecen la ventaja de requerir datos de entrenamiento mínimos y tiempos de entrenamiento reducido, manteniendo la eficacia incluso cuando el conjunto de datos de evaluación diverge sustancialmente del conjunto de entrenamiento. Así, el modelo propuesto puede servir como herramienta de apoyo en las decisiones de agricultores, productores y consumidores.
References
Aparicio Pico, L. E., Devia Lozano, P., and Amaya Marroquin, O. J. (2022). Aplicación de deep learning para la identificación de defectos superficiales utilizados en control de calidad de manufactura y producción industrial: Una revisión de la literatura. Ingeniería, 28(1), e18934. https://dx.doi.org/10.14483/23448393.18934.
Arnal Barbedo, J. G. (2019). Plant disease identification from individual lesions and spots using deep learning. Biosystems Engineering, 180, 96-107. https://dx.doi.org/10.1016/j.biosystemseng.2019.02.002.
Ayoub Shaikh, T., Rasool, T., and Rasheed Lone, F. (2022). Towards leveraging the role of machine learning and artificial intelligence in precision agriculture and smart farming. Computers and Electronics in Agriculture, 198, 107119. https://dx.doi.org/10.1016/j.compag.2022.107119.
Bank, D., Koenigstein, N., and Giryes, R. (2021). Autoencoders. arXiv preprint. https://doi.org/10.48550/arXiv.2003.05991.
Bao, M., Zheng, Z., Chen, J., and Deng, X. (2021). Investigation of citrus hlb symptom variations associated with candidatus liberibacter asiaticus strains harboring different phages in southern china. Agronomy, 11(11), 2262. https://dx.doi.org/10.3390/agronomy11112262.
Beggel, L., Pfeiffer, M., and Bischl, B. (2020). Robust anomaly detection in images using adversarial autoencoders. In U. Brefeld, E. Fromont, A. Hotho, A. Knobbe, M. Maathuis, and C. Robardet (Eds.), Machine Learning and Knowledge Discovery in Databases. Springer International Publishing. https://doi.org/10.1007/978-3-030-46150-8_13",pages="206--222.
Bhardwaj, A., Hasteer, N., Kumar, Y., and Yogesh. (2022). Deep learning based fruit defect detection system. In S. Mekhilef, R. N. Shaw, and P. Siano (Eds.), Innovations in Electrical and Electronic Engineering (pp. 306–313. https://doi.org/10.1007/978-981-19-1677-9_28). Springer Singapore.
Bhole, A., Varsha Kumar. (2021). A transfer learning-based approach to predict the shelf life of fruit. Inteligencia Artificial, 24(67), 102120. https://doi.org/10.4114/intartif.vol24iss67pp102-120.
Bird, J. J., Barnes, C. M., Manso, L. J., Ekárt, A., and Faria, D. R. (2022). Fruit quality and defect image classification with conditional gan data augmentation. Scientia Horticulturae, 293, 110684. https://doi.org/10.1016/j.scienta.2021.110684.
Blasco, J., Cubero, S., and Moltó, E. (2016, 4). Quality evaluation of citrus fruits. In Computer Vision Technology for Food Quality Evaluation: Second Edition (pp. 305–325). Elsevier. https://dx.doi.org/10.1016/B978-0-12-802232-0.00012-8.
Cabrera Ardila, C. E., Alberto Ramirez, L., and Prieto Ortiz, F. A. (2020a). Spectral analysis for the early detection of anthracnose in fruits of sugar mango (mangifera indica). Computers and Electronics in Agriculture, 173, 105357. https://doi.org/10.1016/j.compag.2020.105357.
Cabrera Ardila, C. E., Alberto Ramirez, L., and Prieto Ortiz, F. A. (2020b). Spectral analysis for the early detection of anthracnose in fruits of sugar mango (Mangifera indica). Computers and Electronics in Agriculture, 173, 105357. https://doi.org/10.1016/j.compag.2020.105357.
Cao, V. L., Nicolau, M., and McDermott, J. (2016). A hybrid autoencoder and density estimation model for anomaly detection. In J. Handl, E. Hart, P. R. Lewis, M. López-Ibáñez, G. Ochoa, and B. Paechter (Eds.), Parallel problem solving from nature – ppsn xiv (pp. 717–726). Springer International Publishing. https://doi.org/10.1007/978-3-319-45823-6_67.
Cazzonelli, L., and Kulbach, C. (2023). Detecting anomalies with autoencoders on data streams. In M.-R. Amini, S. Canu, A. Fischer, T. Guns, P. Kralj Novak, and G. Tsoumakas (Eds.), Machine learning and knowledge discovery in databases (pp. 258–274). Springer International Publishing https://doi.org/10.1007/978-3-031-26387-3_16.
Chaturvedi, A., Sharma, S., and Janghel, R. R. (2023). Detection of external defects in tomatoes using deep learning. Journal of Ambient Intelligence and Humanized Computing, 14(3), 2709–2721. https://doi.org/10.1007/s12652-023-04514-y.
Chen, Y., Wu, J., and Cui, M. (2018). Automatic classification and detection of oranges based on computer vision. In 2018 ieee 4th international conference on computer and communications (iccc) (p. 1551-1556. https://doi.org/10.1109/CompComm.2018.8780680).
Cubero, S., Aleixos, N., Albert, F., Torregrosa, A., Ortiz, C., García-Navarrete, O., and Blasco, J. (2014). Optimised computer vision system for automatic pre-grading of citrus fruit in the field using a mobile platform. Precision Agriculture, 15(1), 80–94. https://doi.org/10.1007/s11119-013-9324-7.
Cándido-Mireles, M., Hernández-Gama, R., and Salas, J. (2023). Detecting vineyard plants stress in situ using deep learning. Computers and Electronics in Agriculture, 210, 107837. https://doi.org/10.1016/j.compag.2023.107837.
Dandekar, M., Punn, N. S., Sonbhadra, S. K., and Agarwal, S. (2020). Fruit classification using deep feature maps in the presence of deceptive similar classes. CoRR. https://arxiv.org/abs/2007.05942.
Dhiman, P., Kukreja, V., Manoharan, P., Kaur, A., Kamruzzaman, M. M., Dhaou, I. B., and Iwendi, C. (2022). A novel deep learning model for detection of severity level of the disease in citrus fruits. Electronics, 11(3), 495. https://doi.org/10.3390/electronics11030495.
dos Santos Tanaka, F. H. K., and Aranha, C. (2019). Data augmentation using gans. arXiv preprint. https://doi.org/10.48550/ARXIV.1904.09135.
Fan, S., Li, J., Zhang, Y., Tian, X., Wang, Q., He, X., Huang, W. (2020). On line detection of defective apples using computer vision system combined with deep learning methods. Journal of Food Engineering, 286, 110102. https://doi.org/10.1016/j.jfoodeng.2020.110102.
Food and Agriculture Organization. (2022). World food and agriculture statistical yearbook 2022 (Tech.Rep.).
Food and Agriculture Organization of the United Nations. https://openknowledge.fao.org/handle/20.500.14283/cc2211en.
Futerman, S. I., Laor, Y., Eshel, G., and Cohen, Y. (2023). The potential of remote sensing of cover crops to benefit sustainable and precision fertilization. Science of The Total Environment, 891, 164630. https://dx.doi.org/10.1016/j.scitotenv.2023.164630.
Gao, X., Huang, C., Teng, S., and Chen, G. (2022). A deep-convolutional-neural-network-based semi-supervised learning method for anomaly crack detection. Applied Sciences, 12(18), 9244. https://dx.doi.org/10.3390/app12189244.
George, A., Rao, C. N., and Mani, M. (2022). Pests of citrus and their management. In M. Mani (Ed.), Trends in Horticultural Entomology (pp. 551–575). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-0343-4_17.
Ghazal, S., Qureshi, W. S., Khan, U. S., Iqbal, J., Rashid, N., and Tiwana, M. I. (2021). Analysis of visual features and classifiers for fruit classification problem. Computers and Electronics in Agriculture, 187, 106267. https://doi.org/10.1016/j.compag.2021.106267.
Gokulnath, B. V., and Usha Devi, G. (2021). Identifying and classifying plant disease using resilient LF-CNN. Ecological Informatics, 63, 101283. https://doi.org/10.1016/j.ecoinf.2021.101283.
González-Barbosa, J.-J., Ramírez-Pedraza, A., Ornelas-Rodríguez, F.-J., Cordova-Esparza, D.-M., and González-Barbosa, E.-A. (2022). Dynamic measurement of portos tomato seedling growth using the kinect 2.0 sensor. Agriculture, 12(4), 449. https://doi.org/10.3390/agriculture12040449.
Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep learning. MIT Press. http://www.deeplearningbook.org.
Gron, A. (2017). Hands-on machine learning with scikit-learn and tensorflow: Concepts, tools, and techniques to build intelligent systems (1st ed.). O’Reilly Media.
Hanh, L. D., and Bao, D. N. T. (2022, May 29). Autonomous lemon grading system by using machine learning and traditional image processing. International Journal on Interactive Design and Manufacturing (IJIDeM), 17, 445-452. https://doi.org/10.1007/s12008-022-00926-w.
Hasan, M. M., Salehin, I., Moon, N. N., Kamruzzaman, T. M., Baki-Ul-Islam, and Hasan, M. (2021). A computer vision system for the categorization of citrus fruits using convolutional neural network. In IEEE (Ed.), 2021 International Symposium on Electronics and Smart Devices (ISESD) (p. 1-6. https://doi.org/10.1109/ISESD53023.2021.9501790). IEEE.
Hernández, A., Ornelas-Rodrı́guez, F. J., Hurtado-Ramos, J. B., and González-Barbosa, J. J. (2021). Accuracy comparison between deep learning models for mexican lemon classification. In M. F. Mata-Rivera and R. Zagal-Flores (Eds.), Telematics and Computing (pp. 62–73). Springer International Publishing.
Hu, W., Gao, J., Li, B., Wu, O., Du, J., and Maybank, S. (2020). Anomaly detection using local kernel density estimation and context-based regression. IEEE Transactions on Knowledge and Data Engineering, 32(2), 218-233. https://doi.org/10.1109/TKDE.2018.2882404.
Ibrahim, M., and Kuban, K. (2023). Key generation method from fingerprint image based on deep convolutional neural network model. Nexo Revista Científica, 36, 906-925. https://dx.doi.org/10.5377/nexo.v36i06.17447.
Istiak, M. A., Syeed, M. M., Hossain, M. S., Uddin, M. F., Hasan, M., Khan, R. H., and Azad, N. S. (2023). Adoption of Unmanned Aerial Vehicle (UAV) imagery in agricultural management: A systematic literature review. Ecological Informatics, 78, 102305. https://doi.org/10.1016/j.ecoinf.2023.102305.
Khanramaki, M., Askari Asli-Ardeh, E., and Kozegar, E. (2021). Citrus pests classification using an ensemble of deep learning models. Computers and Electronics in Agriculture, 186, 106192. https://doi.org/10.1016/j.compag.2021.106192.
Lang, C. I., Sun, F.-K., Lawler, B., Dillon, J., Dujaili, A. A., Ruth, J., . . . Boning, D. S. (2022). One class process anomaly detection using kernel density estimation methods. IEEE Transactions on Semiconductor Manufacturing, 35(3), 457-469. https://doi.org/10.1109/TSM.2022.3181468.
Latif, G., Mohammad, N., and Alghazo, J. (2023). Deepfruit: A dataset of fruit images for fruit classification and calories calculation. Data in Brief, 50, 109524. https://doi.org/10.1016/j.dib.2023.109524.
Li, D., Song, Z., Quan, C., Xu, X., and Liu, C. (2021). Recent advances in image fusion technology in agriculture. Computers and Electronics in Agriculture, 191, 106491. https://doi.org/10.1016/j.compag.2021.106491.
Li, E., Wang, L., Xie, Q., Gao, R., Su, Z., and Li, Y. (2023). A novel deep learning method for maize disease identification based on small sample-size and complex background datasets. Ecological Informatics, 75, 102011. https://doi.org/10.1016/j.ecoinf.2023.102011.
Lozano, E., and Archibald, D. (2022). Citrus annual. United States Department of Agriculture. https://www.fas.usda.gov/data/mexico-citrus-annual-7.
Lu, Y., and Young, S. (2020). A survey of public datasets for computer vision tasks in precision agriculture. Computers and Electronics in Agriculture, 178, 105760. https://doi.org/10.1016/j.compag.2020.105760.
Maheshwari, A., Mitra, P., and Sharma, B. (2022). Autoencoder: Issues, challenges and future prospect. In M. Vashista, G. Manik, O. P. Verma, and B. Bhardwaj (Eds.), Recent Innovations in Mechanical Engineering (pp. 257–266. https://doi.org/10.1007/978-981-16-9236-9_24). Springer Singapore.
Memarzadeh, M., Akbari Asanjan, A., and Matthews, B. (2022). Robust and explainable semi-supervised deep learning model for anomaly detection in aviation. Aerospace, 9(8), 437. https://doi.org/10.3390/aerospace9080437.
Michelucci, U. (2022). An introduction to autoencoders. CoRR. https://arxiv.org/abs/2201.03898 .
Momeny, M., Jahanbakhshi, A., Neshat, A. A., Hadipour-Rokni, R., Zhang, Y.-D., and Ampatzidis, Y. (2022). Detection of citrus black spot disease and ripeness level in orange fruit using learning-to-augment incorporated deep networks. Ecological Informatics, 71, 101829 https://doi.org/10.1016/j.ecoinf.2022.101829.
Murean, H., and Oltean, M. (2018, 06). Fruit recognition from images using deep learning. Acta Universitatis Sapientiae, Informatica, 10, 26-42. https://doi.org/10.2478/ausi-2018-0002.
Mzoughi, O., and Yahiaoui, I. (2023). Deep learning-based segmentation for disease identification. Ecological Informatics, 75, 102000. https://doi.org/10.1016/j.ecoinf.2023.102000.
Naranjo-Torres, J., Mora, M., Fredes, C., and Valenzuela, A. (2021). Disease and defect detection system for raspberries based on convolutional neural networks. Applied Sciences, 11(24), 11868. https://doi.org/10.3390/app112411868.
Nowak, B. (2021). Precision agriculture: Where do we stand? a review of the adoption of precision agriculture technologies on field crops farms in developed countries. Agricultural Research, 10(4), 515–522. https://doi.org/10.1007/s40003-021-00539-x.
Palei, S., Behera, S. K., and Sethy, P. K. (2023). A systematic review of citrus disease perceptions and fruit grading using machine vision. Procedia Computer Science, 218, 2504-2519. https://doi.org/10.1016/j.procs.2023.01.225.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://dl.acm.org/doi/10.5555/1953048.2078195.
Peng, K., Ma, W., Lu, J., Tian, Z., and Yang, Z. (2023). Application of machine vision technology in citrus production. Applied Sciences, 13(16), 9334. https://doi.org/10.3390/app13169334.
Puerto Cuadros, E. G. (2024, Nov.). Advanced neural model for spanish spell-checking. Ingeniería, 29(3), e21135. https://dx.doi.org/10.14483/23448393.21135.
Rathnayake, N., Rathnayake, U., Dang, T. L., and Hoshino, Y. (2022). An efficient automatic fruit-360 image identification and recognition using a novel modified cascaded-anfis algorithm. Sensors, 22(12), 4401. https://doi.org/10.3390/s22124401.
Rong, D., Rao, X., and Ying, Y. (2017). Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm. Computers and Electronics in Agriculture, 137, 59-68. https://doi.org/10.1016/j.compag.2017.02.027.
Roy, K., Chaudhuri, S. S., and Pramanik, S. (2021, September). Deep learning based real-time industrial framework for rotten and fresh fruit detection using semantic segmentation. Microsystem Technologies, 27(9), 3365–3375. https://doi.org/10.1007/s00542-020-05123-x.
Schor, N., Bechar, A., Ignat, T., Dombrovsky, A., Elad, Y., and Berman, S. (2016). Robotic disease detection in greenhouses: Combined detection of powdery mildew and tomato spotted wilt virus. IEEE Robotics and Automation Letters, 1(1), 354-360. https://doi.org/10.1109/LRA.2016.2518214.
Shahi, T. B., Sitaula, C., Neupane, A., and Guo, W. (2022). Fruit classification using attention-based mobilenetv2 for industrial applications. PLOS ONE, 17(2), 1-21. https://doi.org/10.1371/journal.pone.0264586.
Shijie, J., Ping, W., Peiyi, J., and Siping, H. (2017). Research on data augmentation for image classification based on convolution neural networks. 2017 Chinese Automation Congress (CAC) [Conference paper]. https://doi.org/10.1109/CAC.2017.8243510.
Siddiqi, R. (2020). Comparative performance of various deep learning based models in fruit image classification. In ACM (Ed.), 11th International Conference on Advances in Information Technology (p. 14). Association for Computing Machinery. https://doi.org/10.1145/3406601.3406619.
Soltani Firouz, H., Mahmoud Sardari. (2022). Defect detection in fruit and vegetables by using machine vision systems and image processing. Food Engineering Reviews, 14(3), 353–379. https://doi.org/10.1007/s12393-022-09307-1.
Tan, A., Zhou, G., and He, M. (2021). Surface defect identification of citrus based on kf-2d-renyi and abc-svm. Multimedia Tools and Applications, 80(6), 9109–9136. https://doi.org/10.1007/s11042-020-10036-y.
Thoidis, I., Giouvanakis, M., and Papanikolaou, G. (2021). Semi-supervised machine condition monitoring by learning deep discriminative audio features. Electronics, 10(20), 2471. https://doi.org/10.3390/electronics10202471.
Toosi, A., Javan, F. D., Samadzadegan, F., Mehravar, S., Kurban, A., and Azadi, H. (2022). Citrus orchard mapping in juybar, iran: Analysis of ndvi time series and feature fusion of multi-source satellite imageries. Ecological Informatics, 70, 101733. https://doi.org/10.1016/j.ecoinf.2022.101733.
Verma, R., and Verma, A. K. (2022). Fruit classification using deep convolutional neural network and transfer learning. In V. E. Balas, G. R. Sinha, B. Agarwal, T. K. Sharma, P. Dadheech, and M. Mahrishi (Eds.), Emerging Technologies in Computer Engineering: Cognitive Computing and Intelligent IoT (pp. 290–301). Springer International Publishing.
Wang, T., Zhao, L., Li, B., Liu, X., Xu, W., and Li, J. (2022). Recognition and counting of typical apple pests based on deep learning. Ecological Informatics, 68, 101556. https://doi.org/10.1016/j.ecoinf.2022.101556.
Yılmaz, E. K., Adem, K., Kılıçarslan, S., and Aydın, H. A. (2023, June). Classification of lemon quality using hybrid model based on stacked AutoEncoder and convolutional neural network. European Food Research and Technology, 249(6), 1655–1667. https://doi.org/10.1007/s00217-023-04245-6.
Zhang, H., Chen, Y., Liu, X., Huang, Y., Zhan, B., and Luo, W. (2021). Identification of common skin defects and classification of early decayed citrus using hyperspectral imaging technique. Food Analytical Methods, 14(6), 1176–1193. https://doi.org/10.1007/s12161-020-01960-8.
Ümit Atila, Uçar, M., Akyol, K., and Uçar, E. (2021). Plant leaf disease classification using efficientnet deep learning model. Ecological Informatics, 61, 101182. https://doi.org/10.1016/j.ecoinf.2020.101182.
How to Cite
APA
ACM
ACS
ABNT
Chicago
Harvard
IEEE
MLA
Turabian
Vancouver
Download Citation
License
Copyright (c) 2024 Angel-Moisés Hernández-Ponce, Francisco-Javier Ornelas-Rodríguez, Juan-Bautista Hurtado-Ramos, Pedro-Alfonso Ramírez-Pedraza, José-Joel González-Barbosa

This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors or holders of the copyright for each article hereby confer exclusive, limited and free authorization on the Universidad Nacional de Colombia's journal Ingeniería e Investigación concerning the aforementioned article which, once it has been evaluated and approved, will be submitted for publication, in line with the following items:
1. The version which has been corrected according to the evaluators' suggestions will be remitted and it will be made clear whether the aforementioned article is an unedited document regarding which the rights to be authorized are held and total responsibility will be assumed by the authors for the content of the work being submitted to Ingeniería e Investigación, the Universidad Nacional de Colombia and third-parties;
2. The authorization conferred on the journal will come into force from the date on which it is included in the respective volume and issue of Ingeniería e Investigación in the Open Journal Systems and on the journal's main page (https://revistas.unal.edu.co/index.php/ingeinv), as well as in different databases and indices in which the publication is indexed;
3. The authors authorize the Universidad Nacional de Colombia's journal Ingeniería e Investigación to publish the document in whatever required format (printed, digital, electronic or whatsoever known or yet to be discovered form) and authorize Ingeniería e Investigación to include the work in any indices and/or search engines deemed necessary for promoting its diffusion;
4. The authors accept that such authorization is given free of charge and they, therefore, waive any right to receive remuneration from the publication, distribution, public communication and any use whatsoever referred to in the terms of this authorization.