Publicado

2015-01-01

FPGA-based translation system from colombian sign language to text

Sistema traductor de la lengua de señas colombiana a texto basado en FPGA

DOI:

https://doi.org/10.15446/dyna.v82n189.43075

Palabras clave:

FPGA, Colombian Sign Language (LSC), Image Processing, Sign Language Recognition, Artificial Neural Networks (en)
FPGA, Lengua de Señas Colombiana (LSC), Procesamiento de Imágenes, Reconocimiento de Lengua de Señas, Redes Neuronales Artificiales (es)

Descargas

Autores/as

  • Juan David Guerrero Balaguera Universidad Pedagógica y Tecnológica de Colombia
  • Wilson Javier Pérez Holguín Universidad Pedagógica y Tecnológica de Colombia
This paper presents the development of a system aimed to facilitate the communication and interaction of people with severe hearing impairment with other people. The system employs artificial vision techniques to the recognition of static signs of Colombian Sign Language (LSC). The system has four stages: Image capture, preprocessing, feature extraction and recognition. The image is captured by a digital camera TRDB-D5M for Altera's DE1 and DE2 development boards. In the preprocessing stage, the sign is extracted from the background of the image using the thresholding segmentation method; then, the segmented image is filtered using a morphological operation to remove the noise. The feature extraction stage is based on the creation of two vectors to characterize the shape of the hand used to make the sign. The recognition stage is made up a multilayer perceptron neural network (MLP), which functions as a classifier. The system was implemented in the Altera's Cyclone II FPGA EP2C70F896C6 device and does not require the use of gloves or visual markers for its proper operation. The results show that the system is able to recognize all the 23 signs of the LSC with a recognition rate of 98.15 %.
Este trabajo presenta el desarrollo de un sistema diseñado para facilitar la comunicación e interacción de personas con discapacidad auditiva severa con las demás personas. El sistema emplea técnicas de visión artificial para el reconocimiento de las señas estáticas de la Lengua de Señas Colombiana (LSC). El sistema tiene cuatro etapas: Captura de la imagen, preprocesamiento, extracción de características y reconocimiento. La imagen es capturada mediante una cámara digital TRDB-D5M diseñada para tarjetas de desarrollo de DE1 y DE2 Altera. En la etapa de preprocesamiento, la seña es extraída del fondo de la imagen mediante el método de segmentación por umbral; posteriormente, la imagen segmentada es filtrada usando una operación morfológica para eliminar el ruido. La etapa de extracción de características está basada en la creación de dos vectores que caracterizan la forma de la mano mediante la que se realiza la seña. La etapa de reconocimiento está constituida por una red neuronal artificial perceptrón multicapa (MLP), la cual actúa como clasificador. El sistema fue implementado en el dispositivo FPGA Cyclone II EP2C70F896C6 y no requiere el uso de guantes o marcadores visuales para su correcto funcionamiento. Los resultados muestran que el sistema tiene la capacidad para reconocer todas las 23 señas estáticas de la LSC con una taza de reconocimiento del 98.15 %.

Referencias

World Health Organization, Deafness and hearing loss [Online], 2014 [date of reference February 10th of 2014]. Available at: http://www.who.int/mediacentre/factsheets/fs300/en/

Instituto Nacional Para Sordos, Estadísticas e información para contribuir en el mejoramiento de la calidad de vida de la Población Sorda Colombiana [Online], 2009 [date of reference March 10th of 2012 ] Available at: http://www.insor.gov.co/historico/images/bolet%C3%ADn%20observatorio.pdf

Ebrahim-Al-Ahdal, M. and Nooritawati, M.T., Review in sign language recognition systems, in 2012 IEEE Symposium on Computers and Informatics, pp. 52-57, 2012. https://doi.org/10.1109/ISCI.2012.6222666

Kausar, S. and Javed, M.Y., A survey on Sign language recognition, in 9th International Conference on Frontiers of Information Technology FIT 2011, pp. 95-98, 2011. https://doi.org/10.1109/FIT.2011.25

Instituto Nacional Para Sordos, Diccionario básico de la lengua de señas colombiana. [Online] 2006 [date of reference March 1st of 2012] Available at: http://www.ucn.edu.co/e-discapacidad/Documents/36317784-Diccionario-lengua-de-senas.pdf

Vogler, C. and Metaxas, D., A framework for recognizing the simultaneous aspects of American sign language, Computer Vision and Image Understanding, 81 (3), pp. 358-384, 2001. https://doi.org/10.1006/cviu.2000.0895

Al-Jarrah, O. and Halawani, A., Recognition of gestures in Arabic sign language using neuro-fuzzy systems, Artificial Intelligence, 133 (1-2), pp. 117-138, 2001. https://doi.org/10.1016/S0004-3702(01)00141-2

Starner, T., Weaver, J. and Pentland, A., Real-time american sign language recognition using desk and wearable computer based video, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (12), pp. 1371-1375, 1998. https://doi.org/10.1109/34.735811

Waldron, M.B. and Kim, S., Isolated ASL sign recognition system for deaf persons, IEEE Transactions on Rehabilitation Engineering, 3 (3), pp. 261-271, 1995. https://doi.org/10.1109/86.413199

Davis, J. and Shah, M., Visual gesture recognition, IEEE Proceedings: Vision, Image and Signal Processing, 141 (2), pp. 101-106, 1994. https://doi.org/10.1049/ip-vis:19941058

Ye, J., Yao, H. and Jiang, F., Based on HMM and SVM multilayer architecture classifier for Chinese sign language recognition with large vocabulary, in IEEE First Symposium on Multi-Agent Security and Survivability, pp. 377-380, 2004. https://doi.org/10.1109/ICIG.2004.44

Hernandez-Rebollar, J.L., Kyriakopoulos, N. and Lindeman, R.W., A new instrumented approach for translating American Sign Language into sound and text, in Sixth IEEE International Conference on Automatic Face and Gesture Recognition FGR 2004, pp. 547-552, 2004. https://doi.org/10.1109/AFGR.2004.1301590

Kim, J.S., Jang, W. and Bien, Z., A dynamic gesture recognition system for the Korean Sign Language (KSL), IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 26 (2), pp. 354-359, 1996. https://doi.org/10.1109/3477.485888

Tolba, M.F. and Elons, A.S., Recent developments in sign language recognition systems, in 8th International Conference on Computer Engineering and Systems ICCES 2013, pp. xxxvi-xlii, 2013. https://doi.org/10.1109/ICCES.2013.6707157

Alex Raj, S.M., Sreelatha, G. and Supriya, M.H., Gesture recognition using field programmable gate arrays, in 2012 International Conference on Devices, Circuits and Systems, pp. 72-75, 2012. https://doi.org/10.1109/ICDCSyst.2012.6188677

Oniga, S., Tisan, A., Mic, D., Buchman, A. and Vida-Ratiu, A., Hand postures recognition system using artificial neural networks implemented in FPGA, in 30th International Spring Seminar on Electronics Technology, pp. 507-512, 2007. https://doi.org/10.1109/ISSE.2007.4432909

Adithya, V., Vinod, P.R. and Gopalakrishnan, U., Artificial neural network based method for Indian sign language recognition, in IEEE Conference on Information and Communication Technologies, pp. 1080-1085, 2013. https://doi.org/10.1109/CICT.2013.6558259

Vargas, L.P., Barba, L., Torres, C.O. and Mattos, L., Sign language recognition system using neural network for digital hardware implementation, Journal of Physics: Conference Series. 274 (1), 2011. https://doi.org/10.1088/1742-6596/274/1/012051

Karami, A., Zanj, B. and Sarkaleh, A.K., Persian sign language (PSL) recognition using wavelet transform and neural networks, Expert Systems with Applications, 38 (3), pp. 2661-2667, 2011. https://doi.org/10.1016/j.eswa.2010.08.056

Admasu, Y.F. and Raimond, K., Ethiopian sign language recognition using Artificial Neural Network, in 10th International Conference on Intelligent Systems Design and Applications, ISDA'10, pp. 995-1000, 2010. https://doi.org/10.1109/ISDA.2010.5687057

Maraqa, M. and Abu-Zaiter, R., Recognition of Arabic Sign Language (ArSL) using recurrent neural networks, in International Conference on the Applications of Digital Information and Web Technologies, ICADIWT 2008, pp. 478-481, 2008. https://doi.org/10.1109/ICADIWT.2008.4664396

Munib, Q., Habeeb, M., Takruri, B. and Al-Malik, H.A., American sign language (ASL) recognition based on Hough transform and neural networks, Expert Systems with Applications, 32 (1), pp. 24-37, 2007. https://doi.org/10.1016/j.eswa.2005.11.018

Yu, S. H., Huang, C. L., Hsu, S. C., Lin, H. W. and Wang, H. W., Vision-based continuous sign language recognition using product HMM, in Asian Conference on Pattern Recognition, ACPR 2011, pp. 510-514 , 2011. https://doi.org/10.1109/ACPR.2011.6166631

Kawahigashi, K., Shirai, Y., Miura, J. and Shimada, N., Automatic synthesis of training data for sign language recognition using HMM, in 10th International Conference On Computers Helping People With Special Needs And Pre-Conference, Vol. 4061, pp. 623-626, 2006. https://doi.org/10.1007/11788713_92

Rekha, J., Bhattacharya, J. and Majumder, S., Shape, texture and local movement hand gesture features for indian sign language recognition, in International Conference on Trendz in Information Sciences and Computing, TISC 2011, pp. 30-35, 2011. https://doi.org/10.1109/TISC.2011.6169079

Premaratne, P., Human computer interaction using hand gestures. Adelaide, South Australia, Australia: Springer, 2014. https://doi.org/10.1007/978-981-4585-69-9

Terasic-Technologies. TRDB-D5M User Guide. [Online] 2010 [date of reference June 1st of 2013] Available at: http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGuide.pdf

Terasic-Technologies. TRDB-LTM User Manual. [Online] 2009 [date of reference June 1st of 2013] Available at: http://www.terasic.com.tw/cgi-bin/page/archive_download.pl?Language=English&No=213&FID=b226168825c32dd5d7064e9a57f42b0b

Gonzalez, R.C. and Woods, R.E., Digital image processing, 2d ed.: Prentice Hall, 2002.

MathWorks®, Image Processing Toolbox. [Online] 2013 [date of reference June 1st of 2013] Available at http://www.mathworks.com/products/image/

Mehrotra, K., Mohan, C.K. and Ranka, S., Elements of artificial neural networks: MIT press, 1997.

Beale, M., Hagan, M.T. and Demuth, H.B., Neural network toolbox, Neural Network Toolbox, The Math Works, pp. 5-25, 1992.

Quiroga, J., Cartes, D., and Edrington, C., Modelamiento basado en redes neuronales de un PMSM bajo fluctuaciones de carga, DYNA, 76 (160), pp. 273-282, 2009.

Abdul-Kadir, N.A., Sudirman, R. and Mahmood, N.H., Recognition system for nasal, lateral and trill arabic phonemes using neural networks, in 2012 IEEE Student Conference on Research and Development SCOReD, pp. 229-234, 2012. https://doi.org/10.1109/SCOReD.2012.6518644

Andersen, T. and Martinez, T., Cross validation and MLP architecture selection, in International Joint Conference on Neural Networks IJCNN '99, Vol. 3, pp. 1614-1619, 1999. https://doi.org/10.1109/IJCNN.1999.832613

Wright, J.L. and Manic, M., Neural network architecture selection analysis with application to cryptography location, in The 2010 International Joint Conference on Neural Networks IJCNN, pp. 1-6, 2010. https://doi.org/10.1109/IJCNN.2010.5596315

Simpson, P., FPGA design: Best practices for team-based design: Springer New York, 2010.

Dimensions

PlumX

Visitas a la página del resumen del artículo

1247

Descargas

Los datos de descarga aún no están disponibles.

Cómo citar

[1]
“Sistema traductor de la lengua de señas colombiana a texto basado en FPGA”, DYNA, vol. 82, no. 189, pp. 172–181, Jan. 2015, doi: 10.15446/dyna.v82n189.43075.

Licencia

Derechos de autor 2015 DYNA

Creative Commons License

Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.

El autor o autores de un artículo aceptado para publicación en cualquiera de las revistas editadas por la facultad de Minas cederán la totalidad de los derechos patrimoniales a la Universidad Nacional de Colombia de manera gratuita, dentro de los cuáles se incluyen: el derecho a editar, publicar, reproducir y distribuir tanto en medios impresos como digitales, además de incluir en artículo en índices internacionales y/o bases de datos, de igual manera, se faculta a la editorial para utilizar las imágenes, tablas y/o cualquier material gráfico presentado en el artículo para el diseño de carátulas o posters de la misma revista.