Real-Time Recognition of Peruvian Sign Language Using Convolutional Neural Networks (CNNs)

Autores/as

DOI:

https://doi.org/10.26439/ciii2025.8660

Palabras clave:

Assistive technology, convolutional neural networks (CNNs), inclusive education, Peruvian Sign Language (PSL), sign language recognition

Resumen

Inclusive education for people with hearing impairments in many countries still lacks accessible technological tools. This work introduces a prototype for automatic translation of the Peruvian Sign Language (PSL) finger alphabet based on convolutional neural networks (CNNs) combined with support vector machines (SVMs). The system recognizes letters in real time without requiring additional sensors or wearable devices. A proprietary dataset containing up to 50 images per class was used for training under controlled conditions. The prototype achieved an average accuracy of 97%, a word error rate (WER) of 15%, and a response time of 1.8–2.0 s and a processing speed of up to 125 frames per second (fps). These results demonstrate the viability of the system as an inclusive educational tool in both controlled environments and real-life school settings.

Descargas

Los datos de descarga aún no están disponibles.

Biografía del autor/a

  • Sonia J. León-Jimenez, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Sonia Joanna León Jiménez holds a Bachelor’s degree in Industrial Engineering from Universidad de Lima, Peru. She is currently an intern in Project Management at Siemens Energy, where she contributes to the management and coordination of projects in the energy sector, with a focus on process planning and optimization. She won first place in the 8th edition of the Regional Energy Efficiency Program organized by Siemens for a proposal recognized for its potential to advance energy efficiency and sustainability, representing Universidad de Lima in an international competition. Her academic projects include “Optimization of Resources and Parameters in Mine Wastewater Treatment through Reverse Osmosis and pH Control,” which focused on improving industrial processes with an emphasis on sustainability and efficiency. Her research interests include automation, project management, and industrial design, with a particular focus on resource optimization, technological innovation, and digital transformation in industrial applications.

  • Claudia León-Chavarri, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Claudia C. Leon-Chavarri holds a Master of Engineering in Civil and Environmental Engineering from the Massachusetts Institute of Technology (MIT), United States, and a Bachelor of Engineering in Industrial Engineering from Universidad de Lima, Peru. She currently teaches in the Industrial Engineering Program at Universidad de Lima, is registered as a RENACYT researcher at Level VII, and has served as Vice President of the MIT Club Peru since 2026. She is General Manager of Ingenio Consultoría & Emprendimiento SAC and previously taught full-time at Universidad Peruana de Ciencias Aplicadas (UPC). She is a Senior Fellow of the IEEE and a member of the ACM. Her publications include “A Production Management Method to Reduce Non-Fulfillment of Orders Based on Lean Tools and Change Management: Case of a Peruvian Apparel Company” and “Process Optimization through the Implementation of SLP, Kanban, Poka-Yoke and TPM to Improve Efficiency in a Metalworking SME,” both published in the International Journal of Engineering Trends and Technology. Her areas of interest include sustainable design, sustainable manufacturing, textile sustainability, water treatment, and photocatalysis.

  • Rafael Chavez-Ugaz, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Rafael Chávez Ugaz holds a Master’s degree in Strategic Business Administration from Centrum Católica, Perú; a Master of Business Administration in General and Strategic Management from Maastricht School of Management, the Netherlands; a degree in Management from the Netherlands; and a Bachelor of Engineering in Industrial Engineering from Universidad de Lima, Peru. He currently teaches in the Industrial Engineering Program at Universidad de Lima, where he is also a researcher and a member of the Faculty of Engineering Council. He previously participated as an associate researcher in technological innovation projects funded by Fondecyt. His publications include “Implementation of Lean Manufacturing and SLP in an SME in the Jewelry Sector: Case Study,” published in the International Congress on Innovation and Trends in Engineering; “Ergonomic Risk Management Model Using RULA, NIOSH and OCRA Techniques in a Metalworking SME,” published in the Proceedings of the LACCEI International Multi-Conference for Engineering, Education and Technology; and “Textile Characteristics of Huacaya Alpaca Fiber, According to Agroecological Zones, Sex and Age in the Puno Region (Peru),” published in the Peruvian Journal of Veterinary Research. His areas of interest include production operations for goods and services, supply chain management, and last-mile logistics.

  • Lucia B. Suni-Chavez, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Lucia Britney Suni Chavez holds a Bachelor’s degree in Industrial Engineering from Universidad de Lima, Perú. She is currently an intern in the Services Department at TKE Elevator, where she has gained pre-professional experience in the industrial services sector and contributes to operational coordination and process management through data analysis and performance monitoring. Her research interests include operations and service management, maintenance and asset management, operations planning and control, performance measurement, and continuous process improvement, with an emphasis on optimization and operational efficiency in industrial settings.

  • Fabricio H. Paredes-Larroca, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Fabricio Humberto Paredes Larroca holds a PhD in Systems Engineering from Universidad Nacional de Ingeniería, a Master’s degree in Automation and Instrumentation from the same institution, and a degree in Industrial Engineering from Universidad de Lima. He has also completed further studies at the Massachusetts Institute of Technology (MIT), earning certifications in Industry 4.0, Internet of Things, Machine Learning, and Smart Manufacturing. He currently serves as a Full Professor and Researcher at Universidad de Lima. His academic career includes teaching experience since 2002 and positions such as Academic Coordinator, Head of the CIM Laboratory, and Member of the School Council. He has participated in scientific projects related to non-potable water treatment, the development of solar photocatalytic reactor prototypes, intelligent process control using fuzzy logic, and technological innovation aimed at improving the quality of life of people with disabilities. He holds more than ten patents registered in Peru, related to automation devices, renewable energy, applied robotics, and solutions for the agricultural sector. He has a strong publication record indexed in Scopus and Web of Science, with research published in high-impact journals on water treatment, applied artificial intelligence, and environmental sustainability. His research focuses on automation, control systems, applied artificial intelligence, smart manufacturing, and industrial engineering.

  • Ezilda M. Cabrera-Gil, Carrera de Ingeniería Industrial, Universidad de Lima, Perú

    Ezilda Cabrera Gil Grados holds a Master’s degree in Business Administration from Universidad ESAN, Perú, and a degree in Industrial Engineering from Universidad de Lima, Peru. She currently works as a professor in the Industrial Engineering Program at Universidad de Lima. She has published the book Linear Programming Models: A Guide to Their Formulation and Solution (Fondo Editorial de la Universidad de Lima, 2017) and several articles in academic conference proceedings, including “Improvement Proposal to Achieve Shrinkage Reduction in the Home Dispatch Process in a Retail Company,” “The Origins and Evolution of Social Commerce: Enhancing E-Commerce Platforms with Social Features,” “Home Office Physical and Psychosocial Ergonomic Effects on the Job Satisfaction of Service Sector Employees in Peru,” and “Study of Carbon Monoxide Levels in the Most Traveled Streets of Metropolitan Lima,” among others. Her main professional area of interest is operations research.

Referencias

[1] Ombudsman’s Office, “Situation of inclusive education in Peru,” Ombudsman’s Office, Lima, Peru, 2022.

[2] Pontificia Universidad Católica del Perú, “Diccionario LSP al Español [Virtual Bilingual Dictionary of Peruvian Sign Language],” 2022. [Online]. Available: https://diccionariolsp.pucp.edu.pe/search-by-text

[3] Yapaykuy. (2023). App Store. Lima, Peru. [Mobile app]. Available: https://apps.apple.com/pe/app/yapaykuy/id6478118708

[4] K. Abbas, R. Kumar, and P. Jain, “Sign language recognition using CNN,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 5, pp. 1941–1953, May 2021.

[5] J. DelPreto, J. Hughes, M. D’Aria, M. de Fazio, and D. Rus, “A wearable smart glove and its application of pose and gesture detection to sign language classification,” IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 10589–10596, 2022, doi: 10.1109/LRA.2022.3191232.

[6] A. Ji, Y. Wang, X. Miao, T. Fan, B. Ru, L. Liu, R. Nie, and S. Qiu, “Dataglove for sign language recognition of people with hearing and speech impairment via wearable inertial sensors,” Sensors, vol. 23, no. 15, Art. no. 6693, 2023, doi: 10.3390/s23156693

[7] A. Radzi, S. Salim, M. H. Abd Wahab, M. M. Abdul Jamil, and T. C. Phing, “Development of a wearable sensor glove for real-time sign language translation". Ann. Emerg. Technol. Comput., vol. 7, no. 5, pp. 25–38. doi: 10.33166/AETiC.2023.05.003

[8] World Health Organization, World Report on Hearing. Geneva, Switzerland: WHO, 2021. [Online]. Available: https://www.who.int/publications/i/item/world-report-on-hearing

[9] H. Knoors, and M. Marschark, “Language planning for the 21st century: Revisiting bilingual language policy for deaf children,” J. Deaf Stud. Deaf Educ., vol. 17, no. 3, pp. 291–305, 2012, doi: 10.1093/deafed/ens018.

[10] C. Tannenbaum-Baruchi, B. Ahad Ha’am, and R. Harari, “Accessibility and support in higher education: A case study of deaf and hard of hearing students’ perceptions,” High. Educ., 2025, doi: 10.1007/s10734-025-01586-x.

[11] H. Wang, et al., “Enhanced open biomass burning detection: The BranTNet approach using UAV aerial imagery and deep learning for environmental protection and health preservation,” Ecol. Indic., vol. 154, Art. no. 110788, 2023, doi: 10.1016/j.ecolind.2023.110788.

[12] L. Findlater et al., “Design preferences and barriers for deaf and hard of hearing users of mobile technology,” in Proc. ASSETS 2019, Pittsburgh, PA, USA, Oct. 28–30 2019.

[13] A. Gangal, A. Kuppahally, and M. Ravindran, Sign Language Recognition with Convolutional Neural Networks, Stanford CS231n Project Report, 2024. [Online]. Available: https://cs231n.stanford.edu/2024/papers/sign-language-recognition-with-convolutional-neural-networks.pdf

[14] A. Kasapbaşı, A. E. A. Elbushra, O. Al-Hardanee, and A. Yilmaz, “DeepASLR: A CNN-based human–computer interface for American Sign Language recognition for hearing-impaired individuals,” Comput. Methods Programs Biomed. Update, vol. 2, p. 100048, Jan. 2022, doi: 10.1016/j.cmpbup.2021.100048.

[15] S. Subburaj, S. Murugavalli, and B. Muthusenthil, “Sign language video to text conversion via optimised LSTM with improved motion estimation,” J. Exp. Theor. Artif. Intell., vol. 37, no. 1, pp. 163-182, 2024, doi: 10.1080/0952813X.2024.2380991.

[16] Y. Wang, H. Jiang, Y. Sun, and L. Xu, “A static sign language recognition method enhanced with self-attention mechanisms,” Sensors, vol. 24, no. 21, p. 6921, 2024, doi: 10.3390/s24216921.

[17] A. Elhagry and R. G. Elrayes, “Egyptian Sign Language recognition using CNN and LSTM,” arXiv, Jul. 28, 2021. [Online]. Available: https://arxiv.org/abs/2107.13647

[18] S. M. S. Swetha, M. S. Muneshwara, V. Venkatesh, S. M. S. Kudremane, R. S. Nair, and V. H. Vismitha, “Developing a translation model to bridge ASL and Indian regional languages using neural networks,” in Proc. 2025 Int. Conf. Comput. Sustain. Intell. Future (COMP-SIF), Bangalore, India, Mar. 21–22, 2025, pp. 1–6.

[19] Indian Patent Office, A sign language translator glove, Patent application IN201841026260 A, Jul. 20, 2018. [Online]. Available: https://patentscope.wipo.int/search/es/detail.jsf?docId=IN224134191&_cid=P20-MFKLFW-88048-1

[20] Indian Patent Office, Sign language translator for deaf and speech impaired, Patent application IN202321041213 A, 2023. [Online]. Available: https://patentscope.wipo.int/search/es/detail.jsf?docId=IN414860224&_cid=P20-MFKLGX-89227-1

[21] Korean Intellectual Property Office, Sign language translator and method thereof, particularly for translating the video of a finger language into a voice or text, Patent KR1020100026701 A, 2010. [Online]. Available: https://patentscope.wipo.int/search/es/detail.jsf?docId=KR5842487&_cid=P20-MFKLHM-90082-1

[22] J. A. Czechowicz et al., “Hearing impairment and poverty: The epidemiology of ear disease in Peruvian schoolchildren,” Otolaryngol. Head Neck Surg., vol. 142, no. 2, pp. 272–277, 2010. doi: 10.1016/j.otohns.2009.10.040

[23] B. Alsharif, A. S. Altaher, A. Altaher, M. Liyas, and E. Alalwany, “Deep learning technology to recognize American Sign Language alphabet,” Sensors, vol. 23, no. 19, p. 7970, 2023. doi: 10.3390/s23197970

[24] D. Kothadiya, C. Bhatt, K. Sapariya, K. Patel, A.-B. Gil-González, and J. M. Corchado, “Deepsign: Sign language detection and recognition using deep learning,” Electronics, vol. 11, no. 11, p. 1780, 2022. doi: 10.3390/electronics11111780

[25] K. Assaleh and M. Al-Rousan, “Recognition of Arabic sign language alphabet using polynomial classifiers,” EURASIP Journal on Advances in Signal Processing, vol. 2005, Art. no. 507614, 2005, doi: 10.1155/ASP.2005.2136

Descargas

Publicado

2026-06-08

Número

Núm. 004 (2025): IV Congreso Internacional de Ingeniería Industrial

Sección

Ponencias

Licencia

Derechos de autor 2026 Congreso Internacional de Ingeniería Industrial de la Universidad de Lima

Creative Commons License

Esta obra está bajo una licencia internacional Creative Commons Atribución 4.0.