Correo Electrónico
DNINFOA - SIA
Bibliotecas
Convocatorias
Identidad U.N.
Published
Artificial Neuronal Networks: A Bayesian Approach Using Parallel Computing
Redes neuronales regularizadas: un enfoque bayesiano usando cómputo paralelo
DOI:
https://doi.org/10.15446/rce.v41n2.55250Keywords:
Empirical Bayes, Nonlinear models, Parallel processing (en)Bayes emp\'irico, modelos no lineales, procesamiento en paralelo (es)
Downloads
An Artificial Neural Network (ANN) is a learning paradigm and automatic processing inspired in the biological behavior of neurons and the brain structure. The brain is a complex system; its basic processing unit are the neurons, which are distributed massively in the brain sharing multiple connections between them. The ANNs try to emulate some characteristics of humans, and can be thought as intelligent systems that perform some tasks in a different way that actual computer does. The ANNs can be used to perform complex activities, for example: pattern recognition and classification, weather prediction, genetic values prediction, etc. The algorithms used to train the ANN, are in general complex, so therefore there is a need to have alternatives which lead to a significant reduction of times employed to train an ANN. In this work, we present an algorithm based in the
strategy ``divide and conquer'' which allows to train an ANN with a single hidden layer. Part of the sub problems of the general algorithm used for training are solved by using parallel computing techniques, which allows to improve the performance of the resulting application. The proposed algorithm was implemented
using the C++ programming language, and the libraries Open MPI and ScaLAPACK. We present some application examples and we asses the application performance. The results shown that it is possible to reduce significantly the time necessary to execute the program that implements the algorithm to train the ANN.
procesamiento automático inspirado en el comportamiento biológico de las neuronas y en la estructura del cerebro. El cerebro es un sistema altamente complejo; su unidad básica de procesamiento son las neuronas, las cuales se encuentra distribuidas de forma masiva compartiendo múltiples conexiones
entre ellas. Las RNAs intentan emular ciertas características propias de los humanos, pueden ser vistas como un sistema inteligente que lleva a cabo tareas de manera distinta a como lo hacen las computadoras actuales. Las RNAs pueden emplearse para realizar actividades complejas, por ejemplo: reconocimiento y clasicación de patrones, predicción del clima, predicción de valores genéticos, etc. Los algoritmos utilizados para entrenar las redes, son en general complejos, por lo cual surge la necesidad de contar con alternativas que permitan reducir de manera signicativa el tiempo necesario para entrenar una red. En este trabajo se presenta una propuesta de algoritmos basados en la estrategia divide y conquista que permiten entrenar las RNAs de una sola capa oculta. Parte de los sub problemas del algoritmo general de entrenamiento se resuelven utilizando técnicas de cómputo paralelo, lo que permite mejorar el desempeño de la aplicación resultante. El algoritmo propuesto fue implementado utilizando el lenguaje de programación C++, así como las librerías Open MPI y ScaLAPACK. Se presentan algunos ejemplos de aplicación y se evalúa el desempeño del programa resultante. Los resultados obtenidos muestran que es posible reducir de manera signicativa los tiempos necesarios para ejecutar el programa que implementa el algoritmo para el ajuste de la RNA.
References
Blackford, L. S., Choi, J., Cleary, A., D'Azevedo, E., Demmel, J., Dhillon, I.,
Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker,
D. & Whaley, R. C. (2012), 'ScaLAPACK users' guide', Netlib's official site.
-03-14. *http://netlib.org/scalapack/slug/index.html
Crossa, J., de los Campos, G., Pérez-Rodríguez, P., Gianola, D., Burgueño, J.,
Araus, J. L., Makumbi, D., Singh, R. P., Dreisigacker, S., Yan, J., Arief, V.,
Banziger, M. & Braun, H.-J. (2010), 'Prediction of genetic values of quantitative traints in plant breeding using pedigree and molecular markers', Genetics 186(2), 714-724.
Foresee, F. D. & Hagan, M. T. (1997), 'Gauss Newton approximation to Bayesian learning', Proc. IEEE International Conference Neural Networks 3, 1931-1933.
Fox, J. (2008), Applied Regression Analysis and Generalized Linear Models, SAGE Publications. *http://books.google.ch/books?id=GKkn3LSSHFsC
Gabriel, E., Fagg, G. E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J. M., Sahay, V., Kambadur, P., Barrett, B., Lumsdaine, A., Castain, R. H., Daniel, D. J., Graham, R. L. & Woodall, T. S. (2004), Open MPI: Goals, concept, and design of a next generation MPI implementation, in 'Proceedings, 11th European PVM/MPI Users' Group Meeting', Budapest, Hungary, pp. 97-104.
Gianola, D., Okut, H., Weigel, K. A. & Rosa, G. J. (2011), 'Predicting complex
quantitative traits with bayesian neural networks: a case study with jersey
cows and wheat', BMC Genetics 87, 7-8.
Girosi, F. & Poggio, T. (1989), 'Representation properties of networks: Kolmogorov's theorem is irrelevant', Neural Computation 1(4), 465-469.
*http://dx.doi.org/10.1162/neco.1989.1.4.465
Kolmogorov, A. K. (1957), 'On the representation of continuous functions of several variables by superposition of continuous functions of one variable and addition', Doklady Akademii Nauk SSSR 114, 369-373.
Lampinen, J. & Vehtari, A. (2001), 'Bayesian approach for neural networks review and case studies', Neural networks 14(3), 259-262.
Lampinen, J., Vehtari, A. & Leinonen, K. (1999), 'Application of bayesian neural network in electrical impedance tomography', 6, 3942-3947.
Lebrón-Aldea, D., Dhurandhar, E. J., Pérez-Rodríguez, P., Klimentidis, Y. C.,
Tiwari, H. K. & Vazquez, A. I. (2015), 'Integrated genomic and bmi analysis
for type 2 diabetes risk assessment', Frontiers in Genetics 6, 1-8.
Levenberg, K. (1944), 'A method for the solution of certain problems in least squares', Quart. Applied Math. 2, 164-168.
MacKay, D. J. (1994), 'Bayesian non-linear modelling for the prediction competition', ASHRAE transactions 100, 5-12.
Marquardt, D. W. (1963), 'An algorithm for least-squares estimation of nonlinear parameters', SIAM Journal on Applied Mathematics 11(2), 431-441.
Neal, R. M. (1996), Bayesian Learning for Neural Networks, Springer-Verlag New York, Inc., Secaucus, NJ, USA.
Nguyen, D. & Widrow, B. (1990), Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights, in 'International Symposium on Neural Networks'.
Okut, H., Gianola, D., Rosa, G. J. & Weigel, K. (2011), 'Prediction of body
mass index in mice using dense molecular markers and a regularized neural
network', Genetics research 93, 189-201.
Pérez-Rodríguez, P., Gianola, D., González-Camacho, J. M., Crossa, J., Manes, Y. & Dreisigacker, S. (2012), 'Comparison between linear and non-parametric regression models for genome-enabled prediction in wheat', G3 2(12), 1595-1605.
Pérez-Rodríguez, P., Gianola, D., Weigel, K. A., Rosa, G. J. M. & Crossa, J.
(2013), 'Technical note: An R package for fitting Bayesian regularized neural networks with applications in animal breeding', Journal of Animal Science 91, 3522-3525.
Prechelt, L. (2012), Early stopping - but when?, in G. Montavon, G. B. Orr &
K.-R. Müller, eds, 'Neural Networks: Tricks of the Trade: Second Edition',
Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 53-67. *https://doi.org/10.1007/978-3-642-35289-8_5
Yoginath, S., Bauer, D., Kora, G., Samatova, N., Kora, G., Fann, G. & Geist, A.
(2009), 'RScaLAPACK: High-performance parallel statistical computing with R and ScaLAPACK'. 2016-01-12. *http://web.ornl.gov/ webworks/cppr/y2001/pres/124121.pdf
How to Cite
APA
ACM
ACS
ABNT
Chicago
Harvard
IEEE
MLA
Turabian
Vancouver
Download Citation
CrossRef Cited-by
1. Nikita A. Kharitonov, Anatoly G. Maximov, Alexander L. Tulupyev. (2019). Recent Research in Control Engineering and Decision Making. Studies in Systems, Decision and Control. 199, p.696. https://doi.org/10.1007/978-3-030-12072-6_56.
2. Fernando Mesa, Rogelio Ospina-Ospina, Diana Marcela Devia-Narvaez. (2023). Pattern recognition for the modification of characteristics using non-linear techniques. Revista UIS Ingenierías, 22(1) https://doi.org/10.18273/revuin.v22n1-2023002.
Dimensions
PlumX
Article abstract page views
Downloads
License
Copyright (c) 2018 Revista Colombiana de Estadística
This work is licensed under a Creative Commons Attribution 4.0 International License.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
