Published
Aprendizaje de selección de acciones en un mundo simple pero impredecible
Keywords:
Aprendizaje por refuerzo, Aprendizaje Q, Agentes autónomos, Animats (es)Reinforcement learning, Q learning, Autonomous agents, Animats (en)
Downloads
Uno de los principales problemas estudiados en la simulación de agentes artificiales autónomos es el de la selección de acciones: un mecanismo que le permita al sistema escoger la acción más apropiada para la situación en que se encuentre, de tal forma que maximice su medida de éxito. El aprendizaje por refuerzo representa un enfoque atractivo para atacar este problema, ya que se basa en la búsqueda de señales de premio y la evasión de señales de castigo mediante un proceso de ensayo y error. En este artículo presentamos al PAISA I, una criatura artificial que aprende a comportarse (seleccionar acciones) utilizando una técnica de aprendizaje por refuerzo (aprendizaje Q) para optimizar la cantidad de comida que puede encontrar en un mundo impredecible, aunque con un espacio estado-acción pequeño.
One of the main problems studied in simulation of artificial autonomous agents is the action-selection: a mechanism that allows the system to choice the more suitable action for the specific situation where it is located, in such a way that maximizes his success measure. The reinforcement learning represents an attractive approach to attack this problem, because it is based in the searching of awards signals and the refusing of punishments by a trial and error process. In this paper, we present the PAISA I, an artificial creature that learns to behave (that is, action-selection) using a reinforcement learning technique known as Q-learning, to optimize the amount of food that he can find in an unpredictable world, although in a small state-action space.
References
[Baird, 1994] Baird, L. C. (1994). Reinforcement Learning in Continuous Time: Advantage Updating. Proceedings of the International Conference on Neural Networks.
[Lin, 1992] Lin, L. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. En: Machine Learning, 8.
[Moriarty et al., 1996] Moriarty, D. E.; Miikkulainen, R. (1996). Efficient reinforcement learning through symbiotic evolution, En: Machine Learning, 22.
[Munos et al., 1994] Munos, R.; Patinel, J. (1994). Reinforcement learning with dynamic covering of state-action: partitioning Q-learning. En: Cliff, D.; Husbands, P; Meyer, J. A.; Wilson, S. W. (Eds), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior. The MIT Press/Bradford Books.
[Peng, 1993] Peng, J. (1993). Efficient Dynamic Programming-based Learning for Control. Tesis doctoral. College of Computer Science of Northeastern University.
[Peng et al., 1996] Peng, J.; Williams, R. J. (1996). Incremental Multi-step Q-Learning. En: Machine Learning, 22.
[Rojas, 1998] Rojas, S. A. (1998). Disertación teórica sobre simulaciones inspiradas biológicamente para el estudio del comportamiento adaptativo. Monografía de grado. Facultad de Ingeniería de la Universidad Nacional de Colombia.
[Sutton et al., 1998] Sutton, R. S.; Barto, A. G. (1998). Reinforcement Learning: An Introduction. The MIT Press.
[Watkins et al., 1992] Watkins, C. J.; Dayan, P (1992). Q-Learning. En: Machine Learning, 8.
License
Copyright (c) 2002 Sergio A. Rojas, José J. Martínez
This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors or holders of the copyright for each article hereby confer exclusive, limited and free authorization on the Universidad Nacional de Colombia's journal Ingeniería e Investigación concerning the aforementioned article which, once it has been evaluated and approved, will be submitted for publication, in line with the following items:
1. The version which has been corrected according to the evaluators' suggestions will be remitted and it will be made clear whether the aforementioned article is an unedited document regarding which the rights to be authorized are held and total responsibility will be assumed by the authors for the content of the work being submitted to Ingeniería e Investigación, the Universidad Nacional de Colombia and third-parties;
2. The authorization conferred on the journal will come into force from the date on which it is included in the respective volume and issue of Ingeniería e Investigación in the Open Journal Systems and on the journal's main page (https://revistas.unal.edu.co/index.php/ingeinv), as well as in different databases and indices in which the publication is indexed;
3. The authors authorize the Universidad Nacional de Colombia's journal Ingeniería e Investigación to publish the document in whatever required format (printed, digital, electronic or whatsoever known or yet to be discovered form) and authorize Ingeniería e Investigación to include the work in any indices and/or search engines deemed necessary for promoting its diffusion;
4. The authors accept that such authorization is given free of charge and they, therefore, waive any right to receive remuneration from the publication, distribution, public communication and any use whatsoever referred to in the terms of this authorization.
