Abstract—In this research, soccer task is investigated among the numerous tasks of deep reinforcement learning. The soccer task requires cooperative behavior. However, it is difficult for the agents to acquire the behavior, because a reward is sparsely given. Moreover, the behaviors of the allies and opponents must be considered by the agents. In addition, in the soccer task, if the agents attempt to acquire high-level cooperative behavior from low-level movements, such as ball kicking, a huge amount of time will be needed to learn a model. In this research, we conduct experiments in which reward shaping and curriculum learning are incorporated into deep reinforcement learning. This enables the agents to efficiently acquire cooperative behavior from low-level movements in a soccer task. The findings of this research indicate that reward shaping and curriculum learning with a designer’s domain knowledge positively influence the agent’s attempt to acquire cooperative behavior from low-level movements.
 

Index Terms—soccer, multi-agent reinforcement learning, reward shaping, curriculum learning, MuJoCo

Cite: Takashi Abe, Ryohei Orihara, Yuichi Sei, Yasuyuki Tahara, and Akihiko Ohsuga, "Step-by-Step Acquisition of Cooperative Behavior in Soccer Task," Journal of Advances in Information Technology, Vol. 13, No. 2, pp. 147-154, April 2022.

Copyright © 2022 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.