Jaeeun's Research

Option learning from action segmentation

Robots learning interactively with a human partner has several open questions, one of which is increasing the efficiency of learning. One approach to this problem in the Reinforcement Learning domain is to use options, temporally extended actions, instead of primitive actions. In this paper, we aim to develop a robot system that can discriminate meaningful options from observations of human use of low-level primitive actions. Our approach is inspired by psychological findings about human action parsing, which posits that we attend to low-level statistical regularities to determine action boundary choices. We implement a human-like action segmentation system for automatic option discovery and evaluate our approach and show that option-based learning converges to the optimal solutions faster compared with primitive-action-based learning.

In this work we show that robots can find options automatically from human-like action segmentation, and that these options enable them to more efficiently learn from demonstration. Our approach is as follows:

• Human-like Action Segmentation: We solve the option discovery problem inspired by human statistical learning from low-level primitive actions. The assumption is that it will be easier to predefine a low-level primitive action set than highlevel actions or possible action sequences.
• Efficiency of the Learned Options: Given the set of options learned in the previous step, we evaluate their effectiveness in speeding up the learning process. We show these options provide efficiency gains with respect to learning an optimal policy with just primitive actions. We also show that our option discovery method shows better performance than a state-of-theart option discovery approach.

Publications

• Jaeeun Shim and Andrea L. Thomaz. "Human-Like Action Segmentation for Option Learning," 20th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011, pp. 455-460.
• Baris Akgun, Kaushik Subramanian, Jaeeun Shim, and Andrea Lockerd Thomaz. "Learning Tasks and Skills Together From a Human Teacher," Association for the Advancement of Artificial Intelligence (AAAI), 2011.
• Aaron Curtis, Jaeeun Shim, Eugene Gargas, Adhityan Srinivasan, and Ayanna M. Howard. "Dance dance Pleo: developing a low-cost learning robotic dance therapy aid," International Conference on Interaction Design and Children (IDC), 2011,149-152.