Visual Task Planning

Planning only with visual information.

The emerging research area of visual task planning attempts to learn representations suitable for planning directly from visual inputs, alleviating the need for accurate geometric models. Current methods commonly assume that similar visual observations correspond to similar states in the task planning space. However, observations from sensors are often noisy with several factors that do not alter the underlying state, for example, different lightning conditions, different viewpoints, or irrelevant background objects. These variations result in visually dissimilar images that correspond to the same task state. Achieving robust abstract state representations for real world tasks is an important research area that the ELPIS lab is focusing on.

Relevant Publications

  1. Comparing Reconstruction-and Contrastive-based Models for Visual Task Planning
    C. Chamzas*, M. Lippi*, M. C. Welle*, A. Varava, L. E. Kavraki, and D. Kragic
    In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022
  2. RAL
    MotionBenchMaker: A Tool to Generate and Benchmark Motion Planning Datasets
    C. Chamzas, C. Quintero-Peña, Z. Kingston, A. Orthey, D. Rakita, M. Gleicher, M. Toussaint, and L. E. Kavraki
    IEEE Robotics and Automation Letters, 2022
  3. State Representations in Robotics: Identifying Relevant Factors of Variation using Weak Supervision
    C. Chamzas*, M. Lippi*, M. C. Welle*, A. Varava, M. Alessandro, L. E. Kavraki, and D. Kragic
    In NeurIPS, 3rd Robot Learning Workshop: Grounding Machine Learning Development in the Real World, 2020