Robotic Navigation using LLM/VLM#

Proposed Study Plan#

My study plan for the duration of my exchange at the Tokyo Institute of Technology focuses on developing an integrated system that combines Robotic Navigation with Large Language Models (LLMs). This interdisciplinary project aims to enable robots to understand human language instructions and to utilize this capability to enhance their navigation systems. Furthermore, by proposing a new structure/methodology, I aspire to meet or exceed current benchmarks in this innovative domain.

Throughout the program, I will regularly consult with my advisors from both the Tokyo Institute of Technology and my home university to ensure the project remains on track and benefits from diverse expert insights. By the end of this exchange program, I expect to construct a robust understanding and demonstrate a novel application of integrating language models with robotic systems, positioning me well for future endeavors in AI and robotics research.

Here is the tenative timeline during my exchange at the Tokyo Institute of Technology:

1. Initial Learning Phase (First two months)

   • Objective: Acquire fundamental knowledge in robotic navigation.
   • Activities:
     • Study key robotic navigation concepts, focusing on sensor operation and essential navigation algorithms.
     • Paper study on the field of robotic navigation with LLM/VLM. Try to identify the insufficiency in previous work and understand the basic structure nowadays.

2. Integration and Development Phase (Third to tenth month):

   • Objective: Develop and refine the integrated Robotic Navigation with LLM system.
   • Activities:
     • Collaborate with peers in the laboratory to deepen understanding and practical skills in robotic navigation.
     • Implement initial models integrating LLMs into robotic navigational frameworks.
     • Conduct iterative testing and modifications based on feedback from professors and lab mates to enhance system performance.
     • Participate in group meetings and discussions to stay aligned with lab goals and share progress.

3. Finalization and Presentation Phase (Eleventh month):

   • Objective: Prepare and deliver a comprehensive presentation of the project outcomes.
   • Activities:
     • Collate and analyze data from the experimental tests.
     • Summarize methodologies, findings, challenges, and areas for future research.
     • Develop a detailed presentation that outlines the project’s scope, methodology, results, and implications.
     • Practice presentation skills with peers to refine content and delivery.