NarrativeFlow links robot motion to language instructions
A Keio study tests a shared motion representation across robot demonstrations
Keio University researchers describe NarrativeFlow in a paper submitted to arXiv on October 1. It predicts robot motion from an instruction and an initial image, using video demonstrations from different robot platforms.
Motion becomes an intermediate step
The method generates motion fields for a separate control policy. Training also uses descriptions of scene changes to emphasize task relevant information.
The wider test covered 13 tasks
Across 260 trials per method on Toyota’s Human Support Robot, the appendix reports rounded average success of 58% versus 45% for Im2Flow2Act. Cup stacking reached 20%. Each model received task specific demonstrations and fine tuning.
What remains unproven
These are authors’ results on one physical robot platform. The project page links a paper and demonstrations, with no public code or weights link. Depth aware motion remains future work. It assumes a fixed camera and a visible robot gripper at the start.
Archival robot hand photograph by Franck V., published in August 2018, under the Unsplash License. This illustrative image does not show the study’s robot or NarrativeFlow. Source supplied JPEG rendition with no local edits. No endorsement is implied.
