How should we build long-horizon manipulation systems? As coding agents become more capable, what interface should connect models to the physical world? We’ve been thinking about this since 2024.
Introducing Retriever, a programming framework for asynchronous, closed-loop robot systems, built over the past year at Stanford and Northeastern.
- Robot agents run in physical time with various feedback loops, currently lacking a good programming abstraction.
- Retriever introduces a time-aware building block, “Flow”, analogous to a PyTorch layer or a Ray Actor.
- Users connect Flows and specify when each runs and what inputs it reads. Retriever handles asynchronous execution.
- Agents are represented as directed graphs with temporal feedback loops for closed-loop, hierarchical systems, decoupling algorithm logic from execution.
We built Retriever-0 agent to demonstrate long-horizon, partially observable tasks combining perception, memory, reasoning, and manipulation.
Here's the system in action. Paper, code, and demos below.
Robots & gadgets
Retriever, a framework for robot agents that run in physical time
Linfeng Zhao, a postdoctoral researcher at Stanford, built Retriever with collaborators at Northeastern. Its time-aware 'Flow' blocks wire perception, memory, reasoning and manipulation into asynchronous feedback loops. The Retriever-0 agent demonstrates long-horizon, partially observable tasks.
148likes
24reposts
67saves
562Kviews
- Role
- Postdoctoral researcher, Stanford
- Posted
