Research

Dexterity lives as much in the head as in the hand.

Under this research thrust, our current focus aims at answering two broad, open questions.

1. Many of today's robot learning systems are kinematic -- they don't explicitly account for dynamics (e.g., forces). How do we build robot learning systems that account for forces as central entities in manipulation?
2. Most robot data collected for modern learning methods attempt to minimize external contact, often just to the extent it is needed for a downstream task. How do we design better hardware, control, and learning algorithms that embrace contacts for more performant and robust manipulation?

Our view is that manipulation is fundamentally about sensing and exerting forces. For example, a robot hand could be holding a golf ball or an egg with near-identical joint angles, but in each case, the joints would be exerting very different forces on the objects held. We work on treating forces as the central sensed and output quantities in a manipulation system. Relatedly, we're focused on designing contact-rich manipulation systems (both at the hardware and software level) that embrace contact with the physical world.

We are a full-stack lab -- this means we also design hardware and sensing capabilities that push the capability envelope of robot manipulation approaches.

We have ongoing research (in many cases, collaborations with other experts from materials science, electronics, mechanical, and biomedical engineering) on designing novel tactile sensors for robots, and incorporating them into modern robot learning methods.

We also are actively working towards designing in-house dexterous hands with tactile and/or force sensing capabilities, palms/wrists, and soft / hybrid robots that exhibit compliance at the material level.

While visual perception is a dominant modality for robot learning today, (a) visual perception alone is less robust and operates primarily when objects are in the field of view of the sensors, (b) tactile sensing has the potential to provide instantaneous, high-frequency feedback that enables dynamic manipulation.

A major thrust of our lab is to explore touch (a.k.a. haptic/tactile) and other multisensory cues that complement visual perception. We collaborate closely with other groups at Hopkins to develop novel tactile sensors that are fast, cheap, and provide rich sensory feedback.

In the near term, we are interested in embedding sensors into deformable skin surfaces, fingers, grippers, etc., enabling easy customization to robot surface geometries and sensing requirements. We're also exploring visuotactile control policy learning and cue integration for tactile sensors and other complementary force sensors.

Designing general-purpose humanoid behaviors involves whole-body planning and control. Think about humanoids navigating through tight spaces, reaching upwards/downwards to retrieve/place objects, and likely leveraging the external environment for support. We study planning and control methods that allow humanoids to perform these behaviors robustly while adapting to uncertainty and unexpected interactions.

We're particularly interested in compliant and contact-aware control, with (and especially without) direct force sensing. We're exploring whole-body teleoperation, shared autonomy, and social interaction/navigation with humanoid robots.

World models enable thinking before acting

Most learned robot policies produce actions conditioned primarily on current observations. World models instead allow a robot to think through an action sequence, before it ever acts. By imagining and comparing multiple plausible future consequences, robots can optimize a sequence of actions most likely to accomplish the task at hand.

While the popular connotation of world models in robotics refers to action-conditioned models that emulate the consequences of robot actions, our lab also draws upon the cognitive-science underpinnings of the term, which, in its broadest sense, refers to mental/internal models we construct that help us perceive and reason about the external world and ultimately drive decision-making.

World models could steer existing policies during deployment, detect imminent failures, help robots adapt in context, or even be used to evaluate -- and eventually train -- control policies.