Nishanth J. Kumar
AI, ML, and Robotics researcher + Ph.D. Student @ MIT CSAIL
I build AI systems that can solve complex, long-horizon tasks in the real-world — whether that’s a digital agent booking a personalized trip or a robot making your favorite tea. My research combines modern (deep) learning with automated planning, search and reasoning to create general-purpose agents that scale in both training and deployment. I tend to draw on ideas from task and motion planning (TAMP), foundation models, reinforcement learning, logic, deep learning, and program synthesis.
I’m currently a final year Ph.D. student with the LIS Group within MIT CSAIL. I’m officially advised by Leslie Kaelbling and Tomás Lozano-Pérez, and have the pleasure of collaborating with many other wonderful people within CSAIL’s Embodied Intelligence Initiative. I’m extremely grateful for support from the NSF Graduate Research Fellowship. I’ve also had the pleasure of interning at Meta FAIR, NVIDIA Research, the Robotics and AI Institute, Vicarious AI, and Uber ATG. Previously, I received an S.M. degree from MIT, an Sc.B. with honors from Brown University, and completed the IB Diploma in my hometown of Coimbatore, India.
Outside of research, I like to lift heavy things, read, play basketball, philosophize, cook, and write both fiction and non-fiction. If you’re interested in learning more about me or reading some of my writing, check out my blog, fiction writing, or social links in the website footer. If you’d like to get in contact, check out this page here. If you’d like to leave me some anonymous feedback (preferably constructive!), see this form.
news
| Jan 20, 2025 | I’m excited to spend Summer 2025 as an intern at FAIR in NYC working on improving long-horizon generation and decision-making for LLMs with Mary Williamson, Jimmy Yang, and Yuandong Tian. |
|---|---|
| Dec 25, 2024 | New paper on inventing symbolic predicates using VLM’s accepted at the LLM4Plan and GenPlan AAAI 2025 workshops! Very excited about continuing to push in this direction and get this stuff onto real-world robots. |
| Sep 03, 2024 | Two new papers on LLMs+TAMP and Language-Guided Abstraction Learning accepted at CoRL 2024. |
research areas
My recent research has been organized around a few distinct directions, which are highlighted below. For a complete and up-to-date list of papers, please see my Google Scholar page.
Planning and Learning with Foundation Models
Foundation models have an implicit ability to sequence high-level skills --- like 'pick', 'place', or 'move to' --- to accomplish a goal, but are unable to reason over low-level joint angles or torques. By combining foundation models with structured planning methods like constraint satisfaction or TAMP, we can create sophisticated and general-purpose systems capable of solving complex tasks directly from vision and natural language input.
Learning to Plan from Demonstrations
Structured search-based methods like TAMP work well with tediously hand-engineered components. By leveraging ideas from program synthesis, we can instead output the components of planning domains --- such as symbolic predicates and operators --- directly from demonstrations and potentially scale systems with data instead of human effort. After learning, we run a planner at test-time to zero-shot complex new tasks via reasoning.
Planning to Learn in Deployment
Robots and web agents are bound to encounter unexpected failures in their deployment environments (new objects, layouts, websites, etc.). Ideally, they should rapidly learn to overcome these failures online. We can leverage planning to not only find alternative ways to recover from failures, but also autonomoously explore an environment to collect data most useful to prevent future failures of the same kind.