Nishanth J. Kumar

AI, ML, and Robotics researcher + Ph.D. Student @ MIT CSAIL

njk-headshot-high-res.png

I am currently a final year Ph.D. student with the LIS Group within MIT CSAIL. My official advisors are Leslie Kaelbling and Tomás Lozano-Pérez, but I have the pleasure of collaborating with many other wonderful people within CSAIL’s Embodied Intelligence Initiative. I’m extremely grateful for support from the NSF Graduate Research Fellowship. I’ve also been lucky to intern at FAIR @ Meta, NVIDIA Research, the RAI Institute, Vicarious AI, and Uber ATG. Previously, I received an S.M. degree from MIT, an Sc.B. with honors from Brown University, and completed the IB Diploma in my hometown of Coimbatore, India.

Outside of research, I like to lift heavy things, read, play basketball, philosophize, cook, and write both fiction and non-fiction. If you’re interested in learning more about me or reading some of my writing, check out my blog, fiction writing, or social links in the website footer. If you’d like to get in contact, check out this page here. If you’d like to leave me some anonymous feedback (preferably constructive!), see this form.

Resume / CV / Scholar / LinkedIn / X/Twitter

news

Mar 12, 2026 I’ve accepted a full-time job offer to be an AI Research Scientist at Meta Robotics Studio. Very excited to work with Jonathan Tompson, Ning Li, Sangbae Kim, and the rest of the team on making helpful, deployable robots a reality!
Jan 25, 2026 I’m on the industry job market for research scientist and engineer positions! Feel free to check out my Resume and CV above, and do reach out if you think I’d be a good fit for your org.
Jan 15, 2026 Two new papers on open-world TAMP and learning symbolic world models from demonstration accepted at RA-L.
Jan 10, 2026 I’ve officially wrapped up my internship at Meta. I learned a lot about research and computer-use agents, and had an incredible time in NYC. Stay tuned for an eventual paper release!
Jan 20, 2025 I’m excited to spend Summer 2025 as an intern at FAIR in NYC working on improving long-horizon generation and decision-making for LLMs with Mary Williamson, Jimmy Yang, and Yuandong Tian.

research

I'm broadly interested in creating AI systems that can autonomously solve complex, long-horizon tasks in the real world. Much of my research has been at the intersection of learning, planning, and foundation models. I've sought to create general-purpose agents that scale well with data (at training time), and additional computation (at test time).

I've listed some selected representative publications below. For a complete and up-to-date list of papers, please see my Google Scholar page.

TiPToP: A Modular, Open-Vocabulary Planning System for Robotic Manipulation
William Shen*, Nishanth Kumar*, Sahit Chintalapudi, Jie Wang, Christopher Watson, Edward S. Hu, Jing Cao, Dinesh Jayaraman, Leslie Pack Kaelbling, Tomás Lozano-Pérez
arXiv, 2026
[Paper] [Code] [Website]
TiPToP is a modular, open-vocabulary planning system that accepts natural language commands and raw pixel inputs to execute long-horizon manipulation tasks—without any robot-specific training data. It compares favorably to a state-of-the-art end-to-end trained VLA, and generalizes across multiple robot platforms with minimal setup required.
From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Models
Ashay Athalye*, Nishanth Kumar*, Tom Silver, Yichao Liang, Jiuguang Wang, Tomás Lozano-Pérez, Leslie Pack Kaelbling
RA-L, 2026
[Paper] [Code] [Website]
We learn symbolic predicates and operators from human video demonstrations, constructing a "symbolic world model" that enables zero-shot generalization to new scenes and tasks via efficient planning.
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints
Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Caelan Reed Garrett
RA-L, 2026
[Paper] [Website]
We use VLMs to write constraints directly into TAMP systems at runtime. This leverages the common-sense and code generation capabilities of VLMs to enable TAMP to solve 'open-world' tasks it does not originally have the concepts to tackle.
VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang, Nishanth Kumar, Hao Tang, Adrian Weller, Joshua B. Tenenbaum, Tom Silver, João F. Henriques, Kevin Ellis
ICLR, 2025
[Paper]
We introduce “Neuro-Symbolic Predicates” that merge symbolic and neural knowledge representations through a first-order abstraction language. Our algorithm invents predicates and learns abstract world models, achieving better sample complexity, stronger out-of-distribution generalization, and improved interpretability compared to hierarchical RL, VLM planning, and other baseline symbolic approaches.
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Nishanth Kumar, Ankit Goyal, Tucker Hermans, Leslie Pack Kaelbling, Tomás Lozano-Pérez, Fabio Ramos
RSS, 2025
[Paper] [Code] [Website]
cuTAMP frames task and motion planning as a backtracking bilevel search over plan skeletons, using differentiable optimization to simultaneously explore thousands of candidate solutions on the GPU. This enables solving highly constrained manipulation problems with non-convex constraints in seconds, substantially improving upon traditional serial approaches.
AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
Jiafei Duan, Wilbert Pumacay, Nishanth Kumar, Yi Ru Wang, Shulin Tian, Wentao Yuan, Ranjay Krishna, Dieter Fox, Ajay Mandlekar, Yijie Guo
ICLR, 2025
[Website] [Paper] [Code]
AHA is a vision-language model for detecting and reasoning over failures in robotic manipulation. Using FailGen, a simulation framework that synthetically generates failure demonstrations by perturbing successful trajectories, we create a large-scale dataset for instruction-tuning. AHA outperforms six state-of-the-art alternatives and generalizes to real-world scenarios for improved error recovery.
Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction
Aidan Curtis*, Nishanth Kumar*, Jing Cao, Tomás Lozano-Pérez, Leslie Pack Kaelbling
CoRL, 2024
[Paper] [Code] [Website] [Spotlight Video]
LLMs are great at common sense but bad at physics; constraint solvers are the reverse. PRoC3S pairs the two so a robot can go from a natural-language instruction and a camera image to a long-horizon manipulation plan— no task-specific engineering required.
Practice Makes Perfect: Planning to Learn Skill Parameter Policies
Nishanth Kumar*, Tom Silver*, Willie McClinton, Linfeng Zhao, Stephen Proulx, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Jennifer Barry
RSS, 2024
[Paper] [Website]
We use planning to guide exploration and data collection in deployment environments so a robot can efficiently and autonomously learn to handle unexpected deployment failures when solving long-horizon tasks.
Learning Efficient Abstract Planning Models that Choose What to Predict
Nishanth Kumar*, Willie McClinton*, Rohan Chitnis, Tom Silver, Tomás Lozano-Pérez, Leslie Pack Kaelbling
CoRL, 2023
[Paper] [Code] [Website]
Most learned transition models predict everything; ours learns to ignore what doesn't matter. Using program synthesis on demonstrations, we discover compact planning operators that focus only on decision-relevant state—making abstract planning dramatically more efficient.
PDDL Planning with LLMs
Solving PDDL Planning Problems with Pretrained Large Language Models
Tom Silver, Varun Hariprasad, Reece Shuttleworth, Nishanth Kumar, Tomás Lozano-Pérez, Leslie Pack Kaelbling
FMDM Workshop @ NeurIPS, 2022
[Paper] [Video]
We investigate few-shot prompting of pretrained LLMs for solving PDDL planning problems across 18 domains. We find that LLM performance stems not only from syntactic pattern matching but also from commonsense understanding of English terms in the PDDL, and propose a simple mechanism for using LLM outputs to guide a heuristic-search planner.
Multi-Object Search using Object-Oriented POMDPs
Arthur Wandzel, Yoonseon Oh, Michael Fishman, Nishanth Kumar, Lawson L. S. Wong, Stefanie Tellex
ICRA, 2019
[Paper] [Video]
We formulate Multi-Object Search (MOS) as an Object-Oriented POMDP, enabling a mobile robot to find multiple objects under uncertainty. The OO-POMDP structure supports factoring the agent’s belief into independent object distributions, allowing belief size to scale linearly rather than exponentially in the number of objects.