Welcome to my Research page! I’m currently a 1st Year Ph.D. student with the LIS Group within MIT CSAIL. I’m officially advised by Leslie Kaelbling and Tomás Lozano-Pérez, though I frequently collaborate with Dylan Hadfield-Menell, Josh Tenenbaum, and many other wonderful people within CSAIL’s Embodied Intelligence Initiative. I’m extremely grateful for support from the NSF Graduate Research Fellowship.
Resume | CV | Google Scholar | Bio | GitHub | Twitter
Research Areas
I’m broadly interested in enabling robots to operate robustly in long-horizon, multi-task settings so that they can accomplish tasks like multi-object manipulation, cooking, or even performing household chores. To this end, I’m interested in combining classical AI planning and reasoning approaches with modern machine learning techniques. My research draws on ideas from reinforcement learning, task and motion planning (TAMP), continual learning, and neurosymbolic AI.
Publications
Conference Papers
![]() |
Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes Nishanth Kumar*, Sean Segal*, Sergio Casas, Mengye Ren, Jingkang Wang, Raquel UrtasunConference on Robot Learning (CoRL) poster, 2021. OpenReview / arXiv / poster Introduces fine-grained active selection via partial labeling for efficient labeling for perception and prediction. |
![]() |
Building Plannable Representations with Mixed Reality Eric Rosen, Nishanth Kumar, Nakul Gopalan, Daniel Ullman, George Konidaris, Stefanie TellexIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020. paper / video Introduces Action-Oriented Semantic Maps (AOSM's) and a system to specify these with mixed reality, which robots can use to perform a wide-variety of household tasks. |
![]() |
Multi-Object Search using Object-Oriented POMDPs Arthur Wandzel, Yoonseon Oh, Michael Fishman, Nishanth Kumar, Lawson L.S Wong, Stefanie TellexIEEE International Conference on Robotics and Automation (ICRA), 2019. paper / video Introduces the Object-Oriented Partially Observable Monte-Carlo Planning (OO-POMCP) algorithm for efficiently solving Object-Oriented Partially Observable Markov Decision Processes (OO-POMDPs) and shows how this can enable a robot to efficiently find multiple objects in a home environment. |
Workshop Papers and Extended Abstracts
![]() |
Task Scoping for Efficient Planning in Open Worlds Nishanth Kumar*, Michael Fishman*, Natasha Danas, Michael Littman, Stefanie Tellex George KonidarisAAAI Conference on Artificial Intelligence, Student Workshop,, 2020. paper
Introduces high-level ideas for how large Markov Decision Processes (MPDs) might be efficiently pruned to include only states and actions relevant to a particular reward function. This paper is subsumed by our arxiv preprint on task scoping.
|
![]() |
Knowledge Acquisition for Robots through Mixed Reality Head-Mounted Displays Nishanth Kumar*, Eric Rosen*, Stefanie TellexThe Second International Workshop on Virtual, Augmented and Mixed Reality for Human Robot Interaction, 2019. paper
Sketches high level ideas for how a mixed reality system might enable users to specify information for a robot to perform pick-place and other household tasks. This work is subsumed by our AOSM work.
|
Preprints
![]() |
Inventing relational state and action abstractions for effective and efficient bilevel planning Tom Silver*, Rohan Chitnis*, Nishanth Kumar, Willie McClinton, Tomás Lozano-Pérez, Leslie Kaelbling, Joshua TenenbaumarXiv, 2022. arxiv / Code
Introduces a new, program-synthesis inspired approach for learning neuro-symbolic and relational state and action abstractions from demonstrations. The abstractions are explicitly optimized for effective and efficient bilevel planning.
|
![]() |
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX Guangyao Zhou*, Nishanth Kumar*, Miguel Lázaro-Gredilla, Shrinu Kushagra, Dileep GeorgearXiv, 2022. arxiv / blog post / code
Introduces a new JAX-based framework that aims to make it easy to build and run inference on probabilistic graphical models (PGM's).
|
![]() |
Task Scoping: Building Goal-Specific Abstractions for Planning in Complex Domains Nishanth Kumar*, Michael Fishman*, Natasha Danas, Michael Littman, Stefanie Tellex George KonidarisarXiv, 2020. arxiv
Introduces a method for how large classical planning problems can be efficiently pruned to exclude states and actions that are irrelevant to a particular goal so that agents can solve very large, 'open-scope' domains that are capable of supporting multiple goals.
|
![]() |
Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control Nishanth Kumar*, Jonathan Chang*, Sean Hastings, Aaron Gokaslan, Diego Romeres, Devesh Jha, Daniel Nikovski, George Konidaris, Stefanie TellexarXiv, 2020. arxiv
Shows how the generalization capabilities of Behavior Cloning (BC) can be improved by learning a policy parameterized by some input that enables the agent to distinguish different goals (e.g. different buttons to press in a grid). Includes several exhaustive experiments in simulation and on two different robots.
|
Theses and Misc. Publications
![]() |
You Only Need What’s in Scope: Generating Task-Specific Abstractions for Efficient AI Planning Nishanth KumarUndergraduate Honors Thesis, Brown University, 2021. paper
Presents a detailed, thesis-style description of my work on Task Scoping. This work is largely subsumed by our 'Task Scoping' preprint.
|
Talks
- Learning Models with Side Effects for Efficient, Generalizable TAMP
MIT LIS Group Meeting. March 4, 2021. - Task Scoping: Generating Task-Specific Abstractions for Planning
MIT LIS Group Meeting. February 12, 2021. - What I’m working on now: Task Scoping and Parameterized Imitation Learning
Intelligent Robot Lab meeting. November, 2019. - Let’s Talk about AI and Robotics
I was interviewed about my work, experiences and advice on research for an episode of the interSTEM YouTube channel. - Action-Oriented Semantic Maps via Mixed Reality
The Second Ivy-League Undergraduate Research Symposuim (ILURS). Best Plenary Presentation Award.
The University of Pennsylvania. April, 2019. - Building intelligent, collaborative robots
Machine Intelligence Conference 2019
Boston University. September 2019.
Awards
- NSF GRFP Fellow
- Berkeley Fellowship (declined)
- Sigma Xi Inductee
- CRA Outstanding Undergraduate Research Award Finalist, 2021
- Tau Beta Pi Inductee
- Heidelberg Laureate, 2020
- Goldwater Scholarship, 2020.
- CRA Outstanding Undergraduate Research Award Honorable Mention, 2020
- Karen T. Romer Undergraduate Teaching and Research Award
- Best Plenary Presentation, The Second Ivy League Undergraduate Research Symposium, 2019
Industry Experience and Research Collaborations
- Vicarious AI, Union City, USA.
Worked with Stannis Zhou, Wolfgang Lehrach, and Miguel Lázaro-Gredilla on developing PGMax - an open-source framework for ML with PGM’s. - MERL, Cambridge, USA.
Worked with Diego Romeres, Devesh Jha and Daniel Nikovski on furthering Learning from Demonstration for industrial robots. - Uber ATG Research, Toronto, Canada.
Worked with Sean Segal, Sergio Casas, Wenyuan Zeng, Jingkang Wang, Mengye Ren and others on a project exploring Active Learning for Self-Driving Vehicles that lead to a paper (read it here!). I had the honor of being advised by Prof. Raquel Urtasun.
Teaching
- Head Teaching Assistant, CSCI 2951-F: Learning and Sequential Decision Making
Brown University, Fall 2019 - Teaching Assistant, ENGN 0031: Honors Intro to Engineering
Brown University School of Engineering, Fall 2018
Selected Press Coverage
- Engineering’s Dwyer, Dastin-van Rijn and Kumar Selected as NSF Graduate Research Fellows
- Bawabe, Kumar, Sam, And Walke Receive CRA Outstanding Undergraduate Researcher Honors
- Nishanth Kumar named 2020 Barry M. Goldwater Scholar
- 2020 Barry Goldwater Scholars Include Many Impressive Indian American Researchers
- Undergrad Nishanth Kumar Wins Best Plenary Presentation At ILURS
- Bayazit, Galgana, Kumar, And Safranchik Win CRA Outstanding Undergraduate Researcher Honorable Mentions