Obligatory disclaimers: this is heavily geared to robotics/embodied AI-adjacent PhD students or new grads looking for research scientist and research engineer roles at tech companies. It is also heavily biased to my particular experience on the market between November 2025 and March 2026. Take it with an appropriately-sized grain-of-salt depending on how different your context is from mine.

Timelines

The rough process for signing an offer at a company in industry is this:

1. The pre-interview phase a. Reach out, network, and try to get the attention of as many places of interest. b. Prep interview materials and start seriously studying for your upcoming interviews
2. Go through interview processes at all the different choices: line up offers
3. Negotiate and sign an offer

Step 1 is often the longest part of the process because it’s the one that is least in your control (you never know how long it might take to get a particular company/recruiter/hiring manager’s attention; more advice on doing this below). Once you’re there, going through step 2 is much more in your control: you can often work with the company or recruiter to speed-up or slow-down processes given a valid reason (e.g. slowing down because you need more time to prepare properly, speeding up because you’re at more advanced stages already with other companies and anticipate deadlines).

As you’ve probably heard before, it tends to be most productive to be at stage 3 simultaneously with multiple different companies (this lets you know what all your options are, and makes it easier for companies to calibrate their own offers and pitches to you based on the other options you have). Given this, here’s the rough timeline I followed for my job search.

I took between 1 and 2 months for stage 1 (November and December), 2 months for stage 2 (January and February) and 3 ish weeks for stage 3 (in March). As a rule of thumb, I think it’s helpful to plan to start Stage 1 around 4-5 months before you intend to graduate, though I haven’t seen much downside in starting earlier.

In general, I’d recommend budgeting 1-2 months for prep, 1-2 months for actual interviews. I wouldn’t worry too much about the time taken for stage 3: that tends to vary significantly based on your situation and the specific companies you’re talking to.

Getting interviews

• Reach out and talk to as many people as possible
  • Don’t be too shy/think that a connection is too weak. One of the most exciting offers I ended up getting came from a Twitter DM for instance.
  • It’s also a good idea to do this to get a sense for interviews + what rough ballparks for salaries are (especially from friends who might be willing to talk about this).
  • In general, especially for competitive roles at well-known companies, I found that it was much much easier to get foot in the door with a referral/if you know someone.
• Protect your time: it’s surprisingly easy to find yourself having days and days where you just talk to recruiters and headhunters about fit. It’s good to book a lot of these in the beginning, but you probably want to taper off over time and only seriously pursue opportunitites you’re interested in.
  • I also found that scheduling a lot of these initial chats can start taking away from your time to do research/prep for other interviews. I found it useful to set aside 1-2 days a week in which I would schedule these chats (e.g. Mondays and Tuesdays) and leave all other days open to protect my time.
• Apply broadly: you very likely have more broad experience than you might think (e.g. I’ve worked with LLMs and VLMs, agents, CV, NLP, genai, etc. in the context of robotics). This often means you will be a surprisingly good fit for roles you didn’t expect to be a fit for (e.g. I ended up successfully recruiting for several LLM-posttraining roles despite predominantly having robotics experience). I found it very useful to explore my options broadly and learn about what different teams and orgs are working on or looking for. In general, I found that most companies and teams did not care too much that I hadn’t previously worked directly on or published in their area of interest (e.g. LLMs and RL for coding), but rather that I had some relevant experience (e.g. I’d worked with LLMs before and had done RL on them for one reason or another) and was willing to learn.

Interview Prep

In general, I’d recommend asking recruiters (or whoever you’re talking to at the company) to give you as much information as possible about what to expect from each interview. I found that there are several situations where recruiters are allowed to give out more information on a call, but not over email, so it is often useful to ask for a brief call to discuss interview information. In addition to the prep described below, I would almost always use an LLM to give me mock problems based on all the information I was given by a recruiter or hiring manager. I’d do as many of these problems as seemed relevant in an effort to practice as much as possible.

Numpy + Pytorch Coding Interviews (50% of all interviews I took)

The vast majority of interviews I had to take (across all roles I had to apply for) ended up being some form of ML or tensor manipulation problem in Numpy or PyTorch (Jax was also an option, but I never used it). These interviews tend to be hard because they require broad conceptual ML knowledge (e.g. remembering the k-means algorithm, knowing how to implement RoPE for an LLM) as well as intuition for tensor manipulation helpers and library functions in numpy/torch. Put simply: you need to have a lot of concepts and programming knowledge at your fingertips: most of these intervies are 45 mins or less and you simply don’t have time to spend time remembering how an algorithm works or re-discovering how basic numpy or torch operations work. I found personally (and heard from several friends) that these interviews deserve their own focused prep, even if you are an ML researcher/practitioner (because many of them will test concepts + tricks you won’t likely be encountering day-to-day).

So what exactly should you study, and how should you practice? I found it very useful to pick a concept and have an LLM (e.g. ChatGPT) generate me a stub file + some test cases simulating an interview. I’d then fill in the stub with a solution and ask the LLM to critique it. If I couldn’t implement it, I’d ask the LLM to give me hints/help me along. I’d repeat this process with spaced-repetition until I felt I could do all the problems I expected to encounter well within an interview time limit. I’ve also included a link for each section to an online course/resource that I used that might be helpful.

Based on the interviews I had, I’d recommend the following curriculum:

1. As Jenya says here: get really good at einsum and adopt Noam Shazeer’s shape suffixes for variable naming. These will come in extremely handy for everything that comes after.
2. Get very comfortable with tensor manipulation and writing vectorized code (e.g. how do you compute the distances between every pair of points in a (N x 3) array?)
   1. Resource: Sasha Rush’s Tensor Puzzlers - these are pretty illuminating to do; though you don’t need to do them all in a single line.
   2. K-means clustering
   3. K-NN classifier
3. Work on implementing various neural network layers from scratch (both forward and backward passes).
   1. Resource: CS 231N; I’d recommend doing all the assignments in particular.
   2. Logistic regression
   3. MLP with softmax
   4. MLP with MSE
   5. 2D convolution layer (for looping is okay for stride, but everything else should be vectorized)
   6. A simple ViT layer
4. Understand LLMs in-and-out (basically hw1 from this)
   1. Resource: CS 336; I’d recommend doing homeworks 1 and 2 thoroughly, and skimming/being familiar with the rest.
   2. Single-headed attention (forward and backward without autograd)
   3. Multi-headed attention (forward only should suffice)
   4. Byte-pair encoding (BPE) and how to write simple tokenizers from scratch
   5. Embedding layer
   6. A full transformer block as found in a modern LLM
   7. KV-caching
   8. (Challenge) simple LLM from scratch: start with some simple text and then implement all the necessary components (tokenization, embedding, transformer blocks, output projection, mapping back to vocab, kv-caching) from scratch in a single python file. See if you can implement everything with no hints/guidance in ~1 hour, and test that you can train and then run inference on the final model!

ML System Design Interviews (15% of all interviews I took)

These tended to be quite varied and dependent on the team I was interviewing for; the topic was often closely related to the team I was interviewing for (e.g. at self-driving car companies, I was asked to design some sub-components of an ML pipeline for driving vs. at home robotics startups, I was asked to design end-to-end manipulation systems). I don’t have too much preparation advice for these interviews aside from (1) knowing your own research very well, and (2) making sure to spend time catching up on state-of-the-art approaches that might be relevant to the team/role you’re interviewing for. In several of these interviews, I was asked to either explain how one of my own papers might be applied to solve a particular design question, or explain how a popular technique from the literature might apply. For example, one interview asked me something along the lines of: “say we want to build a system that can learn to fold laundry from human demonstrations — walk me through how you’d design the data collection pipeline, what policy architecture you’d use, and how you’d improve the success rate online.” This then naturally led into a discussion about tradeoffs between different policy representations (e.g. diffusion policy vs. action chunking with transformers), how much real-world data you’d actually need, and where sim-to-real transfer tends to break down for deformable objects. Similar to the ML/coding interviews above, I often practiced for these by asking LLMs to simulate example interviews and then critique my responses.

From the feedback I was given, my sense was that the main thing interviewers were looking for was a good high-level understanding of different approaches and their particular tradeoffs. For this reason, I think it’s useful to spend some time in the beginning structuring your answer, and also to make sure you start each section by clearly stating your high-level approach and reasoning before diving into the details. Interviewers want to see that you can think about systems at the right level of abstraction and make reasonable design decisions under uncertainty — not that you have every implementation detail memorized.

A few other things I found helpful:

• Start by clarifying the problem. Ask questions about the constraints: what’s the latency budget? How much data do we have? Is this online or offline? What are the failure modes we care most about? This both buys you thinking time and shows you know that design decisions depend heavily on context.
• Draw out a clear pipeline early. Sketching out (even with words) the major components and data flow at the start gives you and the interviewer a shared reference point. It also makes it much easier to zoom in on any particular component when they inevitably ask you to go deeper. The structure of overview-first then dive into details sworked quite well for me in most of these interviews.
• Know the landscape of approaches for your area. For instance, if you’re interviewing at a robotics company, you should be able to speak fluently about the tradeoffs between behavior cloning vs. RL, diffusion policies vs. autoregressive action prediction, end-to-end vs. modular pipelines, etc. You don’t need to have implemented all of these, but you should be able to articulate when you’d reach for one over the other and why.
• Be honest about what you don’t know. I found that interviewers responded well when I said something like “I haven’t worked with X directly, but my understanding is that it works by doing Y, and I’d want to validate that assumption by doing Z.” This came across much better than trying to bluff through a topic I wasn’t confident about.

Conceptual (verbal) interviews (15% of all interviews I took)

One of the downsides of AI and robotics being so broad is that the concepts you might be expected to know are also quite broad. Here is a (non-exhaustive) list of areas I had to know:

• General ML
  • Generative vs. discriminative models
  • Bias vs. Variance
  • Overfitting vs. Underfitting
  • L1 vs. L2 regularization; other forms of regularization
  • Common functions and knowing their gradients
    • Sigmoid
    • BCE
    • Softmax
    • NLL
    • Batchnorm
    • Layernorm
    • Groupnorm
    • Precision, Recall, F1
    • Logistic regression
    • K-means
    • KNN
    • Entropy
    • Cross-entropy
    • KL Divergence
• Reinforcement Learning
  • Policy gradient vs. value function methods
  • On-policy vs off-policy methods
  • Planning methods: Value Iteration and Policy Iteration (be able to write up and explain the the bellman backup)
  • Advantage and baselines
  • Q-learning (know the learning update; understand tabular vs. deep network case)
  • PPO (be able to write out and explain objective)
  • GRPO (and how is it bettter than PPO + where does it fail)
• Diffusion models, flow-matching (know updates equations, where to use each, time embedding, how to use in robotics via diffusion policy, etc.)
• Optimization
  • SGD
  • Momentum
  • RMSProp
  • Adam
  • AdamW
  • Learning rate scheduling approaches (cosine scheduling)
• Foundation models
  • Everything about LLMs
    • Tokenization, embedding layers, architecture design, MoE design, training tricks (the first couple lectures of CS 336 are very useful)
  • How to get to VLMs (conceptually) from LLMs
  • Quantization
• Robotics
  • Robot learning
    • VLAs
      • Popular VLA network architectures such as those used by Physical Intelligence and NVIDIA (in detail), using video model backbones
      • Action chunking and how it is used
    • Diffusion Policy and Flow Matching
    • Data collection + systems setup
    • sim2real RL and associated tooling
  • Controls
    • How does a simple PID controller work? (useful to know how to implement)
    • Impedance control (useful to know how to implement)
    • Operational-space control
  • SLAM
  • Classical robotics
    • FK and IK
    • Motion Planning, RRTs and PRMs
    • TAMP
  • Sensors, Vision and Data Collection
    • Some popular ways to collect data for robot policy learning (UMI, real-to-sim, VR)
    • Common sensors on a robot and what you can do with them (tradeoffs of depth/IR cameras vs. stereo vision, etc.)
    • Data collection and cleaning pipelines

Research interviews (10% of all interviews I took)

These interviews come in two flavors:

1. A ‘fit’ interview with the hiring manager where they ask you about your work and discuss your interest in the role
2. A full-fledged research presentation, followed by Q&A

Of these, the former was much more common than the latter. My biggest recommendation is to prepare several slide decks ranging from 5 - 20 mins long that provide increasingly detailed overviews of your PhD research. Before each fit chat, you can also add a slide at the end about your current interests + what kind of things you’d like to work at if you take the role you’re being considered for.

I don’t have much advice for research talks aside from recommending that you deliberately keep it short. Instead of presenting multiple projects, just give a broad overview of your research at the start and then zoom into one. Often, companies don’t care that much about the overall arc of your PhD (in the way that academia might), but rather more about your specific technical skills. They will ask you a lot of questions and really try to understand what you’re saying. I got the sense that it is much stronger to present one thing well and have your interviewers come away with the belief that you really understand it, over presenting multiple things at a high-level. Also, make sure to talk about how your work is relevant/useful to the role: running out of time before you get to this is common and unfortunate.

Behavioral interviews (10% of all interviews I took)

These are fairly straightforward: make sure to use the STAR method, and consider augmenting to STAR-L (add on a ‘learning’ at the end of your answer and talk about how that leadning has helped you since). I’d also recommend researching the interviewer + the company’s latest work. I had several interviews where I mentioned something about the interviewer/company’s work that was relevant to the conversation and I was told afterwards that this left them with a strongly positive impression about my genuine interest.

The mental game

Probably the hardest part of interviewing for me was playing the “mental game” of interviewing well: coping with the stress, anxiety, and comparison that comes part and parcel with the process. Here are some tips from my experience, in the hope that they will help you play the game better than I did.

Expect to get rejected — a lot. Companies receive far more applicants than they can possibly hire and interviewers will deliberately probe the edges of your knowledge. You will likely bomb an interview at at least one company, and walk out knowing you could have nailed it with one more week of prep. That sting is real. I was rejected after at least one interview from about 50% of the companies I seriously pursued, and colleagues told me that was a pretty good ratio given their own experiences. I think this speech from Roger Federer (my favorite tennis player!) captures a useful mindset: “When you’re playing a point, it has to be the most important thing in the world, and it is. But when it’s behind you, it’s behind you… The best in the world are not the best because they win every point. It’s because they know they’ll lose again and again, and have learned how to deal with it.”

Treat interviewing as a full-time job, and pace yourself accordingly. Between the interviews themselves and the reading, slide prep, and background research each one requires, it is very hard to seriously work on anything else. Ideally, by the time you start interviewing, you’d be mostly done with your research rather than pushing toward a first-author deadline. I made the mistake of scheduling interviews back-to-back with no breathing room, and burned out quickly. Build in recovery days, and make it a priority to protect time for things that have nothing to do with interviewing.

Don’t compare outcomes. If you’re a PhD student in AI or robotics right now, you almost certainly know people who landed eye-popping offers or unusually impressive titles. It is very hard not to use those as your benchmark. I spent more time than I’d like to admit refreshing compensation-sharing threads and mentally measuring my offers against numbers I’d heard secondhand from friends. This was not productive. The reality is that there is a lot of randomness in this process that has nothing to do with how good you are: companies have sudden hiring freezes, another candidate happens to know someone on the team, the role you were perfect for gets restructured the week before your final round. You can do everything right and still not get the outcome you wanted. Try to evaluate your offers on their own terms — does this role excite you, will you learn, does the idea of working with your teammates sound fun — rather than against someone else’s outcome that came from a completely different set of circumstances. And remember that careers are long: not landing your dream role this cycle doesn’t close the door. Perhaps this outcome was the right path for you at this time, and even if not, you can always try again in the future when your experience is deeper and circumstances line up better.

Focus on the conversations, not the verdicts. A mentor of mine gave me a helpful way of looking at interviewing in general: in every round, you get thirty to sixty minutes with a smart person who works in roughly your area and is genuinely curious about you and your work. When I stopped thinking of interviews as judgment and started treating them as conversations, I relaxed, asked better questions, and — ironically — performed better. Even the rounds that went poorly left me with something: a paper I hadn’t read, a new way of thinking about a problem, or just a clearer picture of what I know and don’t know. Treating interviews as just conversations/learning opportunities is certainly hard to keep up, especially when you know you’re being explicitly evaluated, but it helped take away quite a bit of my stress both during the interview itself. It also helped me keep my head up after several rejections.

That’s all I have. If any of this has been helpful, or if you have additional suggestions or thoughts you’d like to share about interviewing based on your experience, feel free to email me: I’d love to hear from you. Best of luck with your interviews!

Thanks to Sahit Chintalapudi, Will Shen, Aidan Curtis and Rohan Chitnis for reviewing early drafts of this post and providing helpful feedback.

Obligatory disclaimer: this is based entirely on my own experience. You ultimately know what’s best for you, so as with all advice, take mine with a grain of salt.

There are already a number of good articles on whether to do an AI PhD or not (see this, this, this, this, and this). These generally discuss the “pros” and “cons” of a PhD, and I won’t repeat them here. Instead, I want to share a different framing that I’ve found much more useful for actually making the decision.

Optimize for fun

One of the best pieces of career advice I’ve read is this essay by Michael Black. I strongly resonate with everything he says: making the optimal career decision at every step is not something that can be planned out. There are too many unknowns, too much randomness, and too many things that will change between now and the future you’re trying to optimize for. Given this, optimizing for “fun” turns out to be quite a good heuristic.

So my high-level advice is straightforward: go do a PhD if it seems like the most fun thing you can do over the long term, and don’t do it if not. I want to emphasize over the long term here. A PhD involves a lot of delayed gratification — months of failed experiments, rejected papers, and uncertainty about whether your ideas will pan out. It’s not always fun day-to-day. But if the overall arc of spending several years deeply exploring research questions that fascinate you sounds like the most exciting thing you could be doing with your time, that’s a very strong and useful signal.

The real question

But how do you actually know if a PhD will be the most fun decision? Here’s the problem: a good PhD is a lot of fun, but a bad PhD is very much not. The experience varies enormously depending on your advisor, your research group, your topic, and the specific environment you’d be working in. So whether a PhD is right for you depends on the options available to you, not the theoretical options that might be available to someone else.

In my opinion, the biggest issue for people making the PhD decision is that they think too much in hypotheticals. They spend a lot of time asking “should I do a PhD?” when the much more useful question is “should I do this specific PhD vs. the other concrete options I have at my disposal?” This is a much more grounded question, and it makes it significantly easier to figure out whether the PhD path is the most fun long-term choice or not.

What to actually do

Given all this, here’s my concrete advice: if you’re interested in AI research and considering a PhD, you should almost certainly apply to programs you’re excited about. At the same time, you should actively pursue research opportunities in industry — residency programs, research engineer roles, or anything else that would let you do the kind of work you’re interested in. If you’re not yet ready for these/not getting a lot of traction (and that’s completely fine!), you should perhaps focus on finding ways to become a stronger applicant (a completely separate question).

The key idea here is that applying broadly gives you concrete options to choose between, rather than forcing you to decide based on abstract pros-and-cons lists. Once you have actual offers in hand — a specific PhD program with a specific advisor vs. a specific industry role at a specific company — you can much more clearly evaluate which one sounds like the most fun over the long term. And if you have a better option in industry, go for it! The goal isn’t to do a PhD for the sake of doing a PhD; it’s to find the path that’s most exciting for you.

What about delayed gratification?

A natural response to all this is to ask whether a PhD is worth doing as a delayed gratification bargain. Maybe none of your PhD options seem particularly fun, and you have more immediately exciting options — a high-paying industry job, a startup opportunity — but you’re convinced you need a PhD for the long-term career you want.

This reasoning can be sound, but the details really matter. How much more exciting are the other options vs. the PhD? How sure are you that a PhD is actually necessary for where you want to go? How intrinsically excited are you about your research topic, and how much are you just chasing the credential?

A few things to consider:

• You likely can’t go wrong. If you have several options that seem roughly equally exciting, there isn’t really a “bad” one. It’s impossible to pick the “optimal” choice given everything you know now, so make a decision based on whatever factors feel right and trust that you’ll course-correct if needed.
• A good PhD is often what unlocks the opportunities you want. AI PhD programs are demanding: classes, quals, mentoring, teaching, funding, and good research on top of all of it. The post-PhD opportunities you’re most excited about — in academia or industry — will likely require a strong publication record and great references. That’s hard enough to achieve in a research area you love, with advisors you enjoy working with. If you’re going in with a “power through and survive” mindset, the delayed-gratification bet might not pay off the way you hope.
• It might be worth generating more options. Many programs allow deferred enrollment, and many advisors are open to industry collaborations or internships during the PhD. It’s worth exploring whether you can find a setup that gives you something closer to the best of both worlds.
• PhDs are long, and the landscape will change. When I started my PhD, GPT-2 had just come out and most people around me thought it was a cute demo. Robots struggled to walk stably or pick up objects in controlled lab settings. The vast majority of robotics jobs were in self-driving, which was supposedly on the verge of revolutionizing transportation. The academic AI job market looked steady. Five years later, AI jobs are dominated by LLMs, self-driving hasn’t delivered nearly what was promised, and federal funding cuts have reshaped academia. Things have worked out well for AI PhDs overall, but this was far from guaranteed at the time. So I’d recommend against doing a PhD if the primary reason is some specific post-grad opportunity — you don’t know if it’ll still exist, or if you’ll still want it (lots of people’s career aspirations change during a PhD), by the time you’re done.
• Ask yourself whether you’d be happy with a random recent alum’s outcome. Look at where the last few students out of the lab or program ended up — their trajectories are the strongest signal you have for what yours could look like. Would you be proud if your own PhD turned out like one of theirs?

My experience

To make all this concrete, here’s my own story. When I was finishing up my undergrad, I was extermely conflicted about whether I should go to industry or pursue PhD programs. I knew I wanted to work on AI — specifically AI/ML for robotics — but wasn’t sure what the best way to pursue this interest would be. I was advised by several researchers I looked up to that, given my undergrad research experience, I should apply directly to relevant roles in industry, especially to AI residency programs. I applied to any and all I could find and didn’t get interviewed for any relevant full-time roles. The best I was able to do were to land two research internships at startups that my undergrad advisors had connections at.

At the same time, though, I had applied to several PhD programs that I was excited about, and ended up getting into a number of them. I ultimately chose my advisors at MIT because it seemed like the most fun option available to me. It also had the added benefit of seeming like a great long-term career option. Frankly, I felt I’d be crazy to turn it down.

Had I gotten into AI residency programs, or landed a full-time research engineer or research scientist role, I might have gone straight into industry instead! But the point is that option was simply not available to me, even though it existed for other people. The hypothetical of “should I do a PhD or go into industry research?” was not the right question for me, because I didn’t actually have the industry research option. The real question was: “given the specific options I have right now, which one sounds the most fun?” And for me, the answer was clearly the PhD I chose.

While my PhD overall had several challenges and periods of significant struggle, it did turn out to be extremely fun overall (as I had originally hoped!). I got to work on interesting problems with people I like and admire. I got to spend time to understand a lot of concepts I only knew at a surface level in undergrad. I also got more clarity on whether I want a career in academia or industry over the long-term. At the end of my PhD, I was able to successfully recruit for several jobs in industry that I could only have dreamed of as an undergrad. In hindsight, choosing to do a PhD was probably the best decision I’ve made.

I think this framing — ground the decision in your actual options, and optimize for long-term fun — makes what can feel like an overwhelming life decision into something much more manageable. You don’t need to figure out the globally optimal career path. You just need to explore your options, and then pick the one that concretely excites you the most.

If any of this has been helpful, or if you’d like to share your own experience with making this decision, feel free to reach out — I’d love to hear from you.

Thanks to Josh Roy, and Tom Silver for reviewing early drafts of this post and providing helpful feedback.

Of course, some version of this question has been on researchers’ minds for a few years now¹. However, in the aftermath of the unprecedented success of ChatGPT and other large-scale ‘foundation models’ on tasks that were thought to be unsolvable just a few years ago, the question was especially topical at this year’s CoRL. Developing a general-purpose robot, one that can competently and robustly execute a wide variety of tasks of interest in any home or office environment that humans can, has been perhaps the holy grail of robotics since the inception of the field. And given the recent progress of foundation models, it seems possible that scaling existing network architectures by training them on very large datasets might actually be the key to that grail.

Given how timely and significant this debate seems to be, I thought it might be useful to write a post centered around it. My main goal here is to try to present the different sides of the argument as I heard them, without bias towards any side. Almost all the content is taken directly from talks I attended or conversations I had with fellow attendees. My hope is that this serves to deepen people’s understanding around the debate, and maybe even inspire future research ideas and directions. If you read this and feel that I’ve misrepresented any of the ideas of positions below, then do write to me or post a clarifying comment below.

With all that established, let’s dive in!

I want to start by presenting the main arguments I heard in favor of scaling as a solution to robotics.

Why scaling might work

• It worked for Computer Vision (CV) and Natural Language Processing (NLP), so why not robotics?: This was perhaps the most common argument I heard, and the one that seemed to excite most people given recent models like GPT4-V and SAM. The point here is that training a large model on an extremely large corpus of data has recently led to astounding progress on problems thought to be intractable just 3-4 years ago. Moreover, doing this has led to a number of emergent capabilities, where trained models are able to perform well at a number of tasks they weren’t explicitly trained for. Importantly, the fundamental method here of training a large model on a very large amount of data is general and not somehow unique to CV or NLP. Thus, there seems to be no reason why we shouldn’t observe the same incredible performance on robotics tasks.
  • We’re already starting to see some evidence that this might work well: Chelsea Finn, Vincent Vanhoucke and several others pointed to the recent RT-X and RT-2 papers from Google DeepMind as evidence that training a single model on large amounts of robotics data yields promising generalization capabilities. Russ Tedrake of Toyota Research Institute (TRI) and MIT pointed to the recent Diffusion Policies paper as showing a similar surprising capability. Sergey Levine of UC Berkeley highlighted recent efforts and successes from his group in building and deploying a robot-agnostic foundation model for navigation. All of these works are somewhat preliminary in that they train a relatively small model with a paltry amount of data compared to something like GPT4-V, but they certainly do seem to point to the fact that scaling up these models and datasets could yield impressive results in robotics.
• Progress in data, compute, and foundation models are waves that we should ride: This argument is closely related to the above one, but distinct enough that I think it deserves to be discussed separately. The main idea here comes from Rich Sutton’s influential essay: the history of AI research has shown that relatively simple algorithms that scale well with data always outperform more complex/clever algorithms that do not. A nice analogy from Karol Hausman’s early career keynote is that improvements to data and compute are like a wave that is bound to happen given the progress and adoption of technology. Whether we like it or not, there will be more data and better compute. As AI researchers, we can either choose to ride this wave, or we can ignore it. Riding this wave means recognizing all the progress that’s happened because of large data and large models, and then developing algorithms, tools, datasets, etc. to take advantage of this progress. It also means leveraging large pre-trained models from vision and language that currently exist or will exist for robotics tasks.
• Robotics tasks of interest lie on a relatively simple manifold, and training a large model will help us find it: This was something rather interesting that Russ Tedrake pointed out during a debate in the workshop on robustly deploying learning-based solutions. The manifold hypothesis as applied to robotics roughly states that, while the space of possible tasks we could conceive of having a robot do is impossibly large and complex, the tasks that actually occur practically in our world lie on some much lower-dimensional and simpler manifold of this space. By training a single model on large amounts of data, we might be able to discover this manifold. If we believe that such a manifold exists for robotics - which certainly seems intuitive - then this line of thinking would suggest that robotics is not somehow different from CV or NLP in any fundamental way. The same recipe that worked for CV and NLP should be able to discover the manifold for robotics and yield a shockingly competent generalist robot. Even if this doesn’t exactly happen, Tedrake points out that attempting to train a large model for general robotics tasks could teach us important things about the manifold of robotics tasks, and perhaps we can leverage this understanding to solve robotics.
• Large models are the best approach we have to get at ‘common sense’ capabilities, which pervade all of robotics: Another thing Russ Tedrake pointed out is that “common sense” pervades almost every robotics task of interest. Consider the task of having a mobile manipulation robot place a mug onto a table. Even if we ignore the challenging problems of finding and localizing the mug, there are a surprising number of subtleties to this problem. What if the table is cluttered and the robot has to move other objects out of the way? What if the mug accidentally falls on the floor and the robot has to pick it up again, re-orient it, and place it on the table? And what if the mug has something in it, so it’s important it’s never overturned? These ‘edge cases’ are actually much more common that it might seem, and often are the difference between success and failure for a task. Moreover, these seem to require some sort of ‘common sense’ reasoning to deal with. Several people argued that large models trained on a large amount of data are the best way we know of to yield some aspects of this ‘common sense’ capability. Thus, they might be the best way we know of to solve general robotics tasks.

As you might imagine, there were a number of arguments against scaling as a practical solution to robotics. Interestingly, almost no one directly disputes that this approach could work in theory. Instead, most arguments fall into one of two buckets: (1) arguing that this approach is simply impractical, and (2) arguing that even if it does kind of work, it won’t really ‘solve’ robotics.

Why scaling might not work

It’s impractical

• We currently just don’t have much robotics data, and there’s no clear way we’ll get it: This is the elephant in pretty much every large-scale robot learning room. The Internet is chock full of data for CV and NLP, but not at all for robotics. Recent efforts to collect very large datasets have required tremendous amounts of time, money, and cooperation, yet have yielded a very small fraction of the amount of vision and text data on the Internet. CV and NLP got so much data because they had an incredible ‘data flywheel’: tens of millions of people connecting to and using the Internet. Unfortunately for robotics, there seems to be no reason why people would upload a bunch of sensory input and corresponding action pairs. Collecting a very large robotics dataset seems quite hard, and given that we know that a lot of important ‘emergent’ properties only showed up in vision and language models at scale, the inability to get a large dataset could render this scaling approach hopeless.
• Robots have different embodiments: Another challenge with collecting a very large robotics dataset is that robots come in a large variety of different shapes, sizes and form factors. The output control actions that are sent to a Boston Dynamics Spot robot are very different to those sent to a KUKA iiwa arm. Even if we ignore the problem of finding some kind of common output space for a large trained model, the variety in robot embodiments means we’ll probably have to collect data from each robot type, and that makes the above data-collection problem even harder.
• There is extremely large variance in the environments we want robots to operate in: For a robot to really be ‘general purpose’, it must be able to operate in any practical environment a human might want to put it in. This means operating in any possible home, factory, or office building it might find itself in. Collecting a dataset that has even just one example of every possible building seems impractical. Of course, the hope is that we would only need to collect data in a small fraction of these, and the rest will be handled by generalization. However, we don’t know how much data will be required for this generalization capability to kick in, and it very well could also be impractically large.
• Training a model on such a large robotics dataset might be too expensive/energy-intensive: It’s no secret that training large foundation models is expensive, both in terms of money and in energy consumption. GPT-4V - OpenAI’s biggest foundation model at the time of this writing - reportedly cost over $100 million and 50 million KWh of electricity to train. This is well beyond the budget and resources that any academic lab can currently spare, so a larger robotics foundation model would need to be trained by a company or a government of some kind. Additionally, depending on how large both the dataset and model itself for such an endeavor are, the costs may balloon by another order-of-magnitude or more, which might make it completely infeasible.

Even if it works as well as in CV/NLP, it won’t solve robotics

• The 99.X problem and long tails: Vincent Vanhoucke of Google Robotics started a talk with a provocative assertion: most - if not all - robot learning approaches cannot be deployed for any practical task. The reason? Real-world industrial and home applications typically require 99.X% or higher accuracy and reliability. What exactly that means varies by application, but it’s safe to say that robot learning algorithms aren’t there yet. Most results presented in academic papers top out at 80% success rate. While that might seem quite close to the 99.X% threshold, people trying to actually deploy these algorithms have found that it isn’t so: getting higher success rates requires asymptotically more effort as we get closer to 100%. That means going from 85% to 90% might require just as much - if not more - effort than going from 40% to 80%. Vincent asserted in his talk that getting up to 99.X% is a fundamentally different beast than getting even up to 80%, one that might require a whole host of new techniques beyond just scaling.
  • Existing big models don’t get to 99.X% even in CV and NLP: As impressive and capable as current large models like GPT-4V and DETIC are, even they don’t achieve 99.X% or higher success rate on previously-unseen tasks. Current robotics models are very far from this level of performance, and I think it’s safe to say that the entire robot learning community would be thrilled to have a general model that does as well on robotics tasks as GPT-4V does on NLP tasks. However, even if we had something like this, it wouldn’t be at 99.X%, and it’s not clear that it’s possible to get there by scaling either.
• Self-driving car companies have tried this approach, and it doesn’t fully work (yet): This is closely related to the above point, but important and subtle enough that I think it deserves to stand on its own. A number of self-driving car companies - most notably Tesla and Wayve - have tried training such an end-to-end big model on large amounts of data to achieve Level 5 autonomy. Not only do these companies have the engineering resources and money to train such models, but they also have the data. Tesla in particular has a fleet of over 100,000 cars deployed in the real world that it is constantly collecting and then annotating data from. These cars are being teleoperated by experts, making the data ideal for large-scale supervised learning. And despite all this, Tesla has so far been unable to produce a Level 5 autonomous driving system. That’s not to say their approach doesn’t work at all. It competently handles a large number of situations - especially highway driving - and serves as a useful Level 2 (i.e., driver assist) system. However, it’s far from 99.X% performance. Moreover, data seems to suggest that Tesla’s approach is faring far worse than Waymo or Cruise, which both use much more modular systems. While it isn’t inconceivable that Tesla’s approach could end up catching up and surpassing its competitors performance in a year or so, the fact that it hasn’t worked yet should serve as evidence perhaps that the 99.X% problem is hard to overcome for a large-scale ML approach. Moreover, given that self-driving is a special case of general robotics, Tesla’s case should give us reason to doubt the large-scale model approach as a full solution to robotics, especially in the medium term.
• Many robotics tasks of interest are quite long-horizon: Accomplishing any task requires taking a number of correct actions in sequence. Consider the relatively simple problem of making a cup of tea given an electric kettle, water, a box of tea bags, and a mug. Success requires pouring the water into the kettle, turning it on, then pouring the hot water into the mug, and placing a tea-bag inside it. If we want to solve this with a model trained to output motor torque commands given pixels as input, we’ll need to send torque commands to all 7 motors at around 40 Hz. Let’s suppose that this tea-making task requires 5 minutes. That requires 7 * 40 * 60 * 5 = 84000 correct torque commands. This is all just for a stationary robot arm; things get much more complicated if the robot is mobile, or has more than one arm. It is well-known that error tends to compound with longer-horizons for most tasks. This is one reason why - despite their ability to produce long sequences of text - even LLMs cannot yet produce completely coherent novels or long stories: small deviations from a true prediction over time tend to add up and yield extremely large deviations over long-horizons. Given that most, if not all robotics tasks of interest require sending at least thousands, if not hundreds of thousands, of torques in just the right order, even a fairly well-performing model might really struggle to fully solve these robotics tasks.

Okay, now that we’ve sketched out all the main points on both sides of the debate, I want to spend some time diving into a few related points. Many of these are responses to the above points on the ‘against’ side, and some of them are proposals for directions to explore to help overcome the issues raised.

Misc. Related Arguments

We can probably deploy learning-based approaches robustly

One point that gets brought up a lot against learning-based approaches is the lack of theoretical guarantees. At the time of this writing, we know very little about neural network theory: we don’t really know why they learn well, and more importantly, we don’t have any guarantees on what values they will output in different situations. On the other hand, most classical control and planning approaches that are widely used in robotics have various theoretical guarantees built-in. These are generally quite useful when certifying that systems are safe.

However, there seemed to be general consensus amongst a number of CoRL speakers that this point is perhaps given more significance than it should. Sergey Levine pointed out that most of the guarantees from controls aren’t really that useful for a number of real-world tasks we’re interested in. As he put it: “self-driving car companies aren’t worried about controlling the car to drive in a straight line, but rather about a situation in which someone paints a sky onto the back of a truck and drives in front of the car”, thereby confusing the perception system. Moreover, Scott Kuindersma of Boston Dynamics talked about how they’re deploying RL-based controllers on their robots in production, and are able to get the confidence and guarantees they need via rigorous simulation and real-world testing. Overall, I got the sense that while people feel that guarantees are important, and encouraged researchers to keep trying to study them, they don’t think that the lack of guarantees for learning-based systems means that they cannot be deployed robustly.

What if we strive to deploy Human-in-the-Loop systems?

In one of the organized debates, Emo Todorov pointed out that existing successful ML systems, like Codex and ChatGPT, work well only because a human interacts with and sanitizes their output. Consider the case of coding with Codex: it isn’t intended to directly produce runnable, bug-free code, but rather to act as an intelligent autocomplete for programmers, thereby making the overall human-machine team more productive than either alone. In this way, these models don’t have to achieve the 99.X% performance threshold, because a human can help correct any issues during deployment. As Emo put it: “humans are forgiving, physics is not”.

Chelsea Finn responded to this by largely agreeing with Emo. She strongly agreed that all successfully-deployed and useful ML systems have humans in the loop, and so this is likely the setting that deployed robot learning systems will need to operate in as well. Of course, having a human operate in the loop with a robot isn’t as straightforward as in other domains, since having a human and robot inhabit the same space introduces potential safety hazards. However, it’s a useful setting to think about, especially if it can help address issues brought on by the 99.X% problem.

Maybe we don’t need to collect that much real world data for scaling

A number of people at the conference were thinking about creative ways to overcome the real-world data bottleneck without actually collecting more real world data. Quite a few of these people argued that fast, realistic simulators could be vital here, and there were a number of works that explored creative ways to train robot policies in simulation and then transfer them to the real world. Another set of people argued that we can leverage existing vision, language, and video data and then just ‘sprinkle in’ some robotics data. Google’s recent RT-2 model showed how taking a large model trained on internet scale vision and language data, and then just fine-tuning it on a much smaller set robotics data can produce impressive performance on robotics tasks. Perhaps through a combination of simulation and pretraining on general vision and language data, we won’t actually have to collect too much real-world robotics data to get scaling to work well for robotics tasks.

Maybe combining classical and learning-based approaches can give us the best of both worlds

As with any debate, there were quite a few people advocating the middle path. Scott Kuindersma of Boston Dynamics titled one of his talks “Let’s all just be friends: model-based control helps learning (and vice versa)”. Throughout his talk, and the subsequent debates, his strong belief that in the short to medium term, the best path towards reliable real-world systems involves combining learning with classical approaches. In her keynote speech for the conference, Andrea Thomaz talked about how such a hybrid system - using learning for perception and a few skills, and classical SLAM and path-planning for the rest - is what powers a real-world robot that’s deployed in tens of hospital systems in Texas (and growing!). Several papers explored how classical controls and planning, together with learning-based approaches can enable much more capability than any system on its own. Overall, most people seemed to argue that this ‘middle path’ is extremely promising, especially in the short to medium term, but perhaps in the long-term either pure learning or an entirely different set of approaches might be best.

What can/should we take away from all this?

If you’ve read this far, chances are that you’re interested in some set of takeaways/conclusions. Perhaps you’re thinking “this is all very interesting, but what does all this mean for what we as a community should do? What research problems should I try to tackle?”. Fortunately for you, there seemed to be a number of interesting suggestions that had some consensus on this.

We should pursue the direction of trying to just scale up learning with very large datasets

Despite the various arguments against scaling solving robotics outright, most people seem to agree that scaling in robot learning is a promising direction to be investigated. Even if it doesn’t fully solve robotics, it could lead to a significant amount of progress on a number of hard problems we’ve been stuck on for a while. Additionally, as Russ Tedrake pointed out, pursuing this direction carefully could yield useful insights about the general robotics problem, as well as current learning algorithms and why they work so well.

We should also pursue other existing directions

Even the most vocal proponents of the scaling approach were clear that they don’t think everyone should be working on this. It’s likely a bad idea for the entire robot learning community to put its eggs in the same basket, especially given all the reasons to believe scaling won’t fully solve robotics. Classical robotics techniques have gotten us quite far, and led to many successful and reliable deployments: pushing forward on them or integrating them with learning techniques might be the right way forward, especially in the short to medium terms.

We should focus more on real-world mobile manipulation and easy-to-use systems

Vincent Vanhoucke made an observation that most papers at CoRL this year were limited to tabletop manipulation settings. While there are plenty of hard tabletop problems, things generally get a lot more complicated when the robot - and consequently its camera view - moves. Vincent speculated that it’s easy for the community to fall into a local minimum where we make a lot of progress that’s specific to the tabletop setting and therefore not generalizable. A similar thing could happen if we work predominantly in simulation. Avoiding these local minima by working on real-world mobile manipulation seems like a good idea.

Separately, Sergey Levine observed that a big reason why LLM’s have seen so much excitement and adoption is because they’r extremely easy to use: especially by non-experts. One doesn’t have to know about the details of training an LLM, or perform any tough setup, to prompt and use these models for their own tasks. Most robot learning approaches are currently far from this. They often require significant knowledge of their inner workings to use, and involve very significant amouns of setup. Perhaps thinking more about how to make robot learning systems easier to use and widely applicable could help improve adoption and potentially scalability of these approaches.

We should be more forthright about things that don’t work

There seemed to be a broadly-held complaint that many robot learning approaches don’t adequately report negative results, and this leads to a lot of unnecessary repeated effort. Additionally, perhaps patterns might emerge from consistent failures of things that we expect to work but don’t actually work well, and this could yield novel insight into learning algorithms. There is currently no good incentive for researchers to report such negative results in papers, but most people seemed to be in favor of designing one.

We should try to do something totally new

There were a few people who pointed out that all current approaches - be they learning-based or classical - are unsatisfying in a number of ways. There seem to be a number of drawbacks with each of them, and it’s very conceivable that there is a completely different set of approaches that ultimately solves robotics. Given this, it seems useful to try think outside the box. After all, every one of the current approaches that’s part of the debate was only made possible because the few researchers that introduced them dared to think against the popular grain of their times.

Acknowledgements: Huge thanks to Tom Silver and Leslie Kaelbling for providing helpful comments, suggestions, and encouragement on a previous draft of this post.

Before I dive into describing all the packages and tools I used, I want to mention 2 things:

1. This article assumes you’re planning to use GitHub to host your project’s repository. A lot of the tools used should integrate well with some other version control setup, but might not because it wasn’t written with any other setup in mind.
2. There are a lot of different tools and libraries covered in this post, and quite a few of them might be overkill for your needs. Feel free to pick and choose specific pieces based on your use-case!

Alright, let’s get started!

Dependency Management and Packaging with poetry

Almost every useful Python package relies on a plethora of other, existing Python packages. While there are many popular tools like pip and conda that help manage and maintain all the various packages and dependencies you might need to prevent the dreaded ImportError, most of these are intended for personal use and don’t seamlessly support all the needs that arise when building an open-source package that will (hopefully) be used and contributed to by hundreds of people. This is where poetry comes in.

Poetry is an easy-to-install tool that seamlessly handles pretty much all the use-cases you might have with developing an open-source package. It also has an intuitive CLI and some nice bells and whistles (one of my favorite: you can specify separate dependencies for developers and users and use these to make separate virtualenvs!). Head over to this page and follow the instructions to install and setup poetry. Then, look under the “Project Setup” heading at this page to generate a pyproject.toml file which poetry treats as a one-stop-shop to manage your project. You can read this page to understand all the basic usecases, and this page to see everything you can do with the CLI, but the general idea of how to use poetry is as follows:

• Activate your poetry environment for development with poetry shell
• Whenever you want to add a new dependency/import a new library as part of development, do poetry add -D <library-name>. If this will be needed for project users (not just developers) as well, simply do poetry add <library-name> (you can also optionally specify particular versions of requirements, and even the source from which to get a specific library).
• If someone else wants to start developing and contributing to your project, they can simply download the project repository and run poetry shell followed by poetry install, and they should have a nice, clean virtualenv with everything needed to run and develop the code (ta-da!).
  • If someone just wants to use your package and isn’t interested in developing, they can run poetry shell followed by poetry install --no-dev.

Publishing your poetry package to PyPI

One of Poetry’s nicest features is its ability to easily publish packages to PyPI so users can install the package painlessly with a simple pip install <package_name>. Before actually publishing your package though, it’s probably a good idea to test that it builds correctly and will upload to PyPI seamlessly.

Testing your package build and upload with Test PyPI

1. If you haven’t already, add some miscellaneous, relevant fields to your pyproject.toml file. Such nice-to-have fields might include a license, a list of package authors and maintainers, a link to a homepage website, etc. You can see the full list of possible fields here; and as a sample, PGMax’s pyproject.toml file can be found here.
2. Also, if you haven’t already, make an account on Test PyPI. Note that this will be entirely separate from any PyPI account you may have registered for with the same username.
3. Go ahead and build your package with the command: poetry build. This should create a dist folder with a .whl and .tar.gz file. Note that it’s probably a good idea to add this dist folder to your .gitignore.
4. Try to install this .whl file in a fresh environment (e.g. create a new Conda environment or venv) with the command: pip install dist/<name_of_.whl_file>. If there are any dependency/other issues with this install, fix them now.
   1. If your package has any tests/example code, try to run these to verify that the install really worked.
5. Add Test PyPI as an alternate package repository to publish to: poetry config repositories.testpypi https://test.pypi.org/legacy/
6. Now, publish your package to Test PyPI: poetry publish -r testpypi. You will need your account credentials for this. If there are any dependency errors, etc. with this resolve them now.
7. To verify that the upload looks good and works as intended, you can view it on Test PyPI and then additionally try to install it in a fresh environment with the command: pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple <your_package_name> (see this StackOverflow answer if you’re curious about what this command does.)
   1. Again, run any test/example code to verify that this install is all good.

If this works, then congrats! You’re now ready to upload your package to PyPI and can be assured that everything will work seamlessly!

Uploading your package to PyPI

1. If you haven’t already, then register for a PyPI account. Note that this is entirely separate from any account you may have on Test PyPI.
2. Publish your package with a simple poetry publish command. You will need your PyPI account credentials.

Ta-da! Now you have a shiny new package on PyPI, ready for anyone to install with a simple pip install <package-name> command!

Code Formatting, Linting and other nice-to-haves

While these things are relatively minor, they can save you (and people who want to use your project) from significant headaches in the future. Since these things are straightforward to setup and become increasingly important as your project grows, I recommend setting them up right at the outset. Specifically, I recommend using:

• black for code formatting
• isort to sort imports and make sure they’re always in the same order (this is extremely useful when reviewing changes on GitHub)
• flake8 for linting
• mypy for type-checking all your code. This might be somewhat annoying during development, but is useful to keep-around because it can help catch some extremely subtle type bugs (plus, you can make it ignore files/functions/lines you don’t want it to look at).

Enforcing Code Formatting with pre-commit hooks

Now, it can be somewhat painful to install and use all these formatting and type-checking tools together. Fortunately, the pre-commit tool can bundle and run all of these tools (and more) for you! What’s more, you can set this up to run whenever a git commit command is executed. To do this, simply:

1. Install pre-commit by following these instructions
   1. If you’re using poetry (which I hope you are!) be sure to add pre-commit as a developer dependency and do this installation (preferably via pip) within your poetry shell. You can accomplish all of this by running poetry add -D pre-commit after having activated your poetry env with poetry shell.
   2. Verify your installation by checking whether the pre-commit --version command runs
2. Add a .pre-commit-config.yaml file to specify the tools you want pre-commit to run
   1. You can have the terminal spit out a sample .pre-commit-config.yaml using the command pre-commit sample-config. Copy pase the output of this command into .pre-commit-config.yaml

   2. If you want you’d like to use all the nice tools specified above (black, isort, flake8 and mypy), copy paste the below into your .pre-commit-config.yaml file

       repos:
       -   repo: https://github.com/PyCQA/isort
           rev: 5.8.0
           hooks:
           -   id: isort
      
       -   repo: https://github.com/ambv/black
           rev:  21.6b0
           hooks:
           -   id: black
               language_version: python3.7
      
       -   repo: https://github.com/pre-commit/pre-commit-hooks
           rev: v4.0.1
           hooks:
           -   id: check-added-large-files
               args: ['--maxkb=5120']
           -   id: trailing-whitespace
               files: (\.py|\.rst|\.yaml|)$
           -   id: check-merge-conflict
      
       -   repo: https://gitlab.com/pycqa/flake8
           rev: 3.9.2
           hooks:
           -   id: flake8
               # To turn this into an error, remove --exit-zero below
               args: ['--config', '.flake8.config','--exit-zero']
               verbose: true
      
       -   repo: https://github.com/pre-commit/mirrors-mypy
           rev: 'v0.902'  # Use the sha / tag you want to point at
           hooks:
           -   id: mypy
               additional_dependencies: [tokenize-rt==3.2.0]
      

   3. Setup pre-commit to run whenever you commit to GitHub by running the pre-commit install command
      1. If you’re using poetry, be sure to run this command within your poetry shell. Alternatively, you can run poetry run pre-commit install.
   4. (Optional) Integrate pre-commit with your IDE so you can run these checks for any file on save, etc. (VSCode extension, IntelliJ Plugin)

      1. You can also just run the command pre-commit run --files <path-to-your-files> to selectively run these tools on specific files. If you’re like me and don’t enjoy typing all this out, you can make a bash alias by adding the below lines to the end of your ~/.bashrc file.

          pc ()
          {
              pre-commit run --files "$1"
          }
         

         Now, you can simply run pc <path-to-yourfile>.

Managing Jupyter Notebooks with jupytext

Jupyter Notebooks are better than plain old Python scripts for a variety of different purposes. Unfortunately, version control isn’t one of them. Keeping Jupyter Notebooks as part of your project can make code review and management an absolute nightmare. Fortunately, there is a solution: Jupytext.

Jupytext essentially creates a text version of your notebook and updates this version whenever you update the notebook (or vice-versa, so you can edit your notebook just by editing the jupytext-produced text file!). This way, you can just commit these text files (that are amenable to version control) to your repo and then generate + edit the corresponding Jupyter notebooks if and when they’re needed. Here’s how to integrate Jupytext into your repo:

1. Add *.ipynb to your .gitignore file within your GitHub repo
2. Install Jupytext by following the instructions from here
   1. If you’re using poetry (which I hope you are!) be sure to add jupytext as a developer dependency and do this installation (preferably via pip) within your poetry shell!
3. Pair all your notebooks with a file in text format by following the instructions here.

And that’s it! Be sure to commit the paired text files to GitHub’s version control.

Testing with pytest

Having a good test suite is absolutely essential for any open-source library that expects to be widely adopted and improve with time. As of this writing, pytest is arguably the most popular choice for a python testing library because of its ease-of-use coupled with exhaustive features.

Installing and setting up PyTest is super easy: simmply follow the steps from the official documentation. If you’re using Poetry (which I hope you are!), make pytest a dependency of your project and install it within your poetry environment by running poetry add pytest within your poetry shell.

You’re now ready to begin writing tests! To get started, follow the guide from the official documentation.

As you’re thinking about how to structure and organize your tests, I’d highly recommend checking out this set of best practices. Ensuring that your tests follow as many of these pointers as is reasonable will save you a lot of headache down the line.

Generating documentation with Sphinx

If you’ve ever experienced the abject horror of having to make changes to a sprawling codebase with no comments or documentation after when everyone that’s worked on it is unavailable, you understand why documentation is important. Now of course, there’s a reason most people don’t like writing documentation: it’s an annoying overhead (especially when you’re just starting to work on your project). Fortunately, there are tools to make writing and maintaining documentation significantly less annoying (and even quite a delight). Enter Sphinx.

Sphinx helps you turn documentation files into a beautiful website that makes it easy for users to navigate through your (potentially massive and sprawling) codebase and understand how different pieces work. What’s more, Read the Docs hosts this website (generally for FREE!) and can be configured to update it whenever you push changes to your repo on GitHub. This way, your documentation is always up-to-date, and accessible to anyone on the Internet (which makes it much more likely that people will want to use and help improve your library!).

Unfortunately, as of this writing, Sphinx can be a bit annoying and tricky to setup well, especially if you’d like to do non-default things to make your documentation look better. There are a lot of different ways to configure Sphinx to build a great-looking website; I’ll describe the one I settled on for PGMax, which was inspired by the structure and setup of the JAX documentation. In particular, this setup generates a separate page in the documentation for every function, class and exception in your codebase (which looks much nicer than cramming everything into a few pages).

Writing DocStrings

For Sphinx to actually do it’s magic, your code needs to have ‘docstrings’. If you haven’t heard of these before, the name is fairly self-explanatory: they’re strings placed throughout the codebase that document everything from a general description of each class to the input/output of a specific function. Docstrings have a specific structure and style to make them easily parseable, and there are 2 major styles to choose from: the Google Style and the NumPy Style. PGMax chose to go the Google route, but there’s no significant reason to choose one over the other. Whatever you do choose, make sure to write docstrings for all applicable parts of your code!

Configuring Sphinx

To get started, follow these instructions to install Sphinx and walk through a setup process to create a documentation folder for you project. PGMax uses the readthedocs theme to format documentation, so I’d certainly recommend installing this as well. Once you’ve completed this, follow this StackOverflow answer to setup your repo to use the autosummary directive correctly (which generates separate pages for each module in your documentation). Feel free to look at PGMax’s documentation folder for reference. If all is now setup correctly, then if you navigate to your project’s documentation folder and run make html command, you should see a build directory get created containing html pages with your documentation. If you open these pages with your browser, you should be able to view what your documentation pages look like! Be sure to click through most if not all of you pages to make sure everything has been generated correctly.

Setting up Read the Docs with GitHub Integration

Now that you have Sphinx building your documentation correctly locally, you can make your own website on readthedocs.io to host and continuously update your documentation so any users can easily access it. To do this, simply follow these instructions (though adapted to your project instead of their tutorial example). Importantly, this integration should automatically update the documentation upon new pull-requests, so all you need to do whenever you update code and documentation is simply make a PR!

Automating things with Continuous Integration via GitHub Actions

If you’ve setup everything up till this point, then you should have a nice development environment and workflow ready-to-go. This is great, but there’s one subtle problem: what if you (or someone contributing to your open-source project) mess up? Suppose for example that you forget to run your pre-commit hooks before pushing to GitHub, or even worse, you forget to run all your tests with PyTest before a major change. There’s currently nothing that prevents you from doing any of these things, and while some of them (like forgetting to lint) might not matter as much, other slip-ups (like forgetting to test and thereby accidentally breaking things) could be extremely annoying and even costly to fix. Enter GitHub Actions.

GitHub Actions essentially allow you to do various things (run tests, lint, publish the package, etc.) upon “triggers” like committing, pushing or opening a Pull Request (PR) on GitHub. There are a lot of useful, interesting and awesome GitHub Actions out there, but I’m going to try to keep things simple and focus on the ones that dovetail well with the tools described above. Specifically, whenever someone pushes a commit, an action should enforce all our pre-commit hooks on any changed code. And whenever someone opens a PR, various actions should:

• Enforce all pre-commit hooks
• Try to install all the Poetry dependencies and make sure it works
• Run all tests with PyTest
• Test that all documentation can be generated, then publish this documentation after PR approval

Additionally, after a new “release” of the repo is made, we want to automatically publish this new version to PyPi.

PGMax uses two different files within its .github/workflows folder: ci.yaml and publish.yaml. Our ci.yaml file specifies all the actions that are run on push and pull-request events, whereas the publish.yaml file specifies actions to be run when a new release is made. Both files are fairly simple and reading their comments along with some of the official GitHub Actions docs should be sufficient to enable you to understand what each part of each file does (if that’s not the case, feel free to post a comment or shoot me a message and I’m happy to help clarify things!). The only piece of setup that’s needed to get these actions to run in any other repo is the creation of a GitHub Secret that enables GitHub to access my PyPI account when it needs to publish a new release of the package. To do this, simply create an API token on PyPI and then crete a new GitHub Secret named PYPI_TOKEN and paste your PyPI API token as the “value” of the secret.

Feel free to directly use PGMax’s YAML files for your own GitHub actions, or pick and choose specific sections adapted to your project’s needs!

Pro Tip: To make your GitHub repo look super professional, you can add a shiny badge to your README to display the status of your tests, documentation builds, etc. To do this, follow these steps.

That’s all folks! I hope these tips and links are helpful, and that you don’t have to spend as much time as I did on figuring out how to setup your shiny new open-source Python library. If you do end up using this guide for your open-source library, do let me know; I’m excited to see what you build!

If I had to give two pieces of advice to anyone writing an SoP, it’d be these: make it both research-dense and skimmable. I think it’s useful to think of an SoP as an application for a research job instead of a college essay: it should focus almost exclusively on things directly relevant to your ability to do good research (namely, your past experience and your interest and ideas going forward). Relentlessly cut-out any fluff. This is especially important because people on PhD admissions committees often skim SoP’s quickly (often because of the sheer volume of applications received). For this reason, it’s also useful to optimize your essay for skimming: do things like having the first 1 or 2 sentences of each paragraph setup what the paragraph will be about, or italicizing/bolding important words/sentences, etc. You shouldn’t take this too far and make your writing too terse or lazy though; remember that eventually, Professors will carefully read your entire SoP!

Good luck with writing and the overall grad school application process!

In my own case, at the start of my freshman year, I had a pretty good idea of what coursework I’d need to do to get a CS degree, and quickly found out the why and how of recruiting for various summer internships. However, I knew next-to-nothing about undergrad research. I didn’t know - for instance - that most Professors’ primary responsibility is to do research, that being a “researcher” is a legitimate career option, or that research experience is important for grad school applications. Fortunately for me, I stumbled into some incredible undergrad research opportunities by happy accident. Research changed my life; I loved (and still love!) it so much that I spent my senior year working to promote undergrad research opportunities within Brown’s CS department so that more students get to take advantage of the opportunities I’d gotten. One of the most common things I heard from students and peers is that they wished they had known more about research and gotten involved sooner. This post is an attempt to address that, as well as share some lessons I’ve learned as an undergrad researcher, by trying to answer three common, big-picture questions about undergrad research: what, why and how?

Table of Contents

• What the heck is Undergrad CS Research?
• Why should I care?
• How should I get involved?
• Some tips and advice
  • Some Do’s and Dont’s of Undergrad Research
  • Some Important Things to Keep in Mind

1. What the heck is Undergrad CS Research?

CS research basically involves trying to push the boundaries of CS as a field and discover new knowledge about computers. If that sounds like a vague, fairly generic description, that’s because it is. One of the cool things about CS research is that it is really broad and hard to draw a precise box around. There are researchers working on everything from AI to predict protein folding to proving things quantum computers won’t be able to do. Pretty much every cool CS-related invention - from Object-Oriented Programming to web search and the protocols of the Internet - started out as a project in a research lab.

This is all well and good, but it says nothing about what undergrad CS research involves. What does an undergrad researcher do on a daily basis? The answer is that, just like the content of CS research itself, it varies. However, there is some general structure in what to expect.

If you join a lab as an undergrad, you’ll probably be mentored by a grad-student (generally, someone working towards their PhD) or post-doc (someone who’s finished their PhD) directly, with occasional meetings with one or more Professors. You’ll get to work on a project that’s directly related to one of the lab’s research directions in a very specific sub-field of CS. Especially in the beginning, you’ll likely spend a good amount of time reading academic papers and getting yourself acquainted with the lab’s work (which can take anywhere from a week to a few months). You may also be asked to work on a “starter project”, like implementing an existing paper or solving a problem-set from a related class, to get you up to speed and give you a taste of the kind of work you’ll be doing [2].

Soon enough, you’ll probably be tasked with working on some relatively well scoped-out part of an existing project. Most of the time, this means designing/implementing some code as part of a larger codebase, or implementing a novel algorithm someone on the project has come up with, or designing and running one or more experiments. More rarely (but common with theoretical CS research), you’ll be asked to help prove a theorem or do something else that’s rather math-heavy like derive the worst-case runtime of a new algorithm. Eventually, you might be asked to help write up the part you worked on as part of an academic paper. What’s common between these different tasks is that they’re usually fairly concrete: someone’s sketched out what needs to be done in broad strokes and your job is to figure out all the details of actually doing it. In this way, these aspects of research are not unlike what a traditional software engineer does.

After working on a few projects, you’ll probably have the opportunity to lead your own project, which will pretty-much simulate what being a grad student or professional researcher would be like. Unlike before, this will involve much less concrete things, like finding an interesting problem to work on and designing a completely novel solution for it. On a day-to-day basis, this will likely involve reading lots of papers (a core activity for a researcher at any stage of their career!), having research-related discussions with your mentors/advisors/collaborators and spending a significant amount of time just thinking. These aspects of research are rather significantly different from software engineering because there are no descriptions of what you need to produce, and it could very well be that no one has done something similar enough before. This means you’re not even really sure where to get started or what sort of tools/frameworks to use. Generally, if you find that you enjoy these rather unstructured, less-concrete parts of research, then it’s a pretty strong sign that you’ll enjoy being a researcher.

Aside from these technical aspects of actually doing your research, you’ll also likely spend time communicating your research by giving talks or poster presentations and attending conferences. However, these commitments are generally few and far between for undergrads when compared to the other aspects of research.

2. Why should I care?

From the above description of CS research, hopefully it’s clear that research offers you the opportunity to (a) pursue an interest in a sub-topic of CS, and (b) learn a lot of interesting/useful skills/knowledge. This is great, but there are also so many other things - like working on a side-project, trying to start a startup, or doing web-dev for that cool new club - that you could do to reap similar benefits. Given this, here are a few benefits that are fairly unique to research:

1. This is (probably) the best chance you’ll get to try it

   Believe it or not, it’s exceptionally difficult to get involved with research if you’re not at a University. I’ve met quite a few people who became interested in research after graduation and their number one regret is that they didn’t take advantage of research opportunities while they were still doing their degree programs. In fact, gaining research experience is one of the most-common reasons that people decide to pursue Master’s degrees. Given that it’ll probably never be as easy to get involved with research as it is for you right now, you should probably at least give it a try.

2. You’ll figure out if you want to be a researcher or not

   It’s possible to split all CS-related career paths you could possibly choose to pursue into 2 categories: research (becoming a research scientist, research engineer, professor, etc.) and not-research (software engineering, product management, consulting, tech entrepreneurship, etc.). The general rule I’ve been told by internship mentors, grad students and Professors is that it is much easier to switch from research to not-research than the other way around. Given this, and the point above, it’s incredibly useful to know if a research career is appealing to you or not. There’s no better way to figure this out for yourself than to get involved with research and seeing if you like it.

3. You’ll massively strengthen grad-school applications (should you choose to apply)

   Gaining research experience is probably the single best thing you can do to strengthen a grad school application (especially for PhD programs!) [3]. So if you’re at all considering going to grad school, getting involved with research is a very good thing to do. If you’re not sure about grad school (like most people are), then it’s still useful to get some research experience because it leaves that door open for you to decide later. It’s much better to have the experience and decide not to apply to grad school than to not have the experience and want to apply. Additionally, as mentioned above, doing undergrad research will essentially simulate what a PhD program program will be like, which will help you decide whether or not you’d even like to apply to grad school in the first place.

4. You’ll get to form strong relationships with a Professor

   Professors are generally extremely cool, intelligent and experienced people who can (and want to) be incredible mentors to your career and life. They can give you valuable advice on everything from what classes to take to how to think about what kind of person you want to be. They can also open doors to cool opportunities (internships, grad-schools, etc.) with strong letters of recommendation and/or connecting you with people they know in academia and industry (and most CS Professors know a lot of interesting people!). In my own case, my undergrad research advisors are the best mentors I’ve had so far, and their advice changed my life and career trajectory; so I’d definitely highly recommend getting to know your Professors better. While you could do this through other avenues (TA’ing, going to a lot of their office hours, etc.) there’s nothing as direct as doing research with them.

5. You’ll strengthen job/internship applications

   Research projects - especially ones that are relevant to roles you might be looking for - are appealing to potential employers. This is especially true if you’re interested in research-adjacent roles (e.g. “research intern”, “research engineer”, “machine learning engineer”, etc.). What’s more, letters of recommendation from a professor can open up opportunities that might not be otherwise available (both the industry internships I did during my undergrad years were roles that I was only able to interview for on recommendation from a Professor). Having said this, I want to make clear that doing research is rarely the best use of your time if getting an industry internship/job is your primary goal; you’d be better off spending time on interview prep, side-projects, open-source contributions, etc.).

3. How should I get involved?

Hopefully at this point, you’re considering trying out this research thing for yourself. So how exactly do you go about getting involved? Fortunately, there are a lot of great existing articles and resources on this. Some posts I’ve found particularly helpful are this one from Cornell CS and this one from UNC Chapel Hill. While these articles include some school-specific information, the general process is roughly the same at any institution. Spend some time searching for information specific to your department and the various labs within it (if you happen to be a Brown student, check out this website and get in touch with the MURA’s!). If you can’t seem to find much, then don’t hesitate to talk to/email a Professor directly. Overall, taking initiative and showing enthusiasm for research opportunities can go a long way towards getting them!

Now, what should you do if there aren’t that many relevant/exciting research opportunities within your school’s CS department? If you’re a citizen or permanent resident of the US, the NSF funds a “Research Experiences for Undergraduates” (REU) program with a bunch of exciting, paid research opportunities in a variety of different topics all across the world. You can find a list of these opportunities here. If this doesn’t apply to you, or none of these opportunities stand out to you, then I’d encourage you to try to find research opportunities through industry internships (e.g. an AI Residency Program), virtual mentorships from established researchers (e.g. this one) or try cold-emailing professors or graduate students at nearby institutions (here are some generally helpful guidelines for writing such an email that’s likely to get a response). If none of that works and you still have the urge to do research, then go for it and forge your own path. Read papers, try to come up with an idea, and start working on it! This will certainly be difficult, but no one is stopping you. Plenty of people (like Tim Dettmers or Andreas Madsen) have successfully pursued “independent” research (there’s even an organization that helps foster this!). Again, I’d highly recommend taking initiative and showing enthusiasm: these are often the most important traits of a researcher and will significantly improve your chances of independent success or getting noticed.

4. Some tips and advice

Now that we’ve covered the basics of what, why, and how, let me share some tips and advice that I’ve gotten over the years and wish I’d known when I started as a bright-eyed bushy-tailed undergrad researcher.

4.1. Some Do’s and Dont’s of Undergrad Research

Do’s

1. Work towards a paper

   If you had to sum up a software engineer’s job in 3 words, “produce good code” would be a fairly apt description. If you had to do a similar thing for a researcher “produce good papers” would be it. Papers are the bread-and-butter of a researcher; pretty much all research projects aim to publish a paper in some capacity. Given this, if you want to get a true taste of what it’s like to be a researcher, you should work towards publishing a paper. Not doing so would be like trying to experience what it’s like to be a software engineer without working on a significant software project! By working towards a paper, you’ll not only get a real sampling of what it’d be like to be a professional researcher, but you’ll also have something tangible to show for it.

   Now, to be clear, I’m not suggesting that you should only work on projects where you’ll be first-author on a paper that’s published at one of the top conferences or journals. I’m also not suggesting that you don’t work on projects whose outcome might not be a traditional conference or journal paper - producing things like technical reports, posters or even deep-dive blog posts can often be just as useful of a learning experience as a traditional paper. It also takes time to practice and develop the various skills that go into good research, and it’s a good idea to start out by doing a small, unpublished starter project, or working on an existing codebase, or by joining a project led by someone more experienced. However, if you want to experience what research is really like (which you should!), you need to be part of at least one project that eventually publishes a paper (or something with similar qualities like a technical report, deep-dive blog post, poster, etc.).

   If you work on any other project (a code release for an existing paper, setting up infrastructure for experimentation, etc.), even if it’s within a research lab, be well-aware that what you’re experiencing isn’t fully representative of the research process, and that the fact that you enjoy this project doesn’t mean you’d enjoy being a professional researcher.

2. Ask for help and advice frequently

   Before I actually got involved with research, I imagined that researchers generally sat alone thinking about problems for long periods of time before having a “eureka” moment and writing some crazy important paper (a la Isaac Newton). I chuckled when I wrote that last sentence because this is actually pretty far from the truth: most research is extremely collaborative (notice how almost every paper you’ll read has more than one author). Sure, it’s sometimes useful to sit and think about a problem alone, but there’s no point banging your head against it repeatedly when you’re stuck: it’s not useful to you or anyone you might be working with.

   Don’t be afraid to ask for help and collaborate with others when you think they can help you get unstuck; discussing problems and brainstorming ideas can be one of the most enjoyable parts of research (it certainly is for me!).

3. Set reasonable expectations with your advisor

   Most undergrads feel the need to impress their advisor, especially when they first start out with research. As a result, it can be tempting to propose ridiculously hard-to-meet deadlines. Avoid doing this; the clearer you are about your time commitments, the easier it’ll be for your advisor to set expectations, and the more likely you’ll be to see your project through to completion instead of burning out and bowing out. Also, in general, it’s better to underpromise and overdeliver on deadlines than the other way around.

Dont’s

1. Flake

   This is unfortunately one of the most-common thing that happens with undergraduate researchers (and why some labs are hesitant to take any at all). Abandoning a research project suddenly and calling it quits is not only a letdown to the researchers you’re working with, but it also prevents you from getting a full taste of research and having something to show for it. Of course, if you truly can’t handle the workload, or actively hate what you’re working on, or aren’t getting enough time, help or guidance from the people you’re working with, then it can be a good idea to call it quits. However, undergrads often flake for simpler, more benign reasons: getting overwhelmed with other commitments and/or feeling stuck and not asking for help.

   Overall, I’d recommend that you stick with a research project to see it through, even if (more likely when) it gets difficult, annoying or uninteresting, because some of the best parts come after the worst ones. Also, sticking with a project will give you a true taste for the entire research process, and leave you with something to show for all your efforts at the end.

2. Feel too bad about not making much progress

   Research is hard, and this is especially true when you’re just starting out. One of the things that makes it so hard is that research progress if often highly non-linear. It’s possible (and somewhat common) to make very little progress on a project despite putting in effort. Projects also fail entirely from time to time. In this way, research is very much unlike course work or personal projects, and often much more frustrating for undergrads used to relatively linear payouts for work put in.

   However, it’s important to realize this isn’t just happening to you: all researchers get stuck on problems and feel frustrated [4], but it does get better with time and experience. As an undergrad, you can try to mitigate the frustration that comes from a lack-of-progress by leaning on your mentors and collaborators more, focusing on what you can control (the time and effort you’re putting in), and even potentially joining multiple projects so that you can switch gears to another project whenever you’re feeling stuck. However, it’s also important to pay attention to your feelings: if you’re finding that you really don’t like the frustration of being stuck, and would much prefer to make more linear progress on your work, then that’s a strong and useful signal that you probably won’t be very happy as a researcher.

3. Assume a Professor’s research is just like that course you took with them

   While Professors generally teach classes that are at least tangentially related to their research, this is not necessarily the case. Also, even if their research does cover the exact topic from a class you took with them, there is usually a big difference between learning material/doing class assignments and doing research. Try to do some of your own research on a lab’s recent work before trying to join it: skim some recent papers and maybe even go to a lab meeting (you’ll usually be welcome!). If you’d like to get a taste for the research process, see if your Professor teaches a seminar or graduate class related to their research and see if you can enroll in it.

4.2. Some Important Things to Keep in Mind

• It’s okay to feel like you don’t know what you’re doing

  It’s easy to feel lost and overwhelmed when trying to do research, especially as an undergrad. This is okay. In fact, it’s normal and believe it or not, most senior researchers feel like they don’t know what they’re doing quite often. This isn’t too surprising when you really think about it: research literally involves straying beyond the boundaries of the field into the unknown. If you know exactly what you’re doing all the time, you’re probably not exploring far enough. One of the most significant and sobering things my undergrad research advisor told me is that the feeling of not knowing what you’re doing doesn’t really ever go away, even as a Professor. The best researchers learn to accept this and operate well despite not truly knowing how exactly things will pan out.

• You’re probably not bothering/annoying them

  Professors and grad students are often incredibly busy juggling a jaw-dropping number of tasks. As a result, it’s easy to feel like emailing them or asking for a meeting when you’re stuck/need help is bothersome and annoying. It’s usually not - especially if you’re blocked/stuck on something where a quick email/meeting could save you hours (and a significant amount of headache). That being said, it’s probably not a good idea to email out a cry for help for every problem you encounter without first trying to solve it.

  In general, a good rule of thumb is that you should have exhausted all the reasonable options you can think of for solving whatever problem you’re confronted with before asking for help. This will not only often help you resolve most problems/develop a better problem-solving intuition, but also will enable you to ask much better questions by describing what you’ve already done and your thoughts on why they didn’t work. An email saying “Stuck on problem x and have no clue what’s going on pls HALP!” is much less productive than one saying “I’ve tried solution attempt a, b, and c to solve problem x and this is what happened in each of the cases, any thoughts on how to proceed?”. If you’re looking for more detailed guidelines on when and how to ask good questions, check out this StackOverflow post.

• You’re a valuable asset to your research group

  As an undergrad, it’s easy to look at the incredibly smart and productive grad students and Professor(s) in your group and feel like your contributions aren’t particularly important or valuable to the group. While you might not be producing as much as a grad student or Professor, that certainly does not mean you aren’t valuable (if that were the case, then the group wouldn’t have hired you and wouldn’t be interested in keeping you around!). In fact, aside from their direct contributions to projects, undergrads can prove valuable in some non-obvious ways. For instance, having an interested undergrad ask basic questions about a project or paper can often force people who’ve been working on the project for a while to rethink some fundamental assumptions and ultimately come up with better ideas (I’ve seen this happen; never hesitate to ask basic questions!). Additionally, working with undergrads helps grad students/post-docs acquire valuable collaboration and mentorship skills, which are part of the reason they chose their jobs in the first place. All in all, your work and presence as an undergrad are often much more valuable than you realize.

• Being an undergrad and doing research is hard: it’s okay to ask for extensions/take some time off.

  Being an undergrad means having a significant course load and a bunch of external commitments to clubs, part-time jobs, etc. Research will always have to be done on top of all this, which is hard to say the least. There will be some weeks when all your commitments overwhelm you and make it hard to get much research work done. This is fine. Grad students and Professors understand this (all of them were once undergrads and many of them likely know what it’s like to be in your shoes). And more often than not, they’re more than willing to accommodate you if you need to push back some deadlines, etc. That being said, it’s useful to try to plan in advance, set reasonable deadlines and expectations for how much time you can commit to research in any given semester, and avoid over-promising.

• Persistence goes a long way

  Research is very much a skill and, like all other skills, takes time and practice to develop. Research also has a long learning curve: it’s difficult to ‘teach’ someone how to do good research with a couple lectures. Rather, ideas, concepts and a good taste for problems tend to sink in with experience over time. So don’t be surprised if you feel like you’re not getting much better at research in the beginning. If you choose to stay at it, you’ll find yourself improving slowly and rather unnoticeably to you, till some day you give a talk or mentor new undergrads yourself and realize how far you’ve come.

That’s it! Hopefully that’s enough to give you an idea of what undergrad research is, why you might want to get involved and how to actually do it, plus some tips and advice that I wish I’d been given when I started out. I hope some of this has been informative, and even convinced you to give undergrad research a try.

Good luck - I can’t wait to hear about what you build :).

[1] As an example, the only substantive guide to undergrad research I’ve been able to find online is this Reddit AMA

[2] If you’re working on a starter project and get stuck, don’t hesitate to reach out to a lab member to ask for help! Contrary to what it might seem like, asking for help will usually be taken as a positive sign that you’re really interested in the work and excited to learn. Far too many undergrads give up after getting stuck on a starter project without even asking for help - so avoid this mistake if you can.

[3] See here, here, and here.

[4] There’s a famous story about how Geoffrey Hinton - a “godfather” of AI, inventor of Deep Learning and giant in the field of CS as a whole - got so frustrated with not being able to come to a satisfying understanding of how the brain works as an undergrad that he gave up on science and research entirely to become a carpenter for a year.

Thanks to Dylan Sam, Roma Patel, and Anna Wei for providing comments and feedback on a draft of this post.

Let’s suppose you want to create a new Python list and initialize it with all 0’s. Python provides a nice, intuitive and convenient way to do this with the * operator:

>>> my_list = [0] * 5

Great, and now you can edit/insert a value into any list index like you’d expect:

>>> my_list[0] = 23
>>> print(my_list)
[23, 0, 0, 0, 0, 0]

Now, suppose you want to kick it up a dimension and initialize a 2D list (i.e, a list of lists) with all 0’s. It’s only natural to use the nice and convenient * operator right?

>>> my_2d_list = [[0] * 5] * 5

Unfortunately, this doesn’t do what you intuitively think it does. To illustrate, let’s go and edit the top-left element

>>> my_2d_list[0][0] = 1
>>> print(my_2d_list)
[[1, 0, 0, 0, 0], 
[1, 0, 0, 0, 0], 
[1, 0, 0, 0, 0], 
[1, 0, 0, 0, 0], 
[1, 0, 0, 0, 0]]

Notice how not just the top-left element, but all elements in the first column got edited to the same value! What gives?

As it turns out, using the * operator to duplicate a list multiple times doesn’t create new lists, but just references the same underlying list object. This is almost never what someone wants when making a 2D (or multi-dimensional) list. But the behavior has to be this way for compatibility reasons (as explained in these nice StackOverflow answers).

If you didn’t know about any of this before, don’t feel bad - I’d been using the * operator to make Python lists for ~5 years before this and somehow never realized this particular quirk…

So, in conclusion, never make a more-than-1D Python list with the * operator. If you want to be extra safe, stop using the * operator to make lists at all.

If you’re here because you’re trying to debug strange Python behavior (hi future me!), and the quirk mentioned here wasn’t at the root of it, check out this GitHub repo for an exhaustive illustration of Python’s weirdnesses.

If there’s one piece of advice I’d give to anyone applying for the GRFP, it’s this: share your materials with as many grad students, professors and other researchers you can! It’s often daunting to share personal or professional essays with others (and the GRFP requires both!), but feedback from more experienced researchers - especially those who might have either written or seen successful GRFP applications in the past - is absolutely invaluable. Of course, you don’t have to listen to incorporate every piece of feedback you get, but often, you’ll be surprised at the suggestions you’ll get and realize you can make your application much stronger by changing a few paragraphs, etc. (this certainly happened to me!).

Good luck with your GRFP applications!

Now that my undergrad journey is coming to an end, I’ve been in a rather reflective mood. Recently, I spent some time reflecting on my journey from getting rejected as an undergrad applicant to MIT to now having committed to join the Ph.D. program in CSAIL. I decided to write some of these thoughts up and email Chris Peterson, who encouraged me to apply to MIT years ago. To my elation, he decided to share my story by publishing my email on the MIT Admissions Blogs! As someone who’s followed these blogs for close to 6 years now, having my own post up there is a huge personal milestone and dream come true. If someone had told me 4 years ago, when I was still trying to console myself after being rejected from MIT, that my college journey would turn out well enough to merit a post on the MIT blogs, I probably would have thought they were frothing-at-the-mouth crazy. Yet somehow, here we are, and I can’t help feel so incredibly lucky and grateful for how things have played out :).

If you happen to be reading this and are currently feeling a bit disappointed or hopeless by a recent rejection or setback (hello future me!), I hope this story brings you some comfort, motivation or hope to keep persisting toward goals you care about.

You can read the actual blog post on MIT Admissions’ site here.

Given this, I thought I’d put my own application online in the hopes that future Goldwater applicants find it helpful. Below, you’ll find my responses to all the short essay questions + my research essay; which represent the bulk of the Goldwater application. If you’re reading this and considering applying to the Goldwater but currently on the fence, I’d highly recommend you try it (especially if you’re a sophomore because you can apply again next year!). If you’re in no way planning to apply for the scholarship but just reading this anyways, I hope it’s at least mildly interesting / entertaining…

In one or two sentences, describe your career goals and professional aspirations (see example below). This statement will be used in publications if you are selected as a scholar or honorable mention.

Obtain a PhD in Artificial Intelligence. Conduct research aiming to develop intelligence algorithms to enable practical and collaborative robots.

What are your career goals and professional aspirations? Indicate which area(s) of mathematics, science or engineering you are considering pursuing in your research career and specify how your current academic program and your overall educational plans will assist you in achieving your career goals and professional aspirations.

Humans have been dreaming of creating intelligent robots for many centuries now. Intelligent and collaborative robots could enable everything from creating an extraterrestrial habitat on Mars to making nuclear fusion reactors a viable source of clean and safe energy. The potential applications of these robots are so vast that it seems like their development could fundamentally change the world for the better, similar to how computers have changed our world by an unprecedented factor in only the last 50 years. The prospect of having widespread intelligent and collaborative machines at our disposal is thus exciting indeed.

Given this, my primary career goal is to accelerate the development of such intelligent and collaborative robots.

I recognize that creating such robots is not a simple task in the slightest. Such intelligent and collaborative robots have not even been created in a laboratory setting yet. Hence, it is clear to me that I need to spend time learning as much as I can about these fields and attempt to advance the state-of-the-art in research as much as I can.

To this end, I am pursuing an undergraduate degree in Computer Engineering. I believe that to create truly useful robots, we must make progress in creating increasingly intelligent algorithms in addition to cheaper, more accurate sensors and actuators. Thus, I see the challenges of robotics as fundamentally being challenges of computer science and engineering.

I chose to study computer engineering because it enables me to gain foundational knowledge in both fields.

Outside of my curriculum, I am a member of a Robotics and Artificial Intelligence lab on campus. As a lab member, I am able to learn and work on independent research projects to advance the current state-of-the-art in various sub-fields like Planning, Reinforcement Learning, Imitation Learning and Human-Robot Interaction. I am also able to constantly learn about the field from my fellow researchers within the lab and around the world.

After graduation, I intend to pursue a Ph.D. in Computer Science with a focus on artificial intelligence for robots. I want to build upon the foundation I will have laid to focus both on developing intelligence algorithms for robots and on creating interfaces to facilitate better human-robot collaboration. At the completion of my Ph.D., I hope to have made significant contributions to the state-of-the-art for robot learning and be well-equipped to help build the robots of our dreams.

Describe an activity or experience that has been important in helping shape or reinforce your desire to pursue a research career in science, mathematics or engineering

My first research internship involved helping a colleague with his project: using data from expert demonstrations to teach a robot how to press buttons. My colleague already had an algorithm; all that was left was to implement it on our lab’s mobile robot. We didn’t think this would take more than two months.

As it turned out, we were very wrong. Our initial attempts resulted in our robot’s arm frequently hitting obstacles. Some weeks of debugging yielded a flaw in our algorithm. Some more weeks and extensive tests revealed our data-collection programs weren’t actually collecting data. Weeks continued to pass until my colleague graduated and we lost hope that the idea would work.

However, on the urging of an advisor, another colleague and I began re-examining the project. We ran meticulous experiments until we finally discovered a small error with our data collection. Days later, approximately one year after having gotten involved in the project, I held my breath and watched the robot press a button.

While button-pressing may seem a trivial task, that moment was one of the most profound experiences of my life. Despite the many setbacks and frustrations, I continue with research to this day because such moments of discovery, of knowing that I have created something significant, novel and potentially impactful, have made everything else well worth it. Research is the most exciting and fulfilling pursuit I have ever known and I’m excited to spend the rest of my life doing it.

Goldwater Scholars will be representative of the diverse economic, ethnic and occupational backgrounds of families in the United States. Describe any social and/or economic impacts you have encountered that influenced your education - either positively or negatively - and how you have dealt with them.

I grew up in my father’s relatively small hometown in South India where opportunities were scarce. My high school had no guidance counselor and the nearest SAT Testing Center was a three-hour drive. However, my parents are both hard-working immigrants to the US who taught me to dream big and then persevere relentlessly for those dreams. So, I did.

I taught myself electronics and computer programming and began competing in national - and eventually international - robotics competitions and science fairs. I took a ninety-minute bus ride every day for six years to attend my city’s only international school. Eventually, I was able to achieve my dream of being accepted to study engineering at a research-driven college in the U.S.

I should emphasize that I’ve had the good fortune of being supported by great people throughout my journey thus far. My parents have never spared expenses on my education or pursuit of opportunity. My high school teachers nominated me for every relevant extracurricular opportunity they could and stayed after school hours to mentor me. My school’s chairman even fully-sponsored our Robotics team when costs became prohibitive.

My experiences have taught me that it’s important to encourage and support students everywhere to pursue opportunity, no matter how scarce it may be. Sometimes, this support can change a life, as it did mine. In this spirit, I hope that my career and life can contribute to making the world have more opportunity for everyone, everywhere.

(PS: If you’re looking to write your Research Essay in LaTex, I’d highly recommend this free, open-source template by 2019 Goldwater Scholar Hannah Richards).