By Sravana Reddy and Ramya Ramakrishnan
Task-oriented dialogue, an area of research within the broader field of conversational AI, is an exciting area centered around building dialogue systems to solve tasks. It is a high-impact area of study as natural language systems become increasingly ubiquitous across consumer applications and the enterprise. In addition, researchers in this field get to work on open research questions with high scientific impact.
Success as an applied research scientist in this field is largely driven by three main objectives: identifying the computational problem underlying the product goal, defining the right metrics of success, and innovating on the state of the art in the form of published research papers and product impact. We’ve identified four key ways new researchers can make sure they’re preparing themselves to succeed in AI research in industry, particularly in task-oriented dialogue.
1) A cultivation for constant learning
AI as a field moves very quickly. The latest advances in machine learning can have significant improvements in performance that come out on what seems like a monthly basis. Papers come out daily, and major areas and directions change every few years. As research scientists, it is important to be up-to-date on the best performing and most efficient models as the state of the art advances. Case in point: GPT-3 may be a useful pre-trained model, but the newer GPT-Neo outperforms GPT-3 on benchmark metrics and is far more computationally efficient. Staying abreast of these advances can result in profound advantages.
That said, it’s virtually impossible to stay on top of all research. We’ve found it best to focus on a couple of areas for a deep focus, while keeping a general awareness of the broader field. So, while you may not need to know the details of how GPT-3 or Megatron-Turing work while reading the latest papers in task-oriented dialogue, you should at least know that these pre-trained language models exist, how they are used, and their limitations (since newer is not always better).
2) Tap mentors and peers
As the latest state-of-the-art methods and papers publish rapidly, having a network of peers thinking about a similar set of research problems can help research scientists stay apprised of the latest research. Having Slack channels with your peers to curate relevant papers is a helpful practice to keep a pulse on the latest methods, as well as to discuss further areas of exploration. Paying attention to a single annual conference and sharing the relevant papers with your peers is more feasible than keeping track of all papers in all conferences every year.
Where possible, identify who you can tap as a mentor. This can be your manager or someone who works in a similar role elsewhere in industry. Mentors can point you to relevant papers in the field, as well as guide you to work cross-functionally across teams. Especially when coming from a different field in machine learning, in-house mentors and peers can help new research scientists become acquainted with well-known models and technical terminology.
3) Work holistically across a team
Moving from academia to industry/product can be a paradigm change. In academia, your peers use similar language and hold a similar view of the world as you do. In industry, research scientists need to be able to work cross-functionally across different disciplines, departments, and teams — who may each be thinking about the same problem but in categorically different ways. You will be working together with engineers, product managers, user/market researchers, and data scientists. To collaborate effectively will require learning your peers’ language and perspective, and understanding the role they have in solving that shared problem. Gaining that shared understanding can take some time but a deliberate effort to foster this holistic collaboration can help realize impactful results.
Having humility in this process can help foster a productive collaboration as well. Sometimes titles and PhD degrees can present an impression of different levels of authority in tackling a problem. It is important to realize that across different disciplines and teams, everyone is an expert in their respective fields working to bring their effort to the collective problem. Having humility in this collaboration can allow each team member to bring their best work forward. In addition to humility, having an open mind can help in learning a lot and solving the problem more effectively.
4) Have a product-level view on the problem you’re solving
Sometimes, in order to better understand the cross-functional problem you’re solving, it helps to momentarily take yourself out of the researcher mindset. The purpose of communicating with different teams is more often than not to better define the problem you’re collectively trying to solve. Once you’ve defined the problem, it’s easier to then shift back to your researcher mindset and focus on how you’ll contribute to the solution.
What’s particularly exciting about task-oriented dialogue is that so much of the research efforts toward these product-level solutions are unprecedented. Open-ended research problems that remain in the field include control of generative models, and abstractive+extractive summarization.
Don’t be afraid to dive in and make mistakes!
As you learn more and become acquainted with the research scientist role, our experience has been that you start to realize how much you don’t already know about the focus area or research specialty. That feeling can be daunting, but you can be surprised how much you do already know and how that knowledge complements the fuller picture provided by your colleagues. It’s a great thing to have different knowledge and skillsets. The future of task-oriented dialogue itself, in fact, may be focused on how to best create human-in-the-loop systems for complimentary AI + human teams.
If you’re a graduate student finishing your degree, entering into industry as a research scientist seem may seem daunting, but the role does become easier over time. It is fine to make measured mistakes as you onboard and become acquainted with the area of specialization, work with cross-functional teams, and understand how to frame the problem. What’s most important is to keep an open mind and learn from the new methods and contexts as you become embedded in the field.
About the authors
Sravana Reddy is a research scientist at ASAPP, where she currently works on transfer learning and domain adaptation. Previously, she worked at Spotify, building a voice assistant for music and enabling podcast discovery through content modeling. She is also interested in computational approaches to understanding language variation. She received her PhD from the University of Chicago.
Ramya Ramakrishnan is a research scientist at ASAPP focused on leveraging generative language models for a variety of applications, such as improving agent training and automating agent tasks. She completed her PhD at MIT, where she worked on human-in-the-loop machine learning and human-robot interaction. She is interested in building robust machine learning models that learn from human feedback and can augment human capabilities in complex tasks.