A Beginner’s Guide to Scientific Roadmapping
Wonderful maps and how to read them
Science needs more catalytic projects so that we can build essential technologies.
How do we find those catalytic projects? One way is by surveying the landscape of science and creating roadmaps that show us how to navigate the tech tree before us.
There are bottlenecks choking progress in virtually every scientific field, and you can identify these with a good roadmap. Once you know where they are, you can conduct research to break through these bottlenecks (or find clever ways to route around them) and unlock essential technologies. Often this unblocking and rerouting requires coordinated leaps forward — projects at such a scale that individual researchers, traditional institutions, and single grants simply cannot tackle alone.
Identifying scientific bottlenecks requires a special kind of forward-looking judgment. You have to look down the road at what will be made possible by unlocking the particular node on the tech tree you’re contemplating or routing around. Even if it’s unsexy or particularly thorny, you have to judge the value of solving a challenge now by the future unlocks it makes possible. Finding and funding catalytic projects means asking: If you knock over this one scientific domino, ‘how many other dominoes fall downstream?’ not ‘how intrinsically novel or interesting is the original domino?’
At Convergent, we find these catalytic ideas by talking to brilliant scientists frustrated by systemic barriers to innovation. Motivated researchers are full of great ideas, especially when you give them permission to engage in out-of-the-box, deconstructive thinking. Even scientists and innovators who have a lot of practice engaging in first-principles thinking can remain constrained by assumptions held by their field unless you set up a space where you can challenge them.
Astro Teller, head of Google X’s moonshot factory, has noted that achieving "10X" progress can sometimes be easier than obtaining incremental improvements because 10x thinking forces researchers to completely resurvey the landscape. By drawing a roadmap that helps you see promising paths to progress, the scientific community can establish a rigorous basis for justifying and directing the "multi-hop" investment needed to tackle the ambitious projects that lead to breakthroughs.
When you pull these conversations together, we’ve found the results typically reveal three key things: (1) which exciting avenues are being choked by a lack of coordinated research or infrastructure; (2) widely-held, and ultimately false assumptions about what advances are tractable; and (3) which destinations are especially compelling to target. Talk long enough with these folks about their ideas in the right ways and you end up with a roadmap towards a rich landscape of scientific and technological possibility.
In this post, I’ll sketch some preliminary design principles for five types of scientific roadmapping, and share some of our favorite examples. These principles and existing roadmaps have helped our team at Convergent Research to direct resources toward projects that we expect to be especially catalytic in the next five years.
There are many, many more transformative and fundable projects out there, and I’m excited for us to share more of our own map of them soon. In the meantime, we hope this serves as a guide that can help you read scientific roadmaps, conduct a survey of your own domain or, perhaps, even pitch a bottleneck-busting vision of your own!
How to build a scientific roadmap
To give you an intuition for how to interpret these artifacts, let’s first take a look at how scientific roadmaps get drawn, from creating a forum, to setting goals and choosing what type of roadmap is right for you. Whether it’s a group of intrepid researchers who want to leap-frog developments in their field or a collection of government funders determined to spur technological advancement, you have to get knowledgeable people together in a room to start thinking and writing and mapping together. Once you have a sense of how those discussions play out, you’ll have a better intuition for how to read the documents that come out of them.
Creating a Forum
Roadmapping begins by getting scientists to think about systematic barriers to innovation that they often don't get a chance to think about, let alone talk about together in any systematic way. Neuroscientists, for example, might wonder why it’s not yet feasible to map the whole mouse brain connectome; or why we can’t yet read and write to the whole human brain. But it’s not feasible to ask that question within a normal grant cycle.
Typical academic career and funding structures do not reward or provide regular venues for such work, unfortunately, so there is rarely sufficient local and shared incentive for individual researchers to pursue roadmapping projects on their own. (Of course, the history of science shows many moments where paradigm-shifting work is rewarded, but these moments aren’t usually produced by design.) Those that do often execute them as passion projects, labors of love that they lug from conference to conference or conduct well after their other critical grant-related work has been offloaded or has spun down.
So, if you want to help researchers get the kind of bird’s-eye view they need of their scientific landscape to do scientific roadmapping you can help by building a supportive forum. Workshops, brainstorming sessions, and collaborative writing exercises where researchers are encouraged to step beyond their immediate research agenda are all excellent venues for this — each is a temporary but potentially transformative environment that creates critical connections and durable artifacts for future use.
Consider ARPA program design workshops, for example, a signature element of the process for launching a high-powered research effort at government R&D agencies like DARPA and IARPA, and now ARPA-H and ARIA. At the inception of any research program a group of program managers, invited scientists, and industry experts gather to evaluate and align on “big if true” research objectives: What if we could predict drug safety and efficacy accurately before clinical trials even begin? What if we could bioprint any organ on demand?
These are hairy, audacious goals that nonetheless reflect real desires on the part of humanity, and could bring tangible benefits to billions of people. Negotiating these goals helps the group to map out research pathways and build a sense of shared ambition between scientists. Large coordinated projects thrive on a bedrock of healthy human relationships and a shared mission, so it’s no mistake that roadmapping starts here. Often, this is a chance for practitioners to revisit the dreams and aspirations that the founders of their field had decades ago, and to evaluate their progress towards reaching those lofty, important goals.
To curate a roadmapping workshop, select a set of participants who can both think outside the box, question assumptions, and ground the subject’s fundamental physical or scientific constraints. To find these people, consult experts and workers in fields who are not only highly knowledgeable but frustrated by systemic barriers to innovation and feel constrained by what’s possible. Often, when I meet these folks I have images of them just busting at the seams, like their lab coat - not to mention their whole lab infrastructure - is too tight. Balance them out with folks that are fresh in the field and can bring fresh perspectives. Each participant enters with a particular expertise and workshop problems are often interdisciplinary.
To borrow a term from the social study of science, you can think of roadmapping exercises as creating what Peter Galison calls “trading zones” between the contributing fields, and the roadmap documents as “boundary objects” that can move between those fields and set a framework for communication between these disciplines. (Lauren Greenspan and her co-authors have written on LessWrong about AI safety cases as useful boundary objects for developing big science efforts in the field.) Participants must therefore catch each other up on relevant background concepts, language, and the state-of-the-art in their contributing fields, align on the true scope of the questions being asked, and actively encourage one another to speculate. Leave the usual stump speeches at home!
You can encourage certain participants to give opening presentations that take risks, setting the tone for others and expanding the range of acceptable discussion and questioning. Future-looking speculation is often discouraged by journal reviewers; in this sort of setting, it’s the perfect tone. As scientists, many of us are pretty good at thinking about why some ideas won’t work; processes like these encourage us to ask what might be possible if this one does?
Roadmapping needs a diverse group of participants but also needs strong leadership. Typically, a cross-disciplinary leader will lead a larger writing exercise that establishes a framework, breaks down the problem into parts, helps each participant contribute to the most relevant parts, synthesizes the contributions from the participants, and melds them into a unified whole, questioning assumptions and making connections along the way.
Consensus is not the goal here, but rather an amalgam of perspectives that, when seen together, uncover a set of specific potential paths beyond the bottleneck — paths that no one participant may be able to fully articulate or even believe in. Whereas surveys of actual physical landscapes are meant to be definitive and durable, scientific roadmaps are wonderfully polyvocal and opinionated and change when new realities about how the universe works become known.
Roadmapping Goals, Structure, and Priorities
One indicator that a topic may be worth roadmapping is the presence of a productive tension: there is widespread agreement about the transformative potential of a goal, but significant debate around its near-term feasibility.
Say that you were those same neuroscientists from earlier, and considered it plausible (though uncertain) whether it was within the boundaries of the laws of physics to create a system that could read from the entire mouse brain. You’d want to move forward by really drilling into the physical limits, as well as figuring out if one could theoretically create such a system for less than all the funding currently available in the field. (One of my first roadmaps from over a decade ago was motivated by conversations like this.)
Or say that you had just gone through a global pandemic and wanted to move beyond pathogen-specific disease detection. You might wonder: can I leverage advances in metagenomic sequencing to detect emerging pathogens in such a discerning way that I can meaningfully separate early pandemic-potential signals from noise?
Finding a question that has this tension (‘this is physically possible and would be very impactful but seems profoundly challenging’) means that you’re in the right place and your topic sits at the frontier of current capabilities - a site where ambitious, innovative thinking can potentially bridge the gap between the goods you can envision and what’s presently achievable. At the same time, being able to define your very ambitious goals precisely is important - simply saying that you want to work on neuroanatomy or Alzheimers is insufficiently defined and will risk your group being pulled in too many directions. On the other hand, being able to say “I want to map the mouse connectome for a large but reasonable sum of money and effort” is quantitative enough to be tractable, posing a challenge that people can really dig their teeth into as they map out their plans to tackle it.
Roadmapping initiatives are designed to reveal these deep-seated gaps in research capabilities and pinpoint opportunities where strategic interventions could yield the most significant scientific returns. Questions central to scientific roadmapping can include:
What common denominator technologies or capabilities could unlock the most progress across multiple future research areas?
What core constraints define what is possible in a domain?
Could there be assumption-violating workarounds for those constraints?
Which problems should be targeted if we weren’t limited by resources, execution or coordination capacity?
Historically, roadmapping has led to ambitious projects like the Human Genome Project, which came out of a series of meetings at Cold Spring Harbor in the ‘80s. The BRAIN Initiative was born from the Brain Activity Map, which was created at a 2011 gathering organized by the Wellcome Trust, the Gatsby and Kavli Foundations, and the Allen Institute of Brain Science. Similarly, astrophysicist Kip Thorne’s theoretical work on detecting gravitational waves laid the foundation for the Laser Interferometer Gravitational-Wave Observatory (LIGO), a project that succeeded over multiple decades of development and collaboration. One inspirational point of view is reflected in how the old ARPA did “studies” — inexpensive, incisive investigations that revealed the deep, often hidden interconnections within complex systems and served as a trigger for directing larger investments toward the most impactful opportunities
Beyond large government initiatives, we believe roadmapping is vital to fund specific opportunities that address critical research gaps. They provide a strategic overview to see what could be achieved with a hypothetical (ambitious but realistic) technology - scientific infrastructure that the existing science establishment can’t build. Roadmaps guide researchers, and institutions toward high-impact, high-return areas. And they build a strong intellectual case for why particular fields and technologies warrant coordinated, large-scale investment.
Types of Roadmaps
Roadmap formats stand in contrast to the typical academic artifacts - journal articles, which report specific novel results, and literature reviews, which usually focus on the past. Instead, roadmaps often take other shapes such as landscape analysis, technical roadmap, pipeline analysis, opportunity analysis, agenda setting manifesto, and vision paper. Below you’ll find a summary of each, their objectives, and why we think they can be powerful.
The field of scientific roadmapping is a nascent one – this preliminary list is not meant to be exhaustive or a perfect categorization. Roadmaps can use hybrid formats, and it may take time to see which ones you need. But it will hopefully illustrate some of the texture of the explorations we think science needs more of. Let us know your favorite ones that we missed!
1. Landscape Analysis:
Think of this like a literature review, but going over the approaches being taken by people, labs, and organizations rather than just analyzing a collection of journal articles. You want to find both what exists and doesn’t so that you can see what should exist, but doesn’t yet. Landscape analyses of these activities and approaches reveal their assumptions, directions and the gaps between and among them. These high-level perspectives help us identify potential opportunities. New tools for AI assisted literature analysis through efforts like those at Future House and the release of OpenAI’s Deep Research means that the marginal cost of a landscape analysis of existing literature is going down fast, but nothing beats candid conversations with the people behind the work.
Purpose: List who is working in a space and what they’re doing, in order to reveal potential synergies or gaps.
Design Considerations:
Comprehensiveness. Does this include all the work that could be relevant, even from non-obvious sources?
Organization. Did the landscape usefully organize the players into a framework that can reveal gaps and opportunities, as well as key differences in assumptions or approaches? (Bonus points if the review introduces a new and useful categorization or framework for thinking about what is being tried and why.)
Wayfinding. Did the review help identify interesting opportunities, assumptions or gaps?
Examples:
2. Technical Roadmap:
These roadmaps outline the potential paths to overcoming major scientific challenges. They focus on specific technical goals and quantitative constraints.
Purpose: State a technical goal and detail all the possible ways of achieving it through R&D: necessary breakthroughs, and possible solutions.
Design Considerations:
Well-defined stretch goal. Is success unambiguously measurable? Is the goal still sufficiently far afield that it would truly stretch the existing scientific enterprise and the existing assumptions within the field to achieve it? Can the goal be decomposed into necessary or sufficient conditions? (e.g., Roadmapping to human-level AI performance on Atari or Go would have been hard in advance because we had no idea what nontrivial qualities were needed in a solution. We don’t have a “physics” of intelligence the way we have a physics of electromagnetism. Yet.)
Completeness. Did you explore all possible routes, including ones far from what is being tried now? Are the approaches considered mutually exclusive and collectively exhaustive (MECE)?
Constraining. Did you find nontrivial limiters (logical, quantitative, or probabilistic) for the different paths to the goal? Did you explore ways to work around those limiters and whether each one is truly fundamental?
Note about one subtype of Technical Roadmap that we love: Take something seemingly impossible and show it is actually just very very hard in a finite number of specific ways. Empower people with a quantitative framework for generating and triaging potential ways forward towards the un-impossible goal. Some examples include Casey Handmer’s writings - like this piece - on what it would take to do supersonic electric flight and a roadmap on whole mouse brain neural activity recording that I worked on. It could be a map of physics-based constraints and potential workarounds, or a MECE assessment (quantitative) for a hard technical challenge facing physical or engineering limits. For problems that have pretty well defined physics constraints, we’ve seen utility from this type of method.
Examples:
Imaging, Sensing, And Communication Through Highly Scattering Complex Media
Reframing Superintelligence: Comprehensive AI Services as General Intelligence
3. Pipeline Analysis:
These deconstruct complex research processes, such as drug development or materials science, into discrete stages. Pipeline analyses examine how resources are allocated and identify opportunities for optimization.
Purpose: Get more of some desired output/product from a given process.
Design Considerations:
Definition. Is the output clearly defined and measurable? (E.g. FDA-approved drugs: yes. Scientific breakthroughs: less so.)
Deconstruction. Is the process divided into conceptually clean steps?
Careful accounting. How much time/energy/money/resources goes into each step? What is the yield at each step?
“Gradient estimation.” How convincing are the estimates of what interventions would increase overall output?
Examples:
4. Opportunity Analysis:
These assess the potential impact of various initiatives, allowing funders to understand the likely outcomes and long-term value of different scientific investments.
Purpose: Estimate the expected value (on the margin) of a funding opportunity.
Design Considerations:
Quantitativeness. Did the author produce convincing numerical estimates of impact?
Thoroughness. Did the author consider all possible outcomes, failure-modes, and counterfactuals?
Examples:
Open Phil uses the Importance, Tractability, Neglectedness framework, like New cause area: Violence against women and girls
DARPA uses The Heilmeier Catechism as a methodology for generating this type of analysis for their programs
VCs write investment memos, like Bessemer's Shopify investment memo
5. Agenda Setting manifesto
Purpose
Pose a concrete, “big if true” vision for a new capability, and sketch the elements of a coordinated research or engineering program that could plausibly solve it.
Design Considerations
Direction. Does it lay out a transformative possibility for creation or improvement? Does it lay out a concrete set of necessary and sufficient action?
Quantitativeness. Does it provide quantitative or at least concrete criteria for success?
Holistic. Does it clearly identify limiting factors? Does it synthesize a holistic picture of how an otherwise dubious sounding goal could be achieved?
Examples
The brain activity map project and the challenge of functional connectomics
Organ Preservation Alliance Roadmap for solving the organ shortage
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Lauren Greenspan on safety cases to motivate AI safety related research programs
6. Vision papers:
A somewhat broader category laying out the shape of a largely uncharted field, often through introducing quite new concepts or frameworks. Michael Nielsen has a great essay about vision papers with some examples here. These go beyond classic agenda setting manifestos in the way they introduce new conceptual frameworks and they may not be as concretely actionable versus intellectually inspirational.
Recent examples in AI:
Roadmaps in any of these formats are tools for identifying, coordinating, and advancing cross-disciplinary initiatives.
At Convergent, we have seen that the roadmaps like this can steer science toward massively enabling technologies. Take this 2018 paper published in ACS Journal of Proteome Research by Harrison Specht and Nikolai Slavov. Specht and Slavov (and others) had long recognized a bottleneck toward medical innovations: researchers’ inability to quantify proteins in single cells. The paper outlined specific ideas to improve single-cell mass spectrometry sensitivity and throughput by orders of magnitude. Think of this work as a technical roadmap (as outlined above) that uncovered key dimensions for improving proteomics. These ideas make a case for devoting resources to particularly catalytic R&D. And it led to the founding of Parallel Squared Technology Institute (PTI), a focused research organization.
A couple parting thoughts…
Roadmapping fits into a broader picture of enabling systems level research leadership. It is necessary because individuals and individual labs cannot make it through these bottlenecks alone.
In today’s research ecosystem, it can be hard to come by the resources, time, and justification for building a roadmap, but this leads to a negative feedback loop where we impoverish the culture of science by promoting thinking focused largely on projects the size of the most common grants (e.g., the NIH R01). Roadmapping promotes a forward-looking and agentic culture of research as well. Eric Drexler discussed this kind of culture and Space Age spirit in describing his inspiration to build a roadmap for nanotechnology:
…the space systems community taught me a way of thinking that harnessed creative vision to physical, quantitative reasoning, in order to explore what could be achieved in new domains of engineering… Satellite launchers and moonships grew out of quantifiable engineering visions: system-level concepts that could be sketched, assessed, and discarded at a rapid pace, evolving through a kind of Darwinian competition. The best concepts would win the resources of time and attention needed to fill in more details, to optimize designs, to apply closer analysis, and after this refinement and testing, to compete again. The prize at the end would be a design refined into fully detailed specifications, then metal cut on a factory floor, then a pillar of fire rising into the sky bearing a vision made real…
In future posts, we will share more about the outcomes of these kinds of roadmapping processes, detailing bottlenecks, landscapes, and the focused research organizations (FROs) addressing them. Stay tuned by subscribing here.
Are you a scientist who wants to lead or join a different type of team? Are you a funder looking to accelerate science? Do you see a bottleneck in science that new bridge-scale tools, systems or datasets could transform?
At Convergent, we want to steer the engine of non-profit deep tech that we are building towards the most positively impactful problems for humanity now and in the long term. Our current open call is a request for FROs in the UK. The deadline for submitting a proposal is March 28. Check out the call and submit your ideas for critical bottlenecks and how to bridge them with essential technologies.
Acknowledgements
Thanks to Milan Cvitkovic and Ales Flidr for helping Convergent develop its culture of scientific roadmapping. Thanks to Anastasia Gamick, Mary Wang, Janelle Tam, Joseph Fridman, Christina Agapakis, and Max Levy for reviewing drafts of this piece.
And if you’ve read this far, here are even more examples that our team likes:
Productive Nanosystems: A Technology Roadmap: Drexler, K. E., et al., 2007
Battery Technology Roadmap: Ma, Jianmin, et al. "The 2021 battery technology roadmap." Journal of Physics D: Applied Physics 54.18 (2021): 183001.
Willy Chertman’s Fertility Whitepaper
Superconducting Optoelectronic Neurons V: Networks and Scaling
Towards ubiquitous metagenomic sequencing: a technology roadmap
Will there be an open call in the US later this year?
FYI the link to Reframing Superintelligence is broken because FHI has closed down. Here's another link to it.
https://owainevans.github.io/pdfs/Reframing_Superintelligence_FHI-TR-2019.pdf