So I think for science overall, across scientific fields, even though there might be flux in our knowledge (which foods are healthy or unhealthy, or what we think dinosaurs look like), this flux is part of the process of improving our understanding of the universe, and often goes hand-in-hand with improvements in our tools and technologies, and our ability to accurately measure our environment.
You write in the book, "What we're really dealing with is the long tail of discovery." What do you mean by this? What are the implications for researchers, institutions, and funding?
Arbesman: When a scientific field is new, there are lot of "low-hanging fruit" to be discovered and discoveries can come quickly and easily. As time passes and a field becomes more mature, successive discoveries are often less seminal or earth-shattering, yet are still important in fleshing out our understanding of the discipline. A small number of discoveries can explain a great deal of the phenomena in a single field, but we still need to examine the ideas and knowledge far out in what I term "the long tail of discovery" in order to have a true picture and understanding of the field.
I'm sure there is one perspective that funders might not want to expend so much effort for this long tail, but I don't think that's the right way to view this. The discoveries out in the long tail ensure that we have an unbiased depiction of the world, and see our surroundings in all of its rich variety and complexity. Therefore, researchers, institutions, and funders should continue working hard to elaborate this long tail of discovery.
What areas of science would you point to where we can expect to see the most rapid developments in the next generation or two? If the development of knowledge proceeds in sometimes predictable ways, can that help us guide funding? Could we use these models to help us find areas that are most due for a breakthrough or other significant achievement? What kinds of factors/constraints can we not account for?
Arbesman: I think the areas that are poised for the most rapid developments are at the boundaries of traditional domains. As science grows, traditional fields become increasingly disconnected from each other. But as scientists can bridge these gaps, connecting one field to another by importing ideas or recognizing similarities in problems, we can get massive developments. Combine computer science with biology and you get the many fields that are loosely grouped under computational biology, and this is one of the fastest-moving fields. Even the field of digital humanities (combining computational and digital tools with the humanities) is poised for rapid growth.
More generally, there are numerous scientists who are developing tools and techniques to predict the growth and birth of scientific fields. For example, work by Carl Bergstrom's lab has examined how new fields develop, such as neuroscience, and Katy Börner's research group has also explored the evolution of research areas. Funding agencies are becoming increasingly interested in this kind of research, as it can help guide funding. For example, the National Science Foundation has a program called Science of Science and Innovation Policy devoted to this.
That being said, it is not easy to predict or determine a breakthrough or significant achievement. But we can sometimes make some headway. For instance, sometimes these kinds of problems can be quantified and smaller steadier discoveries can be viewed as steps along a path towards larger discoveries. For example, discovering the first potentially Earth-like planet can be viewed this way and predictions are possible in this area. Other problems, such as determining when a long unsolved mathematical conjecture will be solved, are more difficult, but probabilistic estimates can still be made.