It’s Time to Remodel Our Policy Making Processes
J. Kennerly Davis
As the world struggles to overcome the devastating consequences of the most severe economic dislocation since the Great Depression, evidence mounts that the epidemiological modeling used to justify draconian lockdowns across the country and around the world was terribly wrong, profoundly flawed and wildly inaccurate in its doomsday projections of cases, hospitalizations, and fatalities.
Most notably, on March 16, modelers at Imperial College in London projected that the Covid-19 virus could kill 500,000 Britons and 2.2 million Americans. Some questioned the assumptions apparently used in the Imperial College model and complained that its software code was not readily accessible for independent review and validation. Others expressed concern that the chief modeler had in the past grossly overstated the potential threats posed by other outbreaks, including the mad cow disease outbreaks in the 1980s and 1990s.
No matter. The psychological and political impact of the Imperial College modeling was immediate and overwhelming. Overnight, its frightening numbers defined the benchmark for public policy responses to the virus across the globe.
Other modeling done by the Institute for Health Metrics and Evaluation at the University of Washington (the “IHME”) has been less apocalyptic but still alarming. Earlier this month it was projecting that more than 147,000 Americans could die by the first part of August. IHME modelers have repeatedly modified their pandemic projections, frequently to correct them downward for actual outcomes that proved to be less dire than predicted. Dr. Birx has publicly questioned the accuracy of IHME modeling.
During the pandemic, many in the commentariat have opined that science should determine policy during the crisis, not politics. We are now seeing where overreliance on faulty science, flawed modeling, can lead us.
Statements placing supposedly superior “science” over supposedly inferior “politics” reflect the long-established progressive, technocratic belief that policy issues in modern times are simply too complicated for ordinary citizens or their elected representatives to understand and effectively deal with.
According to this world view, “experts” are needed to manage the demanding complexities of modern society; credentialed subject matter experts who can plan for us, direct us, and protect us in times of crisis. Such experts, to fulfill their critically important responsibilities, must be free to exercise and apply their professional judgments without interference from unschooled politicians and ordinary citizens. We must defer to the experts. Our elected representatives must rely upon their judgments and directions. As noted, “science” should determine policy, not mere “politics.”
In fact, in a constitutional republic or any democracy worthy of the name, science and scientists should not determine policy, they should inform the policy making process and inform accountable policy makers, those elected officials who are responsible for weighing all the relevant considerations before deciding on a course of action.
We face many serious challenges. Sometimes, challenges explode into crises. Issues of public policy often involve the complexities of science and technology. Subject matter specialists have an important role to play in the formulation of public policy. But that role needs to be carefully and clearly defined as one that informs the process by providing useful subject-specific information. The proper role of unelected technicians and scientists is to inform, but not supplant, the policy determinations of politically accountable elected officials.
The Imperial College modeling has recently been described as one of the most devastating modeling mistakes of all time. One of the most important takeaways from the Covid-19 crisis should be a greatly heightened appreciation of the terrible damage that can result from overly deferential reliance on models and those who promote them.
It is time, past time, for a thorough reassessment of the outsized role that modelers and their models have been playing in the formulation of public policy, the drafting of legislation, and the development of regulations. The scope of this reassessment should include but not be limited to matters relating to public health and epidemiology. Overreliance on modelers and modeling is all too common. Models have, for example, driven and distorted the public discussion of climate policy and other environmental matters for decades.
The goal of this reassessment should be to develop a comprehensive set of protocols for the use of models in the policy process, a comprehensive set of procedural and substantive criteria that modelers must satisfy if they want their output to be taken seriously.
Models are only as good as their data, assumptions, and software. At the very least, modelers who want to be given an opportunity to contribute to the formulation of public policy must commit to absolute transparency, an essential attribute of sound science. Modelers must, in a timely fashion, make their data, assumptions, and software code all publicly available in a form that will support a rigorous crowdsourced independent public review.
Nothing less than complete transparency will do. If experts want to be respected participants in our policy processes, they must earn that respect. In matters of public policy, there can be no such thing as secret science. It’s time for some serious remodeling.