I wrote this response to Sophie Congreso’s Quora question: “Explain the need for a scientific method. What would happen if scientists do not follow an organized process?”

Outside of the ideals of science that have widespread acceptance, that is coming from a realist’s worldview, the methods of science are a standard mode of behavior that mix observations with skeptical evaluation of the results of that observation. That is the empirical side of things. The rational side of things is that the result of observation is data embedded in a wider framework whose placement comes from the logical, and mathematical necessity of the laws of nature. Those laws of course, are not the actual laws on which nature operates, but our best estimate of how nature must operate given our limited observations. If that seems circular, your perception is correct. But the circularity is not pernicious. Observations allow us to adjust the formulation of our laws to fit the reality which is there, not a fantasy we might hope for or expect from historical perception.

The process of science is not tidy in this respect. What we call the scientific method is a heuristic method capable of changing when it seems warranted to suit the object under observation. That is, it is not a logically locked down set of steps, but a set of guidelines that permits a wide latitude in how an observation might be carried out. Michael Shermer in Teach Your Child Science: Making Science Fun For the Both of You reminds us that “A technical definition of the scientific method that is satisfactory to all scholars and scientists is almost impossible to devise.” (37)

What I have given you is a reason to think that the scientific method is required for a sound realism. That is, a realism that has the ability to hold to good and useful results and reject poor ones. The scientific method must, if it is to be retained, have a self-adjusting role in science. Often because of human impatience we grow weary with checking our results and fixing our mistakes. This is natural. The scientific method gives us a mode of patience that provides better surety of success. Every successful scientist is an amalgam of lucky intuition and dogged, detailed, work. Both are required, but not everybody has the patience to carry out the labor that their intuition requires. In addition, the old adage that “The harder I work, the luckier I get” applies here. And even a continuous stream of failures in following the scientific method is instructive. For example, a large part of Albert Einstein’s later work was devoted to finding a Theory Of Everything, because quantum physics and cosmic physics failed every compatibility requirement, except of course our belief that it works together in reality so there must be a theory of how it does so. He explored every avenue his magnificent intuition could take him. And he always had his ear open to the suggestions of others. He didn’t always take their advice but he listened nonetheless. Yet for all his labors, he managed only to eliminate a variety of avenues for finding that TOE. We are closer today to a theory of Quantum Gravity, but we are not there yet.

Shermer outlines four steps in the scientific method: Observation, Generalization, Prediction, Experimentation. But we must add to that assiduous record-keeping. Keeping records permits the scientist, their contemporaries, and successors to explore the object and the method to look, not only for flaws, but insights. It has been said that quantum mechanics is counterintuitive, but that insight is perhaps passé. Today students grow up into a worldview that was deeply non-intuitive, but they absorb it as natural, and so can think clearly in those terms. Quantum mechanics is no longer the bugaboo it once was. There is nothing to be terrified of any more. We have begun to use its insights in ordinary technology, and so, it’s weirdness has become normalized.

This is part of what we call the sociology of science, the transformation of the social sphere of the scientific method into something that more closely models reality. But it has taken a couple generations of scientists to get from quantum weirdness to quantum normality. Reality didn’t change, but we did. So the scientific method is a mode of transforming realist society into something that more closely models reality. Natural laws are adjusted to conform to the truth of reality instead of some failed proposition. The puzzles, however, seem to multiply as our universe is sorted out in this way. But this is a good kind of problem.

This state of affairs is deeply troubling to some people. They look nostalgically to past certainties and universal truths, even though the world in its constant ferment has moved on. But as Michael Shermer said quoting James Burke, “Certainty is a kind of prison. When you are absolutely certain, you are also possibly absolutely blind.” (46) The need for the scientific method is modeled in the need to suffer uncertainty and ambiguity. Uncomfortable as those states of being might be, they are necessary in order to find the truth, and not be settled with something less. Think of using the scientific method as baking a cake. To get the optimum wonderfulness of the cake as properly imagined, you can’t take it out of the oven too soon. But you have to take it out when it’s ready or you’ll likewise spoil it.

Science would disappear without its method. We would be stuck in a world whose mysteries had become concrete and impenetrable. Scientists would become state functionaries, dispensing edicts without the attendant justifications that make the scientific method useful. As it is now, any person may follow the scientific method and gain for themselves a piece of reality that will not fade into oblivion with their demise.