If you ever want an interesting conversation with a physicist, ask them about free will.
On a basic level, free will is the ability to choose between different possible outcomes. For example, Neo freely choosing between the red pill and the blue pill in The Matrix. Most of us feel like we have this ability to choose. Our inner dialog seems to have an act of choosing between options. But a basic assumption of physics is that the universe is governed by physical laws which strictly limits the behavior of physical objects.
In classical Newtonian physics these laws are deterministic, meaning that if you knew the exact initial state of a system, you could predict the state at some future time. At no point would the system choose one outcome over another. It doesn’t make any sense to say that a baseball chooses to fall under the influence of gravity, for example, any more than it would for a skydiver to jump out of an airplane and then choose not to fall. Free will doesn’t come into it.
You might think that quantum theory gives you an out, since quantum mechanics is not deterministic, but that isn’t the case. Free will requires a choice between outcomes, not simply a probabilistic outcome. A common misconception in quantum theory is that quantum objects like photons electrons conspire to prevent us from knowing about a system. In the mathematical formalism of quantum theory, however, the probabilistic outcome is determined by the state of the system. Randomness is not free will.
What’s interesting about all this is that the usual metaphysics of quantum theory assumes some form of free will. Experiments such as the EPR Experiment presume that the observer is free to choose what to measure independent of the state of the quantum system. So basically we assume free will within a system that has no free will. No one ever said that physicists made good philosophers. Of course the counter argument is that even with this implicit observational free will, the predictions of quantum theory match reality to an extraordinary precision. Newton made incorrect assumptions about the nature of space and time, but his physics still works quite well. The same can be said of quantum theory.
There are approaches that try to overcome this metaphysical paradox, such as superdeterminism. The basic idea is that there are non-local “hidden variables” in a quantum system, but that they occur through the historical interactions between system and observer rather than being a real-time conspiracy of the quantum system. The downside of this approach is that the EPR experiment is agnostic on superdeterminism, so you’d need a different kind of experiment to test it. Since quantum theory already works quite well, superdeterminism is relegated to the back burner of physics. It might be interesting in the future, but for now, who cares?
Just to be clear, the fuzzy metaphysics physicists typically use doesn’t mean quantum theory is unfounded. It does, however, point to an interesting conundrum in modern physics that we haven’t fully resolved. Since free will seems intuitively correct, most physicists hold onto it without really thinking about it, and since it works well enough for most physical systems, there’s no real push to change.
But given what we know about the universe, how could we do otherwise?
Paper: Sabine Hossenfelder. Testing superdeterministic conspiracy. Journal of Physics: Conference Series 504 (2014) 012018. arXiv:1105.4326 [quant-ph]