Retrocausality & The Transactional Interpretation of Quantum Mechanics | Ruth Kastner

Theories of Everything 2h11 9 min #19
Retrocausality & The Transactional Interpretation of Quantum Mechanics | Ruth Kastner
Watch on YouTube

Summary

  • The episode explores the transactional interpretation (TI) of quantum mechanics, developed by physicist-philosopher Ruth Kastner as a response to the persistent failures of the conventional quantum formalism. TI builds on John Cramer’s original 1986 proposal and the older Wheeler-Feynman absorber theory (also called the direct action theory of fields). Its central claim is that quantum measurement outcomes—and spacetime itself—emerge from a deeper, non-spatiotemporal realm of real physical possibilities Kastner calls the quantum substratum. The interpretation uses both retarded (forward-in-time) and advanced (past-directed) field processes to explain how definite outcomes arise without requiring conscious observers, and it claims to resolve the measurement problem, nonlocality, and the tension between quantum theory and gravity.

The Measurement Problem and Why the Conventional Theory Fails

  • The measurement problem is the central motivation for TI: the standard (Dirac–von Neumann) quantum formalism has no internal criterion for distinguishing a mere interaction from a measurement that yields a definite outcome.
    • This is illustrated by Schrödinger’s cat: an unstable atom in superposition of decayed/undecayed is coupled to a Geiger counter, a vial of sleeping potion, and a cat. The conventional theory only ever produces larger and larger superpositions (“adding train cars”), never a single definite outcome. The cat is described as both awake and asleep—something we never observe.
    • The problem is twofold: (1) the theory cannot say what counts as a measurement, and (2) it cannot explain why outcomes are probabilistic (the origin of the Born rule).
    • Because the theory cannot define measurement from within, physicists since von Neumann and Heisenberg have resorted to an arbitrary “cut”—appealing to an external conscious observer to collapse the wave function by hand. This is ad hoc and philosophically unsatisfying, especially since there is no principled way to define consciousness.

How the Transactional Interpretation Resolves Measurement

  • TI replaces the conventional assumption that fields propagate unilaterally (only forward in time from emitter) with the direct action theory of fields: both emitters and absorbers are active. Under appropriate conditions, an emitter generates a retarded offer wave, and potential absorbers generate advanced confirmation waves (past-directed adjoint fields).
    • When an offer wave and a matching confirmation wave connect, they form what Cramer called a transaction—a physical interaction that formally corresponds to a projection operator, the mathematical object that represents a measurement outcome.
    • This transaction breaks the superposition and yields a set of clearly defined, distinct possible outcomes (a mixed state), each with a probability given by the Born rule. The Born rule is not postulated but derived from the physics of the direct action theory.
    • The transition from the mixed state to the single observed outcome (the “collapse”) remains genuinely indeterministic—TI does not restore determinism, but it does explain why there are definite outcomes at all.
    • Measurement does not require a conscious observer. Any system capable of absorbing (fulfilling conservation laws) can participate in a transaction. A tree falls in a forest whether or not anyone is there to see it.

Two Levels of Field Interaction

  • TI distinguishes between two kinds of field processes:
    • Virtual photon exchanges (time-symmetric propagators): These are always present among charged particles. They are correlating interactions, not measurements. They do not produce outcomes and are not transactions. A Feynman diagram of two electrons exchanging a virtual photon represents this level—it is one term in a perturbation expansion, not an offer/confirmation event.
    • Real photon transactions: These occur when conservation laws permit a real (on-shell, massless, transversely polarized) photon to be transferred—for example, from an excited atom to an unexcited one. The excited atom emits an offer wave; the unexcited atom generates a confirmation wave. When these match, a transaction occurs, and a real photon is transferred. This is a measurement interaction.
    • The probability of a transaction occurring at any given time corresponds to what the conventional theory calls a decay rate. For macroscopic systems, these probabilities are so overwhelmingly high (99.9999…%) that we perceive a world of continuous, definite objects. At the mesoscopic scale (e.g., buckyballs in a two-slit experiment), the probability can be around 50%, and the quantum behavior becomes visible.

Ontology: What Is Real

  • Kastner is a realist about quantum theory: quantum systems and fields exist physically and independently of observation. However, she rejects the assumption that everything physically real must exist in spacetime.
    • Offer waves and confirmation waves are not little waves propagating forward and backward in spacetime. They are quantum states in a high-dimensional complex Hilbert space—mathematically, they do not have spacetime character. Trying to force them into spacetime (as Cramer originally did) distorts their mathematical nature.
    • Instead, these entities exist in what Kastner calls the quantum substratum—a realm of real physical possibility that is not spatiotemporal. Spacetime is emergent, not fundamental.
    • The ontological commitment is minimal: quantum systems and fields are real, and the formalism of the direct action theory describes how they actually behave. Kastner resists adding further metaphysical substance (she suggests structural realism as a label).
    • Configuration space is not taken as real in the sense of a physical space; it is a mathematical construct. What is real are the quantum systems and their states.

Emergence of Spacetime

  • Spacetime emerges from the quantum substratum through transactions. Kastner uses two metaphors:
    • Iceberg: The tip (what we observe as spacetime events) is real but emergent; the vast submerged portion (quantum possibilities) is equally real but not directly observable.
    • Geode: Quantum possibilities are like mineral-laden fluid filling a hollow rock; transactions cause them to crystallize into the structured crystals of spacetime events.
    • Spacetime, in Einstein’s own terms, is a structured set of point coincidences (events). The coordinates (x, t) are parameters for coordinating observations, not fundamental observables. At the relativistic level, there is no position observable and no time observable—only energy and momentum are fundamental.
    • With colleague Andrea Schlatter, Kastner has derived the Einstein field equations from this picture, including corrections that account for galactic rotation curves without dark matter.

Empirical Distinction and the Nature of Anomalies

  • TI is empirically equivalent to the conventional theory at the level of Born rule probabilities, but it is empirically distinct in a deeper sense: it resolves an anomaly that the conventional theory cannot address.
    • Measurement outcomes are an anomaly for the standard approach. Just as the precession of Mercury’s orbit was an anomaly for Newtonian gravity that Einstein’s relativity resolved, the measurement problem is an anomaly for conventional quantum theory that TI resolves.
    • Critics often demand a new empirical prediction, but this misunderstands the nature of anomalies. The conventional theory does not predict that outcomes occur at all—it is silent on the matter. TI predicts and explains when and why measurement transitions happen.
    • Kastner notes that critics often attack earlier, less developed versions of TI rather than engaging with her relativistic formulation (presented in her 2022 Cambridge University Press book, The Transactional Interpretation of Quantum Mechanics: A Relativistic Treatment).

Retrocausality and Time

  • Despite the use of advanced (past-directed) waves, Kastner does not endorse literal backward-in-time causation in spacetime.
    • The offer and confirmation waves are not spacetime entities; they are processes in the quantum substratum. The appearance of retrocausality is a feature of the formalism, not a description of signals traveling backward through time.
    • She contrasts her view with the two-state vector formalism (TSVF) of Avshalom Elitzur and others, which she regards as only “half the transactional story” (it considers only the offer wave reaching absorbers) and as still inheriting the conventional theory’s inability to define measurement. TSVF also requires stipulating all future measurement outcomes, which implies a block world ontology and forecloses genuine becoming.
    • TI does not support science-fiction-style time travel. Once a spacetime event has crystallized (a transaction has been actualized), it cannot be undone. Possible timelines exist at the level of possibility, not as alternate actualized spacetimes.

Relation to Other Interpretations

  • Many Worlds (Everettian): Arises from the same inability to define measurement. Since the conventional theory cannot say an outcome occurred, it posits that all outcomes occur in branching worlds. TI avoids this by providing a mechanism for definite outcomes without requiring branching.
  • Bohmian mechanics: Has hidden variables and a preferred foliation. TI has neither. TI naturally yields preferred observables (energy/momentum) at the relativistic level without needing to postulate a preferred foliation.
  • Penrose, Elitzur, and others: Kastner is patient with the slow pace of paradigm change (the heliocentric model took ~200 years to be accepted). She attributes resistance not to ignorance but to unexamined metaphysical “training wheels”—assumptions about locality, determinism, and spacetime as the arena of all reality.

Nonlocality, Bell, and Kochen-Specker

  • TI fully embraces nonlocality. Bell inequality violations and the Kochen-Specker theorem are genuine features of quantum theory, and TI does not try to explain them away.
    • Nonlocal influences in the direct action theory do not involve controllable light signals and do not violate the null signaling theorem or relativity. They are “behind-the-scenes” dynamics at the level of possibility that make certain spacetime events more probable than others.
    • The Flatland metaphor: Just as a 3D sphere passing through a 2D plane would appear to the flatland creatures as a mysteriously appearing and disappearing circle (seemingly nonlocal), quantum nonlocality reflects influences from a higher-dimensional (non-spatiotemporal) realm that do not obey the constraints of spacetime.
    • Kochen-Specker (contextuality): The theorem shows that you cannot assign definite values to all observables simultaneously. This is natural in TI: at the level of possibility, systems do not have determinate values for incompatible observables. A system with definite momentum has not yet engaged in the transaction that would actualize a position event. There is no reason to expect non-contextuality at the quantum level.

Gravity and the Theory of Everything

  • TI takes a distinctive approach to gravity: gravity is not a quantum field. The gravitational field is the metrical structure of emergent spacetime—a property of the set of events, not a quantum entity in its own right.
    • This sidesteps the entire problem of “quantizing gravity.” The Einstein equations are derived from the transactional substratum, and the approach naturally accounts for galactic rotation curves without dark matter.
    • TI accommodates the weak and strong forces: these govern unitary transformations and scattering interactions at the level of possibility (the submerged part of the iceberg). Only the electromagnetic field, as a massless gauge field, is responsible for the emergence of spacetime events.
    • Kastner is modest about calling TI a “theory of everything”—physics may not be able to explain consciousness, intentionality, or life. But within the domain of physical questions (quantum theory + gravity), it provides a unified framework.

Free Will and Consciousness

  • TI is genuinely indeterministic: the Born rule gives real probabilities, and which outcome is actualized is not determined. This leaves room for free will, though TI itself does not provide a theory of it.
    • Kastner argues that physics does not rule out free will, contrary to claims by many physicists. Those claims rest on unnecessary metaphysical presuppositions (e.g., that everything must be deterministic or that matter is “dead”).
    • On consciousness: Kastner personally believes consciousness is more fundamental than any physical theory. If one assumes matter is non-sentient (the “Cartesian” or materialist assumption), the hard problem of consciousness is unsolvable by construction. She argues this assumption is optional, not mandatory, and that physics should remain agnostic about the intrinsic nature of matter rather than preemptively declaring it “dead.”
    • She does not claim that rocks or rivers are conscious, but she notes that even Heisenberg spoke of photons “making choices” about whether to pass through a polarizer. Indigenous and Indian philosophical traditions take the sentience of nature seriously, and she has come to respect these perspectives through her study of Indian philosophy and her practice as a yoga teacher.

Open Systems and Generalizability

  • TI is completely general—it is not restricted to closed systems. Lindblad master equations and other open-system formalisms are naturally accommodated. In the conventional theory, the use of probabilistic descriptions in thermodynamics requires an ad hoc “hand wave”; in TI, the physics of transactions provides a principled reason for when and why master equations apply.

The Afshar Experiment

  • The Afshar experiment was claimed to violate complementarity by measuring two non-commuting observables simultaneously. Kastner argues it did nothing of the sort: it was formally equivalent to preparing a particle in a state, confirming that preparation, and then measuring a non-commuting observable—a perfectly standard procedure. The impressive appearance (using a lens and wire grid) does not change the underlying physics.

Advice to Young Researchers

  • Be a student of nature, not an enforcer of metaphysical preferences. Examine your own assumptions: Are you imposing on nature something nature might not be doing? Are demands for locality, determinism, or spacetime as the fundamental arena truly necessary, or are they “training wheels” that have become constraints?
  • Be flexible and a lifelong learner. Kastner herself switched from physics to philosophy after encountering the EPR paradox and nonlocality. She got her PhD at 36, after exploring many directions. She encourages disobedience when the tradition is stuck: Einstein worked in a patent office.
  • Let the data and the mathematics teach you, rather than forcing nature to conform to your preferred metaphysical picture—as Heisenberg did when he followed the data to matrix mechanics.

Killer App

  • TI’s central achievement is theoretical consistency: it solves the measurement problem, derives the Born rule, reconciles quantum theory with general relativity, accounts for galactic rotation curves without dark matter, and does so without hidden variables, preferred foliations, or conscious observers. It provides the “whole phone” on which all other applications run—a physically consistent framework that does not founder on thought experiments like Wigner’s friend or the Frauchiger-Renner paradox.
Back to Theories of Everything