Axiom of Reducibility
How do we punt the asymptote to truth further away?
The ever changing cycle of which AI Lab produces the highest-benchmarking LLM will eventually resonate with the AI community as futile, sooner or later. At the application layer, however, it is increasingly important to contextualize what an LLM can and cannot do. As prediction machines, imitating computation to provide de facto accurate responses suffers with the same achilles heel as foundational logic addressed in Principia Mathematica's Axiom of Reducibility.
In short, the goal in the Axiom of Reducibility is to prove all logic can be derived by using the quickest, most agreeable assumptions possible: that being a mathematical formula. To avoid digging deep into a 100+ year old idea, we introduce two elements, independent of each other, and we operate on them in such a way that they combine into something entirely new.
Many criticized this axiom as arbitrary, because reducing logical conclusions to mathematical principles by explaining one destination with a mathematical route and its twin the black-box logical route is basically improvised.
The issue in the world of LLMs becomes that by separating information into tokens, then using multi-layer inference to arrive at the probabilistic conclusion, it is impossible that even the top model of the moment doesn't pass by paradoxical axioms that lead to immense infallibility. As we continue to scrape for more training data, those paradoxes are swept under the rug, leading to an Asymptote to Truth.
How do we punt this asymptote further away?
We could:
See if they can tell the future by walling off training at one singular point in time.
Constrain them within a time period and challenge them to draw the same associative breakthroughs that we achieved.
If we can document these experiments and their outcomes, we can reverse-engineer how LLMs navigate toward truth despite their inherent limitations. Creating a fixed temporal snapshot, we force the model to extrapolate beyond its knowledge horizon, revealing whether its predictive capabilities stem from genuine reasoning or mere pattern-matching on historical data.
Apple conducted a similar study back in August, where they concluded that AI models are incapable of iterating outside of their training data. This makes sense from the systems engineering perspective: you cannot produce an output that doesn't derive from a singular or a collection of inputs. Cookies cannot come out of an oven without all of the necessary ingredients correctly organized inside of one. However, if we combine elements at a low level, similar to the philosophical ideals of the Axiom of Reducibility we can create foundational takeaways that could be visualized like a summative mind-map, avoiding the computational prediction infallibility we're toying with. This mind-map concept fundamentally exists via the neural nets and multi-dimensional vectors that help synthesize token inputs. However, blocking together step-by-step conclusions may lead to accurate reinforcement learning outcomes. The new big caveat that we'd have to address is avoiding heuristic thinking within LLMs which already plague the design flaw of the human mind.
Moving on, by confining the model to a specific era - say, the Industrial Revolution - and prompting it to link steam power to economic shifts without modern hindsight we test its ability to forge novel associations. We also unlock the ability to create economic forecasting models that can escape gridlock legislation that plagues monetary and fiscal policy. We can delineate the boundaries for our Asymptote to Truth but also develop pathways for refinement, such productizing feedback loops that mimic human trial-and-error learning, pushing the model closer to a more robust World Model. World Model has a cool ring to it.
TLDR
To move the needle, we need:
Computation & Prediction Ledgers
SCMs (Structured Causal Models)
Logical-Outcome Distributions
Tool-Grounded Reasoning
Hope you enjoyed the read.