Modular Neural Logic: How Architecture Shapes Compositional Reasoning
Researchers present THEIA, a modular neural architecture that learns complete Kleene three-valued logic end-to-end without external symbolic solvers. The system uses four dedicated engines for arithmetic, order, set membership, and propositional logic that converge in a final logic module, achieving full K3 rule coverage on a 2M-sample dataset in under 8 minutes. Mechanistic analysis reveals that modularity enables a 'delayed verdict' strategy where upstream engines encode domain-specific variables without committing to final truth values, with the verdict emerging only at the logic boundary, a representational pattern distinct from how monolithic Transformers solve the same problem.
TL;DR
- THEIA learns three-valued logic (K3) through pure neural architecture without symbolic solvers, achieving 12/12 rule coverage across 5 seeds
- Modular design generalizes from 5-step to 500-step sequential composition at 99.97% accuracy, while flat MLPs collapse to chance performance by 50 steps
- Mechanistic probing shows modularity induces delayed verdict strategy: upstream engines encode uncertainty without committing to final truth value until logic boundary
- Transformer baseline reaches equivalent correctness through qualitatively different representational trajectory (contraction then expansion), suggesting distinct compositional strategies between architectures
Why It Matters
This work provides empirical evidence that architectural structure, not just scale or training data, fundamentally shapes how neural networks learn compositional reasoning under uncertainty. The mechanistic findings reveal that modularity enables a specific representational strategy for handling incomplete information, offering insights into why some architectures generalize better to longer sequences and more complex logical compositions than others.
Business Impact
For teams building reasoning systems, AI agents, or verification tools, this demonstrates that architectural choices directly impact compositional generalization and length extrapolation. The 6.5x speedup and superior length generalization suggest modular designs could reduce training costs and improve robustness in production systems that must handle variable-length logical reasoning.
Key Implications
- Modularity is not just an engineering convenience but a computational primitive that enables specific representational strategies for compositional generalization, with measurable advantages over flat architectures at equivalent parameter counts
- The delayed verdict mechanism suggests that uncertainty handling and final commitment can be decoupled in neural architectures, potentially informing design of systems that need to reason under incomplete information
- Different architectures (modular vs. monolithic) solve identical problems through qualitatively different representational trajectories, implying that mechanistic interpretability findings may not transfer across architectural families
What to Watch
Monitor whether the delayed verdict strategy and modularity benefits transfer to other domains beyond logic (e.g., natural language, code, planning). Watch for follow-up work on scaling modular architectures and whether the mechanistic insights about uncertainty encoding inform new training objectives or architectural designs for reasoning systems.
Subscribe to the newsletter
The latest stories and analysis, delivered to your inbox.
Free. No spam. Unsubscribe any time.
