Research 2026 — present
Adaptive Computational Topology (ACT)
Can a network's topology adapt during pretraining?
An experimental pretraining framework investigating whether jointly adapting model weights and computational topology during pretraining can improve the trade-off between model capability and compute cost, compared with fixed architectures of comparable capacity.
Approach
- A research hypothesis, explicitly not a demonstrated result — the joint topology question is still open, while the later architecture-search phase reports a positive result on a narrower question.
- Pre-registered studies with declared decision rules, rather than results read after the fact.
- Compute-aware topology search rather than unconstrained growth.
- Model-importance signals — routing load, gradient norms, Fisher approximations and causal ablation — driving where topology may change.
- Machine-checked claims: a verification script audits the report against its committed artifacts, and the suite pins each arm's recorded result against its own summary, so a claim that drifts from the record fails the build.
- Control and null arms a result has to beat to count, including a do-nothing clause.
Built with
- Python
- PyTorch
- topology search
- experiment tracking
- XLA