Math-AI is an open, collaborative research community pushing the frontier of open research in Mathematical Superintelligence. We pursue foundational work in the open — from novel foundation-model architectures and efficient attention mechanisms to complex reasoning and formal mathematics.
All of our code, models, and write-ups are released openly. Explore the work on GitHub and Hugging Face.
A multi-architecture NanoGPT training and evaluation suite spanning softmax attention and the Fast Weight Attention framework. GitHub
A frontier coding agent that formalizes natural-language problems into Lean 4 theorems and proves them. GitHub
Recasts recurrent sequence modeling as online continual learning with normalized fast-weight rules (Falcon-1/2/3, 1A/2A/3A). GitHub
Fast, memory-efficient exact sampling for large language models via hardware-aware kernels. GitHub
Deep, stacked delta-rule fast-weight learning for expressive linear-time sequence models. GitHub
Group Representational Position Encoding — a group-theoretic foundation for positional encoding. [ICLR 2026] GitHub
Tensor Product Attention — factorized attention that sharply shrinks the KV cache. [NeurIPS 2025] GitHub
Higher-order Linear Attention — capturing higher-order token interactions in linear time. GitHub
A principled design space for KL-regularized policy-gradient algorithms in LLM reasoning. [ICLR 2026] GitHub
Self-distilled policy gradient for stable, low-variance LLM reinforcement learning. GitHub
Beyond Bradley–Terry — a general preference model for language model alignment. [ICML 2025] GitHub
Base language models as zero-shot generative classifiers that curate high-quality mathematical texts. [ACL 2025 Findings] GitHub
For inquiries and collaboration, email us at [email protected].