Independent Research

Prizma

Efficient-attention & local-learning research

Two small-scale threads built the same way — a pre-registered falsifiable bar, a parameter/FLOP-matched baseline, an adversarial referee audit, and binding honest limits. Every number traces to a reproducible script.

by Aylin · Independent Researcher

Two threads

One method, two open questions

Each thread is a falsifiability gate passed — or honestly refused — inside a precisely characterized, small-scale regime. Neither claims large-scale parity.

Lead · Sequence modeling

Prizma-Seq

A Gated-DeltaNet-family sequence mixer. Its novel lever is a parameter-free quadratic feature map (quad2) that makes the per-head carried associative state rectangular (d_h × d_φ, monomials as fixed seeded buffers → 0 added parameters), giving O(1) constant-memory inference.

Parameter-matched against a tuned decoder-only Transformer (RMSNorm + SwiGLU + RoPE), it clears the project's pre-registered §4 diagnostic bar.

Verdict: candidate

Secondary · Continual learning

Prizma

A backprop-free, fully-local predictive-coding learner targeting neuromorphic/analog hardware. Task-boundary-free and task-label-free continual learning via recognition-routing.

In an input-distinguishable (domain-incremental) stream it reaches zero forgetting — beating naive backprop and EWC with no replay, no boundaries, and no weight transport (works with random-feedback DFA).

Verdict: zero forgetting (in-regime)

Prizma-Seq · the §4 bar

Seven legs, parameter-matched vs a tuned Transformer

Pre-registered pass/fail criteria, decided before the runs. Raw A100 result JSONs are committed and cross-checked report ⇄ raw by a 4-referee audit.

Leg Verdict Headline
MQAR (D=128) PASS Parity @ 860K params; solves @ 130K where the matched TF needs ≥ 461K → ≥ 3.5× param-efficiency (coarse grid)
Induction PASS quad2 0.9995 (3/3) vs TF 0.996
Selective-copy PASS selective 0.9991; a fixed-position control isolates content-selectivity
Char-LM (text8) PASS Prizma 1.7496 vs TF 1.7254 BPC — within the +0.05 bar (does not beat TF)
Inference PASS (memory) Constant 17.9 MB state ∀n (28–455× less); measured O(1)-latency crossover at n ≥ 32k (2.4–2.8× faster @ 65k)
Causal ablation PASS quad2 ≫ rand_linear ≈ none ≫ TF — the gain is the quadratic monomials, not "a bigger RNN"
Length-extrapolation WIN (relative) 10× better retention than a RoPE Transformer at 8× train length (absolute acc still only ~0.40)

Inference efficiency

Constant memory, and an O(1) latency crossover at long context

Both models decode step-by-step (KV-cache for the Transformer, an O(1) carried state for Prizma). The Transformer's per-step cost grows with context; Prizma's is flat.

17.9 MB
Constant decode-state footprint for all context lengths n.
28–455× less than the Transformer KV-cache (small config).
n ≥ 32k
Measured latency crossover — beyond this, Prizma decodes faster per step.
Below n ≈ 16k, Prizma is ~1.3–1.6× slower.
2.4–2.8×
Faster per-step decode at n = 65k vs the Transformer.
Long-context-only — not a general speed claim.

Prizma · continual learning

Zero forgetting without boundaries, buffers, or weight transport

Headline result E1 (structured-permuted, 10 seeds, ±95% CI). Prizma sits between replay and the task-id oracle, matching the oracle's zero forgetting without being told the task id. The noRoute ablation shows recognition-routing is the causal mechanism.

Learner ACC FGT ↓ boundaries? buffer? Wᵀ?
backprop MLP 0.445 0.553 — — —
EWC 0.456 0.411 yes — —
replay (buffer 1000) 0.737 0.156 yes yes —
oracle_multihead (upper bound) 0.879 0.000 task-id given — —
Prizma (DFA, no Wᵀ) 0.834 0.000 none none none
Prizma (exact Wᵀ) 0.708 0.000 none none yes
Prizma_noRoute (ablation) 0.446 0.489 — — —
Three-panel continual-learning result: (A) per-task retention through training — Prizma holds flat while backprop decays; (B) forgetting bars over 10 seeds with 95% CI — Prizma ~0 vs backprop/EWC/noRoute high, replay intermediate; (C) separability sweep as input noise blurs the domains.
A. Per-task retention through training — Prizma (solid) holds flat where backprop (dashed) decays.  B. Forgetting (lower is better), 10 seeds ±95% CI — Prizma ≈ 0 against backprop, EWC, replay, and the noRoute ablation.  C. Separability sweep — accuracy degrades gracefully as input noise blurs the domains. (Figure rendered under the project's earlier working name.)

Honest scope & limitations

This project's credibility is its no-spin culture. The binding limits, stated plainly:

  • A candidate, not a proven alternative. Prizma-Seq clears the §4 diagnostic bar at small scale — it is not an established attention replacement.
  • Char-LM does not beat the Transformer. 1.7496 vs 1.7254 BPC: a loss within the pre-registered +0.05 margin, not a win.
  • The latency win is long-context-only. Prizma is ~1.3–1.6× slower below n ≈ 16k; the crossover is at n ≥ 32k.
  • Training is ~5× slower per step (the sequential delta update).
  • No per-FLOP claim. The FLOP-matched Transformer arms were optimization-confounded and are excluded from the verdict.
  • n = 2–3 seeds are descriptive, not a powered equivalence test.
  • No large-scale-LM parity and no backprop-free parity is claimed — both remain open frontiers.
  • The continual-learning result holds only in the input-distinguishable regime. There is no benefit in the fully-ambiguous regime (proven impossible for any single-head learner); it degrades gracefully as domains overlap.
  • quad2 belongs to the Based/Hedgehog feature-map family. The novelty is the rectangular delta-state framing, not the kernel itself.