memory_caching/
├── src/memory_caching/       Stable package surface published as memory-caching
│   ├── layer.py              Memory Caching wrapper
│   ├── backends/             Linear, DLA, Titans, SWLA(c=2)
│   ├── bench/                Benchmark adapters, runners, manifests
│   ├── models.py             Tiny model-backed scientific path
│   └── scientific_manifest.py Scientific artifact truthfulness checks
├── configs/                  Train + benchmark + baseline-tracking configs
├── docs/                     Reproduction, release, API, and claim-boundary docs
├── examples/                 Stable public examples
├── scripts/                  Train / eval / gate / packaging entrypoints
└── tests/                    Backend, API, benchmark, and release-path coverage

This is not official author code. See reproduction_report.md, CLAIM_TO_EVIDENCE_MATRIX.md, and PAPER_PARITY_BLOCKERS.md for the exact claim surface.

uv sync --extra dev
uv run mc list-variants
uv run mc smoke-eval --backend linear --device cpu --warmup-steps 1 --batch-size 1 --seq-len 8 --vocab-size 16 --d-model 8 --num-heads 2

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
mc list-variants

python -m pip install memory-caching
python -c "from memory_caching import MCConfig, MemoryCachingLayer; print('ok')"

• CPU-only example:
  • python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
• CUDA 12.1 example:
  • python -m pip install torch --index-url https://download.pytorch.org/whl/cu121

Install a torch build that matches your local CUDA runtime and driver stack before CUDA workflows.

The supported top-level runtime imports are:

• memory_caching.MCConfig
• memory_caching.MemoryCachingLayer
• memory_caching.SegmentCache
• memory_caching.LinearMemoryBackend
• memory_caching.DLABackend
• memory_caching.TitansBackend
• memory_caching.SWLABackend

For runtime use, prefer:

• layer(x) for the normal forward path
• layer.forward_with_cache(x) when cached segment checkpoints are needed
• layer.inspect(x) when per-token routing/debug rows are needed

Repo tooling is intentionally broader than the public package API. CLI wiring, smoke helpers, benchmark runners, release gates, and report-generation scripts remain repo-level tooling rather than stable semver-tracked runtime surface.

Full API notes:

• PUBLIC_API.md

Namespaced experimental/reference modules now present in the package:

• memory_caching.baselines.LogLinearPP
• memory_caching.loglinear.LogLinearAttentionReference
• memory_caching.loglinear.ChunkedLogLinearAttentionReference

1. Documentation Home
2. Reproduction Report
3. Public API

# List implemented backend/aggregation variants
mc list-variants

# Minimal CPU smoke eval
mc smoke-eval --backend linear --device cpu --warmup-steps 1 --batch-size 1 --seq-len 8 --vocab-size 16 --d-model 8 --num-heads 2

# Debug routing and cache behavior
uv run mc debug-layer --backend linear --aggregation grm --seq-len 8 --d-model 8 --num-heads 2 --out-json outputs/debug/debug_layer.json

# Repository engineering gate
uv run python scripts/reports/release_gate_v1.py --mode repo --out outputs/reports/release_gate_repo_v1.json

# Scientific gate
uv run python scripts/reports/release_gate_v1.py --mode scientific --out outputs/reports/release_gate_scientific_v1.json

For dense command coverage, use:

• BENCHMARK_COMMAND_MATRIX.md
• CONTRIBUTOR_ONBOARDING.md
• PYPI_RELEASE_RUNBOOK.md

Canonical terminology used in this repository:

• engineering scaffold: code quality, reproducibility, packaging, and report-generation integrity
• scientific evidence: model-backed artifacts with non-smoke targets and truthful manifests
• paper parity: faithful reproduction of the paper's reported baselines, metrics, and missing comparison rows

scientific evidence is stricter than the engineering scaffold, but it is still not the same as paper parity.

What a green scientific gate still does not prove:

• full paper parity
• full evaluation evidence for LogLinearPP
• original LogLinearAttention remains a separate future mechanism track
• throughput parity or unpublished internal-author equivalence

Backend-specific limits also remain important:

• linear is an unnormalized matrix-memory reference path, not a normalized linear-attention parity claim
• dla, titans, and swla are mechanism-oriented reference implementations
• titans and swla currently use constant scalar coefficients where the paper presents time-indexed coefficients
• soup is only true state-space mixing when the backend supports state mixing; otherwise the repo uses an explicit output-mixture fallback

Stable published examples:

• examples/minimal_layer.py
• examples/inspect_layer.py
• examples/loglinear_reference.py
• examples/loglinear_chunked_reference.py

Pilot runner:

uv run ./scripts/checks/loglinear_pilot.sh

Current namespaced research/reference surfaces:

• memory_caching.baselines.LogLinearPP
• memory_caching.loglinear.LogLinearAttentionReference
• memory_caching.loglinear.ChunkedLogLinearAttentionReference
• tiny-model families:
  • tiny_loglinear_ref_lm
  • tiny_loglinear_chunked_lm

Sample subset dataset files included for benchmark dry runs:

• examples/longbench_subset.jsonl
• examples/retrieval_subset.jsonl

If you use this repository, cite the original paper and this implementation.

@article{chandra2026memorycaching,
  title={Memory Caching: RNNs with Growing Memory},
  author={Chandra, ...},
  journal={arXiv preprint arXiv:2602.24281},
  year={2026}
}

@software{memory_caching2026,
  title={memory-caching: Community PyTorch Implementation of Memory Caching},
  author={McCleary, Kyle},
  url={https://github.com/kmccleary3301/memory_caching},
  year={2026}
}