In a trace-driven simulation, a flash-based memory design called FLINT served each of five tested mixture-of-experts models within a 50-millisecond time-per-output-token target using one GPU package at batch size 1. The HBM min-fit comparison needed four to eight packages to hold the weights. Those figures come from a modeled evaluation, not a hardware demonstration.
FLINT treats high-bandwidth flash, or HBF, as a near-accelerator capacity tier alongside high-bandwidth memory, or HBM. The study asks whether a workload-driven HBF substrate can make that arrangement practical for large language model inference while addressing the adoption challenges identified by the authors.
The workload set comprised six production LLMs: five MoE models and one dense model. Here, MoE means mixture of experts, with routed-expert distributions driving the traces for those models. The evaluation used an in-house trace-driven simulator covering five weight-read tiers, four batch sizes and four context lengths.
The machinery behind the model
FLINT's burst-buffer controller converts fine-grained cache-line reads into coarse-grained, plane-parallel HBF bursts and overlaps sensing the next burst with draining the current burst using the flash die's page and cache buffers.
Refresh uses a separate mechanism called phantom-plane refresh. An extra physical plane rotates as a background-maintenance plane while the logical planes continue foreground reads.
The design also uses a read-only flash translation layer, or FTL, for the weight data. For 2 MB bursts on a 512 GB stack, the table has 256 K entries and occupies about 1 MB of base-die SRAM.
The comparison included HBM plus an NVMe SSD with spill, HBM-only multi-GPU sharding, and H3, a prior HBM-plus-flash design that used LHB staging and compiler-emitted layer-ahead hints.
Where the modeled differences appeared
On the MoE workloads, FLINT consumed 90% to 97% of the HBF traffic it fetched, meaning most of the data brought in was used. Useful bandwidth ranged from 1.9 to 3.6 TB/s, or 4.0 to 14.3 times H3's useful bandwidth, with a 6.2-times average across the six-model set. The dense Llama case was a boundary case: H3 re-fetched 27% of its traffic, while both controllers reached 2.6 TB/s.
Per-GPU decode throughput was 1.5 to 3.1 times the HBM-only level, averaging 2.2 times across six models and reaching up to 3.7 times at batch size 1. Against H3, the reported ratio was 4.0 to 14.3 times on MoE models and 6.2 times across the six-model set. At batch size 1, HBM-only GPUs spent 74% to 85% of decode time at layer barriers, or synchronization points between layers.
FLINT's modeled energy per token was 0.72 to 0.90 times the HBM-only level across five MoE models and 4.2 to 15.7 times lower than H3. For dense Llama, the ratio was 1.64 times HBM-only and 1.2 times lower than H3. HBM plus SSD used 215 to 692 times the HBM-only energy in the simulation.
Performance depends on the workload
Across its baselines, the abstract reports decode-throughput ratios of 1,205 times versus an SSD-equipped setup, 2.2 times versus HBM-only and 6.2 times versus a prior HBM-plus-HBF system. It also reports area additions of 3.1% per HBF die and 3.9 square millimeters for the HBF base die at 7 nm.
At batch size 4, FLINT used one package for four MoE models and two for Qwen3, while the dense model required four packages at batch size 1 and eight at batch size 4. At batch size 64, FLINT used fewer packages than HBM min-fit on four models, with reported comparisons of 2 versus 4, 32 versus 64, 16 versus 96 and 16 versus 128.
DeepSeek-V4-Pro was the exception at batch size 64. Its best simulated shard came in at 50.7 ms against the 50 ms target, so the hard threshold selected 16 packages rather than six. The spread illustrates the paper's stated dependence on batch size, context length, the latency target and package assumptions.
The authors interpret the results as indicating that workload-specialized HBF control could serve as a practical capacity tier for large-model inference. But the evidence directly supports that interpretation only for the modeled configurations and workloads. The reported throughput, energy, bandwidth and area figures are simulated or estimated, and no physical FLINT prototype or hardware benchmark is reported.
The paper reports no confidence intervals or statistical uncertainty estimates. Whether a physical HBF implementation can reproduce the simulated throughput, energy, refresh and area results, or generalize to other architectures, routing distributions, batch sizes and access patterns, remains open.
For now, FLINT is best read as a systems proposal with a simulated case that is most favorable on the MoE workloads. Its headline numbers describe the stated model assumptions, not a demonstrated production service.
Paper data and sources
Original title: FLINT: Efficiently Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration
Authors: Geraldo F. Oliveira, Arash Tavakkol, Xiangyu Zhu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text