Follow-Up Research · June 2026 · Continuation of Part 1
A second adversarially-verified research pass targeting only primary sources — peer-reviewed papers, arXiv preprints with auditable methodology, and official documentation — in response to the four open questions left unanswered in Part 1.
This is a follow-up to Part 1. The first research pass confirmed one architectural fact and killed 24 claims from grey literature. This pass targeted the four unanswered questions using only arXiv preprints with explicit methodology, peer-reviewed papers, and official documentation. The result: 6 confirmed findings, 2 questions partially answered (Q1, Q3), and 2 questions that remain genuinely unanswered at the primary-source standard (Q2, Q4). The findings are detailed by question below.
Two peer-reviewed or formally published studies were found and verified. Neither covers the specific consumer GPU tiers most practitioners care about (RTX 3060, 3090, 4090 under realistic concurrent workloads) — but they represent the best available primary-source evidence.
No peer-reviewed paper providing reproducible throughput benchmarks specifically for consumer GPU tiers (RTX 3060, 3090, 4090) under realistic multi-user concurrent workloads was found and verified. The XDA Developers paper (arXiv:2410.04466, "Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective") was fetched and processed but yielded claims that failed 1–2 verification — the specific RTX 3090 and M2-Ultra figures could not be independently corroborated. The Argonne repository is the recommended starting point for practitioners who need reproducible data.
Five cost-model papers were found, fetched, and verified. All five failed adversarial verification at 0–3 votes. The one surviving cost-related finding reframes why break-even calculations are inherently difficult.
Every specific cost break-even figure encountered in this research pass was killed. This includes two arXiv preprints (2509.18101 and 2601.09527) that presented auditable frameworks with explicit hardware prices and API rate assumptions. The figures could not be independently corroborated. The underlying methodological problem — that GPU utilization rate is either a fixed assumption or silent 100% in every public LLM cost calculator — was flagged by one paper (2606.11690) but that paper's own numerical claims also failed verification.
The three most specific papers (2509.18101, 2601.09527, 2606.11690) all present numerical claims that verification agents could not corroborate from independent sources. The most likely explanations: (1) figures are derived from pricing snapshots that change faster than the papers' update cycle, (2) hardware cost assumptions (e.g., "RTX 5090 at $2,000") rely on MSRP figures disputed by market pricing, and (3) GPU utilization assumptions baked into cost-per-token calculations are not independently auditable. The methodology gap identified by 2606.11690 — that utilization is systematically underspecified — appears real even if that paper's own numbers didn't survive.
Two papers survived adversarial verification. Both are arXiv preprints without confirmed peer review, but both have explicit, auditable methodology and were verified directly against their published tables. Both apply to LLaMA-family models only — no verified data was found for Mistral, Phi, or Gemma.
Taken together with the Part 1 confirmed finding about GGUF architecture, the verified evidence points to: Q3 and below produce measurable quality loss on reasoning tasks; Q4 is the practical trade-off point where compression is substantial and quality loss is moderate; the 4-bit tier is method-dependent (AWQ vs. OmniQuant diverges at 3-bit); and the Q8 ≈ F16 claim — perhaps the most repeated assertion in local LLM discussions — could not be confirmed from any primary source in either research pass.
No verified empirical data covers quantization effects on Mistral, Phi, or Gemma model families at any level. The two surviving findings apply exclusively to LLaMA-family models. Practitioners deploying Mistral or Phi variants locally have no peer-reviewed primary-source data on quality degradation at their target quantization level.
This was the most thoroughly searched angle and returned zero verified claims. The search covered EMNLP 2024 and 2025 industry tracks, NeurIPS proceedings, arXiv enterprise deployment papers, and company-authored technical reports. The result is not a search failure — it reflects a genuine gap in the primary literature.
No enterprise deployment of a local LLM with a verifiable organizational name, specific use case, and measurable outcomes was confirmed. Several sources came close: the WhatsCode paper (arXiv:2512.05314) describes a named deployment at WhatsApp over 25 months generating 3,000+ accepted code changes — but it is a cloud-based system, not a local LLM deployment. A claim about SkillsFuture Singapore achieving 62.5x faster case processing using GraphRAG with a local model was extracted from an EMNLP industry track source but failed adversarial verification (could not independently confirm the 62.5x figure or the "local" specificity). A "regulator-approved retail banking AI" appeared in one paper but the organizational name was withheld.
There are structural reasons why primary-source evidence for enterprise local LLM deployments is sparse. Organizations in regulated industries — the most likely adopters of local deployment for compliance reasons — have strong incentives not to disclose their AI infrastructure. Conference paper authorship in industry tracks often anonymizes or omits the deploying organization. Vendor case studies (which do name organizations) are systematically grey literature and do not meet a primary-source standard. The gap is real.
Practitioners seeking enterprise deployment examples should search: EMNLP and ACL industry tracks (2024–2025), procurement notices on SAM.gov and similar government portals for public-sector AI infrastructure, and corporate SEC filings and earnings calls for named AI deployment disclosures. These sources are below the primary-source threshold used here but are more likely to contain named organizational data than academic preprints.
The following claims were extracted from primary or near-primary sources but failed adversarial verification. Several are directionally plausible — they failed because independent corroboration was not located within the verification pass, not necessarily because they are false.
| Vote | Claim | Source |
|---|---|---|
| 1–2 | RTX 4090 + AWQ INT4 increases Llama-2-7B from 52 t/s to 194 t/s — the only specific consumer-GPU throughput figure found in a primary source | arXiv:2410.04466 |
| 1–2 | RTX 3090 running Llama2-7B with mixed 2/4-bit quantization achieves 45.2 t/s — the only RTX 3090 figure found in a primary source | arXiv:2410.04466 |
| 0–3 | Apple M2-Ultra running Llama2-7B at 4-bit via llama.cpp achieves 6 t/s single-core and 32 t/s with eight cores | arXiv:2410.04466 |
| 0–3 | Q8_0 achieves near-identical downstream task performance to F16 across MMLU, HellaSwag, TruthfulQA, and GSM8K (widely-held belief, failed from this paper's own data) | arXiv:2601.14277 |
| 0–3 | Q4_K_M causes a 6.58-point IFEval drop vs. F16 — instruction-following disproportionately sensitive to 4-bit quantization | arXiv:2601.14277 |
| 0–3 | Sub-30B on-premise deployment breaks even vs. commercial APIs in 0.3–3 months | arXiv:2509.18101 |
| 0–3 | RTX 5090 delivers 411 t/s for a RAG-8k workload on Qwen3-8B NVFP4 at concurrency 8 | arXiv:2601.09527 |
| 1–2 | Every public LLM cost calculator treats GPU utilization as fixed input or silent 100% assumption — identified as the critical methodological gap in cost modeling | arXiv:2606.11690 |
| 0–3 | 2-bit quantization causes complete language-capability collapse regardless of model size (AWQ on LLaMA-65B: perplexity 7.4×10⁴) | arXiv:2502.13178 |
Two passes of adversarial research have narrowed the unknowns. The questions below are now sharply scoped — they are not general inquiries but specific gaps in the primary literature that practitioners should be aware of when making infrastructure decisions.
R1 What reproducible, peer-reviewed throughput benchmarks exist for consumer GPU tiers (RTX 3060, 3090, 4090) under realistic multi-user concurrent workloads? The two surviving Q1 findings (PowerInfer, Argonne) cover batch-size-1 and enterprise hardware respectively. The consumer-GPU concurrent-request scenario — the most common real-world deployment pattern — has no verified primary-source benchmark.
R2 Does the 3-bit quantization cliff observed for OmniQuant on LLaMA3-8B generalize to Mistral, Phi, and Gemma families? Every verified quantization finding in both passes applies exclusively to LLaMA-family models. Practitioners deploying other architectures have no primary-source quality degradation data at any quantization level.
R3 Given that cloud API price-performance drops 5–10× per year, what utilization rate and workload volume make on-premise deployment economically rational — and what methodology produces a utilization-adjusted break-even that survives adversarial verification? All five cost-model preprints failed; the utilization-underspecification critique (arXiv:2606.11690) appears valid but its own numbers also failed.
R4 What primary-source evidence exists for named enterprise local LLM deployments? The EMNLP and NeurIPS industry tracks were searched and produced zero verified claims meeting the standard. This is a structural gap: regulated-industry adopters have the strongest incentives to deploy locally and the strongest reasons not to publish about it.
Key Sources