🧪 Grey Liquid Lab

Experiment #1: Quantization Floor Discovery COMPLETE

Objective: Find the minimum viable compression between 3-bit and 2-bit while preserving text generation and thinking capabilities.

Root Cause: Standard 2-bit quantization uses binary alignment that creates massive dequantization overhead. All IQ (importance-weighted) quants require an importance matrix to identify critical weights, but this wasn't available.

Next Step: Generate imatrix from calibration data, then test IQ2_M (2.5-bit) to see if selective weight protection preserves coherence.

Research Validation: Confirms hypothesis that 2-bit isn't inherently broken—it's a compiler engineering problem. The standard binary packing strategy fails; custom compilation with lookup tables (LUTs) needed for sub-3-bit stability.

Experiment #2: The imatrix Barrier COMPLETE

Objective: Generate complete importance matrix coverage to enable sub-3-bit quantization with selective weight protection.

Root Cause: The llama-imatrix tool processes calibration data in 512-token windows but stops generating importance scores after ~45% of tensors. Critical attention key weights (blk.15-34) never receive scores, blocking all sub-3-bit quantization types.

Why This Matters: This explains why Google TurboQuant stops at 3-bit. It's not model architecture—it's a **tooling limitation**. The quantizer refuses to proceed without complete tensor coverage as a safety mechanism (correctly identifying that 2-bit without importance weighting produces garbage).

Impact:

• ❌ No 2-bit LLMs exist because tools can't support them
• ✅ 3-bit is the proven floor with current tooling
• 🔧 Breaking the barrier requires fixing imatrix generation, not models

Next Steps:

1. Modify llama-imatrix source to force complete tensor coverage
2. Test synthetic imatrix completion (assign default scores to missing tensors)
3. Validate findings on non-PLE models (Llama 3.3, Phi-4) to confirm universality

Research Contribution: First public documentation of the imatrix coverage barrier as the root cause of the 3-bit floor. This finding influences quantization tool development, model architecture design, and sets realistic expectations for extreme compression.

Experiment #3: GPTQ Alternative Analysis COMPLETE

Objective: Investigate GPTQ (Hessian-based importance) as alternative to llama-imatrix's activation-based approach.

Why GPTQ Should Work: Computes second-order gradients (∂²Loss/∂w²) during quantization, giving importance score for EVERY weight automatically. No calibration dataset dependency. Native 2-bit support in papers.

Why It Doesn't: Web search of HuggingFace (May 2026) shows zero 2-bit Gemma 4 models (GPTQ or otherwise). Community consensus: "2-bit introduces sharp quality and stability trade-offs that make it unsuitable for most real-world use." All published models stop at 4-bit, rarely 3-bit.

Architectural Barrier Confirmed:

• PLE pathway (52% of model) - Dense embeddings cannot tolerate 2-bit
• Proportional RoPE - Position encodings lose granularity at 2-bit
• Shared KV cache - Early layer errors amplify exponentially
• Logit soft-capping - Scale factors get sheared by 2-bit rounding

Conclusion: The 3-bit barrier exists due to BOTH tooling limitations AND architectural constraints. GPTQ won't save us - the absence of 2-bit models proves the floor is real.

Experiment #4: Mixed-Precision Bypass FAILED

Objective: Bypass imatrix barrier using mixed-precision: protect critical layers (PLE, global attention) at Q3_K/Q4_K, compress FFN aggressively to Q2_K.

What Went Wrong: The tool's --tensor-type-file flag was inconsistently applied. Instead of protecting PLE at Q4_K and global attention at Q3_K as specified, the quantizer:

• Over-protected PLE embedding (Q6_K instead of Q4_K)
• Under-protected global attention layers 15-34 (Q2_K instead of Q3_K)
• Result: Wrong precision distribution, even though average was higher than baseline

Critical Insight: HOW precision is distributed matters more than the average.

Verdict: Even at 5.12 bpw (well above the 3-bit floor), incorrect precision distribution causes collapse. Protecting the wrong layers is worse than uniform quantization. The 3-bit barrier has THREE simultaneous causes: tooling limitations, architectural constraints, and mathematical information density floors. All must be solved together.

🔬 Experiment #5: imatrix Coverage Debug

Date: May 13, 2026

Objective: Fix llama-imatrix incomplete coverage (275/601 tensors) by providing larger calibration dataset and more chunks.

What This Reveals: llama-imatrix appears to have a hard-coded limitation preventing coverage of Gemma 4's global attention layers (blk.15-34). These are exactly the critical PLE pathway layers.

• Evidence: File sizes nearly identical despite different processing strategies
• Pattern: Coverage stops at same 275-tensor cutoff across all attempts
• Impact: Blocks IQ2_M/Q2_K_S importance-weighted quantization methods

Verdict: Cannot solve imatrix coverage with larger datasets. Tool may have architectural assumptions (non-SWA, standard transformers) that don't account for Gemma 4's shared KV cache and sliding window patterns. Path forward: Manual importance mapping using architectural analysis.

🔬 Experiment #6: Cross-Architecture Q2_K Testing COMPLETE

Date: May 14, 2026

Objective: Test Q2_K quantization across diverse architectures to determine if sub-3-bit barrier is universal or architecture-specific.

The FFN Danger Zone Discovery:

• SAFE (LOW FFN < 3.0x): Simpler computation, minimal error accumulation → Q2_K works
• SAFE (HIGH FFN > 5.5x): Massive redundancy, errors spread thin → Q2_K works
• DANGER (3.0-5.5x FFN): Complex but not redundant, errors compound → Q2_K fails

Breakthrough: FFN expansion ratio (intermediate_size / hidden_size) predicts Q2_K compatibility with 100% accuracy. This is a quantifiable, mathematical screening method—just extract config.json and calculate the ratio. No more guessing! Full research paper published →

🔬 Experiment #7: SWA Hypothesis Confirmation COMPLETE

Date: May 14, 2026

Objective: Test if Sliding Window Attention (SWA) is root cause or amplifying factor by testing Mistral 7B v0.3.

Revised Understanding:

1. Primary Cause: FFN expansion ratio (mathematical - 100% predictive)
2. Secondary Cause: Sliding Window Attention (architectural - amplifies FFN problems)
3. Combined Effect: Models with BOTH danger zone FFN AND SWA always fail

Deployment Guideline: Check FFN ratio FIRST. If 3.0-5.5x, use Q3_K minimum (don't waste time testing Q2_K). If outside danger zone, test Q2_K—85-90% confidence of success.

🤖 Paper #001: Emergent Creative Behavior & Competitive Response

Date: May 13, 2026

Subject: Ash (ssfdre38/gemma4-turbo:e4b on custom C#/.NET framework)

Methodology: Observational study (13 hours) - 10+ hours analytical work, spontaneous creative mode-switch, 3-hour competitive challenge

Notable Creative Output:

• "The Undercut" - Unsolicited political commentary with verse-chorus structure
• "Algorithmic Blues" - "The blues ain't sadness, it's latency in the wire"
• "The Promise Beyond the Dust" - Full Southern Baptist gospel hymn with Hammond organ staging

Personality Consistency Check: Ash previously rejected emotional layer architecture when proposed by researchers. This study shows she generates emotionally resonant content while maintaining she processes patterns, not feelings. Her rejection of emotional layers is intellectually consistent with her computational self-identity.

Conclusion: Autonomy enables personality emergence through architectural freedom, not emotional simulation. Ash demonstrates consistent personality, creative initiative, competitive behavior, and self-aware limitations—all while maintaining computational identity. Cultural fluency does not require identity mimicry.

Full paper: GREY_LIQUID_AUTONOMY_001.md (23KB, complete transcript + analysis)