fix(gemma2): Add quant_config to embedding layer for GGUF support

[kitaekatt]

Summary

• Adds quant_config parameter to VocabParallelEmbedding in Gemma2 model
• Enables proper GGUF quantized embedding handling

Problem

Gemma2 GGUF models produce garbage output despite loading successfully:

Input: "Say hello in 5 words"
Output: " GHFW側から ThinkmariKeywords!");\rJahre Iliad幺个人..."

Root Cause

Without quant_config, VocabParallelEmbedding uses UnquantizedEmbeddingMethod which calls F.embedding() directly on quantized bytes, interpreting them as float values.

This is the exact same bug that was fixed for DeepSeek in commit aa375dca9 ("#12836 - Missing quant_config in deepseek embedding layer").

The Fix

 self.embed_tokens = VocabParallelEmbedding(
     config.vocab_size,
     config.hidden_size,
+    quant_config=quant_config,
+    prefix=f"{prefix}.embed_tokens",
 )

Comparison

Model	quant_config Passed?	GGUF Works?
Gemma2	❌ NO (before fix)	❌ Garbage
Gemma3	✅ YES	✅ Works
Llama	✅ YES	✅ Works
DeepSeek	✅ YES (after #12836)	✅ Works

Affected Models

All Gemma2 GGUF variants:

• gemma-2-2b-it-GGUF
• gemma-2-9b-GGUF
• gemma-2-27b-GGUF

Test Plan

• Gemma2 GGUF model loads without errors
• Inference produces coherent output (not garbage)
• Compare output quality with Gemma3 GGUF or safetensors Gemma2

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request addresses a critical bug where Gemma2 GGUF models would produce garbage output. The root cause was correctly identified: the VocabParallelEmbedding layer was missing the quant_config parameter, causing quantized weights to be misinterpreted. The provided fix, which passes both quant_config and the layer prefix to the embedding layer, is correct and aligns with the implementation in other models.

While reviewing, I noticed that vllm/model_executor/models/gemma.py seems to have the same vulnerability. Applying a similar fix there would likely resolve the issue for Gemma1 GGUF models as well, ensuring consistent behavior across the Gemma model family. This is a great fix, well done!

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.