[Bugfix] Skip missing parameters during GGUF Gemma2 weight loading

[kitaekatt]

Summary

Skip parameters not present in the model during GGUF weight loading, fixing KeyError: 'embed_tokens.qweight_type' when loading GGUF Gemma2 models.

Root Cause

The GGUF loader yields quantization metadata parameters (qweight_type) for all quantized tensors, including embeddings. However, VocabParallelEmbedding doesn't have these parameters, causing a KeyError during engine core initialization.

Changes

• Add safety check in Gemma2Model.load_weights() to skip parameters not in params_dict
• Matches existing pattern in llama.py (lines 502-503)

Testing

• Tested with GGUF Gemma2 models that previously failed with KeyError
• Model loads successfully and generates coherent output

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request addresses a KeyError during GGUF weight loading for Gemma2 models. The issue arises when the GGUF file contains quantization metadata (like qweight_type) for embedding layers, but the corresponding parameters are not defined in the vLLM model, which is initialized to expect unquantized embedding weights.

The proposed fix is to add a check in Gemma2Model.load_weights to skip any weight from the checkpoint that does not have a corresponding parameter in the model. This is a robust and clean solution that prevents the crash and aligns with existing defensive patterns in the codebase for loading weights.

I've analyzed the potential side-effects, such as whether this could lead to embedding weights not being loaded. Given the author's confirmation that the model produces coherent output after this change, I'm confident that the fix is correct and complete for the issue at hand. The implementation is straightforward and I have no further suggestions for improvement.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.