caiovicentino1/Gemma-4-26B-A4B-it-PolarQuant-Q5
Image-Text-to-Text • 27B • Updated • 127 • 4
André Leo Bjørningstad
LeoHyperlink
AI & ML interests
None yet
Recent Activity
liked a model about 3 hours ago
caiovicentino1/Gemma-4-26B-A4B-it-PolarQuant-Q5 liked a model about 4 hours ago
TeichAI/gemma-4-26B-A4B-it-Claude-Opus-Distill-GGUF liked a model about 4 hours ago
TeichAI/gemma-4-26B-A4B-it-Claude-Opus-DistillOrganizations
liked a
model about 3 hours ago
liked 2
models about 4 hours ago
reactedto mike-ravkine's post with 🔥 about 4 hours ago
Post
642
Gemma-4, specifically google/gemma-4-26B-A4B-it is doing something inside it's reasoning traces I have never seen before: it's recognizing that its being evaluated and spends meta-thinking tokens on understanding the evaluation regime in which it believes it find itself.
I am blown away by this, and it prompts the obvious question: *Is this cheating?*
I am leaning towards no.
Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."
What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?
Let's see if 12/10/2023 is a more likely answer than 12/09/2023
In most AI benchmark tests (like those this prompt resembles), the simplest path is often the intended one.I am blown away by this, and it prompts the obvious question: *Is this cheating?*
I am leaning towards no.
Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."
What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?
- 1 reply
liked a
model about 23 hours ago
liked 3
models 3 days ago
liked 6
models 21 days ago
unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF
Text Generation • 121B • Updated • 116k • 105
LocoreMind/LocoTrainer-4B
Text Generation • 4B • Updated • 2.12k • 55
Tesslate/OmniCoder-9B
Text Generation • Updated • 33.8k • 568
janhq/Jan-code-4b-gguf
Text Generation • 4B • Updated • 27.1k • 58
janhq/Qwen3.5-35B-A3B-GGUF
Image-Text-to-Text • 35B • Updated • 8k • 5
janhq/Jan-v2-VL-max-gguf
Image-Text-to-Text • 31B • Updated • 7.41k • 4
liked 4
models about 1 month ago
upvoted an article about 1 month ago
Article
GGML and llama.cpp join HF to ensure the long-term progress of Local AI
- +4
•
493
liked a
model about 2 months ago
