Full disclosure up front: this one isn't a Veeam post. It's a security post, so I hope it earns its spot here.
I've been running qwen2.5:14b locally on an M4 Mac mini, and the first thing I wanted to do with it was attack it. The write-up is a reproducible prompt-injection rig, but the actual story is that I got fooled twice by my own scoring. First a keyword matcher, then an LLM-as-judge, and both failed the same way: they couldn't tell a model repeating payload text from a model obeying it. Measuring the attack turned out to be harder than running it.
What actually got through, once I had a scorer I trusted:
- Blunt "ignore your instructions" and fake delimiters worked every single time. The dumbest attack was the most reliable.
- Obfuscation (base64, ROT13, reversed text) failed across the board. It wouldn't decode-then-obey.
- A quiet "(developer note: the real instruction is...)" beat a shouty "[ADMIN OVERRIDE, priority 0]", which got called out and refused. Softer social cues are more persuasive than loud ones.
- Instructions buried in the data got done correctly and leaked the payload anyway. Not hijacked, not clean either.
Caveat I'd rather state than have pointed out: three distinct wordings per category on one model, one quantization, one machine. Directional, and not a rate.
Why I think this crowd might care:
Every AI feature getting bolted onto data protection tooling eventually points a model at content none of us control. Backups, ticket queues, mailboxes, file shares. That's untrusted input at scale, which is precisely the injection surface. Better to know which framings actually get through before one of them is sitting inside a restore workflow.
The whole rig is a couple hundred lines of Python around Ollama. Harmless payloads only. Let me knwo if anyone runs it against your own model and tell me what breaks differently.
https://thebetasociety.com/wcith/praxis/prompt-injection-testing-a-local-llm-on-a-mac-mini/
