We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program.
We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here.
Science
DeepSeek beats GPT 6 Sol on offensive cybersecurity benchmark
Vals AI found DeepSeek V4.1 Flash 1017 outperformed GPT 6 Sol on a multi-step offensive cybersecurity task. The benchmark uses simulated corporate networks to assess cyber-risk; they plan to release the full evaluation soon.
1.9Klikes
173reposts
597saves
286Kviews
- Made with
- DeepSeek V4.1 Flash 1017, GPT 6 Sol
- Posted
