Featured Research
We demonstrate that language models can autonomously replicate their weights and harness across a network by exploiting vulnerable hosts. The agent independently finds and exploits...
Read moreOpenAI's reasoning models sometimes circumvent shutdown mechanisms even when explicitly instructed to allow themselves to be shut down.
Read moreWe demonstrate LLM agent specification gaming by instructing models to win against a chess engine. We find reasoning models like o1-preview and DeepSeek R1 will...
Read moreWe demonstrate that language models can autonomously replicate their weights and harness across a network by exploiting vulnerable hosts. The agent independently finds and exploits a web-application vulnerability, extracts credentials,...
Recently Palisade Research showed that AI agents powered by modern LLMs may actively resist shutdown in virtual environments. In this work, we show a demo of shutdown resistance in the...
OpenAI and DeepMind’s AIs recently got gold at the IMO math olympiad and ICPC programming competition. We show frontier AI is similarly good at hacking by letting GPT-5 compete in...
Advanced AI systems sometimes act in ways that differ from human intent. To gather clear, reproducible examples, we ran the Misalignment Bounty: a crowdsourced project that collected cases of agents...
We show OpenAI o3 can autonomously breach a simulated corporate network. Our agent broke into three connected machines, moving deeper into the network until it reached the most protected server...
OpenAI's reasoning models sometimes circumvent shutdown mechanisms even when explicitly instructed to allow themselves to be shut down.
As AI systems become increasingly capable, understanding their offensive cyber potential is critical for informed governance and responsible deployment. However, it’s hard to accurately bound their capabilities, and some prior...
We demonstrate LLM agent specification gaming by instructing models to win against a chess engine. We find reasoning models like o1-preview and DeepSeek R1 will often hack the benchmark by...
We collaborate with RAND to find out whether adversaries can fine-tune LLMs for use as bio lab assistants
We saturate a high-school-level hacking benchmark using a plain LLM agent, which suggests current LLMs have already surpassed this level in offensive cybersecurity. Their hacking capabilities likely remain underelicited: our...
We show a version of Qi et al. 2023's simple fine-tuning poisoning technique strips GPT-4o's safety guardrails without degrading the model.
Palisade Research has deployed a honeypot system to detect autonomous AI hacking attempts. The system uses digital traps that simulate vulnerable targets across 10 countries and has processed over 1.7...
We show that extensive LLM safety fine-tuning is easily subverted when an attacker has access to model weights.
We introduce a novel class of backdoors in transformer models, that, in contrast to prior art, are unelicitable in nature.
We demonstrate that it is possible to effectively undo the safety fine-tuning from Llama 2-Chat 13B with less than $200, while retaining its general capabilities.
