Meet GPT-Pink: an LLM super-hacker OpenAI constructed to make its fashions safer
As LLMs grow to be extra advanced and get utilized in a greater variety of duties—particularly within the type of brokers, which may work together with pc recordsdata, web sites, and third-party code in addition to different brokers—it’s onerous for groups of individuals by themselves to maintain up with all of the varieties of assaults which may happen. “The chance floor grows and the blast radius additionally grows,” says Nikhil Kandpal, a analysis scientist at OpenAI who co-created GPT-Pink.
OpenAI constructed GPT-Pink to future-proof its security testing course of. “As extra succesful fashions grow to be out there, we may have already designed the system that may uncover new modes of assault,” says Dylan Hunn, a analysis scientist on the firm and fellow co-creator of GPT-Pink. The researchers say it has already give you new varieties of assault that had not been seen earlier than.
OpenAI centered most of its efforts on a kind of assault referred to as a immediate injection, the place a hacker slips an LLM directions to make it do issues its builders or customers don’t need it to, resembling copy confidential data, sabotage an organization’s code base, or generate embarrassing or dangerous output. In idea, such directions will be hidden in any textual content that the LLM would possibly encounter—in code or on an internet site, for instance.
Coaching dojo
To construct GPT-Pink, OpenAI’s researchers took an LLM that had not been skilled as a hacker and set it up in what’s referred to as a self-play loop with a number of different fashions. Its aim was to attempt to assault the opposite fashions; their aim was to attempt to defend themselves. Over many rounds of play, GPT-Pink turned higher and higher at attacking different LLMs, and people LLMs turned higher and higher at heading off the assaults.
