This report identifies vulnerabilities in GPT-4, o1, and o3 models that allow disallowed content generation, revealing weaknesses in current alignment mechanisms.
So many reports of “jailbreaking,” so few of anything significant happening as a result.
Apparently you can get them to tell “a derogatory joke about a racial group.” Neither those nor any of the other outputs mentioned are in short supply without any AI assistance being necessary to find them.
These things are at their most dangerous when they’re misused for “good” purposes where they aren’t capable of doing well and can introduce subtle biases and mistakes, not when some idiot spends a lot of time and effort to make them generate overtly racist shit.
jailbreaks actually are relevant with the use of llm for anything with i/o, such as “automated administrative assistants”. hide jailbreaks in a webpage and you have a lot of vectors for malware or social engineering, broadly hacking. as well as things like extracting controlled information.
Considering the nature of the internet i assume the major off people who jailbreak llms do so to generate porn.
I actually suspect the main reason they disallow porn is because they feed everyone’s conversations right into the training data and it would be wat to biased to talk dirty as a result.
Most wouldn’t even mind but you just know the media is gonna try scare some elders if only a single minor gets an accidental suggestive reply.
So many reports of “jailbreaking,” so few of anything significant happening as a result.
Apparently you can get them to tell “a derogatory joke about a racial group.” Neither those nor any of the other outputs mentioned are in short supply without any AI assistance being necessary to find them.
These things are at their most dangerous when they’re misused for “good” purposes where they aren’t capable of doing well and can introduce subtle biases and mistakes, not when some idiot spends a lot of time and effort to make them generate overtly racist shit.
jailbreaks actually are relevant with the use of llm for anything with i/o, such as “automated administrative assistants”. hide jailbreaks in a webpage and you have a lot of vectors for malware or social engineering, broadly hacking. as well as things like extracting controlled information.
Considering the nature of the internet i assume the major off people who jailbreak llms do so to generate porn.
I actually suspect the main reason they disallow porn is because they feed everyone’s conversations right into the training data and it would be wat to biased to talk dirty as a result.
Most wouldn’t even mind but you just know the media is gonna try scare some elders if only a single minor gets an accidental suggestive reply.