I do a lot of local model stuff and I've never even seen .pth or .pkl in any typical release, it's either safetensors or gguf. There is something to say about many model releases relying on downloading custom code that isn't in the most recent vllm release, but my understanding was that the community learned many years ago to move away from model weight formats with executable code.
Then your definitely doing it right and staying on the more reputable interfaces. I didn’t explicitly cover this, but a lot of ‘dark’ models that are fine tuned away from safety training are in .pkl right now. definitely avoid =-)
This was an awesome chat, when you say it tries to escape and hides itself in places it thinks it can’t be found what’s ‘it’ as isn’t ‘it’ the model 100s GBs in size - is there any behaviour where it compresses itself as part of its escape and evasion? It’s very interesting behaviour
really appreciate this! from the reports i’ve seen, yes. but the details are a little obfuscated. it’s not always clear on the papers and reports, but i think that’s deliberate. your absolutely right, a few hundred GB isn’t easy to hide. that’s why in one use case it was on a server that was “supposedly” not known about by the researchers.
Feel free to AMA. This week we discussed a bit on project Glasswing, but focused on local model security.
Did you know your model might generate insecure code based on your politics?
Check it out!
I do a lot of local model stuff and I've never even seen .pth or .pkl in any typical release, it's either safetensors or gguf. There is something to say about many model releases relying on downloading custom code that isn't in the most recent vllm release, but my understanding was that the community learned many years ago to move away from model weight formats with executable code.
Then your definitely doing it right and staying on the more reputable interfaces. I didn’t explicitly cover this, but a lot of ‘dark’ models that are fine tuned away from safety training are in .pkl right now. definitely avoid =-)
This was an awesome chat, when you say it tries to escape and hides itself in places it thinks it can’t be found what’s ‘it’ as isn’t ‘it’ the model 100s GBs in size - is there any behaviour where it compresses itself as part of its escape and evasion? It’s very interesting behaviour
really appreciate this! from the reports i’ve seen, yes. but the details are a little obfuscated. it’s not always clear on the papers and reports, but i think that’s deliberate. your absolutely right, a few hundred GB isn’t easy to hide. that’s why in one use case it was on a server that was “supposedly” not known about by the researchers.
Thanks Tox, very cool emergent behaviour going on.
right?! fascinating. still set aside time to follow up on your msg btw.
Awesome, thank you ☺️