logo

MIT's New Method Flags AI Models Trained on CASM Without Generating It

Posted by sdoering |2 hours ago |5 comments

liquidise 37 minutes ago

This reads like a precision vs recall problem.

If I say every model is trained on CSAM, I too will correctly identify 100% of the models that were. Says little about my false positive rate though.

yk 29 minutes ago

> The approach, detailed in a paper presented at the International Conference on Machine Learning, achieved 100% accuracy in identifying models specialized for CSAM generation.

I know some reasons for 100% accuracy in machine learning, first of all the test set leaking into training data. Or you just accept a silly high false positive rate.

When I was an admin I liked to joke that if you guarantee more than 5 nines, then you are an insurance company and you are planning to pay the penalty instead of actually fulfilling your promise, here the principle is probably the same.

gnabgib 18 minutes ago

What is this spam. No link to today's MIT article[0].. from which most of the sentences and all of the quotes are taken, or the one author Arxiv entry[1]. Quotes as if they talked to the writers.

[0]: https://news.mit.edu/2026/new-method-keeps-kids-safe-from-il... (https://news.ycombinator.com/item?id=48893301)

[1]: https://arxiv.org/abs/2604.25119

andy99 9 minutes ago

Obviously the 100% is wrong. The big concern with using this is there is no way to verify the model results. It’s basically making an assertion it can’t prove. Normally there is som accessible ground truth answer, but here it is illegal to actually prove the model can generate offending material (which is the whole point of this classifier) so there is no recourse and no explanation and no possibility of a human in the loop.

Worse, the ignorant will believe the 100% claim and equate a positive classification with truth.

I should add, it’s an interesting problem space for which there are no good solutions, unfortunately I don’t think this is very helpful and could cause a lot of problems when there are false positives.

16 minutes ago

Comment deleted

crest 39 minutes ago

This sounds a bit too good to be believable. Do they ask the reader to believe it's 100% reliable and their code magically just knows what CASM looks like without looking?!? All without a single working link to the original research. Sounds like the PR spin conman would push in the process of regulatory capture to force his future captive userbase be required by law to become a paying customer at any price or be branded a criminal (aka not rich and connected enough to rape kids). Sorry if I'm jaded.