this post was submitted on 21 Nov 2024
158 points (97.6% liked)

Technology

72894 readers
2548 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

Today, a prominent child safety organization, Thorn, in partnership with a leading cloud-based AI solutions provider, Hive, announced the release of an AI model designed to flag unknown CSAM at upload. It's the earliest AI technology striving to expose unreported CSAM at scale.

top 50 comments
sorted by: hot top controversial new old
[–] TheHobbyist@lemmy.zip 135 points 7 months ago* (last edited 7 months ago) (2 children)

Thorn, the company backed by Ashton Kutcher and which tried to get its way to monitor all messages in the EU via Chat Control. No thanks.

https://fortune.com/europe/2023/09/26/thorn-ashton-kutcher-ylva-johansson-csam-csa-regulation-european-commission-encryption-privacy-surveillance/

[–] Erasmus@lemmy.world 69 points 7 months ago (2 children)

Just remember folks. Kutcher is a slimeball too.

The guy went from a D list star and hanging out with the likes of Danny Masterson and going to Diddy’s infamous parties - to suddenly overnight courting the US government and being the face of ‘helping’ children everywhere.

Yeah right…..

[–] chonglibloodsport@lemmy.world 24 points 7 months ago (2 children)

I’d be wary of calling him guilty by association. Maybe when he realized who he was really hanging out with he was so horrified and disgusted that he just had to get involved and do something to fight back?

[–] Erasmus@lemmy.world 8 points 7 months ago (1 children)

It’s awful coincidental that he seems to hang out with the ‘rapist’ crowd. Even going as far as writing a letter for Masterson as to how nice of a guy he is to try to get him a lenient sentence.

Even Hollywood has ostracized him and his wife - news sites recently reported they were looking to leave the country and let things cool off for a while.

I’m sure everyone is right though that keep posting here, that he is a swell guy who was just in the wrong place at the wrong time, multiple times. Several years worth of multiple times with wrong people. Just a coincidence.

[–] BassTurd@lemmy.world 16 points 7 months ago

The difference between us giving him a benefit of the doubt and claiming innocence and your take, is that you are labeling him a pedophile without proof. That's a significant claim if false, and imo takes an assumption too far. Maybe he's bad and it should be looked into, but saying he did something because he was on a show with and good friends with a guy that happened to be a rapist is wrong.

load more comments (1 replies)
[–] ninekeysdown@lemmy.world 22 points 7 months ago (1 children)

People can grow and change. Not saying he did or didn’t. Just saying that people aren’t a monolith. It’s plausible he just grew and his views changed / evolved.

That being said, it’s highly convenient where he’s positioned himself these days…

[–] sunzu2@thebrainbin.org 3 points 7 months ago

Regime whores are all about proximity

[–] sunzu2@thebrainbin.org 2 points 7 months ago

Has he ever called put the Catholic church or does he only care about pictures of abuse but not actual abuse?

[–] sunzu2@thebrainbin.org 52 points 7 months ago (2 children)

I am a bit confused how it is legal for them to have the training data here?

Like is there anything a corpo can't do?

Like why can't subway Jared and Catholic church "train the AI"

Only half way joking, what's the catch here?

[–] MentalEdge@sopuli.xyz 36 points 7 months ago (1 children)

There are laws around it. Law enforcement doesn't just delete any digital CSAM they seize.

Known CSAM is archived and analyzed rather than destroyed, and used to recognize additional instances of the same files in the wild. Wherever file scanning is possible.

Institutions and corporation can request licenses to access the database, or just the metadata that allows software to tell if a given file might be a copy of known CSAM.

This is the first time an attempt is being made at using the database to create software able to recognize CSAM that isn't already known.

I'm personally quite sceptical of the merit. It may well be useful for scanning the public internet, but I'm guessing the plan is to push for it to be somehow implemented for private communication, no matter how badly that compromises the integrity of encryption.

[–] melroy@kbin.melroy.org 18 points 7 months ago (1 children)

So doesn't that make the law enforcement having the biggest CP collection from everybody? This sounds kinda dangerous...

[–] MentalEdge@sopuli.xyz 26 points 7 months ago* (last edited 7 months ago)

It does. Kinda.

The police are seldom allowed to be in possession of CSAM, except for in terms of grabbing the hardware which contains it in an arrest. The database used in modern detection tools is maintained by NCMEC which has special permission to do so.

And of course there are risks, but it's just digital data. Unless you are creating more, you're not actively harming anyone. And law enforcement absolutely needs that data to take some of the most obvious steps to prevent it being spread further.

Obviously, someone has access, but to get to the actual media files wouldn't be simple. What typically happens, is that anyone wanting to detect CSAM, is given a hashed version of the database. They can then scan their systems for CSAM by hashing any media they are hosting, and seeing whether there are any matches.

Whenever possible, people aren't handling the actual media. But for any detection to be possible to begin with, the database of the actual media does need to be maintained somewhere.

AI is a touchier subject, as you can't train a model to recognize CSAM not already in the database using hashes, so in those cases you have to work with actual real media. This is only recently becoming a thing.

It also leaves open the possibility for false positives. An oft cited example is parents taking pictures of their own children for innocent reasons, or doctors and parents handling images for valid medical reasons. In a system that flagged such content, it would mean someone else would be seeing that "private" content because it was flagged.

load more comments (1 replies)
[–] db0@lemmy.dbzer0.com 44 points 7 months ago* (last edited 7 months ago) (1 children)

It's the earliest AI technology striving to expose unreported CSAM at scale.

horde-safety has been out for a year now. Just saying... It's not a trained AI model in this way, but it's still using Neural Networks (i.e. "AI Technology")

[–] qaz@lemmy.world 8 points 7 months ago* (last edited 7 months ago) (1 children)
[–] db0@lemmy.dbzer0.com 8 points 7 months ago

haha, nah people reported some unexpected censors, and we investigated what part of their prompt might be causing it.

[–] Wes4Humanity@lemm.ee 25 points 7 months ago (1 children)

Man... That AI is going to be so fucked up when it gains sentience

[–] Blackmist@feddit.uk 7 points 7 months ago (1 children)

Skynet's real origin story. We might just deserve judgement day.

load more comments (1 replies)
[–] hendrik@palaver.p3x.de 23 points 7 months ago (4 children)

And will we get that technology to keep the Fediverse and free platforms safe? Probably not. All the predecessors have been kept away for sole use of the big players, despite populism always claiming we need to introduce total surveillance to keep the children safe...

[–] riskable@programming.dev 14 points 7 months ago (1 children)

I was going to say... Sure would be nice to have this feature in all the open source AI image generator tools but you're absolutely right 😩

[–] hendrik@palaver.p3x.de 11 points 7 months ago

Yeah, unless someone publishes even a set of hashes of known bad content for the general public... I kind of doubt the true intentions are preventing CSAM to the benefit of everyone.

[–] db0@lemmy.dbzer0.com 4 points 7 months ago (1 children)

IFTAS is already working with Thorn towards this goal. But you already have access to such technology through my toolset.

[–] hendrik@palaver.p3x.de 2 points 7 months ago* (last edited 7 months ago) (3 children)

This one? I loosely followed your work... Maybe I should try it someday. See how it does on a regular VPS. Thanks for the link to the IFTAS. Seems they have curated some useful links... I'll have a look at their articles. Hope they get somewhere with that. At this point, I don't think there is any blocklist accessible to the average Fediverse admin?!

Edit: Thx, saw your other comment with the link to horde-safety.

load more comments (3 replies)
load more comments (2 replies)
[–] floofloof@lemmy.ca 13 points 7 months ago (5 children)

This seems like a potential actual good use of AI. Can't have been much fun to train it though.

And is there any risk of people turning these kinds of models around and using them to generate images?

[–] Jimbabwe@lemmy.world 24 points 7 months ago (6 children)

If AI was reliable, maybe. MAYBE. But guess what? It turns out that “advanced autocomplete” does a shitty job of most things, and I bet false positives will be numerous.

[–] Chozo@fedia.io 12 points 7 months ago

This is not that kind of AI.

It's possible to have a good AI system, but it takes millions of dollars and several thousand manhours to do, and most companies won't put in the effort.

But, there should always be a human in the loop.

load more comments (4 replies)
[–] FaceDeer@fedia.io 14 points 7 months ago

And is there any risk of people turning these kinds of models around and using them to generate images?

There isn't really much fundamental difference between an image detector and an image generator. The way image generators like stable diffusion work is essentially by generating a starting image that's nothing but random static and telling the generator "find the cat that's hidden in this noise."

It'll probably take a bit of work to rig this child porn detector up to generate images, but I could definitely imagine it happening. It's going to make an already complicated philosophical debate even more complicated.

[–] mspencer712@programming.dev 8 points 7 months ago

I think image generators in general work by iteratively changing random noise and checking it with a classifier, until the resulting image has a stronger and stronger finding of “cat” or “best quality” or “realistic”.

If this classifier provides fine grained descriptive attributes, that’s a nightmare. If it just detects yes or no, that’s probably fine.

[–] catloaf@lemm.ee 7 points 7 months ago

Nobody would have been looking directly at the source data. The FBI or whoever provides the dataset to approved groups, but after that you just say "use all the images in this folder" and it goes. But I don't even know if they actually provide real full-resolution images, or just perceptual hashes, or downsampled images.

And while it's possible to use the dataset to generate new images assuming the training data had full-res images, like I said, I know they investigate the people making the request before allowing access. And access is probably supervised and audited.

[–] Hoimo@ani.social 3 points 7 months ago

Available image generators are already capable of generating those images and they weren't even trained on it. Once a neural network can detect/generate two separate concepts, it can detect/generate the overlap. It won't be as fine-tuned obviously, but can still turn out scarily accurate.

[–] sexual_tomato@lemmy.dbzer0.com 12 points 7 months ago* (last edited 7 months ago) (2 children)

Jesus Christ. If someone ever got their hands on this model they could use it to generate new material. The grossest possible AI model to date

load more comments (1 replies)
[–] Nurse_Robot@lemmy.world 9 points 7 months ago (1 children)

This is a great development, albeit with a lot of soul crushing development behind it I assume. People who have to look at CSAM or whatever the acronym is have a miserable job, so I'm very supportive of trying to automate that away from people.

load more comments (1 replies)
[–] Churbleyimyam@lemm.ee 2 points 7 months ago (2 children)

I think all CSAM should be destroyed out of respect for the victims, not proliferated. I don't care who is hanging onto this material or for what purpose.

[–] Ghostie21@lemmy.world 15 points 7 months ago (2 children)

How is this proliferating csam? Also, how do you expect them to find csam without having known images? It gives a really nice way to check based on hashes without having someone look at every picture on someone's harddrive. With this AI it should greatly help determining new or unknown images while minimizing the number of actual people that have to see that stuff, and who get scarred from looking at such images. The only reason to be against this is if you are looking at CP and want it to be harder to find, or if you don't understand how this technology is being used.

load more comments (2 replies)
[–] sunzu2@thebrainbin.org 5 points 7 months ago

This ain't about the victims... It never was, otherwise churches would NOT exist in current form.

This is about police and corpo state gaining power.

load more comments
view more: next ›