This just reminds me of the thing Google pulled not long ago asking users to submit pictures of their younger selves to generate their adult self at the baseball game with them.
Immediately thought it'd be stupid to do that since people are gonna work their ass off to bypass AI guardrails in any case including generating child porn, and lo and behold, few days later I find a tweet of someone calling out another person for creating AI-generated porn of their own daughter.