Little recourse for teens girls victimized by AI 'deepfake' nudes
Category: News & Politics
Via: perrie-halpern • 11 months ago • 59 commentsBy: Melissa Chan and Kat Tenbarge
S E E D E D C O N T E N T
Teenage girls in the U.S. who are increasingly being targeted or threatened with fake nude photos created with artificial intelligence or other tools have limited ways to seek accountability or recourse, as schools and state legislatures struggle to catch up to the new technologies, according to legislators, legal experts and one victim who is now advocating for a federal bill.
Since the 2023 school year kicked into session, cases involving teen girls victimized by the fake nude photos, also known as deepfakes, have proliferated worldwide, including at high schools in New Jersey and Washington state.
Local police departments are investigating the incidents, lawmakers are racing to enact new measures that would enforce punishments against the photos' creators, and affected families are pushing for answers and solutions.
Unrealistic deepfakes can be made with simple photo-editing tools that have existed for years. But two school districts told NBC News that they believe fake photos of teens that have affected their students were AI-generated.
AI technology is becoming more widely available, such as stable diffusion (open-source technology that can produce images from text prompts) and "face-swap" tools that can put a victim's face in place of a pornographic performer's face in a video or photo.
Apps that purport to "undress" clothed photos have also been identified as possible tools used in some cases and have been found available for free on app stores. These modern deepfakes can be more realistic-looking and harder to immediately identify as fake.
"I didn't know how complex and scary AI technology is," said Francesca Mani, 15, a sophomore at New Jersey's Westfield High School, where more than 30 girls learned on Oct. 20 that they may have been depicted in explicit, AI-manipulated images.
"I was shocked because me and the other girls were betrayed by our classmates," she said, "which means it could happen to anyone by anyone."
Politicians and legal experts say there are few, if any, pathways to recourse for victims of AI-generated and deepfake pornography, which often attaches a victim's face to a naked body.
The photos and videos can be surprisingly realistic, and according to Mary Anne Franks, a legal expert in nonconsensual sexually explicit media, the technology to make them has become more sophisticated and accessible.
A month after the incident at Westfield High School, Francesca and her mother, Dorota Mani, said they still do not know the identities or the number of people who created the images, how many were made, or if they still exist. It's also unclear what punishment the school district doled out, if any.
The Town of Westfield directed comment to Westfield Public Schools, which declined to comment. Citing confidentiality, the school district previously told NBC New York that it "would not release any information about the students accused of creating the fake nude photos, or what discipline they are facing."
Superintendent Raymond Gonzalez told the news outlet that the district would "continue to strengthen our efforts by educating our students and establishing clear guidelines to ensure that these new technologies are used responsibly in our schools and beyond."
In an email obtained by NBC News, Mary Asfendis, the high school's principal, told parents on Oct. 20 that it was investigating claims by students that some of their peers had used AI to create pornographic images from original photos.
At the time, school officials believed any created images had been deleted and were not being circulated, according to the memo.
"This is a very serious incident," Asfendis wrote, as she urged parents to discuss their use of technology with their children. "New technologies have made it possible to falsify images and students need to know the impact and damage those actions can cause to others."
While Francesca has not seen the image of herself or others, her mother said she was told by Westfield's principal that four people identified Francesca as a victim. Francesca has filed a police report, but neither the Westfield Police Department nor the prosecutor's office responded to requests for comment.
New Jersey State Sen. Jon Bramnick said law enforcement expressed concerns to him that the incident would only rise to a "cyber-type harassment claim, even though it really should reach the level of a more serious crime."
"If you attach a nude body to a child's face, that to me is child pornography," he said.
The Republican lawmaker said state laws currently fall short of punishing the content creators, even though the damage inflicted by real or manipulated images can be the same.
"It victimizes them the same way people who deal in child pornography do. It's not only offensive to the young person, it defames the person. And you never know what's going to happen to that photograph," he said. "You don't know where that is once it's transmitted, when it's going to come back and haunt the young girl."
A pending state bill in New Jersey, Bramnick said, would ban deepfake pornography and impose criminal and civil penalties for nonconsensual disclosure. Under the bill, a person convicted of the crime would face three to five years in jail and/or a $15,000 fine, he said.
If passed, New Jersey would join at least 10 other states that have enacted legislation targeting deepfakes, according to Franks, a law professor and the president of the Cyber Civil Rights Initiative, a nonprofit group that combats nonconsensual porn.
The state laws targeting deepfakes vary widely in scope. Some of them, like ones in Texas and Wyoming, make nonconsensual pornographic deepfakes a criminal violation. Other states, like New York, have laws that only allow victims to bring forward a civil suit.
Franks said the laws are "all over the place," noncomprehensive, and the constitutionality of the laws has been called into question.
"So you've got a patchwork of criminal charges, which are going to be difficult in these cases because the perpetrators are going to be minors, so that raises its own questions," she said.
'Probably just the tip of the iceberg'
It's unclear how many young people have been victimized by AI-generated nudes.
The FBI said it is difficult to calculate the number of minors who are sexually exploited. But the agency said it has seen a rise in the number of open cases involving crimes against children. There were more than 4,800 cases in 2022, which grew from more than 4,100 the year before, the FBI told NBC News.
"The FBI takes crimes against children seriously and works to investigate the facts of each allegation in a collective effort with our state, local, and tribal law enforcement partners," the agency said, adding that victims can face significant challenges when trying to stop the spread of the image or get it removed from the internet.
Franks said there are likely a lot more incidents and that they will only increase.
"Whatever we're hearing about that floats up to the surface is probably just the tip of the iceberg," she said. "This is probably happening quite a bit right now, and girls just haven't found out about it yet or discovered it or the school is covering it up."
At Issaquah High School in Washington state, a school district representative said a mid-October incident "involving fake, AI-generated imagery of students" continues to affect the student body.
In the Spanish town of Almendralejo, mothers say dozens of their middle school-aged daughters have been victimized with AI-generated nude photos created with an app that can "undress" clothed photos. Local police in New Jersey, Washington and Spain are investigating the high school cases.
In a June public service announcement, the FBI warned that technology used to create nonconsensual pornographic deepfake photos and videos was improving and being used for harassment and sextortion.
Meanwhile, the National Association of Attorneys General called on Congress in September to study AI's effects on children and come up with legislation that would protect them from those abuses.
In a letter signed by 54 state and territory attorneys general, the group said it was concerned that "AI is creating a new frontier for abuse that makes prosecution more difficult."
"We are engaged in a race against time to protect the children of our country from the dangers of AI," the letter said.
Francesca and her mother said they plan to head to Washington, D.C., in December to personally urge Congress members to act, as they continue to advocate for updated policies within the school system and seek accountability for what happened.
"We all know this is not an isolated incident," Dorota Mani said. "It will never be an isolated incident. This is going to keep happening all the time. We have to stop pretending that it's not important."
The rise in incidents targeting high school girls follows the proliferation of AI deepfake apps and deepfake porn websites where such material is created, shared and sold.
A 2019 report from Sensity, an Amsterdam-based company that tracks AI-generated media, found that 96% of deepfakes created at that point were sexually explicit and featured women who didn't consent to their creation. Many victims are unaware the deepfakes exist.
Franks said there is nothing parents and children can do to prevent the creation of deepfakes using their likenesses. Instead, Franks said schools and local law enforcement need to make an example out of perpetrators in cases that reach the general public, to discourage others from creating deepfakes.
"If you could imagine a dramatic and important response from the school in New Jersey or from the authorities in New Jersey to make an example out of the case, really strict penalties, people go to jail, you might get the discouragement," Franks said.
"In the absence of that, it's just going to become one more tool that men and boys use against women and girls to exploit and humiliate them and that the law basically has nothing to say about."
Tags
Who is online
399 visitors
Humanity is a constant tug of war between the forces that lift up humanity and those that drag it down. It has probably always been that way but technology has vastly accelerated the effects.
Anything uploaded to the Internet is likely to remain there forever, or at least a very long time. It's not too difficult for bad people to manipulate and use those images for nefarious purposes.
It should absolutely be criminal and with a 15+ year sentence. This is basically digital child trafficking.
My girls are not allowed on social media specifically because of this kind of shit.
I’m not defending this but that’s a bit extreme. Punish the people who make the software. Teenagers do stupid shit, but imprisoning them and ruining their life for manipulating photo is far stupider. And your kids don’t need to be on social media to become a victim. All someone needs is their photo.
This is not just "stupid shit" to their victims.
This is not only their victim's current reputation, but will have lasting lifelong consequences for the feelings of self-worth, self-esteem and confidence.
This is psychological warfare by people who need a lesson on boundaries. The lesson needs to have severe enough penalties to mean that it will not be tolerated by society.
So if your son did a stupid thing like hit “enter” on a program that converts a girl’s innocent photo into a naked photo and posting it for laughs amongst his horny teenager friends, you’d be okay with him going prison for fifteen years and having no ability get a job upon release. You might want to rethink that.
Y'Think?? Wow. People need some perspective.
That's the thing. People forget that boys are kids, too.
I had teenage daughters. I would be fine with sending a POS with no respect for my and everyone else's daughters to prison for demeaning them in this manner.
However, I would be fine with a mandatory sentence of 5 years to begin with.
If I had had sons, and had not been able to teach them better respect for females, then I would have been okay to have allowed them to face the consequences of their disgusting and unacceptable treatment of their classmates/acquaintances.
This is not "stupid shit". They chose their targets in order to demean them. Anyone who sees their videos can't unsee them and it will have lasting effects on the treatment and personality of the target.
I would have also testified against my daughters for doing the same type of harm against others - male or female.
Some people have boundaries and some people have few, if any, boundaries. That is why it is impossible for the two groups to socialize or develop any type of meaningful relationship at any age.
I feel sorry for any girl who even dates a boy who would ever do this to anyone for any reason.
Boys are "kids" when it comes to unacceptable behavior? Really?
At what age are they taught boundaries and expected to be respectful of others? 10? 20? 30? Ever?
I have a grandchild that exposed her breasts to a boy online as a teenager. It was a big fucking deal to every adult associated with these two kids. How much prison time should the teenagers serve?
If she was never told that what gets posted online is always gonna be there, then the question becomes how much time should the "adults" serve..
That is not even in the same galaxy as a boy (or girl) taking a girl's picture and inserting it on a person in a porn film.
Your grandchild chose to expose her own breasts. While inappropriate, she chose to demean herself for reasons that she may or may not understand herself. I hope she and the boy received the needed counseling on age appropriate relationships. If your grandchild was stalking the boy, that is another matter entirely and might should have resulted in a restraining order and even more necessary counseling before the situation escalated to the point incarceration was necessary to protect her target.
I feel like you truly don’t understand kids, and if you were her mother you’d be pushing for someone to go to prison over it. They’re kids, they DO do stupid shit, and it’s not automatically some adults’s fault. She’s also an exceptional gymnast that will probably be in the Olympics someday, provided some overbearing adult doesn’t force her into prison for making some stupid kid mistake.
Please. Now you’re sounding like mocowgirl. Parenting is not an exact science, and parents are not the automatic scapegoat for bad kid behavior. If that were true any angry, ornery child could easily have their parent jailed by just acting out.
Thanks, I think she is a very logical person.
Sure, and yet parent's are ultimately responsible for their children aren't they?
Hmmmm? Is that where this leads us?...
You're entitled to your feelings. They are not factual. I understand kids just fine. That is why I taught mine to be individuals first and to protect themselves from the ones who would harm them or lead into places that they might never recover from. Because I raised them in hometown, I knew who they needed to avoid most of the time.
So you completely disregarded my response.
Amen. In eighth grade, one of my classmates exposed his penis to me when the teacher was out of the room. I called him a little bastard and reported him to the teacher after the bell rang. They made him get me out of my next class. He did not know I had already told exactly what he had done and what I had said about it. So he tried to "bargain" with me and said he wouldn't tell I had called him a bastard if I would not tell what he had done. I laughed. He got his ass busted. I don't recall ever having another problem with him for the next 3 years we were in school together.
Nope. But, an ounce of prevention is worth a pound of cure.
Who paid for and gave the phone to use to send pictures of her breasts? I would say that adult could be at fault.
Did a parent ever talk to her about the reasons she should not send nude pictures of herself to anyone?
Did a parent ever talk to her about her biological urges for sex vs a males biological urge for sex?
Again, you completely ignored my comment or can't comprehend it.
Either way, I wish your granddaughter all the best in gymnastics and life.
When it comes to 15-year prison sentences, absolutely.
I had a teenage son and a daughter. Perspective.
Prison. For what? Embarrassing your daughter? That's not batshit or anything.......
You would have a different perspective.
So out of curiosity, if some 13 yr old kid is good with a pencil and he draws a picture with a classmate's face on the topless body of Wonder Woman and passes it around to his buddies... it he going to prison, too? Or is this just a digital crime?
I don't consider projecting a teenage girl as a porn star "embarrassing", I consider it child trafficking/child porn and is illegal. I don't give a tinker's damn of the age of the perp who does it. Also, I consider a teenager, who has the skills to manipulate images in this way to probably be of average intelligence at the very least and therefore should know this is completely unacceptable behavior. If they don't, then jail/prison would definitely be the only place they belong.
Prison, no? Counseling on boundaries and in school, detention for a month? Definitely.
So out of curiosity, is your son obsessed with drawing his classmate's faces on topless bodies of grown women?
How do you know so much about teenage boys' obsessions with demeaning their female classmates and why do you find it acceptable?
So out of curiosity, is there an age that boys mature beyond this mindset or is that how they also spend their adulthood?
Society always makes excuses for teenage boys. If my son had ever done something like this I would have knocked him into next week
The men/women, who support/condone this behavior, are responsible for making our society unsafe for all of us.
This is why you, and people with your attitude, will always have my respect and be considered treasured friends regardless of minor disagreements elsewhere.
This is why people, like your son, would have been acceptable friends/dates for my daughters.
Why thank-you, Mocowgirl!
My son is now 29 and is serving in the Air Force. He always respected his sister and they have a very close relationship today even tho they don't get to see each other much.
Thank you for being you, TG!
I don't do sentiment well.....and especially not on an open forum, but you are one of the reasons I still log on here.
“You would have a different perspective.”
Absurd, apologetic, and apathetic toward understanding the world girls have to navigate. My two sons were never allowed to excuse their behavior with entry into the vapid ‘boys will be boys’ club. (My least favorite phrase of all time) That ‘perspective’ may be foreign to too many and it will only change when strong and powerful women are heard and respected by secure and empathetic men.
Read carefully. I have a daughter, too.
Great. So you would support them going to prison for over a decade for sharing an altered photo of a person who poses nude for money? Really?
That's longer than an average armed robbery sentence, BTW.
Or maybe... just maybe... we could muster some common sense and understand that there are other options between "boys will be boys" and "throw away the key".
Riiiight.
Tell us again what part of that teenage girl's body is on display? Oh, that's right... her face. The same face she shows on Instagram, Twitter, or Snapchat. The same face she shows off at school every day to anyone in the building.
Yeah, yeah. We get it. You're committed to your irrational views. Never mind that nearly every other law in every other instance takes the offender's age into account. None of that matters if girls are embarrassed.
So the computer nerd goes to prison while the art nerd goes to counseling. Right. Sure. That's not inconsistent or anything.
Do you even begin to understand the problem here?
I spent 30 years as a coach.
Please indicate where I said it was acceptable.
Are you incapable of understanding the vast gulf between "acceptable" and "15 years in prison"? You seemed to manage that idea with the art nerd.
Some grow out of it. Some men stay puerile. They're not unlike girls in that regard.
But you're missing 90% of this issue because you can only see it as though your daughters are potential victims. What if your daughter made the image? It's exceedingly well documented that the vast majority of people who demean women are actually other women. It's easier for them because there is no accountability.
So what if one of your daughters had a momentary episode of "mean girl" and put her crush's girlfriend's face on a pornstar? Do we lock her up till she's 33? Really?
Let me help you. Bullshit. We don't. As a society, we are wise enough not to buy the emotional bullshit you are trying to push here.
We pretty much exempt teenage girls from accountability to begin with, so we don't need to make excuses for them. Girl dad speaking from experience.
Well said.
If your son had done this to my daughter, I would have knocked him into next week.
But I would NOT have tried to have him thrown away so that he spent all of his 20's in prison. That's just crazy.
Do cite someone supporting or condoning the behavior.
All I see are people pushing back on your [deleted] stance that this is a prison worthy crime. [deletedc]
[deleted]
What you are missing is that I am female and don't share much of my life experience from being raised in a society with a "boys will be boys" attitude and the damage it caused me and my peers as teenagers and throughout our lives. Still does. That is why "the boys will be boys" crowd view women as only incubators for their sperm who should not have sex unless they also want to be incubators for their offspring.
If my daughters were manipulating porn videos on the web, I would consider that they were a complete waste of oxygen - seriously! There are so many wonderful things for them to do in life- I have never considered porn to be one of them.
As far as my daughters "demeaning" other women as children, teens and now adults? I gave you my answer on porn, so give me another scenario if you want my answer.
I will repeat - if either of my daughter's was manipulating porn, I would consider them a complete waste of oxygen. I am not sure that they would ever have a worthwhile purpose in society since their attitude was to harm others by the time they reached their teens - and probably before if a person was honest about it.
Some people have an obsession with porn and nude bodies that I don't have. Before our children were born, my husband used to bring home Playboy and Penthouse articles. I looked at the pictures and read the articles.
As an adult, I have watched porn. Mostly disgusting. Nothing there that I find mentally or physically healthy for anyone - especially teenagers.
I could probably Google for studies that show the damage that porn causes, but none of that matters to any person who finds this type of behavior any kind of normal or acceptable.
You are incapable of understanding the harm to the victim - so be it.
There is something seriously wrong with the emotional development of boys who hate girls on this level.
It’s not just these boys, it’s the adult SW developers that build this weaponized SW. It’s not duel use technology misused, the developers know what they are building and don’t care.
Yes, it is.
Emotionally immature, irresponsible, deviant and all kinds of toxic adult who is probably a porn addict and possibly a pedophile porn addict.
Raping child videos are big business on the dark web. We damn sure don't need to tolerate this behavior where teens can view it freely or where their victims have it following them every day for the rest of their lives.
I’m probably too old to understand how people that earn their living making such a damaging product can live with themselves.
Again, it’s not like their product is being misused, it’s being used exactly as they intended. We don’t have the metrics to measure the harm, especially to young woman, but I imagine it’s horrendous and growing.
Here's an article about a 14 yr old girl who was kidnapped, videoed being raped and then having her classmates view the rape video on the net.
We can have the metrics to measure the harm - all our society has to do is listen to the women, support the women and develop a zero tolerance for abuse.
It won't take long to separate the women haters from the women lovers.
There are all kinds of reasons why some people develop a hatred for women.
But the most toxic I have found are the ones looking for a mother replacement to abuse. The development of romantic relationship to one of mother/child takes months (even years). I have lived it and did not even see it for what it was until now. These days it is just creepy to even talk with my husband because his responses have been childlike for so long that I used to consider them "normal". They would be "normal" if he was 4 years old - maybe.
I assure you that is exceedingly clear.
I refuse to believe you lack the intellectual wherewithal to envision a middle ground between cavalier disregard and long-term prison sentences. So I am actually quite confused as to your insistence that anyone who pushes back on 15-year prison terms for teenagers is somehow indifferent to any wrongs they may have done. They're not indifferent, they're just being reasonable.
The punishment you support is massively overboard. It's just nonsense. We don't treat kids that way unless they kill somebody.
You conspicuously dodged the 15-year prison term question. The double standard is deafening.
You didn't answer the first one. If your daughter didn't make 8th grade cheer captain and out of spite created a porn-fake of the girl who did, would you still advocate a 15-year sentence?
I sure as hell hope not. I sure as hell hope that scenario causes you to rethink this position in a more rational light.
Here is the part you won't like: Our boys are not lesser beings than your girls. They're children, just like that 8th grade girl is.
Pretty much every teenage boy. It's chemistry. That chemistry is actually vital to the survival of the species.
This is what the victim will face from the "boys will be boys" putting their faces on porn videos.
As I stated elsewhere, 5 years in prison would be the beginning penalty for demeaning anyone in this way.
MORE from above...
I said 5 years should be the beginning to set an example that this is a serious crime and will not be tolerated.
It has lifelong consequences for the victim.
I understand that you are tripling down on your firm belief that girls' feelings are more important than boys lives.
But no mention of the girls who hate other girls and do all sorts of thoroughly nasty shit. The double standards you have are galactic.
5 years - same as for anyone else.
I am sure that I would want her back in my house ever because she would be someone I never really knew in the first place if she was that pathetic.
Who said they are?
The rules should be the same for everyone.
No special pleading that "boys will be boys".
I will reiterate.
The rules should apply to everyone.
The penalties should be the same.
You are talking about a girl who was actually raped.
Do you pretend you don't understand the difference between that scenario and the one we're describing?
For a 13-year-old. That's just insane.
As is this.
Who, besides you, is using that phrase?
There will be no difference to the shaming the girl receives when the boys and parents watch her being fucked on a porn site.
It is just insane that a 13 year old would not be spending time with friends doing things that make life better instead of trying to destroy classmates.
He sounds like someone who has enough hatred of others to be nothing but dangerous so is probably not very well-liked.
It is beginning to sound more like a profile of a Nicholas Cruz or Ethan Crumbley personality.
First....Watching her actually get raped vs watching an entirely different person....and you don't think there is a difference?? Are you listening to yourself?
I'm guessing you're not aware that this has been going on for years with celebrities. So you're trying to tell us all that a real video of real Emma Watson really being raped is no more traumatic to her than a fake video with her face pasted on a porn star.
Are you sure this is the answer you want to go with?
Second... and here is the real problem... you are advocating prison solely for causing a girl to feel a certain way. There is just more wrong with that than we can describe, but we'll just sum it all up right here...
Girls' feelings are not more important than boys' lives.
Don't project female thinking onto boys. There is a very good chance a group of these guys would be giggling their asses off as they do this together, or one kid is working the software while his buddies are egging him on with "do Madison" or "do Hailey" or whoever the pretty girls are.
Boys don't generally think about "destroying" people, and if they do, it's physical, not reputation. That's how girls think.
It's not about hatred. That's how girls think.
No. Again. Just no. Those guys are out torturing the neighbor's cat.
And I would have stopped my husband from knocking you into next week. He is an overprotective/indulgent father. I was the disciplinarian when they were growing up.
Girls get slut shamed...boys...not so much
Why? You have argued that this is normal behavior and the girl is not a really a victim. All she had to worry about is her feelings.
And yet, now you are wanting to assault someone else's child? Why? Is it because of your feelings?
Where is the logic in assaulting a teenager for hurting your feelings?
So incarceration and counseling is unacceptable, but physical assault is okay?
We have laws to protect teenage boys from vigilante violence from adult males for this exact reason - it is vigilante violence and illegal.
I am with you.
We must have enforceable laws to protect our society from descending into vigilante violence.
Castration does not keep rapists from harming/killing others. Violence is not the answer. If it was, we would not need prisons and mental hospitals to protect us from people with little to no boundaries.
The root causes must be identified and addressed. And the people, hardwired for violence, must be dealt with in as humane ways as possible, but separated from society until they can live peacefully with others and obey society's boundaries.
Brain scans may be a beneficial tool that allows better identification of what the best way to treat the person with no boundaries and to protect society.
The religious minded can be an impediment because some believe the person is demon possessed when it really amounts to brain damage that could stem from utero and the brain never develops properly.
Totally agree.
"Franks said there is nothing parents and children can do to prevent the creation of deepfakes using their likenesses. Instead, Franks said schools and local law enforcement need to make an example out of perpetrators in cases that reach the general public, to discourage others from creating deepfakes."
Public execution would be a good start !!!
Seriously?