South Korean Heejin’s Last Saturday, the phone received a Telegram message from an anonymous sender:
“Your photos and personal information have been leaked. Let’s talk.”
In a chat group on the Telegram app, Heejin noticed that his school-age face picture had been attached to the body of an unknown woman appearing in a porn video. Pictures and videos had been distributed on social media.
Heejin, who is studying at the university, told about his situation of the British broadcasting company BBC in the interview. Heejin is not her real name.
The extent of the phenomenon began to be revealed when the editor of the South Korean Hankyoreh newspaper Ko Narin began to investigate matter.
He found dozens of chat groups on Telegram where users shared photos of women and girls they knew and asked other group members to turn the photos into fake porn.
New pictures were added to the groups every minute, and it only took a few seconds to create a photo or video fake, i.e. deepfake, with the help of artificial intelligence. One of the groups had 220,000 members.
Individual universities, high schools and even middle schools had their own groups. In them, users were encouraged to share pictures of their fellow students. According to the reporter, some students had their own rooms, i.e. groups, if there was a lot of material.
In addition, the groups had so-called “humiliation rooms”, the description of which promised: “If you ask someone to be humiliated using a deepfake, we will do it”.
– I was shocked by how systematic the process was, says journalist Ko Narin, who investigated the case, to the BBC.
– The worst thing is that there were more than 2,000 members in a group for underage students at one school.
The phenomenon is also growing in Finland
The phenomenon is not new to Finland either, says a lawyer specializing in online harassment Kaisa Rissanen.
– We get more and more contacts because of fake porn images or videos using artificial intelligence, says Rissanen.
According to Rissanen, who heads the first aid unit of Someturva, which helps victims of social media, the cases in Finland are mostly about either bullying or blackmail with a fake pornographic image.
The most common case is extortion, or sextortion, where the subject is told that a deepfake nude image has been made of him.
– The youngest who have contacted have been around 10 years old.
Rissanen emphasizes that it is a crime in Finland to also distribute nude photos made with the help of artificial intelligence.
– And when we talk about minors, even taking or possessing pictures with your own device is a crime.
According to Rissanen, the ease of using deepfake applications can also lead to thoughtless actions. Often the perpetrator can also be a minor.
– Criminals also take advantage of this, as in the case of South Korea. There, in the groups, schoolchildren or students were encouraged to share pictures of their schoolmates.
Deepfake porn has the same sense of shame as rape
In South Korea, the scandal has caused anxiety for many teenagers and young women.
A South Korean university student interviewed by the BBC says that he has begun to doubt almost all of his male acquaintances.
According to lawyer Kaisa Rissanen, fake photos made with deepfake are associated with a very similar feeling of shame and anxiety as, for example, rape.
– Spreading a pornographic image or video causes long-term stress about where the image has ended up, who has seen it and who will still see it, describes Rissanen.
According to Rissanen, the South Korean case is unprecedentedly extensive.
– There have been cases in Spain and the United States that concern one or two schools, but I have not come across such an extensive case yet.
Rissanen sees the rapid development of easy-to-use artificial intelligence applications as the reason.
Another factor, according to Rissanen, is specifically Telegram. In the application, you can create huge groups where videos are shared.
– Telegram is a private, encrypted messaging application with loose moderation. This makes it possible to distribute illegal content there.
Users are often anonymous, rooms can be set to “secret” mode and their contents can be deleted quickly without a trace.
This has also made Telegram suitable for criminal activities.
Women delete their pictures from social media
In South Korea, more than 500 school and university groups were found on Telegram by last weekend, where students have been encouraged to share pictures of women and girls.
The full number of cases is yet to be ascertained, but many of the victims are believed to be under 16, which is the legal age for sex in South Korea. Also, a large part of the suspected perpetrators are teenagers.
Scores of women and teenagers have since removed their photos from social media or disabled their accounts altogether, fearing they could be misused.
The case has shocked South Korea.
The country’s police announced on Monday that they are considering opening a criminal investigation into Telegram with the help of French authorities. French authorities are accusing Telegram’s Russian founder, Pavel Durov, of crimes related to the app, including enabling the distribution of child pornography.
Telegram tells the BBC that it “actively combats harmful content, including illegal pornography”.
Many of the chat rooms exposed by the media in South Korea have been closed. However, since then, new ones have already taken their place.