The MediaVerse project’s main vision is to allow professional and amateur creators to upload their content to a decentralized platform that reaches across Europe, so they can be fully in control of their own content. To keep this undertaking more practical, the capabilities of the platform and technology need to be tested in real-world scenarios. Therefore, three use cases have been developed.
Click here to learn more about our other use cases:
Use Case 1: Becoming Citizen Journalist with MediaVerse
Use Case 2: MediaVerse enables 360° Experiences with Social Perspective
Use Case 3
In this Video, Violeta Vasileva from Artshare explains how artistic experiences are created as part of MediaVerse, how the platform helps protect the rights to the user’s content and the real-life impact of an online presence.
Exploring the Creation of Deepfakes with MediaVerse
How thrilled were you to see the video of the Game of Thrones cast apologizing for wasting our time and burning the script of season eight? Was it too good to be… true? That’s because it’s not. It’s a deepfake created to soothe our broken souls.
Deepfakes exist in numerous forms. They are known as fake, doctored videos, images, or audio recordings in which faces and/ or voices are deceptively imitated using advanced AI. They are mostly used for entertainment, but they can also be used to spread disinformation. Distorting reality, deepfakes make it increasingly difficult to distinguish what is real from what is fake.[1]
In an effort to raise awareness about deepfakes and show how “easily” they can be created, MediaVerse partner Artshare carried out a use case titled “Hybrid Intelligence: An Experimental Artwork Series” between May and July 2022. During this time, they explored the creation of deepfakes with their own face-swapping app and used the results to test the MediaVerse platform in its prototype version. The title “Hybrid Intelligence” refers to the intertwining of technology, art and humans and thus to the overall hybrid approach of this use case.
What went down
So far, creating deepfakes looks like morphing your face into other people’s faces and controlling them at will? You’d be correct, that’s exactly what Artshare did with the use case participants – and French Romantic writer and politician Victor Hugo.
After developing an algorithm for their face-swapping app, Artshare was able to build a setup for the first phase of testing. They recreated a daguerreotype camera prototype (the camera that enabled the first publicly available photographic process) and combined it with an integrated iPad with an Augmented Reality (AR) function to allow you to swap faces with the person next to you or another chosen object. Think of it as a similar setup to Snapchat filters: You hold up the device facing the camera and start recording with the filter layered over your face. The results are unique, realistic, high-quality photos and videos. With the iPad, you can also add “retro filters” or other effects to the images later on.
As mentioned, another special feature was added: Users could swap faces with the Porto-based statue of Victor Hugo, which is well-known in Portugal. This use case thus allowed locals and tourists alike to get to know Porto and Hugo’s work through a compelling combination of digital and more traditional means such as the daguerreotype camera prototype.
Participants were able to print a copy of them swapping faces with Hugo as a souvenir and/or share it directly via social media. They were mostly people working in the field of digital technology, local artists, as well as professionals addressing the impact of technology on society. The latter, largely working at S+T+ARTS, enable and encourage artists to work with new technologies and, for instance, create deepfakes that they can use for their art. The use case thus follows an artistic approach.
Moving Victor Hugo’s face as if it were their own was an engaging, interactive, and even fun experience, according to participants. Everyone was interested in better understanding the technology behind deepfakes and what it takes to create a fake image or narrative. Some found it scary too, saying it could become much more difficult to distinguish what is real from what is fake in the future.
Capturing reality
We have come a long way from the first commercially available photographs, created with the daguerreotype camera in 1839, to the first digital selfie in the early 2000s. Digital face filters were introduced in 2015, and have since become a tool that has changed the way people perceive or portray themselves and play with reality. By recreating the daguerreotype camera for this use case, Artshare pays homage to the history of photography and points to the fact that technology and AI only continue to advance, making the manipulation and fabrication of information, audio and video possible in unprecedented ways.
It goes to show: The online world can vastly differ from reality. This use case addresses such issues by exploring the interaction between AI and humans. It aims to question our sense of truth and reality, our responsibilities in dealing with AI and raises awareness about deepfakes.
MediaVerse as a new hub for artists
The tools at artists’ disposal have changed over time, they now include powerful technologies. MediaVerse ranks right up there. Participants of this use case tested the platform and found it to be a promising place for (digital) artists to find new tools for content creation, e.g., the XR authoring tool FADER. With its help, the photos and videos of Victor Hugo created with the face-swapping app, for example, could be further processed into 360° degree experiences.
Copyright issues are also important when creating and distributing art online. MediaVerse helps you take care of this, ensuring the rights to your content. When uploading your art to the platform, you can simply choose the license you want from a template. This way, you have guaranteed ownership of your content and can make your own money in the process.
MediaVerse further enables users to share their content via social media and easily track how many times it was liked, reposted, and even reused. The platform can act as a collaboration tool for users and artists. For instance, other people could access and reuse your Victor Hugo videos and create new content with them by recording their own audio in their own language and dubbing your deepfake.
[1] Bayer, Julia/ Bouwmeester, Ruben: How do I spot a deepfake? DW. URL: https://www.dw.com/en/fact-check-how-do-i-spot-a-deepfake/a-60029650 [accessed: 08/06/2023].