SNAPlab

Category: Tools

Film Archives as an Unforeseen Ally of Generative AI

In recent weeks, the boom in generative AI models has begun to shift from text and images to video. Anyone can create an eight-second video using Google’s Veo model. Open AI has announced the launch (currently only in North America and by invitation) of a video platform running on the Sora 2 model. In addition, other platforms are emerging that enable the generation of special forms of content, such as micro dramas.

In our family, video generation has become part of our daily bedtime routine. Every evening, the children simply dictate to me what they want their fairy tale to be about. Gemini first prepares the text for me, and after I finish reading it, we generate the video. I consider this to be one of the greatest advantages of generative AI, as the quality of the fairy tales does not depend on how tired I am.

The problem, of course, is that eight-second videos created in two minutes enchant us now, but in a moment we will consider them the norm and they will no longer be enough for us. And if I understand correctly, the problem for developers is still to keep the characters looking consistent between shots, or to ensure that the resulting videos follow basic cinematic continuity rules. This is not surprising, because AI must first learn to tell stories like filmmakers in order to match them, and it took filmmakers decades to do so. And developers could be helped by a traditional institution well known to film historians – the archive – with its tried and tested rules of film storytelling.

I realized this in connection with The Development of Generative Artificial Intelligence from a Copyright Perspective from May 2025, which was brought to my attention by my friend and lawyer Miroslav Obernauer. The creation of a license for training generative AI is being considered. This would solve the problem of the need to train AI on existing works on the one hand and copyright protection on the other.

And this is where there is room for film archives to become centers for training AI, in addition to their role of preservation, restoration, and dissemination.

Take, for example, the Národní filmový archiv in Prague. Archivists care for hundreds of films made in Czechoslovakia – films that could be used to train generative AI. And their collections include thousands more films from other countries around the world.

Of course, I don’t know if this will happen. But I like the idea that an institution such as an archive, which – let’s be honest – is not one of the driving forces behind the digital revolution and does not exactly come across as cool to the public, could find a new role and become integral part of the creation of modern technologies.

October 6, 2025

ai, archive, artificial-intelligence, cinematic continuity, film archive, technology
Futures Studio – Tour of the new virtual production studio at Royal Holloway

Today, I had the opportunity to attend a two-hour program at Futures Studio at Royal Holloway. It is a virtual production studio serving research and development for the audiovisual industry. The two-hour program flew by.

At the beginning, Peter Richardson talked about the main milestones in the development of special effects from Le Voyage dans la Lune (1902) through Modern Times (1936), Superman (1978) to Mandalorian (2019) and The Batman (2022). The aim was not just to show the effects in the films, but mainly to highlight the gradual improvement of processes, materials, and technologies. And everything led to… LED.

That’s it. Futures Studio. Equipped with an LED projection wall, light panels, a sound system, computers, and simply tools for virtual production.

And yes.

The studio allows the projection of George Costanza’s image.

May 16, 2025

film, industry, virtual production
The imperfect but amazing SAM2: AI video segmentation
As part of the AI bootcamp with Farshad, we tested models that enable photo and video segmentation. In this blog, I’d like to show Meta’s SAM2 application to two cases from the history of audiovisual culture where the model failed. At the same time, I want to demonstrate that far from criticizing SAM2, I find it a fascinating tool.

Before I go on to introduce the examples, I want to explain what is meant by segmentation. For film studies graduates, this term can be confusing because it means something different than what we are used to. In the context of AI analysis of photos and videos, it means entity selection within the photo and video. It is therefore a spatial segmentation that we can select as the area of interest a person, an animal, an object, but also houses, roads, trees or the sky. If we are talking about temporal segmentation – I was instructed by Farshad – we are talking about chapterization. As I wrote, this is unusual for a film studies graduate, but it makes sense.

Now let’s go to the examples. There’s no point in me trying to introduce Meta’s SAM2 model here, because this website does that quite successfully. When I tried the demo, I was naturally wondering how to fool the model. It’s not nice of me, but I couldn’t shake the impression that the sample videos in the demo are too easy to detect.

I knew I wanted to pick out some cases from the history of audiovisual culture, but which ones? What will show the AI segmentation capabilities? So I set some criteria:

– a black and white film

– poor quality video

– crowd scene

– shape change

I then considered the selection of a particular film and scene. In the list were crowd scenes from Soviet war movies, night scenes from horror movies, scenes with T1000 from Terminator 2… I also considered testing the twins (Social Network, Dead Ringers), but that ended up being very easy for SAM2.

In the end, I chose the chase scene from Seven Chances, where the main character is chased by a hoard of brides. The second demo was to include Odo, the shapeshifter from Star Trek: Deep Space Nine. I was very lucky, because these videos were available on youtube. Let’s take a look at the result.

In the chase demo, SAM 2 was tasked with tracking three selected brides. We can see that he was only 2/3 successful. While the orange and blue brides were tagged the entire time, a third green tag traveled between several brides.

As far as Odo is concerned, SAM2 only succeeded sometimes. Here in the demo, we can see that the tendency to stick to the shape of the original object prevailed. I was under the impression that SAM2 is more successful when Odo changes from or to human form, but I don’t have exact data on that.

But the biggest shock for me was in this clip. I accidentally picked out part of the following scene as well, and was surprised that SAM 2 was able to identify Odo even after the cut (in the last two images).
Now, I could go on to criticize SAM2 for not demonstrating a 100% success rate. I could also speculate that there is no substitute for genuine analysis done by a human. I could certainly also speculate on what AI analysis reports when it lacks subjectivity. I’ll leave all that to others.

I will confine myself here to amazement at what a computer model can do. Certainly it can’t yet replace humans, but I can imagine how it could enable us within film studies to analyse a much larger number of films in a short time than we are able to do now. (This was, incidentally, the first thing Peter Krämer replied to me when I emailed him.) I can also imagine quite well that AI will prepare the analytical basis, which the analyst will then work with further. Actually, I don’t see many reasons why we don’t use AI already and the ones I see don’t make much sense to me.
February 3, 2025

ai, analysis, Tools