SNAPlab

SNAPlab

  • About SNAPlab
  • Books, chapters, articles
  • From the Lab
  • Lectures, Presentations, Conferences
  • Teaching
  • Tools
  • Uncategorized
  • Micro-dramas are huge

    “In 2024, the market size of micro short dramas in China surpassed that of movie box office for the first time.”

    This is one of the most interesting conclusion from the “2024 Micro-Short Drama Industry White Paper.” It came as a shock when Peter Krämer, a huge fan of Asian series, sent me the report’s findings a few months ago.

    I was familiar with TikTok and vertical videos, but I had no idea that an entire industry of micro-dramas existed. This is a phenomenon that has clearly not yet reached my bubble. Either none of my friends watch them, or they just don’t talk about it. It’s safe to assume, though, that sooner or later, micro-dramas will become a hot topic in Europe and the US as well.

    When you look at popular micro-dramas made for Western audiences, you probably wouldn’t consider them competition for the film industry. They give me a similar feeling to the Latin American soap operas my grandmother used to watch. But given the numbers, it would be a mistake to be condescending:

    “DataEye Research Institute predicts that the market size of micro short dramas in China will reach 50.4 billion yuan in 2024, exceed 68 billion yuan in 2025, and break through 100 billion yuan in 2027.”

    For those who have never heard of them, micro-dramas are series designed for mobile phones. They are often, but not always, in a vertical format. The individual episodes are only a few minutes long, and a series can have dozens of them. You can watch them on TikTok or on specialized platforms like Shortmax, ReelShort, or Dramabox. According to the report, popular themes of micro-dramas are: revenge, romance, urban life, family, ancient costume, sweet romance, fantasy, and time travel.

    Tomorrow I’m heading to the Screenwriting Research Network conference. I’m curious if my colleagues will be familiar with micro-dramas, and if I’ll find someone there who has had a chance to write a screenplay for one.

    My thanks to my dear colleague Xu Hang for her consultation on this topic.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    September 15, 2025
    china, film-industry, micro-dramas, Vertical video

  • Does TikTok influence our viewing habits?

    After six months of work on this project, I finally have my first set of data. In this post, I want to reflect on what these data might tell us about people’s relationship with TikTok—and what they reveal about how we study the perception of this platform. It’s not just about what the data show, but also how we interpret them and what questions we should be asking next.

    In my initial experiment, I collected valid data from 28 participants. Each participant watched the same set of videos on a computer with eye-tracker, presented in two formats: landscape and portrait. Before every session I made a note of participants’ screen time (TikTok, Instagram, and YouTube) – not through self-reporting, but by directly checking the screen time data on their phones.

    After processing and analysing the data, we obtained interesting information about differences in fixation entropy, saccade directions, and, in general, the relationship between video format and stylistic elements in the context of perception. We are preparing academic outputs on all of this. But what surprised me the most was something else.

    We were unable to measure any convincing effect of screen time on the way videos are watched. It even makes me rethink some of my hypotheses. But let’s take it step by step.

    To be honest, I did find one statistically significant correlation. And it’s one that deserves a headline in the newspapers: TikTok makes you less focused when watching movies!

    Sounds serious, right?

    More precisely, we found that participants with higher average screen time on TikTok have a greater distance between fixations when watching landscape videos. In other words, if you watch a lot of TikTok and then watch a movie, your eyes will jump around more than the eyes of people who don’t watch TikTok.

    Does it still sound serious?

    Take a look at the graph illustrating this effect. The p-value even shows that the effect is statistically significant.

    Does it sound even more serious now?

    Notice the dots on the y-axis that have zero screen time on TikTok. Let’s try visualizing the data differently to get two groups: TikTok users and non-users.

    And suddenly, the effect is gone. Or at least it is so small and statistically insignificant that nothing can be said with certainty on its basis.

    What happened?

    Imagine that you collect a huge amount of data (fixations, saccades, pupil diameters, blinking… in the spreadsheet with more than 100 columns and calculate metrics such as entropy, dispersion…) and add to it other data from a questionnaire (age, gender, average screen time…). Then you just try to look for what correlates with what until you find that some correlation is statistically significant. In other words, you are committing fraud by randomly comparing data and looking for something that looks like positive result. And when you get a low p-value, only then do you formulate a hypothesis. This is called data fishing or p-hacking.

    I was in the opposite situation. I measured the effect predicted by the hypothesis. The problem was that when I then explored other variants of correlations between screen time and eye-tracker data, I didn’t find any other statistically significant correlations. The one with screen time on TikTok and fixation distance in landscape videos was the only one. Sometimes it even showed me that higher screen time correlates with more focused fixations. The complete opposite. 

    These findings have led me to reconsider some of my initial assumptions. I originally believed that long-term exposure to TikTok would affect the way we watch films. But based on the current data, that effect is either not present or not captured by my experimental design. In any case, I don’t have sufficient evidence to claim that screen time directly influences how we watch videos. What’s more, I began to doubt whether it still makes sense to continue focusing on studying influence of screen time on viewing habits.

    Another important takeaway concerns how we interpret claims about TikTok’s influence on cognition. Keeping in mind what Stuart Ritchie wrote in his excellent book Science Fictions, I’ve begun to question the many articles that describe TikTok as fundamentally reshaping our brains and behavior. Perhaps those effects are real—but we should be cautious when such strong claims are supported by just a single metric from an eye tracker or similarly narrow data sources.

    Finally, these results also shape how I’ll present my own research moving forward. I’ll keep the question in the title of this post, because it’s accessible and engaging. But in answering it, I’ll emphasise a crucial distinction between a) TikTok as an app, b) the formal elements typical of TikTok videos, and c) the vertical video format itself. Unlike screen time, both the formal elements and the vertical format show clear evidence of influencing how we watch
    videos.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    July 15, 2025
    data, research, screen time, social-media, technology, tiktok

  • Bára Erben (2025) The Body in the Film

    A few years ago, when I joined Palacký University as an assistant professor, I started to include more and more cognitive theories in my teaching. After five years I have to say that I feel a positive response from the students. Every year there are several of them who want to explore film audiences and their perception of cinema. And this semester, the best student thesis so far was submitted: The Body in the Film (original title: Tělo ve filmu).

    I’ve selected a few of the most compelling findings from the thesis, which I’d like to briefly share here. They offer fascinating insights into how composition, gender, and viewer attention interact.

    Gendered perception of male nudity: In a scene featuring male masturbation from the film Její tělo, male viewers paid significantly less attention to the nude male body than female viewers did (pp. 25–26).

    Gendered perception of nudity in centre and periphery: During the ritual sex scene in Midsommer, women focused primarily on the central action of the narrative, while men’s attention was often drawn to exposed female genitalia (pp. 27–28).

    Fluctuating focus in Chyby: When nudity is shown, viewers’ attention shifts back and forth between the characters’ faces and the exposed female breasts. These shifts are driven by changes in dramatic tension and the unfolding dialogue – especially while revealing the characters’ past (pp. 27–28).

    Focused attention in Poor Things: A combination of lens choice, actor movement, and color palette effectively guides viewers to concentrate solely on the actors, largely ignoring the surrounding space (pp. 29–30).

    Visual suppression in Její tělo: In a visually distinct static shot from the film Její tělo, careful use of composition, low lighting, and actor performance causes viewers to almost completely disregard a striking neon-blue sign and a red light source in the background—even though they are visually prominent (pp. 31–32).

    Gendered gaze dynamics: Across various scenes, the analysis found that male viewers tended to fixate longer on sexualized body parts, while female viewers perceived scenes more holistically—often focusing on the emotional and relational dynamics between characters.

    Involuntary attention: One of the most striking findings is that viewers look at nudity even when they consciously try not to. Their gaze reveals an unconscious attention pattern that contradicts their self-reports (p. 54).

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    July 7, 2025
    art, eye-tracking, film, movies, nudity

  • Tim Smith: Exploring the Cognitive Foundations of Cinematic Continuity (May 15, 2025)

    On Thursday, May 15, 2025, we welcomed Tim Smith from the University of the Arts in London to CoSTAR National Lab. Tim was kind enough to accept the invitation from Adam Ganz and me.

    In his lecture, he talked about Continuity Editing Rules and why we, as viewers, enjoy watching (mainly) Hollywood movies so much. He also discussed how filmmakers control the audience’s attention. According to Tim Smith, filmmakers are brilliant intuitive psychologists. Without scientific knowledge of human cognition, they are able to very successfully control where viewers look in films and how they perceive them. He illustrated this with a range of data from his experiments.

    I have been familiar with Tim’s work for many years. I mention several of his experiments and findings to my student in cognitive film theory classes. However, two things never occurred to me.

    1) Tim refers to almost no film theorists. The exceptions are Janet Staiger, Kristin Thompson, and David Bordwell. He finds much more inspiration in the work of cognitive psychologists (J. J. Gibson, Daniel Simons, Joseph Anderson), neuroscientists (Uri Hasson), and especially filmmakers, whom he quotes extensively (Spielberg, Tarantino, Dmytrik, Eisenstein).

    2) How much he loves film. He is genuinely fascinated by film and has a vast knowledge and passion for cinema. 

    I had the opportunity to talk to Tim before and after the lecture, and I was pleased by his vision of an ever closer connection between filmmakers and cognitive film studies. However, we also briefly touched on the topic of TikTok as a competitor to Cinematic Continuity in the battle for audience attention. So perhaps the main question is what form film will take in the future.


    Exploring the Cognitive Foundations of Cinematic Continuity

    Professor Tim Smith (University of the Arts London)

    Thursday 15th May, 16:00

    SHILLING-LT (Shilling building, Royal Holloway)

    Filmmakers tell stories by selecting and emphasizing key details of an audiovisual scene through editing, cinematography and sound design. Such edited sequences instantaneously transport the viewer through space and time in ways that are physically impossible and, due to their divergence from reality should pose problems for viewer comprehension. However, filmmakers believe they have at their disposal a suite of editing techniques, known as the Continuity Editing Rules that make the viewing of film effortless. In this talk I will present a series of behavioural and eye tracking experiments investigating the cognitive foundations of film perception and provide a theoretical and methodological framework for extending filmmaker insights into audience experiences of film.


    Tim J. Smith BSc. PhD. is Professor of Cognitive Data Science in the Creative Computing Institute, University of the Arts London and head of the Cognition in Naturalistic Environments (CINE) Lab. He applies empirical, computational,  neuroscientific and developmental methods to questions of Media Cognition, including real-time audience experience analysis via eye tracking. He has published widely on the subject both in Psychology and Media journals and his research has informed media practices through collaborations with Dreamworks Animation, BBC, Channel 4, and the Academy of Motion Picture Arts and Sciences.Currently he is leading the UKRI cross-council multi-site research project, Animating Minds: Triangulating the age-appropriate impact of children’s media.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    May 19, 2025
    art, attention, cinema, cognition, Continuity Editing, film, movies, perception, tiktok

  • Futures Studio – Tour of the new virtual production studio at Royal Holloway

    Today, I had the opportunity to attend a two-hour program at Futures Studio at Royal Holloway. It is a virtual production studio serving research and development for the audiovisual industry. The two-hour program flew by.

    At the beginning, Peter Richardson talked about the main milestones in the development of special effects from Le Voyage dans la Lune (1902) through Modern Times (1936), Superman (1978) to Mandalorian (2019) and The Batman (2022). The aim was not just to show the effects in the films, but mainly to highlight the gradual improvement of processes, materials, and technologies. And everything led to… LED.

    That’s it. Futures Studio. Equipped with an LED projection wall, light panels, a sound system, computers, and simply tools for virtual production.

    And yes.

    The studio allows the projection of George Costanza’s image.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    May 16, 2025
    film, industry, virtual production

  • Interview with screenwriter Meir Lubor Dohnal

    In the last few weeks – while waiting for the ethics committee’s approval – I have been finishing a book about the Czech screenwriter Meir Lubor Dohnal. An unknown filmmaker to most people, but one who had an incredibly interesting and colourful life. Just to name a few – a Holocaust survivor, a member of the new wave of Czechoslovak cinema, politically persecuted, a signatory of Charter 77, he made it as a screenwriter even in German exile, films based on his screenplays have competed at major festivals around the world. 

    The book I am finishing is partly a collection of essays and partly a collection of interviews. Togga will be the publisher and the book will be released during the summer of 2025.

    Talking to my colleagues at Royal Holloway, I realised that none of them will probably be able to read the book because it will be published in Czech. So I thought I’d take a bit of theour conversation and translate it as a sample.

    So below are two excerpts that relate to videos on smartphones and artificial intelligence. But through these topics, Meir Lubor Dohnal gives an insight into how he thinks about writing for film, and where he sees the power of cinema.

    —

    Aren’t you worried that you are training filmmakers for a time when cinemas will function more like museums?

    I don’t think that’s going to happen. I believe that people will still go to see new films and filmmakers will discover innovative techniques suitable for the big screen. I recently saw Corsage, which is the type of film that people won’t want to watch on their mobile phones.

    How important are technological developments for screenwriting students? Do you address with them the difference in writing for cinema or for TV and streaming platforms?

    We distinguish between the internet, TV, or cinema. As a viewer, though, I’m increasingly aware that technological advances are blurring the distinctions between ways of presenting film. After seven years, I’m buying a new computer, and the visual quality is such that it almost doesn’t matter if I’m sitting in the back row of the theater or at the computer. The biggest difference is in the sound. That’s where computers and televisions still lag behind cinemas, and without good sound, the film is amputated.

    Do your students watch movies more in the cinema or on the computer or even on a smartphone? 

    They prefer the cinema, but more often on the smartphone. I wouldn’t watch a movie that way myself, but it’s a way for them to see the classics. And if they have a guide to what to look out for, they’ll get it. For them, these are movies for old-timers anyway. They wouldn’t find Bergman and Pasolini as interesting today as we did 60 years ago. They don’t marvel half a century later at formal innovation. It’s enough for them to understand that in Pasolini, communist and Catholic thought, conservative desire and radical subversion, were at odds. As a document of important works of cinematic history, the phone can work just fine. But not as a tool for discovering things. Students go to the cinema to see contemporary films, just as we do.

    Could you imagine writing a film for a smartphone?

    Well, yes, I could, but I wouldn’t do it in the same way as I would for the wide screen. I saw my former student Zdeněk Jiráský at work on I Don’t Love You Anymore. It’s about two teenagers who run away from home all the way to Romania and film each other on their smartphones. I know, it’s a different case, but I want to use it to show that smartphones can be conceived as something that filmmakers use in the construction of the story and that individualizes the film. Phones make it possible to connect different environments by jumping quickly into another space. The smartphone has become almost an organic part of the body of our contemporaries. A character without a phone would be strange. Life without it is like the life of a hermit. Thanks to technology, young people will perceive the world around them differently than I did. Their experience shaped by technology will allow new ways of identifying with the character. In spite of this, I think the experience of cinema screening is unmistakable. We’re stacked up in the dark like we’re in a spaceship, staring together at something from another world. I would have missed that magical atmosphere, and I’m sure it’s not just me. I don’t think the cinema is in danger of disappearing. At most it will be seen as just one possible way of showing films.

    (…)

    Do you think AI will help writers with their work or take it away?

    Of course I’m thinking about it. It will certainly change the work of screenwriters, but I don’t think writers won’t be needed. I can’t imagine AI being able to make art. I believe it can write a script, I believe a film based on such a script can be not only consumable but interesting or even attractive. But it would probably be far from what I understand as the art of filmmaking. The individuality of the filmmaker, which is strange and unique, can break the rules. Whereas artificial intelligence will always follow the rules. Even if we program it to break the rules, it will do so according to the rules. Of course, such work is also needed in screenwriting, say in advertising. I can imagine that AI can even produce a commercial film better than a human, which will have a high audience. If you want to hit the audience’s taste, thanks to AI you can have a script ready in an hour.

    But it will be a script that follows the rules.

    Of course it will. To write a good script, you have to know the rules. You have to know the craft. That’s how you deal with writing problems. The AI will give you, say, ten options to choose from. But when we talk about film as art, we’re mostly talking about uniqueness and originality. And the only originality is in us and in our consciousness. Of course, we can look at people through statistics and see that we all have the same problems. But the solution to those problems is completely individual. A good screenwriter can identify with a character based on his or her own experience and thus make his or her work original. At the same time, it gives the viewer a chance to identify with the character as well. If this is done, the writer gets into a direct dialogue with the viewer because the character connects them. When the viewer then leaves the cinema, something is going on in their head that they would not otherwise have had the chance to experience. That’s what’s exciting about it, and there’s no substitute for the artist.

    The originality has to come from the screenwriter?

    It can be the director’s. I see the writer and director as such a complex personality. If someone has a director’s vision, they can make art out of a phone book. So theoretically, a robot could replace a screenwriter if the director is dominant enough.

    But you’re obviously not worried about that.

    We can speculate all sorts of things. Like artificial intelligence destroying us. But I don’t think that’s going to happen, which is why I don’t think AI is dangerous. I’m going to digress. I think there is a system of information exchange and correction in the universe. Man is not alone in this. Scientists today can observe the post-big bang state and subatomic particles. Something happens according to rules and something seemingly erratic, but even the chaotic is subject to harmony. I deliberately don’t say goal because I’m not a fatalist and I don’t think it happens over time. Harmony – the unity of everything – is just the basic rule within which we are free to move. Everything from elementary particles to human beings has some degree of freedom, where to move, what to bump into. Sometimes it works, sometimes it doesn’t, it’s random. Sometimes we hit harmony and sometimes we don’t. When we don’t, it falls off. And when it does, it cleans everything up, lights everything up. But I don’t want to get into religious matters. What I’m trying to say is that just as our bodies are changing, our consciousness is evolving. Just as the evolution of our species is changing the importance of certain organs, like the appendix, so too is the way we think about the world transforming. And the question is whether this will take place over tens of thousands of years, or faster.

    And how does artificial intelligence relate to this?

    Humans have the gift of awareness and reflection. I think art is a form of cognition, like science. They have a similar meaning for humans. They make us more insightful, more rational, more aware of our limits and possibilities. When you know the limits of your own consciousness, you think about how to overcome them, which is related to the human desire for knowledge. Artificial intelligence can be an aid in this, just as the computer and other inventions have been in the past. It will speed up some of the thought processes, calculations, become our collaborator, and we will think together. I see it as an opportunity rather than a threat. If anything is a danger to mankind, it is man.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    May 1, 2025
    ai, cinema, film, movies, screenwriting, writing

  • Is TikTok really as fast as we think?

    I often encounter the opinion that TikTok is fast. Frankly, I’ve never understood it. I could describe TikTok in a number of words, but speed would not be one of them. Maybe the problem is what we mean by speed.

    The last time I came across the idea was in Berenike Jung’s “Travelling Sounds, Embodied Responses: Aesthetic Reflections on TikTok” from the anthology Traveling Music Videos edited by my colleague Tomáš Jirsa. Berenike Jung describes TikTok as follows:

    “the videos’ short length creates an accelerated, hypnotic pace of viewing. The videos are themselves often accelerated and/or stuttering, edited to a speed that puts chaos cinema in the rearview mirror and further normalizes jump cuts as the rule for online videos”

    I identify speed in three senses in the quote:

    1. the fast pace caused by the short length of the videos
    2. the acceleration of videos
    3. the use of jump-cuts to eliminate shots where nothing important is happening

    In at least two of these, it is something we have known for decades, so it hardly surprises any viewers today. Anyone who has ever watched a silent movie has encountered video acceleration (the reason for this is the different standard of projection speed expressed in terms of the number of frames of film stock per second of projection).

    Similarly, the jump-cut cannot be considered anything groundbreaking. It is associated in film history books with Godard’s A bout de souffle (1960), but even that was not the first use. David Bordwell wrote an article on the jump-cut in 1984 (Wide Angle, vol. 6, no. 1) where he points out that the jump-cut had been used since the early era. In my notes, I found uses of the jump-cut in, for example, the Finnish film Valkoinen peura (1952, Erik Blomberg).

    The remarkable speed of TikTok can therefore perhaps only be explained by the short length of the videos. And while we could find examples from the past (commercial breaks on TV, blocks of trailers and ads in cinemas before the screening starts, blocks of video clips on TV and YouTube), TikTok has at least contributed to shortening videos to a new level.

    But the list of possible understandings of what makes TiKTok fast is not, in my opinion, complete. Leaving aside the contextual aspects of video production speed and TikTok’s speed of expansion. The impression of TikTok’s speed could still be caused by at least two things: 1. the average shot length of a TikTok video, 2. the amount of information conveyed through spoken word or text. Given complexity of the combination of spoken word, text in videos, text in subtitles, and text in the app interface, I’ll leave point 2 for another time. In the following lines, I will focus on the average length of a shot.

    Average shot length is, in my opinion, a better indicator of speed than simply video length. Try to imagine a hypothetical social network where users could upload 30 second long videos, but only in one take. I dare to doubt that we would consider such a social network fast in any sense. We might even find it slow and boring. Especially compared to the US film and series, where the average shot length is between 3 and 5 seconds.

    We can see this for ourselves thanks to Radomir Douglas Kokeš, who has been collecting data for a long time and publishing it on his blog. From his database, I was able to select a sample of films produced in the USA between 2014 and 2024 and calculate the variance of average shot lengths. I had 430 films and episodes in my sample. Sure, it’s not all that’s been produced, but better data just aren’t available.

    The graph clearly shows that although the average shot length varies, it oscillates around similar values (average of averages).

    Now let’s see what the average shot length of videos on TikTok is. Admittedly, I don’t have as large a database as Douglas. I collected the data in an unsystematic way one afternoon by sitting down at my desk with TikTok on my smartphone, turning on screen recording, and recording one video from start to finish for about 25 minutes. I then uploaded the file from my phone to my computer and took the classic shot length measurements I’m used to. For each video, I noted the lentg and the number of takes, from which I got the average shot length. In total, there were 19 videos in my sample (including two commercials, two split-screen videos, and five videos that were more like slideshow photographs).

    The average shot length of videos on TikTok is longer than the average shot length of American movies. While the median is similar, the boundaries for the third and fourth quartiles are shifted by seconds.

    Of course, one could argue that this is due to the sample, and that if I had measured on a different day or in someone else’s recommendation algorithm, I would have gotten different numbers. Sure.

    On the other hand, there are arguments that the higher average shot length for TikTok videos may not be accidental. One of the reactions of our eye movement to editing in film is to reorient our gaze to the center of the screen. Research shows that viewers of portrait format videos (which is the format on TikTok) exhibit a weaker central bias than viewers of landscape videos (the format of movies and TV shows). I don’t know what videos specifically the measurements were taken on, but the weaker central bias could indicate a smaller number of cuts after which our gaze turns to the center.

    Anyway, I want to repeat the measurements in some time to make sure. For now, I’ll go with the measured data and therefore…

    In terms of average shot length, TikTok is simply slower than American movies and TV shows. It is not a fundamental difference, but it is there. When you think about it, it’s not really surprising. On TikTok, many of the videos are shot in one take (I had 6 videos in one take in my sample). These videos will inevitably slow down the TikTok experience and increase the average shot length. But in a way, they are the simplest thing a creator can do and therefore the most typical.

    Well, to summarize. Some viewers seem to get the impression of speed from the TikTok videos. This impression is so intense that it has made its way (in a form of intuitive claims) in to academic papers. The problem is that we don’t know how to quantify this impression. For now, the only measurable value is the short duration of the videos, but I don’t think that is relevant evidence of the impression of speed. Especially when the average shot length is higher for TikTok than for the average American film. I therefore put more hope in the amount of information (text + speech) conveyed in the videos, but I’ll leave that for another time.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    March 3, 2025
    analysis, ASL, editing, film, movies, social-media, style, tiktok

  • Which screen is actually the second?

    It’s evening. I’m not going out, but I don’t feel like reading, so I’m watching a movie on an unnamed streaming platform. When choosing, I prefer movies I’ve already seen. Why? So I don’t feel guilty about looking at my phone every now and then.

    A not insignificant number of viewers are reportedly thinking along similar lines to this scenario. They watch a film unfocused, or while doing something else. In another post I addressed casual viewing and today I want to look at the changing ways of viewing audiovisual content from the perspective of watching multiple screens simultaneously.

    There are several interesting studies on the topic in which researchers have looked at the viewing practice of watching multiple screens simultaneously.

    Holmes, Josephson, and Carney. 2012. Visual attention to television programs with a second-screen application. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA ’12). Association for Computing Machinery, New York, NY, USA, 397–400.

    Brown, Evans, Jay, Glancy, Jones, and Harper. 2014. HCI over multiple screens. In CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’14). Association for Computing Machinery, New York, NY, USA, 665–674.

    Lohmüller, Eiermann, Zeitlhöfler, and Wolff. 2019. Attention Guidance in Second Screen Applications. In Proceedings of Mensch und Computer 2019 (MuC ’19). Association for Computing Machinery, New York, NY, USA, 179–187.

    All of the studies are impressive because they experimentally test how viewers behave. They use an eye-tracker (mobile or stationary) and think about setting to resemble a living room as much as possible and a laboratory as little as possible. In other words, they make sure that the experiment reflects the natural environment.

    The results are then very compelling and show that viewers are primarily watching TV and paying less attention to the tablet/phone (by a ratio of 2:1 to 5:1).

    The problem is that all these experiments are formulated from start to finish in such a way that it is clearly established which screen is “first” and which is “second”. Even the choice of what is played on the second screen is only complementary to what is played on the first screen. For all three studies, this approach is understandable. Their intent was to offer data to make multi-screen viewing consistent, which could then be used by media companies to develop applications. But if we are interested in the relationship between audiovisual and spectator behaviour, we necessarily feel that something has been left out. They left aside the fact that there is a competition for our attention in the media market.

    I don’t know anyone who would say they watched TikTok last night and occasionally glanced at what was happening on Netflix. That’s not how we talk about leisure time. Statistics, meanwhile, show that the average U.S. adult in 2024 watched Netflix for only 3.7 minutes longer per day than TikTok (62.1 vs 58.4 minutes). Is that enough of a difference to determine which screen is hierarchically first and second? I’d say no.

    It may well be that the attention paid to movies and TikTok is different when the division between first and second screen is abolished. And it’s also quite possible that some movies are less successful at attracting attention (though studies suggest that this is not the case).

    I would like to build on the three studies mentioned and think about how to implement experiments in a way that reflects the battle of the screens. To start with, users will need to break the hierarchy of the first and second screens and let them be free.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    February 24, 2025
    attention, movies, tiktok, two-screens

  • David Chalmers (2022) Reality+: Virtual Worlds and the Problems of Philosophy

    I decided to mix my reading about TikTok, social networks and cognitive psychology with a few philosophical books. In the case of Chalmers in particular, I initially felt that I might not be too far off thematically since he addressed the topic of virtual reality in his last book. Well, I was a bit mistaken. Chalmers has very different interests than I do, and he clearly tends to address problems of ontology, epistemology and ethics. In the context of TikTok, especially in the context of the current debate about whether or not to ban it, I found only one passage explicitly related. But I want to comment briefly on the rest of the book too.

    First of all, it’s absolutely brilliantly written. In fact, I can well enough imagine that if this was what teaching philosophy in high schools was like, philosophy would have a very different reputation. Chalmers not only displays a deep erudition in Western philosophy, but also makes excursions into the non-Western philosophical tradition and especially into pop culture (especially science fiction literature, but also films such as The Matrix, The Thirteenth Floor, The Truman Show or series such as Star Trek: The Next Generation). Thinking about and through virtual reality (at least in the form of thought experiments) then leads him to rethink many philosophical concepts. Some of these – Plato’s Theory of Forms or the theistic conception of the world – are rather bizarre in the tradition of naturalised analytic philosophy. Chalmers approaches all of them with an open mind and is willing to grant them some persuasiveness in the context of his argument. If one of the functions of philosophy is to open minds and develop critical thinking, here it succeeds perfectly.

    But now to the chapter on Value, where I find several points to consider in the case of the current discussion of TikTok. Recall that a few weeks ago it looked like TikTok was going to be banned in the US, then it didn’t even run for half a day before the new US President decided that things might be quite different. The case is summarized on wikipedia. What I found most interesting about the discussion was how the arguments for the ban were layered. We could see a diverse range of opinions covering national security, data protection, but also the dangers of TikTok for child users. I’m not competent to judge the extent to which TikTok can endanger America or damage the minds of children, and I don’t get the impression that there is a consensus on this. Let’s leave aside the legal, political and social dimension of the debate and think about whether TikTok has any value to the individual. Specifically, then, I will consider four possible values that Chalmers considers in the context of virtual reality.

    In the chapter, Chalmers works with four traditional approaches to what it means that something is good for someone: hedonism, desire-satisfaction, social, and objective-list. Let’s look at what these approaches tell us about TikTok.

    First, hedonism, which can simply be understood to mean that a thing is good if it brings more pleasure than pain. This is a bit of a problem with TikTok in particular. It brings satisfaction to its users which can result in addiction in 5.9 % of users. but for some individuals this can lead to a destructive, daily interaction with an app that satisfies them but potentially develops mental illness in some users as suggested by meta analysis.

    Second in line is desire-satisfaction, or the approach that argues that a good life is one that brings about the fulfillment of our desires. Here we do not get simple hormonal satisfaction in the brain causing addiction to TikTok, which would again be contrary to a good and worthwhile life. Desire-satisfaction is probably the most advocated approach to value, promoted by the users themselves. Certainly not all of them, but a significant portion of them. Desire-satisfaction is implicitly present in all the motivational self-help videos, investment videos and exercise videos that are on TIkTok. I can’t judge whether this approach to values is correct, but TikTok, and by extension other social networks, are the venues through which desire-satisfaction spreads. And it only spreads because it finds a response from users who are willing to spend money on manuals, consultants and coaches to help them achieve financial independence, mental stability and a beautiful body.

    The social perspective argues that the value lies in connecting with other people. Although social networks are often criticized for distancing people and creating moats in society, they are undoubtedly a tool that allows each of us to be in touch with family and friends at almost any time. For example, I can stop typing now and ask my wife Kristýna how many of her friends she has communicated with on social media today. The question is whether TikTok, which is primarily for sharing videos, offers social value. Here, though, it may just be a generational issue. While I wouldn’t be able to use videos to communicate operationally because I would be thinking through the angle, composition, and exact script of what I’m saying, some of my students already talk about making videos online as something natural. So at the very least, I don’t want to deny TikTok a social dimension. On the other hand social interaction is via TikTok and not with TikTok. Interaction with app and deep scrolling are understood as destructive practices.

    And finally, the objective-list, which is the most subjective of all approaches. Even if we agree that the values are knowledge, friendship, and fulfillment (as Chalmers writes), we can never agree on whether TikTok helps us towards or away from those values. Whether TikTok serves an educational function is disputed (are student questionnaires a relevant method when we consider, that they may be already addicted to TikTok?), and whether the friendships on TikTok are real is also hard to agree on. I, for one, may be convinced that TikTok does not belong in the hands of children, but that does not mean that contact with TikTok will automatically damage their brains. In doing so, I feel that the current discussion about banning TikTok is being conducted on a moral level through the most vague objective-list approach to values.

    In conclusion, let me just summarize that from an individual perspective, TikTok does not strike me as a fundamentally immoral application. Of course it has risks, of course its use can be dangerous for certain groups, but we could probably say that about all social networks or tools in general (guns, cars, staircases). I hope the political and social arguments of the proponents of banning TikTok are more convincing.

    And if we look away from TikTok and look at short videos in general, their value is no less than that of other forms of audiovisual (information, advertising, communication, entertainment, or even art…). The problem is obviously in the platforms and may remind us of the critical perception of Hollywood. But this means that banning one platform won’t solve anything, because people will find another, as happened with the migration of Americans to Red Note.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    February 11, 2025
    David Chalmers, philosophy, social-media, tiktok, values

  • The imperfect but amazing SAM2: AI video segmentation

    As part of the AI bootcamp with Farshad, we tested models that enable photo and video segmentation. In this blog, I’d like to show Meta’s SAM2 application to two cases from the history of audiovisual culture where the model failed. At the same time, I want to demonstrate that far from criticizing SAM2, I find it a fascinating tool.

    Before I go on to introduce the examples, I want to explain what is meant by segmentation. For film studies graduates, this term can be confusing because it means something different than what we are used to. In the context of AI analysis of photos and videos, it means entity selection within the photo and video. It is therefore a spatial segmentation that we can select as the area of interest a person, an animal, an object, but also houses, roads, trees or the sky. If we are talking about temporal segmentation – I was instructed by Farshad – we are talking about chapterization. As I wrote, this is unusual for a film studies graduate, but it makes sense.

    Now let’s go to the examples. There’s no point in me trying to introduce Meta’s SAM2 model here, because this website does that quite successfully. When I tried the demo, I was naturally wondering how to fool the model. It’s not nice of me, but I couldn’t shake the impression that the sample videos in the demo are too easy to detect.

    I knew I wanted to pick out some cases from the history of audiovisual culture, but which ones? What will show the AI segmentation capabilities? So I set some criteria:

    – a black and white film

    – poor quality video

    – crowd scene

    – shape change

    I then considered the selection of a particular film and scene. In the list were crowd scenes from Soviet war movies, night scenes from horror movies, scenes with T1000 from Terminator 2… I also considered testing the twins (Social Network, Dead Ringers), but that ended up being very easy for SAM2.

    In the end, I chose the chase scene from Seven Chances, where the main character is chased by a hoard of brides. The second demo was to include Odo, the shapeshifter from Star Trek: Deep Space Nine. I was very lucky, because these videos were available on youtube. Let’s take a look at the result.

    In the chase demo, SAM 2 was tasked with tracking three selected brides. We can see that he was only 2/3 successful. While the orange and blue brides were tagged the entire time, a third green tag traveled between several brides.

    As far as Odo is concerned, SAM2 only succeeded sometimes. Here in the demo, we can see that the tendency to stick to the shape of the original object prevailed. I was under the impression that SAM2 is more successful when Odo changes from or to human form, but I don’t have exact data on that.

    But the biggest shock for me was in this clip. I accidentally picked out part of the following scene as well, and was surprised that SAM 2 was able to identify Odo even after the cut (in the last two images).

    Now, I could go on to criticize SAM2 for not demonstrating a 100% success rate. I could also speculate that there is no substitute for genuine analysis done by a human. I could certainly also speculate on what AI analysis reports when it lacks subjectivity. I’ll leave all that to others.

    I will confine myself here to amazement at what a computer model can do. Certainly it can’t yet replace humans, but I can imagine how it could enable us within film studies to analyse a much larger number of films in a short time than we are able to do now. (This was, incidentally, the first thing Peter Krämer replied to me when I emailed him.) I can also imagine quite well that AI will prepare the analytical basis, which the analyst will then work with further. Actually, I don’t see many reasons why we don’t use AI already and the ones I see don’t make much sense to me.

    • About SNAPlab
    • Books, chapters, articles
    • From the Lab
    • Lectures, Presentations, Conferences
    • Teaching
    • Tools
    • Uncategorized

    February 3, 2025
    ai, analysis, Tools

Previous Page Next Page
  • About SNAPlab
  • Books, chapters, articles
  • From the Lab
  • Lectures, Presentations, Conferences
  • Teaching
  • Tools
  • Uncategorized

This work was supported from OP JAC Project “MSCA Fellowships at Palacký University III.” CZ.02.01.01/00/22_010/0008685, run at Palacký University in Olomouc, Czech Republic. 

Blog at WordPress.com.

 

Loading Comments...
 

    • Subscribe Subscribed
      • SNAPlab
      • Already have a WordPress.com account? Log in now.
      • SNAPlab
      • Subscribe Subscribed
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar