June 2011

In my last post about the Social TV research context I explained that I decided to focus on generating and evaluating conversational video metadata by eliciting mediated conversations through SocialTV.

Before drilling down into that choice to distill a set of research questions, however, there’s a more basic question about the context of research into SocialTV metadata: why gather SocialTV metadata at all?

The Notube project has a diagram on this issue, which shows the progression of a piece of TV content along a timeline from pre-broadcast to media archive. The assertion is that having more metadata about user preferences enhances the value of TV content because it provides many more opportunities to recommend programmes. Although the Notube project adds a great deal to existing research on recommendation systems, Notube’s central focus follows most existing research in focusing on developing more accurate and sophisticated user profiles (in this case through aggregating media consumption habits from heterogeneous sources on the web).

The way the diagram is re-used in my slide above emphasises the corrolary of the point that it is intended to illustrate: that having more TV programme metadata would also enhance the frequency and accuracy of recommendation for content (and thereby its value) throughout its lifecycle {{2}}.

If metadata (about profiles or programme content) can enhance the value of a TV show and facilitate production, discovery and delivery of TV programmes because it increases the likelihood of that programme being recommended, then it should follow that the greater the richness and referential diversity of that metadata, the more ‘recommendable’ the programme becomes.

This suggestion presupposes that the metadata in question is relevant, rather than a random spamming of references, intended to maximise recommendation in all possible contexts. So the question then becomes: what determines relevance in this context, and, even more importantly, relevance to what?

It may seem self-evident that the metadata about users should be relevant to their consumption habits, and that these habits, recorded and aggregated should constitute their preferences or dispreferences. Even if this widespread assumption holds true, what should programme metadata relate to? Are broadcaster’s assertions about their programes necessarily relevant to the way those programmes are discovered and interpreted? And crucially, in SocialTV – which almost by definition is about the interaction between viewers brokered via a networked TV infrastructure, how do those metadata correlate with the way TV programmes are used as a prop, touch-point, or stimulus to conversation between viewers{{1}}?

Current content discovery heuristics tend to rely on user profiles and broadcaster-provided metadata, without necessarily taking into account the context and quality of interactions between users.

The hypothesis of this research project is that metadata derived from conversations between viewers via SocialTV can provide a crucial additional component to support the interactional possibilities of of SocialTV.

To test this, an experimental scenario will be developed to involve concurrent, co-present viewers of a TV programme in a public multimedia chat system, designed to elicit metadata from their conversation. The transcript of their interactions will then undrego Conversation Analysis. This analysis will provide a baseline to evaluating to what extent conversations around a SocialTV experience can be correlated to a detailed and highly granular ‘top down’ semantic annotation of a TV programme.

An analysis of the data may also be used to test several related hypotheses:

• that conversational metadata are likely to have more divergent subject matter and more external reference than a-priori programme data about actors, characters and plot developments
• that conversational metadata are likely to be more responsive to ways in which the context of the conversation changes{{3}}.

  

So in terms of my earlier exploration of SocialTV as a research context, I can start to narrow my focus onto a few areas.

To achieve the study outlined above, I am proposing to deploy a multimedia discussion interface that will allow co-present concurrent viewers of a TV programme to interact and converse as freely as possible. Although it may have significant design issues, the state of the art in this context is definitely the ‘hot topic’ of Second Screen/Companion Devices. This is not the central point of the study, however, so I will be approaching this part of the project as a design process – building on prior art – which is abundant at the moment – and iterating out a customised version as quickly as possible to find something basic that works well enough for me to get the conversational data I’m looking for.

The other research objectives in the slide above are already in order of priority: eliciting and capturing discussion is the most pressing need for the system. Other functions and features are interesting, but probably out of scope for the time being. However, I might well post some ideas to this blog about how conversational metadata might underpin new approaches to searching, filtering, annotating and segmenting video.

[[1]] There’s still the thorny issue of what determines ‘relevant’ in this context. There may be perceptual studies comparing recommendation engines or other ways of trying to determine relevance of metadata statistically. However, for the purpose of this study, the question is moot. Relevance, especially in terms of conversational metadata, is subjective unless based on evidence of how it is used to broker or support an interpersonal interaction via SocialTV.[[1]] [[2]] This is also core to Notube’s aims and methods: using highly granular metadata about programmes to create ‘serendipitous’ trails through content that can break through the sameness of personalisation recommendation systems that can tend to channel users into relatively static, homogenous clusters of content. [[2]] [[3]]For example, a programme might be broadcast 50 times over 15 years. Conversational metadata associated with the programme might change significantly over time and in the different contexts in which it is watched, accumulating qualified layers of annotations, whereas broadcaster-provided metadata is likely to remain static.[[3]]

As Libby Miller’s presentation of the notube project’s ‘beancounter’ profiling engine points out, there are considerable privacy concerns with user profiling in SocialTV. To what extent will users be willing to share data on their personalised viewing in order to benefit from IPTV services?

Despite privacy concerns, much of the research into SocialTV focusses on
developing and using user profiles to provide more personalised TV services.

Many iTV systems and research projects, take their cue from (mostly)
single-user devices such as smartphones / tablets and computers, and focus
their innovative edge on providing services to users via ‘user profiles’.
However, because of social conventions of TV use (a shared room, a shared piece
of equipment, without a ‘login’ paradigm for use, it is a significant challenge
to find out which user in a household is watching TV, and provide appropriate
recommendations and other services (Yu, Zhou, Hao, Gu, 2006) {{1}}.

This presents practical and theoretical issues to providing single-user and
group-centric services via iTV, for which a variety of approaches are being
adopted, including household and group profile aggregation (Yu et. al, 2006),
(Shin & Woo, 2006){{2}} and context profiling (identifying who is likely to watch
at different times of day/days of week) (Vildjiounaite, Kyllanen, Hannula &
Alahuhta, 2008) {{3}}.

Recent research has developed more complex and multi-dimensional methods for
collecting very detailed logging data in order to identify groups and group
behaviour (for example, channel hopping) and characterising group dynamics (as
homogeneous, eg. group of friends or heterogeneous eg. a family) before
applying recommender systems. (Sotelo, Blanco-Fernandez, Lopez-Nores,
Gil-Solla, Pazos-arias, 2009) {{4}}.

But just as the social dynamic of a group may have a huge impact on the kinds
of recommendations and services that are appropriate in different contexts,
individuals may be just as complex and variable in their tastes depending on
the context and particularly on the interactions they are engaged in at the
time. Furthermore, by dint of their interactions (whether co-present or remote
via social networks or chat), viewers of iTV may be seen as a constantly
remotely connected group or series of groups – both heterogeneous and
homogeneous – that viewers drop into and out of depending on their
communications activity.

In this case, where the make-up of groups and individuals is constantly
shifting, the pre-selection of content by user-profiles may become an obstacle
to the fluidity and fluency of the ways people deploy and share iTV as a means
of interacting with each other.

The research aim of this project is to investigate how people deploy
the components of Social TV in order to interact. Therefore, rather
than looking at user’s choices and behaviours as a way of understanding and
profiling them, this project looks at how users interact with each other as a
way of understanding the media they deploy and the systems they use and adapt
to do so.

From this perspective, the user profile as a way of understanding the user becomes a secondary concern because the user does not ‘deploy’ their profile: it is built up around them, based on data gathered from their interactions with content, networks, services and other users, using predetermined a-priori heuristics.

If the question of interactive TV is about how users interact with each other, rather than how they interact with the TV, then the crucial elements to understand are the fluency and subtlety of the interfaces they have to each other, and how accessible and readily usable the components of iTV can be in their conversations. How readily can viewers find and manipulate media that they want to use to participate in an interaction? How specific can they be about a piece of media or a sub-section of a piece of media that they’re sharing or commenting on? How are their conversations brokered? And what can their interaction tell us about the media they’re using in order to express themselves and interact?

[[1]] Yu, Z., Zhou, X., Hao, Y., & Gu, J. (2006). TV Program Recommendation for Multiple Viewers Based on user Profile Merging. User Modeling and User-Adapted Interaction, 16(1), 63-82. doi: 10.1007/s11257-006-9005-6. [[1]] [[2]]Shin, C., & Woo, W. (2009). Socially aware tv program recommender for multiple viewers. IEEE Transactions on Consumer Electronics, 55(2), 927-932. doi: 10.1109/TCE.2009.5174476.[[2]] [[3]]Vildjiounaite, E., Kyllanen, V., Hannula, T., & Alahuhta, P. (2008). Unobtrusive Dynamic Modelling of TV Program Preferences in a Household, 82-91. [[3]] [[4]] Sotelo, R., Blanco-Fernandez, Y., Lopez-Nores, M., Gil-Solla, A., & Pazos-arias, J. (2009). TV program recommendation for groups based on muldimensional TV-anytime classifications. IEEE Transactions on Consumer Electronics, 55(1), 248-256. doi: 10.1109/TCE.2009.4814442.[[4]]