Saul Albert

Research on aesthetics, technology and social interaction

Archive for ‘April, 2011’

Outline of the Social TV research context

Outline of the Social TV research context

iTV has had a huge impact on all areas of television research, from production, and delivery to discovery and enrichment of television content and communications.

The delivery of TV media, as discussed in an earlier post, is no longer limited to one, static device in the home. Set top boxes (STBs) from Sky or BT, Over The Top (OTT) video / media services like youtube or vimeo, and the increasing use of smartphones, PSPs and other mobile devices has fragmented the experience of TV viewing into a multitude of discrete media delivery contexts.

For example, this video by the German design firm syzygy shows an consumer-centric overview of the technologies and interaction scenarios currently available in iTV. Simply put: the user controls the TV with a fancy remote (based on an iPad or tablet), which synchronises content, games and communications between ‘friends’, with product placement and advertising.

Another significant area of research in iTV is the use of network communications technology to enhance the usability of Electronic Programme Guides (EPGs), which, faced with ever increasing amounts of available content, present significant challenges to broadcasters and consumers to highlight and recommend appropriate content. More sophisticated, browser-like iTV interfaces and devices have opened up the possibility of using techniques for content navigation and selection derived from social networks and other web services.

Vidque - an example of an innovative iTV content discovery service

Vidque – an example of an innovative iTV content discovery service

For example, uses a twitter-like user interaction model of users’ personal ‘streams’ of content (in twitter, 140 characters of text), to share videos. By watching and posting videos, each person’s viewing habits become a channel, which other users can watch and ‘follow’, providing a more personalised and network-centric alternative to the a-priori genre or time-based list format of most EPGs. Vidque incites users to ‘curate’ their streams of videos, expressing something about their tastes and interests, creating another mechanism for them to project their identity into the social network through their choice of media.

iTV’s impact on TV production has only recently started to propose new business models for how media production is funded. For example, independent producers
such as Robert Greenwald, whose politically charged documentaries have found it hard to gain funding from traditional studios or broadcasters, have used the web to connect directly with media consumers who are happy to ‘pre-buy’ the film on DVD to fund its production. Many more of these ‘crowd funded’ projects are being produced in this way via services such as

Resonant Object (ARG) and Where are the Joneses (UGC)

Resonant Object (ARG) and Where are the Joneses (UGC)

As well as funding, iTV has also enabled other, more participatory ways for viewers to participate in TV production. For example Resonant Object is a new science fiction TV series that involved viewers in an expanded narrative via an Augmented Reality Game (ARG), happening in various different channels. Their involvement may extend from sitting in front of a TV to participating in a mobile phone-based game or even attending live events and contributing audiovisual content to what becomes part of the TV show.

Where are the Joneses was a pioneering project by David Bausola at Imagination, which got a brand (Ford in this case) to fund a soap opera which was scripted collaboratively by viewers of the sohw, participating on a public wiki. Their scripts were then produced by a professoinal crew and distributed via Youtube.

The fact that a brand like Ford funded Where are the Joneses demonstrates that this is a viable and potentially very disruptive business model. Rather than Ford paying an advertising agency to do product placement in an established TV show or film, they simply directly commissioned the use of a existing web platforms (wikidot, wordpress and youtube) to enable the voluntary user-generated scripting of the show, and funded the professionals to produce it, demonstrating how the advertising and broadcasting industry could be disintermediated by this kind of TV production process.

More generally, the infrastructure of iTV lends itself more to a network paradigm of communication than traditional broadcast. Opportunities abound for the enrichment of an expanded TV experience, in which viewers, broadcasters and other interest groups can participate in a ongoing conversations about media. Opening up the mass of these conversations about TV (via social media and other networked communications channels) that have always been an important part of the viewer experience, drives the changes to how TV and media is being produced, discovered and delivered.

Soundcloud - a great example of enriching content through user interaction

Soundcloud – a great example of enriching content through user interaction

A recent example of how facilitating people’s interactions with each other can enrich understanding of media is the Berlin-based start-up soundcloud. Started by music professionals who had been used to sending each other audio files by email, and were dissatisfied with the inability to discuss and enrich the media they were sending each other in any level of detail.

In the slide above you can see that comments on the audio track are embedded in the waveform of the audio. Their comments actually segment that track into sections that can be seen to relate to each comments, giving a much more granular set of annotations than is possible via email or on other media sharing platforms such as youtube, where comments are attached to entire media files.

A quick comparison of the quality and tone of comments on soundcloud and youtube could suggest that having to take the time to comment on a specific segment of an audio file encourages more context-relevant and incisive discussion (or discourages other kinds of comments and flames). This ‘in-content commentary’ mechanism also provides much more metadata about what the commenter is talking about, because it relates to a specific segment within the media file, which may have a very different meaning or reference than the rest of the track.

Soundcloud is a great example of how how people can be enabled to mobilise networked media in order to interact with each other, and how that interaction can be harnessed to enrich the experience and annotate the media being discussed.

So much of the effort in iTV research seems focused on trying to understand and profile TV viewers to then try to guess at and satisfy their established tastes. Soundcloud, instead, focuses on stimulating people’s interactions, using media (text/comments), via another piece of media(an audio file), as a way of understanding that media, and enhancing that media in terms of how it can be indexed, discovered, shared and re-used for further interactions.

This approach, which seems comparatively under-researched, highlights another possibility: that of investigating people’s conversations by looking at how they deploy, share and interject text, images, videos etc. in their interactions around other media (eg. TV).

That is the aim of this research project: to investigate mechanisms and heuristics for harnessing discussion between viewers of iTV, as a way of understanding the media they are experiencing and interacting around, and at the same time, looking at how they are deploying media to interact with one another as a way of understanding their conversations.

In short: understanding media through how it’s used in conversation, and conversation through how it uses media.


Leave a comment

Social Film Club project introduction

I’m working at British Telecom at their research centre at Adastral Park, on a project called ‘Social Film Club’. I’m working with a fantastic group there in the Future Applications team, who are all working on new ideas about the possibilities of creating new services and products with the merge of Television and telecommunications infrastructures.

‘Social Film Club’ refers to a whole set of ideas and technologies that BT have already been working with. It builds on work they’ve already been doing with ‘Social TV’, a catch-all term to describe Interactive TV (IPTV) systems that support and extend the sociable aspects of TV viewing {{1}} – both co-present and remotely.

Since both media and communications now use the same IP network infrastructure, television viewing itself is no longer a clearly distinct activity with a specific domestic device and context. When people say they are ‘Watching TV’ they may mean that they are using a specific piece of hardware (‘The TV’ may refer to largest screen in the house, although the same video content may be accessible on smartphones, PCs and tablets). They may be talking about engaging in a set of relationships defined by rights and contracts (for example, a TV
subscription to a specific company with rights to broadcast live sports events){{2}}, and the act itself may involve the interaction of a variety of communications devices and services beyond the time-honoured combination of TV, remote and sofa.

This lack of clarity as to the status and activity of television in a networked media environment coincides with an often-cited ‘fragmentation’ of the ‘traditional’ media environment {{3}}, in which the mass viewership used to be served by a tightly integrated media industry where infrastructure and content production, promotion, delivery and evaluation were all orchestrated by a limited number of organisations.

Cultural ‘reception studies’ of TV soap operas prior to this so-called fragmentation highlighted the social function of TV, such as soap operas being used as a touch point for interpersonal conversations {{4}}, or as a means of social group formation and identification (or counter-identification) {{5}}.

No longer having a constitutive binding ‘sameness’ to what people watch, providing a foil for collective conversation and identity may have contributed to the kinds of social/cultural isolation Putnam discusses in ‘Bowling Alone’, but one of the promises of Social TV is that IP Communications have created new ways for people interact with each other through and with television.

So the broad subject of my research is how the infrastructure IPTV can be mobilised for interpersonal and group interaction.

[[1]] For early examples of use of the term, see Coppens, T., Trappeniers, L., Godon, M.,: AmigoTV: towards a social TV experience. In: Proc. EuroITV 2004, U. of Brighton (2004), and Oehlberg, L., Ducheneaut, N., Thornton, J.D., Moore, R.J., Nickell, E.: Social TV: Designing for Distributed, Sociable Television Viewing. In: Proc. EuroITV 2006, Athens U. of Economics and Business (2006) 251–259 [[1]] [[2]] For example, using the BBC Iplayer to watch ‘catch-up’ TV does not require a ‘TV license’, whereas watching a simultaneous broadcast requires that viewers buy one or face a £1000 fine. (see [[2]] [[3]] See Putnam, Robert. 2000. Bowling Alone. New York: Simon and Schuster. [[3]] [[4]] For example, in Sonia Livinstone’s viewer surveys, something like 40% of participants reported that their soap watching was to provide a regular topic of conversation with friends: Livingstone S M (1988) Why People Watch Soap Operas: An Analysis of the Explanations of British Viewers European Journal of Communication Vol 3 #1London: Sage [[4]] [[5]] Ien Ang’s famous studies of Dutch watchers of the American soap ‘Dallas’ suggested that rather than identifying with the glamorous lifestyle of Texas oil billionaires deicted in the soap, Dutch viewers actually watched while engaging in form of a collective critical distancing. Ang I (1985) Watching Dallas: Soap opera and the melodramatic imagination. London: Methuen [[5]]


Leave a comment

In the Internet of Things, what kind of thing is your TV? And what kind of thing are you?

Researchers designing ‘Social TV’ used conversation analysis to look at how people interact while watching TV (Oehlberg, Ducheneaut, Thornton, Moore, Nickell, 2006){{1}}. They found that viewers integrated the TV into the conversation, making space for it to ‘take turns’ in much the same way as any other conversationalist.

Although the TV is traditionally quite a selfish conversationalist, speaking more than listening (unless you explicitly shut it up with the remote), the prospect of it becoming a more accommodating conversational participant as a networked device is intriguing. For example, when your Boxee remote app on your iphone detects an incoming phonecall, it will pause your TV. That’s not an especially complex interaction, but it points towards the possibility of the TV at least brokering conversation in a more fluid way by letting other devices (your iphone in this case) participate in turn taking.

It would be interesting to see whether a TV that politely paused itself when it detected a sufficiently high level of chatter in the room would encourage or discourage further communication. I suspect the former – like a schoolteacher adopting patient silence with pursed lips while the children settle down. It would be a fun thing to experiment with though.

But the real promise of networked TV is that it becomes more than just a turn-taker, and begins to participate more actively in the conversation.

Perhaps not quite as actively as that, but there are some amusing parallels between some of the proposals for future TV services and the TV-becoming-flesh in Cronenberg’s Videodrome.

For example, some of the most interesting social TV research I’ve found so far – namely the notube project has looked at ways of leveraging network technologies, linked data and Semantic Web strategies for enriching TV viewer’s experience.

Their ‘beancounter’ application uses a variety of sources including social network platforms, to aggregate data about what you’re watching and creates a detailed, machine readable profile of your habits. This profile can then be used to generate better recommendations, or even help to inform and improve your experience of viewing and discussing media. The really difficult bits of this problem – like trying to figure out what is actually being talked about are dealt with gracefully, using ‘good enough’ systems that evaluate multi-lingual natural language text strings and suggest concepts that they may refer to. Ontotext’s LUpedia service provides this entity recognition function for notube. {{2}}

But in a home, where the TV is in a shared space, does the TV learn from each member of the family separately? From listening to discussions at BT, I’ve learned that the ‘problem’ of knowing who is watching the TV, in order to recommend relevant and appropriate content is not going to be solved by having each watcher cumbersomely log in to the TV. Nor is it going to be entirely solved by logged-in or sensed 2nd screen companion devices (not everyone will have one). BT’s immediate strategy will apparently involve a more complex watershed, where what people watch at certain times of day will inform assumptions about who is watching, and what kinds of content to recommend. Communications companies just don’t have this level of access to monitoring our individual behaviours within the home, and there are probably serious privacy and consent implications that will be significant barriers to granting it to them.

And anyway, I’m more interested in what kind of device the TV becomes when it learns from our collective viewing habits, aggregate viewing behaviours and networked discussions. Does the networked TV begin to develop a compound user profile of it’s own? A unique combination of a household’s various proclivities? Is it like the family dog, which everyone interacts with individually and collectively, and is then seen as having a personality, to some extent nurtured through this process.{{3}}

This brings me back to the idea of the TV as a conversational participant. If it can develop a profile, and start to build a model of the various areas of interest and domains of knowledge that a user is interested in, can it participate in a conversation in a more complex way than turn-taking? One of the key ideas of conversation analysis is the notion of ‘repair‘, in which the contingent meanings of utterances between conversational partners are narrowed down and cross-checked for mutual comprehension through all kinds of gestural or verbal cues and repetitions.

Can the TV begin to engage in this level of conversation? Can it’s profile of established interests be used as a source of recommendations that might clarify a misunderstanding of something that has just been said, for example, correcting the misidentification of an actor by people chatting about what they’re watching together on facebook{{4}}. Or could it relieve the co-watcher’s burden of responding to annoying whispered questions during films (‘why is she holding that chainsaw?’) by delving into more complex layers of in-programme dramaturgic medatada and providing some suggested explanations{{5}}.

Can the profiles of individual viewers and their shared TVs then be evaluated quantitatively for similarities over specific periods of time as a measure of the effectiveness of this kind of conversational grounding with various types of content and TV format?

And at what point do we reach the threshold of complexity, fluency and multi-valency beyond which these kinds of interactions with your TV can be thought of as a conversation?

[[1]] Oehlberg, L., Ducheneaut, N., Thornton, J. D., Moore, R. J., & Nickell, E. (2006). Social TV : Designing for Distributed , Sociable Television Viewing. Theater. [[1]] [[2]] I think of this as graceful because it addresses a hugely complex set of contingencies in a simple and contingent way, by issuing a query to a good enough service via a standard API, that does something useful, and assumes that in the future, when there is more linked, semantically enriched data, and more advanced inference services available, the API can just be plugged into those. [[2]] [[3]] I’m aware that this idea that pets do not have Disney-like anthropomorphic personalities is not popular, especially with British people, and I’m not backing it up with anything other than my own supposition that this is the case, and that your animals would eat you in a second if they were hungry enough and you were incapable of defending yourself. [[3]] [[4]] In the lingo of conversation analysis this would be called ‘self-initiated self-repair’ [[4]] [[5]] Other-initiated self-repair [[5]]



Before starting research at BT, I’d never heard the term ‘demonstrators’ used to describe the deliverables of a research project. Getting to grips with the idea has given me a valuable insight into the differences between ‘intrapreneurial’ innovation in corporate contexts, and the gung-ho prototyping I’ve been involved with in many arts/technology/entrepreneurial ventures.

I got a chance to see a variety of demonstrators at a recent BT Innovate and Design ‘Deep Dive’ event, where all the geeks put on suits, polish up their wares, and come and present them to a mix of management/strategy types and other geeks.


For the hardware/infrastructure demonstrators, there were some big green BT link cabinets (such as you sometimes see hanging open, with vast spaghetti tangles of copper twisted pair bulging out) – updated with the latest in fibre coupling systems. Then there were cross-sectional fibreglass models of brick walls, showing how domestic fibre channels could be installed. Those kinds of demonstrators felt a bit like playing in the launchpad at the Science Museum.

For Software demonstrators, the killer apps seemed to be Flash and powerpoint, because they seem to be the most effective ways to illustrate the concepts and proposed functionality of consumer technologies to a broad base of management/strategy people, with varied levels of technical knowledge. {{1}}

The product our team demonstrated in this context was built in Adobe Air/Flash, with a Processing back-end server – which was effective at demonstrating the possibilities of a second screen application for playing, pausing, text chatting and audio-messaging between users of a SocialTV system.

When I had first seen it, and read user test reports, I hadn’t immediately understood what could be learned from the results. The demonstrator literally only did four things: pause, play, send text message and send audio message. I was interested in how users behaved, how they misused or adapted the technology, which wasn’t really possible with such a limited device and such a proscribed mode of use.

Then yesterday I saw this:

GOAB. A TV Experience Concept from Syzygy on Vimeo.

Which basically compiles all the Social TV functionalities and features I’ve seen built or speculated about so far. It’s really very slick, and (as far as I can tell) entirely vaporware-enabled, which is a very good way of communicating to lots of people and testing a market without the complicated and horrible job of actually building any software.

So what I was missing is that ‘demonstrators’ should be seen as mostly non-functional props, designed to make it easier to communicate the possibilities and marketability of certain technologies. Whether they *actually* work or not is immaterial.

Another gem (or mine full of gems) from recent research was the Notube project, which involves my much admired semweb veteran friends Dan and Libby, which has involved building all kinds of prototypes – but not, explicitly, any products. Libby wrote about that very interestingly, after being badgered by product-hungry trade-fair attendees. Her post, which brings up lots of very useful user-centric patterns for Social TV from her previous work at Joost, helped me to understand the different applications of prototypes and ‘demonstrators’.

Dan also pointed me to this  beautiful demonstration of one of notube’s prototypes, using the new Universal Controller for Mythtv built by Steven Jolly, James Barrett and Matt Hammond at BBC R&D. This is the kind of stick-figure prototype I really enjoy: plain and uncomplicated on the outside, but actually rather elegant and complex on the inside.

So, before diving into making, I’m glad I clarified this for myself:

You build a protytpe to learn something about how users and systems behave. You build a demonstrator to learn about how people and organisations react to your product.

In terms of my research, I’m more interested in finding out how users and systems behave, so I’ll be building some prototypes, and not worrying too much about making them look slick.

[[1]] This wasn’t the case for some internally-focused applications, which were basically fully functional systems intended for use by BT and it’s partners for cost-saving/efficiency systems, or the Saturn Project aggregator/classifier for threat detection and analysis from fuzzy data.[[1]]

Leave a comment