My research explores the technology of social interaction at two ends of the spectrum of formalization. At one end, my work on conversational AI asks which features and mechanisms of human social action can be represented and modeled computationally. At the other, I study how people make aesthetic judgements and interact while dealing with underdetermined cultural objects and situations. This program spans multiple, often incompatible disciplines, so my work builds methodological interfaces between them.
I focus on three key questions: what can AI systems and their designers learn from naturalistic social interaction; what can we learn about social interaction using AI systems as research tools; and what can we learn by studying how people work together and collaborate with AI systems in everyday and applied settings such as health and social care? I presented an overview of this research program in Three Meeting Points between CA and AI – my 2020 keynote for the European Conference on Conversation Analysis
GailBot: An automatic transcription system for Conversation Analysis: GailBot is an automated transcription system named after Gail Jefferson, inventor of the transcription system used in Conversation Analysis. It uses state-of-the-art speech-to-text systems and some simple machine learning algorithms to capture and represent paralinguistic features of conversation such as laugher, pitch, speech rate, overlap, and latching.
The interactional coordination of virtual and personal assistants in a homecare setting: key interactional findings of the Adept at Adaptation project on voice interfaces in health and social care, written with Magnus Hamann and Elizabeth Stokoe for the 6th Copenhagen Multimodality DayI
An artificial turn in social interaction research?: a bibliometric review of the current state of the literature between AI, ethnomethodology, and conversation analysis written with Jakub Mlynář, Andreas Liesenfeld, Lynn de Renata Topinková, Wyke Stommel, Lynn de Rijk for the 6th Copenhagen Multimodality Day.
Putting Wake Words to Bed: a paper written with Magnus Hamann for the third conference on Conversational User Interfaces 2021 arguing that voice user interface designers should stop using wake words like “Alexa” and “Hey Siri” that are crowding each other out of the audible environment of the smart home.
Conversation analysis and conversation design: Where the moonshots are: a position paper on the vital missing pieces and (currently) unachievable goals for contemporary approaches to communications AI, written with my Loughborough colleagues Liz Stokoe, Sophie Parslow and Google’s Cathy Pearl
In Case of Emergency, Order Pizza: An Urgent Case of Action Formation and Recognition: paper with William Housley and Elizabeth Stokoe for the 1st International Conference on Conversational User Interfaces, July 2019, highlighting the shortcomings of AI for natural language understanding.
Natural Action Processing: Conversation Analysis and Big Interactional Data: paper with William Housley and Elizabeth Stokoe for Halfway to the Future, Nottingham, November 2019, arguing for the use of naturalistic data and interactional understandings of social action in NLP.
Aesthetics in Interaction
Judgements of taste are often too underdetermined and open ended for the standardized scales and metrics typical in psychology and market research. I study situations in which people must work together, in situ, to establish the relevant criteria for an evaluation, and to figure out who knows (or can claim to know) about whatever is being judged. I outlined this research program in Art as Occasion, a talk at the CAA 2017, and its implications for the creative industries in Measuring Aesthetic Value for Arts Professional.
Moving into step: The embodiment of social structures of action: While dance has often featured in sociological theory, there are relatively few empirical studies that explore the social practices through which people learn to dance together. This paper takes as its point of departure the way that partner dance is often featured as a metaphor to illustrate theories about social order and interaction.
My PhD research Respecifying Aesthetics: Accounting for taste in everyday talk, asked how people do judgements of taste in talk-in-interaction. This is the basis of my book: Accounting for Taste, under contract with Routledge’s Directions in Ethnomethodology and Conversation Analysis.
Rhythmical coordination of performers and audience in partner dance: what can audience members’ rhythmical movements tell us about their experience of a musical dance performance? And what does this reveal about the composition, organization & production of the performance itself?
Multidisciplinary research is essential for understanding the complexity of social action, but often involves incompatible assumptions, data types, and research designs. My research brings together teams with different specialisms and theoretical commitments to create new techniques, tools, and interfaces between methodological boundaries. This involves convening cross-disciplinary research networks to develop new software tools, data corpora, research training resources, and critical methodological literature.
Hannah Pelikan, Andreas Liesenfeld, Kai Cyra and I set up the EMCA/AI open research network to gather international researchers working with AI-related technologies e.g., social robots, voice interfaces & NLP. We now have ~30 members, monthly get-togethers, and a range of joint projects.
Conversation analysis at the ‘middle region’ of public life: Greetings and the interactional construction of Donald Trump’s political persona: a paper with Chase Raymond combining a media analysis focus on the personalization and mediatization of political life with conversation analytic methods.
Repair: The Interface Between Interaction and Cognition: a paper with JP de Ruiter arguing that since we use repair to deal with interactional problems at the surface level of talk, repair can be a point of convergence for studying communication from multiple methodological perspectives.
Improving Human Interaction Research through Ecological Grounding: a paper with JP de Ruiter introducing and adapting inductive research practices and theories from interaction analysis to an audience of meta/open science nerds and psychologists interested in methodological reform.
The conversational rollercoaster: Conversation analysis and the public science of talk: a paper with Charlotte Albury, Marc Alexander, Toby Harris, Emily Hofstetter, Edward Holmes, and Elizabeth Stokoe on the necessity and potential of showing interaction research at public science exhibitions.
An Appeal for a Methodological Fusion of Conversation Analysis and Experimental Psychology: paper with JP de Ruiter on the possibilities and caveats for attempts to bring together two epistemologically (and, to some extent, culturally) incompatible research methodologies and traditions.
The Conversation Analytic British National Corpus: (CABNC) is a open-licensed, detailed conversation analytic re-transcription of naturalistic conversations from a subcorpus of the British National Corpus amounting to around 4.2 million words in 1436 separate conversations. The project aims to produce transcripts usable for both computational and detailed qualitative analysis.