Research

My research explores the technology of social interaction at two ends of the spectrum of formalization. At one end, my work on conversational AI asks which features and mechanisms of human social action can be represented and modeled computationally. At the other, I study how people make aesthetic judgements and interact while dealing with underdetermined cultural objects and situations. This program spans multiple, often incompatible disciplines, so my work builds methodological interfaces between them.

Conversational AI

I focus on three key questions: what can AI systems and their designers learn from naturalistic social interaction; what can we learn about social interaction using AI systems as research tools; and what can we learn by studying how people work together and collaborate with AI systems in everyday and applied settings such as health and social care? I presented an overview of this research program in Three Meeting Points between CA and AI – my 2020 keynote for the European Conference on Conversation Analysis

2022

GailBot: An automatic transcription system for Conversation Analysis: GailBot is an automated transcription system named after Gail Jefferson, inventor of the transcription system used in Conversation Analysis. It uses state-of-the-art speech-to-text systems and some simple machine learning algorithms to capture and represent paralinguistic features of conversation such as laugher, pitch, speech rate, overlap, and latching.

2021

The interactional coordination of virtual and personal assistants in a homecare setting: key interactional findings of the Adept at Adaptation project on voice interfaces in health and social care, written with Magnus Hamann and Elizabeth Stokoe for the 6th Copenhagen Multimodality DayI

2021

An artificial turn in social interaction research?: a bibliometric review of the current state of the literature between AI, ethnomethodology, and conversation analysis written with Jakub Mlynář, Andreas Liesenfeld, Lynn de Renata Topinková, Wyke Stommel, Lynn de Rijk for the 6th Copenhagen Multimodality Day.

2021

Putting Wake Words to Bed: a paper written with Magnus Hamann for the third conference on Conversational User Interfaces 2021 arguing that voice user interface designers should stop using wake words like “Alexa” and “Hey Siri” that are crowding each other out of the audible environment of the smart home. 

2021

Conversation analysis and conversation design: Where the moonshots are: a position paper on the vital missing pieces and (currently) unachievable goals for contemporary approaches to communications AI, written with my Loughborough colleagues Liz StokoeSophie Parslow and Google’s Cathy Pearl

2019

In Case of Emergency, Order Pizza: An Urgent Case of Action Formation and Recognition: paper with William Housley and Elizabeth Stokoe for the 1st International Conference on Conversational User Interfaces, July 2019, highlighting the shortcomings of AI for natural language understanding.

2019

Natural Action Processing: Conversation Analysis and Big Interactional Data: paper with William Housley and Elizabeth Stokoe for Halfway to the Future, Nottingham, November 2019, arguing for the use of naturalistic data and interactional understandings of social action in NLP.

Aesthetics in Interaction

Judgements of taste are often too underdetermined and open ended for the standardized scales and metrics typical in psychology and market research. I study situations in which people must work together, in situ, to establish the relevant criteria for an evaluation, and to figure out who knows (or can claim to know) about whatever is being judged. I outlined this research program in Art as Occasion, a talk at the CAA 2017, and its implications for the creative industries in Measuring Aesthetic Value for Arts Professional.

Upcoming

Moving into step: The embodiment of social structures of action: While dance has often featured in sociological theory, there are relatively few empirical studies that explore the social practices through which people learn to dance together. This paper takes as its point of departure the way that partner dance is often featured as a metaphor to illustrate theories about social order and interaction.

2018-

Drawing interactions project in collaboration with Claude Heath, Sophie Skatch, Toby Harris and Pat Healey develops new techniques, workshops and tools for the transcription, analysis and presentation of social interaction based on traditional artistic methods of life drawing and still life.

2017-

My PhD research Respecifying Aesthetics: Accounting for taste in everyday talk, asked how people do judgements of taste in talk-in-interaction. This is the basis of my book: Accounting for Taste, under contract with Routledge’s Directions in Ethnomethodology and Conversation Analysis.

2015-

Dance as interaction is an ongoing collaboration with Dirk vom Lehn focusing on how novice dancers in a beginners’ workshop become competent members of a partner dance culture, and asking what their methods of learning together can tell us about social and cultural theories of action.

2015

Rhythmical coordination of performers and audience in partner dance: what can audience members’ rhythmical movements tell us about their experience of a musical dance performance? And what does this reveal about the composition, organization & production of the performance itself?

Methodological Interfaces

Multidisciplinary research is essential for understanding the complexity of social action, but often involves incompatible assumptions, data types, and research designs. My research brings together teams with different specialisms and theoretical commitments to create new techniques, tools, and interfaces between methodological boundaries. This involves convening cross-disciplinary research networks to develop new software tools, data corpora, research training resources, and critical methodological literature.

2021

Eef Hogervorst and I recently started the Applied Cognition Technology & Interaction Group (ACTInG) to support interdisciplinary exchange in biomedical, psychological, design, & interactional research into dementia and related disabilities – and our first ACTInG Research Networking Day.

2020

Hannah Pelikan, Andreas Liesenfeld, Kai Cyra and I set up the EMCA/AI open research network to gather international researchers working with AI-related technologies e.g., social robots, voice interfaces & NLP. We now have ~30 members, monthly get-togethers, and a range of joint projects.

2019

Conversation analysis at the ‘middle region’ of public life: Greetings and the interactional construction of Donald Trump’s political persona: a paper with Chase Raymond combining a media analysis focus on the personalization and mediatization of political life with conversation analytic methods.

2019

Drawing as transcription: how do graphical techniques inform interaction analysis? a paper based on the Drawing Interactions project with Claude Heath, Sophie Skatch, Toby Harris, Madeline Miller, and Pat Healey exploring the methodological implications of drawing for interaction research.

2018

Repair: The Interface Between Interaction and Cognition: a paper with JP de Ruiter arguing that since we use repair to deal with interactional problems at the surface level of talk, repair can be a point of convergence for studying communication from multiple methodological perspectives.

2018

Improving Human Interaction Research through Ecological Grounding: a paper with JP de Ruiter introducing and adapting inductive research practices and theories from interaction analysis to an audience of meta/open science nerds and psychologists interested in methodological reform.

2018-

Drawing interactions project in collaboration with Claude Heath, Sophie Skatch, Toby Harris and Pat Healey develops new techniques, workshops and tools for the transcription, analysis and presentation of social interaction based on traditional artistic methods of life drawing and still life.

2018-

The conversational rollercoaster: Conversation analysis and the public science of talk: a paper with Charlotte Albury, Marc Alexander, Toby Harris, Emily Hofstetter, Edward Holmes, and Elizabeth Stokoe on the necessity and potential of showing interaction research at public science exhibitions.

2017

An Appeal for a Methodological Fusion of Conversation Analysis and Experimental Psychology: paper with JP de Ruiter on the possibilities and caveats for attempts to bring together two epistemologically (and, to some extent, culturally) incompatible research methodologies and traditions.

2015

The Conversation Analytic British National Corpus: (CABNC) is a open-licensed, detailed conversation analytic re-transcription of naturalistic conversations from a subcorpus of the British National Corpus amounting to around 4.2 million words in 1436 separate conversations.  The project aims to produce transcripts usable for both computational and detailed qualitative analysis.