My research explores the technology of conversation: what interactional things computers can do, such as helping us work together and they can’t do, such as aesthetic judgements and cultural practices. This interdisciplinary work involves building methodological interfaces between HCI, psychology, and conversation analysis.
Conversational AI
I focus on three key questions: what can AI systems and their designers learn from naturalistic social interaction; what can we learn about social interaction using AI systems as research tools; and what can we learn by studying how people work together and collaborate with AI systems in everyday and applied settings such as health and social care? I presented an overview of this research program in Three Meeting Points between CA and AI – my 2020 keynote for the European Conference on Conversation Analysis
2022
GailBot: An automatic transcription system for Conversation Analysis: GailBot is an automated transcription system named after Gail Jefferson, inventor of the transcription system used in Conversation Analysis. It uses state-of-the-art speech-to-text systems and some simple machine learning algorithms to capture and represent paralinguistic features of conversation such as laugher, pitch, speech rate, overlap, and latching.
2021
The interactional coordination of virtual and personal assistants in a homecare setting: key interactional findings of the Adept at Adaptation project on voice interfaces in health and social care, written with Magnus Hamann and Elizabeth Stokoe for the 6th Copenhagen Multimodality DayI
2021
An artificial turn in social interaction research?: a bibliometric review of the current state of the literature between AI, ethnomethodology, and conversation analysis written with Jakub Mlynář, Andreas Liesenfeld, Lynn de Renata Topinková, Wyke Stommel, Lynn de Rijk for the 6th Copenhagen Multimodality Day.
2021
Putting Wake Words to Bed: a paper written with Magnus Hamann for the third conference on Conversational User Interfaces 2021 arguing that voice user interface designers should stop using wake words like “Alexa” and “Hey Siri” that are crowding each other out of the audible environment of the smart home.
2021
Conversation analysis and conversation design: Where the moonshots are: a position paper on the vital missing pieces and (currently) unachievable goals for contemporary approaches to communications AI, written with my Loughborough colleagues Liz Stokoe, Sophie Parslow and Google’s Cathy Pearl
2019
In Case of Emergency, Order Pizza: An Urgent Case of Action Formation and Recognition: paper with William Housley and Elizabeth Stokoe for the 1st International Conference on Conversational User Interfaces, July 2019, highlighting the shortcomings of AI for natural language understanding.
2019
Natural Action Processing: Conversation Analysis and Big Interactional Data: paper with William Housley and Elizabeth Stokoe for Halfway to the Future, Nottingham, November 2019, arguing for the use of naturalistic data and interactional understandings of social action in NLP.
Aesthetics in Interaction
Judgements of taste are often too underdetermined and open ended for the standardized scales and metrics typical in psychology and market research. I study situations in which people must work together, in situ, to establish the relevant criteria for an evaluation, and to figure out who knows (or can claim to know) about whatever is being judged. I outlined this research program in Art as Occasion, a talk at the CAA 2017, and its implications for the creative industries in Measuring Aesthetic Value for Arts Professional.
Upcoming
Moving into step: The embodiment of social structures of action: While dance has often featured in sociological theory, there are relatively few empirical studies that explore the social practices through which people learn to dance together. This paper takes as its point of departure the way that partner dance is often featured as a metaphor to illustrate theories about social order and interaction.
2018-
Drawing interactions project in collaboration with Claude Heath, Sophie Skatch, Toby Harris and Pat Healey develops new techniques, workshops and tools for the transcription, analysis and presentation of social interaction based on traditional artistic methods of life drawing and still life.
2017-
My PhD research Respecifying Aesthetics: Accounting for taste in everyday talk, asked how people do judgements of taste in talk-in-interaction. This is the basis of my book: Accounting for Taste, under contract with Routledge’s Directions in Ethnomethodology and Conversation Analysis.
2015-
Dance as interaction is an ongoing collaboration with Dirk vom Lehn focusing on how novice dancers in a beginners’ workshop become competent members of a partner dance culture, and asking what their methods of learning together can tell us about social and cultural theories of action.
2015
Rhythmical coordination of performers and audience in partner dance: what can audience members’ rhythmical movements tell us about their experience of a musical dance performance? And what does this reveal about the composition, organization & production of the performance itself?
Methodological Interfaces
Multidisciplinary research is essential for understanding the complexity of social action, but often involves incompatible assumptions, data types, and research designs. My research brings together teams with different specialisms and theoretical commitments to create new techniques, tools, and interfaces between methodological boundaries. This involves convening cross-disciplinary research networks to develop new software tools, data corpora, research training resources, and critical methodological literature.
2021
Eef Hogervorst and I recently started the Applied Cognition Technology & Interaction Group (ACTInG) to support interdisciplinary exchange in biomedical, psychological, design, & interactional research into dementia and related disabilities – and our first ACTInG Research Networking Day.
2020
Hannah Pelikan, Andreas Liesenfeld, Kai Cyra and I set up the EMCA/AI open research network to gather international researchers working with AI-related technologies e.g., social robots, voice interfaces & NLP. We now have ~30 members, monthly get-togethers, and a range of joint projects.
2019
Conversation analysis at the ‘middle region’ of public life: Greetings and the interactional construction of Donald Trump’s political persona: a paper with Chase Raymond combining a media analysis focus on the personalization and mediatization of political life with conversation analytic methods.
2019
Drawing as transcription: how do graphical techniques inform interaction analysis? a paper based on the Drawing Interactions project with Claude Heath, Sophie Skatch, Toby Harris, Madeline Miller, and Pat Healey exploring the methodological implications of drawing for interaction research.
2018
Repair: The Interface Between Interaction and Cognition: a paper with JP de Ruiter arguing that since we use repair to deal with interactional problems at the surface level of talk, repair can be a point of convergence for studying communication from multiple methodological perspectives.
2018
Improving Human Interaction Research through Ecological Grounding: a paper with JP de Ruiter introducing and adapting inductive research practices and theories from interaction analysis to an audience of meta/open science nerds and psychologists interested in methodological reform.
2018-
Drawing interactions project in collaboration with Claude Heath, Sophie Skatch, Toby Harris and Pat Healey develops new techniques, workshops and tools for the transcription, analysis and presentation of social interaction based on traditional artistic methods of life drawing and still life.
2018-
The conversational rollercoaster: Conversation analysis and the public science of talk: a paper with Charlotte Albury, Marc Alexander, Toby Harris, Emily Hofstetter, Edward Holmes, and Elizabeth Stokoe on the necessity and potential of showing interaction research at public science exhibitions.
2017
An Appeal for a Methodological Fusion of Conversation Analysis and Experimental Psychology: paper with JP de Ruiter on the possibilities and caveats for attempts to bring together two epistemologically (and, to some extent, culturally) incompatible research methodologies and traditions.
2015
The Conversation Analytic British National Corpus: (CABNC) is a open-licensed, detailed conversation analytic re-transcription of naturalistic conversations from a subcorpus of the British National Corpus amounting to around 4.2 million words in 1436 separate conversations. The project aims to produce transcripts usable for both computational and detailed qualitative analysis.