Toggle Main Menu Toggle Search

Open Access padlockePrints

Exploring How Interactive Technology Enhances Gesture-Based Expression and Engagement: A Design Study

Lookup NU author(s): Shichao Zhao



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


The interpretation and understanding of physical gestures play a significant role in various forms of art. Interactive technology and digital devices offer a plethora of opportunities for personal gesture-based experience and they assist in the creation of collaborative artwork. In this study, three prototypes for use with different digital devices (digital camera, PC camera, and Kinect) were designed. Subsequently, a series of workshops were conducted and in-depth interviews with participants from different cultural and occupational backgrounds. The latter were designed to explore how to specifically design personalised gesture-based expressions and how to engage the creativity of the participants in their gesture-based experiences. The findings indicated that, in terms of gesture-based interaction, the participants preferred to engage with the visual traces that were displayed at specific timings in multi-experience spaces. Their gesture-based interactions could effectively support non-verbal emotional expression. In addition, the participants were shown to be strongly inclined to combine their personal stories and emotions into their own gesture-based artworks. Based on the participants’ different cultural and occupational backgrounds, their artistic creation could be spontaneously formed.

Publication metadata

Author(s): Zhao SC

Publication type: Article

Publication status: Published

Journal: Multimodal Technologies and Interaction

Year: 2019

Volume: 3

Issue: 1

Online publication date: 27/02/2019

Acceptance date: 23/02/2019

Date deposited: 12/03/2019

ISSN (print): 2414-4088

ISSN (electronic): 2414-4088

Publisher: Multidisciplinary Digital Publishing Institute


DOI: 10.3390/mti3010013


Altmetrics provided by Altmetric


Find at Newcastle University icon    Link to this publication