Toggle Main Menu Toggle Search

Open Access padlockePrints

Cross-modal associations and synesthesia: Categorical perception and structure in vowel-color mappings in a large online sample

Lookup NU author(s):

Downloads


Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


Abstract

We report associations between vowel sounds, graphemes, and colors collected online from over 1,000 Dutch speakers. We also provide open materials, including a Python implementation of the structure measure and code for a single-page web application to run simple cross-modal tasks. We also provide a full dataset of color–vowel associations from 1,164 participants, including over 200 synesthetes identified using consistency measures. Our analysis reveals salient patterns in the cross-modal associations and introduces a novel measure of isomorphism in cross-modal mappings. We found that, while the acoustic features of vowels significantly predict certain mappings (replicating prior work), both vowel phoneme category and grapheme category are even better predictors of color choice. Phoneme category is the best predictor of color choice overall, pointing to the importance of phonological representations in addition to acoustic cues. Generally, high/front vowels are lighter, more green, and more yellow than low/back vowels. Synesthetes respond more strongly on some dimensions, choosing lighter and more yellow colors for high and mid front vowels than do nonsynesthetes. We also present a novel measure of cross-modal mappings adapted from ecology, which uses a simulated distribution of mappings to measure the extent to which participants’ actual mappings are structured isomorphically across modalities. Synesthetes have mappings that tend to be more structured than nonsynesthetes’, and more consistent color choices across trials correlate with higher structure scores. Nevertheless, the large majority (~ 70%) of participants produce structured mappings, indicating that the capacity to make isomorphically structured mappings across distinct modalities is shared to a large extent, even if the exact nature of the mappings varies across individuals. Overall, this novel structure measure suggests a distribution of structured cross-modal association in the population, with synesthetes at one extreme and participants with unstructured associations at the other.


Publication metadata

Author(s): Cuskley CF, Dingemanse M, Kirby S, van Leeuwen TV

Publication type: Article

Publication status: Published

Journal: Behavior Research Methods

Year: 2019

Volume: 51

Pages: 1651-1675

Print publication date: 15/08/2019

Online publication date: 03/04/2019

Acceptance date: 11/01/2019

Date deposited: 09/05/2019

ISSN (print): 1554-351X

ISSN (electronic): 1554-3528

Publisher: Springer

URL: https://doi.org/10.3758/s13428-019-01203-7

DOI: 10.3758/s13428-019-01203-7


Altmetrics

Altmetrics provided by Altmetric


Actions

Find at Newcastle University icon    Link to this publication


Share