Toggle Main Menu Toggle Search

Open Access padlockePrints

Convolutional networks for appearance-based recommendation and visualisation of mascara products

Lookup NU author(s): Dr Chris Holder, Professor Boguslaw ObaraORCiD



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


© 2020, The Author(s). In this work, we explore the problems of recommending and visualising makeup products based on images of customers. Focusing on mascara, we propose a two-stage approach that first recommends products to a new customer based on the preferences of other customers with similar visual appearance and then visualises how the recommended products might look on the customer. For the initial product recommendation, we train a Siamese convolutional neural network, using our own dataset of cropped eye regions from images of 91 female subjects, such that it learns to output feature vectors that place images of the same subject close together in high-dimensional space. We evaluate the trained network based on its ability to correctly identify existing subjects from unseen images, and then assess its capability to identify visually similar subjects when an image of a new subject is used as input. For product visualisation, we train per-product generative adversarial networks to map the appearance of a specific product onto an image of a customer with no makeup. We train models to generate images of two mascara formulations and assess their capability to generate realistic mascara lashes while changing as little as possible within non-lash image regions and simulating the different effects of the two products used.

Publication metadata

Author(s): Holder CJ, Ricketts S, Obara B

Publication type: Article

Publication status: Published

Journal: Machine Vision and Applications

Year: 2020

Volume: 31

Issue: 1

Online publication date: 21/01/2020

Acceptance date: 03/12/2019

Date deposited: 29/04/2021

ISSN (print): 0932-8092

ISSN (electronic): 1432-1769

Publisher: Springer


DOI: 10.1007/s00138-019-01053-5


Altmetrics provided by Altmetric