Toggle Main Menu Toggle Search

Open Access padlockePrints

The Face of Deception: The Impact of AI-Generated Photos on Malicious Social Bots

Lookup NU author(s): Dr Lei ShiORCiD, Professor Aad van Moorsel

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


Abstract

© 2014 IEEE. In this research, we investigate the influence of utilizing artificial intelligence (AI)-generated photographs on malicious bots that engage in disinformation, fraud, reputation manipulation, and other types of malicious activity on social networks. Our research aims to compare the performance metrics of social bots that employ AI photos with those that use other types of photographs. To accomplish this, we analyzed a dataset with 13 748 measurements of 11 423 bots from the VK social network and identified 73 cases where bots employed generative adversarial network (GAN)-photos and 84 cases where bots employed diffusion or transformers photos. We conducted a qualitative comparison of these bots using metrics such as price, survival rate, quality, speed, and human trust. Our study findings indicate that bots that use AI-photos exhibit less danger and lower levels of sophistication compared to other types: AI-enhanced bots are less expensive, less popular on exchange platforms, of inferior quality, less likely to be operated by humans, and, as a consequence, faster and more susceptible to being blocked by social networks. We also did not observe any significant difference between GAN-based and diffusion/transformers-based bots, indicating that diffusion/transformers models did not contribute to increased bot sophistication compared to GAN models. Our contributions include a proposed methodology for evaluating the impact of photos on bot sophistication, along with a publicly available dataset for other researchers to study and analyze bots. Our research findings suggest a contradiction to theoretical expectations: in practice, bots using AI-generated photos pose less danger.


Publication metadata

Author(s): Kolomeets M, Wu H, Shi L, van Moorsel A

Publication type: Article

Publication status: Published

Journal: IEEE Transactions on Computational Social Systems

Year: 2024

Pages: ePub ahead of Print

Online publication date: 09/10/2024

Acceptance date: 12/09/2024

ISSN (electronic): 2329-924X

Publisher: IEEE

URL: https://doi.org/10.1109/TCSS.2024.3461328

DOI: 10.1109/TCSS.2024.3461328


Altmetrics

Altmetrics provided by Altmetric


Funding

Funder referenceFunder name
UK Research and Innovation: EP/W032481/2

Share