Toggle Main Menu Toggle Search

Open Access padlockePrints

Is it feasible to detect FLOSS version release events from textual messages? A case study on Stack Overflow

Lookup NU author(s): Artur Sokolovsky, Dr Thomas Gross, Dr Jaume Bacardit

Downloads


Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


Abstract

© 2021 Sokolovsky et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Topic Detection and Tracking (TDT) is a very active research question within the area of text mining, generally applied to news feeds and Twitter datasets, where topics and events are detected. The notion of "event"is broad, but typically it applies to occurrences that can be detected from a single post or a message. Little attention has been drawn to what we call "micro-events", which, due to their nature, cannot be detected from a single piece of textual information. The study investigates the feasibility of micro-event detection on textual data using a sample of messages from the Stack Overflow Q&A platform and Free/Libre Open Source Software (FLOSS) version releases from Libraries.io dataset. We build pipelines for detection of micro-events using three different estimators whose parameters are optimized using a grid search approach. We consider two feature spaces: LDA topic modeling with sentiment analysis, and hSBM topics with sentiment analysis. The feature spaces are optimized using the recursive feature elimination with cross validation (RFECV) strategy. In our experiments we investigate whether there is a characteristic change in the topics distribution or sentiment features before or after micro-events take place and we thoroughly evaluate the capacity of each variant of our analysis pipeline to detect micro-events. Additionally, we perform a detailed statistical analysis of the models, including influential cases, variance inflation factors, validation of the linearity assumption, pseudo R2 measures and no-information rate. Finally, in order to study limits of micro-event detection, we design a method for generating micro-event synthetic datasets with similar properties to the real-world data, and use them to identify the micro-event detectability threshold for each of the evaluated classifiers.


Publication metadata

Author(s): Sokolovsky A, Gross T, Bacardit J

Publication type: Article

Publication status: Published

Journal: PLoS ONE

Year: 2021

Volume: 16

Issue: 2

Online publication date: 04/02/2021

Acceptance date: 05/01/2021

Date deposited: 26/03/2021

ISSN (electronic): 1932-6203

Publisher: Public Library of Science

URL: https://doi.org/10.1371/journal.pone.0246464

DOI: 10.1371/journal.pone.0246464


Altmetrics

Altmetrics provided by Altmetric


Actions

Find at Newcastle University icon    Link to this publication


Share