Toggle Main Menu Toggle Search

Open Access padlockePrints

Preserving the value of large scale data analytics over time through selective re-computation

Lookup NU author(s): Professor Paolo MissierORCiD, Dr Jacek CalaORCiD, Dr Manisha Rathi



This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).


© Springer International Publishing AG 2017. A pervasive problem in Data Science is that the knowledge generated by possibly expensive analytics processes is subject to decay over time as the data and algorithms used to compute it change, and the external knowledge embodied by reference datasets evolves. Deciding when such knowledge outcomes should be refreshed, following a sequence of data change events, requires problem-specific functions to quantify their value and its decay over time, as well as models for estimating the cost of their re-computation. Challenging is the ambition to develop a decision support system for informing re-computation decisions over time that is both generic and customisable.With the help of a case study from genomics, in this paper we offer an initial formalisation of this problem, highlight research challenges, and outline a possible approach based on the analysis of metadata from a history of past computations.

Publication metadata

Author(s): Missier P, Cala J, Rathi M

Publication type: Conference Proceedings (inc. Abstract)

Publication status: Published

Conference Name: BICOD 2017 31st British International Conference on Databases

Year of Conference: 2017

Pages: 65-77

Print publication date: 14/06/2017

Online publication date: 14/06/2017

Acceptance date: 02/04/2016

Date deposited: 26/08/2017

Publisher: Springer Verlag


DOI: 10.1007/978-3-319-60795-5_6

Library holdings: Search Newcastle University Library for this item

Series Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ISBN: 9783319607948