Browse by author
Lookup NU author(s): Dr Amal Alkhalifa, Dr Marie DevlinORCiD
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
© (2022), (Informing Science Institute). All rights reserved. Aim/Purpose: To encourage students’ engagement in peer assessments and provide students with better-quality feedback, this paper describes a technique for author-reviewer matching in peer assessment systems – a Balanced Allocation algorithm. Background: Peer assessment concerns evaluating the work of colleagues and providing feedback on their work. This process is widely applied as a learning method to involve students in the progress of their learning. However, as students have different ability levels, the efficacy of the peer feedback differs from case to case. Thus, peer assessment may not provide satisfactory results for students. In order to mitigate this issue, this paper explains and evaluates an algorithm that matches the author to a set of reviewers. The technique matches authors and reviewers based on how difficult the authors perceived the assignment to be, and the algorithm then matches the selected author to a group of reviewers who may meet the author’s needs in regard to the selected assignment. Methodology: This study used the Multiple Criteria Decision-Making methodology (MCDM) to determine a set of reviewers from among the many available options. The weighted sum method was used because the data that have been collected in user profiles are expressed in the same unit. This study produced an experimental result, examining the algorithm with a real collected dataset and mockup dataset. In total, there were 240 students in the real dataset, and it contained self-assessment scores, peer scores, and instructor scores for the same assignment. The mock-up dataset created 1000 records for self-assessment scores. The algorithm was evaluated using focus group discussions with 29 programming students and interviews with seven programming instructors. Contribution This paper contributes to the field in the following two ways. First, an algorithm using a MCDM methodology was proposed to match authors and reviewers in order to facilitate the peer assessment process. In addition, the algorithm used self-assessment as an initial data source to match users, rather than randomly creating reviewer – author pairs. Findings The findings show the accurate results of the algorithm in matching three reviewers for each author. Furthermore, the algorithm was evaluated based on students’ and instructors’ perspectives. The results are very promising, as they depict a high level of satisfaction for the Balanced Allocation algorithm. Recommendations: We recommend instructors to consider using the Balanced Allocation algorithm for Practitioners to match students in peer assessments, and consequently to benefit from personalizing peer assessment based on students' needs. Recommendations Several MCDM methods could be expanded upon, such as the analytic hierarfor Researchers chy process (AHP) if different attributes are collected, or the artificial neural network (ANN) if fuzzy data is available in the user profile. Each method is suitable for special cases depending on the data available for decision-making. Impact on Society Suitable pairing in peer assessment would increase the credibility of the peer assessment process and encourage students’ engagement in peer assessments. Future Research: The Balanced Allocation algorithm could be applied using a single group, and a peer assessment with random matching with another group may also be conducted, followed by performing a t-test to determine the impact of matching on students’ performances in the peer assessment activity.
Author(s): Alkhalifa A, Devlin M, Alkhattabi M
Publication type: Article
Publication status: Published
Journal: Journal of Information Technology Education: Innovations in Practice
Year: 2022
Volume: 21
Pages: 136-153
Online publication date: 06/12/2022
Acceptance date: 17/11/2022
Date deposited: 01/06/2026
ISSN (print): 2165-3151
ISSN (electronic): 2165-316X
Publisher: Informing Science Institute
URL: https://doi.org/10.28945/5038
DOI: 10.28945/5038
Altmetrics provided by Altmetric
