Loughborough University
Browse
Design factors accepted.pdf (439.44 kB)

Design factors affecting relevance judgment behaviour in the context of metadata surrogates

Download (439.44 kB)
journal contribution
posted on 2013-09-19, 15:21 authored by Panagiotis Balatsoukas, Ann O'Brien, Anne Morris
This paper reports on the findings of a user study which investigated the effects of structure, highlighting and quantity of information on university students’ interaction with metadata surrogates of learning object repositories (LORs) during the relevance judgement process. The literature review revealed a lack of studies investigating how metadata surrogates should be designed to meet users’ needs and improve the effectiveness of relevance judgement. In order to address this issue, different versions of a prototype system, called METALOR 2, were developed and evaluated in terms of the time needed for users to find relevant information, the accuracy of their relevance judgements and their satisfaction. To evaluate the prototype, a user study was conducted where participants were asked to complete a set of tasks and fill in satisfaction questionnaires. The findings showed that participants performed better and were more satisfied with those versions of the prototype which highlighted metadata elements that included the query terms, used clusters or categories to organize metadata elements in the surrogates and included metadata elements which were relevant to the query or task at hand. The paper concludes with some recommendations for improving the design of metadata surrogates in search result interfaces.

History

School

  • Science

Department

  • Information Science

Citation

BALATSOUKAS, P., O'BRIEN, A. and MORRIS, A., 2010. Design factors affecting relevance judgment behaviour in the context of metadata surrogates. Journal of Information Science, 36 (6), pp. 780 - 797.

Publisher

SAGE Publications on behalf CILIP © The author(s)

Version

  • AM (Accepted Manuscript)

Publication date

2010

Notes

This article was published in the Journal of Information Science [© The author(s)] and the definitive version is available at: http://dx.doi.org/10.1177/0165551510386174

ISSN

0165-5515

Language

  • en