Design factors accepted.pdf (439.44 kB)
Design factors affecting relevance judgment behaviour in the context of metadata surrogates
journal contribution
posted on 2013-09-19, 15:21 authored by Panagiotis Balatsoukas, Ann O'Brien, Anne MorrisThis paper reports on the findings of a user study which investigated the effects of structure, highlighting and
quantity of information on university students’ interaction with metadata surrogates of learning object repositories
(LORs) during the relevance judgement process. The literature review revealed a lack of studies investigating
how metadata surrogates should be designed to meet users’ needs and improve the effectiveness of
relevance judgement. In order to address this issue, different versions of a prototype system, called METALOR
2, were developed and evaluated in terms of the time needed for users to find relevant information, the
accuracy of their relevance judgements and their satisfaction. To evaluate the prototype, a user study was
conducted where participants were asked to complete a set of tasks and fill in satisfaction questionnaires.
The findings showed that participants performed better and were more satisfied with those versions of the
prototype which highlighted metadata elements that included the query terms, used clusters or categories to
organize metadata elements in the surrogates and included metadata elements which were relevant to the
query or task at hand. The paper concludes with some recommendations for improving the design of metadata
surrogates in search result interfaces.
History
School
- Science
Department
- Information Science
Citation
BALATSOUKAS, P., O'BRIEN, A. and MORRIS, A., 2010. Design factors affecting relevance judgment behaviour in the context of metadata surrogates. Journal of Information Science, 36 (6), pp. 780 - 797.Publisher
SAGE Publications on behalf CILIP © The author(s)Version
- AM (Accepted Manuscript)
Publication date
2010Notes
This article was published in the Journal of Information Science [© The author(s)] and the definitive version is available at: http://dx.doi.org/10.1177/0165551510386174ISSN
0165-5515Publisher version
Language
- en