ECS Research
Publications
Intranet Tools

nb. next round of REF2013 will NOT be using data from eprints.ecs, but the central university REF interface.

RSS 1.0 Feed
RSS 2.0 Feed
Atom Feed
 

Evaluating Advanced Search Interfaces using Established Information-Seeking Models

Wilson, M. L., schraefel, m. c. and White, R. W. (2009) Evaluating Advanced Search Interfaces using Established Information-Seeking Models. In the Journal of the American Society for Information Science and Technology, 60 (7). pp. 1407-1422.

Download

[img]
Preview
 Accepted Version
 PDF
2051Kb

Abstract

When users have poorly defined or complex goals search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.

Item Type:Article
Creator/Authors:
Max L Wilson
m.c. schraefel
Ryen W. White
Keywords:IR, Evaluation, Exploratory, Search, Browse, Interactivity
Research Group:Old ECS Groups > Intelligence, Agents, Multimedia
Current ECS Groups > Agents, Interaction and Complexity
Date:July 2009
Information about this record:
Performance Indicator:EZ~03~02~11
Citations:ISI: 6, Google Scholar: 34
Downloads (2010):172
ID Code:14301
Last Modified:23 Sep 2011 10:35
Deposited On:12 Jul 2007 by Wilson, Max

Tools & Metadata

Download Statistics

Last month

Last year

Members of ECS may view the download statistics dashboard for this record.

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in this archive you will be forwarded to the paracite service. Poorly formated references will probably not work.

Bates, M. J. (1979a). Idea tactics. Journal of the American Society for Information Science, 30(5), 280-289.

Bates, M. J. (1979b). Information search tactics. Journal of the American Society for Information Science, 30, 205-214.

Bates, M. J. (1990). Where should the person stop and the information search interface start? Information Processing and Management, 26(5), 575-591.

Bates, M. J. (2002). The cascade of interactions in the digital library interface. Information Processing and Management, 38(3), 381-400.

Beaulieu, M. (1997). Experiments on interfaces to support query expansion. Journal of Documentation, 53(1), 8-19.

Belkin, N. J. (2000). Helping people find what they don't know. Communications of the ACM, 43(8), 58-61.

Belkin, N. J., Cool, C., Stein, A., & Thiel, U. (1995). Cases, scripts, and information-seeking strategies: on the design of interactive information retrieval systems. Expert Systems with Applications, 9(3), 379-395.

Belkin, N. J., Oddy, R. N., & Brooks, H. M. (1982). Ask for information retrieval: parts I and II. Journal of Documentation, 38(2), 3.

Borlund, P. (2003). The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research, 8(3), Paper 152.

Bruza, P., McArthur, R., & Dennis, S. (2000). Interactive Internet search: keyword, directory and query reformulation mechanisms compared. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 280-287l.

Card, S. K., Moran, T. P., & Newell, A. (1980). The keystroke-level model for user performance time with interactive systems. Communications of the ACM, 23(7), 396-410.

Chi, E. H., Pirolli, P., Chen, K., & Pitkow, J. (2001). Using information scent to model user information needs and actions and the web. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems, 490-497l.

Cleverdon, C. W., Mills, J., Keen, M., & Aslib Cranfield Research, P. (1966). Factors determining the performance of indexing systems: College of Aeronautics.

Cool, C., & Belkin, N. J. (2002). A classification of interactions with information. In Proceedings of the 4th International Conference on Conceptions of Library and Information Science, 1-15l.

Fu, W., & Pirolli, P. (2007). SNIF-ACT: a cognitive model of user navigation on the World Wide Web. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems,

Godbold, N. (2006). Beyond information seeking: towards a general model of information behaviour. Information Research, 11(4), Paper 269.

Gong, R., & Kieras, D. (1994). A validation of the GOMS model methodology in the development of a specialized, commercial software application. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 351-357l.

Harman, D. K. (1997). The TREC conferences. Morgan Kaufmann Multimedia Information And Systems Series, 247-256.

Hearst, M. A. (2000). Next generation web search: setting our sites. IEEE Data Engineering Bulletin: Special Issue on Next Generation Web Search, 23(3), 38-48.

Hildebreand, M., Ossenbruggen, J. v., & Hardman, L. (2006). /facet: a browser for heterogeneous semantic web repositories. In Proceedings of the 5th International Conference on the Semantic Web, 272-285l.

Lagergren, E., & Over, P. (1998). Comparing interactive information retrieval systems across sites: The TREC-6 interactive track matrix experiment. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 164-172l.

Liu, B., & Oard, D. W. (2006). One-sided measures for evaluating ranked retrieval effectiveness with spontaneous conversational speech. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 673-674l.

Marchionini, G. (2006). Exploratory search: from finding to understanding. Communications of the ACM, 49(4), 41-46.

Pharo, N. (1999). Web information search strategies: A model for classifying web interaction. In Proceedings of the 3rd International Conference on the Conceptions of the Library and Information Science, 207–218l.

Robertson, G. G., & Mackinlay, J. D. (1993). The document lens. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology, 101-108l.

Robertson, S. E., & Hancock-Beaulieu, M. M. (1992). On the evaluation of IR systems. Information Processing and Management, 28(4), 457-466.

Rocchio, J. (1971). Relevance feedback in information retrieval. The SMART Retrieval System: Experiments in Automatic Document Processing, 313-323.

Salton, G., & Buckley, C. (1990). Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4), 288-297.

Saracevic, T. (1997). The stratified model of information retrieval interaction: extension and applications. In Proceedings of the American Society for Information Science, 313-327l.

schraefel, m. c., Wilson, M. L., Russell, A., & Smith, D. A. (2006). mSpace: improving information access to multimedia domains with multimodal exploratory search. Communications of the ACM, 49(4), 47-49.

schraefel, m. c., Zhu, Y., Modjeska, D., Wigdor, D., & Zhao, S. (2002). Hunter gatherer: interaction support for the creation and management of within-web-page collections. In Proceedings of the 11th International Conference on World Wide Web, 172-181l.

Shneiderman, B. (1981). Direct manipulation: A step beyond programming languages.

Spink, A., Greisdorf, H., & Bateman, J. (1998). From highly relevant to not relevant: examining different regions of relevance. Information Processing and Management: an International Journal, 34(5), 599-621.

Taylor, R. S. (1968). Question-negotiation and information-seeking in libraries (Vol. 29): College and Research Libraries.

White, R. W., Kules, B., Drucker, S. M., & schraefel, m. c. (2006). Introduction. Communications of the ACM, 49(4), 36-39.

White, R. W., Ruthven, I., Jose, J. M., & Rijsbergen, C. J. V. (2005). Evaluating implicit feedback models using searcher simulations. ACM Transactions on Information Systems, 23(3), 325-361.

Wilson, M. L., & schraefel, m. c. (2006). mSpace: what do numbers and totals mean in a flexible semantic browser. In Proceedings of the 3rd International Semantic Web User Interaction Workshop,

Yee, K.-P., Swearingen, K., Li, K., & Hearst, M. (2003). Faceted metadata for image search and browsing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 401-408l.

Zamir, O., Etzioni, O., Madani, O., & Karp, R. M. (1997). Fast and intuitive clustering of web documents. Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, 287–290.

Zhang, J., & Marchionini, G. (2005). Evaluation and evolution of a browse and search interface: relation browser. In Proceedings of the 2005 National Conference on Digital Government Research, 179-188l.

Corrections

ECS staff and postgraduates may modify this record

   Welcome from Deputy Head of School (Research) Research Prospectus Industrial Partnerships New Research Students Notes for Guidance New Research Students Notes for Guidance
The ECS EPrints Repository supports OAI 2.0 with a base URL of http://eprints.ecs.soton.ac.uk/cgi/oai2

EPrints is free software developed by the University of Southampton to facilitate Open Access to research.		 EPrints