Title: Mental models of information retrieval systems
Abstract: Search engines have become cultural artifacts. Almost all of us use them. We ‘Google’ people, etc. Search engines are gatekeepers to content, Services, and opportunities. Thus, those who know how to make the most of search engines are better able to participate in society. Search engines are exciting! They give us an opportunity to generate interest in technology and consider how technology can make Information better. People bring mental models in their interaction with a retrieval System. These models allow people to reason about how the System operates, what kind of input should be provided and what the Output means. There appears to be great diversity in our mental models for information Systems. These models are often very simple, they contain misconceptions, and they evolve as we experience new Systems. The papers in this session look at different aspects of mental models that employ when interacting with information retrieval Systems. Savage-Knepshield investigated factors that may affect searchers' mental model construction, including: the level of detail that a system's conceptual model presents to the user through explicit training, repeated system exposure, and type of searching task. Tenopir and Wang observed science students finding information in electronic Journals for a simulated dass assignment. Student verbalizations of their reactions to the search process, electronic Journals, and information found allow the authors to draw some conclusions about students' mental models and their problem-solving strategies. Hendry and Efthimiadis' research investigates how people conceptualize search engines. What do they know about how they work? What common misunderstandings do they have? How do they express their understandings? In short, what is the nature of people's mental models for search engines? To answer these questions the authors prompted university students to draw Sketches of how an internet search engine works. Search engines have become cultural artifacts. Almost all of us use them. We ‘Google’ people, etc. Search engines are gatekeepers to content, Services, and opportunities. Thus, those who know how to make the most of search engines are better able to participate in society. Search engines are exciting! They give us an opportunity to generate interest in technology and consider how technology can make information better. The aim of this research project is to investigate how people conceptualize search engines. What do they know about how they work? What common misunderstandings do they have? How do they express their understandings? In short, what is the nature of people's mental models for search engines? Data was collected from 265 undergraduate, masters, and doctoral level students at the University of Washington. Students were asked to “draw a labeled sketch of how search engines work.” In this paper we will present the results of this study which have implications in educational assessment and curriculum design on one hand, and information system design on the other. David Hendry is Assistant Professor in the Information School, University of Washington. He received his Ph.D. in Computer Science from the Robert Gordon University in 1996. For his dissertation, he developed a user-interface architecture for implementing diverse families of information retrieval applications. He joined the dot.com movement in 1997, spending two years at a start-up that commercialized collaborative filtering. Over the next three years, as Manager of User Interface Research at Terra Lycos, he studied consumer web applications and helped teams create better user experiences. His research and teaching interests are human-computer interaction, interdisciplinary design, design and evaluation of information Systems, and end-user programming. Efthimis Efthimiadis is Associate Professor at the Information School, University of Washington, Seattle, WA. His expertise is in the area of user-centered design and evaluation of Information retrieval Systems. His research focuses on methods for improving access to databases, in methods that incorporate user preferences in the retrieval techniques, and in the use knowledge organization applications for search and navigation. Efthimiadis' research in the area of query expansion is concemed with the evaluation of ranking algorithms and the study of the searching behavior of end-users. Little is known about the relationships between people's mental models (MM) of complex Systems, the conceptual models that are presented to them, and their performance in such Systems apart from the observation that they are significant. A two-part experiment investigated factors that may affect information searchers' MM construction including: the level of detail that a System's conceptual model presents to the user through explicit training (none, minimal, in-depth), repeated System exposure (with the same or different type of searching task), and type of searching task (simple or complex). Forty-eight experienced searchers performed three searches during the first part of the experiment returning one weck later to perform three more searches. Data demonstrated that exposure to more information about the system's internal Operation through an explicit conceptual model (in-depth condition) led to the construction of a MM that was more congruent with the system's Operation and that searchers with highly congruent MMs experienced greater search satisfaction and found the System easier to use. Although those with high congruency did not have superior performance relative to those with low congruency, their performance improved over trials. Re-exposure to the same type of task in part two of the experiment resulted in greater search satisfaction regardless of task type and better performance for those who were re-exposed to the more complex search task. Pam Savage-Knepshield is a former Distinguished Member of Technical Staff with Lucent Technologies/Bell Labs, where she was a human factors specialist since 1988. In 2002, she joined Northrop Grumman Information Technology as a Senior Human Factors Engineer supporting the Transportation Security Administration's efforts to improve aviation security. Within the past year, she joined the Army Research Laboratory at Fort Monmouth as a Research Psychologist supporting efforts to understand and improve the operational and tactical design of military Systems. She received a PhD in Psychology from Rutgers University and was the recipient of the 2002 ASIST/UMI Doctoral Dissertation Award. Her research interests include problem-solving and decision-making strategies that enhance task performance and improving System usability by using good human factors methods and practices early in the front-end design process. Many recent studies have found that undergraduate students turn first to Web search engines to locate information for class-related assignments. Scholarly Journal articles are sometimes located in these searches, only within a larger context of web pages. Science Journals Systems such as Eisevier's ScienceDirect provide access to peer-reviewed science Journals and Journal articles, but many students are unaware of these Systems or the attributes of peer-reviewed Journals. In a larger study of the use of peer reviewed Journals in the undergraduate science classroom, we observed science students finding information in Journal article Systems for a simulated dass assignment. Student verbalizations of their reactions to the search process, Journals system, and information found allow us to draw some conclusions about students' mental models of electronic Journal Systems and their problem-solving strategies during the interactions in addition to their perceptions of peer reviewed e-journal articles. Carol Tenopir is a Professor at the School of Information Sciences at the University of Tennessee, Knoxville. She is the author of five books, including, Communication Patterns of Engineers (John Wiley for IEEE Press, 2004) with Donald W. King and over 200 Journal articles. She is the recipient of the 1993 Outstanding Information Science Teacher Award from the American Society for Information Science/Institute for Scientific Information, the 2000 ALISE Award for Teaching Excellence and the 2002 ASIST Research Award. Dr. Tenopir holds a PhD degree in Library and Information Science from the University of Illinois. Peiling Wang is an Associate Professor at the School of Information Sciences at the University of Tennessee, Knoxville. She holds a Ph.D. in information science from the University of Maryland and an M.S. in information science and B.S. in Chemical Engineering from China. Her research areas include information seeking and retrieval, user-system interactions and usability, user modeling and system design, cognitive behaviors, knowledge structures, data mining and knowledge discovery, research methodologies and methods, and citation analysis. She is an active member of ASIST and a frequent Speaker at Professional meetings. Dr. Wang is the recipient of the 1994 ASIST Doctoral Forum Award and the 1999 best JASIST paper award.