Article

Towards task-based personal information management evaluations

Authors:

David Elsweiler,

Ian RuthvenAuthors Info & Claims

SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 23 - 30

https://doi.org/10.1145/1277741.1277748

Published: 23 July 2007 Publication History

Abstract

Personal Information Management (PIM) is a rapidly growing area of research concerned with how people store, manage and refind information. A feature of PIM research is that many systems have been designed to assist users manage and refind information, but very few have been evaluated. This has been noted by several scholars and explained by the difficulties involved in performing PIM evaluations. The difficulties include that people re-find information from within unique personal collections; researchers know little about the tasks that cause people to re-find information; and numerous privacy issues concerning personal information. In this paper we aim to facilitate PIM evaluations by addressing each of these difficulties. In the first part, we present a diary study of information re-finding tasks. The study examines the kind of tasks that require users to refind information and produces a taxonomy of refinding tasks for email messages and web pages. In the second part, we propose a task-based evaluation methodology based on our findings and examine the feasibility of the approach using two different methods of task creation.

References

[1]

R. Boardman, Improving tool support for personal information management, Ph.D. thesis, Imperial College London, 2004.

[2]

P. Borlund, The iir evaluation model: A framework for evaluation of interactive information retrieval systems, Information Research 8 (2003), no. 3, paper no. 152.

[3]

K. Byström and K. Järvelin, Task complexity affects information seeking and use, Information Processing and Management 31 (1995), no. 2, 191--213.

Digital Library

[4]

R. G. Capra and M. A. Perez-Quinones, Re-finding found things: An exploratory study of how users re-find information, Tech. report, Virginia Tech, 2003.

[5]

R. G. Capra and M. A. Perez-Quinones, Using web search engines to find and refind information, Computer 38 (2005), no. 10, 36--42.

Digital Library

[6]

R. G. Capra and M. A. Perez-Quinones, Factors and evaluation of refinding behaviors., SIGIR 2006 Workshop on Personal Information Management, August 10-11, 2006, Seattle, Washington, 2006.

[7]

E. Cutrell, D. Robbins, S. Dumais, and R. Sarin, Fast, flexible filtering with phlat, Proc. SIGCHI '06 (New York, NY, USA), ACM Press, 2006, pp. 261--270.

Digital Library

[8]

M. Czerwinski, E. Horvitz, and S. Wilhite, A diary study of task switching and interruptions, Proc. SIGCHI '04, 2004, pp. 175--182.

Digital Library

[9]

S. Dumais, E. Cutrell, J. Cadiz, G. Jancke, R. Sarin, and D. C. Robbins, Stuff i've seen: a system for personal information retrieval and re-use, Proc. SIGIR '03:, 2003, pp. 72--79.

Digital Library

[10]

D. Elsweiler and I. Ruthven, Memory and email re-finding, In preparation for ACM TOIS CFP special issue on Keeping, Re-finding, and Sharing Personal Information (2007).

[11]

D. Elsweiler, I. Ruthven, and C. Jones, Dealing with fragmented recollection of context in information management, Context-Based Information Retrieval (CIR-05) Workshop in CONTEXT-05, 2005.

[12]

D. Elsweiler, I. Ruthven, and C. Jones, Towards memory supporting personal information management tools, (to appear in) Journal of the American Society for Information Science and Technology (2007).

Digital Library

[13]

D. Harman, What we have learned, and not learned, from trec, Proc. ECIR 2000, 2000.

[14]

P. Ingwersen, Information retrieval interaction, Taylor Graham, 1992.

Digital Library

[15]

D. Kelly, B. Bederson, M. Czerwinski, J. Gemmell, W. Pratt, and M. Skeels (eds.), Pim workshop report: Measurement and design, 2005.

[16]

D. Kelly and J. Teevan, (to appear in) personal information management, ch. Understanding what works: Evaluating personal information management tools, Seattle: University of Washington Press., 2007.

[17]

B. H. Kwasnik, How a personal document's intended use or purpose affects its classification in an office, SIGIR'89 23 (1989), no. SI, 207--210.

Digital Library

[18]

M. W. Lansdale, The psychology of personal information management., Appl Ergon 19 (1988), no. 1, 55--66.

[19]

L. Palen and M. Salzman, Voice-mail diary studies for naturalistic data capture under mobile conditions, CSCW '02: Proceedings of the 2002 ACM conference on Computer supported cooperative work, 2002.

Digital Library

[20]

M. Ringel, E. Cutrell, S. Dumais, and E. Horvitz, Milestones in time: The value of landmarks in retrieving information from personal stores., Proc. INTERACT 2003, 2003.

[21]

G. Robertson, M. Czerwinski, K. Larson, D. C. Robbins, D. Thiel, and M. van Dantzich, Data mountain: using spatial memory for document management, Proc. UIST '98:, 1998.

Digital Library

[22]

K. Rodden, How do people organise their photographs, BCS IRSG 21st Annual Colloquium on Information Retrieval Research, Glasgow, Scotland, 1999.

Digital Library

[23]

D. C. Rubin and A. E. Wenzel, One hundred years of forgetting: A quantitative description of retention, Psychological Bulletin 103 (1996), 734--760.

[24]

A. J. Sellen and R. H. R. Harper, The myth of the paperless office, MIT Press, Cambridge, MA, USA, 2003.

Digital Library

[25]

P. Vakkari, Task complexity, problem structure and information actions: Integrating studies in on information seeking and retrieval., Information Processing and Management 35 (1999), 819--837.

Digital Library

[26]

P. Vakkari, A theory of task-based information retrieval, Journal of Documentation 57 (2001), no. 1, 44--60.

Cited By

Levine DTheron LHaffejee SUngar M(2024)Resilience to Depression Among Emerging Adults in South Africa: Insights From Digital DiariesEmerging Adulthood10.1177/2167696824127331912:5(694-709)Online publication date: 7-Aug-2024
https://doi.org/10.1177/21676968241273319
Zhou YDou ZWen J(2023)Enhancing Potential Re-Finding in Personalized Search With Hierarchical Memory NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.312606635:4(3846-3857)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TKDE.2021.3126066
Frummet AElsweiler DLudwig B(2022)“What Can I Cook with these Ingredients?” - Understanding Cooking-Related Information Needs in Conversational SearchACM Transactions on Information Systems10.1145/349833040:4(1-32)Online publication date: 11-Jan-2022
https://dl.acm.org/doi/10.1145/3498330
Show More Cited By

Index Terms

Towards task-based personal information management evaluations
1. Information systems
  1. Information retrieval

Recommendations

Factors in Personal Information Management Evaluation
Advances in Visual Informatics
Abstract
Personal Information Management (PIM) is the domain that involves methods and procedures to study how people keep, manage, maintain and re-find information. Related studies show that re-finding could be a difficult and time-consuming task due to ...
Exploring the design space for personal information management tools
CHI EA '01: CHI '01 Extended Abstracts on Human Factors in Computing Systems

The storing and retrieval of personal and work-related information from large capacity storage devices has become tedious and haphazard especially for individuals that are not technologically or computer literate. There must be a better a way to help ...
Exploring social dimensions of personal information management with adults with AD/HD
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing Systems

Studies in personal information management (PIM) have primarily examined PIM behavior as an individual activity. In this paper, we discuss social dimensions of PIM, more specifically, socially derived PIM activities. The biggest challenge adults with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

July 2007

946 pages

ISBN:9781595935977

DOI:10.1145/1277741

General Chairs:
Wessel Kraaij
TNO, The Netherlands
,
Arjen P. de Vries
CWI, The Netherlands
,
Program Chairs:
Charles L. A. Clarke
University of Waterloo, Canada
,
Norbert Fuhr
University of Duisburg-Essen, Germany
,
Noriko Kando
National Institute of Informatics, Japan

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR07

Sponsor:

SIGIR07: The 30th Annual International SIGIR Conference

July 23 - 27, 2007

Amsterdam, The Netherlands

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
1,910
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Levine DTheron LHaffejee SUngar M(2024)Resilience to Depression Among Emerging Adults in South Africa: Insights From Digital DiariesEmerging Adulthood10.1177/2167696824127331912:5(694-709)Online publication date: 7-Aug-2024
https://doi.org/10.1177/21676968241273319
Zhou YDou ZWen J(2023)Enhancing Potential Re-Finding in Personalized Search With Hierarchical Memory NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.312606635:4(3846-3857)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TKDE.2021.3126066
Frummet AElsweiler DLudwig B(2022)“What Can I Cook with these Ingredients?” - Understanding Cooking-Related Information Needs in Conversational SearchACM Transactions on Information Systems10.1145/349833040:4(1-32)Online publication date: 11-Jan-2022
https://dl.acm.org/doi/10.1145/3498330
Godoy MRusu CUgalde J(2022)Information Consumer eXperience: A Chilean Case StudySocial Computing and Social Media: Applications in Education and Commerce10.1007/978-3-031-05064-0_19(248-267)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-05064-0_19
Zhou YDou ZWen JCaverlee JHu XLalmas MWang W(2020)Enhancing Re-finding Behavior with External Memories for Personalized SearchProceedings of the 13th International Conference on Web Search and Data Mining10.1145/3336191.3371794(789-797)Online publication date: 20-Jan-2020
https://dl.acm.org/doi/10.1145/3336191.3371794
Ashkan AMetzler DPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Revisiting Online Personal Search Metrics with the User in MindProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331266(625-634)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331266
Zürn XBroekhuijsen Mvan Gennip DBakker SZijlema Avan den Hoven EAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)Stimulating Photo Curation on SmartphonesProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298947(255-259)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298947
Al Nasar MMohd MAli N(2019)Factors in Personal Information Management EvaluationAdvances in Visual Informatics10.1007/978-3-030-34032-2_46(523-536)Online publication date: 2019
https://doi.org/10.1007/978-3-030-34032-2_46
Vianna DKalokyri VBorgida AMarian ANguyen T(2019)Searching heterogeneous personal digital tracesProceedings of the Association for Information Science and Technology10.1002/pra2.2256:1(276-285)Online publication date: 18-Oct-2019
https://doi.org/10.1002/pra2.22
Tamine LDaoud M(2018)Evaluation in Contextual Information RetrievalACM Computing Surveys10.1145/320494051:4(1-36)Online publication date: 25-Jul-2018
https://dl.acm.org/doi/10.1145/3204940
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents