ECS Research
Publications
Intranet Tools

nb. next round of REF2013 will NOT be using data from eprints.ecs, but the central university REF interface.

RSS 1.0 Feed
RSS 2.0 Feed
Atom Feed
 

Wicked Problems and Gnarly Results: Reflecting on Design and Evaluation Methods for Idiosyncratic Personal Information Management Tasks

Bernstein, M., Van Kleek, M., Karger, D. and schraefel, m. c. (2007) Wicked Problems and Gnarly Results: Reflecting on Design and Evaluation Methods for Idiosyncratic Personal Information Management Tasks. Technical Report 21-09-07-CHI08-ep14668, Electronics and Computer Science, Southampton. (Unpublished)

Download

[img]
Preview
 PDF
3245Kb

Abstract

This paper is a case study of an artifact design and evaluation process; it is a reflection on how right thinking about design methods may at times result in sub-optimal results. Our goal has been to assess our decision making process throughout the design and evaluation stages for a software prototype in order to consider where design methodology may need to be tuned to be more sensitive to the domain of practice, in this case software evaluation in personal information management. In particular, we reflect on design methods around (1) scale of prototype, (2) prototyping and design process, (3) study design, and (4) study population.

Item Type:Technical Report
Creator/Authors:
Michael Bernstein
Max Van Kleek
David Karger
m.c. schraefel
Keywords:evaluation, design, methods, methodology
Research Group:Old ECS Groups > Intelligence, Agents, Multimedia
Current ECS Groups > Agents, Interaction and Complexity
Date:2007
Information about this record:
Downloads (2010):93
ID Code:14668
Last Modified:23 Sep 2011 10:35
Deposited On:10 Oct 2007 by schraefel, monica

Tools & Metadata

Download Statistics

Last month

Last year

Members of ECS may view the download statistics dashboard for this record.

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in this archive you will be forwarded to the paracite service. Poorly formated references will probably not work.

3M Digital Post It Notes. http://www.3m.com/

2. Circus Ponies Notebook. http://www.circusponies.com/

3. Google notebook. http://www.google.com/notebook/

4. Microsoft OneNote. http://office.microsoft.com/en-us/onenote/default.aspx

5. Stikkit. http://stikkit.com/ ]

6. Barkhuus, L. and Rhode, J.A., From Mice to Men - 24 years of Evaluation in CHI. in Extended Proceedings of CHI 2007, (2007).

7. Bellotti, V., Dalal, B., Good, N., Flynn, P., Bobrow, D.G. and Ducheneaut, N., What a to-do: studies of task management towards the design of a personal task list manager. in Proc. CHI 2004, (Vienna, Austria, 2004), ACM Press, 735-742.

8. Bellotti, V., Ducheneaut, N., Howard, M. and Smith, I., Taking email to task: the design and evaluation of a task management centered email tool. in Proc. CHI 2003, (Ft. Lauderdale, Florida, USA, 2003), ACM Press, 345-352.

9. Bernstein, M., Van Kleek, M., Karger, D. and schraefel, m. Information Scraps: Eluding Our Personal Information Tools. In Submission to Transactions on Information Systems, http://eprints.ecs.soton.ac.uk/14231/.

10. Bernstein, M.S., Van Kleek, M., schraefel, m.c. and Karger, D.R., Management of personal information scraps. in Extended Abstracts CHI 2007, (New York, NY, USA), ACM Press, 2285-2290.

11. Blandford, A.E. and Green, T.R.G. Group and Individual Time Management Tools: What You Get is Not What You Need. Personal Ubiquitous Comput., 5 (4). 213-230.

12. Buchenau, M. and Suri, J.F. Experience prototyping. DIS 2000. 424-433.

13. Buxton, B. Sketching User Experiences. Morgan Kaufmann, 2007.

14. Campbell, C. and Maglio, P., Supporting notable information in office work. in Proc. CHI 2003, (New York, NY, USA), ACM Press, 902-903.

15. Czerwinski, M. and Horvitz, E. An Investigation of Memory for Daily Computing Events, 2002. citeseer.ist.psu.edu/czerwinski02investigation.html

16. Dumais, S., Cutrell, E., Cadiz, J.J., Jancke, G., Sarin, R. and Robbins, D.C. Stuff I've seen: a system for personal information retrieval and re-use Proc. SIGIR 2003, ACM Press, Toronto, Canada, 2003.

17. Hayes, G., Pierce, J.S. and Abowd, G.D., Practices for capturing short important thoughts. in Proc. CHI 2003, (New York, NY, USA), ACM Press, 904-905.

18. Jones, W. and Cronin, B. Personal Information Management. in Anonymous ed. Annual Review of Information Science and Technology (ARIST), Information Today, Medford, NJ, 2007, 453-504.

19. Kalnikaite, V. and Whittaker, S., Software or wetware?: discovering when and why people use digital prosthetic memory. in Proc. CHI 2007, (New York, NY, USA), ACM Press, 71-80.

20. Kelley, D. and Teevan, J. Understanding What Works: Evaluating PIM Tools. in Jones, W. and Teevan, J. eds. Personal Information Management, 2007.

21. Lin, M., Lutters, W.G. and Kim, T.S., Understanding the micronote lifecycle: improving mobile support for informal note taking. in Proc. CHI 2004, ACM Press, 687-694.

22. McGrath, J.E. Methodology matters: doing research in the behavioral and social sciences. Human-computer interaction: toward the year 2000. 152-169.

23. Plaisant, C., The challenge of information visualization evaluation. in Proc. AVI 04, (New York, NY, USA), ACM Press, 109-116.

24. Rettig, M. Prototyping for tiny fingers. Communications of the ACM, 37 (4). 21-27.

25. Rittel, H.W.J. and Webber, M.M. Dilemmas in a general theory of planning. Policy Sciences, 4 (2). 155-169.

26. Rodden, K. and Wood, K.R., How do people manage their digital photographs? in Proc. CHI 2003, (New York, NY, USA), ACM Press, 409-416.

27. schraefel, m.c., Zhu, Y., Modjeska, D., Wigdor, D. and Zhao, S., Hunter gatherer: interaction support for the creation and management of within-web-page collections. in Proc. WWW 02, (New York, NY, USA), ACM Press, 172-181.

28. Sellen, A., Fogg, A., Aitken, M., Hodges, S., Rother, C. and Wood, K., Do Life-Logging Technologies Support Memory for the Past? An Experimental Study Using SenseCam. in CHI, (2007).

29. Smith, G., Baudisch, P., Robertson, G., Czerwinski, M. and Meyers, B. GroupBar: The TaskBar Evolved OZCHI, 2003. citeseer.ist.psu.edu/article/smith03groupbar.html

30. Van Kleek, M., Bernstein, M.S., schraefel, m. and Karger, D.R., GUI--Phooey! The Case for Text Input. in To appear in Proc. UIST 2007.

31. Yee, K.-P., Swearingen, K., Li, K. and Hearst, M., Faceted metadata for image search and browsing. in Proc. CHI 2003, (New York, NY, USA), ACM Press, 401-408.

Corrections

ECS staff and postgraduates may modify this record

   Welcome from Deputy Head of School (Research) Research Prospectus Industrial Partnerships New Research Students Notes for Guidance New Research Students Notes for Guidance
The ECS EPrints Repository supports OAI 2.0 with a base URL of http://eprints.ecs.soton.ac.uk/cgi/oai2

EPrints is free software developed by the University of Southampton to facilitate Open Access to research.		 EPrints