Utilizing Scenario-based Design for Statistical Information Services
By
Syracuse University

1. known to be "do-able" by study participants,
2. represent real tasks (or types of tasks) of interest to statistical agencies
3. fleshed out with details on approaches to and answers for the scenarios.

In short, they represent a library of vetted scenarios that other projects may find helpful. The scenarios are available in the appendices of Developing and Evaluating Scenarios for Use in Designing the National Statistical Knowledge Network, available at http://ils.unc.edu/govstat/papers/scenario_paper_nov_14_2002.doc.

Our project grew out of the increasing attention that has been directed at the role statistical information plays in the lives of people in all walks of life. The statistical agencies of the United States (along with other statistics-producing organizations) have responded to this attention by providing increasing access to their data often in conjunction with tools to support their use. This has lead to accelerated design efforts to provide tools, systems, etc. to enact the "Statistical Knowledge Network". The NSF in its Digital Government Initiative (and through other programs) has provided funding to a number of projects associated with statistical information. Our team is one of those and has chosen a scenario-based approach to design in our work. This approach focuses our work and has provided a vehicle for technology transfer between the team and our partnering agencies. Additionally, our scenarios can provide a launch pad for usability testing for the statistical community as a whole as it moves to integrate information and services in a knowledge network.

Scenario-based design employs user interaction scenarios, or narratives, as the source of guidance for design requirements. These narratives describe how an archetypal person (with a set of goals, behaviors, and knowledge) would carry out a series of interactions with a system. The articulation of the scenario enables designers to understand the features of the situation (e.g. needs analysis), determine appropriate system action (e.g. design requirements analysis), and document those (Rosson and Carroll, 2001). The approach "exploits the complexity and dynamics of the design domain" (Carroll, 2000, p. 45) enabling designers to better understand real tasks and the constraints upon them.

Our approach to scenario development utilized participatory design to ensure stakeholder buy-in and to enable rapid development of scenarios that met certain criteria. Prior to scenario development, we established a set of criteria that the final scenarios would need to meet. These criteria reflected the general guidelines for scenarios (i.e., ability to illuminate key issues) as we and our partner agencies initially understood the key issues for our project. In addition, they needed to reflect the partnership of many researchers and many agencies (both in terms of topical coverage and design challenges illustrated). The specific criteria were that the scenarios:

• Represent real information needs of the general population that cross agency; boundaries and/or levels of government;
• Be compelling to agencies and study team;
• Represent real types of information needs that might be expressed;
• Demonstrate some integration "issues" such as terminological differences (within and across agencies), variable (and classes within) differences, variety of information sources needed to address the need, mismatches between user perceptions of data and agency perceptions, unclear, unavailable metadata, etc.;
• Lead to design insight relevant to study team efforts. Agency partners were asked for possible scenario ideas as well as for information about key integration challenges they faced within their agency.

Ideas were solicited via an email listserv as well as from onsite and phone interviews with agency personnel.

A first set of twenty scenarios was "floated" by the study team to the agency partners and specific feedback on the scenarios gathered. Comments from all participating agencies were gathered and summarized and used to generate a second set of scenarios. The task scenarios (reduced to fifteen) were then searched (on the web) by team personnel to begin to understand the specific integration challenges represented by a given scenario, the information that was available to address the scenario, provide further information on how the scenario might be better formulated, and gain insights on how existing sites were addressing integration challenges similar to ours and how they were presenting data.

These fifteen scenarios have now been used extensively in our project in a variety of ways:

• Usability tests of existing and developing tools,
• User studies designed to identify information to be incorporated in a metadata schema,
• Exemplars used in demonstrations of tools developed by the project.
  
  This work has provided rich detail about the scenarios that might be utilized by other projects. The scenarios are available in the appendices of Developing and Evaluating Scenarios for Use in Designing the National Statistical Knowledge Network, available at http://ils.unc.edu/govstat/papers/scenario_paper_nov_14_2002.doc
  
  Research papers detailing studies that have used the scenarios are:
  
  Ceaparu, I. (2003). Finding Governmental Statistical Data on the Web: A Case Study of FedStats. IT & Society 1(3), 1-17. Available online at http://www.stanford.edu/group/siqss/itandsociety/v01i03/v01i03a01.pdf
  
  Denn, S., Haas, S.W., and Hert, C.A. (2003). Statistical metadata needs during integration tasks. In: DC-2003 (September 28-October 2, 2003, Seattle, WA.) Accessed online at: http://www.siderean.com/dc2003/301_Paper50.pdf.
  
  References
  Carroll, John M. (2002). Making Use: Scenario-based Design of Human-Computer Interactions. Cambridge, MA: MIT Press. Rosson, Mary Beth and Carroll, John M. (2001). Usability Engineering: Scenario-based Development of Human Computer Interaction. Morgan-Kaufman.