021:264 Program Evaluation - U/Iowa SLIS July 2001

Suggested Text: Wallace, Danny P. and Connie Van Fleet. 2001. Library Evaluation: A Casebook and Can-Do Guide. Englewood, CO: Libraries Unlimited.

Required readings: in browsing box; 1 copy of Wallace and Van Fleet in University Reserve Room

In-Class Course Assignments

Prior to class each day, choose a reading that interests you from the ones listed for the upcoming afternoon; study it in detail. Class time will be spent discussing, critiquing, and learning from the readings–especially the case studies. For each piece, those who read it will present it to the rest of the class.

Day 1: Review of course requirements. Instructor’s introduction of program evaluation, along with class discussion of the following topics and the reading: what is a “program”, what is meant by “evaluation”, what are benefits and limitations to evaluation, what happens after an evaluation is completed.

Read the following for class discussion:

Wallace, Danny P. and Connie Van Fleet. 2001. Library Evaluation: A Casebook and Can-Do Guide. Englewood, CO: Libraries Unlimited.

Chapter 1-The Culture of Evaluation and Chapter 6-The Nature of Evaluation. NOTE: This book will be cited as Wallace/Van Fleet hereafter in the syllabus.

Day 2: User Groups Outside and Inside the Library/Information Agency. Evaluating to determine levels of success in meeting the needs of outside groups, for example the disabled, academic faculty, K-12 teachers and students, and children. Discussion will include evaluating some inside groups: staff and volunteers, and staff satisfaction with library processes (using e-resources and the OPAC as examples). Read 1 of the following articles:

Edwards, Kirstie et al. “Evaluation of User Interface Design to Optimize Access to Library Databases for People Who Are Motor Impaired,” Information Technology and Libraries 16/4 (December 1997): 175-181.

Everhart, Nancy. 1998. “[Chapter] 7: Usage,” in her Evaluating the School Library Media Center: Analysis Techniques and Research Practices. Englewood, CO: Libraries Unlimited.

McCune, Bonnie F. and Charleszine “Terry” Nelson. 1995. “[Chapter] 6: Evaluations and Records,” in their Recruiting and Managing Volunteers in Libraries. NY: Neal-Schuman.

Russell, Carrie. “Using Performance Measurement to Evaluate Teams and Organizational Effectiveness,” Library Administration & Management 12/3 (Summer 1998): 159-165.

Weingart, Sandra J. “When Questions Are Answers: Using a Survey to Achieve Faculty Awareness of the Library’s Electronic Resources,” College & Research Libraries 61/2 (March 2000): 127-134.

Young , Diana. “Evaluating Children’s Services,” Public Libraries (Spring 1984): 20-22.

Day 3: Collections–Physical and Virtual. Evaluating their use, quality, currency, and diversity, along with their relationship to user needs.

Everyone must read Wallace/Van Fleet pages 117-128, and 165-182, and then 1 of the following articles/cases:

Barkett, Gina R. 2001. “Case Study 4.3: Conducting a Citation Analysis,” in Wallace/Van Fleet, pages 153-164 (starting with “Method @ A Glance” and “Case @ A Glance”).

Everhart, Nancy. 1998. “[Chapter] 3: Collections,” in her Evaluating the School Library Media Center: Analysis Techniques and Research Practices. Englewood, CO: Libraries Unlimited.

Froehlich, Thomas J. 2001. “Case Study 5.1: Developing Search Engine Evaluation Criteria,” in Wallace/Van Fleet, pages 183-198 (starting with “Method @ A Glance” and “Case @ A Glance”).

Halliday, Blane. 2001. “Case Study 4.2: Identifying Library Policy Issues with List Checking,” in Wallace/Van Fleet, pages 138-152 (starting with “Method @ A Glance” and “Case @ A Glance”).

Kahn, Miriam J. 2001. “Case Study 4.1: Using Qualitative Criteria to Evaluate Reference Resources,” in Wallace/Van Fleet, pages 129-137 (starting with “Method @ A Glance” and “Case @ A Glance”).

Rhine, Leonard. “The Development of a Journal Evaluation Database Using Microsoft Access,” Serials Review (Winter 1996): 27-34.

Day 4: Public Services. Evaluating signage (as an example of physical arrangements), queuing at service points, influence of personnel behavior on user satisfaction, user-librarian interaction, and circulation patterns.

Everyone must read Wallace/Van Fleet pages 79-86 and then 1 of the following articles/cases:

Bosman, Ellen and Carol Rusinek. “Creating the User-Friendly Library by Evaluating Patron Perceptions of Signage,” Reference Services Review 25/1 (Spring 1997): 71-82.

Czopek, Vanessa. “Using Mystery Shoppers to Evaluate Customer Service in the Public Library,” Public Libraries 37/6 (November-December 1998): 370-371.

Davis, Hazel M. and Ellen Altman. “The Relationship between Community Lifestyles and Circulation Patterns in Public Libraries,” Public Libraries 36/1 (January-February 1997): 40-44.

Gatten, Jeffrey N. and Carolyn J. Radcliff. 2001. “Case Study 3.2: Assessing Reference Behaviors with Unobtrusive Testing,” in Wallace/Van Fleet, pages 103-115 (starting with “Method @ A Glance” and “Case @ A Glance”).

Radcliff, Carolyn J. and Barbara F. Schloman. 2001. “Case Study 3.1: Using the Wisconsin-Ohio Reference Evaluation Program,” in Wallace/Van Fleet, pages 87-102 (starting with “Method @ A Glance” and “Case @ A Glance”).

Tillotson, Joy, Janice Adlington, and Cynthia Holt. “Benchmarking Waiting Times,” College & Research Libraries News 59/10 (November 1997): 693-694, 700.

Day 5: The Library and Its External Community. Evaluating programs for accreditation, and for community needs and supports. “Communities” typically include groups and entities, such as teachers, the school, the municipality, the college or university, the company, the governing board of different kinds of libraries. Course wrap-up.

Read 1 of the following articles or group of articles:

Clougherty, Leo et al. “The University of Iowa Libraries’ Undergraduate User Needs Assessment,” College & Research Libraries 59/6 (November 1998): 572-584.

Coleman, Paul and Ada D. Jarred. “Regional Association Criteria and the Standards for College Libraries: The Informal Role of Quantitative Input Measures for Libraries in Accreditation,” Journal of Academic Librarianship 20-/5-6 (November 1994): 273-284.

Group of 3 articles: Hennen, Thomas J., Jr. “Great American Public Libraries: HAPLR Ratings, Round Two,” American Libraries 30/8 (September 1999): 64-68; Hennen, Thomas J., Jr. “Great American Public Libraries: HAPLR Ratings, 2000,” American Libraries 31/10 (November, 2000): 50-54; and Scheppke, Jim, “The Trouble With Hennen,” Library Journal 124/19 (November 15, 1999), 36-37.

Group of 3 articles and a sample survey--all by Guy St. Clair: “Ask the Customers,” “Ask the Customers (II),” “The Information Audit (III): You’ve Got the Data–What Does It Mean?”, and “Information Audit (Sample Survey)”, in The Information Audit: An SLA Information Kit. 1996, pp. 22-32 (also published in The One-Person Library v.11/nr.9 (January 1995), 11/10 (February 1995), 11/11 (March 1995), and 11/9 (January 1995), respectively).

Jones, A. James, Carrie Gardner, and Judith L. Zaenglein. “Desperately Seeking Standards: Bridging the Gap from Concept to Reality,” Knowledge Quest 26/3 (March-April 1998): 38-42.

Stanat, Ruth.1996. “The Strategic Information Audit,” in The Information Audit: An SLA Information Kit, pages 1-21 (also published in Stanat, Ruth. The Intelligent Corporation. AMACOM, 1990).

Outside of Class Course Assignments

1.You are required to read the material that will be discussed in each day’s class prior to that day’s class. You also should work on your Program Evaluation Plan (see following).

2. Course Assignment–A Program Evaluation Plan

By no later than the second day of class, you must select a program that you would like to evaluate (a real program is preferable). Over the course of the class, you should assess the relevance and value to your specific program of various evaluation methods, select the appropriate method(s), and write up an evaluation plan and process. Since learning over the duration of the July 16-20 class week may result in changes to your evaluation plan, the final plan can be turned in the week after, but it must be postmarked to the instructor by no later than Wednesday, July 25, 2001. Paper: length of 5-7 pages double spaced with 2 inch margins. Note that a Wallace/Van Fleet case study or their “Evaluation Action Plan” 9 steps (see their Chapter 1, pages 4-10) may serve as useful models for structuring your own plan.

Grading

Class Participation/Discussion = 70%

Program Evaluation Plan Paper = 30%

WARNING: This course will be conducted as a seminar, which is a collaborative and incremental knowledge-building process. This means each student must participate in order for the content of the course to be realized. If you are very uncomfortable engaging in oral learning, please do not enroll in the course this summer.