Today's post is from Silvia Vong and Angela Henshilwood, librarians at the University of Toronto. Silvia and Angela participated in the creation of the Threshold Achievement Test for Information Literacy. They were recently invited to talk about that experience at their university. Here they summarize the discussion.
By Silvia Vong and Angela Henshilwood
University of Toronto
On January 11, 2019, librarians at the University of Toronto (U of T) gathered for an instruction-focused professional development day. The theme of this year’s PD day was “Instructional Design for Librarians.” The day included refreshers on backward design, anticipatory sets, best practices for developing learning outcomes, and more. A good part of the second half of the day was dedicated to assessing our teaching and how to design the best assessment based on what we want to know about our students’ learning. The organizers were familiar with Project SAILS and other standardized tests but wanted to learn more about the new Threshold Achievement Test for Information Literacy (TATIL). So they invited us to give a lightning talk about TATIL because of our experience as test item writers.
Our aim for the talk was to give participants background information on TATIL, how it has been implemented at other libraries, and the benefits and challenges to consider with online testing. We drew background information from the TATIL website as well as Carolyn Radcliff’s presentation slides (used with her permission – see Clarke and Radcliff, 2018). We decided to touch on the challenges with online assessment since our past conversations with some librarians revolved around how students engage with online content in assessment as well as what features should be considered when identifying a well-designed online assessment tool. We found the best way to discuss the benefits and challenges was to frame the discussion around themes. Morgan and O’Reilly (2005) had outlined ten key qualities for online assessment that seemed to match areas of concerns we had heard from our colleagues. They write that the ten key qualities of good online assessment are:
- A clear rationale and consistent pedagogical approach
- Explicit values, aims, criteria and standards
- Relevant authentic and holistic tasks
- Awareness of students’ learning contexts and perceptions
- Sufficient and timely formative feedback
- A facilitative degree of structure
- Appropriate volume of assessment
- Valid and reliable
- Certifiable as students’ own work
- Subject to continuous improvement via evaluation and quality enhancement
(Morgan & O’Reilly, 2005, pp. 86-87)
We were able to address many of these points as librarians who participated in the development of test items, including some work related to the cognitive interview stages. In addition, it was easy to explain each point since many of these points were considered in the design of TATIL and information about rationale, aims, criteria, and steps in the development are transparent to librarians (see TATIL – Background). The point that piqued some interest was the continual evaluation and improvement of the test. As we were explaining our role with TATIL, we received questions about how and why we were involved. Our participation stemmed from our interest and experience with e-learning as we both had some involvement in developing online assessment tools in past roles and institutions. The fact that librarians were involved in the development of the tool gives us great confidence that test items are practical and reflective of the current IL landscape. As we developed our test questions we drew from our own experiences teaching and assessing IL in the classroom.
We appreciated the opportunity to talk about our involvement with TATIL and we’re confident that participants took away a better understanding of this particular assessment tool.
Clarke, K. & Radcliff, C. (December 7, 2018). Information Literacy Assessment for Instruction Improvement and Demonstration of Library Value: Comparing Locally-Grown and Commercially-Created Tests. Paper presented at Library Assessment Conference 2018, Houston, Texas. Presentation slides. Blog post summary.
Morgan, C., & O’Reilly, M. (2005). Ten Key Qualities of Assessment Online. In S. Howell, & M. Hricko (Eds.), Online Assessment and Measurement: Foundations and Challenges (pp. 86-101). Hershey, PA: IGI Global. doi:10.4018/978-1-59140-720-1.ch004