We're gearing up for big things this summer. By the end of July we expect to have performance standards set for the first two modules. We'll use the results of the field tests to establish criteria (i.e. cut scores) for how well we expect students to do on the test when they are entering college, when they've completed the bulk of their GE, and when they're ready to graduate with a bachelor's degree. This is a major step forward for making the test ready to be used for course-level, program-level, and institutional assessment.
Over the past few months, we've also been thinking more about the role of dispositions in students' IL outcomes. We know from the research on learning mindsets by Andrea Dweck and her colleagues that it’s vitally important for educators to instill in students the belief that they can develop their aptitudes through consistent effort. Students who believe that their intelligence or skills are already fixed and cannot improve over time are more likely to struggle in their courses and may not persist to achieve their academic goals.
The California Academic & Research Libraries, the state chapter of ACRL, held its biennial conference in Costa Mesa March 31-April 2. I was there to present a poster describing our approach to analyzing the Framework. I outlined our process for developing student outcomes and performance indicators. And I explained the rhetorical analysis that resulted in our four Dispositions: Toleration for Ambiguity, Feeling Responsible to the Community, Productive Persistence, and Mindful Self-Reflection. (You can read more about Richard Hannon’s work with the Dispositions here.)
Earlier that day, per-conference workshop participants had met with Allison Carr and Talitha Matlin from Cal State University, San Marcos, to grapple with the Framework and apply constructivist teaching approaches to develop new lesson plans and activities. Discussions like these are helping librarians to bridge the divide between their practices and the Framework’s aspirations. It was a wonderful opportunity for librarians to spend some focused time making the Framework practical.
Throughout the conference, what I heard from my colleagues working at colleges and universities from across the state is that the Framework remains an inspiring but daunting document. We discussed its value as a renewed vision for IL and encouraged one another to keep up the challenging but rewarding work of adapting the Framework to our needs. I heard this message from my community college colleagues who were trying to determine how much of the Framework to bite off in any given research session and in their program, overall. And I heard it from my research university colleague who is trying to incorporate the Framework’s knowledge practices as well as its dispositions into their campus-wide discussions about assessing institutional student learning outcomes.
The efforts of our advisory board and consultants to distill the Framework in order to create TATIL offer one piece of the foundation on which we will continue to build the future of information literacy.
I have been serving as the Rhetoric Consultant on the TATIL test since 2014, where one of my main responsibilities was to analyze the Framework in order to prepare the board to write outcomes and performance indicators. In this post, I will be giving a brief overview as to how I went about that process of understanding the Dispositions.
In order to be able to test for the Dispositions, or what the Framework calls the “affective, attitudinal, or valuing dimension of learning,” we needed to determine what kinds of latent traits sat beneath the surface-level descriptions of these Dispositions within the Framework. Studying the Framework, I was initially confused about the distinction between Dispositions and Knowledge Practices because many dispositions appeared to be bound-up with an understanding of core information literacy concepts. For example, in Information Has Value, the Framework says that competent students will “value the skills, time, and effort needed to produce knowledge.” The problem is that in order to value the time and effort required to create knowledge, students must first have learned what is involved in creating these kinds of texts; so, unless students are explicitly taught the multiple stages of forming research questions, collecting data, analyzing, and then synthesizing the information into a cohesive text, how can they have developed an attitude that values the rigor of this process? Thus, an important question emerged: How do we isolate the affective trait from the knowledge implied in the Dispositions?
There's still time to participate in field testing one or more modules this semester. This is great opportunity to contribute to the effectiveness and rigor of the test. If you're interested, please contact me (firstname.lastname@example.org) or Rick Wiggins (email@example.com) to get started.
We continue to make strides in developing the test. We've just completed cognitive interviews and usability testing for the third test module and we are writing items for the final module, 4: The Value of Information. Thanks to our talented team of test question writers, we are making exciting progress.
I had a chance this month to check in with Carrie Donovan to find out what she's thinking about the Framework now that it's been a little more than one year since ACRL filed the document. Carrie is Assistant Dean for Research & Instruction Services at Ferris State University’s Ferris Library. She is a curriculum designer and facilitator for ACRL's Assessment in Action. She also serves as the ACRL Instruction Section Member-at-Large, and as the ACRL Liaisons Training and Development Committee Vice-Chair. Carrie has been a member of the TATIL Advisory Board since 2014.
I asked her the following questions and she shared her insights.
We are now ready to start cognitive interviews to get students' feedback about Module 3: Research & Scholarship. We are also starting to write items for our final module, Module 4: The Value of Information. That means we're more than half way through with test development. And we just keep getting more intrigued with the depth of the Framework the more we work with it.
One of the exciting things about the Framework is the way the writers identified the “dispositions” that constitute the affective facets of information literacy. From the beginning of brainstorming about a new IL test way back in spring 2014, we’ve known that we wanted to address dispositions, as well as knowledge, in any new instrument we created. We found a way to do that with scenario-based problem solving items. And we’ve continued to deepen our understanding of dispositions by searching the education literature.