June was a big month for the Threshold Achievement Test since we held a productive item writing meeting on the 13th and made significant progress on streamlining our student outcomes. Here are a few photos from the day we spent writing test questions at Hill Street Café in Oceanside, California.
As spring semester gave way to summer sessions, we had our first in-person item writing meeting for the Threshold Achievement Test. Advisory board members in southern California met up at Hill Street Cafe in Oceanside to spend the day drafting test questions that will assess learners' ability to evaluate sources and identify the process used to create information. Writing items gave us the chance to further refine our performance indicators, as we recognized unclear wording and "double barrel" indicators that were trying to describe two behaviors at once.
In May, Michelle Dunaway began compiling a set of IL dispositions based on her research, including review of the Project Information Literacy reports. These efforts will facilitate development of the scales we're planning to use to assess students' values and self-efficacy related to evaluating sources and identifying the information creation process. We are excited to meet the challenge of creating a test that will assess students' knowledge, skills, and abilities as well as their dispositions. Research on measuring affective characteristics is helping us to clearly define our approach. If you're interested, you might check out Instrument Development in the Affective Domain by McCoach, Gable, and Madura.
In June, we're looking forward to additional item writing meetings. And Carrick Enterprises will be hosting a booth at ALA in San Francisco. Rick and Carolyn will be available to answer questions and share more about the test. And I'll be dropping by the booth, too. Don't miss your chance to take our personality quiz and find out which of the IL Frames you are! We hope we'll see you there!
Here’s just a quick update about what we did in April with the Threshold Achievement Test for Information Literacy, since we’re busy writing test items and enjoying the spring weather. 🙂
We’re happy to introduce Michelle Dunaway and Lettycia Terrones as the two newest members of our Advisory Board. Michelle is a librarian who is currently pursuing a PhD in Education Evaluation & Research at Wayne State University in Detroit. Lettycia is the Education Librarian at Cal State Fullerton and she was a high school English and ESL teacher for seven years before becoming a librarian.
The advisory board members have been drafting and refining performance indicators for the first test module we’re developing. Here’s a sampling of some we’re working on:
- Identify reasons for selecting sources that represent a variety of perspectives, values, and judgments.
- Distinguish when it is appropriate to use a primary scholarly article versus a review article.
- Arrange a sample set of sources into their appropriate positions on the information cycle.
- Match descriptions of audience values to descriptions of the authorial expertise they would find most authoritative.
And if you’re interested in learning more about the test development process and research-based item formats, we recommend the following books that have been influential to us so far:
- Downing, S. M., & Haladyna, T. M. (2006). Handbook of test development. Mahwah, NJ: L. Erlbaum.
- Haladyna, T. M. (1997). Writing test items to evaluate higher order thinking. Boston, MA: Allyn and Bacon.
- Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge.
We're looking forward to an exciting summer. Keep checking back for updates. 🙂
In March, at the ACRL Conference in Portland, Carrick Enterprises sponsored an exhibit booth. Rick, Carolyn, and their colleague Paula McMillen fielded questions about the Threshold Achievement Test. Many librarians from institutions as varied as chiropractic colleges, state universities, and community colleges from across the country expressed interest in taking part in the field test scheduled for this fall.
The field test will generate data that we’ll use to set criteria for scoring the test and will give us the chance to analyze individual items. It will give the participating institutions a snapshot of their students’ information literacy knowledge practices and dispositions.
But the discussions at the booth weren’t all business. Attendees also got to take a Facebook-style quiz to reveal which of the IL frames best matched their personality. If you missed your chance to take the quiz and find out if you know yourself, your friends, and the Frames as well as you thought you did, then plan to stop by the Project SAILS, Carrick Enterprises booth at ALA in San Francisco at the end of June. And as you check out everyone’s name badges, look for their Threshold Achievement Test stickers that proclaim their Frame.
February has been a pivotal month for the Threshold Achievement Test, as we've transitioned our focus from defining Information Literacy concepts and dispositions to bringing those concepts to life as test items. The item writing taskforce spent February drafting our first ideas for test questions and we took steps toward identifying a workflow for the process going forward. Thanks to the dedication of volunteers from the advisory board we’re getting the chance to try out many types of test questions and many ways to gather evidence of students’ IL knowledge, experiences, and values. The item development process will continue throughout March.
Also in March, Rick Wiggins and Carolyn Radcliff, along with other representatives from Carrick Enterprises, will be sharing news and answering questions about Project SAILS and the new test at the ACRL 2015 Conference in Portland, OR. Rick will be available to provide additional updates on the next steps in our test development, including preliminary plans for identifying colleges and universities that are interested in beta testing modules of the new test as early as fall 2015. If you think you might be interested, stop by the exhibitor booth to learn more or contact Rick. We hope we’ll see you in Portland!