February has been a pivotal month for the Threshold Achievement Test, as we've transitioned our focus from defining Information Literacy concepts and dispositions to bringing those concepts to life as test items. The item writing taskforce spent February drafting our first ideas for test questions and we took steps toward identifying a workflow for the process going forward. Thanks to the dedication of volunteers from the advisory board we’re getting the chance to try out many types of test questions and many ways to gather evidence of students’ IL knowledge, experiences, and values. The item development process will continue throughout March.
Also in March, Rick Wiggins and Carolyn Radcliff, along with other representatives from Carrick Enterprises, will be sharing news and answering questions about Project SAILS and the new test at the ACRL 2015 Conference in Portland, OR. Rick will be available to provide additional updates on the next steps in our test development, including preliminary plans for identifying colleges and universities that are interested in beta testing modules of the new test as early as fall 2015. If you think you might be interested, stop by the exhibitor booth to learn more or contact Rick. We hope we’ll see you in Portland!
We’re pleased to hear that on February 1, the ACRL executive board voted to file the Framework for Information Literacy for Higher Education. This decision confirms the value of our efforts to envision a new approach to assessing students’ information literacy. The richness of the Framework and its alignment with librarians’ ongoing efforts to engage information literacy’s deep questions and troublesome knowledge, serve as inspiration to our advisory board as we advance to the next stages of test development. At the same time, we are glad that a final decision about the future of the Standards will be made at a later date, once librarians in the field have had ample opportunity to apply the new Frames and fully explore their strengths and limitations. We see our test development efforts as one facet in this wide-ranging project of investigating the affordances of the Framework.
In January, the advisory board studied the final version of the Framework. We found that the performance indicators and outcomes that we drafted in the fall closely matched many of the concepts described in the Framework. Reviewing the frames again inspired us to organize our extensive lists of indicators into core components of our key IL concepts. After further refining our lists of information literacy practices and dispositions, we defined four modules for development.
A taskforce of advisory board members formed to write test items that will take advantage of HTML 5 capabilities to experiment with a variety of innovative item types. If you’re looking for more information on innovative item types, you’ll find that Christine Harmes, Cynthia Parshall, and Kathleen Scalise have developed helpful taxonomies. The item writing taskforce is now engaged in the iterative process of writing items and performance indicators. Based on what we learn from the process of writing our first module and doing preliminary usability studies with our early item prototypes, we will begin writing items for the three remaining modules in the spring.
We're really excited about our latest progress and we're looking forward to updating you again soon!
Happy New Year!
In December, despite the hectic pace of the end of the semester, the Advisory Board reviewed lists of knowledge practices and dispositions that we created throughout the fall. This review was the next step in our process of defining the elements of information literacy that this new test will assess.
At our meeting on December 19, the Board discussed how the draft Framework for Information Literacy for Higher Education can guide our decisions about the behaviors, knowledge, and values we’ll ask students to demonstrate on the test. Advisory Board members are now working in small groups to further refine our definitions and prepare the guidelines we need to have in place in order to start writing test items in February.
This month we also made a major step forward in the process of selecting a name for the new test. Watch this space for exciting announcements!
At ALA MidWinter in Chicago we expect to hear that the Framework for IL in Higher Education has been finalized and our Advisory Board members in attendance at the conference will be meeting face to face (some of us for the first time).
We’re looking forward to having a fantastic 2015! We hope you are, too.
In November we made significant headway in our test development. One essential component of designing a test is defining the concepts/skills we’ll assess and identifying the levels of performance we’ll expect to see students demonstrate. Since this will be an assessment of undergraduates’ IL, we’re drafting plans for a test that will differentiate among students at a beginning level (at or near entry), an intermediate level (at or near completion of lower-division coursework), and an emerging expert level (at or near completion of a bachelor’s degree).
The Advisory Board generated lists of the IL skills, concepts, and dispositions that we have observed among college students throughout their education. We organized those observations so that they fit into the 6 frames of the latest draft of the ACRL Framework for IL in Higher Education. This provided us our first model of how students’ IL changes as they encounter the threshold concepts at the heart of the 6 frames. We will continue to refine our model and use it to guide our development of test items beginning in early 2015.
We have also added details to our plans for using innovative item types that take advantage of the flexibility that’s possible in computer based testing. Innovative items can include images and offer alternative response modes so that we can increase the fidelity that the questions have to students’ research experiences and gauge their knowledge practices and dispositions in ways that have not been possible with traditional multiple-choice questions. For example, innovative items might use an image of typical search results and ask students to select a set of appropriate sources given a specific information need. Determining our full range of item types is the next phase before we begin writing items.
Finally, in November, we began discussions with an expert in educational adaptive technology. We are using resources like Web Accessibility in Mind to incorporate elements of universal design at this early stage of planning for test item types and response systems.
We’re looking forward to a busy winter as we conclude our planning process and begin test development. I’ll keep you up-to-date with another post soon.
Hi, everyone. I'm April Cunningham, the project lead, and I'm going to be posting here regularly to update you on our progress.
October’s been a big month for our plans to develop a new IL test based on the draft ACRL IL framework. We’re moving full-speed ahead because we believe the framework will be adopted by ACRL at the ALA Mid Winter Conference in Chicago, January 30-February 3, 2015.
This month we created an Advisory Board of librarians and educators deeply committed to IL. We’ll be meeting with them and engaging them in online discussions so their collective expertise will guide the plans for the test.
So far we’ve discussed what types of results the test should generate so that it’s meaningful as a tool for planning and learning.
Next we’ll work on defining which information literacy practices students should be able to demonstrate at each stage in their development as undergraduates. We have our work cut out for us, but it’s a very exciting conversation!
Please welcome our Advisory Board (updated 12/7/14):
||Information Literacy and Assessment Librarian at Misericordia University in Dallas, PA
||Political Science professor at Ohio State University in Mansfield, OH
||Assistant Director of the Champlain College Library in Burlington, VT
||Head of Teaching and Learning for Wells Library at Indiana University in Bloomington, IN
||Librarian; student in the Education Evaluation & Research PhD Program at Wayne State University in Detroit, MI
||Dean of the Library at Cal State San Marcos in San Marcos, CA
||Director of Hoover Library at McDaniel College in Wesminster, MD
||English Composition professor at Palomar College and Saddleback College in southern California
||Head of Academic Programs and Course Support for Cline Library at Northern Arizona University in Flagstaff, AZ
||Administrator of Support Services at Encinitas Unified School District in Encinitas, CA
||Chemistry professor at Pomona College in Pomona, CA
||Director of Library Services at College of Marin in Kentfield, CA
||Instruction Librarian at East Los Angeles College in Monterey Park, CA
||Professor in the Educational Leadership Department, Coordinator of the Community College, Higher Education Specialization for the Educational Doctorate, and Director of the Center for Research on Educational Access and Leadership (C-REAL) at Cal State Fullerton in Fullerton, CA
||Teacher Librarian at Poway High School in Poway, CA
||Librarian and Instructional Designer in Los Angeles, CA
||Instructional Design Coordinator for the UC San Diego Library in La Jolla, CA
||Senior Research Fellow at Northwest Evaluation Association in Portland, OR