Archive for April 2011
Jeffrey Cordell is Instructional Pedagogy Librarian in the Undergraduate Library. As part of the Assessment Working Group, he has helped to roll out the new evaluation forms being used to assess teaching across the library.
Using Assessment to Build Instruction Strategy
Over the past two years, the library has designed a set of evaluation forms for instruction sessions; over the course of fall semester, we generated reports on those evaluations for each instructor. At a workshop in late February, individual reports were given out and discussed. I want to give a sense of what we talked about in that meeting and to share some thoughts about evaluations and their uses for teaching.
If you weren’t able to attend the workshop and would like a copy of your report, contact Jen Green at email@example.com and ask for one. As you look over your report, it’s important to remember that while it can tell you much about your teaching, it is not a core sample of who you are as a teacher. Anytime we represent the experience of teaching and learning numerically, we engage in a kind of fiction or wishful thinking that says that we can capture experience quantitatively—that what happens in the classroom can be anything like adequately expressed through numbers. It would be nice to think so, because it would mean that the experience of one classroom could be replicated exactly in another, as if all instructors with a 4.5 were doing exactly the same thing. Rather, the kinds of numbers we see on course evaluation forms are rough approximations of the experience of being a student and inevitably do little to convey the rich complexity of what happens in the classroom, where each instructor works from his or her distinct, incommensurable qualities as a teacher.
Because they are such crude tools, course evaluations can sometimes seem to measure only the entertainment value of a class. However, I do think they measure something useful, and that something is how students feel about their experience in the classroom. At first glance, “feelings” may seem too subjective, even irrelevant to the process of finding information, to bother measuring. After all, learning is not about feelings; it’s about acquiring information. But learning is always bound up with one’s emotions, and, as Plato told us long ago, learning at its best matches the thrill of falling in love, and is, indeed, indistinguishable from it. When we truly grasp an idea, we feel it lodge in ourselves, and that always carries with it change, change in our perceptions and, therefore, in who we are. More pragmatically, there is a sense in which learning becomes more readily available to our consciousness when we’re aware that we are learning. So, the use of course evaluations comes in part from the measure, however tentative and approximate, of students’ perception of their learning in a class. While course evaluations cannot do the work of systematic assessment of skills (for that, we need tests), they can suggest to us, however crudely, how students perceive their experience in the classroom and whether they find that experience valuable.
The reports that the library has generated also help us, as instructors, to get a fuller sense of how our teaching fits into the larger mission of the library. The reports present your numbers against the mean for all the responses from a given evaluation form (that is, when you look at your report, you can see whether your numbers are higher, lower, or similar to the library mean for that report, be it “intro,” “advanced,” etc). That, in turn, can give you a sense of what, in your classes, students seem to appreciate and where you might want to focus your attention as you revise your teaching techniques and in-class exercises. Often, it’s not even so much a matter of changing how you teach as it is explicitly underscoring for students what it is they’re learning. An interactive in-class exercise may not be perceived as interactive by your students until you say that it is. Additionally, the evaluation forms give a sense of what we, as instructors at the library, value and hope to achieve. On that level, these reports are part of an ongoing conversation that may take in questions such as: what do we consider successful numbers at the library? How do we want to use them (for example, they could be used as part of teaching portfolios)? What kinds of institutional support would we like to have available to instructors who find things they want to change in their teaching based on these reports? And so forth.
I have had many years experience in reading and using course evaluations, but, being new to the world of the library, am curious to hear your opinions of the new evaluations and reports, and to hear your ideas about how you are going to use the information they give us.
On Thursday, March 17th, 2011, the Instructor College sponsored an informal panel discussion with three librarians — Gabriel Duque, Rebecca Hill, and Karen Reiman-Sendi — and Chad Hershock, an assistant director from the UM Center for Research on Learning and Teaching CRLT about the use of iClickers in library instruction. Many of Chad’s recommendations were included in CRLT Occasional Paper No. 22: Teaching with Clickers, a white paper that includes information on student/faculty attitudes toward clickers, best practices and challenges. The information below summarizes the workshop conversation about question design, challenges that librarians have faced and advice librarians can offer.
- Include “I don’t know” as a question option. You don’t want the students to guess the answer. You want to know whether they know the answer.
- You could ask, “Have you had a library session before?” to get a sense of where to start the session. You could also ask if they had used Mirlyn or what databases they had used. You could also ask questions that are more task-based (“Have you ever checked out a book?”) instead of tool-based (“Have you ever used Mirlyn?”) to get a sense of what they have and haven’t done with library resources.
- After showing students a new procedure or process, you might ask them if it would be helpful to show another example to give you a sense of whether it would be okay to move on with the lesson.
- Asking international students where they are from with the clickers is a good ice breaker.
- It is easy to revise a question in “real time.”
- Ask students for their favorite movie (give them some choices) and then show them how to look it up in Mirlyn.
- Students don’t need a toy in class. Use the clickers and design the questions with a purpose.
- Decide ahead of time the percentage of students you need to get the question correct. Will it be okay with you if some students get the question incorrect and you still have to move on? How will you manage these situations?
- Using the clickers and prepping by creating slides can add time to your instructional load. Consider if there is something that you can use repeatedly. Manage in-class discussion of clicker answers appropriately. If you are co-teaching, have one instructor manage the mechanics of the clicker software during the class while another facilitates the discussion. Sometimes a student can help.
- Sometimes the batteries run out of power. There should be extra batteries in the boxes. If not or if you use the last ones, please notify Donna McCauley after class. Many times there are more clickers than students so you can easily exchange one clicker for another.
- Use the instructor remote to advance slides in a PowerPoint presentation. Using the keyboard in conjunction with the remote can sometimes cause the PowerPoint to freeze.
- Try it once and see what happens.
- Set up class session ahead of time.
- Make sure the PIN number is in the Instructor remote.
- Remember that students liked to play with clickers so stop the voting before you show the graph if you don’t want students to change their answers.
- A bar graph is very easy for the students to read.
- Most undergraduates are familiar with clickers so you don’t have to explain how to use them.
- Think about your teaching style to determine how this tool can help you. Do you need to break up lecturing? Do you often call on students in class? How do they respond now when you ask a question?
- Grad students seem eager to talk in class. Undergrads may be less enthusiastic so this tool may help with participation.
- It may be difficult to use clickers with hands-on technology classes because students may have to move from one physical piece of technology to another which may feel awkward.
- There are other online polling tools out there that are free and web- or mobile-based. You may want to try them.
- Clickers are not a magic tool. The focus of the class will still be on you.
- Share the data you collect with the faculty member or GSI.
On April 11, Karen Reiman-Sendi led an Instructor College-sponsored brown bag discussion of using web-based guides in instruction, touching on the research literature surrounding the use of guides and our local data about patron use of guides. The slide presentation from the event is available. Attendees collaboratively developed the following best practice statements:
Best Practices for Course-Related and Workshop-Related Guides (April 2011)
- Use a course-related/workshop-related guide:
- to solve a specific problem/to accomplish a task or assignment
- to provide easy accessibility to needed information
- as an opportunity to work with a faculty member
- Think about logical organization of material to meet needs of your audience and the identified educational outcomes
- Avoid too much text – extraneous information is awful – and keep a balance between content and white space
- Include some visual interest appropriate to the purpose of the guide (but don’t rely on color to do this)
- Use screencasts or screenshots appropriate to the task at hand, to illustrate points, strategies, concepts, etc. (See MLibrary Instructional Videos guide)
- Avoid using too many tabs (“pages”) but do use tabs to help define “modules” or sections of the guide
- Help students choose the appropriate resources, information, strategies, etc. by using smaller box content, by using headings within boxes, by providing some “navigation” within and between your tabs (“pages”)
- Provide not just a list of appropriate resources for a specific assignment, but include strategies for understanding the assignment/for completing the assignment, based on your educational goals/outcomes for the guide
- Keep in mind that guides will be viewed on mobile devices and by individuals with visual challenges (accessibility)
- Take care in using drop-down menus in tabs (“pages”) because these areas may be difficult to see or to navigate
- Wherever possible collaborate with colleagues on guide creation to eliminate unnecessary duplication
- Provide uploaded files of your instructional slides and handouts, as appropriate
- Keep the guide up to date and/or take it “offline” when no longer needed
- When providing a list of resources, include short annotations to help students choose
Provide links to related guides where appropriate
- Provide a link to the course/workshop/session evaluation
- Include your profile box on the first tab (“page”) of guide as well as the Ask a Librarian contact box
- Background of guide must be white
- Title of course guide should be the course catalog label, e.g. AMCULT 209: History of American Popular Music
- Guide description should include the purpose of the guide, and ideally the name of the instructor and academic term, e.g. Key information resources and services for completing the honor’s thesis. Prof. John Doe. Winter 2011.
- Instructor profile must appear on “home” tab/page in right column. The profile box can be labeled “Library Contact” or “Library Instructor” or “Workshop Instructor” etc.
- “Library Help” box appears below the instructor profile box and contact information
- Tabs/pages will vary from course to course but might include “Introduction,” “Your Assignment,” “Finding Articles,” “Finding Data,” “How to [do something],” “Citation Styles,” etc.
- When published, include the “course_guide” or “technology_guide” tag
- Set guide to “private” at the end of the current term if the content will not be taught in the following term