[This is a guest post by Janine Utell, who is an Associate Professor of English at Widener University in Pennsylvania. She teaches composition and 19th and 20th century British literature; she has also facilitated a number of on- and off-campus workshops on writing, critical thinking, and general education. Recently at ProfHacker, she’s written on “Practical Wisdom and Professional Life.” You can follow Janine on Twitter: @janineutell --@jbj]
Figuring out whether something is working, and whether it could work better, is a subject near and dear to ProfHacker hearts (see here and here). What if you’ve worked out a pedagogical strategy that you feel pretty committed to, you sense a high level of engagement, but you’re not sure it’s as effective as it could be. You want to know if there are tweaks to be made, ways of doing it better.
I am pretty committed to a discussion-based pedagogy and how it allows for collaborative learning over the life of a course (for other ProfHacker posts on fostering collaborative learning, see here and here). I’ve developed strategies to create good discussion, to facilitate broad and deep involvement, and to synthesize the contributions of the classroom community. I feel like my classes are going best when the room is a bit rowdy, when interactions lead to debates and eurekas. But due partly to assessment work on my campus and partly to collaboration with colleagues in a different discipline around designing a study of student writing, I decided to create a project of my own to investigate the effectiveness of my practice. I wanted a more robust picture of what’s going on in my classroom and whether it’s working.
I’m sure a number of ProfHacker readers have experience with this, and I’d call upon more seasoned folks to participate in the comments section with suggestions and advice. But what if you want to design and implement a small qualitative study of your teaching, perhaps for presentation or publication but mainly just to see if it works, when you have limited, time, resources, and not very much experience?
If you’ve ever needed more than one person to explain the difference between quantitative and qualitative, this is for you. If there’s enough reader interest and response, if we get some good suggestions and advice, or if readers get an idea from this post that they’d like to do a study of their own and share experiences, a follow-up post is a possibility, with more tips as well as what worked and what didn’t.
So: How to start designing a study of your own teaching: Some quick and easy steps…
- Define your question. Keep it small. Something along the lines of: Does this technique work? Then ask: how does the question connect to course objectives and assignments you’re already using? Is there something in your syllabus and course design you’re currently doing you can draw upon?
- Do some reading. Tap into your network. What have other people done that you can use? Is there anyone on your campus you can team up with or ask for advice? One of my preliminary steps was to host a Twitter #engchat session on discussion in teaching. This gave me some great ideas.
- Brainstorm ways you would know whether the thing you’re doing works…or not. Again, how are you using assignments currently that might be useful here? Could you look at different aspects of assignments you’ve already designed? Are there other things you could look at: not just tests or essays but class discussion, one-on-one time with students in the office? Consider asking different questions of the student artifacts you collect every semester, not just did they learn and what, but how, where’s the evidence, where are the gaps.
- How will you create ways to collect data? Formal writing? More metacognitive/reflective writing? Video classroom work? Create transcripts of discussions or office meetings? Use a message board/blog/wiki? Focus groups? Interviews or a surveys? A mix might yield more robust data and give you a multi-faceted perspective on how your students are learning. I usually look to my students’ written work for evidence of learning, but I think I might be able to do more.
- How will you analyze the collected data to answer your question? If a content analysis of some of your artifacts—written, video, transcripts of class discussion—would yield some interesting stuff, see if you can get access and training on nVivo. Are there students in methods courses who could help with coding and analysis? You can also do an analysis yourself: think about categories that show up in artifacts or transcripts. For instance, maybe you want to look for words that indicate confidence with material or shifts in critical thinking, or use by students of specialized vocabulary that reveals disciplinary mastery. I’ve been thinking about ways to capture the classroom dynamic, the network of interactions, and then connecting that to the knowledge that gets generated during discussion.
- This leads to a pretty important logistical/bureaucratic/ethical necessity: If you’re studying student learning and there’s any chance you’ll go public, you will almost certainly need approval from your school’s Institutional Review Board (IRB). In fact, you should probably talk to your IRB anyway just to make sure your work here is ethical. A number of disciplines have formulated best practices for working with human subjects, as you can see from this bibliography put together by the Conference on College Composition and Communication; your institution will most likely have a committee charged with reviewing applications. You may need IRB approval to seek grants, and it might affect other aspects of your planning and timeline especially as you’re designing your course and assignments, so talk to your reps.
- Visit your campus faculty development office, if you’ve got one. The first thing I did was sit down with the Director of our Office of Pedagogical Support. He was encouraging, helped me frame my research question, and assisted me with looking at what I’m already doing to get started with data collection. This person may also be able to help you connect with other faculty, and should you decide to expand for publication or presentation, s/he may be able to suggest venues.
Is there any area of your teaching you’d like to investigate? Have you designed a study of your teaching strategies and have ideas to share?
Photo by Flickr user naosuke ii / Creative Commons licensed
