Joys and Pains of Rubrics (working title)

I confess that I really did not begin as a big fan of assessment, especially not for Creative Writing. On the surface our field does not seem to lend itself to analysis by a rubric, where everything has to be understood and ranked based on a grid. Furthermore, I teach in a small program and the data we are able to generate from the small sampling of students each year seem statistically insignificant. Nonetheless, I am required to go through assessment, and if I must spend a significant amount of time and energy doing something, I want to find some value in it.

One value can be found in actually reading our Senior Portfolios carefully. We use these as our primary assessment tool, yet though we have collected them for the past ten years or so, I confess that I have only begun to read them cover-to-cover since we began assessment. It is a joy to read these portfolios without the need to place a grade on them. We have mapped out our curriculum and defined our program objectives, and we are only required to assess a couple objectives each year. So when I read portfolios, I only need to assess two narrowly defined sets of skills and can enjoy the rest. The value of reviewing our students’ work happens despite assessment, but the assessment process forces me to do it.

So, though I question the validity of the final outcome of assessment, I can believe in the process. If the goal is to improve the way we teach, then there may be more value in it. As I said, I teach in a small program, and when it was even smaller, I might have argued I had no need of assessment. I could intuitively understand more about my students and my program with much less effort than I could learn by going through assessment. Assuming I was committed to doing this on an ongoing basis, individual self-assessment would be preferable to institutionalized assessment. Yet now that my program has grown to include two faculty, an institutionalized process has its advantages.

Assessment gives me a reason to get together with my colleague and discuss what and how we teach without evaluating what he does. We collaborate and evaluate each other, and we are assessing the strengths and weaknesses of the program as a whole. My colleague teaches fiction and I teach poetry, but we both teach a cross-genre introductory Creative Writing class, thus it is imperative that we both understand how the other teaches each genre. Assessment provides a platform for this conversation. I do not believe that there is only one way to teach creative writing nor that we need to standardize the way we teach, but we do need to understand each other’s methods.

Herein lies the greatest challenge for assessment, however. Creative Writing is inherently paradoxical. When I initially considered sitting down and coming up with a rubric to assess creative writing, it felt impossible. When I consider what I want my students to be able to do, there are too many options. I don’t want them to rhyme unless they can rhyme well. I want them to write realist fiction, unless of course they don’t. For every rule or skill I could come up with, I could think of one or more exceptions. So the question became, “How can you assess something when there is no one right way to do it?” I won’t pretend to have all the answers, but I do want to present a few possible solutions to this conundrum.

The first step toward a solution, I feel, is to consider broadly defined learning outcomes for assessment. When I consider rhyme, for instance, I don’t think that all poets should even use it. But I do think all poets should have a good sense of how sound works in a poem, whether that is a free verse poem or a formal, rhymed poem. So I opt for the broader objective of being able to use rhyme or other sound techniques. For fiction I might look at how well students structure plot and the narrative arc of the story. In both cases, I want to consider several ways of fulfilling this objective to give students with different writing styles a chance to excel.

When constructing a rubric, there are two basic options. I could opt for a holistic rubric, which would list one criterion: for example, use of sound techniques such as rhyme, alliteration, assonance, and consonance. If I chose this approach, I would have to come up with one description for each level of ranking in my rubric: what constitutes excellent use of sound, adequate use of sound, or poor use of sound. Personally, I would find this confusing, since my descriptions would be too convoluted and contradictory. An acceptable use of sound might include end-rhyme that isn’t predictable or it might include no end-rhyme at all but some internal rhyme. An acceptable use of time management and narrative techniques might involve a clear chronological progression, or it might include effective use of flashbacks, but it wouldn’t include a confusing structure, inefficient chronology or over-used flashbacks.

Because the choices are complex and because my goal is to develop a clear statement of criteria, I prefer the analytical rubric over the holistic form. An analytical rubric allows me to break the broad learning outcome down into ways a student might demonstrate proficiency in this area. A student need not excel on all the criteria, so I try to set them up to reflect my real expectations for this outcome and to allow students to score well if they do meet these expectations. Remember, talking about expectations with my colleague is one of the main values I find in the process, so establishing clear criteria that match our common expectations is extremely important. How students actually score is less important, though testing the criteria on actual student writing does validate whether our descriptions are useful ones or not.

A few examples may help clarify the process. Use of end-rhyme is one issue I face when teaching about sound in poetry. Many of my students come to me expecting poetry to rhyme and using very predictable rhyme combinations. I am actually happy and feel I’ve taught them something, if they stop using end-rhyme altogether, so I put no use of end-rhyme at level 3 (acceptable), along with limited use of end-rhyme (some of which is predictable). Level 2 included use of some predictable end-rhyme in a serious poem, and Level 1, unacceptable, involved many predictable, forced rhymes. I included the qualifier, “in a serious poem” to account for surprising, humorous or ironic use of predictable, forced rhymes, which I might include in level 4 or 5, depending on how well it is done. Similarly, I developed criteria and rankings for the use of off-rhyme, internal rhyme, rhyme scheme, alliteration, and assonance and consonance. I also included one criterion for the overall use of sound in the poem as a way to give a little more weight to the areas of strength for each student.

Similarly in fiction, I expect any story to have some narrative arc, a beginning, middle, and end with tensions or conflict, complications or rising action, and some sense of change in a conclusion. I look at both linear and non-linear narrative techniques, and I include a criterion for the use of narrative breaks, regardless of whether they represent forward or backward movement in time. And I rank each student on the overall use of time and narrative structure.

Students can be expected to score high in some areas and lower in others, though I try to set my average expectations for each criterion as a 3. I set our program's objectives for these learning outcomes to be that 90% of our seniors would have an average ranking of 3 or higher, and we may find that we can set our objective higher than that. For our entry-level class, though, we set the objective as an average rating of 2 or higher to show that they have been introduced to these concepts but haven’t achieved a level of mastery yet.

When we applied these rubrics in real-world assessment, we found that they worked relatively well. They were relatively easy to apply to student writing, and yielded the results that matched our general impression of our students’ writing, though we did see the need to better calibrate the way we apply the rubric, especially in assessing our introductory class. In general, we met our objectives, though we did learn that our students favor traditional linear narratives and avoid end-rhyme, perhaps too much. We agreed that we could push them to experiment with other options more, though given our small student population, these results may not be typical. Over a period of several years, we might find an acceptable level of experimentation by some students. We did “close the loop” with some recommendations, notably to make the Senior Portfolio into a one-credit class, allowing us another chance to mentor our students and encourage the kind of experimentation we see is somewhat lacking.

Though I am relatively satisfied that these rubrics are both specific enough and flexible enough to assess complicated and contradictory learning outcomes in creative writing, I don’t believe they are perfect. You’re welcome to use them, but your expectations may be different than mine and the way you teach poetry and fiction may be different. I hope these rubrics give you some ideas on how to assess your programs, but let me end with a few alternatives that I might consider using in the future. If I were to assess a graduate program, I would obviously need to adjust my ranking, combing 1 and 2 into the first level, perhaps, and breaking up 4 and 5 into the top three levels. Or I might consider adding a 0 level to the rubric. Another option that I was tempted to use is an “N/A” column, so that some criteria could be skipped if they don’t apply to an individual student’s work. I might want to limit the number of N/A’s I could assign. And finally, I’ve considered using a weighted rubric that could be weighted differently for each student, depending on which criteria apply best to the students’ work. In the end, though I like it conceptually, I think this approach might get too complicated to actually score, and it might look too loosey-goosey to the outside world.

Ultimately, I want to develop rubrics that help me and my colleagues communicate about what we expect of our students. I want rubrics that make sense and are easy to apply to student writing. And I want rubrics that yield results that match my intuition and lead to improvements in teaching or confirm the quality of current teaching methods. If this is the result, I can put up with the bureaucracy and the busywork, and maybe even find some joy in assessment.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License