Education Emptiness

I have read too much Roger Schank. And I've been thinking too much. This is something best avoided, especially when marking student essays. Why? Because it makes you question what it is we're doing in Higher Education.

Assessing Essays

Marking essays is a soul-destroying task, as you have too little time to spend on each essay, and you have a large pile of essays to process.  Most students spend days and weeks on preparing their essays, so it always feels wrong to read and assess them in about half an hour, including writing up your feedback. This is very unsatisfactory, but otherwise one simply cannot turn around the marking in the allocated time.

But the worst thing about marking is its reductionist nature. An essay is a complex piece of writing, comprised of style, argument, expression of knowledge, understanding, interpretation, analysis, discussion etc etc. And all these different dimensions get conflated into a single point on a one-dimensional scale: a grade between about 40 and 70. This is just not right.

Many essays end up having the same numerical grade assigned to it, but they are not really comparable. One student might write eloquently but superficially, another provides deep insights with terrible grammar. One student might have a great idea, but not much understanding of the underlying concepts. Another one has solidly learned all what was required but lacks the creativity to apply the principles to a given problem. Yet, they all get the same numerical value. Different feedback, sure, but that does not really count for anything.

School children get detailed reports (besides a few simple letter grades), but in HE there are simply not the resources to do this, as there are too few staff and too many students, unless you are in Oxbridge. In principle that should not be an insurmountable problem.

You're doing it wrong!

After grading a mathematical crime is committed: the numerical grades are added up and averaged. This is simply not possible. The numbers are not numbers, they are labels that look like numbers. If we assigned the essays letters, then it would be more obvious: what is the average of A and B? But that is a completely different issue to be discussed on another day...

Essentially, then, after a lot of adding and averaging, the whole three years a student spends at university is reduced to a single label again, the degree classification. This is again an enormous reduction of a multitude of information into a single point out of four. And this point decides what possible career a student can then pursue...

As the degree class is so important (and expensive, especially for the incoming cohorts of students), this tends to be at the forefront of students' minds. This is of course a wild generalisation, and there are many exceptions, but from my experience a lot of students are primarily interested in getting good grades. Learning becomes secondary, and only the means to the end of achieving good grades. That means, curiosity, a central ingredient for successful learning, suffers, or rather, is redirected into finding out how to get grades. One cannot really blame students for trying to game the system, which is essentially what they learn to do in the end.

It does work elsewhere...

Postgraduate work is different, though, as it is less regimented. And, more importantly I think, PhD students do not get a grade. It's pass or fail. You either get a PhD, or you don't. There are of course, differences: you could get through with major corrections, minor correction, or no corrections. But nobody will know whether you scraped through with a 'revise and re-submit' or sailed through without any required corrections. If it works for PhDs, why not for UGs as well?

The problem is that we've got too many undergraduates, so there needs to be some differentiation. But why? Who wants it? Presumably those who employ graduates, so that they can see who is better or worse. But does the degree class really reflect vocational ability? I would doubt that, but in the end it is just another filter to reduce the 52 applications for each graduate job to a manageable number. With PhDs this is not so much of an issue, as you can get a more rounded picture by looking at their previous grades or even publications.

In-Conclusion

So what is the solution to this dilemma? It basically requires a system change, which is probably not feasible. Employers want differentiation, universities want to climb up the league tables (which nowadays tend to include employability metrics), and students want to have something that distinguishes them from the crowd. But in the process, education suffers. Learning is not really the focus of HE, and we're just churning out graduates who are good at spotting what is needed to get a good grade and doing just that.

We're assessing far too much, and it destroys what I think universities are all about: expanding your horizons, applying your knowledge and curiosity to interesting problems, be able to fail tasks without jeopardising your future career, and generally maturing and learning stuff.

Catching up...

The last few weeks have been very busy. Nothing like a bout of marking to mess up your basic schedule! Now that marking is out of the way, normality is slowly returning. Still, there are things to do, and blogging tends not to be the one with the highest priority.

In my Frameworks module, the podcasting was the thing to suffer most from marking. A colleague told me that one of her AST students was sad about it, and I feel somewhat bad myself. I will try and get another podcast put together this week. In the meantime I set the students a task that involved them making their own podcasts, and slowly the results are coming in. Some are really good! They will be made available on the course blog, finally a way to get some activity going there. With any luck, students will not only post their podcasts, but also listen to each others and comment on them.

Language Foundation is slowly ticking along, and finally we're getting to the point where students feel it making 'click' during the grammatical analyses. They are hard work, but by constantly practicing them I believe students will better understand how they work, and feel more self-confident. Only the exam will tell, of course!

Then, almost out of the door is a research project proposal. Only a few things to sort out, and then the big question will be if there is enough money still around within the AHRC to fund it. This leads me back to my main gripe with the current model of research funding: the effort that has gone into this project proposal could have been spent on doing quite a lot of the actual work. Add to that the time spent by university admin people checking the figures, and by the AHRC administrators, and the academic reviewers, etc, and you will find that you probably have spent more money altogether on that proposal then it would cost to just do it. And that is assuming it will get funding, otherwise all that money would simply be wasted.

I can see that this doesn't work with your average science project, even if not all of them cost as much as the Large Hadron Collider, but many smaller humanities projects should just be funded directly. Cut the red tape, avoid the frustration of having your proposals rejected, and put some trust in your academics!

Enough ranting for today, still some things to do before today's list is empty.

Audio Feedback Revisited

I have now actually tried this out. My final year option only has half a dozen students, and this seemed to be an ideal opportunity. And I think it worked.

Firstly, it does take a bit more time. On average I have recorded about 10 minutes of commentary per essay. After working out the best way to use the digital recorder I did not have to post-edit the recording, but I still needed to listen to it, which means another 10 minutes. Administrative stuff is fairly easy, as the recorder records straight to MP3, I only need to rename the file with the student id (which I announce at the beginning of the recording). The files are around 2 MB in size. Not sure about how to distribute them yet; for the time being they're all on a CD ROM that goes with the pile of essays.

However, my feedback is obviously a lot more detailed than it would have been just in writing. I do hope that the students find this useful, and I intend to ask them what they think. They will also get a(n ultra-short) traditional feedback form, as there has to be something on paper.

This whole thing was intended as an experiment, and on occasion I might try this again (let's see what the external examiner thinks of it first!), but I do not intend to persuade the department at large to do the same. It might not suit everybody, and for larger modules it is indeed a lot more time-consuming. And I wouldn't want to get anybody to change the way they work.

To be continued...

The use of "use" is used to make use of the use of "use"

Looking through a stack of student essays and exam scripts makes you spot patterns. One pattern that I noticed today is that of superfluous use of words such as "use", especially in sentences like "the use of evaluation is used to indicate such and such". Why not simply write "evaluation is used to indicate ...", or even "Here, evaluation indicates..." etc. Shorter, more to the point, and sounding less bloated and repetitive.

Another common pattern seems to be using the wrong verbs with abstract nouns. Hypothesis are being fortified, an author enhances a concept, and various other examples which of course now escape my mind. Next time I shall keep a list, which I can then throw back at my students in the hope that the use of this list can be used to improve their academic written use of English.

Silence

Not much going on, teaching-wise. And little time to write new blog posts, for it is marking time. And I have to get a paper finished by last week...

Once marking is out of the way, fairly normal service will resume!

Audio Feedback

Today I took part in an on-line seminar on providing feedback to students not in writing, but in audio format. The seminar itself was interesting, as people from all over the UK were participating, and there was a combination of 'chat' (written) and audio/video (mostly from the people organising it).

One project reported on an experiment where students were given audio feedback over the course of a year, and they all liked it (and so did the staff). With speaking, you can say more in the same time you'd need to write it, and it's also a richer medium -- presumably less ambiguous due to tone of voice etc. The time required to produce it is variable, from shorter to same to longer, and that probably depends on how streamlined the whole process can be made. Here is a summary of some of the advice given.

One of my seminar groups whose essays are now in my marking pile is fairly small, ideal circumstances to try it out. After consulting both the head of department and the exams officer I'll give it a go, but will have to provide a traditional feedback sheet as well. I've ordered a digital recorder (which can record straight to mp3), and will start marking their essays as soon as it has arrived.

Just need to think of a way to elicit feedback from the students to see what they thought of it. And a way to deliver it! Presumably using the dreaded WebCT. Maybe one thing it is actually useful for.

Marking

The marking season hasn't even started yet, and I'm already tired of it. At present I'm looking through a pile of formative assignments, which are exam-style questions to prepare students for what is awaiting them in a few weeks' time. And I don't like doing it.

Why not? Is this the typical whingeing about an unpleasant job? Partly. I can think of many things I'd rather do, and that I still have to do once the marking is finished. But that in itself is not the reason. Giving feedback should be one of the central tasks in education, but the ways we have available for it are not very satisfactory.

Ideally I would write an essay about the length of the students' submission to give them proper feedback about all the things I like and don't like about their work. Because the student will read it at some point, and might come back to me with questions about it, I have to be very specific, and have to make sure they can be read out of context (ie without having the student essay still fresh in mind). This is very difficult to do. And would consume more time than is available.

It is also the case that it is very hard to phrase feedback properly. I can easily read a text and decide whether I like it or not, but then telling somebody else why I came to that conclusion is difficult. Often these points are on an almost sub-conscious level, and hard to verbalise. It would also be a lot easier if it could be delivered in a face-to-face conversation, as interactivity would be much better than simply a list of bullet points with good and bad points about the student's work. Resource shortage however makes that impossible, apart from practical considerations.

Maybe it would be an idea to try out what other people have done: record feedback orally, and then send it to the students. It would be quicker to say something, rather than to write it down in great detail, but dealing with sound files might be more time-consuming in the end.

So it remains the only option for the moment, filling in feedback forms, printing them off, attaching them to the paper, and hoping that the students will be able to use them to improve their work. Unsatisfactory!