On-line Education?

I have last term completed the on-line module in Artificial Intelligence offered by Stanford's Sebastian Thrun and Google's Peter Norvig–both top-academics and experts in their field. I guess it was successful, as I received a grade of 79% (a 'first' in UK terms, but I have the suspicion it doesn't work like that). Given the minimal effort I put in (mainly due to lack of time) I could very likely have achieved a better result with some extra work. But with a full-time job it's not so easy to put aside 10 hours a week for doing so, which was the amount of time recommended by the course leaders.

So I got 79%, but did I learn anything? And do the 79% reflect my achievements? And what was the overall learning experience like?

First, the learning experience: the module was delivered as a series of short low-tech video lectures, interspersed with multiple-choice or number-entry quizzes. Then there was homework (multiple-choice and number-entry quizzes) and a mid-term and final exam (both multiple choice and... you get the idea).

The lectures were interesting: it was a camera view top-down on a piece of paper on which the lecturers would (hand-)write, not just a filmed 'lecture'. The tone was informal and friendly, and Thrun's charming German accent made me almost feel at home. And I also learned–from the view head-shot video sequences–that Peter Norvig likes colourful shirts.

The quizzes, however, were rather limited. There was the problem of turning quite complex material into a simple format, and also (which I found hardest) missing context. As a result the questions were often trivial side-aspects, or impossible to answer due to ambiguity (judging from the few forum posts I looked at, many other people had the same issue). You can interpret a question in many different ways, especially if you need to take into account external constraints which have not been clearly specified.

Quite often you get an answer wrong, and then look at the explanation of the proper outcome, and you think "oh right, that's how they meant it".

I quite struggled with Bayes networks, and I consistently got the wrong answers when asked how many independent parameters I would need to describe one. To this day I do not know why I need to know this. I can guess, but it wasn't really explained. Formal logic was one of the things I felt very comfortable with, as I had covered that in my own UG studies as a computational linguist, but I only got 1 out of 4 points in the final exam question, as I made one small error; based on that the subsequent answers were also wrong.

My best results were in computer vision–100%. And that's even though I'm short-sighted! But do I really understand computer vision so much better than all the other areas of AI? No. Thing is, all that was asked in the relevant quizzes was basic maths. There was a simple formula, relating various parameters such as focal length and distances to each other, and all you had to do was resolve the equation for different values and work out the result. I would have been able to do this beforehand, and didn't even learn that in the course. Still, I was assessed on it and scored 100%. But anybody with basic maths would have been able to do that, even without watching a single minute of any of the class videos.

So my first criticism is: the quizzes were not designed properly. There is a lot more one can do with multiple choice question, but Thrun and Norvig didn't do it. The assessments felt like an ad-hoc addition, along the lines of "I need a quiz now, so what could I ask?".

My second criticism is the way the scoring worked. One slight mistake, nil points. In a real exam you would get points for results which are wrong, but only because of a mistake in an intermediate step. An all-or-nothing approach is not very helpful.

Is this the future of education? Are on-line classes like this all we need? I don't think so. Apart from the implementation–it'd be easy to come up with some better quizzes–it's also quite detached. There is little direct interaction (impossible with 140,000+ students), and at times you feel a bit lost. It is obvious that this was an experiment, and as such it is not possible to expect wonderful and perfect results, but there is still a long way to go.

Did I learn anything I would not have learned from reading a book? Probably not. The main advantage for me was to have the pressure of getting through the weekly session before the hand-in date, which makes you put aside time you would otherwise spend on something else. So in that respect it is alright; and the fact that it was delivered on-line was convenient as you could choose the time when you wanted to study it. But while this is good for a supplementary course, I am glad I did have proper seminars and lectures when I went to university.

While you can't argue with a free course (you did get more than you paid for!), there is still a lot of scope for improvement for this particular type of course, an on-line distance course, and I cannot see it replacing 'proper' seminars any time soon. But it was overall an interesting experience, if only to find out what 'real' teaching should be like.

Pointless Quizzes

This morning I completed the University's on-line Diversity Training. In principle a good idea, as it raises awareness about disadvantaging students (or members of staff) where you didn't think you were, but in practice just another thing to do during an already full schedule. And much of it was not relevant for me anyway, as I am not in a position to determine the level of pay of my fellow members of staff, either male or female.

What struck me, however, was a particularly bad quiz. I've been thinking about this in the context of the Stanford AI-Class (on which I will post soon - it'll be finished in two weeks' time), and here it came up again: there were two short on-line multiple-choice quizzes embedded in the course.

The first one was so simple that I could just guess the right answers without having needed to read the previous text. If they are so glaringly obvious that anybody can get them right just with a bit of common sense then it does not really contribute to a good view on the course as a whole - it just comes across as—literally!—a box-ticking exercise.

But the second quiz was even worse, and not only because I got some of the answers wrong. There were various scenarios given, and the four choices you had to choose from were: was this case a) victimisation b) direct discrimination c) indirect discrimination or d) nothing illegal.

Why on Earth do I need to know the difference between 'direct' and 'indirect' discrimination? As far as I am concerned, I need to know what is legal, and what is not, in other words there are only two relevant categories for me: 'discrimination' or 'no discrimination'. So I got several answers wrong because I chose 'direct' when the answer was clearly 'indirect' or the other way round. This was just plain annoying. I can see that I need this when wanting to work in the legal field of employment tribunals, but as a simple bod delivering seminars and lectures about language to students I couldn't care less—as long as I know that I'm not doing anything that would count as illegal discrimination.

But this seems to be a frequent pattern with many on-line tests. Because the people creating such tests have not thought about them properly they come up with spurious details that they ask for, just so that they can have four possible choices, when ideally you should start at the learning outcomes—what do the people taking the quiz need to have learned, and how can we test this?

It is perfectly possible to do really good and useful on-line multiple-choice quizzes, but it requires work and thinking about the purpose of it. Otherwise it just annoys people and makes them want to do things to you that I could not possibly mention on this blog.

More on this topic in a few weeks when I will be discussing my experiences with the on-line AI course...

Why I deleted my FaceBook account

I have just deleted my Facebook account. I cannot remember exactly when I joined, but it was probably 5 or 6 years ago. A while ago I already removed most information about me (such as what music or books I liked), as I felt increasingly uncomfortable with FB's way of making more and more of your information about you available to other people, unless you explicitly disallowed it. This did not feel very honest to me.

And then today came the proverbial straw: I read two (unrelated) posts about FB in direct succession which convinced me that it was finally time to cut the cord. The first [1], showed how FB does not really 'log you out' when you log out - it keeps certain cookies in place which can identify you. I don't use many public computers (especially not with FB) so this does not overly concern me, but I see this as yet a further violation of default expectable privacy.

The second article was vaguely similar, and shows how FB can track where you have been, and other sites can post on your 'wall' when you simply read a webpage. This is just silly. I'm - again - not overly concerned about this (along the lines that I don't generally do things which are illegal or immoral), but on top of that it just contributes to the already existing information overload. If I need to care that person X read webpage Y then I would expect X to tell me. I don't want a stream of activities swamped with reports what websites people I know have visited.

Anyway, those two articles were enough to sway me far enough to permanently delete my account. Not sure what 'permanently' means in this context. For at least the next 14 days FB keeps my account in case I'll change my mind, and I don't exactly trust it to delete anything for real anyway. Remember, in the FB business model, FB's assets are you, its users and their data, which they mine and sell on to other people.

Will I miss FB? I haven't really used it that much in the first place. I'm much more active on Twitter, which is somewhat less intrusive and has fewer opportunities to do stuff with my data. I won't now not as easily be updated on what some family members who live abroad are doing, but there are other ways of keeping in touch. The main issue is our postgraduate students (I am coordinating our English Language PG students) - they have recently set up a FB page, which I now won't be able to see. But most things are still posted on a traditional mailing-list anyway.

On the positive side, I no longer need to deliberate whether I accept somebody who wants to be my friend or snub them if I only vaguely know them. Fewer decisions to make equals more happiness.

It feels weird, cutting the cord, as it did with any account on any system I spent a reasonable amount of time on, but in the long run I don't think I will shed any tears over it. I'm just concerned that FB will spread its tentacles out further, so that at some point in the future everybody is expected to have a FB account, and you cannot do certain things without one. Matrix, anyone?








[1] Apologies for the shortened (and thus opaque) links - they're directly copied from the corresponding tweets

Plagiarism - setting an example

Germany's Defence secretary, Karl-Theodor zu Guttenberg, resigned today. What that has to do with a blog on learning and teaching, you ask? The reason for his resignation was that he plagiarised large parts of his PhD thesis, though he said this was only because he was so busy with his job (MP) and family (two daughters) and he didn't notice that he copied some hundred or so pages. His supervisor (retired) was of course shocked, as he was one of his best students, and he wouldn't have believed any accusations of plagiarism by the so-called "Baron zu Googleberg". There's the old blind trust in 'honourable' people again...

Being a conservative, Guttenberg could count on the support of the conservative press, such as BILD, which ran articles in which it was emphasised that he was a good minister, doing a splendid job for the boys in Afghanistan, and that he shouldn't resign because a couple of academics have lost all sense of proportion and demand his resignation over a piddly little academic infelicity. Those people in their ivory towers, out of touch with the real world of our troops dying in the Hindukush, how dare they ruin the career of this brilliant man... just because of some stupid footnotes!

But he finally resigned; and his PhD was taken away earlier by Bayreuth University, which is none too happy about the PR implications. And now we have a good example that we can show our students: even the powerful can fall if they commit plagiarism! While plagiarism should indeed reason enough to not be fit for public office, Guttenberg himself of course states it was not the only reason... showing that he doesn't seem to bothered about his integrity.

There is also an interesting linguistic side-aspect to this: in English we talk about plagiarism, meaning the process of plagiarising something. In German you talk about the Plagiariat, the product of copying, rather than the process. Of course you can 'verb' it into Er plagiarisierte seine Doktorarbeit, but that sounds rather awkward. The offender, a plagiarist, is also not directly lexicalised in German.

And to end on a happy note, here's a joke I heard on Twitter:
Fragt der Praktikant im Verteidigungsministerium "Wo ist denn der Kopierer?" Antwort: "Auf Truppenbesuch in Afghanistan" (Intern at the ministry of defence asks "where's the copier[*]?" Answer: "Visiting the troops in Afghanistan"). Who says Germans don't have a sense of humour?!




[*] Kopierer in this usage would generally be understood to mean 'photocopier'

Numerical illiteracy

I had a look at our free local newspaper, the Birmingham Mail Extra. In this issue is an article titled "Suburb is now crime hotspot". This article has several problems, which are indicative of the problems of representing what happens 'out there' in the form of a newspaper article.

Quinton had very little burglaries, "averaging just one or two burglaries a month" over the last nine years. But now, crime has "shot up by 29%", more than the rest of Birmingham where crime only rose by 21%.

I find this rather misleading, and I worked through an example with the kids at the dinner table, which illustrates the problem:

Suburb A had 10 burglaries in 2009, and 12 in 2010. That is an increase of 20%.
Suburb B had 4 burglaries in 2009, and 5 in 2010. Increase of 25%.
Suburb C had one burglary in 2009, and two in 2010. A whopping 100% increase.

First lesson: percentage increase is meaningless unless everybody started in the same place. Why do developing economies have higher growth rates than Europe and the US? Because the percentage looks bigger. If you double your output from 100 cars to 200 cars you increase by 100%, but to maintain that rate you will have to produce an additional 200 cars the year after, and then 400, 800 etc.

Second lesson: without a fixed reference point (such as burglaries per 1000 inhabitants) you have no idea whether the risk of your house being burgled is high or low. Everything is relative, but you still need a fixed point to evaluate things.

Third lesson: this is not mentioned, but is that difference between 29% and 21% statistically significant? With "one or two burglaries a month", the variation seems rather high, given those small numbers. So if there was one extra or one fewer burglaries, how would the 29% change?

Final lesson: be careful with calling places "crime hotspot" when your statistics are that shaky.

Now I don't expect a staff reporter at the Birmingham Mail to apply the same rigour as a scientist (or any other, non-science researcher), but the way the article is presented is simply misleading. To me this sounds like a scare story being created out of some random statistics. I don't know whether that is the case, and maybe Quinton is really a crime-ridden area, but I cannot tell from the few facts given to me in the article. The reason for this development given in the paper is that there are now 5 fewer policemen covering Quinton. Is there really such a cast-iron correlation between policemen and crime rate? How about the influence of the recession on crime?

I now have more questions and know less than I did before reading that article!

The cuts start now...

My two elder daughters (who are at primary school) are currently getting free language tuition at a nearby secondary school. That school has specialist status for languages, and provides those additional classes for free as an after-school activity for the younger ones. Apart from teaching the kids Spanish/French/German/... I guess they also benefit themselves through advertising their school. No idea how much the kids actually learn, but it is a fun class, they enjoy it, and if they can speak a few French phrases in addition then that's great. It definitely gets them interested in foreign languages, and that is a bonus given the current attitude to those in England.

However, we yesterday got a letter from the school. The new coalition government has abolished the 'specialist status' scheme, and presumably that means that any funding that was connected to this status is going to disappear. From April onwards, the letter states, the school cannot afford to provide those additional language classes any longer.

This, of course, is not the school's fault. But it is really sad that foreign language classes for primary school kids are disappearing, and it shows that this government got its priorities wrong. I'm not too worried about my own kids, as they get a lot of exposure to other languages, but children from mono-lingual families are now losing out. Not all parents can afford to pay for language classes provided by, eg, Club Français, after school.

In my view, learning foreign languages is vital in the modern global world. Even though many people learn English as a second language it severely limits you if you can only speak one language. And since modern languages are no longer compulsory at A-level, recruitment for modern language departments at English universities has gone down.

Given that the British economy has little manufacturing left, languages are vital if we want to compete on the global market. Cutting down spending in education is the wrong way to move forward. But this cabinet of millionaires does seem to have other priorities.

Who wants to be a millionaire?

The schools secretary, Michael Gove, is quoted in today's Guardian with a comment on tuition fees (arguing that fees are not the barrier to university, it's a fault that lies with schools): "Someone who is working as a postman should not subsidise those who go on to become millionaires."

This is just wrong on so many levels.

Firstly, that also means that a postman should not enable students to train as teachers, nurses, doctors, engineers, and any number of necessary careers without which our society would not function. This leads straight on to the second point: a university degree does not make you a millionaire. On the contrary, I would guess that most millionaires do not have a degree, because they spent their time building a successful career in business rather than reading Shakespearian sonnets and analysing the intricacies of subject-verb agreement in different polynesian languages.

I do not have time to investigate the full Sunday Times Rich list (which is hidden behind a paywall in any case), and in some cases it would not help, as some people are already millionaires before they went to university (and soon will have to be...), but here is an unscientific overview of some prominent millionaires and what I could find out about them on Wikipedia:

Richard Branson, had a "poor academic record"; he holds an honorary degree from Loughborough. His first successful business venture started when he was just 16, so he did of course have better things to do than go to uni. Still, he's #212 in Forbes' list of rich people in the world.

Alan Sugar left school at 16. No academic career.

James Dyson: I assumed he started out as an engineer, but in fact he studied interior design at the Royal College of Art before moving into engineering. No mercy for people following in his footsteps with humanities teaching slashed.

Peter Jones, of Dragon's Den fame, ended his academic career after A-levels to become a businessman.

Theo Paphitis, "began his entrepreneurial activities by running his school tuckshop, at the age of 15." Again, no mention of an academic career.

Duncan Bannatyne - was in the navy for a while, but no sign of a university degree.

Rachel Elnaugh wanted to study art history, but was apparently rejected at 5 universities. That didn't stop her from becoming an entrepreneur. Not sure, though, if she still is a millionaire.

Simon Woodroffe went on the road with Rod Stewart etc after acquiring two O-levels.

Doug Richard is actually one of the 'Dragons' who has a degree, a BA in Psychology from University of California at Berkeley and a Juris Doctor at the school of Law, University of California at Los Angeles. (source). But Gove will be pleased to see it was not funded by UK taxpayers.

James Caan left school at the age of 16. His involvement with academia (Harvard Business School) only started after he made his fortune.

Deborah Meaden is the only other academic dragon: she studied business at Brighton Technical College, which is a further education college.


This of course is not a proper sample, but one gets the impression that most of those successful entrepreneurs have been too busy in their early adolescence to pursue degrees.

And finally, as I do not want to turn this into a long rant, with a large number of UK students going into higher education (though I believe Labour's target of 50% university attendance has not been achieved), our country should be awash with millionaires. And working at a university, I should be surrounded by them. Unless my colleagues are hiding something from me, I think I must live on another planet than the one Michael Gove is on...