The rest of the (Moodle) world is eagerly anticipating Moodle 2.0, but I would like to tell you about what I have been doing for most of the last year, but which you won't be able to have until Moodle 2.1 - unless, that is, you are a student or teacher with the Open University, in which case you will be using it from this December.
What I have done is to rewrite a large chunk of the Moodle quiz system. What chunk is that? Well, first you can split a quiz system into two main parts. There is the quiz part, which says, "This quiz comprises these questions, and will be open to students between these dates". It tracks the student and they attempt the quiz, and stores their total score. Then there is the part that deals with the details of the individual questions within each quiz.
The question part can again be split in two. There is the question bank which lets the teacher create and store questions. For example "This is a multiple choice question where the student must select one of these three options, and it is an 'Elementary maths' question." Then there is the code that controls what happens when a student attempts a question "The student sees three radio buttons and a Submit button, and when they click the button we compute a score as follows and show this feedback." That second bit is what I call the question engine, and that is what I have rewritten.
However, you cannot just change the question engine in isolation. There are knock-on effects. For example, the quiz module still maintains overall control of things, even though it delegates a lot of the details to the question engine. So there are places where the quiz says things like "Dear question engine, please display this question now", or "Dear question engine, the student submitted this data, please process it", or "Dear question engine, the teacher wants to see all students responses to all questions in this quiz, give me the data to display." All those places have to change when the question engine changes.
There were also small changes required to the question bank. mainly because the new question engine has some new features that need some extra options stored with each question. So, the question bank needs to store the new options; let teachers edit them; back them up and restore them; import them and export them; and so on.
Altogether, my year's work added about 52,000 lines of new code and removed about 25,000 lines of old code (or, if you prefer, added 27,000 lines and altered 25,000 lines). At least that is the size of the change that I committed to the OU's CVS server last Friday, just in time to make the feature-freeze for the December update of our VLE. For comparison, the whole of Moodle 2.0 is about 1,600,000 lines of code, although that includes several large third-party libraries.
I am sure that there will be some minor bugs still to be found and fixed, but this new code has already had extensive testing from my colleagues Phil Butcher and Paul Johnson, so I am confident that the remaining bug-fixes will be minor.
There is much more I want to write about the new question engine, but I think this introductory post is already long enough. Therefore, I will split the remainder of what I want to say into separate posts which I hope to publish over the next week or so.
Tuesday, October 5, 2010
Tuesday, September 14, 2010
On the other side of the fence
I've been having problems with my teeth recently. Fortunately, none of it has been painful, but cycling home from yet another dentist's appointment this afternoon, I suddenly had a thought that the experience I was having with my dentist might be a bit like the experience of a non-developer who encounters a bug in the Moodle quiz. Allow me to explain.
I think it has been a total of four trips to the dentist over the summer. First I went for a regular check-up. That revealed that one of my fillings was cracked, so there was another appointment to drill it out and redo it. Then a filling in another tooth fell out one Friday evening, so I could not get anything done about it until the following week, and over the weekend a bit of the tooth next to the hole broke off, which was really worrying. (Luckily, as I said above, it was not painful.) So anyway, that required a big filling to fill the resulting hole. Then part of the new filling broke off, so today's appointment was to re-do the missing bit. Hopefully that remedial fix will work. If the new bit breaks off again, I will have to go back to have the whole filling drilled out and redone.
With all this going on, I fear I have been starting to have unkind thoughts like: "Is my dentist competent?" "Should so many things go wrong over a few months?" "How do I know if this is normal?". This is exacerbated by what appears to me to be a slightly casual attitude on her part. I expect that these really are routine problems, and rather boring to her. I, however, am worried, so I would have appreciated a more concerned-seeming bedside manner.
As I say, it was just after I had climbed on my bike to go back to work this afternoon, that I had the epiphany that this is probably how someone feels when they come to the Quiz forum after encountering a problem in one of their Moodle quizzes. To them it seems like some terrible problem that has them really concerned. They describe their symptoms, and I read it and think "oh yes, the problem is probably in that bit of code, let me do a quick fix." And then maybe I screw up and introduce a regression, but when that is pointed out, it too is easily fixed. To me it seems like some routine and minor matter, but I have never really thought how the process of fixing bugs feels to someone who is not a software developer, and who does not really understand what is going on. Will they still trust Moodle?
Now that I have thought about it, will I do anything? I fear I am unlikely to change my 'bedside manner'. My time seems to be more than taken up with actually doing the bug fixes, and other development, to spend too much time being nice to people. Still, I will try to make a bit more effort in future know I now how it feels to be on the other side of the situation.
Before finishing, allow me to point out (particularly to American readers) that all these trips to the dentist have cost me very little or nothing. In particular, the two recent appointments to replace the filling that had (partly) fallen out cost me nothing. Thank you NHS. I hope the new government does not cut you to death.
While I am writing, I will also share the news that I finally bought an apartment of my own over the summer, after years of renting. It is very nice, and worth the hassles of dealing with solicitors. Hopefully now that the stresses of dealing with the move are behind me I can concentrate more on Moodle development, although there are still a few minor things to deal with like getting some more furniture. Over the weekend I ordered some new sofas, including some bright orange cushions. I can't think where I got the idea for such a daring colour ;-)
Friday, July 9, 2010
Book review: Moodle 1.9 Extension Development
Yes, I think it does. The authors, Mike Churchward and Jonathan Moore, are two experienced Moodle developers (they both work for Canadian Moodle Partner Remote Learner) so they can write authoritatively on the subject.
One issue with a book like this is that the examples given are, necessarily, fairly basic. To illustrate key techniques and ideas, a book must explain using the simplest example that makes the point. The question is, when you come to solving real problems, will the techniques you have learned expand to cope? Well, this is where the experience of the authors counts. They are telling you the right way to do things that works for real applications, even if they are only using simple examples to illustrate them.
The book does a thorough job of covering just about every type of Moodle plugin there is. Of course, some plugin types get more space than others, with the two most important, blocks and activity modules getting the most space. Therefore, some other plugin types, like question types, and gradebook plugins, are covered rather briefly. Between the chapters on the different types of plugins are chapters on more general topics like security, accessing the database, and so on.
Anyone who has had code reviewed by me will know that I get really pedantic when I review something. As I was reading the book I made a list of minor errors, or points where I disagreed with the authors. From the 300 pages of the book, I only found 22 things to put on my list, and none of them are interesting enough to mention here. (I did send the list to Jonathan.) So, I think this book has a very high standard of accuracy.
This book does assume you already know how to program in PHP, and write HTML and CSS. I think that was the right decision. There are plenty of excellent books out there that will teach you to write general web application in PHP, and it would be silly to duplicate those in a book that is uniquely about writing code for Moodle.
It is unfortunate timing that this book was released only a few months before Moodle 2.0. Moodle 2.0 does change quite a lot of the rules for how to do Moodle development, and so a lot of the details in the book will soon be out of date. However, don't let that stop you from getting this book. We have just talked about how this book helps you make the jump from being a general PHP developer to being specifically a Moodle (1.9) developer. Well, from there to being a Moodle 2.0 developer is just another small step. You won't be wasting much time if you learn about Moodle 1.9 first, and anyway, some people will still be running Moodle 1.9 for some time to come, and it will be a while before there is a book about Moodle 2.0 development on sale.
This should go without saying, but programming is an activity that you actually need to do to understand. You won't become an expert Moodle programmer just be reading a book. You will become a Moodle programmer by actually trying to write Moodle code, and learning from your own mistakes, and from the code other people have written in the past. What a book like this will do for you is that it will help you avoid a lot of the really basic mistakes, and it will set you off on the right path. So it will make your own learning-by-doing much more efficient, but I cannot replace the doing. Also, I would like to point out that while this is the only book about Moodle development, it is certainly not the only resource to help you learn Moodle development. If you are interested in this book, you should also look at the Developer documentation on Moodle Docs and the Introduction to Moodle Programming course on http://dev.moodle.org/.
Overall, if you want to learn Moodle development, this is a good book to help you attain your goal. Sure, you can get a lot of the information for free online, but in this book the authors set it out clearly and in a logical order. The information in this book has been written by expert Moodle developers and then carefully reviewed, so you can read the book without being on your guard for misinformation. You would have to be more careful just using the information Google finds for you online. So, as I say, this book lives up to expectations. If you want a book on Moodle development, get this one, and don't worry too much about Moodle 2.0 making it out-of-date.
Wednesday, June 9, 2010
Computational Knowledge
I spent today at the London Computational Knowledge Summit. A rather pretentious name for a meeting organised by Wolfram about Wolfram|Alpha, Mathematica, and what these sorts of tools can do. They way they see it is that if Google is a means to retrieve facts, Wolfram|Alpha aspires to be a way to use information. For example, governments have started dumping a lot of data into the public domain, but how easy is it for a citizen to extract meaningful information from that data?
For someone used to Moodle and open source, I was deeply struck by how Cathedral (as opposed to Bazaar) the whole Wolfram/Mathematica world is. That may be, in part, a natural consequence of when they started. Wolfram has been working on Mathematica for twenty years. That is an impressive achievement, and it means they have been working on it since before Linux was a gleam in Linus Torvald's eye. They started at a time when the commercial model was how most software was written, and they are clearly still an American commercial software development business at heart.
It is interesting to ask what would have happened of Mathematica had been GPL. Would it have developed more or less than it has as a commercial project? Wolfram would not have been able to pour as much money into it, but would a community have done as much, or more? I am not sure. Clearly the commercial model has been highly successful for driving the development of Mathematica. It is a very cool, and very sophisticated tool. On the other hand, one can dream about a world where the Mathematica engine is free, and as a result shipped on every OLPC XO. What effect would that have on the world over the next twenty years? (Quiz, can you name Mathematica's closes open source rival? I can, but only because it is used in STACK. Speaking of which, it was nice to renew my acquaintance with Chris Sangwin today.)
Another things that struck me about the Wolfram world is how insular it is. Some of the speakers we heard clearly spend all their time thinking about Mathematica. They have a really amazing tool, and, having what they hope is the ultimate sledge-hammer, they seem to view most problems as nuts to be cracked. It made me worry about whether the Moodle community seems that insular from the outside.
However, the day was not just about Maths and Mathematica, but more about the democratisation of knowledge. How can a concerned citizen reach meaningful conclusions about the world using the data that is now available? Conrad Wolfram set the scene by talking about this mission. He started with two slides, one showing a page from Principia Mathematica, and then a page from a modern mathematical paper. They both comprised text, diagrams and equation. Certainly, modern type-setting is better, and the diagrams are now in colour, but that is not much to show for 350 years progress. Is this the best way to publish scientific knowledge in the 21st Century? No, it isn't. For instance, it is silly to show a static graph. It should be a graph that the reader can manipulate, and the raw data should be hidden behind it in a form that you can extract to perform you own analyses on if you choose. Is the paper about a model? If so the model should be embedded there, and you should be able to experiment with varying the inputs and the assumptions to see what happens. If you are interested, you should be able to get at the source code to see exactly what was implemented. So, rather than a document being a static, one-way, low-bandwidth form of information transmission, it should be an interactive things that allows the "reader" to engage in a two-way dialogue.
Of course, Mathematica is the perfect tool for authoring such documents, or so Wolfram would like you to think. To encourage this, they are talking about a CDF - computational document format - which would be an 'open' standard based on a subset of Mathematica's capabilities, with a 'free' player for all common platforms. Think PDF and Adobe. Will it take off? Well, does the rest of the world trust Wolfram enough to adopt their format? Can the offer enough more than HTML5? I am not sure.
The next talk was by Andrew Dilnot from Oxford University. He talked very engagingly about the difficulty with statistics and gave some nice topical examples of the need to interpret numbers in context. Presented with a numerical fact, you need to ask some basic questions: is this big or small? going up or down? where did it come from? For example, the smoking ban has saved the NHS £8million. Well, that is small. The NHS annual budget is £100billion. Why are the media making such a big deal of the story? His talk could be summarised as: All numbers are wrong, but using numbers is still much better than not using numbers.
There were a couple of talks about that were more pretty than interesting. John Barrow from Cambridge talked about the history of scientific images, and Alan Joyce showed some of the cool things you can do with Mathematical. There were also some glimpses behind the scenes of Wolfram|Alpha. As I say, very Cathedral. There is no web spidering or crowd-sourcing here. At one end it is about Wolfram people going out and finding the most authoritative data sources they can get, and loading them into a format their Mathematica code can use. At they other end they are training the Alpha system to take natural language queries, decide which methods to apply to what date, and select visualisation to display the results, in order to automatically give the best possible answer to that question. They gave an interesting parallel for this. In the past, all actors had to appear on stage to whatever live audience could fit into the theatre. Today, film and television allow most people to see the best actors in the world. Can the expertise of the best thinkers in different fields be made accessible in a similar way? Well, that is their goal. The initial results are impressive, but these are only the very early stages of a highly ambitious project and there is a lot more to do.
In the afternoon, Conrad Wolfram talked about his thoughts on maths education. This has traditionally been focussed on teaching students how to perform computations by hand. But, said Conrad, this is only one part of what you need to know to be a mathematician. The full steps are:
At the end, Stephen Wolfram gave a presentation by video link summarising some of the themes of the day.
Overall, it was a very interesting day. While I don't think it quite lived up to its pretentious title, it was certainly more than just a Wolfram marketing exercise. I would like to thank the Open University for sending me. I can see myself using Wolfram|Alpha in future for certain sorts of queries, but it will not replace Google as my day-to-day search engine. (I don't think it aspires too.)
For someone used to Moodle and open source, I was deeply struck by how Cathedral (as opposed to Bazaar) the whole Wolfram/Mathematica world is. That may be, in part, a natural consequence of when they started. Wolfram has been working on Mathematica for twenty years. That is an impressive achievement, and it means they have been working on it since before Linux was a gleam in Linus Torvald's eye. They started at a time when the commercial model was how most software was written, and they are clearly still an American commercial software development business at heart.
It is interesting to ask what would have happened of Mathematica had been GPL. Would it have developed more or less than it has as a commercial project? Wolfram would not have been able to pour as much money into it, but would a community have done as much, or more? I am not sure. Clearly the commercial model has been highly successful for driving the development of Mathematica. It is a very cool, and very sophisticated tool. On the other hand, one can dream about a world where the Mathematica engine is free, and as a result shipped on every OLPC XO. What effect would that have on the world over the next twenty years? (Quiz, can you name Mathematica's closes open source rival? I can, but only because it is used in STACK. Speaking of which, it was nice to renew my acquaintance with Chris Sangwin today.)
Another things that struck me about the Wolfram world is how insular it is. Some of the speakers we heard clearly spend all their time thinking about Mathematica. They have a really amazing tool, and, having what they hope is the ultimate sledge-hammer, they seem to view most problems as nuts to be cracked. It made me worry about whether the Moodle community seems that insular from the outside.
However, the day was not just about Maths and Mathematica, but more about the democratisation of knowledge. How can a concerned citizen reach meaningful conclusions about the world using the data that is now available? Conrad Wolfram set the scene by talking about this mission. He started with two slides, one showing a page from Principia Mathematica, and then a page from a modern mathematical paper. They both comprised text, diagrams and equation. Certainly, modern type-setting is better, and the diagrams are now in colour, but that is not much to show for 350 years progress. Is this the best way to publish scientific knowledge in the 21st Century? No, it isn't. For instance, it is silly to show a static graph. It should be a graph that the reader can manipulate, and the raw data should be hidden behind it in a form that you can extract to perform you own analyses on if you choose. Is the paper about a model? If so the model should be embedded there, and you should be able to experiment with varying the inputs and the assumptions to see what happens. If you are interested, you should be able to get at the source code to see exactly what was implemented. So, rather than a document being a static, one-way, low-bandwidth form of information transmission, it should be an interactive things that allows the "reader" to engage in a two-way dialogue.
Of course, Mathematica is the perfect tool for authoring such documents, or so Wolfram would like you to think. To encourage this, they are talking about a CDF - computational document format - which would be an 'open' standard based on a subset of Mathematica's capabilities, with a 'free' player for all common platforms. Think PDF and Adobe. Will it take off? Well, does the rest of the world trust Wolfram enough to adopt their format? Can the offer enough more than HTML5? I am not sure.
The next talk was by Andrew Dilnot from Oxford University. He talked very engagingly about the difficulty with statistics and gave some nice topical examples of the need to interpret numbers in context. Presented with a numerical fact, you need to ask some basic questions: is this big or small? going up or down? where did it come from? For example, the smoking ban has saved the NHS £8million. Well, that is small. The NHS annual budget is £100billion. Why are the media making such a big deal of the story? His talk could be summarised as: All numbers are wrong, but using numbers is still much better than not using numbers.
There were a couple of talks about that were more pretty than interesting. John Barrow from Cambridge talked about the history of scientific images, and Alan Joyce showed some of the cool things you can do with Mathematical. There were also some glimpses behind the scenes of Wolfram|Alpha. As I say, very Cathedral. There is no web spidering or crowd-sourcing here. At one end it is about Wolfram people going out and finding the most authoritative data sources they can get, and loading them into a format their Mathematica code can use. At they other end they are training the Alpha system to take natural language queries, decide which methods to apply to what date, and select visualisation to display the results, in order to automatically give the best possible answer to that question. They gave an interesting parallel for this. In the past, all actors had to appear on stage to whatever live audience could fit into the theatre. Today, film and television allow most people to see the best actors in the world. Can the expertise of the best thinkers in different fields be made accessible in a similar way? Well, that is their goal. The initial results are impressive, but these are only the very early stages of a highly ambitious project and there is a lot more to do.
In the afternoon, Conrad Wolfram talked about his thoughts on maths education. This has traditionally been focussed on teaching students how to perform computations by hand. But, said Conrad, this is only one part of what you need to know to be a mathematician. The full steps are:
- Ask an interesting question about the world.
- Translate that real world question into mathematics.
- Compute the solution.
- Translate that solution back into real-world terms, and find some way to validate whether your answer makes sense.
At the end, Stephen Wolfram gave a presentation by video link summarising some of the themes of the day.
Overall, it was a very interesting day. While I don't think it quite lived up to its pretentious title, it was certainly more than just a Wolfram marketing exercise. I would like to thank the Open University for sending me. I can see myself using Wolfram|Alpha in future for certain sorts of queries, but it will not replace Google as my day-to-day search engine. (I don't think it aspires too.)
Thursday, March 25, 2010
When do students submit their online tests?
I am currently studying an Open University course (M888 Databases in Enterprise systems. There was an assignment due today, and like many students, I submitted only an hour before the deadline.
That got me thinking, are all students really like that? Well, I don't have access to our assessment submission system, but I do work on our Moodle-based VLE, so I can give you the data from there.
This graph shows how many hours before the deadline students submit their Moodle quizzes (iCMAs in OU-speak)
That is not exactly what I was expecting. Certainly, there is a bit of a peak in the last few hours, but there is another peak almost exactly 24 hours before that, with lesser peaks two and three days before.
Note that all our deadlines are at noon (it used to be midnight, but that changed a few months ago). The graph above is consistent with our general pattern of usage. The following graph shows what time of day students submitted their quiz attempts. It is same shape as our general load graph for most OU online systems.
I don't know what, if anything, this means, but I thought it was interesting enough to share.
By the way, if you want to compute these graphs for your own Moodle, here are the database queries I used:
That got me thinking, are all students really like that? Well, I don't have access to our assessment submission system, but I do work on our Moodle-based VLE, so I can give you the data from there.
This graph shows how many hours before the deadline students submit their Moodle quizzes (iCMAs in OU-speak)
That is not exactly what I was expecting. Certainly, there is a bit of a peak in the last few hours, but there is another peak almost exactly 24 hours before that, with lesser peaks two and three days before.
Note that all our deadlines are at noon (it used to be midnight, but that changed a few months ago). The graph above is consistent with our general pattern of usage. The following graph shows what time of day students submitted their quiz attempts. It is same shape as our general load graph for most OU online systems.
I don't know what, if anything, this means, but I thought it was interesting enough to share.
By the way, if you want to compute these graphs for your own Moodle, here are the database queries I used:
-- Number of quiz submissions by hour before deadline
SELECT
(quiz.timeclose - qa.timefinish) / 3600 AS hoursbefore,
COUNT(1)
FROM mdl_quiz_attempts qa
JOIN mdl_quiz quiz ON quiz.id = qa.quiz
WHERE
qa.preview = 0 AND
quiz.timeclose <> 0 AND
qa.timefinish <> 0
GROUP BY
(quiz.timeclose - qa.timefinish) / 3600
HAVING (quiz.timeclose - qa.timefinish) / 3600 < 24 * 7
ORDER BY
hoursbefore
-- Number of quiz submissions by hour of day
SELECT
DATE_PART('hour', TIMESTAMP WITH TIME ZONE 'epoch' + timefinish * INTERVAL '1 second') AS hour,
COUNT(1)
FROM mdl_quiz_attempts qa
WHERE
qa.preview = 0 AND
qa.timefinish <> 0
GROUP BY
DATE_PART('hour', TIMESTAMP WITH TIME ZONE 'epoch' + timefinish * INTERVAL '1 second')
ORDER BY
hour
Thursday, October 15, 2009
The Open University's approach to plagiarism
I spent the day at an OU internal conference on plagiarism. Until now, most of my engagement with this subject has been people coming to the Moodle Quiz Forum expecting technological magic bullets. There seems to be a depressing assumption in some quarters that preventing students from copying or pasting during a quiz attempt, or from using a spell checker, in some way helps the validity of an assessment. And using Google during a quiz attempt is surely cheating, it couldn't possibly be a valuable study skill for the 21st century?
Anyway, it is just not possible to enforce these kinds of restrictions from a web server application like Moodle. Image, for a moment, what would happen to the web if any advertiser could stop you leaving their horrible web site and going to Google to find something better. If you really want to set up a restricted environment (akin to the traditional invigilated/proctored exam hall) you need client-side software installed on the computer the student will be using. If you are looking for something like this, take a look at Safe Exam Browser, and open-source effort built on Firefox, which is now integrated into Moodle 1.9.6 (due for release soon). See MDL-19145.
I am pleased to say that the OU approach is diametrically opposed to this wishful chasing after technological magic bullets. The conference today started with two keynotes by Jude Carroll from Oxford Brookes University, who has been thinking about these issues for the last nine years, and who is an excellent speaker. She has the very clear summary of the issue: Universities have the responsibility/duty/privileged to accredit student's learning. They give the student a bit of paper saying that they have learned something, and in our society that bit of paper has a certain amount of value. So universities need to measure learning, and that is where assessment comes in. The real problem with plagiarism is that it breaks the link between what the student submits for assessment, what they have learned. If a student writes something in their own words, then that writing is good evidence of the extent to which they have taken in the ideas of a course. If they have just copied and pasted someone else's words, you cannot assess what they have learned. Also, teachers do not set assessment tasks just to measure students. We hope that performing the tasks helps the student to learn.
Jude also made the point that exactly is plagiarism does depend on the context. For example the level of experience of the student, the learning that is being assessed, what you are allowed to assume as common knowledge. Also, not all plagiarism is cheating, and not all cheating is plagiarism. Dealing with plagiarism must be done in an appropriate and proportionate way. To start with, understanding issues of academic honesty is an important learning outcome from a degree course, but it needs to be taught, it is not innate. It cannot be taught as an optional add-on ("the plagiarism lecture") but instead it must be embedded in the context of the subject early in a degree program. (By and large the OU does this in its introductory course.) When plagiarism happens, the response has to be proportionate. Early in a program of study, students need to be given feedback to help develop good academic practices. Later on, students should be expected to understand these issues, and plagiarism becomes a disciplinary issue, but still their need to be a range of fair and proportionate penalties.
The OU has clearly had a group of people thinking hard about this over the last few years, and that has resulted in a new plagiarism policy that was adopted last June. This conference was part of disseminating that policy more widely. There are now a range of penalties for various types of plagiarism, and a new role, 'Academic Conduct Officers' who can deal with most issues of plagiarism that go beyond what the student's individual tutors can deal with. This lets many cases be dealt with quickly, so that only the most serious cases have to be referred to the Central Disciplinary Committee.
Another part of the OU's response to this issue is a new web site Developing good academic practices, which I am afraid you will only be able to see if you have some sort of OU login, student or staff. That is a Moodle site in our VLE. It is a mixture of text to read, with a few audio clips, with a quiz at the end of each section. There is also a summary quiz at the end which draws questions randomly from all the other quizzes. The estimated study time for the whole site is 2 hours. It seems to be well done, and for me it is pleasing to see the quiz module used well, even if it is only multiple-choice and matching questions. The course has about 2300 quiz attempts at the moment.
Finally, the OU does use two technological means to detect plagiarism, CopyCatch and Turnitin, on a proportion of submitted work. However, this is only used to flag up possible cases that are then reviewed by a human. The third form of detection is that tutors marking the students' work can often spot problems. However, overall, the amount of plagiarism detected at the OU seems to be lower than that reported elsewhere. It is interesting to speculate why that might be.
Anyway, it is just not possible to enforce these kinds of restrictions from a web server application like Moodle. Image, for a moment, what would happen to the web if any advertiser could stop you leaving their horrible web site and going to Google to find something better. If you really want to set up a restricted environment (akin to the traditional invigilated/proctored exam hall) you need client-side software installed on the computer the student will be using. If you are looking for something like this, take a look at Safe Exam Browser, and open-source effort built on Firefox, which is now integrated into Moodle 1.9.6 (due for release soon). See MDL-19145.
I am pleased to say that the OU approach is diametrically opposed to this wishful chasing after technological magic bullets. The conference today started with two keynotes by Jude Carroll from Oxford Brookes University, who has been thinking about these issues for the last nine years, and who is an excellent speaker. She has the very clear summary of the issue: Universities have the responsibility/duty/privileged to accredit student's learning. They give the student a bit of paper saying that they have learned something, and in our society that bit of paper has a certain amount of value. So universities need to measure learning, and that is where assessment comes in. The real problem with plagiarism is that it breaks the link between what the student submits for assessment, what they have learned. If a student writes something in their own words, then that writing is good evidence of the extent to which they have taken in the ideas of a course. If they have just copied and pasted someone else's words, you cannot assess what they have learned. Also, teachers do not set assessment tasks just to measure students. We hope that performing the tasks helps the student to learn.
Jude also made the point that exactly is plagiarism does depend on the context. For example the level of experience of the student, the learning that is being assessed, what you are allowed to assume as common knowledge. Also, not all plagiarism is cheating, and not all cheating is plagiarism. Dealing with plagiarism must be done in an appropriate and proportionate way. To start with, understanding issues of academic honesty is an important learning outcome from a degree course, but it needs to be taught, it is not innate. It cannot be taught as an optional add-on ("the plagiarism lecture") but instead it must be embedded in the context of the subject early in a degree program. (By and large the OU does this in its introductory course.) When plagiarism happens, the response has to be proportionate. Early in a program of study, students need to be given feedback to help develop good academic practices. Later on, students should be expected to understand these issues, and plagiarism becomes a disciplinary issue, but still their need to be a range of fair and proportionate penalties.
The OU has clearly had a group of people thinking hard about this over the last few years, and that has resulted in a new plagiarism policy that was adopted last June. This conference was part of disseminating that policy more widely. There are now a range of penalties for various types of plagiarism, and a new role, 'Academic Conduct Officers' who can deal with most issues of plagiarism that go beyond what the student's individual tutors can deal with. This lets many cases be dealt with quickly, so that only the most serious cases have to be referred to the Central Disciplinary Committee.
Finally, the OU does use two technological means to detect plagiarism, CopyCatch and Turnitin, on a proportion of submitted work. However, this is only used to flag up possible cases that are then reviewed by a human. The third form of detection is that tutors marking the students' work can often spot problems. However, overall, the amount of plagiarism detected at the OU seems to be lower than that reported elsewhere. It is interesting to speculate why that might be.
Wednesday, October 14, 2009
What I'm working on
I've been meaning to post here for a while about what I am working on, but I keep posting in the Moodle Quiz Forum instead, because what I have to say seems more relevant there.
Anyway, the short answer is that I am rewriting the Moodle Question Engine. That document explains the what and the why. I want to post here about the how.
The question engine is the part of Moodle that processes what happens as a student attempts questions as part of a quiz (or other activity). What makes this a relatively difficult problem is that there are two independent degrees of variation. A question may be one of many different question types (multiple choice, short-answer, essay, ...) and there are various ways that a student may interact with it (adaptive mode, non-adaptive mode, manually graded, ...). Plus, people seem to think that when quizzes are used for summative assessments, it is quite important that there are no bugs and that performance is good ;-).
The complexity of the problem, and my natural bias, means that I am taking a very object-oriented approach. More so than is normally used in Moodle. Separate objects, with clearly defined responsibilities, helps me understand the problem and have confidence that what I am doing will work.
If you read the document linked to above, you will learn that I have been dissatisfied with this part of the quiz code for years, however it is really only within the last year that I have worked out the overall approach to solving this problem nicely. The key realisation is that you need to apply the strategy pattern twice, once for the bits that vary with the question type, and once for the bits that vary with the interaction model. The other important part of the plan is a cleaner database structure that more closely maps to data we need to store.
However, to implement all that requires quite a lot of code. I need to build the core system that can support all the different interactions and question types. Then I need to create all the interactions to replicate Moodle's existing functionality and the new features we want to add (Certainty Based Marking, and what we are calling the Interactive mode). Then I need to modify all the existing Moodle question types to fit into the new system. And I need to do all this with robust bug-free code that we can confidently use for summative assessment.
Over the last year I become increasing coverted to test-driven development. Certainly I wrote a lots of tests while eating my elephant, and this time round I am being very thorough with testing, although I must admit I still don't always write the tests first.
I am also proceeding very iteratively. I started with a test that took a single true-false question through being attempted according to the simplest of the interactions, and wrote some code that used roughly the right classes to make that test pass. Then for about a week I did little more than refactor that code.
It turned out that most of my variables and classes had the wrong name. For example, at one point I had classes called question_state and question_states, which was clearly a recipe for confusion. It was only after I had working code that I was able to think of the name question_attempt_step (one step in a question_attempt) for one of those. I also came to realise that grades were stored in three separate ways in different places, and so I ought to make sure I was using a consistent naming scheme for my variables. Then I did a bit of moving methods between classes as the exact boundaries of responsibility of different classes became clearer. Should I have been able to get all that right first time? Well I think that would be impossible.
I also started gradually removing simplifying assumptions that worked for my initial test true-false question, but not more generally. That was the point at which I started extending my code to other question types and interactions. For example I wrote a test case a walking an essay question through the manual grading interaction, and exposed some new issues that had to be accounted for; and just today I introduced multiple-choice questions, which randomise the order of the choices when you start an attempt, but then you have to store that random order somewhere so it can be used throughout the attempt. I have also already implemented the certainty based marking interaction.
There is still much to do. My code that outputs the HTML for a question is still a rough hack of the current code that lets the tests pass. I need several rounds of refactoring before that will be cleaned to my satisfaction. There are sill five more interactions to write, although each one is getting quicker than the one before. Then I have to convert all the existing question types. I need code that stores the current state of everything in the database and later reads it back. Finally, the nasty bits, which are upgrading from the current database structure to the new one, and re-implementing backup and restore, including restore of old backups.
If you were paying attention in that last paragraph, you will have deduced that my code does not yet store anything in the database! I am taking an approach with the domain objects completely unaware of the database (which makes testing very easy) and I am planning to use the data mapper pattern to coordinate loading and saving the necessary data efficiently. I am pretty sure I know how to do that, however, I have not tried to write it yet because I want to re-read the relevant chapters of Patterns of Enterprise Application Architecture, and my copy only recently got back from Australia, and I won't be able to retrieve it from my parents' house until this weekend.
In terms of time-scales, I am really hoping to get this done by about Christmas, and in time to go into Moodle 2.0, since it breaks a number of APIs. However, I first have to implement this in the OU's Moodle 1.9 code-base and then port it to Moodle 2.0, so no promises. This may have to wait until Moodle 2.1.
Anyway, that is what I have been doing, and so far it has been very enjoyable. Write tests, make them pass, then refactor aggressively with the tests as a safety net is an approach that is working really well for me. These are quite big scary changes, but right now I am feeling confident that it will all work out, and we will end up with a really solid question engine upon which to base an enhanced quiz and other Moodle activities.
Anyway, the short answer is that I am rewriting the Moodle Question Engine. That document explains the what and the why. I want to post here about the how.
The question engine is the part of Moodle that processes what happens as a student attempts questions as part of a quiz (or other activity). What makes this a relatively difficult problem is that there are two independent degrees of variation. A question may be one of many different question types (multiple choice, short-answer, essay, ...) and there are various ways that a student may interact with it (adaptive mode, non-adaptive mode, manually graded, ...). Plus, people seem to think that when quizzes are used for summative assessments, it is quite important that there are no bugs and that performance is good ;-).
The complexity of the problem, and my natural bias, means that I am taking a very object-oriented approach. More so than is normally used in Moodle. Separate objects, with clearly defined responsibilities, helps me understand the problem and have confidence that what I am doing will work.
If you read the document linked to above, you will learn that I have been dissatisfied with this part of the quiz code for years, however it is really only within the last year that I have worked out the overall approach to solving this problem nicely. The key realisation is that you need to apply the strategy pattern twice, once for the bits that vary with the question type, and once for the bits that vary with the interaction model. The other important part of the plan is a cleaner database structure that more closely maps to data we need to store.
However, to implement all that requires quite a lot of code. I need to build the core system that can support all the different interactions and question types. Then I need to create all the interactions to replicate Moodle's existing functionality and the new features we want to add (Certainty Based Marking, and what we are calling the Interactive mode). Then I need to modify all the existing Moodle question types to fit into the new system. And I need to do all this with robust bug-free code that we can confidently use for summative assessment.
Over the last year I become increasing coverted to test-driven development. Certainly I wrote a lots of tests while eating my elephant, and this time round I am being very thorough with testing, although I must admit I still don't always write the tests first.
I am also proceeding very iteratively. I started with a test that took a single true-false question through being attempted according to the simplest of the interactions, and wrote some code that used roughly the right classes to make that test pass. Then for about a week I did little more than refactor that code.
It turned out that most of my variables and classes had the wrong name. For example, at one point I had classes called question_state and question_states, which was clearly a recipe for confusion. It was only after I had working code that I was able to think of the name question_attempt_step (one step in a question_attempt) for one of those. I also came to realise that grades were stored in three separate ways in different places, and so I ought to make sure I was using a consistent naming scheme for my variables. Then I did a bit of moving methods between classes as the exact boundaries of responsibility of different classes became clearer. Should I have been able to get all that right first time? Well I think that would be impossible.
I also started gradually removing simplifying assumptions that worked for my initial test true-false question, but not more generally. That was the point at which I started extending my code to other question types and interactions. For example I wrote a test case a walking an essay question through the manual grading interaction, and exposed some new issues that had to be accounted for; and just today I introduced multiple-choice questions, which randomise the order of the choices when you start an attempt, but then you have to store that random order somewhere so it can be used throughout the attempt. I have also already implemented the certainty based marking interaction.
There is still much to do. My code that outputs the HTML for a question is still a rough hack of the current code that lets the tests pass. I need several rounds of refactoring before that will be cleaned to my satisfaction. There are sill five more interactions to write, although each one is getting quicker than the one before. Then I have to convert all the existing question types. I need code that stores the current state of everything in the database and later reads it back. Finally, the nasty bits, which are upgrading from the current database structure to the new one, and re-implementing backup and restore, including restore of old backups.
If you were paying attention in that last paragraph, you will have deduced that my code does not yet store anything in the database! I am taking an approach with the domain objects completely unaware of the database (which makes testing very easy) and I am planning to use the data mapper pattern to coordinate loading and saving the necessary data efficiently. I am pretty sure I know how to do that, however, I have not tried to write it yet because I want to re-read the relevant chapters of Patterns of Enterprise Application Architecture, and my copy only recently got back from Australia, and I won't be able to retrieve it from my parents' house until this weekend.
In terms of time-scales, I am really hoping to get this done by about Christmas, and in time to go into Moodle 2.0, since it breaks a number of APIs. However, I first have to implement this in the OU's Moodle 1.9 code-base and then port it to Moodle 2.0, so no promises. This may have to wait until Moodle 2.1.
Anyway, that is what I have been doing, and so far it has been very enjoyable. Write tests, make them pass, then refactor aggressively with the tests as a safety net is an approach that is working really well for me. These are quite big scary changes, but right now I am feeling confident that it will all work out, and we will end up with a really solid question engine upon which to base an enhanced quiz and other Moodle activities.
Subscribe to:
Posts (Atom)