Tim's blog

What I want to build next

2011-09-27T00:10:00.000+01:00

Earlier this summer I finally finished the new Moodle question engine, which was released as part of Moodle 2.1. As you might expect with such a large change, a number of minor bugs were not spotted until after the release, but I (and others) have fixed quite a lot of them, and we will continue to fix more. I want to say "thank you" to everyone who has taken the time to report the problems they encountered. Pleasingly, some people, including Henning Bostelmann, Tony Levi, Pierre Pichet, Jamie Pratt, Joseph Rézeau and Jean-Michel Vedrine have not only been sending in bug reports, but also submitting bug fixes. I would like to thank them in particular. I don't know whether this means that the new Moodle development processes are working well and encouraging more contributors, or that I released the new question engine full of trivial bugs.

At the moment, apart from fixing bugs, we are about two months away from the end of the OU's one-year project to move from Moodle 1.9 to 2.x and implement a lot of new features at the same time. In the eAssessment area, we had about 30 work-packages to do, of which finishing the question engine was by far the biggest, and we have about 6 left to go. Most of the remaining tasks are at least started, but finishing them is what I, and the developers on my team, will be doing in the near future.

I have, however, been thinking ahead a bit, and I have an idea for what I would like to build, should I be given the opportunity. Honesty compels me to say these are not my ideas. I stole them from other people, and there are proper acknowledgements at the end of this post. I wanted to post about this because: 1. in my experience, if you post about your half-baked ideas, people will be able to suggest ways to make them better; and 2. I am hoping that at least one course-team at the OU will see this and say "we would love to use this in our teaching" because that might persuade the powers that be to let me build this.

Rationale

The Moodle quiz is a highly structured, teacher-controlled tool for building activities where students attempt questions. What I want to create is a more open activity where students can take charge of their learning using a bank of questions to practice some skill where the computer can mark their efforts and give feedback. For the sake of argument, I have been calling this the "Question practice" activity module.

The entry page

When a student goes into a Question practice activity, they see a front screen that lists all the categories in the question bank for this activity.

Next to each category, there are statistics for how the student has performed on that category so far. For example, it might say "recently you scored 19/21 (90%); all time you scored 66/77 (86%).” The categories are nested, and there is a subtotal for each category.

At the bottom of the page is an Attempt some questions… button. This takes the student to the …

Start a session form

… where they set up what practice they would like to do. Students can select which categories they want to attempt questions from. They may also be able to choose how many questions they want. For example "Give me 10 questions", "As many as possible in 20 minutes", or "Keep going until I say stop". The teacher will probably be able to constrain the range of options available here.

Once they are satisfied, the they clicks the "Start session" button. This takes them to the …

Attempt page

… which shows the student the first question, chosen according to the criteria they set. There will probably be a display of running statistics "In this session you have attempted 0 questions so far". The question will contain the controls necessary for attempting the question. There will also probably be a "Please stop, I'm bored" button, so the student can leave at any time.

When they get back to the front page, the statistics will have been updated.

If the student crashes out of a session, then when they go back in, the front page will have a "Continue current session" button.

Overall activity log

One batch of attempting questions will be called a 'practice session'. The system will keep track of all the sessions that the student has done, and what they achieved during each session.

The front page will have a link to a page that lists all of the student's sessions, showing what they achieved in each. This provides more detail than is visible on the front page.

Possible extensions

That is the key idea. Here are some further things that could be added to the basic concept.

Milestones

The system could recognise targets, goal, or achievement (I'm not sure of the best name). That would be something like "Attempt more than 10 questions from the Hard category, and score more than 90%". If the student achieves that target at any time, they system would notice, and the achievement would be recorded on the front page and in the session log in an ego-boosting way (e.g. a medal icon).

The whole point of this activity is to be as student-driven as possible, so should students be able to define their own targets or goals? Should students be able to set goals for each other?

Locks / Conditional access

The activity could also have locks, so that the student cannot access the questions in the Multiplication category until after they have scored more than 80% in the Hard addition category. Of course, unlocking a new category could be an achievement. We appear to be flirting with the gamification buzz-word here, so I will stop.

Performance comparison

Should there by any way for students to compare their performance, or achievements, with their peers? We are definitely getting to features that should be left until version 2.0. Let's get a basic system working first, but make sure it is extensible.

How hard would this be to build

I think this would not require too much work because a lot of the necessary building blocks already exist in Moodle. The question bank already handles questions organised into categories, and we would just use that. Similarly, the attempt page and practice sessions are very easy to manage with the new question engine.

The real work is in two places. First, building the start attempt form, and then writing the code that randomly selects questions based on the options chosen. Second, deciding what statistics to compute, and then writing the code to compute them.

Of course, before we can start writing any code, there are still a lot of details of the design to decide. Also one most not forget things like backup and restore, creating the database, and all the usual Moodle plumbing.

Overall, I think it would take a few months work to get a really useful activity built.

Credit where credit is due

I said earlier that I got most of these ideas from other people. To start with, things like this have been mooted in the Moodle quiz forum over the years. The discussions there usually start from Computerised Adaptive Testing, whereas this idea is about student-driven use of questions. I think the latter is more interesting. (As a mathematician, I think CAT is an interesting concept. I just don't think it would make a useful Moodle activity.)

The real inspiration for this came at a meeting in London at the start of 2011. That meeting was at UCL with Tony Gardiner-Medwin who has already built a system something like this, but stand-alone, not in Moodle; and David Emmett from University of Queensland, Brisbane (who was giving a seminar). David had been hoping to get a grant to build something like this proposal (in Moodle) but that did not pan out. We did, however, have a very interesting discussion, and that is where I got the key idea that this sort of question practice was most interesting if you could give the student control of their own learning as much as possible.

We have also discussed ideas like this on-and-off for a long time at the OU. There has, however, been a lot of other things we needed to deal with first. We had to do a lot of work getting the quiz system working to our satisfaction (a strand of work that eventually lead to the new question engine). We had to sort out the reporting of grades, including working with Moodle HQ on the new gradebook in Moodle 1.9, and integrating Moodle with our student information system. We had to make a new question types that our users wanted. Only now can we start to think seriously about the last piece of the jigsaw: more activities that use all the question infrastructure we have built. I hope this post is a useful starting point for discussing what one of those activities might be.

The Good, the Bad and the Ugly

2011-08-06T14:37:00.001+01:00

It was the best of times, it was the worst of times, ... It has certainly been a mixed week.

The good

... was that I helped three OU developers submit their first bug fix through the new Moodle development process: MDL-27631, MDL-28517 and MDL-28620. Hopefully those fixes all get through integration review next week.

The Bad

... was that the time had finally come to deal with a hot potato that we have been tossing around for some months; and, to mix metaphors, when the buck stopped, I was in the the wrong place at the wrong time.

As part of some new question types we are developing, we want students to be able to type responses that include superscripts and subscripts. For example 3×10⁸ ms^-1 or SO₄^2-. We have an old implementation of this, done six years ago for OpenMark (for example this or this), but that never worked in Safari, and is a bit dodgy generally. We want a new, reliable implementation that works in IE, Firefox, Chrome and Safari.

Plan A: back near the start of spring, I quickly knocked up a partial solution using the YUI 2 Rich Text Editor library. It mostly worked, but there were issues. It did not work consistently across browsers, and it lets you nest superscripts inside subscripts inside superscripts which just gets confusing, so we want to prevent that.

I had a sneaking suspicion how hard it would be to get from my quick partial solution to a robust implementation. Therefore I moved on to other things, and tried to unload this job onto three other people in turn. There were plenty of other more urgent tasks on our todo list.

Time passed, and many of the other things got done, so at the start of the week I realised that creating this input widget could not be put off any longer. I also felt it was unfair to expect other developers to deal with a crappy job that I was not prepared to do myself, so I decided to have another go.

The other thing that had changed is that while attempting to implement this, Colin and Wale had both eliminated some blind alleys, and suggested some promising ideas. Therefore, I was continuing from a far better place than where I left off. Even so, it was a long week.

The Ugly

Plan B: Although we had a partial implementation in YUI 2, I did not want to continue with that. Moodle is trying to move away from YUI 2 and to YUI 3 as soon as possible. So, my first attempt was to use the YUI 3(.3) Rich Text Editor. As the docs make clear. That is not finished yet. It is also not terribly well documented. With hindsight, I now realise that it provides only a very thin rapper around the native editing facilities provided by web browsers. Therefore it does not really help with browser inconsistencies.

Plan C: Since the Rich Text Editor is only beta, I decided to have a look at what was new in YUI 3.4, which is due for release soon. The answer is that they have made quite a lot of progress - in the sense that if you are trying to walk from London to Edinburgh, you have made quite at lot of progress by the time you reach Milton Keynes. Compounded with the fact that it is hard to find any documentation for pre-release version of YUI, this approach also failed.

At this point, I decided to do a bit of reading. I found two excellent articles from Opera that explained exactly how contentEditable works in web browsers. I also found a good cross-browser compatibility table. That made me realise that YUI was hardly doing anything to help with cross-browser differences.

Plan D: Now that I knew roughly what the browsers were doing, I briefly toyed with the idea of implementing the widget entirely myself in plain JavaScript. Once again, getting something basic working was not too hard, but I had not even started to tackle the cross-browser differences.

Then I realised that TinyMCE, which Moodle uses, tends to work really well across browsers. It is a bit slow to load, because it is a huge mass of code, but perhaps all that code is there for a reason. A quick play with superscript and subscript in the Moodle HTML editor in various browsers confirmed that TinyMCE must be working around most of the problems. So I dived into the TinyMCE code with the original idea of stealing the bits I needed to make Plan D work. It did not take much looking for me to develop a new-found respect for how hard TinyMCE is working to keep different web browsers in line. I did not want to have to replicate all that.

Plan E: So I finally concluded I should just use TinyMCE directly. That is using a very large sledge-hammer to crack a nut, but at least it should work. Indeed, it was mostly just a matter of setting the right configuration options. What made it particularly good is that there is an option to limit which tags can be nested inside other tags. That robustly prevents people from nesting superscript inside subscript, etc.

I was very nearly there, but there were two more requirements. We did not want pressing enter to insert a line-break, and because we were only dealing with a single line of input, we wanted to use the up and down arrow keys as shortcuts for superscript and subscript. The only way I could find to do that was to write a simple TinyMCE plugin. Fortunately, that is well documented.

The end

I got there eventually. The code needs to be cleaned up, tested some more, and integrated into the question types we are building, but I don't foresee any problems doing that.

I would like to thank the Moxie Code, who make TinyMCE, even though they have completely ignored the patch I sent them some time ago in relation to MDL-27890. I would also like to thank Olav Junker Kjær, who wrote the Oracle blog posts, which were the most useful thing I read. Also, the team behind Firebug. I can't imagine doing JavaScript development without that debugging tool. Finally Colin, Wale and Jamie, who I tried to dump this on, and who in return gave me helpful ideas.

Keeping the discipline of not changing Moodle core

2011-07-07T00:31:00.000+01:00

We have said in the past that at the OU we made too many changes to core code in our Moodle 1.9 system, and that as we moved to Moodle 2, we would make far fewer. The release of Moodle 2.1 provides a good opportunity to stop and reflect on how we are doing.

Exactly how many core changes we had made in 1.9 seems to depend on who you ask. It was something of the order of one or two thousand depending on how you count. As a result, every time there is a new Moodle 1.9.x release, someone (Derek) has to do a couple of days painstaking merging to upgrade to the new version.

Moodle 2.1 was released on Friday. On Monday afternoon we decided to try upgrading our development branch to it. The merge (literally git merge MOODLE_21_STABLE) only took a few hours, and that was most mostly a matter of thinking before typing git checkout --theirs to resolve most of the conflicts in favour of the Moodle 2.1 code. Then we had to test test install, upgrade, and basic functionality before pushing the merge to our central repository.

But, how many OU-specific changes do we have in core code right now? Well, the answer appears to be eight. Let me explain that number.

To control the core code changes, we use a simple approval procedure. Each change must be proposed by one of the leading developers. They do this by opening a special sort of ticket in our issue tracking system. That serves as a permanent record of the change, and is also a place to log any discussion. The other leading developers then review the proposal. For the change to be approved, at least one other leading developer must endorse it with a +1 vote, and there must not be any -1 votes. Votes are normally accompanied by an explanation of why that developer is voting that way.

After a suitable time for votes, the issue is declared either accepted or rejected. OU-specific changes can be rejected for two reasons.We may decide that it is not acceptable to change core to implement this feature, so we drop the feature; or we think of some devious way to achieve the feature without changing core code.

If a change is approved, then the code is written. Well, in some cases the code will already have been written, because you can have a much more informed debate about whether a certain change is a good idea if you can see exactly what the proposed change is. Once the code is written and approved, it is committed to out git repository and the issue moves into state 'Code committed'. Finally, we may find a way to get rid of the OU-specific change in future. The most common way that happens is if we contribute the change upstream to moodle.org. For example the new Moodle question engine was an ou-specific change as long as we were using it in Moodle 2.0, but now we have upgraded to Moodle 2.1, it is standard code. Therefore, that issue has now changed status to 'No longer required'.

Overall, our we, have 22 ou-specific change issues in our bug tracker. The break-down is:

Rejected: 2
New (under discussion): 4
Approved (but not yet implemented): 1
Code committed: 8
No longer required: 7

Most of the 'Code committed' changes are pretty boring. For example three of them are bug-fixes to the questionnaire module that we have submitted upstream, but which have not been reviewed and accepted by the questionnaire maintainers yet. Therefore, those three will almost certainly end up as 'No longer required' in due course. Another example is that we want to customise the "Database connection failed / It is possible that the database is overloaded or otherwise not running properly" page that you get when Moodle fails to connect to the database. If Moodle can't connect to the database, then it cannot load the configuration, and so cannot determine which theme to use to display the error. Therefore, the only way customise that page is to edit lib/setup.php.

The one 'serious' ou-specific change we have is some hacking around in course/lib.php to support one of our custom modules called 'subpage' (not released yet, but we hope to share it eventually). Given more time, we might be able to find a more elegant way to handle these changes, but we don't have that sort of time at the moment.

While we have controlled the core code changes, we do have written a lot of custom plugins. Those range from big things like forumng and ouwiki, to small things like a local plugin that just implements a single web-service function. I'm afraid I don't have a complete list, but we must have more than 50 plugins by now. As far as I am aware, the upgrade from 2.0 to 2.1 did not break any of them.

Performance and Scalability

2011-04-12T09:25:00.001+01:00

When you set up a web application, often you will start small with everything running on one server. Everything, in this case, typically means the application and the database and data files. That is nice and simple. It has the advantage that everything it fast because it is all one the one server.

The capacity is, however, limited. Suppose the load on your application increases. You can get some way just by upgrading the one server, adding more memory and faster processors, but that will only get you so far.

Eventually, you will have to scale out. You will get a number of separate web servers, with a load-balancer to distribute the incoming requests between them. All the web servers will connect to a shared database sever, or cluster of database servers. The files will probably go on a separate file server.

While this increases the total load that the whole system can support, it means, paradoxically, that processing a single request is slower. For example, if you switch from one server to three servers (application server, database and files) your site will not support three times as many users. The scalability will not be linear. That is because every connection to the database or to get a file now has to travel over the network. Accessing something across a network tends to be an order of magnitude slower than accessing something on the same server.

The above is all standard knowledge about scaling web applications. I have been thinking about about it recently because it explains the way my working life has been evolving. Just over six months ago I was working essentially on my own, re-developing the Moodle question engine. I had been working away like that for a year, and I had got a lot done.

Since then, things have changed, and I am now managing a team including three other developers, and two out-sourced development contracts. It has been particularly 'bad' this last couple of weeks as one development period of the project came to an end and I had to review a lot of code, and then I had to sort out everything we were supposed to be doing for the next three months. I am starting to wonder if I will every get any of my own development work done at all!

That is, however, just some exasperation showing. I know really that this has just been a brief spell with an excessive amount of administration. Overall I am happy that the OU is investing so much in its eAssessment systems (and the other parts of its VLE); as a team we are achieving more than I could on my own; but right now my inner geek would really like to go and hide in a cave for a while and just write code undisturbed.

Moodle bug tracker

2011-03-09T23:26:00.000Z

Today, between fixing bugs and reviewing code, I spent a bit of time tinkering with my dashboard in the Moodle bug tracker. I was trying to make it as clear as possible which issues need my attention. I am quite pleased with the result:

The issue statistics widget does not just show you the pretty graphs, it also makes it easy to get at those issues. For example, if I click on 1.9.12 in the My targetted issues box, then I am taken to a list of those 11 issues. That particular widget I have used for a while, the new parts are the boxes just under there.

My: Ongoing pull requests I added to make it easy to find the things I have submitted for inclusion in next week's weekly build (hopefully). Thanks to Eloy, that filter is now available to everyone in the jira-developers group.

The next two boxes let me quickly get to issues with patches attached. There is an emerging convention of adding the label patch to such issues, where the attached code needs to be reviewed. This makes finding such issues very much easier. The whole point of the new development processes is to encourage more people to contribute patches, and then ensure those patches get looked at, rather than just sitting there for years. (Here is an example I found yesterday of what used to happen: MDL-13983). Therefore, as quiz maintainer, I need to be able to see easily if anyone has submitted any relevant patches. I also want easy access to bugs with patches that I created or commented on.

Having brought it up, can I say that I am quite happy with how the new processes are working so far. My impression is that since they were introduced, I have received more usable bug fixes for the quiz that in the past. I am not sure how much causality one can claim there, however, since as well as the new processes, we also had the Moodle 2.0 release. Moodle 2.0 has plenty of minor bugs that are ripe for fixing. So, it may just be that we are seeing lots of bug fixes because there are lots of bugs.

At the other end, it has made it a bit easier to get my code reviewed. Well, finished code where I have created a PULL request certainly gets is reviewed. It is still sometimes a problem to get comments on work-in-progress, because everyone is so busy.

Etiquette for questions

2011-02-23T01:31:00.001Z

I have been working hard at converting the new Moodle question engine to work in Moodle 2.0, aiming at a deadline this Friday (25th February). On Friday we should have a first OU version of Moodle 2.0 with all the key features so that we can start testing, even though students won't get onto the new system before July. I have have basically finished the question engine, give or take a few features that are not needed for testing, and this week I am just doing some final tidying up of the code.

Hopefully, next week I can start the process of getting it reviewed for inclusion in Moodle 2.1. As I say, there are some gaps in the functionality that will need to be filled in before it can actually be committed, but there is a lot of code to be reviewed (lucky Eloy!) and so I hope we can kick off the process.

So, my excuse for not blogging about the new question engine recently is that I have been too busy working on it to write about it. In the last few days, however, I encountered a couple of nice ideas that would be easy to implement using the flexibility the new question engine gives, and I want to describe them. First, I need to remind you of one key point about the new system:

Question behaviours

As I explained before a key idea in the new question engine is that of question behaviours. Whereas a question type lets you have either a multiple-choice, a drag and drop, or a short-answer question, a behaviour controls how the student interact with the questions, of whatever type. For example, the student may have to type in answers to each question in the quiz, then submit everything, and only then are the questions marked. This is known as the "Deferred feedback" behaviour. Alternatively, the student may answer one question, have their answer marked immediately. If they are wrong, they get a hint and can then immediately have another go. If they get it right on the second or third try, they get fewer marks. This is called the "Interactive with multiple tries" behaviour.

When I was first working on this, I did wonder whether it was perhaps over-kill to make behaviours fully-fledged Moodle plugins. It seemed to me that I had already implemented all the types of behaviour anyone was likely to want. It turns out I was wrong. Here are three ideas for new behaviours have I have come across since had that naive thought.

Explain your thinking behaviour

The concept here is that, in addition to presenting the question to the student for them to answer in the usual way, you also give them a text-area with the prompt "Explain your answer". When the submit the question is graded as usual. Moodle does not do anything with the explanation, other than to store it, and re-display it later when the student or their teacher reviews their attempt. The point is that the student should reflect upon and articulate their thought processes, and the teacher can then see what they wrote, which might be useful for diagnosing what problems the students are having.

I'm not sure that this would really work. Would the students really bother to write thoughtful comments if there were no marks to be had? However, this would be relatively easy to implement, so we should build it and see what happens in practice. The teacher could always manually adjust the marks based on the quality of the reflection, if that was necessary to incentivise students.

I'm afraid I cannot remember who suggested this idea. It was a post in the Moodle quiz forum some time ago, just after I had implemented the behaviour concept and was thinking that my initial list of behaviours was all anyone could possibly want.

gnikram desab-ytniatreC

This idea I only came across yesterday evening, in a blog post from people in the OU's technology faculty. It is a slightly strange twist on certainty-based marking.

With classic CBM, the students answers the questions, and also says how certain they are they got it right (for example, on a three-point scale). The student will only get full marks if they get the question right, and are very certain that they were right. If, however, they express high certainty and are wrong, they are penalised heavily with a big negative mark. To maximise their score, the student must accurately gauge their level of knowledge. This hopefully promotes reflection, and self awareness by the student of their level of knowledge.

The idea from the OU technology faculty is to do this backwards, for multiple choice questions. Rather than getting the student to answer the question and then select a certainty, you first show them just the question stem without the choices, and get them to express a certainty. Only then do you show them the choices and let them chose what they think is the right answer.

Again, I am not sure if this would work, but it is sufficiently easy to do by creating a new behaviour plug-ing (and a some change to the multiple-choice question type so that you can output just the question, without the choices) that it has to be worth a try.

Free text responses with a chance to act on feedback

This last idea I only heard about this morning. There was a session of the OU's "eLearning community" all about eAssessment, which naturally I attended. This is a monthly gathering with a number of different presentations on some eLearning topic. The first three talks were about specific courses that have recently adopted eAssessment, and how students had engaged with that, what effect the effect had been on retention and pass rates, and so on. That was interesting, but not what I want to talk about here. The final talk was by Denise Whitelock from the OU's Institute of Educational Technology who has just completed a review of recent research into technology-enhance assessment for HEA that should be published soon. Here, I just want to pick up on one specific idea from her talk.

I'm afraid that again, I don't recall who deserves credit for this idea. (Once Denise's review is published, I will have a proper reference, but I did not take notes this morning.) It was another UK university that had done this. It was in the context of language teaching. The student had to type a sentence in answer to the question, then the computer graded that attempt and gave some feedback. Then, the student was immediately allowed to revise their sentence in light of the feedback, and get it re-marked. The final grade for the question is then a weighed sum of the first mark and the second mark. You need to get the weights right. The weight for the first try has to be big enough that the student tries hard to get the question right on their own before seeing the hints, and the weight for the second try, though smaller, also has to be big enough so that the student bothers to revise their response.

Now, the OU is currently creating a Moodle question type that can automatically grade sentence length answers using an algorithm that my colleague Phil Butcher implemented the first version of in 1978! (When I say we are creating this, what I actually mean is that we have contracted Jamie Pratt a free-lance Moodle developer to implement it to our specification.) Anyway, once you have that, the idea of allowing two tries, with feedback after the first try, and a final grade that is a weighted sum of the marks for the two tries, is just another behaviour.

So, my initial thought that people would not have many ideas for interesting new behaviours seems to have been wrong. The flexibility I built into the system is worth having.

Should you listen to futurologists?

2011-02-09T14:00:00.003Z

Educause just published their annual survey describing "six areas of emerging technology that will have significant impact on higher education and creative expression over the next one to five years".

This got circulated round the team at work and I rather cynically asked "so, what did they predict last year then?" My colleague Pete Mitton took that question and ran with it to produce the following analysis:

OK, as I have a full set of Horizon Reports on my hard disk, here's a summary of their predictions for the years 2004-11.

I've pushed some titles together where the wording is different but the intent is the same (for example they've used mobile computing/mobiles/mobile phones in the past with the same meaning).

The numbers in the table are the time-to-adoption horizon in years.

	2004	2005	2006	2007	2008	2009	2010	2011
User-created content			1	1	1
Social Networking		4-5	1	1
Mobiles			2-3	2-3		1	1	1
Virtual Worlds				2-3
New Scholarship and Emerging forms of publication				4-5
Massively Multiplayer Educational Gaming				4-5
Collaboration Webs					1
Mobile Broadband					2-3
Data Mashups					2-3
Collective Intelligence					4-5
Social Operating Systems					4-5
Cloud Computing						1
The Personal Web						2-3
Semantic-Aware Applications						4-5
Smart Objects						4-5
Open Content							1
Electronic Books							2-3	1
Simple Augmented Reality		4-5	4-5				2-3	2-3
Gesture-based computing							4-5	4-5
Visual Data Analysis							4-5
Game-based learning		2-3	2-3					2-3
Learning analytics								4-5
Learning Objects	1
Scaleable Vector Graphics	1
Rapid Prototyping	2-3
Multimodal Interfaces	2-3
Context Aware Computing aka Geostuff	4-5		4-5			2-3
Knowledge Webs	4-5
Extended Learning		1
Ubiquitous Wireless		1
Intelligent Searching		2-3

Of course, the purpose of a report like this is not to accurately predict the future. The aim is rather to stimulate informed debate about the technologies that are coming up. Within our team, at least, they seem to have succeeded.

I thought, however, that this analysis was interesting enough to share. It provides some context for year's predictions. More generally it shows how difficult it is predict future technology trends.

How (not) to sell your open source development services

2010-11-02T17:39:00.001Z

It must be terrible trying to sell software development services if you work in Open Source. The restrictions of the GPL appear to make it almost impossible to demonstrate your previous work to prospective clients. I would like to offer the following advice, which is my attempt to distil what appears to be current best practice in dealing with this tricky situation.

Do not make it possible for your potential client to find any examples of code you have written. If you cannot avoid it completely, make it as difficult for them as possible. For example, if you have created a certain sort of plugin, just say "We have created a new Moodle block." Do not give any clues as to which block that might be. If the open source project provides a convenient place to upload and share your contributions, try to avoid making any code you have produced easy to find there. Even if you have been forced to share your code in these places, do not on any account provide your potential client with a link to the example of work that you are most proud of. It is fairer to let them search and find a representative sample of your work, if they are clever and patient enough to do so.

Your development staff are, or course, all faceless drones who are nothing to be proud of. Your potential client does not care who will actually be doing the work they are requesting. This is particularly important when the open source project has a strong community. It must not be possible to identify your staff in the recognised list of project contributors; nor should it be easy to discover how they have contributed to the on-going development of the project's code by looking in the project's issue tracker.

It is important is to dress your proposal up in meaningless marketing-speak. If your client cannot complete their buzzword bingo card while reading your document, they just won't hire you. Including a PowerPoint presentation with impressive diagrams and more platitudes about your company can create a particularly strong impression.

The new question engine - how it works

2010-10-20T23:31:00.000+01:00

In my last blog post I promised more details of the new question engine "in a week or so". Unfortunately, things like work (mainly fixing minor bugs in the aforementioned question engine); rehearsing for a rather good concert that will take place this Friday; and buying curtains for my new flat, have been rather getting in the way. Now is the time to remedy that.

Last time I explained roughly what a question engine was, and that I had made big changes to the one in Moodle. I now want to say more about what the question engine has to do, and how the new one does it.

The key processes

There are three key code-paths:

To display a page of the quiz:

Load the outline data about the student's attempt.
Hence work out which questions are on this page.
Load the details of this student's attempt at those questions within this quiz attempt.
Get the display settings to use (should the marks, feedback, and so on be visible to this user).
Taking note of the state of each question (from Step 3) update the display options. For example, if the student has not answered the question yet, we don't want to display the feedback now, even if we will later.
Using the details of the current state of each question, and the relevant display options, output the HTML for the question.

The bits in italic are the bits done by the quiz. The rest is done by the question engine.

To start a new attempt at the quiz:

From the list of questions in the quiz, work out the layout for this attempt. (This is only really interesting if you are shuffling the order of the questions, or selecting questions randomly.)
Create an initial state for each question in the quiz attempt. (This is when things like the order of multiple choice options are randomised.)
Write the initial states to the database.

To process a student's responses to a page of the quiz.

From the submitted data, work out which attempt and questions are affected.
Load the details of the current state of those question attempts.
Sort all the submitted data, into the bits belonging to each question. (The bits of data have names like 'q188:1_answer'. The prefix before the '_' identifies this as data belonging to the first question in attempt 188, and the bit after the '_' identifies this as the answer to that question.)
For each question, process its data to work out whether the state has changed and, if so, what the new state is. This is really the most important procedure, and I will talk more about it in the next section.
Write any updated states to the database.
Update the overall score for the attempt, if appropriate, and store it in the database.

These outlines are, of course, oversimplifications. If you really want to know what happens, you will have to read the code.

There are other processes which I will not cover in detail. These include finishing a quiz attempt, re-grading an attempt, fetching data in bulk for the quiz reports, and deleting attempts.

The most important procedure

This is the step where we take the data submitted by the student for one particular question and use it to update the state of that question.

Moodle has long had the concept of different questions types. It can handle short-answer questions, multiple-choices questions, matching questions, and so on. Naturally, what happens when updating the state of the question depends on the question type. That is true for both the old and new code.

Now, however, there is a new concept in addition to question types. The concept of 'question behaviours'.

In previous versions of Moodle, there was a rather cryptic setting for the quiz: Adaptive mode, Yes/No. That affected what happened as the student attempted the quiz. When adaptive mode was off, the student would go through the quiz entering their response to each question. Those responses were saved. At the end, they would submit the quiz and everything would be marked, all at once. Then the student could review their attempt (either immediately, or later, depending on the quiz settings) to see their marks and the feedback. When adaptive mode was on, the student could submit each question individually during the attempt. If they were right first time, they got full marks. If they were wrong, the got some feedback and could try again for reduced marks.

The problem with the previous version of Moodle was the way this was implemented. There was a single process_responses function that was full of code like "if adaptive mode, do this, else do that". It was a real tangle. It was very common to change the code to fix a bug in adaptive mode (for example), only to find that you had broken non-adaptive mode. Another problem was the essay question type, which has to be graded manually by the teacher. It did not really follow either adaptive or non-adaptive mode, but was still processed by the same code. That lead to bugs.

A very important realisations in the design of the new question engine was identifying this concept of a 'question behaviour' as something that could be isolated. There is now a behaviour called 'Deferred feedback' that works like the old non-adaptive mode; there is an 'Adaptive' behaviour; and there is a 'Manually graded' behaviour specially for the essay question type. Since these are now separate, you can alter one without risking breaking the others. Of course, the separate behaviours still share common functions like 'save responses' or 'grade responses'. We now also have a clean way to add new behaviours. I made a 'certainly-based marking' behaviour, and a behaviour called 'Interactive', which is a bit like the old Adaptive mode but modified to work exactly how the Open University wants.

It takes two to tango, and three to process a question

In order to do anything, there now has to be a three-way dance between the core of the question engine, the behaviour and the question type. Does this just replace the old tangle with a new tangle (of feet)? Fortunately there is a consistent logic. The request arrives at the question engine. The question engine inspects it, and passes it on to the appropriate behaviour. The behaviour inspects it in more detail, to work out exactly what need to be done. For example is the student just saving a response, or are they submitting something for grading. The behaviour then asks the question type to do that specific thing. All this leads to a new state that is passed back to the question engine.

So, the flow of control is question engine -> behaviour -> question type except, critically, in one place. When we start a new attempt, we have to choose which behaviour to use for each question. At this point the question engine directly asks the question type to decide. Normally, the question type will just say "use whatever behaviour the quiz settings ask for", but certain question types, like the essay, can instead say "I don't care about the quiz settings, I demand the manual grading behaviour."

If you like software design patterns, you can think of this as a double application of the strategy pattern. The question engine uses a behaviour strategy, which uses a question type strategy (with the subtlety that the choice of behaviour strategy is made by the question type).

Summary

So that is roughly how it works. A clear separation of responsibility between three separate components. Each component focussing on doing one aspect of the processing accurately, which makes the system highly extensible, robust and maintainable. Of course, everyone says that about the software they design, but based on my experiences over the last year, first of building all the parts of the system, and then of fixing the bugs that were found during testing, I say it with some confidence.

Introducing the new Moodle question engine

2010-10-05T23:24:00.003+01:00

The rest of the (Moodle) world is eagerly anticipating Moodle 2.0, but I would like to tell you about what I have been doing for most of the last year, but which you won't be able to have until Moodle 2.1 - unless, that is, you are a student or teacher with the Open University, in which case you will be using it from this December.

What I have done is to rewrite a large chunk of the Moodle quiz system. What chunk is that? Well, first you can split a quiz system into two main parts. There is the quiz part, which says, "This quiz comprises these questions, and will be open to students between these dates". It tracks the student and they attempt the quiz, and stores their total score. Then there is the part that deals with the details of the individual questions within each quiz.

The question part can again be split in two. There is the question bank which lets the teacher create and store questions. For example "This is a multiple choice question where the student must select one of these three options, and it is an 'Elementary maths' question." Then there is the code that controls what happens when a student attempts a question "The student sees three radio buttons and a Submit button, and when they click the button we compute a score as follows and show this feedback." That second bit is what I call the question engine, and that is what I have rewritten.

However, you cannot just change the question engine in isolation. There are knock-on effects. For example, the quiz module still maintains overall control of things, even though it delegates a lot of the details to the question engine. So there are places where the quiz says things like "Dear question engine, please display this question now", or "Dear question engine, the student submitted this data, please process it", or "Dear question engine, the teacher wants to see all students responses to all questions in this quiz, give me the data to display." All those places have to change when the question engine changes.

There were also small changes required to the question bank. mainly because the new question engine has some new features that need some extra options stored with each question. So, the question bank needs to store the new options; let teachers edit them; back them up and restore them; import them and export them; and so on.

Altogether, my year's work added about 52,000 lines of new code and removed about 25,000 lines of old code (or, if you prefer, added 27,000 lines and altered 25,000 lines). At least that is the size of the change that I committed to the OU's CVS server last Friday, just in time to make the feature-freeze for the December update of our VLE. For comparison, the whole of Moodle 2.0 is about 1,600,000 lines of code, although that includes several large third-party libraries.

I am sure that there will be some minor bugs still to be found and fixed, but this new code has already had extensive testing from my colleagues Phil Butcher and Paul Johnson, so I am confident that the remaining bug-fixes will be minor.

There is much more I want to write about the new question engine, but I think this introductory post is already long enough. Therefore, I will split the remainder of what I want to say into separate posts which I hope to publish over the next week or so.

On the other side of the fence

2010-09-14T23:11:00.000+01:00

I've been having problems with my teeth recently. Fortunately, none of it has been painful, but cycling home from yet another dentist's appointment this afternoon, I suddenly had a thought that the experience I was having with my dentist might be a bit like the experience of a non-developer who encounters a bug in the Moodle quiz. Allow me to explain.

I think it has been a total of four trips to the dentist over the summer. First I went for a regular check-up. That revealed that one of my fillings was cracked, so there was another appointment to drill it out and redo it. Then a filling in another tooth fell out one Friday evening, so I could not get anything done about it until the following week, and over the weekend a bit of the tooth next to the hole broke off, which was really worrying. (Luckily, as I said above, it was not painful.) So anyway, that required a big filling to fill the resulting hole. Then part of the new filling broke off, so today's appointment was to re-do the missing bit. Hopefully that remedial fix will work. If the new bit breaks off again, I will have to go back to have the whole filling drilled out and redone.

With all this going on, I fear I have been starting to have unkind thoughts like: "Is my dentist competent?" "Should so many things go wrong over a few months?" "How do I know if this is normal?". This is exacerbated by what appears to me to be a slightly casual attitude on her part. I expect that these really are routine problems, and rather boring to her. I, however, am worried, so I would have appreciated a more concerned-seeming bedside manner.

As I say, it was just after I had climbed on my bike to go back to work this afternoon, that I had the epiphany that this is probably how someone feels when they come to the Quiz forum after encountering a problem in one of their Moodle quizzes. To them it seems like some terrible problem that has them really concerned. They describe their symptoms, and I read it and think "oh yes, the problem is probably in that bit of code, let me do a quick fix." And then maybe I screw up and introduce a regression, but when that is pointed out, it too is easily fixed. To me it seems like some routine and minor matter, but I have never really thought how the process of fixing bugs feels to someone who is not a software developer, and who does not really understand what is going on. Will they still trust Moodle?

Now that I have thought about it, will I do anything? I fear I am unlikely to change my 'bedside manner'. My time seems to be more than taken up with actually doing the bug fixes, and other development, to spend too much time being nice to people. Still, I will try to make a bit more effort in future know I now how it feels to be on the other side of the situation.

Before finishing, allow me to point out (particularly to American readers) that all these trips to the dentist have cost me very little or nothing. In particular, the two recent appointments to replace the filling that had (partly) fallen out cost me nothing. Thank you NHS. I hope the new government does not cut you to death.

While I am writing, I will also share the news that I finally bought an apartment of my own over the summer, after years of renting. It is very nice, and worth the hassles of dealing with solicitors. Hopefully now that the stresses of dealing with the move are behind me I can concentrate more on Moodle development, although there are still a few minor things to deal with like getting some more furniture. Over the weekend I ordered some new sofas, including some bright orange cushions. I can't think where I got the idea for such a daring colour ;-)

Book review: Moodle 1.9 Extension Development

2010-07-09T22:33:00.001+01:00

It is a book that the community has needed for a long time, a book that tells you how to write Moodle code. Now it is finally here. Does it live up to expectations?

Yes, I think it does. The authors, Mike Churchward and Jonathan Moore, are two experienced Moodle developers (they both work for Canadian Moodle Partner Remote Learner) so they can write authoritatively on the subject.

One issue with a book like this is that the examples given are, necessarily, fairly basic. To illustrate key techniques and ideas, a book must explain using the simplest example that makes the point. The question is, when you come to solving real problems, will the techniques you have learned expand to cope? Well, this is where the experience of the authors counts. They are telling you the right way to do things that works for real applications, even if they are only using simple examples to illustrate them.

The book does a thorough job of covering just about every type of Moodle plugin there is. Of course, some plugin types get more space than others, with the two most important, blocks and activity modules getting the most space. Therefore, some other plugin types, like question types, and gradebook plugins, are covered rather briefly. Between the chapters on the different types of plugins are chapters on more general topics like security, accessing the database, and so on.

Anyone who has had code reviewed by me will know that I get really pedantic when I review something. As I was reading the book I made a list of minor errors, or points where I disagreed with the authors. From the 300 pages of the book, I only found 22 things to put on my list, and none of them are interesting enough to mention here. (I did send the list to Jonathan.) So, I think this book has a very high standard of accuracy.

This book does assume you already know how to program in PHP, and write HTML and CSS. I think that was the right decision. There are plenty of excellent books out there that will teach you to write general web application in PHP, and it would be silly to duplicate those in a book that is uniquely about writing code for Moodle.

It is unfortunate timing that this book was released only a few months before Moodle 2.0. Moodle 2.0 does change quite a lot of the rules for how to do Moodle development, and so a lot of the details in the book will soon be out of date. However, don't let that stop you from getting this book. We have just talked about how this book helps you make the jump from being a general PHP developer to being specifically a Moodle (1.9) developer. Well, from there to being a Moodle 2.0 developer is just another small step. You won't be wasting much time if you learn about Moodle 1.9 first, and anyway, some people will still be running Moodle 1.9 for some time to come, and it will be a while before there is a book about Moodle 2.0 development on sale.

This should go without saying, but programming is an activity that you actually need to do to understand. You won't become an expert Moodle programmer just be reading a book. You will become a Moodle programmer by actually trying to write Moodle code, and learning from your own mistakes, and from the code other people have written in the past. What a book like this will do for you is that it will help you avoid a lot of the really basic mistakes, and it will set you off on the right path. So it will make your own learning-by-doing much more efficient, but I cannot replace the doing. Also, I would like to point out that while this is the only book about Moodle development, it is certainly not the only resource to help you learn Moodle development. If you are interested in this book, you should also look at the Developer documentation on Moodle Docs and the Introduction to Moodle Programming course on http://dev.moodle.org/.

Overall, if you want to learn Moodle development, this is a good book to help you attain your goal. Sure, you can get a lot of the information for free online, but in this book the authors set it out clearly and in a logical order. The information in this book has been written by expert Moodle developers and then carefully reviewed, so you can read the book without being on your guard for misinformation. You would have to be more careful just using the information Google finds for you online. So, as I say, this book lives up to expectations. If you want a book on Moodle development, get this one, and don't worry too much about Moodle 2.0 making it out-of-date.

Computational Knowledge

2010-06-09T23:59:00.011+01:00

I spent today at the London Computational Knowledge Summit. A rather pretentious name for a meeting organised by Wolfram about Wolfram|Alpha, Mathematica, and what these sorts of tools can do. They way they see it is that if Google is a means to retrieve facts, Wolfram|Alpha aspires to be a way to use information. For example, governments have started dumping a lot of data into the public domain, but how easy is it for a citizen to extract meaningful information from that data?

For someone used to Moodle and open source, I was deeply struck by how Cathedral (as opposed to Bazaar) the whole Wolfram/Mathematica world is. That may be, in part, a natural consequence of when they started. Wolfram has been working on Mathematica for twenty years. That is an impressive achievement, and it means they have been working on it since before Linux was a gleam in Linus Torvald's eye. They started at a time when the commercial model was how most software was written, and they are clearly still an American commercial software development business at heart.

It is interesting to ask what would have happened of Mathematica had been GPL. Would it have developed more or less than it has as a commercial project? Wolfram would not have been able to pour as much money into it, but would a community have done as much, or more? I am not sure. Clearly the commercial model has been highly successful for driving the development of Mathematica. It is a very cool, and very sophisticated tool. On the other hand, one can dream about a world where the Mathematica engine is free, and as a result shipped on every OLPC XO. What effect would that have on the world over the next twenty years? (Quiz, can you name Mathematica's closes open source rival? I can, but only because it is used in STACK. Speaking of which, it was nice to renew my acquaintance with Chris Sangwin today.)

Another things that struck me about the Wolfram world is how insular it is. Some of the speakers we heard clearly spend all their time thinking about Mathematica. They have a really amazing tool, and, having what they hope is the ultimate sledge-hammer, they seem to view most problems as nuts to be cracked. It made me worry about whether the Moodle community seems that insular from the outside.

However, the day was not just about Maths and Mathematica, but more about the democratisation of knowledge. How can a concerned citizen reach meaningful conclusions about the world using the data that is now available? Conrad Wolfram set the scene by talking about this mission. He started with two slides, one showing a page from Principia Mathematica, and then a page from a modern mathematical paper. They both comprised text, diagrams and equation. Certainly, modern type-setting is better, and the diagrams are now in colour, but that is not much to show for 350 years progress. Is this the best way to publish scientific knowledge in the 21st Century? No, it isn't. For instance, it is silly to show a static graph. It should be a graph that the reader can manipulate, and the raw data should be hidden behind it in a form that you can extract to perform you own analyses on if you choose. Is the paper about a model? If so the model should be embedded there, and you should be able to experiment with varying the inputs and the assumptions to see what happens. If you are interested, you should be able to get at the source code to see exactly what was implemented. So, rather than a document being a static, one-way, low-bandwidth form of information transmission, it should be an interactive things that allows the "reader" to engage in a two-way dialogue.

Of course, Mathematica is the perfect tool for authoring such documents, or so Wolfram would like you to think. To encourage this, they are talking about a CDF - computational document format - which would be an 'open' standard based on a subset of Mathematica's capabilities, with a 'free' player for all common platforms. Think PDF and Adobe. Will it take off? Well, does the rest of the world trust Wolfram enough to adopt their format? Can the offer enough more than HTML5? I am not sure.

The next talk was by Andrew Dilnot from Oxford University. He talked very engagingly about the difficulty with statistics and gave some nice topical examples of the need to interpret numbers in context. Presented with a numerical fact, you need to ask some basic questions: is this big or small? going up or down? where did it come from? For example, the smoking ban has saved the NHS £8million. Well, that is small. The NHS annual budget is £100billion. Why are the media making such a big deal of the story? His talk could be summarised as: All numbers are wrong, but using numbers is still much better than not using numbers.

There were a couple of talks about that were more pretty than interesting. John Barrow from Cambridge talked about the history of scientific images, and Alan Joyce showed some of the cool things you can do with Mathematical. There were also some glimpses behind the scenes of Wolfram|Alpha. As I say, very Cathedral. There is no web spidering or crowd-sourcing here. At one end it is about Wolfram people going out and finding the most authoritative data sources they can get, and loading them into a format their Mathematica code can use. At they other end they are training the Alpha system to take natural language queries, decide which methods to apply to what date, and select visualisation to display the results, in order to automatically give the best possible answer to that question. They gave an interesting parallel for this. In the past, all actors had to appear on stage to whatever live audience could fit into the theatre. Today, film and television allow most people to see the best actors in the world. Can the expertise of the best thinkers in different fields be made accessible in a similar way? Well, that is their goal. The initial results are impressive, but these are only the very early stages of a highly ambitious project and there is a lot more to do.

In the afternoon, Conrad Wolfram talked about his thoughts on maths education. This has traditionally been focussed on teaching students how to perform computations by hand. But, said Conrad, this is only one part of what you need to know to be a mathematician. The full steps are:

Ask an interesting question about the world.
Translate that real world question into mathematics.
Compute the solution.
Translate that solution back into real-world terms, and find some way to validate whether your answer makes sense.

(OU Maths students, note the similarities to the Modelling cycle as taught in OU courses.) Historically, maths education has focussed on Step 3. However, these days computers are just better than humans at that. At the same time, there has never been a greater demand for people who are good at the other three steps (in business, to guide public policy, and so on). So we need to rethink the maths curriculum. To put it another way, is it cheating if you use Wolfram|Alpha to do your maths homework? Well, if it is, then we are probably assessing the wrong things.

At the end, Stephen Wolfram gave a presentation by video link summarising some of the themes of the day.

Overall, it was a very interesting day. While I don't think it quite lived up to its pretentious title, it was certainly more than just a Wolfram marketing exercise. I would like to thank the Open University for sending me. I can see myself using Wolfram|Alpha in future for certain sorts of queries, but it will not replace Google as my day-to-day search engine. (I don't think it aspires too.)

When do students submit their online tests?

2010-03-25T12:17:00.000Z

I am currently studying an Open University course (M888 Databases in Enterprise systems. There was an assignment due today, and like many students, I submitted only an hour before the deadline.

That got me thinking, are all students really like that? Well, I don't have access to our assessment submission system, but I do work on our Moodle-based VLE, so I can give you the data from there.

This graph shows how many hours before the deadline students submit their Moodle quizzes (iCMAs in OU-speak)

That is not exactly what I was expecting. Certainly, there is a bit of a peak in the last few hours, but there is another peak almost exactly 24 hours before that, with lesser peaks two and three days before.

Note that all our deadlines are at noon (it used to be midnight, but that changed a few months ago). The graph above is consistent with our general pattern of usage. The following graph shows what time of day students submitted their quiz attempts. It is same shape as our general load graph for most OU online systems.

I don't know what, if anything, this means, but I thought it was interesting enough to share.

By the way, if you want to compute these graphs for your own Moodle, here are the database queries I used:

-- Number of quiz submissions by hour before deadline
SELECT 
    (quiz.timeclose - qa.timefinish) / 3600 AS hoursbefore,
    COUNT(1)

FROM mdl_quiz_attempts qa
JOIN mdl_quiz quiz ON quiz.id = qa.quiz

WHERE
    qa.preview = 0 AND
    quiz.timeclose <> 0 AND
    qa.timefinish <> 0

GROUP BY
    (quiz.timeclose - qa.timefinish) / 3600

HAVING (quiz.timeclose - qa.timefinish) / 3600 < 24 * 7

ORDER BY
    hoursbefore

-- Number of quiz submissions by hour of day
SELECT 
    DATE_PART('hour', TIMESTAMP WITH TIME ZONE 'epoch' + timefinish * INTERVAL '1 second') AS hour,
    COUNT(1)

FROM mdl_quiz_attempts qa

WHERE
    qa.preview = 0 AND
    qa.timefinish <> 0

GROUP BY
    DATE_PART('hour', TIMESTAMP WITH TIME ZONE 'epoch' + timefinish * INTERVAL '1 second')

ORDER BY
    hour

The Open University's approach to plagiarism

2009-10-15T19:54:00.005+01:00

I spent the day at an OU internal conference on plagiarism. Until now, most of my engagement with this subject has been people coming to the Moodle Quiz Forum expecting technological magic bullets. There seems to be a depressing assumption in some quarters that preventing students from copying or pasting during a quiz attempt, or from using a spell checker, in some way helps the validity of an assessment. And using Google during a quiz attempt is surely cheating, it couldn't possibly be a valuable study skill for the 21st century?

Anyway, it is just not possible to enforce these kinds of restrictions from a web server application like Moodle. Image, for a moment, what would happen to the web if any advertiser could stop you leaving their horrible web site and going to Google to find something better. If you really want to set up a restricted environment (akin to the traditional invigilated/proctored exam hall) you need client-side software installed on the computer the student will be using. If you are looking for something like this, take a look at Safe Exam Browser, and open-source effort built on Firefox, which is now integrated into Moodle 1.9.6 (due for release soon). See MDL-19145.

I am pleased to say that the OU approach is diametrically opposed to this wishful chasing after technological magic bullets. The conference today started with two keynotes by Jude Carroll from Oxford Brookes University, who has been thinking about these issues for the last nine years, and who is an excellent speaker. She has the very clear summary of the issue: Universities have the responsibility/duty/privileged to accredit student's learning. They give the student a bit of paper saying that they have learned something, and in our society that bit of paper has a certain amount of value. So universities need to measure learning, and that is where assessment comes in. The real problem with plagiarism is that it breaks the link between what the student submits for assessment, what they have learned. If a student writes something in their own words, then that writing is good evidence of the extent to which they have taken in the ideas of a course. If they have just copied and pasted someone else's words, you cannot assess what they have learned. Also, teachers do not set assessment tasks just to measure students. We hope that performing the tasks helps the student to learn.

Jude also made the point that exactly is plagiarism does depend on the context. For example the level of experience of the student, the learning that is being assessed, what you are allowed to assume as common knowledge. Also, not all plagiarism is cheating, and not all cheating is plagiarism. Dealing with plagiarism must be done in an appropriate and proportionate way. To start with, understanding issues of academic honesty is an important learning outcome from a degree course, but it needs to be taught, it is not innate. It cannot be taught as an optional add-on ("the plagiarism lecture") but instead it must be embedded in the context of the subject early in a degree program. (By and large the OU does this in its introductory course.) When plagiarism happens, the response has to be proportionate. Early in a program of study, students need to be given feedback to help develop good academic practices. Later on, students should be expected to understand these issues, and plagiarism becomes a disciplinary issue, but still their need to be a range of fair and proportionate penalties.

The OU has clearly had a group of people thinking hard about this over the last few years, and that has resulted in a new plagiarism policy that was adopted last June. This conference was part of disseminating that policy more widely. There are now a range of penalties for various types of plagiarism, and a new role, 'Academic Conduct Officers' who can deal with most issues of plagiarism that go beyond what the student's individual tutors can deal with. This lets many cases be dealt with quickly, so that only the most serious cases have to be referred to the Central Disciplinary Committee.

Another part of the OU's response to this issue is a new web site Developing good academic practices, which I am afraid you will only be able to see if you have some sort of OU login, student or staff. That is a Moodle site in our VLE. It is a mixture of text to read, with a few audio clips, with a quiz at the end of each section. There is also a summary quiz at the end which draws questions randomly from all the other quizzes. The estimated study time for the whole site is 2 hours. It seems to be well done, and for me it is pleasing to see the quiz module used well, even if it is only multiple-choice and matching questions. The course has about 2300 quiz attempts at the moment.

Finally, the OU does use two technological means to detect plagiarism, CopyCatch and Turnitin, on a proportion of submitted work. However, this is only used to flag up possible cases that are then reviewed by a human. The third form of detection is that tutors marking the students' work can often spot problems. However, overall, the amount of plagiarism detected at the OU seems to be lower than that reported elsewhere. It is interesting to speculate why that might be.

What I'm working on

2009-10-14T22:22:00.001+01:00

I've been meaning to post here for a while about what I am working on, but I keep posting in the Moodle Quiz Forum instead, because what I have to say seems more relevant there.

Anyway, the short answer is that I am rewriting the Moodle Question Engine. That document explains the what and the why. I want to post here about the how.

The question engine is the part of Moodle that processes what happens as a student attempts questions as part of a quiz (or other activity). What makes this a relatively difficult problem is that there are two independent degrees of variation. A question may be one of many different question types (multiple choice, short-answer, essay, ...) and there are various ways that a student may interact with it (adaptive mode, non-adaptive mode, manually graded, ...). Plus, people seem to think that when quizzes are used for summative assessments, it is quite important that there are no bugs and that performance is good ;-).

The complexity of the problem, and my natural bias, means that I am taking a very object-oriented approach. More so than is normally used in Moodle. Separate objects, with clearly defined responsibilities, helps me understand the problem and have confidence that what I am doing will work.

If you read the document linked to above, you will learn that I have been dissatisfied with this part of the quiz code for years, however it is really only within the last year that I have worked out the overall approach to solving this problem nicely. The key realisation is that you need to apply the strategy pattern twice, once for the bits that vary with the question type, and once for the bits that vary with the interaction model. The other important part of the plan is a cleaner database structure that more closely maps to data we need to store.

However, to implement all that requires quite a lot of code. I need to build the core system that can support all the different interactions and question types. Then I need to create all the interactions to replicate Moodle's existing functionality and the new features we want to add (Certainty Based Marking, and what we are calling the Interactive mode). Then I need to modify all the existing Moodle question types to fit into the new system. And I need to do all this with robust bug-free code that we can confidently use for summative assessment.

Over the last year I become increasing coverted to test-driven development. Certainly I wrote a lots of tests while eating my elephant, and this time round I am being very thorough with testing, although I must admit I still don't always write the tests first.

I am also proceeding very iteratively. I started with a test that took a single true-false question through being attempted according to the simplest of the interactions, and wrote some code that used roughly the right classes to make that test pass. Then for about a week I did little more than refactor that code.

It turned out that most of my variables and classes had the wrong name. For example, at one point I had classes called question_state and question_states, which was clearly a recipe for confusion. It was only after I had working code that I was able to think of the name question_attempt_step (one step in a question_attempt) for one of those. I also came to realise that grades were stored in three separate ways in different places, and so I ought to make sure I was using a consistent naming scheme for my variables. Then I did a bit of moving methods between classes as the exact boundaries of responsibility of different classes became clearer. Should I have been able to get all that right first time? Well I think that would be impossible.

I also started gradually removing simplifying assumptions that worked for my initial test true-false question, but not more generally. That was the point at which I started extending my code to other question types and interactions. For example I wrote a test case a walking an essay question through the manual grading interaction, and exposed some new issues that had to be accounted for; and just today I introduced multiple-choice questions, which randomise the order of the choices when you start an attempt, but then you have to store that random order somewhere so it can be used throughout the attempt. I have also already implemented the certainty based marking interaction.

There is still much to do. My code that outputs the HTML for a question is still a rough hack of the current code that lets the tests pass. I need several rounds of refactoring before that will be cleaned to my satisfaction. There are sill five more interactions to write, although each one is getting quicker than the one before. Then I have to convert all the existing question types. I need code that stores the current state of everything in the database and later reads it back. Finally, the nasty bits, which are upgrading from the current database structure to the new one, and re-implementing backup and restore, including restore of old backups.

If you were paying attention in that last paragraph, you will have deduced that my code does not yet store anything in the database! I am taking an approach with the domain objects completely unaware of the database (which makes testing very easy) and I am planning to use the data mapper pattern to coordinate loading and saving the necessary data efficiently. I am pretty sure I know how to do that, however, I have not tried to write it yet because I want to re-read the relevant chapters of Patterns of Enterprise Application Architecture, and my copy only recently got back from Australia, and I won't be able to retrieve it from my parents' house until this weekend.

In terms of time-scales, I am really hoping to get this done by about Christmas, and in time to go into Moodle 2.0, since it breaks a number of APIs. However, I first have to implement this in the OU's Moodle 1.9 code-base and then port it to Moodle 2.0, so no promises. This may have to wait until Moodle 2.1.

Anyway, that is what I have been doing, and so far it has been very enjoyable. Write tests, make them pass, then refactor aggressively with the tests as a safety net is an approach that is working really well for me. These are quite big scary changes, but right now I am feeling confident that it will all work out, and we will end up with a really solid question engine upon which to base an enhanced quiz and other Moodle activities.

Herding cats

2009-08-24T11:47:00.000+01:00

Just before I left Perth, Martin advertised for a Project Manager for Moodle HQ. I think this is probably a good thing, if a good person applies and is appointed. However, I also think it will be a very difficult job to do.

Martin always has too much to do, and any time he can identify some tasks that can be delegated, it is a good thing. Early in the history of Moodle HQ he found Mike and Darlene who run a lot of the commercial side of the Partners network. More recently he appointed Helen to act as community co-ordinator, and Jordan as system admin. All those were successful appointments, and having watched form the sidelines I have seen that Martin will not recruit someone unless he is sure they are right for the job (important when you are running a small business) and Martin is a good judge of people.

The other reason a project manager would be a good idea is clear to anyone who had been following the Moodle 2.0 development, which still has a long way to go until it is finished. Someone with more time than Martin has to ride herd on the process can only help us get 2.0 out of the door.

However, it is not quite that simple. On the one hand, I have studied the Open University Project Management course from their Postgraduate Computing program. That is an excellent course, and as a result I know a bit about and am interested in what project managers do. On the other hand, I also know the fairly independent way that I and other core developers like to work. That is why I chose the title of this blog post. (Not because of some people's user profile pictures!)

To start with, the Moodle project is not even a project in the sense that Project Managers use the word. A typical definition is "A project is a temporary endeavor, having a defined beginning and end," but Moodle itself will not end (we hope). Still, some of the things we do, for example, the Moodle 2.0 release, are projects in this sense. However, we know there will also be another project called the Moodle 2.1 release later. Therefore, if we contemplate a short-cut that would let us release Moodle 2.0 sooner, we have to balance the short-term benefits with the long-term costs of cleaning things up in a later release. Classic project management tends to think about one project at a time.

That ties in with Moodle's choice of feature-driven, not date-driven releases. We will pick a set of features to be included in a release, and we will release when they are done, tested and bug-fixed, however long that takes. Well that is the general policy, but a certain amount of pragmatism is applied when interpreting it. This is a fairly unusual way to run a project. For example, with the Open University's Moodle-based VLE we operate the other way. We have date-based releases, with the dates announced a long way in advance, but we do not guarantee what features will be an any given release very far in advance.

Then there is the the nature of the work. One area where project management is generally successful is in construction projects. The cliché old maths problem "If it takes 2 men 7 hours to dig a trench 6 metres long ..." has at least some relevance to reality there. There is even the Bluebook cost guide that has established estimates for certain types of construction work.

In contrast, as that classic of software project management "The Mythical Man Month" says, "adding manpower to a late software project makes it later." What software developers do is much closer to an architect drawing the plans than to a builder laying bricks. The compiler (or PHP interpreter) is the builder. This is hardly new, and is why agile software-development methodologies were invented. If Moodle's new project manager is going to adopt a Methodology, I hope the choose an agile one.

Another difficulty is that software developers are not fungible. If there is some tricky job involving security you probably want Petr to do it. If it is to do with the quiz, it would be best if I did it. If it is related to backup and restore, no one wants to do it, and so it ends up with Eloy ;-). If those people do the job, it will be quicker. On the other hand, sometimes you want to give the job to someone else, to increase the bus number of the project. While this sort of think makes the project harder to manage, it is the sort of thing that a good project manager knows how to deal with - once they get to know the people involved.

Another consideration is that developers are highly distractible. That is, while they are working on one thing, they may suddenly notice that something is a bit messy, and decide to rewrite it or refactor it, even if that was not part of the original plan. When considering the long-term health of the moodle, that might be the right thing to do, even if, in the short term it delays the next release. At other times, it might just be self-indulgance and cause bugs. Would a project manager be able to tell the difference? Anyway, as some-time tiddlywinks world champion Andy Purvis once told my, after thrashing me soundly, "Strategy should be a mixture of planning and opportunism."

A final potential problem is that the Moodle core developers are a highly motivated group of people. We care a lot about Moodle, and have demonstrated that over the years. We don't need to be told what to do, and a project manager who tried could piss a lot of people off very rapidly and do a lot of harm to the project. Fortunately, I think the chance of Martin appointing someone like that is tiny. However, I think is an important point to make, that the Moodle Project Manager should be a colleague and peer of the developers (with a particular set of skills). That is, they should manage the project, not the developers.

So, I have said that I think a Moodle Project Manager is a good idea, but I have also said why I think it would be a very difficult job to do. So what do I think the Moodle Project Manager can usefully contribute (that I, as a developer, would not object to ;-)).

A good project manager is always a good communicator, and in multiple directions. The Moodle Project Manager will need to communicate with the developers to track as clearly as possible what development is going on, and what still needs to be completed before the next release. They then need to communicate that picture of where we are to all the Moodle administrators waiting for the next release, including the Moodle Parters. They also need to communicate it back to the developers to help them see what progress is being made and what the priorities. Naturally they will be working closely with Martin. They also need to communicate progress to Translators and third-party plugin authors, to help translators have their translations ready as soon to the main release as possible.

A project manager is also good at extracting clarity from a confusing situation, which is really another part of what I just wrote. They can take reports from a lot of people, and a more-or-less vague list of outstanding tasks, and combine that into a picture of where we are now, and where we are going that other people can understand.

Once you can communicate well, and as a result derive a clear picture of where we are and where we are going, then the actually project planning - getting the right people to work on the right tasks next - is relatively easy, but still requires some knowledge of the people and the tasks. Then there are the people skills to help keep people motivated and focussed on the next release without trying to over-manage them.

Finally, and particular around the release time, a project manager can improve the processes. For example, does Moodle need string and user-interface freezes in the run up to the release, to help translators and theme designers? (The answer needs to be a consensus of all involved, so more communication.)

The project manager will also be one of the people in a position to hear what new features people are most desperate to have next, and so will be able to contribute to planning the scope of future releases.

So, it is a challenging job, but I think someone is needed to do it.

On the busses

2009-08-13T10:09:00.002+01:00

On Monday I was privileged to to accompany Jo, Linda and Mary on two of their busses. Jo and I made our way to the middle of a street by the Thames in Pimlico, just opposite the old Battersea Power station, meeting Mary at one end of Vauxhall bridge, and Linda at the 24 head stop. Apparently that is the proper name for the bus stop at the end of a bus route.

The plan had been to get the 24 all the way to the other end at Hampstead Heath, and then get the 168 back all the way south to Old Kent Road. However, it turned out due to an accident in Tottenham Court Road, the 24s were not going beyond Victoria. We got kicked off there, and were faced with a potential rules crises. Jo, displaying a frightening knowledge of the London bus network, almost immediately suggested that we could get the C2 from Victoria to the other side of the Heath, and then walk across to the 168 head stop and so salvage some of the day.

Speaking of the rules of their busses odyssey, they can do other busses out of order, but are only allowed to add them to the blog in order. So, since the C busses will only appear after all the numbered busses, and the As and the Bs, that bit of the trip will probably get blogged in about 5 years time. Who know what the rules say if the route if modified in the mean time?

Having walked across the park, we found not only 168 busses, but also plenty of 24s, and the news that the route was now clear. (Actually, the busses going south use Gower street, not Tottenham Court Road.) So, in the event we were able to complete the rest of the 24 route in reverse, finishing at Victoria, from where Jo and I went to Oxford Circus for a very nice Curry at the Kerala.

It's a strange hobby, but it keeps my mother out of mischief of a Monday and it was nice to be able to share it with her. Being a geek, I hope I can be there when they do the 42 route. And I am pleased I persuaded my mother to blog. Now that I have added a hit counter to their site (thanks to http://www.statcounter.com/) we will be able to see whether anyone is interested, or whether the only visitors are friends and family, or people trying to find travel information. I have now also added a little search form to let people easily find TfL bus route maps from the blog.

One step at a time

2009-07-14T03:09:00.004+01:00

An important thing to realise about Open Source development is that nothing is ever finished and nothing is ever perfect. (If it was, we would all be out of a job ;-)). Open source is never a project [1], but an eternal quest, and the relevant questions for planning are only ever: "What should we do next?" "What are the biggest problems now?" with the focus on next and now. The quest can only proceed one step at a time, and you know there will be more steps in the future.

Of course, you must have a bold vision of where you are all heading, so everyone is moving in roughly the same direction, but that should not involve details. The details of what you will work on next year are better left until next year. You will be a year older and wiser then, and the state of the art will have moved on.

An important aspect of "what should we do next?" is that it must be somewhere you can get to in one step from where you are now. (Some of the steps in Moodle 2.0 have been quite large.) Open Source proceeds best by accumulating incremental change. That is my belief anyway.

1. A project is a temporary endeavor, having a defined beginning and end ...

(This evolved out of a message I was writing to my Google Summer of Code mentee Olli Savolainen. Thanks Olli.)

The geeks don't really matter

2009-06-22T06:02:00.009+01:00

I was just reading this criticism of Moodle, and the thing that strikes me about Bryan Ollendyke's criticisms are the reactions of a software geek. They are also very familiar. I had a similarly horrified reaction the first time I saw the Moodle code (and got into trouble with my boss at the time for speaking my mind at an inopportune moment ;-)) and three and a half years later the same criticisms still apply.

But what I have come to learn over the last three and a half years is that it does not really matter. That is somewhat humbling. I am a professional software developer. I have worked hard to learn to write good code; to build and appreciate elegant and powerful software architectures; to design usable interfaces with modern UI idioms; and so on. However, as I say, that seems not to be very important. Teachers all over the world use Moodle to create great (and ordinary) learning experiences; teachers who move from other learning management systems tend to like Moodle better than whatever they were using before; and all sorts of people from all over the place seem to be able to write plugins. All that, despite Moodle not being a text-book example of software design.

So, are Bryan and I and others with similar opinions just software engineering snobs? Well, probably yes, up to a point, but it is our job to care about these things. Other things being equal, we should do things the right way. The point is that some of the other things are much more important than the cool (or not) technical things going on behind the scenes. The most important thing is education. Moodle's genesis was in Martin Dougiamas's postgraduate studies of online education; but more important is that the Moodle project, led by Martin, has always focussed on engaging teachers, as well as developers, in the Moodle community. To use the HCI jargon, Moodle has always practiced user-centred design.

Now, if you are an expert in HCI design, and you have seen parts of Moodle like the quiz settings form (screen-shot right), then that last sentence might make you laugh. That form is a typical long Moodle form with a million fields you can fill in. In the technical details it does not follow recent user interface best practice. However, suppose you really want to simplify it. The best approach would be to find some options that nobody really needs, and remove them. Well, believe me, I have tried on several occasions. I have convinced myself that some feature is pointless and posted in the quiz forum. Of course, I quickly get a lot typical 'How dare you! I love it that feature. I'll throw my toys out of the pram if you remove it.' responses. Then, invariably, someone will make a well reasoned post that explains a compelling educational situation where that feature really makes a difference, and I will realise why that feature was added in the first place, and why it would be a mistake to remove it.

That is the thing. All those features that bloat the interface are there because someone wanted them badly enough to implement them. Even if they were not a professional developer, and even if their code is not first rate. On the whole, it is probably better for Moodle to have those features than not, even if that means that the interface is not great. We are not here primarily to build beautiful interfaces. We are here to let teachers build beautiful learning experiences. Of course, that is not an excuse to just ignore defects in the interface. We should, and do, go back and apply HCI know-how to improve the interfaces of existing features; and these days we do try to think about UI design for new features before we implement them. We are learning.

To reiterate my main points in summary: While we developers should strive to do the best job we can, technically, we should not be snobbish about it. We must remember that the most important people for Moodle are the teachers and learners, and it is our job to serve them. We should not reject contributions from people who know lots about education, even if they do not write code as well as we do.

PHP global variables are not necessarily evil

2009-04-17T15:31:00.006+01:00

Global variables in software are generally a bad idea. They serve to magically teleport information from one place in a program to another in a way that is very hard to follow. Thus they cause subtle bugs, make maintenance difficult, and kill kittens.

In a simple program, it is normally easy to avoid global variables. When you call a function or method to get it to do something for you, you can probably just pass all the necessary inputs to the function as parameters, and get all the results back in the return statement. The flow of data is easy to follow.

However, in a large, complex system, that may become untenable. For example many, but not all functions in Moodle need a database connection to get information or update it. Does that mean all functions in Moodle end up taking a $db parameter, whether they need them or not, just so they can pass it on to any functions they in turn call? Well, we don't do that, because that way lies madness. Instead we have a 'global' $DB object, and that is not all. We also have $CFG, $COURSE, and so on. Imaging if you needed many of those inside your function. Calling it with all those parameters would be a pain.

Now, I just put the word 'global' in quotes, why was that? Well, Moodle is a web application, so the process that is running is a web server, probably with multiple threads executing simultaneously. A true global variable would be accessible from all threads, and last forever. Something you declare in PHP as a global is only accessible for the duration of one server request, and not accessible from any other thread. In Java, that would be called a ThreadLocal, not a global.

So, a PHP global is not truly a global. However, if abused, it can still be a way to teleport data from one place to another during the processing of a single request. The point I want to make is that there are non-evil ways to use it. In Patterns of Enterprise Application Architecture Martin Fowler describes the Registry pattern (sorry, that online summary is inadequate). That is an object in your program which all the other parts can get hold of, and from which they can get the service objects, like the database connection, that they must depend on. In web applications, you often want dependancies with thread local scope. My argument is that the PHP global keyword is a language feature that implements a thread-local registry for you with no effort. Thus it is a good thing if used properly.

What is using it properly? Well it all depends on the kind of things you store there. Are you making common dependancies easy to find, or are you magically teleporting information. I think that Moodle does mostly use this feature properly. Our global variables like $DB, $CFG and $COURSE are things that can legitimately be stored in and accessed from a Registry. However, we have some horrors, like $CFG->pagepath and $CFG->stylesheets, which I am currently trying to exorcise from Moodle 2.0.

One advantage of storing your dependencies in a Registry, as opposed to, say, making them singletons, is that it makes them substitutable. This becomes important when you try to write unit tests. When unit testing, you often need to switch in a test double in place of one of the dependencies. Well, with a Registry you can swap one in during test set up, and switch it back out during tear down. I have been doing that a lot recently in Moodle, and it works. (You could not do it, for example, when you accessing the database in Moodle 1.9. There, the database connection was hidden in functions like get_records, and could not be substituted. Now we have $DB->get_records with $DB in the 'global' registry where it can be substituted.) Some other global-like mechanisms, like static methods or singletons are not substitutable, and so make testing much harder, if not impossible.

Of course, some people would argue that you still should not use the global keyword, that instead you should implement your own Registry class (an example). I disagree. Appropriate use of language features tends to create more idiomatic code.

How do you eat an elephant?

2009-04-08T15:40:00.000+01:00

Conductors say the funniest things some times. Shaun uttered the above gem at a rehearsal recently, seemingly apropos nothing. The answer is, of course, one bite at a time. It turned out that he was trying to tell us that we would one day get to grips with the first movement of the César Franck symphony in D minor, if we rehearsed it a bit at a time - yes, even the cellos.

The symphony I think I can cope with. (Those bits I can't play properly, I will be able to fake ;-)) Instead, my personal elephant is currently the Navigation changes for Moodle 2.0. Although it is described as Navigation changes, it is much, much more than that. Blocks / themes / filters / rendering / pagelib / navigation changes would be more apt, and that is at least an African elephant, if not a mammuthus sungari.

Fortunately, I have Shaun's advice to guide me, so I sharpened my metaphorical butcher's knife and started trying to dissect it into manageable chunks. Unfortunately I only got part way, because I started thinking about the filters bit of it, which I had previously designed in detail. Since it is more fun to write code than to write documentation, and because I had been reading a book about unit testing and test driven development, and wanted to try some, I got seduced. Still, it is a worthwhile feature and the implementation is going well. You can follow the development in MDL-7336.

Of course, once that is done, I must get back and finish the breakdown of all the Navigation work, especially as some of it needs funding, and so we need some estimates. That summarises what I have been working on in Moodle land recently.

By the way, the book I mentioned (xUnit Test Patterns) is excellent, if you want to read an 800 page book about unit testing. If you want a lighter introduction to the topic, Pragmatic Unit Testing, or the Unit Testing chapter of PHP in Action, are probably better bets.

Avoiding duplication and increasing flexibility in software: what about version control?

2009-02-18T13:13:00.004Z

These thoughts were catalysed by Section 4.2.4 "Once and only once" of PHP in Action where they preach the evils of duplicated code. This is something I strongly believe in. (My best exorcism to date is this Moodle commit.)

Unfortunately, the example they pick to illustrate their point utterly fails. Their example of what not to do is making a custom version of an application in a hurry by copying the whole thing and editing a few lines. Well, duh! Put the code in git and make a branch for the custom version. And, of course, when you do have time, come back and refactor, at which point git diff/merge/rebase ... will probably be a big help.

Now, this is a form of software engineering that open source developers do all the time (while having wild flame wars about which version control system is best). But, when computer scientists dream about paradigms like object orientation, design patters, refactoring, and so on, as ways to reduce the need for code duplication and increase the flexibility of software designs, do they consider the role of version control systems? I don't recall reading anything.

The point about flexibility is particularly interesting in the context of opens source software written in in an interpreted language. Given my background, I am of course thinking about Moodle, which is a PHP web application. Suppose, by way of example, you want to change some part of the processing that occurs when a student submits a quiz, and Moodle works out what score they deserve. The 'proper' design patterns way to do this is:
1. Check: Is the processing algorithm factored into a separate class, as recommended by the Strategy pattern? If not, refactor.
2. Subclass the default Strategy to implement your customisations.
3. Configure things so the factory methods instantiate your Strategy class, rather than the default one.

Alternatively, you could just hack the code. Preferably as a custom branch in your version control system. When a new Moodle version is released, just upgrade and merge (or pull and rebase). One could even try to claim that this way is better, because if the code you hacked changes between releases, the merge may fail, helping you realise that you need to rework your customisation. If you don't notice the problem until you get to testing (which may happen anyway) it will be much harder to find and diagnose.

Another example is this post in the Moodle quiz forum. Someone wanted to tweak something in Moodle, they were told to change the definition of a constant in a library file. Note that because there is no duplication of magic numbers throughout the code, it was only necessary to change the number in one place, so there is some proper design going on here.

One could, of course, have a separate configuration file for all the settings like this in Moodle, but does that really make anyone's life easier? If you think of all the things people might want to tweak, that would be one big file. Also, it would move the constant definition further from where it is used, reducing the cohesion in the code. You could also handle this by making this number an admin setting, stored in the database, and editable through the web interface. In this case, I would argue that that is bad UI design. 3 is a perfectly good default for almost everyone, and Moodle already has more than enough configuration settings. Sufficiently few people need to adjust this, that telling those that do to edit the code is an adequate interface.

You could call this approach "The whole code is a configuration file". In the abstract, you would not say it was good design, but for obscure configuration options like this, it may be the best way.

So, in a world with version control and interpreted languages, what is flexible and easy-to-modify software?

(To head off the obvious comments, I had better re-enforce that I do like nice clean software design, and design patterns, and so on; but I also spend most of my life working on Moodle where parts of the code are not like that; and somehow, most of the time, it just does not seem to matter. Millions of people around the world happily use and customise Moodle despite the lack of design patterns in the code. Should I be sad, or happy?)