One step at a time

An important thing to realise about Open Source development is that nothing is ever finished and nothing is ever perfect. (If it was, we would all be out of a job ;-)). Open source is never a project [1], but an eternal quest, and the relevant questions for planning are only ever: "What should we do next?" "What are the biggest problems now?" with the focus on next and now. The quest can only proceed one step at a time, and you know there will be more steps in the future.

Of course, you must have a bold vision of where you are all heading, so everyone is moving in roughly the same direction, but that should not involve details. The details of what you will work on next year are better left until next year. You will be a year older and wiser then, and the state of the art will have moved on.

An important aspect of "what should we do next?" is that it must be somewhere you can get to in one step from where you are now. (Some of the steps in Moodle 2.0 have been quite large.) Open Source proceeds best by accumulating incremental change. That is my belief anyway.

1. A project is a temporary endeavor, having a defined beginning and end ...

(This evolved out of a message I was writing to my Google Summer of Code mentee Olli Savolainen. Thanks Olli.)

The geeks don't really matter

I was just reading this criticism of Moodle, and the thing that strikes me about Bryan Ollendyke's criticisms are the reactions of a software geek. They are also very familiar. I had a similarly horrified reaction the first time I saw the Moodle code (and got into trouble with my boss at the time for speaking my mind at an inopportune moment ;-)) and three and a half years later the same criticisms still apply.

But what I have come to learn over the last three and a half years is that it does not really matter. That is somewhat humbling. I am a professional software developer. I have worked hard to learn to write good code; to build and appreciate elegant and powerful software architectures; to design usable interfaces with modern UI idioms; and so on. However, as I say, that seems not to be very important. Teachers all over the world use Moodle to create great (and ordinary) learning experiences; teachers who move from other learning management systems tend to like Moodle better than whatever they were using before; and all sorts of people from all over the place seem to be able to write plugins. All that, despite Moodle not being a text-book example of software design.

So, are Bryan and I and others with similar opinions just software engineering snobs? Well, probably yes, up to a point, but it is our job to care about these things. Other things being equal, we should do things the right way. The point is that some of the other things are much more important than the cool (or not) technical things going on behind the scenes. The most important thing is education. Moodle's genesis was in Martin Dougiamas's postgraduate studies of online education; but more important is that the Moodle project, led by Martin, has always focussed on engaging teachers, as well as developers, in the Moodle community. To use the HCI jargon, Moodle has always practiced user-centred design.

Now, if you are an expert in HCI design, and you have seen parts of Moodle like the quiz settings form (screen-shot right), then that last sentence might make you laugh. That form is a typical long Moodle form with a million fields you can fill in. In the technical details it does not follow recent user interface best practice. However, suppose you really want to simplify it. The best approach would be to find some options that nobody really needs, and remove them. Well, believe me, I have tried on several occasions. I have convinced myself that some feature is pointless and posted in the quiz forum. Of course, I quickly get a lot typical 'How dare you! I love it that feature. I'll throw my toys out of the pram if you remove it.' responses. Then, invariably, someone will make a well reasoned post that explains a compelling educational situation where that feature really makes a difference, and I will realise why that feature was added in the first place, and why it would be a mistake to remove it.

That is the thing. All those features that bloat the interface are there because someone wanted them badly enough to implement them. Even if they were not a professional developer, and even if their code is not first rate. On the whole, it is probably better for Moodle to have those features than not, even if that means that the interface is not great. We are not here primarily to build beautiful interfaces. We are here to let teachers build beautiful learning experiences. Of course, that is not an excuse to just ignore defects in the interface. We should, and do, go back and apply HCI know-how to improve the interfaces of existing features; and these days we do try to think about UI design for new features before we implement them. We are learning.

To reiterate my main points in summary: While we developers should strive to do the best job we can, technically, we should not be snobbish about it. We must remember that the most important people for Moodle are the teachers and learners, and it is our job to serve them. We should not reject contributions from people who know lots about education, even if they do not write code as well as we do.

PHP global variables are not necessarily evil

Global variables in software are generally a bad idea. They serve to magically teleport information from one place in a program to another in a way that is very hard to follow. Thus they cause subtle bugs, make maintenance difficult, and kill kittens.

In a simple program, it is normally easy to avoid global variables. When you call a function or method to get it to do something for you, you can probably just pass all the necessary inputs to the function as parameters, and get all the results back in the return statement. The flow of data is easy to follow.

However, in a large, complex system, that may become untenable. For example many, but not all functions in Moodle need a database connection to get information or update it. Does that mean all functions in Moodle end up taking a $db parameter, whether they need them or not, just so they can pass it on to any functions they in turn call? Well, we don't do that, because that way lies madness. Instead we have a 'global' $DB object, and that is not all. We also have $CFG, $COURSE, and so on. Imaging if you needed many of those inside your function. Calling it with all those parameters would be a pain.

Now, I just put the word 'global' in quotes, why was that? Well, Moodle is a web application, so the process that is running is a web server, probably with multiple threads executing simultaneously. A true global variable would be accessible from all threads, and last forever. Something you declare in PHP as a global is only accessible for the duration of one server request, and not accessible from any other thread. In Java, that would be called a ThreadLocal, not a global.

So, a PHP global is not truly a global. However, if abused, it can still be a way to teleport data from one place to another during the processing of a single request. The point I want to make is that there are non-evil ways to use it. In Patterns of Enterprise Application Architecture Martin Fowler describes the Registry pattern (sorry, that online summary is inadequate). That is an object in your program which all the other parts can get hold of, and from which they can get the service objects, like the database connection, that they must depend on. In web applications, you often want dependancies with thread local scope. My argument is that the PHP global keyword is a language feature that implements a thread-local registry for you with no effort. Thus it is a good thing if used properly.

What is using it properly? Well it all depends on the kind of things you store there. Are you making common dependancies easy to find, or are you magically teleporting information. I think that Moodle does mostly use this feature properly. Our global variables like $DB, $CFG and $COURSE are things that can legitimately be stored in and accessed from a Registry. However, we have some horrors, like $CFG->pagepath and $CFG->stylesheets, which I am currently trying to exorcise from Moodle 2.0.

One advantage of storing your dependencies in a Registry, as opposed to, say, making them singletons, is that it makes them substitutable. This becomes important when you try to write unit tests. When unit testing, you often need to switch in a test double in place of one of the dependencies. Well, with a Registry you can swap one in during test set up, and switch it back out during tear down. I have been doing that a lot recently in Moodle, and it works. (You could not do it, for example, when you accessing the database in Moodle 1.9. There, the database connection was hidden in functions like get_records, and could not be substituted. Now we have $DB->get_records with $DB in the 'global' registry where it can be substituted.) Some other global-like mechanisms, like static methods or singletons are not substitutable, and so make testing much harder, if not impossible.

Of course, some people would argue that you still should not use the global keyword, that instead you should implement your own Registry class (an example). I disagree. Appropriate use of language features tends to create more idiomatic code.

How do you eat an elephant?

Conductors say the funniest things some times. Shaun uttered the above gem at a rehearsal recently, seemingly apropos nothing. The answer is, of course, one bite at a time. It turned out that he was trying to tell us that we would one day get to grips with the first movement of the César Franck symphony in D minor, if we rehearsed it a bit at a time - yes, even the cellos.

The symphony I think I can cope with. (Those bits I can't play properly, I will be able to fake ;-)) Instead, my personal elephant is currently the Navigation changes for Moodle 2.0. Although it is described as Navigation changes, it is much, much more than that. Blocks / themes / filters / rendering / pagelib / navigation changes would be more apt, and that is at least an African elephant, if not a mammuthus sungari.

Fortunately, I have Shaun's advice to guide me, so I sharpened my metaphorical butcher's knife and started trying to dissect it into manageable chunks. Unfortunately I only got part way, because I started thinking about the filters bit of it, which I had previously designed in detail. Since it is more fun to write code than to write documentation, and because I had been reading a book about unit testing and test driven development, and wanted to try some, I got seduced. Still, it is a worthwhile feature and the implementation is going well. You can follow the development in MDL-7336.

Of course, once that is done, I must get back and finish the breakdown of all the Navigation work, especially as some of it needs funding, and so we need some estimates. That summarises what I have been working on in Moodle land recently.

By the way, the book I mentioned (xUnit Test Patterns) is excellent, if you want to read an 800 page book about unit testing. If you want a lighter introduction to the topic, Pragmatic Unit Testing, or the Unit Testing chapter of PHP in Action, are probably better bets.

Avoiding duplication and increasing flexibility in software: what about version control?

These thoughts were catalysed by Section 4.2.4 "Once and only once" of PHP in Action where they preach the evils of duplicated code. This is something I strongly believe in. (My best exorcism to date is this Moodle commit.)

Unfortunately, the example they pick to illustrate their point utterly fails. Their example of what not to do is making a custom version of an application in a hurry by copying the whole thing and editing a few lines. Well, duh! Put the code in git and make a branch for the custom version. And, of course, when you do have time, come back and refactor, at which point git diff/merge/rebase ... will probably be a big help.

Now, this is a form of software engineering that open source developers do all the time (while having wild flame wars about which version control system is best). But, when computer scientists dream about paradigms like object orientation, design patters, refactoring, and so on, as ways to reduce the need for code duplication and increase the flexibility of software designs, do they consider the role of version control systems? I don't recall reading anything.


The point about flexibility is particularly interesting in the context of opens source software written in in an interpreted language. Given my background, I am of course thinking about Moodle, which is a PHP web application. Suppose, by way of example, you want to change some part of the processing that occurs when a student submits a quiz, and Moodle works out what score they deserve. The 'proper' design patterns way to do this is:
1. Check: Is the processing algorithm factored into a separate class, as recommended by the Strategy pattern? If not, refactor.
2. Subclass the default Strategy to implement your customisations.
3. Configure things so the factory methods instantiate your Strategy class, rather than the default one.

Alternatively, you could just hack the code. Preferably as a custom branch in your version control system. When a new Moodle version is released, just upgrade and merge (or pull and rebase). One could even try to claim that this way is better, because if the code you hacked changes between releases, the merge may fail, helping you realise that you need to rework your customisation. If you don't notice the problem until you get to testing (which may happen anyway) it will be much harder to find and diagnose.


Another example is this post in the Moodle quiz forum. Someone wanted to tweak something in Moodle, they were told to change the definition of a constant in a library file. Note that because there is no duplication of magic numbers throughout the code, it was only necessary to change the number in one place, so there is some proper design going on here.

One could, of course, have a separate configuration file for all the settings like this in Moodle, but does that really make anyone's life easier? If you think of all the things people might want to tweak, that would be one big file. Also, it would move the constant definition further from where it is used, reducing the cohesion in the code. You could also handle this by making this number an admin setting, stored in the database, and editable through the web interface. In this case, I would argue that that is bad UI design. 3 is a perfectly good default for almost everyone, and Moodle already has more than enough configuration settings. Sufficiently few people need to adjust this, that telling those that do to edit the code is an adequate interface.

You could call this approach "The whole code is a configuration file". In the abstract, you would not say it was good design, but for obscure configuration options like this, it may be the best way.


So, in a world with version control and interpreted languages, what is flexible and easy-to-modify software?


(To head off the obvious comments, I had better re-enforce that I do like nice clean software design, and design patterns, and so on; but I also spend most of my life working on Moodle where parts of the code are not like that; and somehow, most of the time, it just does not seem to matter. Millions of people around the world happily use and customise Moodle despite the lack of design patterns in the code. Should I be sad, or happy?)