Is learning design like UML?

A couple of weeks ago, I attended the #design4learning conference, which was conveniently on my doorstep at the Open University. Jenny Gray has already written her summary of the conference (and she though she was a bit late writing it up!)

I would like to highlight the point the organisers made with the conference name. Calling learning design "learning design" is a misnomer. You cannot design learning. Learning is something that goes on inside the student's head, perhaps most effectively under the support and guidance of a teacher. Therefore, you can only "design for learning", whatever it is you are designing: a course, a activity, a learning community, …. I think this is more than just semantic pedantry. We should all remember this, particularly when thinking about educational technology. There is no magic bullet that guarantees learning will occur. Just things that are more or less likely to encourage students to learn. (Having said this, I am going to just write "learning design" in the rest of this post, since it is so much easier!)

The main thought I wanted to share here is, however, something else. After two interesting days at a conference all about learning design, I cannot recall a single diagram shown by any speaker where I thought, "that is a graphical representation of the design of a bit of learning." Was I right to expect to see that? I don't know, but I have seen other presentation about tools like CompendiumLD in the past so I know it can be done. Pondering this as I cycled home, I got to thinking about the type of design I do know about: design of software, and though of an interesting comparison.

Software developers have a well-established way to draw the design of their software, called UML (better description on Wikipedia). Let me say immediately that I am not trying to suggest UML as a way to represent learning designs. Rather, I think it is interesting to think about how developers do (or more often don't) use UML to help their work. Can that tell us anything about how and whether teachers might engage with learning design?

There are two different ways to use UML. There is the quick-and-dirty, back-of-the-envelope way, where you draw of a part of the system to help explain or communicate a particular aspect of your design. This is the way I use UML as can be seen, for example, in this documentation I wrote. You include the details that are relevant to making your point, and leave out anything that does not help.

The other way to use UML is much more elaborate. It is called "Model Driven Architecture" which I studied as part of OU module M885. In MDA, you try to draw complete diagrams of the design of your system using a very precise dialect of UML, dotting all the 'i's and crossing all the 't's. Then, using a software tool (that you probably had to buy at great expense) you press the magic button, and it creates all your classes and interfaces for you. Then you just need to fill in all the implementations. At least, that is the promise. As I say, I studied this as part of a postgraduate computing course. It was of some academic interest, but I have never seen anyone write software this way (though a some people do, if the references in the course are to be believed). I expect more people have bought expensive MDA tools than have actually used them. In a previous generation, the same was true of CASE tools that also failed to live up to their promises.

So what, if anything, can this tell us about learning design? Well, I can see exactly the same split happening. There will be hype about magic systems where you input your learning outcomes, and sketch your learning design, press a magic button, and hey, presto, there is your Moodle course. It won't work outside of research labs, but some vendors will try to commercialise it, and a some institutions will fall for it and end up with expensive white elephants.

On the other hand, it would be good to see a common notation emerge to represent learning designs. This would help teachers communicate with each other, and perhaps with students, about how their teaching is supposed to work. A good feature of UML is that it is really very natural. Most developers can understand most of a UML diagram without having to be taught a lot of rules. There are several types of diagram to represent different things, but they are the kinds of things people drew anyway before UML was invented. The creators of UML just picked one particular way of drawing each sort of diagram, and endorsed it, in an attempt to get everyone talking (drawing) a common language. If you want to draw highly detailed UML diagrams, you need to learn a lot of rules, but you can get a long way just by copying what you see other people do, which is a sign of an effective language. It would be nice to see such a language for communicating about learning.

The new question engine - how it works

In my last blog post I promised more details of the new question engine "in a week or so". Unfortunately, things like work (mainly fixing minor bugs in the aforementioned question engine); rehearsing for a rather good concert that will take place this Friday; and buying curtains for my new flat, have been rather getting in the way. Now is the time to remedy that.

Last time I explained roughly what a question engine was, and that I had made big changes to the one in Moodle. I now want to say more about what the question engine has to do, and how the new one does it.

The key processes

There are three key code-paths:

To display a page of the quiz:

1. Load the outline data about the student's attempt.
2. Hence work out which questions are on this page.
3. Load the details of this student's attempt at those questions within this quiz attempt.
4. Get the display settings to use (should the marks, feedback, and so on be visible to this user).
5. Taking note of the state of each question (from Step 3) update the display options. For example, if the student has not answered the question yet, we don't want to display the feedback now, even if we will later.
6. Using the details of the current state of each question, and the relevant display options, output the HTML for the question.

The bits in italic are the bits done by the quiz. The rest is done by the question engine.

To start a new attempt at the quiz:

1. From the list of questions in the quiz, work out the layout for this attempt. (This is only really interesting if you are shuffling the order of the questions, or selecting questions randomly.)
2. Create an initial state for each question in the quiz attempt. (This is when things like the order of multiple choice options are randomised.)
3. Write the initial states to the database.

To process a student's responses to a page of the quiz.

1. From the submitted data, work out which attempt and questions are affected.
2. Load the details of the current state of those question attempts.
3. Sort all the submitted data, into the bits belonging to each question. (The bits of data have names like 'q188:1_answer'. The prefix before the '_' identifies this as data belonging to the first question in attempt 188, and the bit after the '_' identifies this as the answer to that question.)
4. For each question, process its data to work out whether the state has changed and, if so, what the new state is. This is really the most important procedure, and I will talk more about it in the next section.
5. Write any updated states to the database.
6. Update the overall score for the attempt, if appropriate, and store it in the database.

These outlines are, of course, oversimplifications. If you really want to know what happens, you will have to read the code.

There are other processes which I will not cover in detail. These include finishing a quiz attempt, re-grading an attempt, fetching data in bulk for the quiz reports, and deleting attempts.

The most important procedure

This is the step where we take the data submitted by the student for one particular question and use it to update the state of that question.

Moodle has long had the concept of different questions types. It can handle short-answer questions, multiple-choices questions, matching questions, and so on. Naturally, what happens when updating the state of the question depends on the question type. That is true for both the old and new code.

Now, however, there is a new concept in addition to question types. The concept of 'question behaviours'.

In previous versions of Moodle, there was a rather cryptic setting for the quiz: Adaptive mode, Yes/No. That affected what happened as the student attempted the quiz. When adaptive mode was off, the student would go through the quiz entering their response to each question. Those responses were saved. At the end, they would submit the quiz and everything would be marked, all at once. Then the student could review their attempt (either immediately, or later, depending on the quiz settings) to see their marks and the feedback. When adaptive mode was on, the student could submit each question individually during the attempt. If they were right first time, they got full marks. If they were wrong, the got some feedback and could try again for reduced marks.

The problem with the previous version of Moodle was the way this was implemented. There was a single process_responses function that was full of code like "if adaptive mode, do this, else do that". It was a real tangle. It was very common to change the code to fix a bug in adaptive mode (for example), only to find that you had broken non-adaptive mode. Another problem was the essay question type, which has to be graded manually by the teacher. It did not really follow either adaptive or non-adaptive mode, but was still processed by the same code. That lead to bugs.

A very important realisations in the design of the new question engine was identifying this concept of a 'question behaviour' as something that could be isolated. There is now a behaviour called 'Deferred feedback' that works like the old non-adaptive mode; there is an 'Adaptive' behaviour; and there is a 'Manually graded' behaviour specially for the essay question type. Since these are now separate, you can alter one without risking breaking the others. Of course, the separate behaviours still share common functions like 'save responses' or 'grade responses'. We now also have a clean way to add new behaviours. I made a 'certainly-based marking' behaviour, and a behaviour called 'Interactive', which is a bit like the old Adaptive mode but modified to work exactly how the Open University wants.

It takes two to tango, and three to process a question

In order to do anything, there now has to be a three-way dance between the core of the question engine, the behaviour and the question type. Does this just replace the old tangle with a new tangle (of feet)? Fortunately there is a consistent logic. The request arrives at the question engine. The question engine inspects it, and passes it on to the appropriate behaviour. The behaviour inspects it in more detail, to work out exactly what need to be done. For example is the student just saving a response, or are they submitting something for grading. The behaviour then asks the question type to do that specific thing. All this leads to a new state that is passed back to the question engine.

So, the flow of control is question engine -> behaviour -> question type except, critically, in one place. When we start a new attempt, we have to choose which behaviour to use for each question. At this point the question engine directly asks the question type to decide. Normally, the question type will just say "use whatever behaviour the quiz settings ask for", but certain question types, like the essay, can instead say "I don't care about the quiz settings, I demand the manual grading behaviour."

If you like software design patterns, you can think of this as a double application of the strategy pattern. The question engine uses a behaviour strategy, which uses a question type strategy (with the subtlety that the choice of behaviour strategy is made by the question type).

Summary

So that is roughly how it works. A clear separation of responsibility between three separate components. Each component focussing on doing one aspect of the processing accurately, which makes the system highly extensible, robust and maintainable. Of course, everyone says that about the software they design, but based on my experiences over the last year, first of building all the parts of the system, and then of fixing the bugs that were found during testing, I say it with some confidence.

What I'm working on

I've been meaning to post here for a while about what I am working on, but I keep posting in the Moodle Quiz Forum instead, because what I have to say seems more relevant there.

Anyway, the short answer is that I am rewriting the Moodle Question Engine. That document explains the what and the why. I want to post here about the how.

The question engine is the part of Moodle that processes what happens as a student attempts questions as part of a quiz (or other activity). What makes this a relatively difficult problem is that there are two independent degrees of variation. A question may be one of many different question types (multiple choice, short-answer, essay, ...) and there are various ways that a student may interact with it (adaptive mode, non-adaptive mode, manually graded, ...). Plus, people seem to think that when quizzes are used for summative assessments, it is quite important that there are no bugs and that performance is good ;-).

The complexity of the problem, and my natural bias, means that I am taking a very object-oriented approach. More so than is normally used in Moodle. Separate objects, with clearly defined responsibilities, helps me understand the problem and have confidence that what I am doing will work.

If you read the document linked to above, you will learn that I have been dissatisfied with this part of the quiz code for years, however it is really only within the last year that I have worked out the overall approach to solving this problem nicely. The key realisation is that you need to apply the strategy pattern twice, once for the bits that vary with the question type, and once for the bits that vary with the interaction model. The other important part of the plan is a cleaner database structure that more closely maps to data we need to store.

However, to implement all that requires quite a lot of code. I need to build the core system that can support all the different interactions and question types. Then I need to create all the interactions to replicate Moodle's existing functionality and the new features we want to add (Certainty Based Marking, and what we are calling the Interactive mode). Then I need to modify all the existing Moodle question types to fit into the new system. And I need to do all this with robust bug-free code that we can confidently use for summative assessment.

Over the last year I become increasing coverted to test-driven development. Certainly I wrote a lots of tests while eating my elephant, and this time round I am being very thorough with testing, although I must admit I still don't always write the tests first.

I am also proceeding very iteratively. I started with a test that took a single true-false question through being attempted according to the simplest of the interactions, and wrote some code that used roughly the right classes to make that test pass. Then for about a week I did little more than refactor that code.

It turned out that most of my variables and classes had the wrong name. For example, at one point I had classes called question_state and question_states, which was clearly a recipe for confusion. It was only after I had working code that I was able to think of the name question_attempt_step (one step in a question_attempt) for one of those. I also came to realise that grades were stored in three separate ways in different places, and so I ought to make sure I was using a consistent naming scheme for my variables. Then I did a bit of moving methods between classes as the exact boundaries of responsibility of different classes became clearer. Should I have been able to get all that right first time? Well I think that would be impossible.

I also started gradually removing simplifying assumptions that worked for my initial test true-false question, but not more generally. That was the point at which I started extending my code to other question types and interactions. For example I wrote a test case a walking an essay question through the manual grading interaction, and exposed some new issues that had to be accounted for; and just today I introduced multiple-choice questions, which randomise the order of the choices when you start an attempt, but then you have to store that random order somewhere so it can be used throughout the attempt. I have also already implemented the certainty based marking interaction.

There is still much to do. My code that outputs the HTML for a question is still a rough hack of the current code that lets the tests pass. I need several rounds of refactoring before that will be cleaned to my satisfaction. There are sill five more interactions to write, although each one is getting quicker than the one before. Then I have to convert all the existing question types. I need code that stores the current state of everything in the database and later reads it back. Finally, the nasty bits, which are upgrading from the current database structure to the new one, and re-implementing backup and restore, including restore of old backups.

If you were paying attention in that last paragraph, you will have deduced that my code does not yet store anything in the database! I am taking an approach with the domain objects completely unaware of the database (which makes testing very easy) and I am planning to use the data mapper pattern to coordinate loading and saving the necessary data efficiently. I am pretty sure I know how to do that, however, I have not tried to write it yet because I want to re-read the relevant chapters of Patterns of Enterprise Application Architecture, and my copy only recently got back from Australia, and I won't be able to retrieve it from my parents' house until this weekend.

In terms of time-scales, I am really hoping to get this done by about Christmas, and in time to go into Moodle 2.0, since it breaks a number of APIs. However, I first have to implement this in the OU's Moodle 1.9 code-base and then port it to Moodle 2.0, so no promises. This may have to wait until Moodle 2.1.

Anyway, that is what I have been doing, and so far it has been very enjoyable. Write tests, make them pass, then refactor aggressively with the tests as a safety net is an approach that is working really well for me. These are quite big scary changes, but right now I am feeling confident that it will all work out, and we will end up with a really solid question engine upon which to base an enhanced quiz and other Moodle activities.

PHP global variables are not necessarily evil

Global variables in software are generally a bad idea. They serve to magically teleport information from one place in a program to another in a way that is very hard to follow. Thus they cause subtle bugs, make maintenance difficult, and kill kittens.

In a simple program, it is normally easy to avoid global variables. When you call a function or method to get it to do something for you, you can probably just pass all the necessary inputs to the function as parameters, and get all the results back in the return statement. The flow of data is easy to follow.

However, in a large, complex system, that may become untenable. For example many, but not all functions in Moodle need a database connection to get information or update it. Does that mean all functions in Moodle end up taking a $db parameter, whether they need them or not, just so they can pass it on to any functions they in turn call? Well, we don't do that, because that way lies madness. Instead we have a 'global' $DB object, and that is not all. We also have $CFG, $COURSE, and so on. Imaging if you needed many of those inside your function. Calling it with all those parameters would be a pain.

Now, I just put the word 'global' in quotes, why was that? Well, Moodle is a web application, so the process that is running is a web server, probably with multiple threads executing simultaneously. A true global variable would be accessible from all threads, and last forever. Something you declare in PHP as a global is only accessible for the duration of one server request, and not accessible from any other thread. In Java, that would be called a ThreadLocal, not a global.

So, a PHP global is not truly a global. However, if abused, it can still be a way to teleport data from one place to another during the processing of a single request. The point I want to make is that there are non-evil ways to use it. In Patterns of Enterprise Application Architecture Martin Fowler describes the Registry pattern (sorry, that online summary is inadequate). That is an object in your program which all the other parts can get hold of, and from which they can get the service objects, like the database connection, that they must depend on. In web applications, you often want dependancies with thread local scope. My argument is that the PHP global keyword is a language feature that implements a thread-local registry for you with no effort. Thus it is a good thing if used properly.

What is using it properly? Well it all depends on the kind of things you store there. Are you making common dependancies easy to find, or are you magically teleporting information. I think that Moodle does mostly use this feature properly. Our global variables like $DB, $CFG and $COURSE are things that can legitimately be stored in and accessed from a Registry. However, we have some horrors, like $CFG->pagepath and $CFG->stylesheets, which I am currently trying to exorcise from Moodle 2.0.

One advantage of storing your dependencies in a Registry, as opposed to, say, making them singletons, is that it makes them substitutable. This becomes important when you try to write unit tests. When unit testing, you often need to switch in a test double in place of one of the dependencies. Well, with a Registry you can swap one in during test set up, and switch it back out during tear down. I have been doing that a lot recently in Moodle, and it works. (You could not do it, for example, when you accessing the database in Moodle 1.9. There, the database connection was hidden in functions like get_records, and could not be substituted. Now we have $DB->get_records with $DB in the 'global' registry where it can be substituted.) Some other global-like mechanisms, like static methods or singletons are not substitutable, and so make testing much harder, if not impossible.

Of course, some people would argue that you still should not use the global keyword, that instead you should implement your own Registry class (an example). I disagree. Appropriate use of language features tends to create more idiomatic code.