Releases

Good news, everyone! We’ve found a way to get useful information about attempts on Numbas tests out of Blackboard, and present it like this:

The current situation at Newcastle is that all of our in-course Numbas tests which count towards credit are run through a Moodle server set up specifically for the purpose, even though our institutional VLE is Blackboard.

The reason for that boils down to the fact that Blackboard doesn’t make it easy to analyse data to do with SCORM packages: the built-in SCORM reports don’t give much useful information and are tedious to generate, and it’s unclear where in the database the SCORM data lies. If a student claims that their answer was marked incorrectly, we have no way of checking it because Blackboard only gives you the student’s reponse to an interaction, and not the expected response. And sometimes that’s not enough: it’s much easier to work out where a student’s gone wrong if you can load up the test as they saw it. SCORM has a review mode which does that, and while I was able to add support for that to the open-source Moodle server, Blackboard is a black box and brooks no intervention.

Can you work out where this student went wrong?

Read the rest

Development of Numbas has continued apace over the Summer break. I’m about to go on holiday for a couple of weeks, so I thought I’d write a development log to keep you up to date with all the latest changes.

The biggest change is that I completely rewrote the default theme to use the Bootstrap framework. As well as making everything look more “modern”, it should make using Numbas on smaller screens a lot easier. When the screen is below a certain width, the question list collapses into a sliding menu, which you can reveal by clicking on the icon at the top left of the screen. The old layout, with all the navigation bumped to the bottom, led to a lot of scrolling up and down.

Other changes

The Numbas runtime:

• New JME functions: len(set) (code), reverse(list) and indices(list,value). (code, documentation)
• There’s now a version of the table function which doesn’t require a list of column headers. (documentation)
• The shuffle function now works on ranges as well as lists. (code, documentation)
• “Match choices with answers” parts now have a couple of layout options, which let you remove certain elements from the grid. You might want to do this when your grid is symmetric (for example, when asking the student to state which elements of a set are equivalent to each other). (code, documentation)
• Fixed a bug in the random number generator seed which caused a warning in Chrome. (issue)
• The code to count significant figures in a number now copes with E notation. (issue)
• The value of cmi.session_time in the SCORM data model is now set properly (code)
• The “noLeadingMinus” simplification rule rewrites -0 to 0. (code)
• Added an option to not show the results page when the exam is finished. (issue)
• When a question only has one part, there’s no longer a “submit part” button in the part feedback box. Instead, there’s just the “submit answer” button at the bottom of the question. (code)
• As part of the groundwork for enabling adaptive marking, part objects now have a getCorrectAnswer method which returns the correct answer to the part in a given scope. (code, documentation). Each part also has a method studentAnswerAsJME which returns the student’s answer to the part as a JME data type. (documentation)
• Part marking scripts need to store some information which the validation script uses to decide what feedback to give. This should now be stored in this.validation. (code)
• Fixed a bug in Numbas.jme.display.mergeRuleset which led to some rules going missing. (code)
• If the “minimum/maximum number of marks” options in a multiple choice part are empty, use 0. (issue)
• The JME function zip(lists) no longer gets stuck in an infinite loop if you give it no arguments. (code)
• There’s now a function Numbas.jme.tokenToDisplayString which turns a JME token into a representative string, and a dictionary Numbas.jme.typeToDisplayString which defines how that behaves for each data type. (code, documentation)
• Fixed a bug where names of expected variables in “mathematical expression” didn’t have excess whitespace trimmed. (issue)
• Added a function Numbas.util.nicePartName which gives a human-readable identifier for a part. (documentation)
• Nested unary minus and plus now get brackets around them when rendered as LaTeX. (issue)
• The source code for each part type is now in a separate file, and the builtin JME functions are in a separate file to the core JME interpreter. This should make the code easier to maintain. (code)
• Added display-only JME functions sub(name,index) and sup(name,index) to display variable names with arbitrary subscripts or superscripts. (code, documentation)
• The logic around marking parts with zero marks has changed so we can give more useful feedback. Previously, parts with zero marks available just weren’t marked, but sometimes you want a part to be marked for adaptive marking, or just to get some feedback. Parts with zero marks now show a tick or a cross, even when they don’t contribute to the total score. (code)

The Numbas editor:

• Fixed the logic to decide when to show the delete button for questions and exams. (issue)
• Fixed a bug where links to pages on the same domain as the Numbas editor were made relative. (issue)
• Added a lot of links to help pages that had been missing from the various part editor tabs. (issue)
• Fields which take a JME expression use a monospace font, so they’re more readable. (issue)
• Question and exam descriptions are sanitized to remove bad HTML. (somebody put a whole form element in their description!) (code)
• The question search page now takes an exclude_tags parameter so you can exclude questions with certain tags. There’s no user interface for this yet. (code)
• The sorting of tags on the question edit page is now case insensitive. (issue)
• Added a feedback label “Needs to be tested”. (code, documentation)
• Fixed a bug involving custom functions whose names contain a capital letter. (issue)

A long-standing limitation of Numbas has been the inability to offer “error-carried forward” marking: the principle that we should only penalise them once for making an error in an early part. When calculations build on each other, an error in the first part can make all the following answers incorrect, even if the student follows the correct methods from that point onwards.

Numbas now has a system of adaptive marking enabled by the replacement of question variables with the student’s answers to question parts.

Read the rest

We’ve just updated the Numbas editor with another long-awaited feature: a “shopping basket” to keep questions while you browse the database. You can create an exam containing the questions in your basket with one click, or select individual questions from the basket inside the question editor. This resolves a long-standing problem with the exam editor – when there can be dozens of questions with the same name, and copies of those questions, it was very hard to find the questions you want in the miniature search interface included in the question editor.

I’ve updated the “create an exam” tutorial with instructions on how to use the new “shopping basket” feature.

Read the rest

It’s a good idea to work as part of a team when writing a Numbas question: you can proofread each other’s questions, suggest changes, and give the final OK when a question works. I’ve just added a few new features to the Numbas editor which make collaborating with others a bit easier. Read the rest

We’ve just updated the Numbas editor with a “Licence” field on both questions and exams, which lets you specify the licence under which you allow other people to use your work. To begin with, you can pick from “All rights reserved” or any of the Creative Commons licences.

When you search for questions or exams, you can filter the results by what you’re allowed to do with them – freely reuse, modify, or reuse commercially. If you apply any of these filters, content with no licence specified will be excluded, so please make sure you specify a licence for your material!

Happily, everything produced at Newcastle University and uploaded to the mathcentre Numbas editor is available under a CC BY (free to reuse with attribution) licence.

Note that these licences aren’t enforced by the software – they’re just there to help people do the right thing when compiling open-access resources. If you don’t want your material to be publicly available, the solution is still to use the access controls to make your questions and exams hidden by default.

Since I’m here, I’ll do a general development log as well. Read the rest

We’ve got a pretty big new feature for Numbas this week – a whole new part type! The matrix entry part asks the student to enter a matrix.

In the past, we’ve hacked together questions where we ask the student to enter a matrix by putting a load of gapfills in a table. That was very laborious to set up, and the user experience for the student wasn’t particularly great.

The matrix entry part is meant to make all this a lot smoother – you give a JME matrix variable as the correct answer, and the student is shown a (optionally resizable) grid of cells to enter their answer.

It works like this:

The student picks the size of their answer, then enters numbers in each of the cells. It’s marked as correct if the matrix they give exactly matches the one you give.

If you want, you can lock the size of the answer matrix, so the student doesn’t have to be shown the size picker interface. That looks like this:

There are a few other options:

• You can award the marks proportionate to the number of correct cells, instead of a simple pass/fail.
• You can mark cells as correct if they’re within a given margin of error of the correct answer.
• The precision restriction options from the number entry part are also available here.
• You can allow the student to enter their answers as fractions (this is also available for number entry parts now)

You can see all that in action in this demo exam I created, containing a few simple-ish matrix entry questions. And of course, there’s documentation.

One thing that many people have asked for over the years is the ability to check whether the variables generated by a question have a desired property, and “re-roll” them if not. I’ve been reluctant to implement this because it can lead to sloppy question design when a method that’s guaranteed to work can often be found with a little bit of thinking.

Read the rest

Here’s another round-up of recent development work on Numbas.

Version tracking

The big new feature is version tracking – every change made to a Numbas question or exam is saved in the database, and you can revert to old versions in the new “Editing history” tab. Read the rest

Here’s another round-up of the development work on Numbas in the past month-and-a-bit.

A little attribution

I’ve added a small attribution footer to the bottom of the default theme. It looks like this:

Now that people can upload their own themes, we wanted an unobtrusive way of noting that tests are built using Numbas. If you don’t want it there, you can use a modified theme to remove it, but we’d prefer you kept it. Read the rest