Numbas v5.0

It’s time for another major version of Numbas. This release includes a radically new way of presenting questions, as well as a few other helpful new features.

We’ve made a short video to demonstrate the new features:

We’ve also made a demo exam so you can try it out yourself.

Explore mode

“Explore mode” is a completely new style of question. Rather than presenting the student with a fixed list of parts that they must answer, in explore mode the student can choose their own path through a question, with the available routes depending on how they have performed on previous parts.

My initial mental model for explore mode was a “choose your own adventure” book: the student would be given a prompt and some activity, and then they could choose where to go next from a list of options.

I then wrote down a few use cases to guide the design:

• Offer a selection of hints at varying degrees of helpfulness.
• Walk the student through an iterative algorithm, giving feedback on each step.
• Allow the student to choose the method they want to use to solve a problem.
• Take free input from the student, such as measurements or an example of an object, then ask them questions about it.
• Ask the student to define the criteria for a test, then assess their decisions based on those criteria.

Writing “Explore” questions is a bit more complicated than “Show all parts” questions – you have to set up how the parts link together, think about how variable replacements will work, and come up with a sensible marking scheme. I’ve tried to make the editing interface as intuitive and helpful as possible.

This new mode enables a huge range of new activities in Numbas. I can’t wait to get started writing more questions in explore mode, and to see what others come up with.

See the documentation on explore mode.

Alternative answers

A long-standing difficulty of Numbas has been recognising specific incorrect answers and giving tailored feedback. Similarly, the standard marking for most parts assume there’s only one correct answer, up to some kind of equivalence.

“Alternative answers” lets you set up copies of a part, with their own settings and expected answer, number of marks to award and a specific feedback message. When the student submits an answer, the version of the part that matches best is used.

Alternative answers are good for situations where you have a fixed number of cases that you want to check for.

Use cases for alternative answers:

• Accept any of several correct answers, for example \(x\) or \(-x\), or a precise answer and one obtained by rounding in intermediate steps.
• Award partial credit for an answer that is ‘nearly right’.
• Give a relevant feedback message for an answer corresponding to an error you expect the student to make (commonly known as ‘mal-rules’). The paper Diagnosing student errors in e-assessment questions makes the case for catching these.

See the documentation on alternative answers.

Menu mode

This is a small change in terms of functionality, but represents a change in how Numbas material can be presented.

There is now a “Question navigation mode” option for exams. The “Sequential” option is the classic presentation, with a list of numbered questions displayed in a sidebar. However, the “Choose from a menu” mode presents the student with a table of available questions, which they can pick and choose from. By default, the question’s name as it appears in the editor is used, but you can set a different name if you want.

The idea is that this mode is more appropriate for material which students access for revision or practice, to make it easier for them to pick just the subjects that they want to work on.

There’s no “End Exam” button in menu mode – menu mode is for purely formative use and each question should stand on its own.

See the documentation on menu mode.

Folders

Organising content in the Numbas editor has always been a challenge. We’ve made it a little bit easier to arrange content within projects with the addition of folders.

Rather than searching for an item by name, you can now navigate through a folder hierarchy.

See the documentation on organising content inside a project.

Other changes

Accessibility

• There is an accessibility statement for Numbas exams, and another one for the Numbas editor.
• Part input warnings now have the “alert” role. (code)
• The Numbas logo now has alt text. (code)
• Input boxes have title attributes, and choices for multiple-choice parts are grouped in <fieldset> tags. (code)
• The default theme and the Numbas editor use aria-live throughout to alert accessibility tools to changes in the interface. (code)
• The default theme respects the browser’s prefers-reduced-motion setting, avoiding transition effects. (code)
• The colours of some interface elements have changed to improve contrast, to meet WCAG AAA standards. (code)
• The tick/cross feedback icons are hidden from screenreaders, instead there’s some screenreader-only text of the form “Your answer is correct”, etc. (code)
• Input boxes have autocapitalization turned off for mobile keyboards.

Numbas runtime

• filter accepts any list-like object. (code)
• The new function prod computes the product of the elements of a list of numbers. (documentation, thanks to Grant Gollier)
• The new function split_regex splits up a string at every occurrence of a substring matching a given regular expression. (documentation)
• The new function substitute substitutes values for free variables in an expression. (documentation)
• The new function diff differentiates an expression with respect to a given variable. (documentation)
• The function latex can now take a sub-expression, which will be converted to a string of LaTeX code. (documentation)
• The ‘relative tolerance’ and ‘absolute tolerance’ arguments for isclose are optional, with default values of \(10^{-15}\).
• A new “Show results page” option for exams specifies when the results page is shown. The options are “never”, “when entering in review mode”, and “on completion”. (documentation)
• In a gap-fill part with only one gap, no gap name header is shown in the feedback. (code)
• Parts which use adaptive marking have their ‘expected answer’ updated when the parts referred to in adaptive marking are submitted. (code)
• You can now apply a penalty when adaptive marking is used. (documentation)
• The standard marking algorithm for “mathematical expression” parts no longer checks that the student uses exactly the same set of variables as the expected answer. (code)
• SVGs embedded in content areas with <img> tags are replaced with <object> tag equivalents so that they scale nicely. (code)
• There is an exam-wide “enable steps” option which you can use to turn off steps for all questions. (documentation, issue)
• The initial value of the seed for the random number generator is saved in the SCORM suspend data. (code, thanks to Danil Polikarik)
• The displayed percentage total for the exam is always rounded down, so that it doesn’t look like you’ve achieved a threshold when you haven’t quite. (code, issue)
• There’s a flatFractions flag for displayed expressions, which renders fractions on one line instead of stacked. (documentation, thanks to tpilewicz)
• The ‘mathematical expression’ part now automatically detects expressions in the form of a formula, and sets up the variable value generators accordingly. (code)
• The “printed worksheet” theme displays the number of marks available for each part. (code)
• Exams can now be created from a JSON description, without a display. This was already possible for questions and parts. Numbas exams can now run headlessly, for example for remarking an attempt after a bug is fixed. (code)
• The “printed worksheet” theme now displays the exam’s introduction text before the questions.
• There are some new signals under Numbas.signals: ‘exam ready’, ‘display ready’, and ‘Numbas initialised’. (code)
• The operation e^x uses the built-in JavaScript function Math.exp(x) rather than Math.pow(Math.E,x), so it produces the same result as exp(x). (code)
• Lists and dictionaries can have a trailing comma just before the closing bracket, e.g. [1,2,3,]. (code)
• The “mathematical expression” part has some new settings to do with juxtaposition of names: “force single letter variables names?”, “allow unknown functions?”, and “use implicit function composition?”. (documentation)
• There are some new exam-level options to do with feedback shown in review mode: “show score in review mode?”, “show part feedback messages in review mode?”, “show expected answers in review mode?” and “show question advice in review mode?” (documentation)
• Variable names with different annotations are considered to be different. (documentation)
• The ‘matrix entry’ part has minimum/maximum number of rows/columns settings. (documentation)

Numbas editor

• There is now an “explore” page which shows public projects and recently published questions and exams. (See it on numbas.mathcentre.ac.uk)
• There’s now a web-based editor for extensions, and documentation for extensions can be displayed within the editor. (documentation)
• You can grant other users access to view or edit your extensions, similar to exams and questions. (documentation)
• In the marking algorithm testing panel, the order of variables follows their order in the variables tab. (code)
• The variable editor shows where in the question the current variable is used, such as content areas and part settings. (issue)

• Headers for editor tabs are styled differently when they’re “in use” – when any of the settings in that tab are different to the defaults. (issue)
• You can now download and upload custom part type definitions. (code)
• Content areas only become editable when clicked. This greatly improves the performance of the editor. (code)
• The editor homepage displays a “tip of the day”. (issue)

• You can only view an exam if you can view all of its questions. (issue)
• The “editing history” tab shows the last feedback label set by each person who has set a feedback label. (code)

• You can now choose to receive an email each time you get a notification on the editor, such as for feedback on an item.
• When you open a notification about feedback on an item, you are taken straight to that item’s “editing history” tab. (code)
• On the part marking algorithm tab’s marking test panel, the expected answer is shown above the input for the student’s answer. (issue)

Localisation

Since v4.0, Numbas has been translated into Hebrew, Turkish, Vietnamese and Chinese, bringing the total up to 18 languages.

As ever, there are new pieces of text in the Numbas interface that need to be localised. If you speak a language in addition to English, please go to our localisation page and write some translations.

Contributors

Thanks to the following people for contributing code to Numbas in the last year:

• Christian Lawson-Perfect
• Chris Graham
• David Hadas
• Grant Gollier
• Danil Sergeev
• tpilewicz

There are always plenty of opportunities for you to start contributing to Numbas. See our page on contributing to Numbas for more information. Our GitHub issues labelled “good first issue” are a good place to start.

Give it a go!

You can use all these new features right now on numbas.mathcentre.ac.uk. If you run your own instance of the Numbas editor, follow the upgrade instructions in the documentation.

If you have any questions about the new features, or spot a bug, you can post on the Numbas user group, file an issue on GitHub, or email us.