Development update, September 2024: Numbas v8.0

Here’s an update on Numbas developments since March. We made a separate post announcing the release of v4.0 of the Numbas LTI provider.

We’ve just released v8 of Numbas. There are a couple of technically small but highly noticeable changes to the Numbas runtime: auto-submission of answers and a revamp of the exam feedback settings.

A video showing the changes in Numbas v8.0.

Automatic submission of answers

The most common issue that students have with the Numbas interface is that they either forget to submit answers, or think that they should only submit once at the end of the exam.

It’s been a principle of the design of Numbas from the start that students can submit answers as many times as they like with no penalty. Students can make mistakes and should be able to change their minds at any point until the whole assessment is finished. Different people interpret the word “submit” in different ways, and students have different exam techniques.

We’d like to be as generous as possible when marking a student’s work: it isn’t right that a student should lose marks because of how they interact with the system.

For a long time we’ve been thinking about whether it would be possible to automatically mark students’ answers, instead of asking them to press a “submit” button. It’s important that students understand what they need to do, and that they’re given feedback that makes it clear what the system has done with their input.

While talking about our late work policy, we decided that the marks for a Numbas exam should match the current state of the answer inputs at any point in time. That meant that we’d have to mark answers automatically.

I hope that students will find this better than the old process: they can work through an exam, and their answer for one part is marked as soon as they move on to the next part. Or, there’s still a “save answer” button which forces the answer to be marked, which might be psychologically important to some students.

In order to avoid making the subsequent content move around as feedback for previous parts is added to the display, the feedback for automatically-marked parts is initially hidden and can be revealed by clicking a “show feedback” button. If the student clicks the “save answer” button, the feedback is shown immediately. Our thinking is that if a student has moved on to the next part, we can presume they’re not interested in reading the feedback for the previous part right now. They’ll still see the tick or cross and number of marks awarded, if those are enabled by the exam author.

Revamped feedback settings

The number of different settings controlling what feedback students get has grown over the years, and had become an unwieldy list of checkboxes and drop-downs, with names that were quite hard to interpret.

We’ve rearranged these settings into a single grid, allowing you to control which stage of the exam the student will be shown each kind of feedback, if at all.

Additionally, we’ve split the “review” stage after the exam is finished in two: once you end the exam, it is “submitted”, and there is a subsequent “review” stage after that where you might show the student more feedback. This expands on the old “reveal answers to all questions” setting, which controlled a lot of things at once.

So you can allow a student to see their scores immediately after they finish the exam, but not reveal the expected answers for each part until a deadline has passed, for example.

There are a few new combinations of settings that weren’t possible with the old settings, but all existing exams will continue to behave as they did before the update.

Numbas runtime

Enhancements

  • Added a new JME function escape_html function which takes a string and escapes any HTML special characters. (code)
  • The image lightbox is now activated by pressing a button which appears over the top-right of each image. (code, issue)
  • There is a new data type representing anonymous functions (lambdas). Functions which take an expression as an argument, such as map and filter, now also take anonymous functions. (code, issue, documentation)
  • The mathematical expression part now has an option to warn the student that their answer doesn’t match the required pattern before submission, or to prevent submission. (issue, documentation)
  • There is a new function ExamDisplay.addQuestion, which is called when a question is added to the exam. The theme can use this to do question-specific things. (code)
  • There is a new optional built-in constant to use j as the name for $\sqrt{-1}$, the imaginary unit. (issue)

Bug fixes

  • Scope.deleteVariable also deletes constants with that name. (code, issue)
  • Fixed localisation of decimals in tokenToDisplayString and treeToJME. (code, issue)
  • Fixed a typo in the vector(list) function. (code)
  • Numbas.jme.calculus.differentiate runs basic simplification rules before differentiating (code)
  • The answered state of parts is shown even when marks are not shown. (code, issue)
  • Mathematical expression part: the check if the student’s answer is empty takes place in a separate note. (code)
  • Gap-fill parts are invalid if any gap is invalid. (code, issue)
  • simplifyFractions doesn’t lose extra terms in the denominator. (code)
  • The list of displayed part feedback messages is only cleared if there really are none. (code, issue)
  • Numbas.display.typeset waits until there’s a JME scope. (code)
  • Word-breaking is allowed in some more parts of info pages. (code)
  • A custom JavaScript marking script which runs after the built-in one doesn’t run if the original script was waiting for pre-submit tasks. (code, issue)
  • The build time is added as a query parameter for scripts.js and styles.css, so they’re not cached from old builds. (code)
  • Part.submit saves best_alternative and script_result on the part. (code, issue)
  • Some speed optimisations for the JME parser, particularly to do with variable names. (code)
  • Fixes to do with scientific notation. (code, issue)
  • The slideVisible and fadeVisible binding handlers in the default theme have been removed. (code, issue)
  • createPartFromJSON runs initDisplay if there’s a question and it has a display. (code)
  • Added some margin below the intro text in menu mode. (code)
  • Added a definition of * for rational by number. (code)
  • Worksheet theme: no max-height on questions in screen media. (code)
  • Worksheet theme waits for document ready before init. (code)
  • The init routine can cope with an empty string for the SCORM cmi.entry value. (code)
  • Improved jme.find_valid_assignments so that it immediately returns the first good assignment (code, issue)
  • Numbas.jme.substituteTree makes new tree objects for each argument of for: (code)
  • Made some improvements to the routine converting number values to decimal. (code)

Extensions

  • JSXGraph: Updated JSXGraph to 1.9.2 (code)
  • Permutations: An error is thrown if you try to make an element bigger than $10^6$. (code, issue)
  • Quantities: fixed a bug with the TeX representation of percentages. (code)
  • Quantities: incompatible units can’t be compared. (code)
  • Spreadsheets: added a download_sheet function, which creates a link to download the given spreadsheet. (code, documentation)
  • Statistics: functions are marked as random or deterministic as appropriate, to help optimise attempt suspend data. (code)

Programming

  • When preloading pyodide, you can give a list of question resource files to make available. (code, documentation)
  • In R code, the character \r is replaced with \n. (code, issue)
  • Code is evaluated within a separate context for each question, so that one question’s resources aren’t available to code run in another question. (code)
  • Upgraded webR to v0.4.0. (code)

Numbas editor

  • Removed duplicate feedback in the part testing tab. (code, issue)
  • In the custom part type editor there is now a link to go back to the list of custom part types. (code)
  • The “Range of numbers” and “Random number from a range” variable data types take any expression in the input boxes. (code, issue)
  • Added “How do I” examples on using data entered by the student to mark later parts. (documentation)
  • In the question editor, the runtime question instance is remade after regenerating variables. (code)
  • The URL to fetch a zip of an exam or question is allowed from all origins, so a page on another domain can fetch the zip file of an exam. (code)
  • Form controls in the editors have ID attributes, to help assistive technology establish their label. (code, issue)
  • In the question editor, JME value previews use the question’s scope. (code)
  • Added an example of using the random function with non-integer step size (documentation)
  • Added a section on marking gap-fill answers as a set to the “how do I” page (documentation)

Numbas LTI provider

  • Fixed lockdown app links for LTI 1.3. (code)
  • AGS line items are cached unless trying to create a new line item. (code, issue)
  • Fixed the quick-query buttons on the attempt SCORM listing so that they don’t reload the page. (code)
  • LTI 1.1 launch: the instance_guid is used when getting the context for the launch. (code)
  • Added a management command to migrate user aliases from LTI 1.1 to LTI 1.3 for Blackboard. (code, issue)
  • The title of LTI 1.3 resource links is loaded from the resource link claim, if present. (code)
  • When running an attempt, the appropriate user identifier for the resource is displayed, instead of always using the sub value from the LTI consumer. (code)
  • Fixed the cookie check for LTI 1.1 links. (code)
  • Fixed the check for access changes affecting a user in LTI 1.3 resources. (code, issue)
  • The server only tries reporting scores through LTI 1.1 if there’s no LTI 1.3 link for the resource. (code)
  • LTI 1.3: the LIS claim is used for the user’s consumer user ID. (code)
  • The request to fetch the attempt data when running an attempt now returns the ID of the attempt, which might be different to the one requested if its suspend data was broken. (code)

What’s next

In the Numbas runtime, I’m working on a rewrite of the default theme’s stylesheet to support dark mode and to allow students to change all of the colours used, not just the page background and text colour. I haven’t got that ready in time for the start of our term, so it’s not included in this release.

In the LTI tool, we’ve been designing a way of enabling a “late work” policy, where students can choose to keep working or start an attempt after the first deadline, in return for a credit penalty. Our plan is to have a separate “due date” setting for resources, in addition to the “available until” date. Attempts would end automatically at the due date, but students could choose to re-open attempts and keep working on them up until the “available until” date.

We’re planning on testing this feature at Newcastle for a semester before hopefully releasing it in the new year.