Development update: August 2022

Sorry for the long gap since the last development update: the user meeting, EAMS, and work at Newcastle have consumed all of my time. Now it’s the summer, and I can take a moment to reflect on what I’ve done since March.

We had two student interns working for us for a couple of weeks in July: Will McNestry and Aleksas Bagdonas. They each contributed a few new features to the Numbas runtime, tackling some things that had been on the to-do list for a while.

I’ve mainly spent my time trudging through the ever-growing list of issues on GitHub, adding features and fixing long-standing bugs.

George Stagg left us at the end of June to work at RStudio. We’re interviewing his replacement in a couple of weeks – the new role will have a lot more time dedicated to Numbas development.

Development

The runtime API documentation is now at docs.numbas.org.uk/runtime_api, instead of GitHub Pages.

There is now a schema for the .exam format used to store exam and question definitions, at numbas.org.uk/schema.

With Aleksas and Will, I put together a new set of instructions for setting up a development environment, and making changes to the runtime. Hopefully this will make it easier for new people to get involved in Numbas development!

Unit tests and a development editor

Following a couple of accidentally-introduced bugs that hit some people just as they were running high-stakes exams, I’ve spent a lot of time looking at improving testing in the development process.

There are now unit tests for the whole life cycle of a Numbas exam, and they can be run automatically. Whenever a commit is made to the Numbas git repository, GitHub will run the unit tests and report any errors.

Since the end of June, while I’ve been cracking away at the to-do list, I’ve only been working on a development branch, so none of my changes since then are live on the mathcentre editor.

In order to give other people a chance to test new features before they go on the main editor, I’ve set up a “beta” editor at numbas.mathcentre.ac.uk/beta. This is completely separate from the main editor, so you’ll need to create a new account in order to use it. It is not intended to be robust or to store material that will be used with students; assume that anything you create there will eventually be lost.

It would really help, if you have a spare moment, to write a question or two on the beta editor.

Here’s a summary of changes I’ve made which are now on the beta editor, and will be merged in to the main branch next month:

  • I’m looking at rearranging the layout of the editor, to better use space. In the question and exam editors, the tab headings now run horizontally above the editing area, leaving more width for editor controls. It’s not finished, and I’m not committed to any of the changes, so I’d really appreciate your feedback!
The Numbas question editor, showing tab headers running horizontally.
  • There’s a little pencil icon next to the name in the question and exam editors, since it’s not obvious that you have to go to the “settings” tab to change it. (code)
  • In the question editor, only one function’s definition is shown at a time, like the variable editor. (issue)
  • When a gap is defined but not used in the part’s prompt, a warning message is shown. (issue)
  • The part’s marking algorithm and the area to test answers and run unit tests are now in separate tabs. (code)
  • The marking matrix editor for “match choices with answers” parts now scrolls if it gets too wide. (issue)
  • The code uses more recent JavaScript features, dropping support for old browsers like Internet Explorer 11, as described above.
  • You can set an adaptive marking variable replacement for a gap-fill, and it applies to all of the gaps. (issue from 2016!)
  • There is an option in the matrix entry part to pre-fill some cells. (issue from 2015!)
A 2×2 matrix input with the first row filled in with 5, 4, and non-editable.
  • Functions can be marked as ‘deterministic’ – they always produce the same output given the same input. Using this, question variables which can be deterministically reproduced from other variables are not saved to the SCORM suspend data, saving a lot of space. (issue)
  • When substituting variables into an expression value, the “plain” number notation is always used, avoiding errors when your locale’s default number notation is not the English one. (issue)
  • Number values store the precision that they represent, either as decimal places or significant figures. When they’re displayed to the student, this precision is used, including trailing zeros if necessary. (issue)
  • Improved the accessibility of modal dialogs: the header is read out, and focus is on the start of the text. (issue)
  • In exams with a time limit, the remaining time is always visible. (issue)
  • Variables are substituted into the labels on “next part” option buttons. (issue)
  • In menu mode, LaTeX in question names is rendered. (issue)
  • “Number entry” parts deal with precision restrictions better when the answer is given in scientific notation: only the precision of the significand is considered. (issue)
  • You can put 0 in the “initial number of rows/columns” settings for matrix entry parts to use the same dimensions as the expected answer. (issue)
  • Functions added by an extension are only available in questions using that extension. (issue)
  • You can turn off the hint saying “lose N marks” on “next part” buttons. (issue)
  • You can use implicit multiplication after a postfix operator. (issue)
  • In marking algorithms, you can give the apply function several names of notes to apply. (issue)
  • When pattern-matching a function name, the scope’s case-sensitivity setting is respected. (issue)
  • Added a JME function random_integer_partition, to produce a random partition of an integer into a given number of parts. (issue)
  • There is now a display flag matrixCommas to control whether commas are used between elements in the same row of a matrix or vector. (issue)
  • When a question variable has more than one definition, an error is thrown. (issue)
  • The decimal function rewrites all number literals in its definition to be decimal values, so precision isn’t unexpectedly lost by an intermediate calculation in floating-point number values. (issue)
  • Fractions with a minus sign on the denominator are accepted. (issue)
  • Simplification rules collecting numbers together avoid incorrectly producing ‘infinity’ when exceeding the maximum value representable by a floating-point number. (issue)
  • There are two new simplification rules, turned off by default, called rationaliseDenominators and reduceSurds. (issue)
  • There’s a more helpful error message when a reference is made to another part that doesn’t exist. (issue)

Numbas runtime

I’ve tagged v6.3 of the Numbas runtime on GitHub.

We made a decision to drop support for Internet Explorer, so that we can use more recent JavaScript features supported by the vast majority of browsers in use now. We decided that we would use features supported by at least 95% of browsers in use, as measured by caniuse.com.

Enhancements

  • alisahq and Yitao-Huang added an is_scalar_multiple function, for testing when one vector is a scalar multiple of another. (issue, documentation)
  • There’s now an index page in the tests directory, to make it clearer what tests are available. (code)
  • I’ve published a schema for the .exam format used to store Numbas exam and question definitions.
  • When you move to another part in explore mode, the viewport is scrolled so that the top of the part is visible. (code)
  • The printed worksheet theme doesn’t show the marks available for a part if “show maximum score” is turned off. (issue)
  • The password input field on the exam front page resizes to fit its contents. (code)
  • Empty variable definitions are silently ignored. (issue)
  • In the JavaScript API, there are now events for just about everything that happens in the Exam, Question and Part objects.
  • There’s a new simplification rule powerPower which applies the power law: (x^a)^b = x^(a*b). (code, documentation)
  • Aleksas added two new number theory functions, divisors(n) and proper_divisors(n). (issue, documentation)
  • When resuming an attempt at an exam which does not allow pausing, the screen says “Attempt resumed” instead of “Paused”. (issue)
  • Aleksas and Will added three new functions for concatenating matrices: augment, stack and combine_diagonally. (documentation)
  • The “pattern to match” setting for mathematical expression parts has variables substituted into it. (issue)
  • When you try to embed the same HTML variable in a second place, an error is thrown. (issue)
  • Lots more characters are valid in JME variable names. (issue)
  • When a part warning box would go outside the viewport, it instead sticks to the right of the viewport, so the contents aren’t squashed or hidden. (issue)

Changes

  • gcd(±infinity,?) returns 1. (code)
  • The simplifyFractions rule now also rewrites a/(b/c) to (a*c)/b, de-nesting fractions.
  • When variables are substituted into JME expressions, numbers are never written in scientific notation. (code)
  • The password input field on the exam front page no longer converts all letters to uppercase. (issue)
  • CSS files in an extension’s standalone_scripts directory aren’t collated into the combined exam CSS file. (code)
  • The wiggle room for floating-point error in the accepted values for number entry parts is now defined to be at the 12th significant figure, rather than the 12th decimal place. (issue)
  • The marking algorithm for mathematical expression parts produces a value of nothing in the studentExpr note when the student’s answer is invalid. (code)

Bug fixes

  • The print view no longer clips off questions to a fixed height. (issue)
  • The resultsEqual function deals with all number-like data types. (code)
  • Fixed a bug that led to explore mode questions being restored with the parts in the wrong order. (code)
  • The credit for a part is reset to 0 before applying feedback from an alternative answer. (issue)
  • The correct answer to a custom part type is correctly unwrapped to a JS value, no matter its data type. (code)
  • The equality test for dict values deals with the case where the second dictionary has a key that the first doesn’t. (code)
  • The simplification rule that pulls negation out of products deals with imaginary numbers. (code)
  • When finding variable substitutions in JME expressions, curly braces inside strings don’t count. I recorded myself fixing this bug. (issue)
  • Fixed a bug when adding a complex number value to a decimal value. (code)
  • decimal values are always rendered in the form dec("...") when converting a JME expression to a string. (code)

Numbas editor

I’ve tagged v6.3 of the Numbas editor on GitHub.

Enhancements

  • Anshita Baid fixed content areas so that they don’t show “click to edit” when you can’t edit them. (issue)
  • Added a page about security to the documentation.
  • Maheshwaran Dhandapani made some improvements to the header shown at the top of the screen when previewing a question or exam. (issue)
  • You can now change the order of checklist items in a queue. (code)
  • The list of items in a queue is now paginated, and you can see completed items. (issue)
  • A queue can define a list of status options for items, to help with managing the review process. (issue)
  • You can assign yourself to review an item in a queue. (issue)
  • “Extension” type parts always use a custom marking algorithm, and the checkbox is not shown. (issue)
  • There’s now an “abstract mathematical expression” data type for variable definitions, producing an expression type value, with a live preview rendering. (issue)
  • There’s a “nothing here” message when trying to add an item to a queue, with nothing in your basket or recently-viewed items list. (issue)
  • In a part’s marking algorithm tab, tick or cross icons are shown next to feedback items. (issue)
  • When you press the Enter key while inside a LaTeX expression, a line break is inserted instead of a paragraph break. This has been annoying me for almost a full decade! (issue)

Changes

  • In the question and exam editors, the “Test Run” button is now just labelled “Run”. This is more appropriate for exams using the printed worksheet theme. (issue)
  • When there’s an error in a marking algorithm, the JavaScript stack is no longer shown. (issue)

Bug fixes

  • Fixed a bug which caused an error while trying to replace a file in a theme. (code)
  • Fixed a timing issue where the question’s variables are ready before a marking test is, leading to an error. (code)
  • Newly-created unit tests no longer share their variable values with the question’s variable preview. (issue)
  • When uploading a custom part type, a unique ID is picked. (issue)
  • The “upload a part type” button on your profile’s “Part types” page is always shown, even when you haven’t created any part types yet. (issue)
  • After saving changes to an entry in a queue, you stay on that entry’s page. (issue)
  • Fixed a bug with moving items from a folder to another project. (code)
  • Fixed a bug in giving another user access to a theme or extension. (code)
  • Will fixed the queue index view so queues with no checklist items don’t show a progress bar under each item. (issue)
  • Improved the logic to automatically add variable definitions based on references. (issue)
  • Questions and exams whose names consist solely of emoji no longer cause an error. (issue)
  • Text in the code editor is no longer all bold. (issue)