The Numbas Blog

We’re happy to announce the release of a Basic LTI 1.1 tool provider for Numbas exams.

One of the more complicated parts of using Numbas is getting it to work with your Virtual Learning Environment (VLE). We designed Numbas to use the SCORM standard, which ideally would allow it to run in any SCORM-compliant VLE without any configuration or input from the server administrator. However, there have always been a couple of wrinkles in that plan: not all VLEs support SCORM, and some of those that claim to don’t do it properly.

Blackboard’s SCORM player has a few long-standing bugs and missing features which mean that we haven’t recommended it for serious use. Since we can’t fix those problems ourselves, we’ve spent a long time trying to find a way work around Blackboard’s problems. Additionally, when a large contingent of Norwegian lecturers visited us for the MatRIC colloquium this April, we discovered that very few institutions in Norway use VLEs which support SCORM. Someone suggested we look at LTI, since many more VLEs seem to support it. Read the rest

The vast majority of development work on the Numbas system has been carried out by the e-learning unit of Newcastle University’s School of Mathematics and Statistics. In particular, almost all the code was written by me.

I’ve set aside time to write up information on how you can get involved with the Numbas project. I’ve greatly expanded our “Contributing to Numbas” page, listing the different ways you can help. I’ll reproduce the top part of that page here:

We encourage any kind of contribution to the Numbas project, whether it’s a bug fix, a new feature or extension, documentation, or just a suggestion of a feature you’d like to have.

We are always interested in feedback from users, especially those with interesting ideas on how to develop and improve the system.

Here’s how you can contribute to the Numbas project

• Report bugs, or suggest features you’d like to see, on our issue trackers.
• Blog about how you use Numbas.
• Write some documentation.
• Add a feature or fix a bug by contributing code.
• Translate Numbas into your language.

Read the rest

You can now embed GeoGebra applets in Numbas questions and, using GeoGebra’s new exercises feature, award the student marks based on constructions within the applet. This is a huge step forward, making it much easier to include interactive diagrams in Numbas questions.

Here’s a video showing how to embed a GeoGebra applet in a Numbas question, and award the student marks if they complete a certain construction. There are even steps, giving marks for each stage of the process!

You can use the values of Numbas question variables in the definitions of objects in the GeoGebra worksheet, meaning that diagrams can accurately reflect the rest of your question.

In this question, the gradient of the slope and coefficient of friction are randomly generated in Numbas, then passed to the GeoGebra applet.

I’ve put together a small demo exam with a couple of questions showing some ways you can use GeoGebra inside Numbas.

To get started using GeoGebra in your own Numbas questions, read the extension’s documentation.

I’m very proud to announce the release of Numbas 2.0, which features a completely rewritten editing interface and a reorganised item database.

We’ve added some very helpful new features, and changed the way the database is organised to make working in groups much easier. All exams and questions in the editor database are now organised into projects, which provide a simple way of collecting together material relating to a particular course or activity in one place.

Projects allow you to automatically grant editing rights to a group of collaborators, keep track of changes that have been made to your content, and filter out irrelevant material. Project-level comments make it easier to coordinate writing, testing, and deployment of questions and exams with your team members.

We’ve rebuilt the editing interface from the ground up, to make it cleaner and easier to use. Read the rest

Good news, everyone! We’ve found a way to get useful information about attempts on Numbas tests out of Blackboard, and present it like this:

The current situation at Newcastle is that all of our in-course Numbas tests which count towards credit are run through a Moodle server set up specifically for the purpose, even though our institutional VLE is Blackboard.

The reason for that boils down to the fact that Blackboard doesn’t make it easy to analyse data to do with SCORM packages: the built-in SCORM reports don’t give much useful information and are tedious to generate, and it’s unclear where in the database the SCORM data lies. If a student claims that their answer was marked incorrectly, we have no way of checking it because Blackboard only gives you the student’s reponse to an interaction, and not the expected response. And sometimes that’s not enough: it’s much easier to work out where a student’s gone wrong if you can load up the test as they saw it. SCORM has a review mode which does that, and while I was able to add support for that to the open-source Moodle server, Blackboard is a black box and brooks no intervention.

Can you work out where this student went wrong?

Read the rest

Here at the School of Mathematics and Statistics at Newcastle University, we provide computer-based assessment through Numbas to all of our first- and second-year undergraduate maths modules, as well as many other courses at later stages or for other schools. We write these questions on a private Numbas editor, so that we don’t accidentally reveal content to students, and so we can control editing access through our centralised IT system.

However, it’s always been our policy that everything we create will end up on the public editor at numbas.mathcentre.ac.uk, shared under a CC-BY licence. It’s been quite a while since we last copied anything over, so I’ve spent this morning copying over just about everything we’ve got. It’s all organised under a new account, Newcastle University Mathematics and Statistics. At the moment, we’ve got 101 exams and 497 questions – that’s a lot of maths!

This academic year we’ve reinvented our first-year syllabus, so at the moment only content for our semester 1 modules is on the public database. All of our second year modules are there, however, as well as third-year courses on group theory, coding theory, and number theory and cryptography.  I’ve also transferred each of the exams we’d made available on mathcentre.ac.uk to the school account.

I’ve written a few extensions to add data types and functions necessary for a few of our more advanced courses, which are also available on the editor and on GitHub. They are:

• A data type to manipulate polynomials
• A data type and functions to work with permutation groups
• Functions to solve and demonstrate various algorithms in optimisation theory
• Data types and functions to work with linear codes

While you can pick and choose from our questions, a good way in is to look at our exams, which collect together questions on similar topics. Our first-year modules have been split up into chapters corresponding to more detailed topics, but the other courses were only collected into weekly assignments, so I’ve just made a single exam for each of those modules, containing all the relevant questions.

We’re proud to announce that we are organising an international conference on E-Assessment in Mathematical Sciences, which will take place on the 13th-14th September 2016, here in Newcastle upon Tyne, UK. Read the rest

Development of Numbas has continued apace over the Summer break. I’m about to go on holiday for a couple of weeks, so I thought I’d write a development log to keep you up to date with all the latest changes.

The biggest change is that I completely rewrote the default theme to use the Bootstrap framework. As well as making everything look more “modern”, it should make using Numbas on smaller screens a lot easier. When the screen is below a certain width, the question list collapses into a sliding menu, which you can reveal by clicking on the icon at the top left of the screen. The old layout, with all the navigation bumped to the bottom, led to a lot of scrolling up and down.

Other changes

The Numbas runtime:

• New JME functions: len(set) (code), reverse(list) and indices(list,value). (code, documentation)
• There’s now a version of the table function which doesn’t require a list of column headers. (documentation)
• The shuffle function now works on ranges as well as lists. (code, documentation)
• “Match choices with answers” parts now have a couple of layout options, which let you remove certain elements from the grid. You might want to do this when your grid is symmetric (for example, when asking the student to state which elements of a set are equivalent to each other). (code, documentation)
• Fixed a bug in the random number generator seed which caused a warning in Chrome. (issue)
• The code to count significant figures in a number now copes with E notation. (issue)
• The value of cmi.session_time in the SCORM data model is now set properly (code)
• The “noLeadingMinus” simplification rule rewrites -0 to 0. (code)
• Added an option to not show the results page when the exam is finished. (issue)
• When a question only has one part, there’s no longer a “submit part” button in the part feedback box. Instead, there’s just the “submit answer” button at the bottom of the question. (code)
• As part of the groundwork for enabling adaptive marking, part objects now have a getCorrectAnswer method which returns the correct answer to the part in a given scope. (code, documentation). Each part also has a method studentAnswerAsJME which returns the student’s answer to the part as a JME data type. (documentation)
• Part marking scripts need to store some information which the validation script uses to decide what feedback to give. This should now be stored in this.validation. (code)
• Fixed a bug in Numbas.jme.display.mergeRuleset which led to some rules going missing. (code)
• If the “minimum/maximum number of marks” options in a multiple choice part are empty, use 0. (issue)
• The JME function zip(lists) no longer gets stuck in an infinite loop if you give it no arguments. (code)
• There’s now a function Numbas.jme.tokenToDisplayString which turns a JME token into a representative string, and a dictionary Numbas.jme.typeToDisplayString which defines how that behaves for each data type. (code, documentation)
• Fixed a bug where names of expected variables in “mathematical expression” didn’t have excess whitespace trimmed. (issue)
• Added a function Numbas.util.nicePartName which gives a human-readable identifier for a part. (documentation)
• Nested unary minus and plus now get brackets around them when rendered as LaTeX. (issue)
• The source code for each part type is now in a separate file, and the builtin JME functions are in a separate file to the core JME interpreter. This should make the code easier to maintain. (code)
• Added display-only JME functions sub(name,index) and sup(name,index) to display variable names with arbitrary subscripts or superscripts. (code, documentation)
• The logic around marking parts with zero marks has changed so we can give more useful feedback. Previously, parts with zero marks available just weren’t marked, but sometimes you want a part to be marked for adaptive marking, or just to get some feedback. Parts with zero marks now show a tick or a cross, even when they don’t contribute to the total score. (code)

The Numbas editor:

• Fixed the logic to decide when to show the delete button for questions and exams. (issue)
• Fixed a bug where links to pages on the same domain as the Numbas editor were made relative. (issue)
• Added a lot of links to help pages that had been missing from the various part editor tabs. (issue)
• Fields which take a JME expression use a monospace font, so they’re more readable. (issue)
• Question and exam descriptions are sanitized to remove bad HTML. (somebody put a whole form element in their description!) (code)
• The question search page now takes an exclude_tags parameter so you can exclude questions with certain tags. There’s no user interface for this yet. (code)
• The sorting of tags on the question edit page is now case insensitive. (issue)
• Added a feedback label “Needs to be tested”. (code, documentation)
• Fixed a bug involving custom functions whose names contain a capital letter. (issue)

A long-standing limitation of Numbas has been the inability to offer “error-carried forward” marking: the principle that we should only penalise them once for making an error in an early part. When calculations build on each other, an error in the first part can make all the following answers incorrect, even if the student follows the correct methods from that point onwards.

Numbas now has a system of adaptive marking enabled by the replacement of question variables with the student’s answers to question parts.

Read the rest

Last week I gave a demonstration of Numbas at the CAA Conference in Zeist. As part of the submission process I had to submit a formal paper, which it turns out isn’t being included in the published proceedings because it was on the practice track. Instead, I thought I’d reproduce the paper here, since it offers a good, brief overview of Numbas. If you’d like the PDF version of the paper, click here.

[Could not find the bibliography file(s)Numbas is a free and open-source e-assessment system developed at Newcastle University. This demonstration highlights the key features of Numbas and the design considerations for an e-assessment tool focused on mathematics. Read the rest