Supporting Transition to University with Numbas outside the Maths Department

Chris Graham and Christian Lawson-Perfect
Newcastle University.

What is Numbas?

Numbas is an open-source system developed by the e-Learning Unit of Newcastle University's School of Maths and Stats, based on many years of use, experience and research into e-assessment.

It's aimed at numerate disciplines.

It creates SCORM-compliant exams which run entirely in the browser, compatible with VLEs such as Blackboard and Moodle.

History

Numbas follows the CALM model.

At Newcastle, we used the commercial system i-assess for six years before switching to Numbas.

Development began in 2011 with the aim of replacing i-assess.

Design goals

Scalable, reliable and accessible to a broad range of users.
Good-looking and easy for students to use.
Used by question authors who aren't experts, without support.

Open access

Compiled Numbas tests are SCORM packages: they're completely self-contained. Perfect for open access resources.

We have established a community of authors and users producing quality open-access material.

A set of free revision resources is available on mathcentre.ac.uk.

Public database

The public database at numbas.mathcentre.ac.uk contains thousands of questions created by hundreds of authors.

Organising such a large bank of questions is difficult! We've put a lot of work into making it easier.

Projects collect together material belonging to a particular course or module.

Content is hidden by default - must be explicitly published to the public database.

Current use

100+ universities in the UK and around the world.
1,000+ users registered on public editor.
3,000+ exams; 14,000+ questions.

Newcastle Maths & Stats

Summative Use:

Core modules use CBAs for assessment.
Diagnostic test for incoming students.

Formative Use: Computer-aided assessment is great for practice. Students can try randomised questions over and over until they're happy:

Bank of practice material in Blackboard, used extensively by students.
Practice for diagnostic test posted to incoming students on a USB key.

Using Numbas outside the Maths department: Challenges

Challenges:

In many other subject areas, an understanding of mathematics or statistics is critical to students' progress.
Students transitioning to University have disparate mathematical backgrounds.

Using Numbas outside the Maths department: Solutions

Solutions:

Numbas is used to tackle these challenges, with diagnostic and formative use in accounting, agriculture, biomedical sciences, business studies, engineering, psychology, physics, animal science (and growing!).
Pre-entry course for non-standard applicants to STEM subjects.
We work closely with our maths support service, Maths Aid, including on the Academic Skills Kit website

Case Study: Psychology

Cohort of 150 students.

Entry criteria: AAA/AAB with at least one science A-level. Plus GCSE B in Maths.

Students take a stage 1, semester 1 module covering practical methods and basic mathematics.

In semester 2 they take a module with a significant statistics component.

Students have:

A broad range of mathematical qualifications. In reality students have GCSE A* to C and this has been found to be the strongest predictor of performance in the semester 2 statistics module*.
Different timescales since they have last touched maths. Example cohort: 40% have A-level maths, all of which attained >60% in the semester 2 module, however the A-level grade did not significantly affect performance*.

Module leaders are aware of students falling behind due to limitations with their mathematics.

^*Based on research by Peter Gallagher, Lecturer in Neuropsychology, Newcastle University

Case study: Tackling the transition problem

Suitable questions were created as part of a summer student project.

Students were given a week to practice the material at the start of semester 1, and received automated feedback on their attempts.

The test was then opened in an assessed mode, after which students were asked which topics they would like help with.

Students were offered a timetabled group session with our Maths Aid tutor, or could attend as part of the one-to-one service.

Students scoring less than 90% were recommended to take the further support on offer.

Case study: Student feedback

"I had not had to use this maths knowledge in two years and although I could do the calculations themselves easily, I did need to remind myself how to do them, which the practice helped with."

"It helped to refresh things i had not done in years"

"The test itself was not that difficult. Even now in statistics lectures I see that what we are doing and will do is way more complicated. MORE HELP IS NEEDED."

17 students requested assistance with one or more of 7 topics.

Maths Aid tutor wrote Numbas questions for each of these topics: offered inside the Blackboard folder for the module.

5 students attended the Maths Aid one-to-one service.

Remedial session was poorly attended (but had been rescheduled).

Thank you to Yoav Tadmor, Peter Gallagher, Vicky Hall and Hayley Moore who contributed to the Psychology project in various ways.

Enabling Numbas use outside the Maths department

The psychology model, or similar, can be applied to many different subject areas.

How do we make it easy for colleagues?

Ensure that Numbas works seamlessly with our VLE.
Make existing maths questions easily accessible.
Offer documentation and workshops to staff.

Integration with the VLE

Numbas can use the SCORM standard to integrate with compliant VLEs, such as Moodle and Blackboard.
Or there's a Basic LTI tool provider which works better than most SCORM implementations.
Or can be used without a VLE.

Support for colleagues

We also need to offer support for staff:

Site-specific documentation, with the support of our University teaching and learning team.
Staff workshops.

Summary

A sophisticated mathematical e-assessment tool such as Numbas has applications far beyond the maths department.

In particular, students transitioning to University can benefit from a tool that can diagnose weaknesses and give instant feedback.

Implementation of a tool used in this broader context requires institutional support.

Thanks!