<section class="section1"><h1>Numbas e-assessment in Covid times</h1> <p style="text-align: center; font-size: 1.1em;">Chris Graham</p> <p style="text-align: center; font-size: 0.85em;">Director of E-Learning<br/>School of Maths, Stats & Physics, Newcastle University</p> <figure class="figure1" data-title="Logos"><img alt="Logos" src="../../../static/images/logos.png" width="100%"/></figure><hr/> <section class="section2"><h2>This talk</h2> <p>About Numbas</p> <div class="fragment"> <p>How we've used Numbas during remote teaching</p> </div><hr/> </section><section class="section2"><h2>About Numbas</h2> <p>A web-based e-assessment system designed for mathematical subjects.</p> <div class="fragment"> <p>Developed at Newcastle University for 10 years.</p> </div><div class="fragment"> <p>Free and open source.</p> </div><div class="fragment"> <p>Used around the world.</p> </div><hr/> </section><section class="section2"><h2>Why use maths e-assessment?</h2> <p>Online assessment of procedural, mathematical questions has two golden features:</p> <div class="fragment"> <p>🏆 <strong>Instant feedback</strong></p> <p>Get immediate feedback and receive a full solution for each instance of the question.</p> </div><div class="fragment"> <p>🏆 <strong>Randomisation</strong></p> <p>Practise similar questions over and over again.</p> </div><hr/> </section><section class="section2"><h2>What do we use Numbas for?</h2> <p>Pre-entry material, transition, diagnostic tests, in-course practice, in-course assessment, final exams...</p> <div class="fragment"> <p>A quick look at a Numbas test:</p> <p><a class="button" href="https://numbas.mathcentre.ac.uk/exam/3401/ltds-demo/embed/" target="_blank">Demo test ></a></p> </div><hr/> </section><section class="section2"><h2>Units example</h2> <p> </p> <iframe src="https://numbas.mathcentre.ac.uk/question/93354/simple-unit-example/embed/?token=acff6d0b-60c3-4b06-b1ec-1311c7f46eee" style="width: 100%; height: 430px; -webkit-transform: scale(1.1); border: 0;"></iframe> <hr/> </section><section class="section2"><h2>Remote learning trends</h2> <p>A few trends from the past year:</p> <ul> <li>More diverse and extensive formative assessment</li> <li>Dramatic increase in use in engineering</li> <li>Remote 'labs' using Numbas</li> <li>Hybrid tests</li> <li>Final exams</li> </ul> <hr/> </section><section class="section2"><h2>Formative assessment</h2> <figure class="figure2" data-title="Logos"><img alt="Logos" src="../../../static/images/physics-lthe/handout.png" style="float:right; width: 600px; margin-left: 30px;"/></figure><p>We've seen more formative use; more appreciation of the "weekly quiz".</p> <div class="fragment"> <p>Increasing use in physics, engineering, chemistry...</p> </div><div class="fragment"> <p>More diverse applications to ask questions involving Python, R, MATLAB, SPSS, Minitab.</p> </div><hr/> </section><section class="section2"><h2>Engineering uptake</h2> <canvas id="myChart1"></canvas> <hr/> </section><section class="section2"><h2>Remote Labs</h2> <figure class="figure3" data-title="Logos"><img alt="Logos" height="450px" src="../../../static/images/physics-lthe/gearslab.png"/></figure><p><a class="button" href="https://numbas.mathcentre.ac.uk/question/52463/experiment-proof-of-concept/embed/" style="float:right; font-size: 0.8em;" target="_blank">Stripped-down example</a></p> <hr/> </section><section class="section2"><h2>Hybrid assessments</h2> <p>Used for in-course assessment and final exams</p> <p>Typically 60-80% auto-marked. Focus manual marking where it is needed.</p> <figure class="figure4" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/hybrid.png" width="900px"/></figure><hr/> </section><section class="section2"><h2>Final exams</h2> <p>"4 in 24" hybrid format used across all of our stage 1 modules.</p> <figure class="figure5" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/startend.png" width="100%"/></figure></section><section class="section2"><h2>The format of the assessment(s) was easy to understand</h2> <figure class="figure6" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/understand.png" width="100%"/></figure></section><section class="section2"><h2>The flexibility of starting at any point in the 24 hours was useful to me</h2> <figure class="figure7" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/flexibility.png" width="100%"/></figure></section><section class="section2"><h2>I had enough time to complete the assessment questions</h2> <figure class="figure8" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/time.png" width="100%"/></figure></section><section class="section2"><h2>I worked continuously on the assessment(s) during my timed period</h2> <figure class="figure9" data-title="Hybrid assessment"><img alt="Hybrid assessment" src="../../../static/images/assessment-surveys/continuous.png" width="100%"/></figure></section><section class="section2"><h2>Feedback</h2> <blockquote> <p>I felt like the university did a good job at giving us a flexible exam schedule in order to make up for the limitations of technology and some of the things that can go wrong</p> </blockquote> <div class="fragment"> <blockquote> <p>The flexibility of beginning time is so useful especially when many students have jobs etc </p> </blockquote> </div><div class="fragment"> <blockquote> <p>being able to start whenever you want within the 24hrs gives much more flexibility and allows you to better plan out revision.</p> </blockquote> </div></section><section class="section2"><h2> </h2> <blockquote> <p>I think the problem with the assessments is the fact that once you start you cant stop at all, with many people who aren't in work or who have meetings working at home it makes a lot of distractions that cant really be avoided</p> </blockquote> <div class="fragment"> <blockquote> <p>4 hours was plenty time to Google and try figuring it out with friends</p> </blockquote> </div><hr/> </section><section class="section2"><h2>Using Numbas</h2> <p style="text-align: center; font-size: 1.1em;"> Mathcentre public Numbas Editor at <a href="https://numbas.mathcentre.ac.uk">numbas.mathcentre.ac.uk</a></p> <p style="text-align: center; font-size: 1.1em;">Open source LTI tool at <a href="numbas-lti-provider.readthedocs.io">numbas-lti-provider.readthedocs.io</a></p> <hr/> </section><section class="section2"><h2>Public Numbas Editor</h2> <p><strong>Open to everyone.</strong> </p> <p>5,000+ users at 1,000+ institutions registered on public editor.</p> <div class="fragment"> <p><strong>Collect ready-made questions into a custom test.</strong> </p> <p>8,000+ questions and exams released for free reuse under an open access licence. Translated into 15 languages.</p> </div><div class="fragment"> <p><strong>Or write your own.</strong></p> </div><hr/> </section><section class="section2"><h2>Thanks for listening</h2> <p>Contact me at <a href="mailto:christopher.graham@newcastle.ac.uk">christopher.graham@newcastle.ac.uk</a></p> <p>Useful links:</p> <ul> <li>Numbas Website <a href="https://numbas.org.uk">numbas.org.uk</a></li> <li>Numbas Editor <a href="https://numbas.mathcentre.ac.uk">numbas.mathcentre.ac.uk</a></li> </ul> </section></section>