CMALT - Andrew Davey

5. Specialist area: A bespoke online self-test system for students

Details

Requirement: As well as the core areas, Candidates are required to demonstrate evidence of independent practice in one or more specialist options. This reflects the fact that, although there are common areas of work in this area, practice is extremely diverse and everyone specialises in something different.

Your chosen specialist option might be defined by your manager, your professional body or your own personal interests. You can also define your own specialist topics if none of the recommendations here reflect your role and interests. However, when doing this, you should keep in mind that such areas should be specialist - that is, they should not be things that the majority of practitioners in this area would do.

Once the specialism has been chosen, evidence could include: Reports, papers or presentations you have written; a job description plus written statements supporting your specialist knowledge from colleagues, clients or managers; active membership of professional or other bodies; certificates of completion of specialist training programmes or courses.

Description and reflection

In this section I will reflect on the process of creating a bespoke self-test system for students, which ties in with my specialist areas of testing systems and learning object creation and development. I have been developing this new system for students to use when reviewing packaged sets of learning objects (toolkits) on a particular topic. This system, which is still a work-in-progress, will be included with the individual online toolkits as a core feature of each package.

We held a team planning meeting to outline the requirements for the system, and I started by putting together an experimental alpha version. This used basic styling (based on the learning objects within the packages) for ease of access, and a simple set of 10 multiple-choice questions built in HTML. One particular point I had to bear in mind was that the testing system is going to be hosted online, and any users who have answered some questions but have a connection or computer failure should be able to resume from where they left off. Ultimately, I intend to make this a part of a database system; however for the initial development and testing phase I felt it would be a more agile development to rely on browser cookies to store this information.

Once I was happy with the functionality of the code and the basis of the presentation I ran it past the rest of the team to check for any functional or user issues. In December, we conducted a piloting session with a group of 25 Masters students who are studying English Language testing, giving them time to read through the content of the package before embarking on the test.

Their overall feedback on the system and the concepts behind it were positive, but there were some limitations which were apparent. The most significant complaint was that the user interface looked dull and dated. I felt a little disappointed by this, as the students knew it wasn't a final version, but in the feedback meeting we held it was decided that a more graphical approach was required. This is a particular challenge because there are few interesting ways to illustrate a vocabulary test. However, I have experimented with different designs to add more visual interest, ready for a second pilot to run at the start of May.

The main advantages of developing a bespoke quiz system are the ability to customise and the ability to integrate the system with a database. Although I have not done this with the current version yet, the final version will be integrated in such a way to keep students' scores linked with the activities they have completed.

I also needed a system that could be developed independently and was not tied to a separate login system. This was the main disadvantage of the alternative route of using Moodle quizzes - the platform requires a login and integrating the existing login system with Moodle would have been both complex and potentially problematic (especially when Moodle upgrades are released, as each version would require testing). I also felt that this would have been a bloated solution, whilst a custom quiz system is more streamlined.

The self-test encapsulates some of the options within Moodle that I felt were advantageous, such as the ability to deduct marks after multiple attempts at a question, but in a more versatile way. For example, in the self-test, the scoring scale is 10/10 for a first attempt, 5/10 for a second attempt, 2/10 for a third attempt and 0/10 for a fourth attempt.

Evidence

Here are several screenshots of how I adapted the quiz system to fit the feedback from the students.