Notes from a seminar on e-assessment held on Tuesday 7th January 2014 at the offices of Ofqual in Coventry

Context

RNIB had carried out some research into accessibility of e-assessment and Ofqual wished to expand this beyond VI to other candidates through a seminar. Inevitably the report and much of the discussion – but not all – focussed on accessibility for VI candidates.

Present today: AQA, BATOD, BDA (Dyslexia), BIS, CCEA, City & Guilds, Communicate-Ed, FAB, JTech Dis, IBO, Lycol, OCR, Ofqual, Pearson (Edexcel), RNIB, Welsh Government, WJEC

Summary of the main points of the meeting:

• It was established at the beginning that the use of e-assessment by awarding bodies is not currently widespread.

Sally Cain, RNIB, spoke about her research into e-assessment

• No-one has the responsibility for providing guidance; this is a concern
• There is currently growing interest in developing e-assessment
• This is not a matter of paper tests in electronic form but a different kind of assessment
• A variety of accessibility features can be used from different font sizes, colours and interactivity to the provision of signing and subtitles
• The development of HTML5 has improved functionality
• Although there are standards for accessibility for the internet and software there is nothing for e-assessment
• It is unlikely to be used for large cohorts of candidates eg core GCSEs for practical reasons
• There are currently problems caused by different types of assistive technology not being able to read e-assessment materials
• BATOD pointed out that although multiple choice questions for example were helpful to some groups the linguistic challenges they cause could penalise other groups eg those who are deaf or have dyslexia.
• I highlighted the problems experienced in Northern Ireland over the e-assessed literacy and numeracy assessments at primary level as an example of the need to learn from mistakes and experience
• I also asked what research there had been into the response of users and how had specific expertise been used in developing e-assessment. No response was forthcoming.
• It should be possible to agree minimum standards despite the need for ABs to be competitive

Other presentations concentrated on accessibility for those with visual impairment

Other important points made during the meeting and presentation

• Important for accessibility to be considered not just at the beginning but at every point during assessment development
• User testing is crucial
• Extra time can be accommodated but the functionality needs to be able to permit candidates to go back as well as forward

Ofqual’s research into e-assessment

• Ofqual is exploring the opportunities and risks; it is important to check whether validity of assessment is affected
• Ofqual needs to be able to regulate effectively
• Trying to determine what is going on and where
• Will include a literature review and interviews with key stakeholders
• Intends to report in summer 2014
• Ofqual has set up an internal working group because different arms of the organisation are concerned with different aspects: eg validity, accessibility, regulation

In the afternoon four questions were discussed in working groups

Is it possible to design assessments allowing universal access or will alternates always be needed?

In the future it should be possible eg language modification at source for the vast majority but still some candidates will need special arrangements; timescale is an issue

What are the main considerations for the provision of amendments to online activity as opposed to alternates?

Cost of making the amendments and the need to ensure comparability

When does an alternative activity become materially different from the original?

When the mark scheme required is different; it must not be or the assessment is different and comparability not possible

Is guidance needed and if so in what form?

Consensus was yes; there could be minimum standards for developers and awarding bodies but this could not cost huge amounts of non-existent money; perhaps JCQ could oversee and be involved?

Next steps

• Awarding Bodies to take ideas back and to meet to discuss further with JCQ and FAB
• Disability groups to gather together any relevant guidance and information they already have

Next steps for BATOD

• Report back to NEC
• Seek advice and information from BATOD members about guidance which already exists and also gather other ideas about e-assessment and deaf candidates
• Contribute to the discussions/guidance to be developed by disability groups under leadership of RNIB

Paul Simpson, 7th January 2014