Code of Practice – Use of the OMR (Optical Mark Reader) CAA (Computer Assisted Assessment) Bureau Service for Undergraduate and Postgraduate Exams

 

 

1.      Prior to the examination please adhere to the following procedures:

 

1.1.  The Internal Examiner shall undertake training, with the CAA Support Officer, in the use of the OMR system (unless training has been undertaken on a previous occasion).

 

1.2.  The Internal Examiner shall inform the Student Records & Examinations Office of his/her intention to use the OMR system. This to be done via the Examination Information Sheet and to be undertaken at 6 to 8 weeks in advance of the examination session. This will allow the Student Records & Examinations Office time to schedule OMR exams as early as possible in the Examination Period, enabling prompt processing.

                             

1.3.  The CAA Support Officer will flag any OMR exams to the Student Records & Examinations Office, and, similarly, the Student Records & Examinations Office will contact the CAA Support Officer with details of OMR examinations.

 

1.4.  Where an OMR examination is a combined question-and-answer test paper, the Internal Examiner will send the pink Examination Information Sheet to the CAA Support Officer as means for signing-off the printing of the combined test paper. Exams with separate OMR answer sheets should be treated normally; the pink sheet should accompany the exam paper directly to the Student Records & Examinations Office.

 

1.5.  The CAA Support Officer will supply the required number of student answer sheets or combined question-and-answer test papers (as appropriate) to the Student Records & Examinations Office, including an agreed number of spares, and will supply the lecturer admin sheet or lecturer master copy (of the combined test paper), together with an information pack, to the Internal Examiner.

 

1.6.  The Internal Examiner shall verbally instruct candidates in the OMR procedure prior to the exam, using the information pack supplied by the CAA Support Officer. No further instruction, verbal or otherwise, shall be given to Candidates in the Examination Hall.

 

1.7.  The Internal Examiner, after seeking advice from the University Disability and Additional Needs Service, shall cater for the requirements of any students, with disabilities or additional needs, taking the examination. Upon request, the Student Records & Examinations Office will supply Student Answer Sheets for students with learning difficulties and disabilities, directly to the Internal Examiner or to his/her Department.

 

1.8.  The Student Records & Examinations Office, on the day of the examination, will supply the Student Answer Sheets for all students, other than students with learning difficulties and disabilities, who will be catered for in accordance with 1.5 above.

 

1.9.  Candidates must provide their own pencils and erasers for the examination.

 

 

2.      Following the examination, the procedures to be adopted are:

 

2.1.  The Invigilator shall collect the student answer sheets/combined test papers at the end of the Examination, checking that they are in good condition (not stapled, tagged, folded, torn etc). Any scripts that are not in good condition should be put to the top of the pile.

 

2.2.  The Student Records & Examinations Office will have the OMR student answer sheets/combined test papers ready for collection by the Internal Examiner (or representative) from the examinations venue after the exam has finished. The Student Records & Examinations Office will need to see an ID card prior to releasing any sheets.

 

2.3.  The Internal Examiner will visually check the OMR forms - checking ID numbers are 'coded' in as well as written, that incorrect answers have been erased, that pencil, not biro, has been used, that OMR bubbles have been filled in appropriately and that scripts are in good condition (see 2.1 above). Any scripts which do not satisfy these criteria should be put to the top of the pile and drawn to the attention of the CAA Support Officer when the scripts are brought to Professional Development for processing. They will require manual checking after processing. Additionally, if there are any unexpected issues with the content of the test, these should be flagged to the CAA Support Officer before marking.

 

2.4.  The Internal Examiner will then take the student answer sheets/combined test papers, together with the lecturer answer sheet/lecturer master copy (in the case of the combined test papers), to Professional Development. Please contact Martin Ashby (x.3735) to check availability. A receipt will be issued, if required.

 

2.5.  Professional Development shall provide a secure system for processing (see Service Level Statement).

 

2.6.  The CAA Support Officer will notify the Internal Examiner as soon as the results are ready for collection. This is normally within a working week from receipt at PD (see Service Level Statement).

 

2.7.  The Internal Examiner is responsible for the hand marking of any Student Answer Sheets rejected by the OMR System and for manually checking any scripts separated out at visual checking.

 

2.8.  The Internal Examiner, or another member of departmental staff, should manually check at least 5% of the OMR scripts.

 

2.9.  The CAA Support Officer will be available to assist in further analyses of results if required.

 

 

3.      Resit examinations – changes from above procedure

 

3.1.  Resit examinations are usually not optically marked, due to the low numbers of candidates.

 

3.2.  When a resit examination previously used OMR there will be a requirement for OMR-style stationery in order to capture candidates’ answers. The Student Records & Examinations Office will provide this to internal examiners on request. This will be in the form of a standard 60-question/A-E response sheet, supplied by the CAA Support Officer.

     

November 2007

Service Level Statement - The Use of the Optical Mark Reader (OMR) Computer Assisted Assessment (CAA) Bureau Service

 

This document describes the service for internal clients; normally the Internal Examiner. It is intended for Professional Development (PD) staff and clients.

 

Availability

The OMR Bureau is based at Towers Bungalow 1 and is staffed from 09:30 to 15:00 Monday to Friday. The CAA Support Officer, Martin Ashby, is contactable via extension 3735 (voicemail-enabled) or email (m.c.ashby@lboro.ac.uk). PD Reception in Herbert Manzoni Building will receive any scanned sheets out of usual OMR Bureau hours.

Procedure for creating and handling OMR stationery.

Professional Development offers an in-house bureau service for optically marking CAA tests, which can be either separate answer sheets or combined test papers – the CAA Support Officer is happy to advise on which is more appropriate. Student answer sheets or test papers will be marked using the following procedures:

 

  1. Separate answer sheets

 

    1. The Client gives PD two weeks notice (phone/email) of a test taking place. Required information is:

                                                              i.      Date of test

                                                            ii.      Module code of test

                                                          iii.      Number of students taking test

                                                           iv.      Number of questions in test (e.g. 45)

                                                             v.      Number of options per question (e.g. A-E)

 

    1. PD will issue appropriate stationery promptly.

 

    1. The Client brings the student answer sheets and lecturer admin sheet (which contains the correct answers and information about the test) to the CAA Support Officer or PD reception as appropriate.

 

    1. The CAA Support Officer will process forms within 1 working week of receipt (dependent on workload) and will produce the reports, notifying the client by email as soon as the results are ready. Results will be delivered as email attachments (see below). Test papers are kept securely and returned in agreement with the client.

 


  1. Combined question-and-answer test papers

 

    1. The Client gives PD three weeks notice (phone/email) of a test taking place (noting any departmental deadlines with respect to formal examinations). Combined test papers require a collaborative effort between client and CAA Support Officer in order to create pedagogically appropriate and scannable test papers. Required information includes:

                                                              i.      Date of test

                                                            ii.      Module code of test

                                                          iii.      Number of students taking test

                                                           iv.      Unformatted text file comprising test questions and options, including any test rubric and marking scheme information.

 

    1. The CAA Support Officer will either create the required layout or provide a Word template and support for the client to prepare the layout themselves. In any case, the CAA Support Officer will ensure that the test paper is in the correct format, scannable and usable (taking DANS accessibility and Student Records & Examinations Office requirements into account). The process is expected to take a number of drafts.

 

    1. Once the client is happy to proceed, the CAA Support Officer will print the required number of test papers and agree secure distribution with the client. One copy will be marked ‘Lecturer Master Copy’ upon which the client will record the correct answers. One further copy will be used by the CAA Support Officer to create a scannable test template.

 

    1. The Client returns the taken test papers and lecturer master copy to the CAA Support Officer or PD reception as appropriate.

 

    1. The CAA Support Officer will process forms within 1 working week of receipt (dependent on workload) and will produce the reports, notifying the client by email as soon as the results are ready. Results will be delivered as email attachments (see below). Test papers are kept securely and returned in agreement with the client.

Required Input

  1. When using separate answer sheets:

 

A complete set of completed sheets for each test consisting of

  • A Lecturer admin sheet
  • The Student Answer sheets

Or

  1. When using combined question-and-answer test papers

 

A complete set of completed test papers consisting of

  • A Lecturer master copy
  • The students’ test papers

 

Important note: student IDs are required on both types of form, not only as a written number but also in machine-readable form by filling in a grid of OMR bubbles. Please make sure that any test papers returned to PD have the ID grid filled in.

Output

The following results files will be sent as email attachments:

 

  1. Results in student alphabetical order (PDF)
  2. Frequency distribution of module marks (PDF)
  3. Graph-based item analysis report (PDF)
  4. Excel spreadsheet data, comprising 6 worksheets:
    1. Item analysis
    2. Marks in student alphabetical order
    3. Marks in mark order
    4. Marks in ID order
    5. Marks in mark order (alternative data)
    6. Marks in ID order (alternative data)

 

  1. Individual student feedback forms detailing the student’s exact responses as scanned are available on request.

Feeding Forms through the OMR

Forms delivered to PD in poor condition or unreadable by the scanner will be kept separately and returned with the output to the client, for manual marking.

 

Torn sheets, crumpled sheets, missing pages in combined test papers (usually statistical tables, graphs or lists of formulae), incomplete/missing ID are all examples of problems which slow down PD’s ability to turn around clients’ test results.

 

Final word

PD will attempt to provide help or advice to anyone with an e-assessment-related query.

   

November 2007