“We strongly believe that our next step towards enhancing the quality of our assessments is moving towards Online-Onscreen marking of exams to improve transparency, reliability, and accuracy of our results”
Prof. Nasrullah Khan Yousafzai
Chairman-BISE, Peshawar
Board of Intermediate and Secondary Education, Peshawar (BISE-Peshawar)
About
BISE, Peshawar is a regional board working under the Ministry of Education, Khyber Pakhtunkhwa. Ministry of Education Khyber Pakhtunkhwa (KP) is determined to provide quality education enabling all citizens to reach their maximum potential and produce responsible and skilled humans. To achieve the mission, BISE-Peshawar is taking lead in adapting modern technologies to facilitate students to save their time by quick access to their roll number slips, results, DMCs, registration, model papers, and related information regarding registration/enrolment, affiliation, examination schedule and camp offices from their homes.
Why
BISE, Peshawar is a regional board working under the Ministry of Education, Khyber Pakhtunkhwa. Ministry of Education Khyber Pakhtunkhwa (KP) is determined to provide quality education enabling all citizens to reach their maximum potential and produce responsible and skilled humans. To achieve the mission, BISE-Peshawar is taking lead in adapting modern technologies to facilitate students to save their time by quick access to their roll number slips, results, DMCs, registration, model papers, and related information regarding registration/enrolment, affiliation, examination schedule and camp offices from their homes.
How
01-ANSWER SCRIPT AND QUESTION PAPER DESIGNING
In collaboration with BISE-Peshawar, RedMarker Systems designed an appropriate digital answer script that satisfied requirements of both BISE-Peshawar as well as RedMarker Systems’ rMarker. Question paper was also redesigned in order for it to be aligned with the newly designed answer sheet.
02-DESIGNING OF RUBRIC
RedMarker Systems held a detailed meeting with subject specialists that set the exam paper to formulate rubric usage for the allocated subject/ exam papers. These rubrics were later used to ensure e-markers correctly mark students’ responses
03- INVIGILATION STAFF TRAINING
It is extremely important for the invigilation staff to be accustomed to the new answer sheets as they will eventually guide students to fill out all the required details properly. For this purpose, free trainings were imparted by BISE, Peshawar to ensure all invigilating staff had understanding of the new answer scripts prior to the exam conduction.
04- E-MARKER TRAININGS
With the introduction of rMarker as a tool to mark students’ responses online and onscreen, teachers were trained to efficiently mark answer scripts. This training was held at BISE-Peshawar where selected teachers were given hands-on training and experience of using rMarker.
05- EXAMS CONDUCT AND STUDENT TRAINING
For each exam conducted within the scope of this pilot study, 15 minutes were reserved before the commencement of exam for the invigilators to guide students to fill out all the required details properly. This process was extremely important to ensure students’ answer sheets were rightly associated with their roll numbers to remove any chances of errors.
06- SORTING, CUTTING & SCANNING OF ANSWER SCRIPTS
Once all answer scripts were received by BISE-Peshawar, a team of experts in sorting, cutting and scanning from RedMarker Systems were sent to organize this process at a secure location provided by BISE-Peshawar within their premises. Strict controls were set out to ensure secrecy, security and anonymity of students while maintaining efficiency and professionalism.
07- E-MARKING
All e-markers were registered with RedMarker Systems and were given their unique login details well in advance of marking process to begin. All e-markers began marking exams as instructed by BISE-Peshawar and completed all batches within the time-period specified.
08- RESULT GENERATION
As one of the key features of rMarker, all results are compiled in real-time and can be retrieved at any given time to generate result. These results can be of a particular student, class wise, examination center wise, subject wise, school wise, geographical location wise etc. Accurate results were generated in our pilot project.
09- COMPLEX MICRO-LEVEL DATA ANALYSIS
For questions that require rubric-based marking, e-markers were compelled to grade students’ responses according to the pre-defined rubric, set at the time of question paper making. This data was then represented in a structured form for further macro/ micro level data analysis essential for policy making.
The Results
Graph 1
Graph 1 represents the combined variance for both the subjects. The minimum to maximum variance experienced for both the subjects was 0% to 48%. The graph represents the number of teachers that fall within variance ranges where the largest group of 21 teachers showed a variance between 19% to 29% and only 13 teachers lied in the variance range of 10% to 19%.
Graph 2
Graph 2 represents teacher performance for the English subject only where the minimum to maximum variance experienced for English was 0% to 45%. We can see that in English alone the 22 teachers showed variances between 18% to 27% followed by 9 teachers showing variances ranging from 9% to 18%.
Graph 3
Graph 3 represents teacher performance for the Computer Sciences subject only where the minimum to maximum variance experienced for the subject was 0% to 38%. It is important to mention here that due to a small number of computer science papers only 7 teachers were used for the checking exercise.
Key Analysis
Dividing the pilot phase findings into 3 sections, it is observed that:
- Graph 1 represents combined variance to study the impact of on-screen marking
- Graph 2 represents the use of Rubric-based marking in English’s exam papers to study the results of rubric-based marking
- Graph 3 represents the variance of the control group i.e. computer science’s exam papers, to study the biases introduced in traditional marking practices
While comparing Graph 2 and Graph 3, increased variance is seen in Graph 3 due to the absence of rubric-based marking. This highlights the fact that traditional marking may often be biased or prejudiced by the marker. Thus, it signifies the importance of rubrics-based marking that omits chances of biased marking practices and ensures quality marking.
While Graph 1 cumulatively shows the impact of the quality on-screen marking. Decreased variance renders online on-screen marking an improved tool for marking and grading assessments.
