GB2547782A

GB2547782A - Assesment system

Info

Publication number: GB2547782A
Application number: GB1701524.9A
Authority: GB
Inventors: Aitken Christoffer
Original assignee: Psittacus Systems Ltd
Current assignee: Psittacus Systems Ltd
Priority date: 2016-02-04
Filing date: 2017-01-31
Publication date: 2017-08-30
Anticipated expiration: 2037-01-31
Also published as: GB2547782B; GB201701524D0; GB201602047D0

Abstract

An assessment system for enabling a user to complete an assessment on a computing device including an image capturing device is disclosed. The system comprises a module configured to determine the operational status 101 of the image capturing device and causing it to be switched 104 or maintained in an operational mode for capturing images of the user and providing said captured images for on-line viewing from a remote location. The system further comprises a module configured to retrieve 105 or generate an assessment resource and to display 106 the resource within a user interface on a screen of the computing device, the user interface generating focus data indicative of user interaction. A further module is configured to monitor the focus data 107 and, if it is determined that a blur event 108 has occurred such that the user interface is no longer active, an error signal is generated. In response to the error signal, the assessment may be terminated 103 to prevent any further input from said user. In a preferred embodiment, images captured by the camera during the assessment are compared with stored identification data to verify the identity of the user.

Description

ASSESSMENT SYSTEM

This invention relates generally to an assessment system and more particularly, but not necessarily exclusively, to an assessment system configured to enable on-line examination processes to be effectively and substantially automatically invigilated.

It is well known for learners in respect of a particular course or qualification to be required to sit a Multiple Choice Examination (MCQ) at the end of the learning process in order to obtain the respective qualification. A conventional process for obtaining a qualification would be as follows: 1. The learner orders the course; 2. The learner is registered with the relevant Awarding Body or Organisation for the respective qualification; 3. The learner attends a scheduled classroom training session in order to gain the relevant knowledge taught by an appropriate tutor/lecturer; 4. Once the course has finished, and the learner is ready to be assessed, they are required to attend at a specified time and location to sit an examination. 5. Generally, the tutor/lecturer (or an independent invigilator) will attend the examination session to check all learners’ identification and start the examination in accordance with an allotted time. 6. Once the exam is finished, the invigilator collects the examination papers and posts them on to the Awarding Body to be marked. 7. Once all papers have been marked, the Awarding Body issues full certificates to the learner, again by post.

There are certain parameters that must be applied to all invigilated examinations using MCQs: 1. The invigilator is supplied with test papers, of which there are a number of versions (e.g. three or four), and these are distributed to the learners at random to try to ensure that learners sitting close together within the examination venue have different test papers, wherever practical and possible. 2. The invigilator is required to ensure that learners observe the malpractice policies and guidelines, which will include the prohibition of computing devises, notes, etc. 3. The invigilator is also required to verify that the person physically sitting the examination is, in fact, the person registered for the qualification by checking identification documents.

Online learning is becoming increasingly prevalent in respect of, for example, continued Professional Development (CPD) courses and Ofqual regulated qualifications. Thus, a learner accesses learning and course materials and resources on-line and can study in their own time and/or join online classroom sessions and one-to-one tutorials using webcams. Many such courses are also assessed using Multiple Choice Examinations (MCQs). However, in view of the risk of malpractice, such as a different person sitting the examination or cheating by the use of references such as on-line resources or notes, it us conventionally still necessary for candidates of such courses to physically attend a specified venue at a given time to sit their examination in the presence of an invigilator.

There is an ongoing desire to provide an on-line course that is entirely on-line, and does not require the attendance by a candidate at an examination venue, or the physical presence of an invigilator, in order to enable the candidate to sit their final examination. However, the strict requirement for an invigilator to verify the identity of the examination candidate and also the need to ensure that the candidate observes the requisite guidelines to prevent cheating, has meant that, even if the examination itself is taken on-line, an invigilator has been required to be physically present.

It is an object of aspects of the present invention to provide a technical solution to at least some of the issues outlined above, and provide a system for enabling examinations for regulated qualifications to be taken on-line without requiring the physical presence of an invigilator, thereby circumventing the need for the candidates and an invigilator to attend a specified location at a specified time in order to sit an examination.

In accordance with an aspect of the present invention, there is provided an assessment system for enabling a user to complete an assessment on a computing device including an image capturing device, the system comprising: a module configured to determine the operational status of said image capturing device and causing it to be switched or maintained in an operational mode for capturing images of said user and providing said captured images for on-line viewing from a remote location; a module configured to retrieve or generate an assessment resource and display data representative thereof within a user interface element on a screen of said computing device for interaction therewith by said user, said user interface element having focus data associated therewith only when it is in a mode in which a user can interact with the data therein; a module configured to monitor said focus data associated with said user interface element and, in the event that said focus data is removed from said user interface element as a result of a blur event in respect thereof, generating an error signal; and a module for receiving said error signal and, in response thereto, terminating the assessment to prevent any further input from said user.

In an exemplary embodiment, the system further comprises a posture analysis module. The posture analysis module may comprise an image recognition module configured to analyse a posture of the user based on video or images of the user, the video or images being captured by the image capturing device during the assessment. The posture analysis module may further comprise a monitoring module configured to raise a posture event when the posture of the user deviates from a predetermined authorised posture, the posture event generating the error signal.

In an exemplary embodiment, the image recognition module may be further configured to detect a face of the user and the monitoring module amy further configured to define an authorised region and to raise the posture event when at least a portion of the detected face moves outside the region. The image recognition module may also be configured to detect a face of the user and a side view of a head of the user, and the monitoring submodule is configured to raise the posture event when the side view of the head of the user is detected. The posture analysis module may be further configured to analyse recorded video or images after the assessment is finished or to analyse the video or images in real time during the assessment.

In an exemplary embodiment, the system may be configured to cause a fail flag to be set in respect of said assessment in the event that an error signal is received.

The system may include a repository, accessible by a third party on-line from a remote location, for receiving and storing said images of said user captured during said assessment.

The images captured of said user may comprise still image frames captured periodically or at random intervals during said assessment. In this case, the repository may be configured to receive and store identification data in respect of said user for comparison with one or more of said images captured of said user during said assessment.

In an exemplary embodiment, the system may be configured, in response to an error signal and/or at the end of said assessment, to transmit data representative of a completed or partially completed assessment to said repository for storage therein. The repository may include a flag status associated therewith, wherein said flag status is selectively set to pass or fail in respect of a specified assessment in relation to a specified user. In an exemplary embodiment, the system may be configured to transmit data representative of an assessment, a respective user and a respective flag status to a remote location.

The system may be configured to receive data representative of a user input in respect of said assessment and, using image recognition software, determine entries to be correct or incorrect, and output data indicative that a respective user has passed or failed said assessment based on said determination.

In accordance with another aspect of the present invention, there is provided a computer program product configured, when loaded and run on a computing device to perform an assessment method comprising the steps of: determining the operational status of said image capturing device and causing it to be switched or maintained in an operational mode for capturing images of said user and providing said captured images for on-line viewing from a remote location; retrieving or generating an assessment resource and display data representative thereof within a user interface element on a screen of said computing device for interaction therewith by said user, said user interface element having focus data associated therewith only when it is in a mode in which a user can interact with the data therein; monitoring said focus data associated with said user interface element and, in the event that said focus data is removed from said user interface element as a result of a blur event in respect thereof, generating an error signal; and receiving said error signal and, in response thereto, terminating the assessment to prevent any further input from said user.

These and other aspects of the present invention will be apparent from the following specific description, in which embodiments of the present invention are described, by way of examples only, and with reference to the accompanying drawings, in which:

Figure 1 is a flow diagram indicating principle steps performed by an assessment system according to an exemplary embodiment of the present invention.

Figure 2 is a block diagram showing components of the system according to an exemplary embodiment of the present invention.

Thus, referring to Figures 1 and 2 of the drawings, when the user has accessed the assessment system 200 on-line at step 100, and elected to sit an MCQ, the system 200 is configured to determine, using a module 210 at step 101, the operational status of the user’s webcam 220 and, if it is switched off, the system is configured to generate a dialog box (not shown) requesting the user’s permission to switch it on (step 102). If the user blocks/declines the request, the assessment will be terminated (step 103) and the attempt will be registered as a fail.

If the user accepts the request, the webcam 220 is switched on, at step 104, to capture images of the user whilst they are completing the assessment.

Next, at step 105, the system 200 is configured to select and retrieve, using a module 230, one of a number of assessment ‘papers’ (not shown) and, at step 106, display its contents in an interface element or ‘window’ on the user’s screen (not shown).

At this point, the user interface element (not shown) containing the assessment will have the focus. The term ‘focus’ is known in computing as indicative of the component of a graphical user interface which is selected to receive input. Moving the focus away from a specific user interface element is known as a blur event in relation to that element. Typically, the focus is withdrawn from a user interface element by giving another element the focus. In most cases, the focus is removed from a user interface element by clicking a pointing device outside of that element, although it is also possible to achieve this via the keyboard in many cases. Thus, if the user now clicks outside of the user interface element containing the assessment, it will lose focus (i.e. be the subject of a blur event).

Thus, at step 107, the system 200 is configured to monitor, using a module 240, the focus of the user interface element in which the assessment is displayed, and identify if a blur event occurs (i.e. the focus is removed from that user interface element, indicating that the user has clicked outside of it). This may, of course, indicate that the user has attempted, say, to open a new window or otherwise consult other on-line resources contrary, the normal code and rules of procedure and practice in relation to the taking of an assessment.

Thus, at step 108, if a blur event is identified in relation to the user interface element in which the assessment is displayed, the system is configured to generate, using a module 250, an error signal, in response to which the assessment is immediately terminated. Further user interaction with the assessment is blocked, the assessment, thus far completed is transmitted to a remote repository 270, and a fail result is registered for that user against that attempt (at step 103).

The system may further comprise a posture analysis module 260 for monitoring the posture of a user, and particularly the user’s head, while undertaking the assessment. At step 109, video or image data supplied from the webcam 220 may be fed to the posture analysis module 260 in real time whilst the user is completing the assessment or may recorded and analysed by the module 260 after the assessment. The posture analysis module 260 comprises an image recognition module 261 which may be pre-configured, by subjecting it to a machine learning algorithm which trains the image recognition module to recognise a users face based on training data provided as an input to the algorithm, to digitally recognise a face of the user in the field of view of the webcam 220. A monitoring module 262 also associated with the posture analysis module 260, subsequently defines a two-dimensional authorised region, such as an ellipse, on the screen, the region being coplanar with a two dimensional projection of the face of the user on the screen. The projection of the face of the user should be present within the region at least most of the time or at all times during the assessment. The region corresponds to a real life position of the user, the head of the user being substantially within the central area of the field of view of the webcam 220, and facing the screen of the machine on which the assessment is taken. The posture analysis module 260 then monitors the position of the projection and if the projection crosses the boundaries of the authorised region, thereby entering an unauthorised region, for a predetermined amount of time and/or by a predetermined distance, a posture event will be generated.

Alternatively, the image recognition module 261 may be pre-configured to detect a front and side, i.e. a profile, of the head of the user. In this case, the authorised region is not necessary, as the monitoring module 262 will raise the posture event whenever the profile of the face of the user is detected anywhere within the field of view of the webcam 220. Likewise, additional parameters such as time period within which the head of the user remains turned away from the screen may be analysed by the monitoring submodule in the decision process about raising the posture event.

The posture event corresponds to a situation in which the user turns their head away from the screen. It is envisaged that such events may correspond with attempts of users to look away from the screen to consult with texts or to copy answers from a person sitting nearby, for example.

Alternatively, posture analysis by the posture analysis module 260 as described above may be performed on the recorded video or image data after completing the assessment by the user and during an analysis of the assessment by the examiner, for example. This option requires less computational resources as the video or image data recorded during the assessment may be conveniently communicated for processing to a suitable computationally efficient machine, which might not be available during the assessment.

Thus, if a posture event is identified at step 110, the system 200 is configured to generate an error signal, in response to which the assessment is immediately terminated. Further user interaction with the assessment is blocked, and the assessment which may only be partially completed is subsequently transmitted to a remote repository. In these circumstances, the assessment system may record a fail result for that user against the examination (at step 103).

If, however, no blur event or posture event is determined during the course of the assessment, the assessment continues until a) a predetermined time has elapsed, and/or b) the user has indicated on the assessment that they are finished. Either or both of these events may be considered to be a ‘stop’ event. At step 111, a ‘stop’ event causes further interaction with the assessment by the user to be blocked, and assessment data derived from the completed assessment is transmitted to a remote repository 270 for marking (either automatically, such as by using image recognition software or by an invigilator).

The repository 270 included within the system 200 has a file (not shown) for each registered candidate. Within each file, there is stored identification data uploaded by each candidate prior to the assessment. Image data captured, for example, in the form of a series of periodic stills or, alternatively, as a video file during the course of the assessment may also be stored within the file, together with the completed assessment for that user, once received. An invigilator can access the file to view the captured images and verify the identity of the user taking the assessment by comparison of the images captured with the identification document stored in the file. Once verified, the invigilator can mark the file accordingly. Additionally, a pass/fail flag may also be stored within the file to indicate whether the user has passed or failed the assessment taken. The completed file can thus be transmitted (electronically or otherwise) to the awarding body (not shown) to issue the results.

It will be apparent to a person skilled in the art, from the foregoing description, that modifications and variations can be made to the described embodiments without departing from the scope of the invention as defined by the appended claims.

Claims

1. An assessment system for enabling a user to complete an assessment on a computing device including an image capturing device, the system comprising: a module configured to determine the operational status of said image capturing device and causing it to be switched or maintained in an operational mode for capturing images of said user and providing said captured images for on-line viewing from a remote location; a module configured to retrieve or generate an assessment resource and display data representative thereof within a user interface element on a screen of said computing device for interaction therewith by said user, said user interface element having focus data associated therewith only when it is in a mode in which a user can interact with the data therein; a module configured to monitor said focus data associated with said user interface element and, in the event that said focus data is removed from said user interface element as a result of a blur event in respect thereof, generating an error signal; and a module for receiving said error signal and, in response thereto, terminating the assessment to prevent any further input from said user.

2. A system according to claim 1, further comprising a posture analysis module for monitoring a posture of the user during the assessment.

3. A system according to claim 2, wherein the posture analysis module comprises an image recognition submodule configured to analyse a posture of the user based on video or images of the user, the video or images being captured by the image capturing device during the assessment.

4. A system according to claims 2 or 3, further comprising a monitoring module configured to raise a posture event when the posture of the user deviates from a predetermined authorised posture, the posture event generating the error signal.

5. A system according to claims 3 and 4, wherein the image recognition module is further configured to detect a face of the user and the monitoring module is further configured to define an authorised region and to raise the posture event when at least a portion of the detected face moves outside the region.

6. A system according to claim 3 and 4, wherein the image recognition module is further configured to detect a face of the user and a side view of a head of the user and the monitoring module is configured to raise the posture event when the side view of the head of the user is detected.

7. A system according to any of the claims 2 to 6, wherein the posture analysis module is configured to analyse recorded video or images after the assessment is finished.

8. A system according to any of the claims 2 to 6, wherein the posture analysis module is configured to analyse the video or images in real time during the assessment.

9. A system according to any preceding claim, configured to cause a fail flag to be set in respect of said assessment in the event that an error signal is received.

10. A system according to any preceding claim, including a repository, accessible on-line from a remote location, for receiving and storing said images of said user captured during said assessment.

11. A system according to any of the preceding claims, wherein said images captured of said user comprise still image frames captured periodically or at random intervals during said assessment.

12. A system according to claim 10, wherein said repository is configured to receive and store identification data in respect of said user for comparison with one or more of said images captured of said user during said assessment.

13. A system according to any of the preceding claims, configured, in response to an error signal and/or at the end of said assessment, to transmit data representative of a completed or partially completed assessment to said repository for storage therein.

14. A system according to claim 13, wherein said repository includes a flag status associated therewith, wherein said flag status is selectively set to pass or fail in respect of a specified assessment in relation to a specified user.

15. A system according to claim 13 or claim 14, configured to transmit data representative of an assessment, a respective user and a respective flag status to a remote location.

16. A system according to any of the preceding claims, configured to receive data representative of a user input in respect of said assessment and, using image recognition software, determine entries to be correct or incorrect, and output data indicative that a respective user has passed or failed said assessment based on said determination.

17. An assessment system substantially as herein described and/or with reference to the accompanying drawing.

18. A computer program product configured, when loaded and run on a computing device to perform an assessment method comprising the steps of: determining the operational status of said image capturing device and causing it to be switched or maintained in an operational mode for capturing images of said user and providing said captured images for on-line viewing from a remote location; retrieving or generating an assessment resource and display data representative thereof within a user interface element on a screen of said computing device for interaction therewith by said user, said user interface element having focus data associated therewith only when it is in a mode in which a user can interact with the data therein; monitoring said focus data associated with said user interface element and, in the event that said focus data is removed from said user interface element as a result of a blur event in respect thereof, generating an error signal; and receiving said error signal and, in response thereto, terminating the assessment to prevent any further input from said user.