WO2015099581A2

WO2015099581A2 - Method and system of employee competency assessment field of the invention

Info

Publication number: WO2015099581A2
Application number: PCT/RU2014/000995
Authority: WO
Inventors: Evgenij Valentinovich LUR`E
Original assignee: Zakrytoe Aktsionernoe Obshchestvo "Ekopsi"
Priority date: 2013-12-27
Filing date: 2014-12-26
Publication date: 2015-07-02
Also published as: EA201400144A1; WO2015099581A3

Abstract

This invention belongs to the methods and systems which allow assessing remotely the personnel of entities. More specifically, the invention allows assessing competencies through remote observation of personnel behavior in simulated situations. The technical result of the invention (personnel assessment method) is an increase in the observed assessment parameters, causing more complete information received in assessing the participant, greater information reliability due to the possibility of monitoring the assignment accomplishment progress, which allows reducing the likelihood of prompts, cheat sheets, technical tools being used to help accomplish the assignments; more objective assessment due to the possibility of identifying misprints, correct line of reasoning despite incorrect answer and similar situations.

Description

METHOD AND SYSTEM OF EMPLOYEE COMPETENCY ASSESSMENT

FIELD OF THE INVENTION

This invention belongs to the methods and systems that allow assessing remotely personnel of entities. More specifically, the invention allows assessing competencies through remote observation of personnel behavior in simulated situations.

BACKGROUND OF THE INVENTION

Personnel assessment is a critical element of an entity's (organizations) personnel management system. Personnel assessment allows assessing its quality from the standpoint of conformity to the entity's business goals and strategy.

Personnel assessment is performed during different phases of the personnel management system functioning for a variety of purposes, specifically:

- in determining personnel requirements in the course of calendar planning. This phase involves assessing the existing workforce capacity and shaping personnel retention requirements;

- in selecting personnel with a view to determining the eligibility of candidates for vacant positions;

- in determining the need for personnel development. This phase involves assessing the existing versus required personnel levels and determining the necessity of training specific employees, checking the conformity of employee career levels to their competency;

- in personnel certification carried out on a regular basis for assessing workforce capacity level and developing a broad range of regulatory impacts, specifically:

• for taking measures related to remunerations, promotions, sanctions, dismissals;

• for personnel rotation planning;

• for succession pool planning;

• for developing employee motivation and stimulation.

Typically, the remote assessment task is accomplished through indirect interaction between the assessee and the assessing expert: the assessee completes a questionnaire or undergoes online testing, fulfills a competitive analytical or creative assignment, which is further submitted to the expert for assessment. The expert is a person specifically trained to observe, describe, classify and assess the behavior of the assessment event participant(s) in specific exercises, to interview and to align his/her assessments with other experts involved in the assessment event. The expert assesses a participant based on the findings. Such solution has the following drawbacks: first, inability to assess the assessee's communication skills and the associated competencies; second, the impossibility of expert assessment of behavior while executing a task; third, the impossibility of controlling whether or not the assessee fulfills assignments on his/her own.

The reliability, completeness and objectivity of the information received are critical to personnel assessment. Information reliability is the ability to be perceived correctly. Generally, information reliability is achieved through:

• specifying the time of events which details are delivered;

• comparing data received from different sources;

• timely discovery of misleading information;

· removal of distorted information, etc.

Information may be deemed complete if the minimal set of indicators it comprises is sufficient for making the right decision.

With standard testing techniques and remote personnel assessment, it is difficult to check the reliability of information received, the lack of third-party intervention when the participant fulfills an assignment (use of cheat sheets, wireless headphones and other technical tools).

Data completeness is crucial to assessment. It implies the availability of the entire necessary set of parameters being observed, ensuring the highest quality of assessment. When some parameters are missing, the quality and accuracy of assessment worsen greatly, for example, when assignments only include tests that show whether the participant answered any given assignment correctly or incorrectly, ignoring the possibility of misprint, accidental mistake and whether or not the test has been passed.

One of the existing methods is described in patent application US2012/007717 A1 "Competency assessment tool", March 29, 2012:

Key competencies of the participating employee are assessed using the competency assessment tool. The developer determines key competencies out of any set of topics and well-developed questions and demonstrates tasks for the corresponding training goals according to the principal competencies. Participants, in their turn, accomplish demonstration tasks assessed by observers and answer questions. Grade is calculated based on assessment and answer, with a view to determining how well the participant demonstrates training goals and competency, and then communicated to the participant. When the participant's score drops below the existing threshold, it is recommended that the administrator prepare development plan for the participant. In some options, the participant is an expert in the discipline and checks connection between the training goals and questions, demonstration tasks.

The major drawbacks of the above invention are asynchronous interaction between the participant and the expert, hampering the assessment of the participant's behavior while being assessed, and the lack of interactive exercises, which allow assessing the task executing progress as well as the result, the flexibility of decisions, change of argumentation focus depending on the exercise statement and other task executing qualities assessed based on the behavior observed in solving various analytic cases or in different simulated situations in the course of role- playing.

One of the existing methods is described in patent application US2003/0208752 "Employee candidate computer and web-based interactive assessment software and method of employee candidate assessment", November 6, 2003. Declared is the following:

Computer system for employee candidates and web-based interactive assessment software, the method of assessing the employee candidate's competencies and skills, offering potential candidates a series of employee-customer scenarios and associated questions requiring an interactive answer, in which the type of answers allows employers to select more effectively potential employees who have the qualifications necessary for a given position.

The invention's drawback is the impossibility of interaction between one or more assessees and one or more assessing experts within a single scenario. As a consequence, nothing but the assignment product is assessed whereas the product of analytical activity is inherent in the presentation of results, thus increasing the influence of the assessee's presentation skill on other qualities being assessed.

The above-mentioned technical solutions have the following drawbacks:

incomplete information received while assessing the participant, deteriorating the accuracy and quality of assessment, reduced information reliability, apparent from non-identifiability of the supporting materials and technical tools affecting the progress of assessment,

high level of subjectivity resulting from the impossibility of assignment progress monitoring, hampering the detection of correct solution despite incorrect response (such as misprints, etc.).

SUMMARY OF THE INVENTION

This invention is aimed at eliminating the inherent drawbacks of the existing solutions. The underlying approach offers the following benefits:

• effective remote assessment of multiple participants

· synchronous interaction between the participant and the expert that allows not only assessing the result but also using the information accompanying the progress of exercises, namely: behavior, task executing progress, the flexibility of decisions, etc.

• more accurate and complete assessment of the participant's competencies. The technical result of the invention (personnel assessment method) is an increase in the observed assessment parameters, causing more complete information received in assessing the participant, greater information reliability because of the possibility of monitoring the assignment progress, which allows reducing the likelihood of prompts, cheat sheets, technical tools being used to help accomplish the assignments; more objective assessment because of the possibility of identifying misprints, correct line of reasoning despite incorrect answer and similar situations.

An increase in the observed parameters is achieved by broadcasting video stream and ongoing (real-time) information about the progress of exercises from the participant in the course of assessment, including, but not limited to, the information about the current status of assignment, the text entered in the exercise input fields, the status of control elements (button pressed/released, flag selected, switch selected, etc.).

Some options of the invention offer feedback from the expert transmitted to the participant, which allows asking questions and finding out the progress of task executing by the participant.

The technical effects achieved when using the invention include greater efficiency of using time and technical resources of the system because of scheduling the accomplishment of exercises and commutations, thus shortening downtime and causing optimal software and hardware load. When the system is used by multiple participants and one expert, the processing rate equals to the duration of assessment session multiplied by the number of participants. When two or more experts are used by multiple participants, the processing rate decreases logarithmically.

Another technical effect of the invention is enhanced efficiency of communication channels since video broadcasting is carried out as needed rather than on an ongoing basis.

The use of the tasks sequence development (planning) method increases the speed of scheduling and lowers the likelihood of planning errors.

The employee's competency assessment method consists of the following steps:

• Obtainment of the assessment scenario comprising:

^■ at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

^■ order of executing the above-mentioned tasks;

^■ data about at least one assessee;

^■ data about at least one assessing expert;

• Consecutive execution of tasks according to the assessment scenario, in which case

- if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert.

· Processing of the assessment results.

The employee's competency assessment method, according to one of implementation options, consists of the following steps:

• Obtainment of the assessment scenario comprising:

^■ at least two tasks, in which case

at least one task is an exercise

- at least one task is commutation with the expert

^■ data at least about two assessees

^■ data at least about two assessing experts ■ order of executing the above-mentioned tasks for each participant

■ order of carrying out commutations between experts and participants

· Execution of tasks in the sequence received during the previous step for each participant, in which case

- if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert in the order for accomplishing commutations received during the previous step • Processing of the assessment results.

Data about the assessee comprise at least full name and data for authorization in the system, including at least login and password.

The assessing expert data comprise at least full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least login and password.

Information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed.

In certain implementation options, the assessment scenario is received from remote data storage or remote server.

In certain implementation options, the assessment scenario is received from local data storage.

An exercise may be performed in the form of a test, role-playing, analytic case, e- mail emulator.

In certain implementation options, the order for executing tasks may be changed by the expert.

In certain implementation options, the order for executing tasks by participants is established as follows: • At least the following is received:

- the list of participants comprising at least two participants;

- the list of tasks comprising at least two tasks;

- the list of experts comprising at least two experts;

• A task executing schedule is made for each participant;

• At least one connection is made between each participant and the expert from the expert list for all tasks in the schedule, in which case the schedule of tasks must comprise at least two tasks with references to different experts from the expert list.

The above method for establishing the order of exercises to be followed by participants may undergo the following modification, in which the shaping of participants' task executing schedule occurs in two phases: a. A preliminary sequence of tasks is made, in which case the sequence for execution of tasks by participants in even positions on the list differs from the sequence for participants in odd positions on the list; b. The operator prepares an optimal schedule based on the preliminary sequence of tasks.

Such methods may be implemented as computer appliance.

The employee competency assessment system includes:

a. one or more data storage devices;

b. one or more processors;

c. tools ensuring network interaction with remote computer systems;

d. at least one display;

e. at least one input device;

f. one or more computer programs,

where one or more computer programs are read from one or more data storage devices and run with one or more processors, with one or more programs comprising the following instructions:

^■ Obtainment of the assessment scenario comprising:

o at least two tasks, in which case

at least one task is an exercise;

at least one task is commutation with the expert.

o order of executing the above-mentioned tasks; o data about at least one assessee;

o data about at least one assessing expert; ■ Consecutive execution of tasks according to the assessment scenario, in which case

if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert.

^■ Processing of the assessment results.

The employee competency assessment system in one of the implementation options includes: a. one or more data storage devices;

b. one or more processors;

c. tools ensuring network interaction with remote computer systems; d. at least one display;

e. at least one input device;

f. one or more computer programs,

^■ Obtainment of the assessment scenario comprising:

o at least two tasks, in which case

at least one task is an exercise;

at least one task is commutation with the expert.

o data at least about two assessees;

o data at least about two assessing experts;

o order of executing the above-mentioned tasks for each participant;

o order of carrying out commutations between experts and participants.

^■ Execution of tasks in the sequence received during the previous step for each participant, in which case

if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert in the order for accomplishing commutations received during the previous step.

■ Processing of the assessment results.

The assessee data comprise at least full name and data for authorization in the system, including at least login and password;

The assessing expert data comprise at least full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least login and password; Information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed. In certain implementation options, the assessment scenario is received from remote data storage or remote server;

In certain implementation options, the assessment scenario is received from local data storage;

An exercise may be performed in the form of a test, role-playing, analytic case, e- mail emulator;

In certain implementation options, the task executing sequence may be changed by the expert;

In certain implementation options, the sequence of task executing by the participants is formed as follows: a. At least the following is received:

• the list of participants comprising at least two participants;

• the list of tasks comprising at least two tasks;

• the list of experts comprising at least two experts;

b. A task executing schedule is made for each participant;

c. At least one connection is made between each participant and the expert from the expert list for all tasks in the schedule, in which case the schedule must comprise at least two tasks with references to different experts from the expert list.

In certain invention implementation options, the shaping of participants' task executing schedule occurs in two phases:

a. A preliminary sequence of tasks is made, in which case the sequence for execution of tasks by participants in even positions on the list differs from the sequence for participants in odd positions on the list; b. The operator prepares an optimal schedule based on the preliminary sequence of tasks.

OVERVIEW OF CHARTS

Fig 1. - Functional chart of the assessment system.

Fig 2. - Chart of commutation between experts and participants in the 2 experts per 6 participants scenario.

Fig3. - Chart of commutation between experts and participants in the 2 experts per 4 participants scenario.

Fig 4. - 2 participants per 3 experts scenario.

Fig 5. - Session run delay time screen. Fig 6. - Exercise screen.

Fig 7. - Interface of the participant-expert interaction on the participant 's side.

Fig 8.—Session completion screen.

Fig 9 -Expert 's pre-session screen.

Fig 10 - Expert 's screen while the participant proceeds with the exercise.

Fig 11. -Expert 's screen during the interview.

Fig 12.—Expert 's screen after the session is over.

Fig 13.—Inbasket exercise chart. DETAILED DESCRIPTION OF DRAWINGS

Fig. 1 illustrates the functional chart of the assessment system. Scenario creation tools (2) are used to create a scenario or a family of scenarios (5) for the project, sharing the same content and differing in the number of experts and assessees. Then one scenario is selected from those assigned to the project for carrying out the assessment session. The session involves consecutive presentation of content (6) to the assessment event participants (9) and selective commutation of video communication (4) between the experts (1) and participants (9) according to the scenario. Content is downloaded (6) from external (8) or internal (7) sources. If necessary, any phase of the session (exercisee or video conference) for a given participant may be extended or postponed by one of the assessing experts (1).

Fig. 2 illustrates the chart of commutation between experts and participants in the following scenario: 2 experts per 6 participants. The Oy-axis shows the numbers of participants; the Ox-axis, the sequence numbers of video conferences. The points on the graph correspond to each meeting between an expert and a participant. The assessment session completion was followed by the joint work of experts to process and describe the results. As can be seen from the graph, each expert conducted 21 video conferences and each participant was involved in 7 video conferences. The ratio in the 6 participants per 2 experts scenario is 3 participants per expert during the session.

Fig. 3 illustrates the chart of commutation between experts and participants in the following scenario: 2 experts per 4 participants. In this scenario, each expert dealt with 2 participants per session. The axes of the graph are denominated similarly to the previous scenario option. As can be seen from the graph, each expert conducted 14 video conferences and each participant was still involved in 7 video conferences, i.e. the assessment event content for each participant does not depend on a specific scenario - instead, the scenario changes exclusively the model of load allocation among the experts and determines the content presentation sequence and the model of video streams commutation. Fig. 4 illustrates the schedule of a session involving three participants and two experts. The starting time of exercises for the third participant in the schedule is shifted relative to the first two in order to prevent interviews in the exercises of all the three participants from occurring simultaneously.

Fig. 5 shows the session run delay time screen. When entering the assessment session interface before it starts, the participant sees the session (10) schedule and the session run delay time indicator (1 1). Once the session starts, the screen is automatically updated and replaced by the participant's main screen.

Fig. 6 illustrates the exercise screen. While the participants is performing self- guided exercises and preparing for interactive exercises, the participant's interface displays the session schedule (10), highlighting the current exercise and the time remaining until the end of the current activity (14). The major workspace (17) comprises the assignment text or the participant's response field. Buttons above the workspace are used to switch between the assignment text and response (15, 16). When the participant enters his/her answer, the input text is displayed to the expert as entered. In the course of preparation, the participant is able to see his/her own video stream (12). Upon expiry of the intended preparation or exercise completion time, the participant is automatically switched to the interview mode.

Fig. 7 illustrates the interface of the participant-expert interaction on the participant's side. Overall, the participant's interface remains the same; the only differences are newly-added video frame for the expert's video stream (13) and blocked text input in the response field. Upon expiry of the intended exercise completion or interviewing time, the screen shows the assignment and instructions for the next exercise.

Fig. 8 illustrates the session completion screen. Upon completion of the last exercise/interview, the participant is redirected to the session completion screen.

Fig. 9 illustrates the expert's pre-session screen. When entering the assessment session interface before it starts (Fig. 9), the expert sees the time feed (20), highlighting the moments of interaction with the assessee (21). The time feed may be rescaled. The expert is able to select any event on the time feed, in which case space 26 will show the list of competencies (27, 27A) and their respective indicators (28) assessed in the selected exercise/interview. It is also possible to display on screen the documents related to the selected exercise/interview - fulfillment instructions, participant instructions, interview questions, assignment text, participant's response (which is empty before the exercise begins) and other documents by selecting the relevant point in the heading of space 26 or 30. Before the session starts, the participant's video frame is replaced by the message "Video will appear after the session start" (25). The session control panel (22) comprises two timers - timer from the beginning of the current activity (reset to zero before the session start), timer until the end of the current activity (before the session shows the time remaining until the session start) - and the "Start" and "Postpone" buttons. The session is deemed to have begun after the "Start" is pressed; if the participant has already entered the session, the broadcasting of his/her video stream begins. Pushing the "Postpone" button will postpone the session start time.

Fig. 10 illustrates the expert's screen while the participant proceeds with the exercise. After the session starts, the expert's interface looks similarly (Fig. 10) except that the message (25) is replaced by the participant's video frame (12) and the "Start" button (23) disappears from the session control panel. The comment field (29) is activated in the space which specifies the competencies being assessed (30). The comments added by the expert during the session are available while the results are being processed. Fig. 11 illustrates the expert's screen during the interview. When the participant moves on to the interview, the expert's video frame (13) and video stream control buttons such as sound transmission disconnection (33) and video transmission disconnection (34) appear in the interface. The session control panel (22) shows additional buttons for moving the current exercise to the session end (35) and for early exercise completion (36). When the participant moves on to the next exercise, the expert's interface status becomes similar to the pre-session status (Fig. 9); after the "Start" button is pushed, the expert's interface commences the participant's preparation monitoring (Fig. 10).

Fig. 12 illustrates the expert's screen after the session is over. Upon completion of all exercises/interviews in the session, the expert's interface looks as follows: the session control panel has two buttons - switch to results processing (37) and save grades assigned during the session subsequent processing (38). The participant's video frame is replaced with a static message (25).

Fig. 13 illustrates the Inbasket exercise chart. The input xml file presets the initial system status (3), the acceptable user actions (2) and the applicable processing rules (4). While doing the exercise, the assessee may only perform actions permitted by the exercise settings, namely: respond to letters, forward letters, complete tests, create calendar events, etc. Each action is accompanied by checking whether or not the rule related to the action being performed exists. If it does, the relevant scenario branch is activated; otherwise, the exercise scenario remains unchanged and the participant proceeds with the exercise.

DETAILED DESCRIPTION OF THE INVENTION

The general process of system functioning is outlined in Fig. 1.

The employee's competency assessment method consists of the following steps: • Obtainment of the assessment scenario comprising:

■ at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

■ order of executing the above-mentioned tasks;

^■ data about at least one assessee;

^■ data about at least one assessing expert;

• Integration session.

If necessary, the method described above may be modified as follows:

a. the assessment scenario comprises:

• at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

• data at least about two assessees;

• data at least about two assessing experts;

• order of executing the above-mentioned tasks for each participant;

• order of carrying out commutations between experts and participants.

b. execution of tasks in the sequence received during the previous step for each participant, in which case

if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert in the order of accomplishing commutations that was received during the previous step.

The above method may also be implemented as a software and hardware complex.

The employee's competency assessment system, including:

a. one or more data storage devices; b. one or more processors;

e. at least one input device;

f. one or more computer programs,

^■ Obtainment of the assessment scenario comprising:

o at least two tasks, in which case

^•S at least one task is an exercise;

• at least one task is commutation with the expert.

o data about at least one assessing expert;

^■ Consecutive execution of tasks according to the assessment scenario, in which case

^■ Processing of the assessment results.

In certain implementation options, the system may undergo the following modifications: a. the assessment scenario comprises:

• at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

• data at least about two assessees;

• data at least about two assessing experts;

• order of executing the above-mentioned tasks for each participant;

• order of carrying out commutations between experts and participants. b. execution of tasks in the sequence received during the previous step for each participant, in which case

In different implementation options, the method and the system may undergo the modifications described below.

The assessee data comprise at least a full name and data for authorization in the system, including at least a login and a password;

The assessing expert data comprise at least a full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least a login and a password;

• text entered in the input field;

· status of control elements;

• status of the exercise being performed.

In certain implementation options, the assessment scenario is received from remote data storage or remote server;

An exercise may be performed in the form of a test, role-playing, analytic case, e-mail emulator;

In certain implementation options, the task executing sequence may be changed by the expert; In certain implementation options, the order of exercises to be followed by participants is formed as follows: a. At least the following is received:

• the list of participants comprising at least two participants;

• the list of tasks comprising at least two tasks;

· the list of experts comprising at least two experts;

b. A task executing schedule is made for each participant; c. At least one connection is made between each participant and the expert from the expert list for all tasks in the schedule, in which case the schedule of tasks must comprise at least two tasks with references to different experts from the expert list.

a. A preliminary sequence of tasks is made, in which case the sequence for execution of tasks by participants in even positions on the list differs from the sequence for participants in odd positions on the list;

b. The operator prepares an optimal schedule based on the preliminary sequence of tasks.

Below is a step-by-step description of the system and method operation.

Step O

The assessment is to be preceded by the creation of accounts for assessees and the assessing experts (if not yet available), in which case the following data are to be preset for the assessee:

• Full name;

· Data for authentication.

The data to be preset for the expert are as follows:

• Full name;

• Data about the entity;

• Assessment system access rights;

· Data for authentication.

Data for authentication may include both common login/password pairs and authentication via digital certificate, smart card and other tools.

The assessment system access rights show which actions may be performed, which system sections are accessible, and which projects are available to the assessing expert. Inside the projects, the expert may be entitled to act as observer during the sessions, to conduct integration sessions, to change exercise instructions, etc. Following authentication, the assessee moves on to the session run delay time screen (Fig. 5); the expert, to the list of sessions that he/she is appointed to observe. From the list of sessions, he/she may navigate to the needed (up-to-date) session.

In advance, before the assessment (assessment session) start, it may be necessary to create a project, using the scenario creation tool (3) and the assessment scenario or family of scenarios (2) sharing the same content and differing in the number of experts and assesses.

"Scenario" means data comprising the schedule of execution of exercises and commutations within the assessment session for all participants and all assessing experts. Such schedule is exemplified in Fig. 2.

A single set of exercises may correspond to several scenarios differing in the initial quantity of experts and/or assessees, which allows adjusting flexibly to the assessment environment.

"Assessment session" means a set of exercises performed by each assessee with a view to assessing his/her professional and personal qualities.

"Project" means a group of assessment sessions using the same set of exercises and tests for executing the same assessment task.

In certain invention implementation options, the schedule is created using special automatic and semi-automatic algorithms. One of the semi-automatic scheduling options is the following algorithm:

The initial arrangement of exercises is preset by the simplest rearrangements (participant 1 is doing exercise A while participant 2 is doing exercise B and vice versa). Subsequent rearrangements in the schedule are carried out by the editor or another person in charge. That said, the algorithm checks and ensures that the observer does not work simultaneously with several participants or more than one expert work with one participant (in case of multiple experts in a scenario).

Moreover, the algorithms used in certain invention options may include, but not be limited to, genetic scheduling algorithms.

The video streams commutation chart as exemplified by a fragment of the two experts per three participants scenario is shown in Fig. 3. According to the chart, after the scenario loading, as soon as the assessment session starts, expert 1 is connected to participant 1; expert 2, to participant 2; meanwhile, participant 3 is still waiting for the assessment session to begin and is shown a message specifying the time remaining until connection of the expert. Upon completion of introduction and exercise 1, expert 1 is connected to participant 3, to whom this moment marks the beginning of the assessment session. Meanwhile, participants 1 and 2 are doing the exercise requiring no interaction with the expert and expert 2 is a having a break. Having finished the exercise without the observer, participants 1 and 2 are connected to expert 2 and expert 1 respectively. Unless expert 1 has finished video conference with participant 3 by that time, participant 2 is shown a message asking to wait for connection to the expert.

STEP 1

To initiate the assessment session, both the participant and the expert need to authorize in the system, typically using the login/password pair. Then the participant connects to the assessment session and the expert gets the system interaction interface.

Within the session, scenario is run on the user device, in which case content is presented according to the scenario. Content includes exercises, textual instructions and video instructions concerning exercises, as well as commutations with the expert.

Scenario is the sequence in which exercises are to be performed by the user and the order of commutations with the expert. Scenario may be either stored in a local data storage on the user device or received from remote data storages and remote servers.

The system comprises different types of exercises (see the details below).

Exercise type: Inbasket. When performing an Inbasket exercise, the assessee is able to perform the main actions in email client, namely: 1. To manage files and to move letters.

2. To respond to a letter.

3. To move a letter to the calendar.

4. To create a letter.

5. To use the calendar and to view events.

6. To create an event.

7. To use the address directory.

Apart from standard email client functions, the exercise interface allows integrating interfaces for completing questionnaires. Exercise type: case (analytic case). The idea of the exercise is that the assessee is given information for analysis and the instruction comprising wordings of information handling goals and objectives. As a rule, information for analysis and analytical task are related to the professional field of the assessee or that of the position sought by the assessee. The assessee analyzes independently the information received within the preparation time limits. Furthermore, he/she is allowed to write down his/her ideas, to mark the key points of the materials provided, to prepare the plan of his/her answer, etc. using the response field. When the time given for information analysis and preparation is over, the participant communicates with the expert by providing him/her with the analysis results, answering questions, etc.

Exercise type: role-playing. The idea of the exercise is that the assessee is given the instruction comprising the description of the current situation and the assessee's position in a certain simulated situation related to his/her professional field or that of the position sought by the assessee. The instruction also comprises the wordings of the assessee's goals and objectives in the upcoming interaction with other participants in the simulated situation. A short time for preparation is directly followed by role-playing - the participant meets the expert acting as his/her opponents in the simulated situation. Role-playing interaction is of limited duration. Once it is over, the participant and the expert proceed to discussing the quality of role-playing interaction, solutions and the degree of goals achievement by the participant.

The assessment session also includes the commutations of video connection between the assessee and the expert. Video connection commutation allows the expert to conduct a structured interview with the assessee for the purpose of assessing behavioral manifestations of the assessed qualities in non-role-playing situations and the assessee's experience.

Video connection commutation is based on the following principle:

(a) video stream is broadcast from each assessee while performing exercises and commutations with the expert, with the exception of breaks inside the scenario;

(b) during the video conference, the participant's video stream is broadcast to the expert participating in the meeting according to the scenario;

(c) during the video conference, video broadcasting from the expert to the corresponding participant begins. Video broadcasting may be suspended or terminated at the expert's sole discretion; (d) outside the video conference, video stream is broadcast from the assessee to the expert whose meeting comes first, specifically unless he/she is busy with another video conference;

(e) in certain invention implementation options, all video streams are saved in the video storage (4) and are available for reproduction both in the course and upon completion of the assessment session.

When entering the assessment session interface before it starts (Fig. 5), the participant sees the session schedule (10) and the indicator of the time remaining until the session beginning (11). The session beginning is accompanied by the automatic screen update and replacement by the participant's main screen, which is implemented in two options - the participant's screen during self-directed work (Fig. 6) and the participant's screen during interaction with the expert (Fig. 7). While the participants is performing self-guided exercises and preparing for interactive exercises, the participant's interface displays the session schedule (10), highlighting the current exercise and the time remaining until the end of the current activity (14). The major workspace (17) comprises the assignment text or the participant's response field. Buttons above the workspace are used to switch between the assignment text and response (15, 16). When the participant enters his/her answer, the input text is displayed to the expert as entered. In the course of preparation, the participant is able to see his/her own video stream (12). Upon expiry of the intended preparation or exercise completion time, the participant is automatically switched to the interview mode (Fig. 7). Overall, the participant's interface remains the same; the only differences are a newly-added video frame for the expert's video stream (13) and a blocked text input in the response field. Upon expiry of the intended exercise completion or interviewing time, the screen shows the assignment and instructions for the next exercise (Fig. 6). Upon completion of the last exercise/interview, the participant is redirected to the session completion screen (Fig. 8).

When entering the assessment session interface before it starts (Fig. 9), the expert sees the time feed (20), highlighting the moments of interaction with the assessee (21). The time feed may be rescaled. The expert is able to select any event on the time feed, in which case space 26 will show the list of competencies (27, 27A) and their respective indicators (28) assessed in the selected exercise/interview. It is also possible to display on screen the documents related to the selected exercise/interview - fulfillment instructions, participant instructions, interview questions, assignment text, participant's response (which is empty before the exercise begins) and other documents by selecting the relevant point in the heading of space 26 or 30. Before the session starts, the participant's video frame is replaced by the message "Video will appear after the session start" (25). The session control panel (22) comprises two timers - timer from the beginning of the current activity (reset to zero before the session start), timer until the end of the current activity (before the session shows the time remaining until the session start) - and the "Start" and "Postpone" buttons. The session is deemed to have begun after the "Start" is pressed; if the participant has already entered the session, the broadcasting of his/her video stream begins. Pushing the "Postpone" button will postpone the session start time. After the session starts, the expert's interface looks similarly (Fig. 10) except that the message (25) is replaced by the participant's video frame (12) and the "Start" button (23) disappears from the session control panel. The comment field (29) is activated in the space which specifies the competencies being assessed (30). The comments inserted by the expert during the session are available while the results are being processed. When the participant moves on to the interview, the expert's video frame (13) and video stream control buttons such as sound transmission disconnection (33) and video transmission disconnection (34) pop up in the interface. The session control panel (22) shows additional buttons for moving the current exercise to the session end (35) and for early exercise completion (36). When the participant moves on to the next exercise, the expert's interface status becomes similar to the pre-session status (Fig. 9); after the "Start" button is pushed, the expert's interface commences the participant's preparation monitoring (Fig. 10). Upon completion of all exercises/interviews in the session, the expert's interface looks as follows: the session control panel has two buttons— switch to results processing (37) and the saving of grades assigned during the session subsequent processing (38). The participant's video frame is replaced with a static message (25).

STEP 2

The assessment session completion is followed by the assessment results processing and description step. The system offers two methods for arranging the step, namely: the online method and the offline method. In the first case, the results of each participant are processed through the system web interface, in which case the processing of results may involve either one or two (or more) experts, working remotely or otherwise. In case of 2 remote experts, the results processing interface is used as a separable desktop; changes are synchronized automatically without any extra actions from users. It is possible to discuss the results online through video conferencing. The principle of video connection organization video connection organization is similar to the broadcasting of stream video from the expert during the assessment session: video broadcasting may start and end at the expert's sole discretion; however, unlike video conference during the assessment session, video connection in the integration session is available throughout the processing and description of results. In case two (offline processing of results), all data about the participant's progress with exercises, as well as the comments given by the assessing experts may be downloaded for processing to a local application. In addition, experts directly discuss the grade for each competency of each participant in the assessment event, adducing arguments in favor of any given grade, relying on the data stored in or downloaded from the system. The progress of discussion is similar to the convergence of grades after the assessment center. The final grade is recorded in the system through the online convergence interface. The integration session products are grades for competencies and the generated automatic report. An integration session is deemed incomplete unless the final grade is missing at least for one competency.

This invention improves the efficiency, completeness and quality of remote assessment, thus optimizing the time spent idle by participants and ensuring that hardware resources are utilized efficiently.

Claims

1. The employee's competency assessment method, including the following steps:

• Obtainment of the assessment scenario comprising:

^■ at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

^■ order of executing the above-mentioned tasks;

^■ data about at least one assessee;

^■ data about at least one assessing expert;

• Processing of the assessment results.

2. The method according to claim 1, wherein the assessee data comprise at least full name and data for authorization in the system, including at least a login and a password;

3. The method according to claim 1 , wherein the assessing expert data comprise at least full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least a login and a password;

4. The method according to claim 1, wherein information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed.

5. The method according to claim 1, wherein the assessment scenario is received from a remote data storage or a remote server;

6. The method according to claim 1, wherein the assessment scenario is received from a local data storage;

7. The method according to claim 1 , wherein an exercise may be performed in the form of a test, role-playing, analytic case, e-mail emulator;

8. The method according to claim 1, wherein the order for executing tasks may be changed by the expert;

9. The employee's competency assessment method, including the following steps:

• Obtainment of the assessment scenario comprising:

^■ at least two tasks, in which case

- at least one task is an exercise;

- at least one task is commutation with the expert.

^■ data at least about two assessees;

^■ data at least about two assessing experts;

^■ order of executing the above-mentioned tasks for each participant;

^■ order of carrying out commutations between experts and participants.

• Execution of tasks in the sequence received during the previous step for each participant, in which case

- if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert in the order for accomplishing commutations received during the previous step.

• Processing of the assessment results.

10. The method according to claim 9, wherein the assessee data comprise at least full name and data for authorization in the system, including at least a login and a password;

11. The method according to claim 9, wherein the assessing expert data comprise at least full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least a login and a password;

12. The method according to claim 9, wherein information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed.

13. The method according to claim 9, wherein the assessment scenario is received from a remote data storage or a remote server;

14. The method according to claim 9, wherein the assessment scenario is received from a local data storage;

15. The method according to claim 9, wherein an exercise may be performed in the form of a test, role-playing, analytic case, e-mail emulator;

16. The method according to claim 9, wherein the order for executing tasks may be changed by the expert;

17. The method according to claim 9, wherein the order of exercises to be followed by participants is formed as follows:

a. At least the following is received:

• the list of participants comprising at least two participants;

• the list of tasks comprising at least two tasks;

• the list of experts comprising at least two experts;

b. A task executing schedule is made for each participant;

18. The method according to claim 17, wherein the forming of participants' task executing schedule occurs in two phases:

19. The method according to claim 9, wherein the preliminary task executing order to be followed by participants is formed according to claim 17 or 18;

20. The employee's competency assessment system, including:

a. one or more data storage devices;

b. one or more processors;

e. at least one input device;

f. one or more computer programs,

■ Obtainment of the assessment scenario comprising:

o at least two tasks, in which case

at least one task is an exercise; at least one task is commutation with the expert.

o data about at least one assessing expert;

^■ Consecutive execution of tasks according to the assessment scenario, in which case if the task being executed is an exercise, video stream and information about the current exercise progress are transmitted from the assessee at least to one expert.

^■ Processing of the assessment results.

21. The system according to claim 20, wherein the assessee data comprise at least full name and data for authorization in the system, including at least a login and a password;

22. The system according to claim 20, wherein the assessing expert data comprise at least a full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least a login and a password;

23. The system according to claim 20, wherein information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed.

24. The system according to claim 20, wherein the assessment scenario is received from a remote data storage or a remote server;

25. The system according to claim 20, wherein the assessment scenario is received from a local data storage;

26. The system according to claim 20, wherein an exercise may be performed in the form of a test, role-playing, analytic case, e-mail emulator;

27. The system according to claim 20, wherein the order for executing tasks may be changed by the expert;

28. The employee's competency assessment system, including

a. one or more data storage devices;

b. one or more processors;

e. at least one input device;

f. one or more computer programs,

^■ Obtainment of the assessment scenario comprising:

o at least two tasks, in which case

at least one task is an exercise;

at least one task is commutation with the expert.

o data at least about two assessees;

o data at least about two assessing experts;

o order of executing the above-mentioned tasks for each participant;

o order of carrying out commutations between experts and participants.

^■ Processing of the assessment results.

29. The system according to claim 28, wherein the assessee data comprise at least a full name and data for authorization in the system, including at least a login and a password;

30. The system according to claim 28, wherein the assessing expert data comprise at least a full name, data about the entity, the assessment system access rights, data for authorization in the system, including at least a login and a password;

31. The system according to claim 28, wherein information about the current exercise progress comprises one of the following elements or their combination:

• text entered in the input field;

• status of control elements;

• status of the exercise being performed.

32. The system according to claim 28, wherein the assessment scenario is received from a remote data storage or a remote server;

33. The system according to claim 28, wherein the assessment scenario is received from a local data storage;

34. The system according to claim 28, wherein an exercise may be performed in the form of a test, role-playing, analytic case, e-mail emulator;

35. The system according to claim 28, wherein the order for executing tasks may be changed by the expert;

36. The system according to claim 28, wherein the order for executing tasks may be changed by the expert;

37. The system according to claim 28, wherein the order of exercises to be followed by participants is formed as follows:

a. At least the following is received:

• the list of participants comprising at least two participants;

• the list of tasks comprising at least two tasks;

• the list of experts comprising at least two experts;

b. A task executing schedule is made for each participant;

38. The system according to claim 37, wherein the forming of participants' task executing schedule occurs in two phases: