KR101827651B1 - Method and apparatus for rating service - Google Patents

Abstract

The present invention relates to an evaluation service method and a device thereof. According to an embodiment of the present invention, the evaluation service method comprises: a step of grouping a plurality of registered examination objects into a predetermined number, and defining an examination unit; a step of transmitting information for an examination object for each examination unit to a plurality of evaluator terminals, and requesting an evaluation for each of the examination objects included in each examination unit; a step of receiving an examination score or ranking information of each examination object for each examination unit from each of the evaluator terminals, and storing the same; a step of selecting ranking for the plurality of examination objects by using the examination score or the ranking information; and a step of performing capability evaluation of an evaluator based on the ranking of the selected examination object, and the examination score or the ranking information of the evaluator. The present invention can provide a fair evaluation result based on an intuitive evaluation of a plurality of evaluators without relying on evaluation results of minor evaluators.

Description

[0001] METHOD AND APPARATUS FOR RATING SERVICE [0002]

At least some embodiments of the present disclosure relate to an evaluation service method and apparatus for fairly and objectively evaluating a number of audiences.

In beauty pageants and various public auditions, there are many cases where a group of experts evaluate a plurality of applicants. In such a case, there may be a problem in the selection of the experts themselves, and an unfair evaluation can be made according to the personal tendency of the experts.

In addition, in order to secure such a problem under the mobile environment, mobile voting is performed by public voters such as viewer participation.

However, even in such cases, it is often the case that a large number of cognitive efforts are required which require the involvement and concentration of the evaluation, and the results are often distorted depending on the degree of evaluation of individual evaluators

Therefore, there is a need for a new method of evaluation that improves the evaluation process and gives higher reliability to the evaluation results based on fairness and objectivity.

An embodiment of the present invention is intended to provide an evaluation service method and apparatus capable of drastically improving fairness and objectivity.

The evaluation method according to an embodiment of the present invention includes the steps of grouping a plurality of objects to be registered into a predetermined examination unit, transmitting information about the object to be examined to each of the examination units by the examination unit, A step of receiving examination scores or ranking information of each examination subject for each examination unit from each examination terminal and storing the examination scores or ranking information of each examination subject, And performing evaluation of the competitor based on the ranking of the selected examination subject and the examination score or ranking information of the examiner.

According to an embodiment of the present invention, an evaluation apparatus includes a storage unit for storing information on a plurality of examination subjects to be registered, a communication unit for transmitting and receiving information for evaluation with a plurality of examiner terminals, And a control unit for defining a plurality of examination units, wherein the examination unit definition unit defines a plurality of examination units by grouping the plurality of examination units by a predetermined number, A judging requesting unit for transmitting the judging score or the ranking information of each examination subject for each evaluation unit received from a plurality of judge terminals through the communication unit to the storage unit, The order of ranking for a number of audiences, the ranking of the selected audiences, and the auditor's assessment points And an evaluator capability evaluation unit for evaluating the competence of the evaluator based on the number or ranking information.

The examination request unit of the evaluation service apparatus according to the embodiment of the present specification can refer to past competence evaluation results of the examiner to select the examinee who is to perform the evaluation.

The examination unit definition unit according to an embodiment of the present invention confirms whether or not a ranking is selected according to a predetermined selection criterion, and when a plurality of examination scores or ranking information of each examination group for each group is received and stored according to an identification result, Based on the examination score or ranking information of the subject to be evaluated, and subjects the subject to be continuously evaluated and regroups based on the same score or the same ranking information.

The judge competency evaluation unit according to an embodiment of the present invention may determine the evaluation score or rank information evaluated by the judge based on the plus and minus scores or accuracy Evaluate the competence of the judges while giving a score according to the number.

The ranking unit according to the embodiment of the present invention disregards the examination score or rank information of the corresponding examination unit received after reaching a predetermined target return value for the same examination unit.

These general and specific aspects may be implemented by using a system, method, computer program, or any combination of systems, methods, and computer programs.

According to one embodiment of the present disclosure, it is possible to provide a fair evaluation result based on an intuitive evaluation of multiple auditors without depending on the evaluation result of the minority evaluators.

According to one embodiment of the present invention, an evaluator can evaluate an individual competency by merely evaluating an object to be evaluated, and an individual competency is given an accuracy weight by comparing with a plurality of evaluation result values. Therefore, The reliability of the result can be increased.

1 is a block diagram illustrating an evaluation system interworking with a plurality of evaluation results according to an embodiment of the present invention.
2 is a block diagram illustrating an evaluation apparatus according to an embodiment of the present invention.
3 is a flowchart for explaining an evaluation method of an evaluation apparatus according to an embodiment of the present invention.
4 is a flowchart for explaining re-grouping of the evaluation method of the evaluation apparatus according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In describing the embodiments, descriptions of techniques which are well known in the technical field to which this specification belongs and which are not directly related to this specification are not described. This is for the sake of clarity without omitting the unnecessary explanation and without giving the gist of the present invention.

For the same reason, some of the components in the drawings are exaggerated, omitted, or schematically illustrated. Also, the size of each component does not entirely reflect the actual size. In the drawings, the same or corresponding components are denoted by the same reference numerals.

Prior to the description of the present invention, the terms used in this specification, the subjects to be audited, the auditors, the auditors, and the capability evaluation will be described.

In this specification, the term 'object to be judged' is an object to be judged for determining a grade, a ranking (score), or an admission, and may be a candidate for various contests, aesthetic object such as a picture piece, A work such as a contest or sound recording, and the like.

In the present specification, the term 'auditor' is a subject for determining a rating, a ranking, or an admission for an audition object through the evaluation system of the present invention.

In this specification, 'judging' refers to a process by which the 'judge' decides the grade, ranking, or admission for the 'subject of examination'.

In this specification, 'capacity assessment' refers to the process by which the 'evaluator' evaluates the level of 'competence' for examining 'subject'.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described with reference to the drawings for explaining an evaluation service method and an apparatus according to embodiments of the present invention.

1 is a block diagram illustrating an evaluation system interworking with a plurality of evaluation results according to an embodiment of the present invention.

Referring to FIG. 1, an evaluation system linked to a plurality of evaluation results according to an embodiment of the present invention includes at least one evaluation target registration terminal 10, an evaluation apparatus 100, and a plurality of reviewer terminals 20, As shown in FIG.

The registration target terminal 10 for examination includes a mobile phone, a smart phone, a personal digital assistant (PDA), a personal computer (PC), a tablet, and the like as a terminal for registering an examination subject in the evaluation apparatus 100 and requesting examination services. A personal computer (PC), a notebook, and the like.

The communication network may be any of a variety of communication methods such as TCP / IP, Local Area Network (LAN), WIFI, Long Term Evolution (LTE), Wideband Code Division Multiple Access (WCDMA) May be implemented using at least some of them. Many communications are performed through the communication network, but in the following description, the description of the communication network is omitted in order to simplify the explanation.

The judging terminal 20 is a terminal for the judge to input the examination result of the examination subject and provide the examination result to the evaluation apparatus 100. The examination terminal 20 includes a mobile phone, a smart phone, a personal digital assistant (PDA), a personal computer (PC) A personal computer (PC), a notebook, and the like.

The evaluation apparatus 100 first receives information on the examination subject from one or more examiner terminals 20, and registers the examination subject.

The evaluating apparatus 100 requests the examinees to examine the registered examination subject, receives the examination result, and selects the ranking of the examination subject.

In addition, the evaluating apparatus 100 evaluates individual competencies of the judges based on the judging results of the judges and the ranking results of the selected judges.

The specific operation of each component of the evaluation apparatus 100 will be described later in detail with reference to Figs. 2 to 4. Fig.

2 is a block diagram illustrating an evaluation apparatus according to an embodiment of the present invention.

2, an evaluation apparatus according to an embodiment of the present invention includes a communication unit 110, a judging unit definition unit 120, a judging request unit 130, a ranking selecting unit 140, a judging competency evaluating unit 150 and a storage unit 160. The examination unit definition unit 120, the examination request unit 130, the rank selection unit 140, and the examiner capability evaluation unit 150 may be collectively referred to as a control unit.

The communication unit 110 is a data transmission / reception device provided in the evaluation apparatus 100, and transmits / receives information for reviewing an examination target with an external terminal through a communication network.

The audit unit definition unit 120 groups the plurality of audit objects to be registered into a predetermined audit unit.

The examination requesting unit 130 transmits information about the examination subject to a plurality of examiner terminals by the examination unit through the communication unit 110 to request examination for each examination subject included in the examination unit.

The rank selection unit 140 stores the examination scores or ranking information of each examination subject for each examination unit received from a plurality of examination terminals through the communication unit 110 in the storage unit, The rankings are selected for the subject of examination.

In addition, when the rank selection unit 140 reaches a predetermined target return value for the same examination unit, the ranking selection unit 140 can terminate the reception of examination information for the examination unit. Needless to say, the target return value can be set flexibly according to the customer's needs.

The judging person competency evaluation unit 150 evaluates the competence of the judging person on the basis of the ranking of the object to be judged and the judging score or ranking information of the judging person.

The storage unit 160 stores information on a plurality of examination objects to be registered.

The storage unit 160 stores programs and data necessary for the operation of the evaluation apparatus 100, and can be divided into a program area and a data area.

The program area includes a program for controlling the overall operation of the evaluation apparatus 100, an operating system (OS) for booting the evaluation apparatus 100, an application program necessary for reproducing the multimedia contents, And application programs necessary for evaluator competency evaluation. The data area is an area in which data generated according to the operation of the evaluation apparatus 100 is stored. The data area includes information of a subject to be evaluated, a score or rank information of the subject to be registered, an examination result of the subject, And the results of the competency evaluation of the user.

3 is a flowchart for explaining an evaluation method of an evaluation apparatus according to an embodiment of the present invention.

For example, contest participants upload an application or photographs (which may include a moving picture) required by the contest to the evaluation apparatus 100 through the registration terminal 10, and perform participant registration (registration of interest for examination).

The evaluation apparatus 100 groups a plurality of examination objects registered through the registration terminal 10 by predetermined examination units (S110).

For example, each contest participant is randomly grouped by a predetermined number, e.g., six. One group consisting of six people will be the judging unit.

Information about the object to be audited is transmitted to a plurality of auditor terminals by each audit unit, and an audit is requested for each audit object included in the audit unit (S120).

(A), (b), (c), (d), (e), and (f) to the judging panel. At this time, it is possible to directly provide an interface for conducting the examination or to transmit the address to the web page where the examination can be performed. The reviewer may also provide a brief profile of the subject.

A reviewer who will conduct the examination can be selected by referring to the results of competence evaluation of past reviewers. For example, an auditor with an excellent competency evaluation may be selected at a selection contest last year. However, since it is difficult to evaluate the competence of the judges with only one evaluation, it is possible to give a predetermined number of evaluation times as a condition. That is, the evaluation request message can be transmitted to the judging terminal selected and selected on the basis of the evaluation result of the judges, to the judges who have participated in the evaluation of the evaluation more than three times in the similar selection contest.

The examination score or ranking information of each examination subject for each examination unit is received from each judge terminal and stored (S130).

The judge who is requested to examine the group received by the judge terminal performs the examination based on the subjective criterion of the examination subject and transmits the examination result to the evaluation apparatus through the examination terminal. The judge can conduct the examination in the order of 1 to 6 according to his / her subjective judgment about 6 persons (a, b, c, d, e, f) Here, the examination may be the order of preference, the order of favorability, but it is not limited thereto.

At this time, if the target return value defined in advance for the same judging unit is reached, the judging score or the ranking information for the subsequent judging unit received can be ignored.

For example, when the target return value is 1000, the examination result for the 1 group is processed as the examination result value valid only up to 1000 times according to the received order, and the examination result value received after 1001 times is not reflected in the examination of the examination subject have.

The evaluation apparatus selects a ranking for a plurality of examination subjects using the examination score or ranking information received from the examination terminal (S140).

The evaluation apparatus compiles the examination results of the examination subjects included in the group for each group and performs the comprehensive examination.

The evaluator's capability evaluation is performed based on the ranking of the selected examination subject and the examination score or ranking information of the examiner (S150).

By comparing the examination results of the judges by the unit examination groups with the comprehensive evaluation results, it is possible to objectify the examination ability of the judges. For example, judges in a group can objectify the examination capability by comparing the result of the comprehensive examination with the result of the examination of the applicant and receiving the defined score corresponding to the matching number.

At this time, the judge evaluates the judging ability or ranking information of the judging object or the ranking information according to the selected number based on the plus and minus points or the accuracy based on the comparison result of the final judging object. do. The reason for using negative scores is that if you participate in the evaluation simply, you can receive a lot of evaluation scores, which can lower the credibility of evaluator competency evaluation.

4 is a flow chart illustrating re-grouping of an evaluation method according to an embodiment of the present invention.

After step S140 of FIG. 3, if it is desired to continue the examination, that is, it is confirmed whether a predetermined number of examination objects have been selected (S210).

If a desired number of objects are not selected yet, the object to be continuously screened is selected and re-grouped based on the scores or ranking information of a plurality of objects to be inspected by each group (S220).

For example, the ranking of a group of audiences may be judged by a plurality of auditors by constructing a new group with the same ranking (or rating) of another group. Thereafter, through the steps S230 and S240, the overall ranking is determined based on the cumulative score of each ranking, and all the participants can confirm the individual ranking.

The plurality of judged objects to be re-grouped are transmitted to the plurality of judging terminals by the information on the objects to be judged for each of the re-grouped groups, and the respective examination objects included in the judging unit are requested for examination (S230).

The judging score or ranking information of each judging target of the group re-grouped from each judging terminal is received and stored (S240).

The ranking is selected for the plurality of examination objects using the examination score or ranking information (S250).

When step S250 is finished, the process proceeds to step S160 in Fig.

If it is determined in step S210 that a desired number of examination objects have been selected, the process directly proceeds to step S160.

As a result of this repetition, the ranking or rankings of each screening target for the whole screening object, that is, the whole screening object, can be expressed in percentage or percentage.

At this point, it will be appreciated that the combinations of blocks and flowchart illustrations in the process flow diagrams may be performed by computer program instructions. These computer program instructions may be loaded into a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, so that those instructions, which are executed through a processor of a computer or other programmable data processing apparatus, Thereby creating means for performing functions. These computer program instructions may be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner. Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible for the instructions to perform the processing equipment to provide steps for executing the functions described in the flowchart block (s).

Herein, the term " part " used in the present embodiment means a hardware component such as software or an FPGA or an ASIC, and 'part' performs certain roles. However, 'part' is not meant to be limited to software or hardware. &Quot; to " may be configured to reside on an addressable storage medium and may be configured to play one or more processors. Thus, by way of example, 'parts' may refer to components such as software components, object-oriented software components, class components and task components, and processes, functions, , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided in the components and components may be further combined with a smaller number of components and components or further components and components. In addition, the components and components may be implemented to play back one or more CPUs in a device or a secure multimedia card.

It will be understood by those skilled in the art that the present specification may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present specification is defined by the appended claims rather than the foregoing detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents are included in the scope of the present specification Should be interpreted.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is not intended to limit the scope of the specification. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

10: Registration terminal
20: a plurality of judging terminals
100: evaluation device
110:
120: Evaluation unit definition section
130:
140: rank selection unit
150: Examiner Competency Evaluation Department
160:

Claims (5)

Grouping a plurality of objects to be registered into predetermined examination units;
Transmitting information on an object to be audited to a plurality of auditor terminals for each audit unit and requesting audit for each audit object included in the audit unit;
Receiving and storing examination scores or ranking information of each examination subject for each examination unit from each examination terminal;
Selecting a ranking for a plurality of examination objects using examination scores or ranking information; And
Performing evaluation of the evaluator's competence based on the ranking of the selected review object and the evaluation score or ranking information of the reviewer
Lt; / RTI >
The method of claim 1, further comprising the steps of: transmitting, to each of the plurality of reviewer terminals by the review unit, requesting review of each of the review objects included in the review unit,
Step of selecting a person to be judged to perform the examination by referring to the past evaluation results of the evaluator's competency
Further comprising the step of: delete The method according to claim 1,
After the step of selecting a ranking for a plurality of examination subjects using the examination score or ranking information,
Confirming whether a ranking has been selected according to a predetermined selection criterion;
Selecting an examination subject to be continuously screened based on the examination scores or ranking information of a plurality of examination subjects for each group and re-grouping based on the same score or the same ranking information according to the result of the verification;
Transmitting information on a plurality of re-grouped objects to a plurality of reviewer terminals and requesting examination of each of the objects included in the review unit;
Receiving and storing examination scores or ranking information of each examination subject of the group from each examination terminal;
Selecting a ranking for a plurality of examination subjects using the examination scores or ranking information
Further comprising the step of: The method according to claim 1,
The step of evaluating the competitor based on the ranking of the selected examination subject and the examination score or ranking information of the examiner,
The examiner's competency score is given according to the number of matches based on the plus score and the minus score or the accuracy in accordance with the comparison result of the judging score or the ranking information judged by the judge on the object to be judged, . A storage unit for storing information on a plurality of objects to be registered;
A communication unit for transmitting and receiving information for examination with a plurality of judge terminals;
A judging unit definition unit for grouping a plurality of registered objects to be registered into a predetermined judging unit;
An examination requesting unit for transmitting information on an examination subject to a plurality of examiner terminals for each examination unit through the communication unit to request examination for each examination subject included in the examination unit; And
The examination score or ranking information of each examination subject for each examination unit received from a plurality of examination terminals through the communication unit is stored in the storage unit and a ranking is selected for a plurality of examination subjects using the examination score or ranking information Ranking order; And an evaluator competence evaluation unit for evaluating the competence of the evaluator based on the ranking of the selected evaluation target and the evaluation score or ranking information of the evaluator
Lt; / RTI >
Wherein the examination requesting unit selects an examiner to perform the examination by referring to the competency evaluation result of the examiner of the examiner competency evaluation unit.

Images