KR20170064881A - Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor - Google Patents

Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor Download PDF

Info

Publication number
KR20170064881A
KR20170064881A KR1020150170894A KR20150170894A KR20170064881A KR 20170064881 A KR20170064881 A KR 20170064881A KR 1020150170894 A KR1020150170894 A KR 1020150170894A KR 20150170894 A KR20150170894 A KR 20150170894A KR 20170064881 A KR20170064881 A KR 20170064881A
Authority
KR
South Korea
Prior art keywords
evaluator
evaluation
participant
evaluation result
weight
Prior art date
Application number
KR1020150170894A
Other languages
Korean (ko)
Inventor
이수진
송정욱
Original Assignee
주식회사 크라우디즌
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 크라우디즌 filed Critical 주식회사 크라우디즌
Priority to KR1020150170894A priority Critical patent/KR20170064881A/en
Publication of KR20170064881A publication Critical patent/KR20170064881A/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07B—TICKET-ISSUING APPARATUS; FARE-REGISTERING APPARATUS; FRANKING APPARATUS
    • G07B7/00—Holders providing direct manual access to the tickets

Landscapes

  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Theoretical Computer Science (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Game Theory and Decision Science (AREA)
  • Development Economics (AREA)
  • Educational Technology (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The present invention relates to an evaluation apparatus considering an evaluator tendency, an online evaluation system and a method considering the evaluator tendency, and an evaluator evaluates an answer of a participant in various entrance examinations or competition evaluations that can be performed on-line, The evaluator considers the evaluator's tendency to maximize the objectivity and fairness of the evaluation but to give a reasonable weight to the evaluation by reducing the score deviation according to the evaluator's evaluation tendency by weighting the evaluation result in consideration of the evaluation tendency of the evaluator, And an online evaluation system and method considering an evaluator tendency.

Description

[0001] The present invention relates to an evaluating apparatus considering an evaluator tendency, an online evaluating system considering the evaluator tendency, and an online evaluating system.

The present invention relates to an evaluation apparatus considering an evaluator tendency, an online evaluation system and a method considering an evaluator tendency, and an evaluator evaluates an answer of a participant in various entrance examinations or competition evaluations that can be performed on-line, The evaluator considers the evaluator's tendency to maximize the objectivity and fairness of the evaluation but to give a reasonable weight to the evaluation by reducing the score deviation according to the evaluator's evaluation tendency by weighting the evaluation result in consideration of the evaluation tendency of the evaluator, And an online evaluation system and method considering an evaluator tendency.

As the information and communication technology develops, many technologies for implementing various business models are appearing on a large scale, and a large number of participants, such as entrance examination, competition, or competition, Evaluation and evaluation results are often provided online.

In the case of enrollment, competition, or contest, the number of participants is large. Therefore, the evaluator must select a large number of candidates in advance, and the evaluator will distribute the entire evaluator's answer to the whole evaluator according to a certain method.

Conventionally, since it is difficult for each of the evaluators to evaluate all the participants, any participant is evaluated only by a specific evaluator or a group of some evaluators. Therefore, according to the evaluator's tendency, meet a generous evaluator or a strict evaluator There is a possibility that the objectivity and fairness will be damaged.

As one of the prior arts for conducting various evaluations online, Korean Patent Laid-Open No. 10-2005-0008433 " On-line employment support system and method using the same " , Which is a specialized online recruiter in the related field that is connected to the Internet.

However, Korean Patent Laid-Open No. 10-2005-0008433 also includes a technology for simply providing a specific evaluation result on-line, and has the same problems as the above-described conventional technology.

An object of the present invention is to overcome the above problems and to provide a method and system for evaluating an evaluation result of an evaluator in evaluating an answer of a participant in various entrance examination evaluation or competition evaluation which can be performed on- The evaluator's tendency to consider the evaluator's tendency and to suggest a method to maximize the objectivity and fairness of the evaluation but to give a reasonable weight by reducing the score deviation according to the evaluator's tendency And to provide an online evaluation system and method that are considered.

The present invention relates to an evaluation device that takes into consideration an evaluator tendency, and it is an object of the present invention to provide an evaluation device that receives primary evaluation result information from a separate terminal, apparatus, or server, analyzes the evaluation tendency information of the evaluator included in the primary evaluation result information, The final evaluation result can be calculated by weighting and correcting the primary evaluation result according to the information, and the final evaluation result can be output.

According to another aspect of the present invention, there is provided an online evaluation system that takes into consideration an evaluator tendency, comprising participation participation means for receiving participation application, participant information and entry information from a participant terminal, An evaluation result receiving means for receiving an evaluation result, and a control means for performing a T-test or a Pearson correlation method on the evaluation results of the first evaluator group and the second evaluator group of the evaluator group, And an evaluation result providing means for providing the final evaluation result to the participant terminal; A participant terminal for transmitting an entry request, participant information, and entry information to the central server and receiving a final evaluation result from the central server; And an evaluator terminal for receiving the evaluation object information from the central server and transmitting the primary evaluation result to the central server.

At this time, among the weights given by the evaluation result correction means, the weights according to the T-test method

Figure pat00001
Is calculated according to the following equation.

Figure pat00002
=
Figure pat00003

Figure pat00004

(In the above equation,

Figure pat00005
Is a participant or evaluator for obtaining the weight,
Figure pat00006
The questionnaire for evaluation of the evaluation subject,
Figure pat00007
silver
Figure pat00008
of
Figure pat00009
Data for obtaining a weight,
Figure pat00010
silver
Figure pat00011
This decision
Figure pat00012
It means a bundle of scores.)

In this case, among the weights given by the evaluation result correction means, a weight according to the Pearson correlation method

Figure pat00013
Is calculated according to the following equation.

Figure pat00014

Figure pat00015

(In the above equation,

Figure pat00016
Is a participant or evaluator for obtaining the weight,
Figure pat00017
The questionnaire for evaluation of the evaluation subject,
Figure pat00018
silver
Figure pat00019
of
Figure pat00020
Data for obtaining a weight,
Figure pat00021
silver
Figure pat00022
This decision
Figure pat00023
It means a bundle of scores.)

In this case, the final evaluation result calculated by the evaluation result correction means is expressed by the following equation.

Figure pat00024

(In the above equation,

Figure pat00025
The questionnaire for evaluation of the evaluation subject,
Figure pat00026
Is the weight calculated according to the T-test method,
Figure pat00027
Means the weight calculated according to the Pearson correlation method.)

At this time, the first evaluator group is one of the evaluator groups related to the same participant, and each evaluator belonging to the evaluator group related to the same participant is set as the first evaluator group and the calculation according to the T-test or the Pearson correlation method is repeated .

According to another aspect of the present invention, there is provided an online evaluation method that takes into consideration an evaluator tendency, the method comprising: (A) receiving a participation application, participant information and entry information from a participant terminal; (B) the central server transmits evaluation target information to the evaluator terminal and receives a first evaluation result from the evaluator terminal; (C) The central server applies a T-test or a Pearson correlation method to the evaluation results of the first evaluator group and the second evaluator group among the evaluator groups related to the same participant, by weighting and correcting the primary evaluation results Calculating a final evaluation result by correcting; And (D) the central server providing the final evaluation result to the participant terminal.

At this time, the weighting method in the online evaluation method considering the evaluator tendency can adopt the weighting method in the online evaluation system considering the tendency of the evaluator.

According to the present invention, in evaluating an answer of a participant in an entrance exam evaluation or a competition evaluation which can be carried out on-line, the objective evaluator can maximize the objectivity and fairness of evaluation by reducing the score deviation according to the evaluator's evaluation tendency However, there is an effect of maximizing the rationality of the method of weighting.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of an on-line evaluation system considering an evaluator tendency according to a preferred embodiment of the present invention; FIG.
FIG. 2 is an overall flowchart of an online evaluation method considering an evaluator tendency according to a preferred embodiment of the present invention. FIG.

Before describing the specific details for the practice of the invention, terms and words used in the specification and claims should be construed to enable the inventor to properly define the concept of a term in order to best describe its invention It should be interpreted as meaning and concept consistent with the technical idea of the present invention.

It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an evaluating apparatus considering an evaluator tendency according to a preferred embodiment of the present invention will be described in detail.

The evaluation device considering the evaluator tendency according to the preferred embodiment of the present invention can be applied to various situations in which evaluation is performed on participants such as various examinations, tests, evaluations, competitions, and competition held on-line or off-line, May be used as a component or module of any server, or may be embodied and used as a computer program or the like.

The evaluation apparatus considering the evaluator tendency according to the preferred embodiment of the present invention receives primary evaluation result information for each participant from a separate terminal, apparatus, server, or the like, and evaluates each evaluator included in the primary evaluation result information And the final evaluation result is calculated and outputted by weighting and correcting the primary evaluation result according to the evaluation tendency information of each evaluator by analyzing the inclination information.

At this time, before the evaluating apparatus considering the evaluator tendency according to the preferred embodiment of the present invention receives the first evaluation result information, the function of receiving the participation acceptance from the participant, the function of selecting the evaluator, May be similar to the functions performed by the participation reception means 110 and the evaluation result reception means 120 of the central server 100 in the online evaluation system taking into account the evaluator tendency to be described later. The function of providing the evaluation result after the evaluation apparatus considering the evaluator tendency according to the example outputs the final evaluation result may be provided by the evaluation result providing means 140 of the central server 100 in the online evaluation system taking into account the evaluator tendency to be described later And these functions may be performed by a separate terminal, a device, a server, or the like.

More specifically, the functions performed by the evaluation apparatus in consideration of the evaluator tendency according to the preferred embodiment of the present invention will be described in more detail. Of the evaluator groups related to the same participant, the first evaluator group and the second evaluator group T-test or Pearson correlation method is applied to the evaluation result, and the final evaluation result is obtained by multiplying the evaluation result before weighting by the weight.

The first evaluator group may be a specific first evaluator and the second evaluator group may be an evaluator group other than the first evaluator who has evaluated the same evaluation object as the first evaluator. After calculating the result of calculation by T-test or Pearson correlation method for one evaluator (first evaluator), the same calculation is repeatedly performed for the second evaluator, the third evaluator, etc., When the performance is completed, the final weighting result can be derived.

The evaluation result before weighting may be the result of the primary evaluation for each participant. The weight may be a weight according to the T-test or the Pearson correlation method, or a weight obtained by summing the weights according to the two methods.

In using the T-test method, the T-test calculation result can be determined as a weight by adjusting (scaling) the value from 0 to 1, etc., and if the calculation is mathematically impossible such as a denominator being 0 in the T-test calculation The weight can be set to zero. Likewise, when using the Pearson correlation method, the result of calculating the Pearson correlation can be determined as a weight by adjusting (scaling) the value from 0 to 1. For example, if the denominator is zero in the pearson correlation calculation, Can be set to zero.

The evaluating apparatus according to the preferred embodiment of the present invention can calculate the integrity information of the evaluator to be evaluated. The integrity information may be a value obtained by dividing the number of evaluations completed by the number of evaluations.

In addition, the specific weighting and evaluation result correction function performed by the evaluation apparatus considering the evaluator tendency according to the preferred embodiment of the present invention is similar to the function performed by the evaluation result correction means 130 in the online evaluation system described later .

Hereinafter, an online evaluation system considering an evaluator tendency according to a preferred embodiment of the present invention will be described in detail with reference to FIG.

1 is an overall configuration diagram of an online evaluation system considering an evaluator tendency according to a preferred embodiment of the present invention.

1, an online evaluation system considering an evaluator tendency according to a preferred embodiment of the present invention includes a central server 100, a participant terminal 200 and an evaluator terminal 300, and the central server 100 May be composed of one or a plurality of servers, and the participant terminal 200 and the evaluator terminal 300 may be one or more.

In the preferred embodiment of the present invention, the on-line evaluation is assumed to be an online competition, and the present invention is not limited thereto, but an arbitrary participant is evaluated on-line through various examinations, tests, Can be included.

The central server 100 is a server of a company that provides an online evaluation service considering an evaluator tendency according to a preferred embodiment of the present invention and includes a participation accepting unit 110, an evaluation result receiving unit 120, an evaluation result correcting unit 130 And an evaluation result providing means 140.

First, the participation accepting means 110 receives the participation application, participant information, and entry information from the participant terminal 200.

The participant accepting means 110 can also provide the event information to the participant terminal 200 via the web, the mobile web, or a mobile application.

In the present invention, the tournament holding information may include the name of the tournament, the purpose of the tournament, the outline of the tournament, the details of the tournament, the qualification of the tournament reception, the tournament reception period,

In the present invention, an entry is an object to be evaluated by a participant in various contests and may be various fields such as art, music, literature, photographs, and images, and the entry information may be received on-line It will be possible.

Next, the evaluation result receiving means 120 selects the evaluator, distributes the evaluation amount for each evaluator, transmits evaluation target information to the evaluator terminal 300, and receives the primary evaluation result from the evaluator terminal 300. [

When the evaluation result receiving means 120 selects the evaluator, the evaluator provides the evaluator candidate information to the evaluator terminal 300 via the web, the mobile web, or the mobile application, receives the evaluation application and the evaluator information from the evaluator terminal 300 . The evaluator information may be information about the evaluator individual and may include the evaluator expert field, evaluator evaluable field, evaluation history, and the like.

In addition, the evaluation result receiving means 120 receives evaluator information for various fields in advance from a separate server for evaluator selection, builds an evaluator database, and transmits evaluator information to the evaluator's expert field, evaluator's evaluable field, And the like. The evaluator information included in the evaluator database may be evaluator information accompanying the evaluation request received from the evaluator terminal 300, evaluator information collected separately from another server, and the like.

The evaluation result receiving means 120 extracts, from the evaluator database, an evaluator who takes the field of the tournament held in consideration of the field of the tournament held when there is a specific tournament held, as a field of expertise or an evaluable field, It is possible to select an evaluator by transmitting the evaluation request to the evaluator terminal 300 of the evaluator and receiving the evaluation acceptance from the evaluator terminal 300 of the evaluator who transmitted the evaluation request.

Further, when the evaluation result receiving means 120 distributes the evaluation amount, the participant who applied for participation may be divided into two or more participant groups, and two or more of the selected evaluators may be assigned to each participant group. At this time, the evaluation amount of each evaluator can be distributed so that the number of evaluations that one participant receives is equal.

The evaluation result receiving means 120 transmits the evaluation amount distributed to the specific evaluator terminal 300, receives the evaluation progress status, the evaluation completed amount information, and the primary evaluation result from the specific evaluator terminal 300, 200 to the evaluator information assigned to him / her.

When the evaluator evaluates the entry of the participant, the evaluation method may be a quantitative evaluation and a qualitative evaluation, and the two methods may be mixed. In the quantitative evaluation method, the highest point and the score section can be previously designated by the 5-point scale and the 7-point scale for each evaluation item and presented to the evaluator, and the section of each evaluation item can be presented with a decimal point number , The evaluator can select scores from among the examples presented, as well as directly input between the highest score and the lowest score. For example, a particular evaluator may identify an entry for a particular participant, identify the criteria for evaluation, and select, for example, four of the predetermined scores 1 through 5 as an example of quantitative evaluation, The evaluation can be performed.

Next, the evaluation result correction means 130 analyzes the evaluation results of each evaluator for each participant, analyzes the evaluative tendency of each evaluator, and assigns a weight to the first evaluation result according to the evaluation tendency of the evaluator, thereby correcting the final evaluation result .

Specifically, the evaluation result correction means 130 applies a T-test or a Pearson correlation method to the evaluation results of the first evaluator group and the second evaluator group among the evaluator groups related to the same participant, The result obtained by multiplying the evaluation result by the weight is calculated as the final evaluation result.

The first evaluator group may be a specific first evaluator and the second evaluator group may be an evaluator group other than the first evaluator who has evaluated the same evaluation object as the first evaluator. After calculating the result of calculation by T-test or Pearson correlation method for one evaluator (first evaluator), the same calculation is repeatedly performed for the second evaluator, the third evaluator, etc., When the performance is completed, the final weighting result can be derived.

The evaluation result before weighting may be the result of the primary evaluation for each participant. The weight may be a weight according to the T-test or the Pearson correlation method, or a weight obtained by summing the weights according to the two methods.

In using the T-test method, the T-test calculation result can be determined as a weight by adjusting (scaling) the value from 0 to 1, etc., and if the calculation is mathematically impossible such as a denominator being 0 in the T-test calculation The weight can be set to zero. Likewise, when using the Pearson correlation method, the result of calculating the Pearson correlation can be determined as a weight by adjusting (scaling) the value from 0 to 1. For example, if the denominator is zero in the pearson correlation calculation, Can be set to zero.

The evaluation result correction means 130 can calculate the integrity information of the evaluator to be evaluated. The integrity information may be a value obtained by dividing the completed evaluation number by the assigned evaluation number.

Hereinafter, weighting of the evaluation result correction means 130 and calculation of the final evaluation result will be described in detail.

The definitions of the symbols to be described below are as follows.

Figure pat00028
= n-th participant is not included in the calculation object until the n-th participant has filled the evaluation quota assigned to him / her,
Figure pat00029
.

Figure pat00030
= As the n-th evaluator, if one person is evaluated,
Figure pat00031
.

Figure pat00032
= It can be an evaluator or a participant as an object to obtain a weight.

Figure pat00033
,
Figure pat00034
.
Figure pat00035
= Subject to be evaluated For the evaluation of the entry, for example, the score for each item may be from 0 to 5.

Figure pat00036
,
Figure pat00037
,
Figure pat00038
= T-test (T-statistics, statistical formula), and the subscripts mean (item number, weight 1).

Figure pat00039
,
Figure pat00040
,
Figure pat00041
= Pearson correlation is the first weight obtained using the formula, and the subscripts mean (item number, weight 2).

Figure pat00042
=
Figure pat00043
of
Figure pat00044
This is data for obtaining a weight.

Figure pat00045
=
Figure pat00046
This decision
Figure pat00047
It is a bundle of scores.

In the following, we will explain the weighting method by way of example,

Figure pat00048
,
Figure pat00049
.
Figure pat00050
Are separately configured.

Figure pat00051
This list of participants
Figure pat00052
.

Figure pat00053
To those evaluating participants
Figure pat00054
1 point, 3 point, and 1 point, respectively,
Figure pat00055
Is as follows.

Figure pat00056
=
Figure pat00057

Figure pat00058
A list of people who have evaluated one of the three
Figure pat00059
At this time
Figure pat00060
Are excluded.
Figure pat00061
Can be expressed as follows.

Figure pat00062
=
Figure pat00063

Now, we can calculate about the mean.

mean (

Figure pat00064
) Is calculated as follows.

r count (

Figure pat00065
) = [2, 1, 1]

count (

Figure pat00066
) = 4

c sum (

Figure pat00067
) =
Figure pat00068
(Sum) = [2, 2, 1]

c mean (

Figure pat00069
) =
Figure pat00070
=
Figure pat00071
= [1, 2, 1]

sum (

Figure pat00072
) = sum (c sum (
Figure pat00073
)) = 5

mean (

Figure pat00074
) = sum (c sum (
Figure pat00075
)) / count = (2 + 2 + 1) / 4 = 1.25

mean (

Figure pat00076
) Can be obtained as follows.

count (

Figure pat00077
) = 3

sum (

Figure pat00078
) = 1 + 3 + 1 = 5

mean (

Figure pat00079
) =
Figure pat00080
= 5/3 = 1.66

Now, you can get the variance (var), var (

Figure pat00081
) Can be obtained as follows.

Figure pat00082
=
Figure pat00083

=

Figure pat00084

sum (

Figure pat00085
) = 1 + 1 + 4 + 1 = 7

var (

Figure pat00086
) =
Figure pat00087

= (1/4) (7-5 * 5/4) = 0.1875

var (

Figure pat00088
) Can be obtained as follows.

Figure pat00089
= [1 * 1, 3 * 3, 1 * 1] = [1, 9, 1]

sum (

Figure pat00090
) = 11

var (

Figure pat00091
) =
Figure pat00092

= (1/3) (11-5 * 5/3) = 0.88

Now,

Figure pat00093
, The weight by the T-test method can be obtained as follows.

Figure pat00094

= (1.25 - 1.66) / (0.1865 / 4 + 0.88 / 3) = -0.41 / 0.339 = -1.206

About everyone

Figure pat00095
, And assumes that there are four people who need to obtain the weight.

Figure pat00096

min (

Figure pat00097
) = -1.206

max (

Figure pat00098
) = 5

Figure pat00099
=
Figure pat00100

=

Figure pat00101

=

Figure pat00102

= [0, 0.35, 0.67, 1]

If the denominator is 0 in the above equation,

Figure pat00103
If the denominator is 0, then min (
Figure pat00104
), Which is the same as giving 0 as a weight.

And weight

Figure pat00105
, and the weight by the Pearson correlation method can be obtained as follows.

Figure pat00106

= [2, 2, 1] - 1.25 = [0.75, 0.75, -0.25]

Figure pat00107

Figure pat00108

m diff (

Figure pat00109
) = [-0.66, 1.34, -0.66]

Figure pat00110

Figure pat00111

=

Figure pat00112

=

Figure pat00113

= 0.675 / 1.460 = 0.46

About all people

Figure pat00114
, And assumes that there are four people who need to obtain the weight.

Figure pat00115

Figure pat00116

=

Figure pat00117

=

Figure pat00118

=

Figure pat00119

The final evaluation result can be expressed as follows.

Figure pat00120

Figure pat00121

Figure pat00122

Finally, the evaluation result providing means 140 provides the final evaluation result to the participant terminal 200, and concurrently or separately with the final evaluation result, the first evaluation result, the total number of participants, the prize- Lt; / RTI >

In addition, the participant terminal 200 transmits the participation application, participant information, and entry information to the central server 100.

Specifically, the participant terminal 200 receives competition holding information from the central server 100 via the web, mobile web, or mobile application, provides participation application, participant information, and entry information to the central server 100, As a result of the primary evaluation, the final evaluation result, the total number of participants, the prize-specific prize by each participant, the evaluator information, and the like can be received from the central server 100.

The participant terminal 200 may be a variety of terminals such as a personal computer, a mobile communication terminal, and a tablet PC.

The evaluator terminal 300 receives the evaluation amount from the central server 100 and transmits the primary evaluation result to the central server 100.

Specifically, the evaluator terminal 300 receives the evaluator recruitment information from the central server 100 via the web, mobile web, or mobile application, provides evaluation application and evaluator information to the central server 100, And receives the evaluation amount from the central server 100 to transmit the evaluation progress status, the evaluation completion amount information, and the primary evaluation result.

The evaluator terminal 300 may be a variety of terminals such as a personal computer, a mobile communication terminal, and a tablet PC.

Hereinafter, an online evaluation method considering an evaluator tendency according to a preferred embodiment of the present invention will be described in detail with reference to FIG.

2 is an overall flowchart of an on-line evaluation method considering an evaluator tendency according to a preferred embodiment of the present invention.

First, as shown in FIG. 2, the central server 100 receives participation application, participant information, and entry information from the participant terminal 200 (S10).

The step S10 may include the step of the central server 100 transmitting the tournament holding information to the participant terminal 200 via the web, the mobile web, or the mobile application.

Next, the central server 100 selects an evaluator, distributes the evaluation amount for each evaluator, transmits evaluation target information to the evaluator terminal 300, and receives the primary evaluation result from the evaluator terminal 300 (S20).

The step S20 includes the steps of extracting, from the evaluator database, the evaluator who takes the field of the tournament held in consideration of the field of the tournament held when the specific server is held, Transmitting the evaluation request to the evaluator terminal 300 of the evaluator, and selecting the evaluator by receiving the evaluation acceptance from the evaluator terminal 300 of the evaluator that transmitted the evaluation request.

In step S20, the central server 100 provides evaluator recruitment information to the evaluator terminal 300 through a web, a mobile web, or a mobile application. The evaluator terminal 300 receives evaluator information and evaluator information from the evaluator terminal 300 A step of receiving evaluator information in various fields in advance from a separate server, a step of building an evaluator database, a step of categorizing and storing the evaluator information into an evaluator special field, an evaluator evaluable field, an evaluation record, . ≪ / RTI >

In step S20, the central server 100 divides the participant who has made the participation application into two or more participant groups, and assigns two or more evaluators among the selected evaluators to each participant group so as to equalize the evaluation counts received by one participant And distributing an evaluation amount of each evaluator.

In step S20, the central server 100 transmits an evaluation amount distributed to a specific evaluator terminal 300, and a step of transmitting, from the specific evaluator terminal 300, the evaluation progress status, the evaluation completed amount information, , And transmitting the evaluator information allocated to the participant terminal 200 to the participant terminal 200. [

Next, the central server 100 calculates a final evaluation result by weighting and correcting the primary evaluation result according to the evaluation tendency of the evaluator (S30).

In particular, the weighting in step S30 may be performed by the T-test method or the Pearson correlation method as described above.

Finally, the central server 100 provides the final evaluation result to the participant terminal 200, and transmits the result of the primary evaluation, the total number of participants, the award details per participant, and the like (S40).

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications can be made without departing from the invention. Accordingly, all such modifications and variations are intended to be included within the scope of the present invention.

100: central server
110: participation reception means
120: evaluation result receiving means
130: evaluation result correction means
140: Evaluation result providing means
200: participant terminal
300: Evaluator terminal

Claims (15)

In an evaluation device that takes into consideration evaluator tendencies,
The primary evaluation result information is received from a separate terminal, a device or a server, and the evaluating tendency information of the evaluator included in the primary evaluation result information is analyzed, and the primary evaluation result is weighted according to the analyzed evaluation tendency information, And evaluating the evaluator tendency to calculate the final evaluation result and output the final evaluation result.
The method of claim 1,
Among the weights given to the first evaluation result, the weight according to the T-test method
Figure pat00123
Is calculated in accordance with the following equation.
Figure pat00124
=
Figure pat00125

Figure pat00126

(In the above equation,
Figure pat00127
Is a participant or evaluator for obtaining the weight,
Figure pat00128
The questionnaire for evaluation of the evaluation subject,
Figure pat00129
silver
Figure pat00130
of
Figure pat00131
Data for obtaining a weight,
Figure pat00132
silver
Figure pat00133
This decision
Figure pat00134
It means a bundle of scores.)
The method of claim 1,
Among the weights given to the primary evaluation results, weights according to the Pearson correlation method
Figure pat00135
Is calculated in accordance with the following equation.
Figure pat00136

Figure pat00137

(In the above equation,
Figure pat00138
Is a participant or evaluator for obtaining the weight,
Figure pat00139
The questionnaire for evaluation of the evaluation subject,
Figure pat00140
silver
Figure pat00141
of
Figure pat00142
Data for obtaining a weight,
Figure pat00143
silver
Figure pat00144
This decision
Figure pat00145
It means a bundle of scores.)
The method of claim 1,
And the final evaluation result is expressed by the following equation.
Figure pat00146

(In the above equation,
Figure pat00147
The questionnaire for evaluation of the evaluation subject,
Figure pat00148
Is the weight calculated according to the T-test method,
Figure pat00149
Means the weight calculated according to the Pearson correlation method.)
5. The method according to any one of claims 1 to 4,
The primary evaluation result information includes the evaluation result information of the first evaluator group and the second evaluator group of the evaluator group related to the same participant, the first evaluator group is one of the evaluator groups related to the same participant, Wherein each of the evaluators belonging to the evaluator group is evaluated as a first evaluator group and the calculation according to the T-test or the Pearson correlation method is repeatedly performed.
In an online evaluation system that considers evaluator tendencies,
Participant accepting means 110 for receiving participation application information, participant information and entry information from participant terminal 200, receiving evaluation object information from evaluator terminal 300 and receiving a primary evaluation result from the evaluator terminal 300 And a T-test or a Pearson correlation method is applied to the evaluation results of the first evaluator group and the second evaluator group among the evaluator group related to the same participant An evaluation result correction means 130 for calculating a final evaluation result by applying and correcting the evaluation result to the participant terminal 200, and an evaluation result providing means 140 for providing a final evaluation result to the participant terminal 200;
A participant terminal (200) for transmitting a participation application, participant information and entry information to the central server (100) and receiving a final evaluation result from the central server (100); And
And an evaluator terminal (300) for receiving the evaluation object information from the central server (100) and transmitting the primary evaluation result to the central server (100).
The method of claim 6,
The weighting value according to the T-test method among the weights given by the evaluation result correction means 130
Figure pat00150
Is calculated according to the following equation: < EMI ID = 4.0 >
Figure pat00151
=
Figure pat00152

Figure pat00153

(In the above equation,
Figure pat00154
Is a participant or evaluator for obtaining the weight,
Figure pat00155
The questionnaire for evaluation of the evaluation subject,
Figure pat00156
silver
Figure pat00157
of
Figure pat00158
Data for obtaining a weight,
Figure pat00159
silver
Figure pat00160
This decision
Figure pat00161
It means a bundle of scores.)
The method of claim 6,
The weighting value according to the Pearson correlation method among the weights given by the evaluation result correction means 130
Figure pat00162
Is calculated according to the following equation: < EMI ID = 4.0 >
Figure pat00163

Figure pat00164

(In the above equation,
Figure pat00165
Is a participant or evaluator for obtaining the weight,
Figure pat00166
The questionnaire for evaluation of the evaluation subject,
Figure pat00167
silver
Figure pat00168
of
Figure pat00169
Data for obtaining a weight,
Figure pat00170
silver
Figure pat00171
This decision
Figure pat00172
It means a bundle of scores.)
The method of claim 6,
Wherein the final evaluation result calculated by the evaluation result correction means (130) is expressed by the following equation: < EMI ID = 4.0 >
Figure pat00173

(In the above equation,
Figure pat00174
The questionnaire for evaluation of the evaluation subject,
Figure pat00175
Is the weight calculated according to the T-test method,
Figure pat00176
Means the weight calculated according to the Pearson correlation method.)
10. The method according to any one of claims 6 to 9,
The first evaluator group is one of the evaluator groups related to the same participant and each evaluator belonging to the evaluator group related to the same participant is regarded as the first evaluator group and the calculation according to the T-test or the Pearson correlation method is repeatedly performed On-line evaluation system considering the tendency of evaluator to make.
In the online evaluation method considering the evaluator tendency,
(A) the central server (100) receiving participation application, participant information and entry information from the participant terminal (200);
(B) transmitting the evaluation object information to the evaluator terminal (300) and receiving a primary evaluation result from the evaluator terminal (300);
(C) The central server 100 performs a T-test or a Pearson correlation method on the evaluation results of the first evaluator group and the second evaluator group among the evaluator groups related to the same participant, To calculate a final evaluation result; And
(D) The central server (100) provides the final evaluation result to the participant terminal (200).
12. The method of claim 11,
Among the weights given in the step (C), the weights according to the T-test method
Figure pat00177
Is calculated in accordance with the following equation.
Figure pat00178
=
Figure pat00179

Figure pat00180

(In the above equation,
Figure pat00181
Is a participant or evaluator for obtaining the weight,
Figure pat00182
The questionnaire for evaluation of the evaluation subject,
Figure pat00183
silver
Figure pat00184
of
Figure pat00185
Data for obtaining a weight,
Figure pat00186
silver
Figure pat00187
This decision
Figure pat00188
It means a bundle of scores.)
12. The method of claim 11,
The weight according to the Pearson correlation method among the weights given in the step (C)
Figure pat00189
Is calculated in accordance with the following equation.
Figure pat00190

Figure pat00191

(In the above equation,
Figure pat00192
Is a participant or evaluator for obtaining the weight,
Figure pat00193
The questionnaire for evaluation of the evaluation subject,
Figure pat00194
silver
Figure pat00195
of
Figure pat00196
Data for obtaining a weight,
Figure pat00197
silver
Figure pat00198
This decision
Figure pat00199
It means a bundle of scores.)
12. The method of claim 11,
Wherein the final evaluation result calculated in the step (C) is expressed by the following equation.
Figure pat00200

(In the above equation,
Figure pat00201
The questionnaire for evaluation of the evaluation subject,
Figure pat00202
Is the weight calculated according to the T-test method,
Figure pat00203
Means the weight calculated according to the Pearson correlation method.)
12. The method of claim 11,
Wherein the first evaluator group in the step (C) is one of the evaluator groups related to the same participant, and in the step (C), each evaluator belonging to the evaluator group related to the same participant is set as the first evaluator group, test or pearson correlation method is repeatedly performed on the basis of evaluator tendency.
KR1020150170894A 2015-12-02 2015-12-02 Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor KR20170064881A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150170894A KR20170064881A (en) 2015-12-02 2015-12-02 Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150170894A KR20170064881A (en) 2015-12-02 2015-12-02 Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor

Publications (1)

Publication Number Publication Date
KR20170064881A true KR20170064881A (en) 2017-06-12

Family

ID=59219716

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150170894A KR20170064881A (en) 2015-12-02 2015-12-02 Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor

Country Status (1)

Country Link
KR (1) KR20170064881A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024005329A1 (en) * 2022-06-27 2024-01-04 주식회사 레몬베이스 Method, apparatus, and system for providing artificial intelligence-based member evaluation feedback service

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024005329A1 (en) * 2022-06-27 2024-01-04 주식회사 레몬베이스 Method, apparatus, and system for providing artificial intelligence-based member evaluation feedback service

Similar Documents

Publication Publication Date Title
US8204778B2 (en) Behavioral profiles in sourcing and recruiting as part of a hiring process
US9378486B2 (en) Automatic interview question recommendation and analysis
US20170337287A1 (en) Intelligent integrating system for crowdsourcing and collaborative intelligence in human- and device- adaptive query-response networks
CN109976997B (en) Test method and device
Wang et al. Cost-effective quality assurance in crowd labeling
US20080144882A1 (en) System and method for determining like-mindedness
KR20170042286A (en) Systems and methods for data-driven identification of talent
AU2001268318A2 (en) Computer-implemented system for human resources management
EP1299820A1 (en) Computer-implemented system for human resources management
Garapin 1 et al. Does trust mean giving and not risking? Experimental evidence from the trust game
JP7127848B2 (en) Recommendation system and recommendation method
KR20170064881A (en) Assessment Apparatus Considering Assessor Tendency, Online Assessment System Considering Assessor Tendency and Method therefor
KR20170073330A (en) Online Assessment System Considering Assessor Tendency and Method therefor
CN109472455B (en) Activity evaluation method, activity evaluation device, electronic equipment and storage medium
CN114944219A (en) Mental scale recommendation method and device based on artificial intelligence and storage medium
JP6588176B2 (en) Computer system for analyzing evaluation results and method and program executed in the computer system
Kim et al. Investigating the efficacy of crowdsourcing on evaluating visual decision supporting system
CN113780610A (en) Customer service portrait construction method and device
US20190213520A1 (en) Method and system for marker community to access aggregated marking technologies
Ilieva et al. Virtual Gaming Platform Customer Experience Evaluation
CN113988291B (en) Training method and device for user characterization network
US20210383329A1 (en) Method of distributing functional element unit tasks of crowdsourcing-based project, apparatus therefor, and computer program therefor
Breaban et al. Prudence, personality, cognitive ability and emotional state
Ahmadabadi et al. Design and Evaluation of Crowd-sourcing Platforms Based on Users Confidence Judgments
Kröger et al. Comparison of different question formats eliciting point predictions

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E601 Decision to refuse application