KR20090096040A - On-line statistical process for comparative evaluation between multi parties and system for the same - Google Patents

Abstract

An on-line statistical processing method for comparative evaluation between multi parties and a system for the same are provided to perform self diagnosis and object evaluation about contents having logical, subject and creative features without a professional evaluator. A database(310) comprises a code database and a set database. The code database stores the first codes in which each sub data is converted into lower codes. The set database stores the second codes set by upper codes. An analysis engine(320) statistically processes the first codes or the second codes. An analyzed result creation engine(330) converts the statistically processed result into visual information.

Description

ON-LINE STATISTICAL PROCESS FOR COMPARATIVE EVALUATION BETWEEN MULTI PARTIES AND SYSTEM FOR THE SAME}

The present invention relates to an on-line statistical processing method for multilateral comparison evaluation. More specifically, the multi-dimensional comparative evaluation result is statistically processed by analyzing the content to be evaluated as sub data and encoding the coded information. We propose a method and system for the same.

As the world becomes politically, economically and culturally unified, diversity and individual values are respected, and comprehensive and logical thinking skills are required rather than fragmentary thinking.

In line with these trends, efforts are being made to develop and enhance the abilities of logic, thinking, and creativity in the field of education, and the development of educational methods and evaluation methods is required.

However, in the case of content with characteristics such as logic, subjectivity, originality, for example, thesis, essays, opinions, statements, etc., a professional evaluator must be present for accurate evaluation. There are limitations, and there are fundamental limitations as an evaluation method due to constraints of time and space.

On the other hand, recent technological developments have made it possible for online learning to guarantee free and effective learning across space and time. Advancement of IT technology has enabled a large number of users to participate in the learning process at the same time and effectively improve the ability by two-way communication in online learning, but online learning has various advantages such as low learning cost, overcoming the limitation of learning place, and autonomy of learning time. Nevertheless, there is a limitation that objective evaluation of learning results, in particular, quantitative and quantitative evaluation of contents requiring logic and creativity, such as reading ability or writing ability, is difficult.

Referring to FIG. 1, which illustrates an example of an online learning method, the educational content provider 110 may provide a language learning service based on a network 130 to an online user (learner) 120.

The provider may deliver various educational contents 112 to the user, and the user who has undergone the learning process may feed back the homework assignment 114 on the learning result to the provider. The provider receiving the assignment may improve the learning by evaluating the user's level of learning and passing the assessment and scoring 116 results to the user.

However, due to the nature of language learning content, the evaluation of a task presented by a user has a disadvantage in that it can be performed in a very subjective manner depending on the evaluator. For example, referring to FIG. 2, when there are a plurality of evaluators 116a, 116b, and 116c for one task 114, different evaluation results 117a, 117b, and 117c are derived. If the evaluation is made by only one evaluator, the assignments delivered by multiple users will not be able to be quickly evaluated, which may not take advantage of the online education.

As such, traditional online learning, especially language learning, is difficult to objectively evaluate, and thus, the credibility of educational assessment is not only causing user dissatisfaction, but also lowering the level of learning level, which has significant problems in terms of educational performance. something to do.

In addition, in various fields such as public opinion evaluation and analysis, foreign language learning, psychotherapy, online counseling, etc., when multi-dimensional three-dimensional evaluation is required, there is a problem in that it is difficult to secure objectivity and thus cannot take advantage of online technology.

Therefore, more fundamental alternatives are needed to enable high-quality online education or evaluation for the purpose of securing logic, acquiring foreign languages, and improving the comprehension of literary works.

SUMMARY OF THE INVENTION The present invention has been made under the technical background described above, and has an object of enabling self-diagnosis and objective evaluation of content having characteristics such as logic, subjectivity, originality, etc. without a professional evaluator.

It is another object of the present invention to enable mutual comparison evaluation within a non-fixed population.

Still another object of the present invention is to provide a method for confirming objective and quantitative evaluation results in three dimensions in real time.

Various objects and features of the present invention will be more specifically presented through preferred embodiments.

In order to achieve the above object, according to an aspect of the present invention, a code data generation step of analyzing the target content for evaluation into a plurality of sub-data to change each sub-data into sub-codes, and specifying the code data A step of generating set data which is combined into groups according to a condition and set into a plurality of higher codes; collecting online selection information of subjects to be evaluated for the code data or set data; and collecting the collected selection information statistically. It provides a method for online statistical processing for multi-party comparison evaluation comprising the step of processing to, and providing the results of the statistically processed selection information on-line as visual information.

According to another aspect of the invention, the step of collecting the content for evaluation from a plurality of subjects through a wired or wireless network, and analyzing the collected content as a plurality of sub-data according to a specific condition, each sub-data A code data generation step of changing a code to lower codes, a set data generation step of combining the code data into groups according to a specific condition and setting the plurality of higher codes; Statistically processing the set data, and providing the results of the statistically processed selection information on-line as a visual information online provides a statistical processing method for the multilateral comparison evaluation.

The generating of the set data may include generating a plurality of first higher codes that combine and set the code data into groups according to a specific condition, and by combining the first higher codes by group according to a specific condition. And setting to two higher codes.

The visual information may map selection information for code data or set data to, for example, a normal distribution, wherein the visual information includes the selection of a separate expert group together with the results of statistical processing of the selection information collected from the subjects. May contain information. In addition, the visual information may include a statistical processing result at a first time point and a change in the statistical processing result at a second time point different from the first time point.

The statistical processing may classify subjects under specific conditions, and statistically process only the selection information within a specific group of classified subjects.

The step of collecting the selection information from the subjects is to transmit the code data or set data to the subjects through a wired or wireless network, and the subjects are provided as a response to the code data or set data. Collecting the information through a wired or wireless network.

The present invention also provides a server for performing wired and wireless data communication and data control, a first database for storing content to be evaluated, and analyzing the content into a plurality of sub data according to a specific condition to subcode each sub data. A second database for storing the first codes changed into a plurality of second codes; a third database for storing the second codes in which the codes are combined into groups according to a specific condition and set into a plurality of higher codes; and the first code It provides a statistical processing system for online multi-party comparison evaluation including a statistical processing engine for statistically processing the second or second codes, and a visual information conversion engine for converting the statistically processed results into visual information. In this case, the server may collect the content to be evaluated from the online evaluators through a wired or wireless network.

The present invention also provides a server for performing wired / wireless data communication and data control, a first database for storing content to be evaluated, and analyzing the content into a plurality of sub data according to a specific condition, thereby sub-codes each sub data. A second database for storing the first codes changed into a second database; a third database for storing the second codes in which the codes are combined into groups according to a specific condition and set into a plurality of higher codes; and the first codes Or a statistical processing engine for statistically processing the response information of the evaluators online on the second codes, and a visual information conversion engine for converting the statistically processed result into visual information. Provide a processing system.

According to the present invention, objective evaluation is possible without a professional evaluator about content having characteristics such as logic, subjectivity, originality, and the like. In particular, an absolute evaluation through comparison with a comparative evaluation and a comparison group may be simultaneously performed by mutual comparison evaluation in a non-fixed subject group.

By statistically analyzing the comparative evaluation results and presenting them visually, the evaluation results can be confirmed three-dimensionally, the trend of evaluation results can be changed over time, and the evaluation tool can be updated in real time.

Therefore, the present invention can be effectively applied to a wide range of fields such as language, essay, foreign language, society, economy, law, various learning fields, analysis fields such as public opinion polls, and medical fields such as psychotherapy.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

The present invention provides real-time multi-party comparison by providing the subject to be evaluated through the wired or wireless network or by analyzing the content to be evaluated collected from the subject into sub-data in the service providing server and setting the coded information in groups. It is key to conducting evaluations and visually presenting the results of statistical processing of multilateral comparisons.

This multilateral comparison allows objective evaluation without the effort and effort of professional evaluators, and the evaluation tool can obtain self-advancing effect (self-update effect) according to the change (size, level, etc.) of the evaluated group. Real-time changes of criteria are possible. The larger the size of the assessed group, the more accurately the evaluator's assessment results (e.g., learning level, knowledge level, psychological state, etc.) can be identified and compared to the responses or attitudes of peer groups, expert groups, or the entire group. Its location can be determined objectively and in real time.

In particular, in essay and thesis analysis, it is possible to overcome the limitations of multi-answer type answers such as four-choice answers, so it is very effective to objectively grasp the level of logic, creativity, and subjective thinking ability. It may be possible to solve the problem of a uniform evaluation method.

Figure 3 shows the steps of encoding and setting content in a method according to the invention.

In the encoding step S1, target data for evaluation are analyzed into a plurality of sub data to generate code data for changing each sub data into sub codes. In the set step S2, the code data is combined for each group according to a specific condition to generate set data obtained by setting the plurality of higher codes. In the case of complex content, one or more set steps S3 may be further added as necessary.

According to the field to which the present invention is applied, a step of collecting selection information of subjects on the code data or set data is subsequently added. Alternatively, content for evaluation may be collected in advance from a plurality of subjects through a wired or wireless network.

After statistically processing the selected information collected from the subjects (or a plurality of set data related to the plurality of subjects), the statistical processing results are visually provided to the subjects online via a wired or wireless network. do.

contents  Analysis and sub data

In the present invention, a preceding step for multilateral comparison evaluation and statistical processing is to analyze the content to be evaluated. In the case of content that consists of a plurality of sentences, such as a thesis, essays, opinions on specific issues, and statements about psychological state, and has logical relevance, it is necessary to analyze the content as quantified data for objective comparison and evaluation.

FIG. 4 schematically shows an example of analyzing a language content (language learning content) 200 from a multifaceted perspective in order to evaluate it in a logical and objective manner. It contains one or more viewpoints, such as literary works such as poetry, novels, and essays, articles, and information materials in various fields. 230, information 240 about values.

Information related to these various points of view may be a useful base for objectively analyzing language content. In addition, this information can be systematically analyzed and encoded as an objective unit of evaluation.

FIG. 5 schematically illustrates that the language learning content 200 is converted (or derived) into analysis information 202, 204, 206, and 208 classified into a plurality of issues based on information of various viewpoints. The number of the analysis information may vary depending on the content and format of the language learning content, and the four issues shown are merely exemplary. The analysis information may be further divided into sub-analysis information 202a, 204a, 206a, and 208a including each detailed issue.

If the content being evaluated is a statement of opinion on a particular event, it may be possible to code each piece of data, for example, by analyzing the view of the cause of the event, the view of the course of the event, and the view of the outcome of the event. .

In addition, for medical contents related to psychotherapy, for example, each data may be encoded by analyzing emotions of past experiences, feelings of the present state, thoughts or feelings about the future, and the like.

As such, the process of obtaining the sub data through the analysis of the content may vary according to the type and characteristics of the target content. Analysis information derived based on various viewpoints from the content is converted into digital information (encoded) and used as basic data for performing a multilateral comparison evaluation according to the present invention.

Meanwhile, in the present invention, the analysis information derivation method described with reference to FIGS. 4 and 5 may also be used to analyze the content to be evaluated collected from the person to be evaluated.

Coding Analysis Data

After obtaining the sub data by analyzing the contents of the content from a multi-dimensional point of view (or at the same time as the analysis process), the sub data is encoded.

6 codes, for example, one-dimensional sub data analyzed based on a plurality of viewpoints. The sub data derived from different viewpoints A have a linear relationship with each other and may be encoded as a, b, c, and d, respectively. The codes a, b, c, and d may correspond to, for example, sub data obtained after analyzing content in terms of people, backgrounds, events, and values.

Content analysis can be performed not only in one-dimensional form but also two-dimensional or three-dimensional. In this case, a more complicated form of encoding process is required according to the multi-faceted analysis.

For example, referring to FIG. 7, first viewpoints A related to a person, a background, an event, and a value, and second viewpoints related to a single structure, a complex structure, and a deep structure that are analyzed in a formal aspect of content content composition ( It can be seen that B) is arranged in a matrix. Mixed perspectives formed at the point where each perspective meets through the matrix arrangement of these perspectives are encoded in a more complicated form (a1, a2, a3, b1, b2, b3, c1, c2, c3, d1, d2, d3). do. This matrix-type coding enables accurate comparison and evaluation of articles with complicated contents and logic, including articles and essays.

Meanwhile, referring to FIG. 8, among the views A and B arranged in a matrix structure, the views B related to the single, complex, and deep structures are divided into other subdivided third views. Can be. Although arranged two-dimensionally for the sake of schematic drawing, the arrangement of these views may be three-dimensionally illustrated. In other words, three basic perspectives are linked to each other to further code data of various sub-units (a1, a2, a3, a4, a5, a6, a7, a8, a9, b1, b2, b3, b4, b5, b6, b7, b8, b9, c1, c2, c3, c4, c5, c6, c7, c8, c9, d1, d2, d3, d4, d5, d6, d7, d8, d9)

Combination and Set of Codes

According to the present invention, in order to subjectively evaluate subjective and logical thinking among the subjects in the subject group, subject the codes to specific conditions after coding the content analysis data. According to the combination.

Codes thus set are set as the upper code by combining a series of codes from the previous lower codes. Higher-level codes are the basis on which subjects can provide statistical distribution through comparisons, rather than comparisons with correct answers, and allow assessment tools to evolve over time and over time. It is a necessary basis.

As described above, according to the set of codes, the present invention is capable of comparing and evaluating logical power, understanding ability, and analysis power for various types of contents differently from the evaluation method requiring a correct answer among four choices.

For example, as shown in Table 1 below, the content may be analyzed to extract various sub data and code the sub data.

TABLE 1

Perspective 1 Character Incident background Values Viewpoint 2 simple complex conflict Deepening simple complex conflict Deepening simple complex conflict Deepening simple complex conflict code A1 A2 A3 A4 B1 B2 B3 B4 C1 C2 C3 C4 D1 D2 D3

By extracting the codes one by one from each item of viewpoint 1 and setting a code group in which four codes are combined, a higher level code that can represent one logical flow related to viewpoint 1 may be generated.

The generating of the set data from the coded sub data may include, for example, a plurality of first higher codes AA and AB which combine and set the code data into groups according to a specific condition as illustrated in FIG. 9. Generating a set of a plurality of second higher codes (α, β) by combining the first higher codes in groups according to a specific condition. It may include.

Of course, the set step may be further subdivided into a plurality of steps according to the type or characteristic of the content to be evaluated, and the number of higher codes set may increase.

On the other hand, in the present invention, by providing an example distribution of the evaluation results of the peer group, the comparison target group, or the expert group to which the evaluation target belongs, it is possible to objectively evaluate its own logical power and the like.

For example, a question about the nature or inclination of the main character (A) of the novel could be presented with an optional answer from the following point of view.

1) simple, 2) complex, 3) conflict, 4) deepening

For example, the results of the selection of various groups of subjects (learners) for the proposed answers may be statistically processed as shown in Table 2 below.

TABLE 2

Elementary group Secondary group High school group Adult group Expert group simple 10% 30% 35% 40% 10% complex 25% 40% 5% 30% 80% conflict 5% 15% 30% 10% 8% Deepening 60% 15% 30% 10% 2%

When the answer chosen by the learner is presented as statistical data, the greater the number of learners, the more secure the objectivity of the evaluation criteria is. In addition, by presenting the frequency of selecting a specific answer rather than presenting the correct answer to the question, it is possible to objectively grasp the various tendencies of answer selection. In other words, rather than presenting the correct answer when evaluating a person A, it is possible to embrace the main types of answers as well as to suggest how the frequency of items for peer A and expert groups to evaluate the person A varies. You can also embrace. Evaluation in this manner may be an indirect means of establishing social manners such as consideration, tolerance and understanding that are likely to be neglected in a competitive society.

10 illustrates an example of a procedure for multilateral comparison evaluation and statistical processing according to an embodiment of the present invention.

For example, in the case of an evaluation system related to online learning, a plurality of sub data (and sub codes or set codes) related to content for evaluation are transmitted online from a service providing server to a user (learner) (step S1). The user receiving the sub data transmits a response to the sub data, or response information (selection information) for a lower code related to the sub data or an upper code in which the codes are set, and receives the response information from the server. (Step S2). The received information is analyzed and processed through a separate encoding process (without encoding process if the response information includes the code information) (step S3). The received information can be classified according to user level, age, region, and the like, and statistically processed to finally generate an analysis result with visual information (step S4).

FIG. 11 shows an example of such visual information, and shows a result of statistically analyzing the frequency of selection of a higher code set for each evaluation group.

The evaluation result (I) of the target group, the evaluation result of the comparison group (II), and the evaluation result of the expert group (III) are presented together, so that an objective evaluation is possible. In addition, the evaluation results can be visually provided by various groups such as grades, rankings, regions, and ages.

It can be easily and intuitively grasped by statistically processing the number of opinions or thoughts of the evaluation group for specific contents, and the level or location (M) of the learner (or learner group) is also effective for self-diagnosis.

Unlike the traditional method of evaluating piecemeal answers by presenting only correct answers, it is possible to objectively grasp the level and location of individual learners (or group of learners) by comparing and evaluating a plurality of subjects within the evaluation group. It will be possible to indirectly acquire the virtues of consideration, tolerance, and understanding that are easily neglected in society.

12 shows an example in which selection information on code data or set data is plotted as a normal distribution (Gaussian distribution). The statistical processing results visualized by the normal distribution are easy to determine the level or position of learners in the evaluation group. In this case, objective comparison can be achieved by presenting the evaluation result (I) of the evaluation target group, the evaluation result of the comparison group (II) and the evaluation result of the expert group (III).

In particular, the visualization with a normal distribution is very effective in paralleling supplementary services for improving learning by accurately assessing the level or location in peer group or comparison group in online learning or psychotherapy.

FIG. 13 illustrates a change in an evaluation result over time, and visually confirms a change in a learner's level or position M2 from the learner's level or position M1 at a first time point. Can be.

As described above, the multilateral comparison evaluation and statistical processing method according to the present invention can effectively improve the learning effect in parallel with the learner (or indexing by level of the learner group, the course cooking selection according to the learner level, etc.) in the online learning system. There is a particular advantage in learning content that requires logic or thinking.

14 illustrates a system configuration for a multilateral comparison evaluation and statistical processing method according to the present invention.

The service providing server 300 performing wired / wireless data communication and data control includes a database 310, an analysis engine 320, and an analysis result generation engine 330 through a device provided therein or separately.

The database 310 stores a first database for storing the content to be evaluated and a second code for analyzing the content into a plurality of sub data according to a specific condition and storing first codes for changing each sub data into sub codes. A database (code database) and a third database (set database) for storing the second codes in which the codes are combined into groups according to a specific condition and set into a plurality of higher codes, and in some cases, It may further include additional sub-databases.

The analysis engine 320 corresponds to a statistical processing engine for statistically processing the first codes or the second codes, and the analysis result generation engine 330 converts the statistically processed result into visual information. Corresponds to the information conversion engine.

The server may collect the content to be evaluated from the online evaluators 410 and 420 through the wired or wireless network 305 or transmit the content to the evaluators.

In the present invention, the engine refers to software or hardware that performs essential functions necessary for multilateral comparison evaluation and statistical processing, or a storage medium storing the software. In addition, in the present invention, the analysis engine 320 and the analysis result generation engine 330 may exist as separate software or hardware from each other, but may be implemented as a systemized engine by integrating two or more into one.

On the other hand, in the present invention, the communication to receive the data provided by the user (learner) via a wired or wireless network to the server, or to transmit the data to the user from various databases or various engines controlled by the server or server The unit may be provided separately.

According to the present invention, objective evaluation can be performed in real time by converting content to be quantitatively quantified for evaluating multilateral comparison. Thus, according to the present invention, for example, by overcoming the low reliability of online learning, it is possible to improve the structure of the online learning industry and to create a new educational business model in various ways. In addition, it can be effectively applied to various areas that need objective comparative evaluation in response to social needs.

The present invention has been exemplarily described through the preferred embodiments, but the present invention is not limited to such specific embodiments, and various forms within the scope of the technical idea presented in the present invention, specifically, the claims. May be modified, changed, or improved.

1 is a schematic diagram showing an example of an online learning method.

2 is a schematic diagram illustrating a learning evaluation method of an online learning method.

Figure 3 is a flow chart showing the coding and set-up step for multilateral comparison evaluation according to the present invention.

4 is a schematic diagram illustrating an example of analyzing content into a plurality of sub data.

5 is a schematic diagram illustrating another example of analyzing content into a plurality of sub data.

6 is a schematic diagram illustrating an example in which content is encoded based on a plurality of viewpoints.

Fig. 7 is a schematic diagram showing an example in which content is encoded based on a plurality of viewpoints in a matrix method.

8 is a schematic diagram showing another example in which contents are encoded based on a plurality of viewpoints in a matrix manner.

9 is a schematic diagram showing an example in which codes are combined and set.

10 is a flow chart showing a processing procedure of an embodiment according to the present invention.

11 is a graph showing an example of the results of the multilateral comparison evaluation according to the present invention.

12 is another graph showing the results of the multilateral comparison evaluation according to the present invention.

Figure 13 is another graph showing the result of multilateral comparison evaluation according to the present invention.

14 is a schematic diagram showing a system configuration for multilateral comparison evaluation and statistical processing according to the present invention.

*** Explanation of symbols for the main parts of the drawing ***

300: server 305: network

310: Database 320: Analysis Engine

330: analysis result generation engine 410, 420: user (learner)

Claims (15)

A code data generation step of analyzing the target content for evaluation into a plurality of sub data and changing each sub data into sub codes; A set data generation step of combining the code data into groups according to a specific condition and setting the plurality of higher codes; Collecting on-line information of the subject's selection of the code data or set data; Statistically processing the collected selection information, Providing visual information online for results of statistically processed selection information; On-line statistical processing for multilateral comparison evaluation. The method of claim 1, wherein the set data generation step Generating a plurality of first higher codes obtained by combining the code data into groups according to a specific condition; And combining the first higher codes into groups according to a specific condition and setting the plurality of second higher codes into a plurality of second higher codes. 2. The method of claim 1, wherein the visual information is a normal distribution of selection information for code data or set data. The method of claim 1, wherein the visual information includes selection information of a separate expert group together with the result of statistical processing of the selection information collected from the evaluators. The method of claim 1, wherein the statistical processing classifies the subjects under specific conditions, and statistically processes only selection information within a specific group of the classified subjects. Treatment method. The method of claim 1, wherein the visual information includes a statistical processing result at a first time point and a change in the statistical processing result at a second time point different from the first time point. Method of processing statistics. The method of claim 1, wherein collecting the selection information Transmit the code data or set data to the subjects to be evaluated via wired or wireless network, Gathering, via a wired or wireless network, the information provided by the subjects as a response to the code data or the set data via a wired or wireless network. Collecting content for evaluation from a plurality of subjects through a wired or wireless network; Analyzing the collected contents into a plurality of sub data according to a specific condition to generate code data for changing each sub data into sub codes; Generating the set data in which the code data are combined into groups according to a specific condition and set into a plurality of higher codes; Statistically processing the plurality of set data associated with the plurality of subjects, Providing visual information online for results of statistically processed selection information; On-line statistical processing for multilateral comparison evaluation. The method of claim 8, wherein the set data generating step Generating a plurality of first higher codes obtained by combining the code data into groups according to a specific condition; And combining the first higher codes into groups according to a specific condition and setting the plurality of second higher codes into a plurality of second higher codes. 9. The method of claim 8, wherein the visual information is a normal distribution of selection information about code data or set data. The method of claim 8, wherein the visual information includes statistical processing results of data encoding and setting content of a separate expert group together with data encoding and setting content collected from an evaluator. On-line statistical processing for multilateral comparison evaluation. 10. The method of claim 8, wherein the visual information includes a statistical processing result at a first time point and a change in the statistical processing result at a second time point different from the first time point. Method of processing statistics. A server performing wired and wireless data communication and data control, A first database for storing the content to be evaluated, A second database for analyzing the contents into a plurality of sub data according to a specific condition and storing first codes in which each sub data is changed into lower codes; A third database storing second codes in which the codes are combined into groups according to a specific condition and set into a plurality of higher codes; A statistical processing engine for statistically processing the first codes or second codes; A visual information conversion engine that converts statistically processed results into visual information. Statistical processing system for online multilateral comparison evaluation. The statistical processing system for online multi-party comparison evaluation of claim 13, wherein the server collects the content to be evaluated from online evaluators through a wired or wireless network. A server performing wired and wireless data communication and data control, A first database for storing the content to be evaluated, A second database for analyzing the contents into a plurality of sub data according to a specific condition and storing first codes in which each sub data is changed into lower codes; A third database storing second codes in which the codes are combined into groups according to a specific condition and set into a plurality of higher codes; A statistical processing engine for statistically processing the response information of the evaluators online on the first codes or the second codes; A visual information conversion engine that converts statistically processed results into visual information. Statistical processing system for online multilateral comparison evaluation.

Images