KR102084861B1 - System and method for analyzing and utilizing how authors refer to an event in a web document to predict the distribution of how much readers consider such document credible - Google Patents

Abstract

Disclosed are a method for analyzing and utilizing the manner in which an author refers to an event in a web document to predict credibility distribution of readers for a given document, and a system thereof. According to an embodiment of the present invention, the method for predicting a reader credibility distribution comprises the steps of: calculating event credibility for each event information extracted from the web documents by considering authors of the web documents as virtual readers; receiving a document for performing credibility distribution prediction from a user; and predicting the credibility distribution of the received document of each of reader groups distinguished according to a plurality of preset demographic characteristics based on the event credibility calculated for each event information.

Description

System and method for analyzing and utilizing how authors refer to an event in a web document to predict the distribution of how much readers consider such document credible}

The present invention relates to a technique for predicting a reader's reliability distribution for a document. More specifically, the author's reliability of an event is calculated by analyzing the expression used by the web document authors by referring to the event in the web document. The present invention relates to a method and system for automatically predicting the distribution of the reliability of a web reader by considering a web document author as a virtual reader.

Since the advent of the Internet, the amount of text content that can be encountered in everyday life has been increasing rapidly, and various online text providing platforms such as internet news, social media, blogs, and internet libraries have continued to grow in size.

It is impossible for each user to read all the textual content provided by a huge online platform. Due to these limitations, when users encounter text content through the platform, they decide whether to read the text content at all, read it quickly, or take time to read it. However, since such decisions involve significant cognitive burden, there is a need for decision support to mitigate them.

In connection with the decision making, methods have been proposed that preferentially provide texts that are expected to be satisfied by the user after reading them. Among them, a method of using the reliability of the text content and the author's confidence index has been proposed and is currently used in many text content providing platforms.

Korean Patent Publication No. 10-1284788, which is a technique of a conventional embodiment, relates to a "question answering device and method based on reliability", with the aim of evaluating reliability of a document from various angles such as quality, source, and correct answer extraction strategy. The source reliability of each correct candidate document is measured by using the relationship between the source of the correct candidate document and the user query and the source reliability of the corresponding correct candidate document. This technique measures the reliability of extraction strategy for the correct candidate documents using the goodness of fit between the extraction strategies. The prior art also utilizes six quality assessment qualities: currency, availability, information-to-noise ratio, authority, popularity, and cohesiveness.

Korean Patent Publication No. 10-1859620, which is a technique of another conventional embodiment, relates to a method and system for recommending content based on reliability in an online social network, and collects other user actions in addition to user actions in an online social network. To provide a reliability-based content recommendation method that calculates a complex, collects behavioral data of users who are active in online social networks, calculates user reliability using collected user behavioral data, and Collects content data, and calculates content reliability using the user behavior data and the collected content data.

However, the related arts may cause the following problems when assisting the process of determining whether the user reads the text content.

First, since reliability is calculated by providing a single number, it is difficult to satisfy two users at the same time when there are two users who trust one text content to different degrees. That is, after reading the given text content, the user A may have high reliability, and the user B may have low reliability. In this situation, if the prediction reliability is set high and provided to A and B collectively, the satisfaction of B after reading may be lowered. On the contrary, if the reliability is set low, the satisfaction of A may be lowered. If the reliability is set to the middle value, the satisfaction of both A and B may be lowered.

Second, since trust in text content can be determined according to the subjectivity of the individual, even if the document reliability is calculated based on the personal information of each user, it is unlikely that it will be different from the trust that will be seen after the specific user actually reads the document. This is high. In particular, when using the reliability calculated by one number, a situation in which the reliability provided before the user reads the document and the reliability shown after the actual reading is repeated may be repeated, thereby lowering the user's satisfaction.

Third, the decision about whether or not to read text content is related to a variety of personal factors, such as the user's interests, available time, and emotional state at the time of reading. You may feel that you have not been provided with helpful decision-making information.

Embodiments of the present invention calculate the author's confidence in an event by analyzing the expression used by the web document authors to refer to the event in the web document, and regard the web document author as a virtual reader for the document input by the user. Therefore, the present invention provides a method and system for automatically predicting the reliability distribution that readers in the general public will see.

According to an embodiment of the present invention, a method for predicting reader confidence distribution includes calculating event reliability for each event information extracted from the web document by considering authors of the web document as virtual readers; Receiving a document for performing a reliability distribution prediction from a user; And predicting a confidence distribution for the input document of each of the reader groups distinguished according to a plurality of preset demographic characteristics based on the event reliability for each of the calculated event information.

The calculating of the reliability of the event may include collecting the web document and extracting each of the event information from the collected web document; Calculating event reliability for each of the extracted event information; And an event reliability distribution which is a statistical distribution of the number of readers according to the value of the event reliability based on the value of the collected event reliability, the sum of values of event reliability calculated for the same event information among the extracted event information. And generating, and the predicting may predict the reliability distribution of the user's event information extracted from the input document using the event reliability distribution of each of the user's event information.

The demographic characteristics may include a first item about the gender of the reader, a second item about the age of the reader, a third item about the job of the reader, and a subject of interest to the reader, which are subjects of determining the reliability of the document. And a fourth item for and a fifth item for whether the reader is an expert on the subject of the document.

The calculating of the event reliability may include calculating the event reliability based on the number of positive expressions, negative expressions, confidence expressions, distrust expressions, consent expressions, and non-motion expressions used by the author in referring to the event in the web document. Can be calculated.

Furthermore, the reader confidence distribution prediction method according to an embodiment of the present invention may further include providing the user with a reliability distribution of each of the predicted reader groups.

According to an embodiment of the present invention, a reader confidence distribution prediction system includes a virtual reader data collector configured to calculate event reliability for each event information extracted from the web document by considering authors of the web document as virtual readers; And a document for performing a reliability distribution prediction from a user, and inputting the document to each of the reader groups distinguished according to a plurality of demographic characteristics preset based on the event reliability for each of the calculated event information. And a user input document processing unit for predicting a reliability distribution for the.

The virtual reader data collection unit collects the web document, extracts each of the event information from the collected web document, calculates an event reliability of each of the extracted event information, and the same event information among the extracted event information. Summing the values of the event confidence calculated for and generating an event confidence distribution that is a statistical distribution of the number of readers according to the value of the event confidence based on the value of the collected event confidence, wherein the user input document processing unit The reliability distribution of the event information of the user extracted from the input document may be predicted by using the event reliability distribution of each of the user's event information.

The demographic characteristics may include a first item about the gender of the reader, a second item about the age of the reader, a third item about the job of the reader, and a subject of interest to the reader, which are subjects of determining the reliability of the document. And a fourth item for and a fifth item for whether the reader is an expert on the subject of the document.

The virtual reader data collector may calculate the event reliability based on the number of positive expressions, negative expressions, confidence expressions, distrust expressions, consent expressions, and non-motions expressed by the author in referring to the event in the web document. Can be.

The user input document processor may provide the user with a reliability distribution of each of the predicted reader groups.

According to embodiments of the present invention, web document authors analyze an expression used by referring to an event in the web document to calculate the author's confidence in the event, and a virtual reader of the web document author for the user input document. Can be used to automatically predict the distribution of confidence among readers in the general public.

According to embodiments of the present invention, large-scale data that enables improvement of reliability distribution prediction accuracy can be collected and utilized in a low-cost manner without extending the cost-critical prediction reference corpus.

The technique according to the embodiment of the present invention may have the following effects.

(1) In the present invention, when two users trust one text content to a different degree, after reading the corresponding text content, the user A shows high reliability, and the user B shows low reliability, By predicting the overall distribution of confidence that readers will see and providing them to A and B, how are people with confidence that correspond to the confidence of each of A and B being distributed, and how are those with different confidence from each other being displayed? By showing that there is a high satisfaction level for both users.

(2) The present invention provides information on the reliability distribution seen by the reader in general for a given text content. The confidence displayed by the user after reading the given text content corresponds to the partial information contained in the distribution of confidence provided to the user. Therefore, it is determined whether the information matches the common sense line of the user unless the user collects the confidence of other readers through a direct questionnaire. Although it can be estimated roughly, it is difficult to determine whether a given confidence distribution is an accurate result. Therefore, the reliability distribution result provided to the user may be judged to be a wrong result, thereby preventing the user satisfaction from being lowered.

(3) The present invention predicts the distribution of reliability that each group of readers classified according to a plurality of demographic characteristics such as gender, age, expert status, and so on for a document input from the user and provides the result to the user. To determine whether to read the text content, the user may refer to the reliability distribution provided in the process of self-reviewing related factors such as available time, interest, and emotional state. The present invention provides detailed information on what kind of reliability others have for a given text content, inducing empathy from the user and making the user feel that they have been provided with sufficient decision support information, thereby providing overall user satisfaction. Can increase.

The present invention is a system that assists a person in making a decision based on the reliability information of a plurality of readers, and can be applied and applied to various platforms that need to provide reliable content, and the reader's reliability distribution for news articles. It can be used for public opinion analysis through affirmative sentences, reinforcement of reliable sentences for existing sentences, self-evaluation and feedback on reliability of spoken sentences during conversation.

In addition, the present invention can be applied to an online news platform, SNS, an Internet document providing platform, etc. requiring reliable content provision.

1 illustrates a configuration of a system for predicting reliability distribution according to an embodiment of the present invention.
FIG. 2 illustrates a configuration of an embodiment of the virtual reader data collector of FIG. 1.
3 is a block diagram of an embodiment of the user input document processor of FIG. 1.
Figure 4 shows an exemplary view for explaining the output result of the independent reliability distribution prediction system of the present invention.
5 illustrates an example of a prediction reference corpus included in an original reliability distribution prediction system.
6 illustrates an example of an event confidence database included in an original confidence distribution prediction system.
FIG. 7 illustrates an operation flowchart of an embodiment by the virtual reader data collector shown in FIG. 1.
FIG. 8 is a flowchart illustrating an example of an operation performed by the user input document processor illustrated in FIG. 1.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only the present embodiments to make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" components, steps, operations, and / or elements referred to may include the components of one or more other components, steps, operations, and / or elements. It does not exclude existence or addition.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly.

Hereinafter, with reference to the accompanying drawings, it will be described in detail preferred embodiments of the present invention. The same reference numerals are used for the same elements in the drawings, and duplicate descriptions of the same elements are omitted.

Embodiments of the present invention calculate the author's confidence in an event by analyzing the expression used by the web document authors to refer to the event in the web document, and regard the web document author as a virtual reader for the document input by the user. Therefore, the main point is to automatically predict the distribution of the reliability of the readers.

That is, the present invention automatically predicts the reliability distribution, which is a statistical distribution of the reliability that other readers than the user would read and display, instead of providing the reliability as a single number, and present the distribution to the user.

Predictive models are required to automatically predict the reliability distribution. Especially, in order to build a predictive model through machine learning, corpus, a corpus, which is a document corpus that stores the distribution of reliability collected through actual questionnaires. need. In this case, in general, the more documents that the reliability distribution is measured and the more people who participated in the survey, the accuracy of the reliability distribution prediction increases. However, a high cost is required for collecting the reliability distribution information of readers in such a question-based manner for a large amount of documents.

The present invention can collect and analyze large-scale data in a low cost manner that allows for improved reliability distribution prediction accuracy, without resorting to corpus extension through additional reliability questionnaires.

1 illustrates a configuration of a system for predicting reliability distribution according to an embodiment of the present invention.

Referring to FIG. 1, a reader reliability distribution prediction system 100 according to an embodiment of the present invention includes a user input document processor 120, a virtual reader data collector 110, and a prediction reference corpus 130.

The user input document processing unit 120 automatically predicts and outputs a reliability distribution seen by the public with respect to a document input by the user.

For example, the user input document processing unit 120 receives a document from the user and outputs a reliability distribution to be read by the reader. In this case, the output reliability distribution may differ according to a plurality of demographic characteristics that specify the reader group, and the reliability that one reader evaluates for one document is -5 at a very unreliable + level. It can be an integer up to 5 points (very reliable). Of course, the scope of the present invention is not limited thereto, and may be practiced for a range of reliability of various aspects.

Here, the demographic characteristics are the first item of the gender of the reader who is the subject of determining the reliability of the document, the second item of the age of the reader who is the subject of the reliability of the document, the subject of the reliability of the document. The third item on the reader's job, the fourth item on the subject of interest of the reader, and the subject of judgment on the reliability of the document, whether the reader is an expert on the subject of the document. It may include at least one item of the fifth item for.

The demographic characteristic shown in FIG. 4 illustrates a case in which all five items from the first item to the fifth item are included. However, the scope of the present invention is not limited thereto, and may be implemented in all cases including one or more items as described above. As illustrated in FIG. 4, the user input document processing unit 120 automatically predicts a reliability distribution of various reader groups, which are divided according to a plurality of demographic characteristics, for a document input by a user and calculates a result of the result. Output it to the user and provide it.

Here, the reason why a reader group divided according to a plurality of demographic characteristics is provided as a distribution type rather than a number of figures is that two people included in the same reader group have different reliability for one document. Because it can be seen. In other words, even if two people have the same gender, age, occupation, subject of interest, and expertise, the two documents may have different reliability values for one document. Therefore, the system according to the embodiment of the present invention is defined as a distribution form rather than a single number in order to reflect the individual difference in the reliability that is expected to not be resolved despite the classification of reader groups through a plurality of demographic characteristics. Output the reliability prediction result.

In FIG. 4, reader group 1 is a reader group of a man in his 40s who is a non-specialist in the service industry with an interest in life / health, and may exhibit a distribution of reliability biased with low reliability for input documents. A reader group of women in their twenties who are experts in the education industry with an interest in learning, and can show a distribution of confidence biased with high confidence in input documents.

As described above, the user input document processing unit 120 automatically predicts a reliability distribution of each reader group classified according to demographic characteristics of the document input by the user, and outputs the result to the user. Through this, when the user encounters text content, the reader reliability distribution prediction system according to an embodiment of the present invention may (1) not read the text content at all, (2) read it quickly, or (3) take time. In the process of deciding whether or not to read carefully, decision support information may be provided.

Providing users in the form of a statistical distribution of whether or not they trust a given document by other readers, especially groups of people with different demographic characteristics, is a way to draw empathy between the user and other readers. It can be interpreted that the system according to an embodiment of the present invention through which the user is provided with sufficient information in the process of determining whether to read the given text content.

The prediction criterion corpus 130 stores readers' reliability distributions for each document collected through a direct reliability questionnaire, and the corpus is in the process of constructing a predictive model through machine learning that automatically predicts the reliability distribution. It can be used as a learning standard.

FIG. 5 illustrates an example of a prediction criterion corpus included in an independent reliability distribution prediction system. As illustrated in FIG. 5, the prediction criterion corpus is divided according to a plurality of document sets and a plurality of demographic characteristics for each document. Store the distribution of confidence that the reader group (or group) actually showed through the questionnaire.

Preferably, the plurality of document sets stored by the prediction criterion corpus may include various kinds of documents on various topics that may be encountered in everyday life.

For example, the subject of a document may include life, health, politics, policy, economics, the environment, and the type of document may include SNS posts, blog posts, online news, online forum posts, research papers, books. have. Preferably, the subject and type of each document subject to a direct questionnaire throughout the corpus should not be uniform. This is because the prediction reference corpus 130 collected through a questionnaire in the system according to an embodiment of the present invention is used as a learning criterion for automatically predicting a reliability distribution.

For example, if the predictive criteria corpus 130 includes only documents on life / health and the results of the reliability questionnaire that readers see for these documents, the predictive model learned through the corresponding predictive criteria corpus is a document on political / policy. It would be inappropriate to predict the reliability distribution of and the expected results would be different from the reliability distributions that the actual readers would see.

The virtual reader data collection unit 110 analyzes the degree of trust expressed by the author directly in the document for each event in a given web document, performs the analysis on all documents found on the web, and analyzes the analysis result. Statistical processing stores the author's confidence distribution for each event in a database, for example, an event confidence database.

FIG. 6 shows an example of an event trust database included in the reader confidence distribution prediction system. As shown in FIG. 6, the event trust database indicates the degree of trust expressed by the authors through text for an event mentioned in a web document. It is converted into a statistical distribution of the number of authors according to the reliability value (hereinafter referred to as "event reliability distribution") and stored. That is, an event confidence distribution is generated and stored based on each confidence value.

In this case, an event may be defined as a predicate exposed in the text, a semantic role labeled with a semantic role for the corresponding predicate, and a set of words corresponding to each semantic. Can be.

FIG. 6 is an event corresponding to the meaning of the sentence "tense competition between Company A and Company B is expected in the next 10 years" (hereinafter referred to as "example event"), "predicate: expected", "subject: company A and Company B "," object: tension, "and" temporal: the next decade "include examples of the distribution of the confidence that the authors of a web document reveal through text directly within the document.

For example, Author_1, the author of an SNS article (Doc_1), disagreed to the example case by writing in Doc_1, a sentence in "Doctor A and B that would have tension between Company A and Company B over the next decade". Author_2, author of Online News (Doc_2), wrote in Doc_2, "It will be possible after 10 years to see if it is true that Company A and Company B have a tense relationship over the next decade." The statement shows that there is a neutral trust in the example case, and Author_3, the author of the blog post (Doc_3), writes in Doc_3: “Reviewing the evidence base and analyzing historical trends, It is very clear and reasonable reasoning that Firm B will have a tense relationship. ” Author_4, author of the research document (Doc_4), writes in Doc_4, "In economic terms, when two firms with more than a certain level begin to compete, one company merges with another. We can see that this study shows, so we can infer that Company A and Company B are unlikely to be in tight competition for the next decade. ”

Trust of each author may be statistically converted into an event confidence distribution and stored in the event trust database 115 as shown in FIG. 6.

The event confidence distribution stored in the event trust database 115 is used as data to assist the user input document processing unit 120 in predicting the reliability distribution of the document input from the user. This kind of data utilization can be interpreted in such a way that the author of the web document is considered a virtual reader of the document entered by the user. In other words, the authors of a web document are written for the purpose of describing their opinions and knowledge, but in such documents, the authors can measure their own degree of trust in a specific event as illustrated in FIG. 6 (Author 1 ~ 4). It can be seen directly through the text. The process of collecting opinions expressed through text by the authors of web documents can be performed at a lower cost than the process of collecting readers' trust through a direct questionnaire.

The reader reliability distribution prediction system according to an embodiment of the present invention collects the reliability revealed by the authors about the event on the web in a low cost manner through the virtual reader data collection unit 110, and utilizes this information. The reliability distribution of the user input document processing unit 120 may be improved.

FIG. 2 illustrates a configuration of an embodiment of the virtual reader data collector of FIG. 1.

As shown in FIG. 2, the virtual reader data collecting unit 110 includes a web document collecting unit 111, an event extracting unit 112, an event reliability processing unit 113, a statistical processing unit 114, and an event trust database 115. ).

The web document collecting unit 111 collects web documents collectible on the web.

Here, the web document to be collected by the web document collection unit 111 may include SNS posts, blog posts, online news, online forum posts, research papers, books.

Furthermore, the web document collection unit 111 collects the date information together with each web document when the web document publication date is accessible together with information on a plurality of demographic characteristics for specifying the author. Preferably, the web document collection unit 111 distinguishes the authors through the unique identifiers when the unique identifiers of the authors are accessible, and if there are a plurality of documents written by the same author, they can be distinguished that they are created by one author. Make sure

The event extracting unit 112 extracts event information from a web document.

In this case, an event may be defined as a predicate and a set of words corresponding to a semantic region determined according to semantic role labeling as described above, and the semantic extraction may be performed through a semantic extractor such as DeepSemanticRoleLabeling or PathLSTM. Can proceed.

The event credibility processing unit 113 uses the expression of one or more of the affirmative expression, the negative expression, the trust expression, the distrust expression, the consent expression, and the disagreement expression used by the author as a virtual reader to refer to the event. Calculate the reliability.

In this case, the event reliability may be calculated by Equation 1 below.

[Equation 1]

Here, PositiveCount means the number of positive expressions used by the author in referring to the event in the web document, NegativeCount means the number of negative expressions used in the web document by the author in referring to the event, and TrustCount is the number of negative expressions used by the author in the web document. DistrustCount refers to the number of times the trust expression was used while referring to an event, DistrustCount refers to the number of distrust expressions used by the author to refer to an event within a web document, and AgreeCount refers to the number of times the author used to refer to the event within a web document. DisagreeCount may mean the number of disagreements expressed by the author in referring to the event in the web document.

Here, α ₁ to α ₆ mean non-negative real numbers representing weights, respectively, and when the weight of one value is 0, it means that the number of occurrences of the corresponding expression is not considered. For example, the case where α ₅ is 0 means that the number of occurrences of the synonym expression is not taken into account. And at least one of α ₁ to α ₆ has a non-zero value.

Preferably, the above-described positive expression, negative expression, confidence expression, distrust expression, consent expression, and disagreement expression may include a positive meaning expression word set, a negative meaning expression word set, a trust meaning expression word set, a distrust meaning expression word set, and a synonym meaning. It can be distinguished by using the expression word set and the non-synonymous expression word set. At this time, the word sets can be constructed through the annotation of linguistic experts.

In addition, the event reliability processing unit 113 determines whether a specific expression (eg, affirmative expression) is used while referring to a corresponding event, and one or more words of a predicate corresponding to the event and a word corresponding to each semantic domain are included. The calculation may be performed based on whether the word set corresponding to the specific expression to be targeted (eg, a positive expression word set in the case of a positive expression) is directly connected on the dependency tree.

Here, the dependency tree of each sentence can be extracted through a parser such as StanfordCoreNLP.

For example, when the event predicate is directly connected on the dependency expression tree of affirmative expression words and sentences, the event reliability processor 113 considers the corresponding sentence as a positive reference to the event, and the predicate and each meaning of the event. If none of the inverse words is directly linked to a positively-expression word in the sentence's dependency tree, the sentence is considered to not refer to the event positively.

The statistics processing unit 114 collects event reliability values that the plurality of authors reveal for the same event in various web documents and converts the event reliability distribution, which is the distribution of the number of authors according to the event reliability values. Here, an example of the event reliability distribution is as shown in FIG. 6.

In this case, the statistical processor 114 may classify the authors according to a plurality of demographic characteristics and unique identifiers of the authors collected through the web document collector 111.

For example, if the statistical processor 114 cannot distinguish the unique identifiers of the authors or if the demographic characteristics of the plurality are not known, the statistical processor 114 sets the demographic characteristics of the corresponding authors as “unknown” and different documents. It is assumed that each is written by different authors, and that the author's confidence in the corresponding event is represented by Equation 1 for each document when the same author mentions the same event in different ways in two or more different documents. The average of the calculated reliability scores is used.

Furthermore, the statistics processing unit 114 collects the reliability values of the authors corresponding to each author group classified according to the plurality of demographic characteristics and the group having a demographic characteristic of "unknown", For each author group, the event reliability distribution is stored in the event trust database 115 as shown in FIG. 6.

The event trust database 115 is a database that stores all information related to the events of the present invention.

FIG. 3 illustrates a configuration of an embodiment of the user input document processing unit of FIG. 1.

As illustrated in FIG. 3, the user input document processing unit 120 includes an input unit 121, a preprocessor 122, a reliability distribution predicting unit 123, and an output unit 124.

The input unit 121 receives a document that is a target of reliability distribution prediction from a user.

The preprocessor 122 proceeds to extract the event from the document input by the user.

Here, the event extraction method may be the same as the event extraction method performed by the event extraction unit 112 of the virtual reader data collection unit 110 described above.

The reliability distribution predicting unit 123 loads an event reliability distribution corresponding to each event extracted from a document input from a user from the event trust database 115, and then receives all event reliability distributions for each event and receives the event reliability distribution. Use this to predict the document reliability distribution, which is the confidence distribution that each author group, which is classified according to a plurality of demographic characteristics, is expected to appear for a document input from a user.

At this time, the reliability distribution predicting unit 123 may predict the document reliability distribution of the document input by the user using a model learned through supervised learning from the event trust database 115 and the prediction reference corpus 130. Can be.

In this case, the learning model uses text information in a document stored in the prediction criterion corpus and learning event distributions corresponding to events extracted in the document as an input criterion. The document reliability distribution measured through the actual questionnaire can be retrieved from the prediction criterion corpus 130 and used as an output criterion.

The output unit 124 outputs the document reliability distribution predicted by the reliability distribution predicting unit 123 to the user input document.

As such, the system according to an embodiment of the present invention calculates the author's confidence in the event by analyzing the expression used by the web document authors to refer to the event in the web document, and the web document author for the user input document Considering a hypothetical reader, you can automatically predict the distribution of confidence that readers across the public will see.

In addition, the system according to the embodiment of the present invention, for two users who trust one text content to a different degree, after reading the corresponding text content, the user A shows high reliability, the user B is low When showing confidence, readers are expected to predict the overall distribution of confidence and provide it to A and B so that there is a distribution of people whose confidence corresponds to that of A and B, and that differs from what they see. By showing how people are distributed, you can provide high satisfaction for both users.

FIG. 7 illustrates an operation flowchart of an embodiment by the virtual reader data collector shown in FIG. 1.

Referring to FIG. 7, the method by the virtual reader data collecting unit includes a web document collecting step S310, an event extracting step S320, an event reliability processing step S330, and an event statistics processing step S340.

The web document collection step S310 is a step of collecting a web document, and the event extraction step S320 is a step of extracting an event from the collected web document. The event reliability processing step (S330) includes (1) positive expression, (2) negative expression, (3) trust expression, (4) distrust expression, (5) used by the author as a virtual reader in referring to the event in the collected web document. (6) calculating the event reliability, which is the author's confidence in the event, using at least one expression among the expressions of consent and (6) disagreement. Collecting event reliability values and converting them into a distribution of the number of authors according to the event reliability values and storing them in a database.

FIG. 8 is a flowchart illustrating an example of an operation performed by the user input document processor illustrated in FIG. 1.

Referring to FIG. 8, the method by the user input document processing unit includes an input step S410, a preprocessing step S420, a reliability distribution prediction step S430, and an output step S440.

 The input step S410 is a step of receiving a document that is a target of document reliability distribution prediction from the user, and the preprocessing step S420 is a step of extracting an event from the document input by the user. In the reliability distribution prediction step (S430), event statistical information corresponding to an event extracted from a document input from a user is received as an input from a database, and respective reader groups divided according to a plurality of demographic characteristics are input to the document input from the user. It is a step of predicting the reliability distribution to be seen, and the output step (S440) is a step of outputting the predicted reliability distribution to the user.

Although the descriptions of the methods of FIGS. 7 and 8 are omitted, each step constituting FIGS. 7 and 8 may include all the contents described with reference to FIGS. 1 to 6, which will be understood by those skilled in the art. Self-explanatory

The system or apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the systems, devices, and components described in the embodiments may include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs). ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and may configure the processing device to operate as desired, or process independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to the embodiments may be embodied in the form of program instructions that may be executed by various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or, even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (10)

Calculating event reliability for each event information extracted from the web document by considering the authors of the web document as virtual readers in the virtual reader data collection unit;
Receiving a document for performing a reliability distribution prediction from a user in a user input document processor; And
Predicting, by the user input document processing unit, a reliability distribution of the input document of each of the reader groups distinguished according to a plurality of preset demographic characteristics based on the calculated event reliability of each of the calculated event information;
Including,
The step of calculating the event reliability
Collecting the web document and extracting each of the event information from the collected web document;
Calculating event reliability for each of the extracted event information; And
The event reliability distribution, which is a statistical distribution of the number of readers according to the value of the event reliability, is collected based on the value of the collected event reliability. Steps to Create and Save to Database
Including,
The predicting step
Predicting a reliability distribution of the event information of the user extracted from the input document by using the event reliability distribution corresponding to each of the user's event information extracted from the document input from the user among the generated event reliability distributions ,
The case information
A predicate that is revealed in the text of the web document, a semantic role labeled with a semantic role for the predicate, and a set of words corresponding to each semantic,
The predicting step
The document inputted by the user using a learning model learned through supervised learning from a predictive reference corpus that stores the distribution of the reader's reliability for each document collected through the information stored in the database and the direct reliability questionnaire. Predict reliability distribution,
The learning model
Text information in a document stored in the prediction criterion corpus and learning reliability distributions corresponding to the events extracted in the document during the learning are retrieved from the database as input criteria, and the document measured through the actual questionnaire for the document. An independent reliability distribution prediction method for importing a reliability distribution from the prediction reference corpus as an output reference.
delete The method of claim 1,
The demographic characteristic is
A first item for a gender of the reader, a second item for the age of the reader, a third item for the occupation of the reader, a fourth item for the subject of interest of the reader, and a subject for determining the reliability of the document; And at least one of a fifth item of whether the reader is an expert on a subject of a document.
The method of claim 1,
The step of calculating the event reliability
A reader that calculates the event reliability based on the number of positive expressions, negative expressions, confidence expressions, disbelief expressions, consent expressions, and non-movement expressions used by the author in referring to the event. Reliability Distribution Prediction Method.
The method of claim 1,
Providing the user with a confidence distribution of each of the predicted reader groups
Reader reliability distribution prediction method further comprising.
A virtual reader data collection unit that considers authors of a web document as virtual readers and calculates event reliability for each event information extracted from the web document; And
Receives a document for performing a reliability distribution prediction from a user, and for each of the input document of each of the reader groups distinguished according to a plurality of demographic characteristics preset based on the event reliability for each of the calculated event information User input document processing unit for predicting the reliability distribution
Including,
The virtual reader data collection unit
Collects the web document, extracts each of the event information from the collected web document, calculates an event reliability for each of the extracted event information, and calculates an event reliability calculated for the same event information among the extracted event information Sum the values of, generate an event reliability distribution which is a statistical distribution of the number of readers according to the value of the event reliability, and store in the database,
The user input document processing unit
Predicting a reliability distribution of the event information of the user extracted from the input document by using the event reliability distribution corresponding to each of the user's event information extracted from the document input from the user among the generated event reliability distributions ,
The case information
It is a set of predicates and semantic roles labeled for the predicate and the corresponding predicates revealed in the text of the web document, and a set of words corresponding to each semantic domain.
The user input document processing unit
The information inputted by the user using a learning model learned through supervised learning from a predictive reference corpus that stores readers' reliability distributions for each document collected through the information stored in the database and the direct reliability questionnaire. Predict reliability distribution,
The learning model
Text information in a document stored in the prediction criterion corpus during learning and event reliability distributions corresponding to events extracted in the document are retrieved from the database as input criteria, and the document measured through the actual questionnaire for the document. An independent reliability distribution prediction system for importing a reliability distribution from the prediction reference corpus as an output reference.
delete The method of claim 6,
The demographic characteristic is
A first item for a gender of the reader, a second item for the age of the reader, a third item for the occupation of the reader, a fourth item for the subject of interest of the reader, and a subject for determining the reliability of the document; And at least one of a fifth item of whether the reader is an expert on a subject of a document.
The method of claim 6,
The virtual reader data collection unit
A reader that calculates the event reliability based on the number of positive expressions, negative expressions, confidence expressions, disbelief expressions, consent expressions, and non-movement expressions used by the author in referring to the event. Reliability Distribution Prediction System.
The method of claim 6,
The user input document processing unit
And providing the user with a reliability distribution of each of the predicted reader groups.

Images