JP7523379B2 - Data evaluation system and submission evaluation method - Google Patents

Description

The present invention relates to the content evaluation of data such as posts made via SNS (Social Networking Service) etc.

Patent Document 1 states that "a false post filter device includes an event identification unit that analyzes the content of a post notifying the occurrence of an event and identifies the location where the event occurred; a contact identification unit that searches a contact database that stores data that associates locations observed by one or more devices with contacts of management entities that manage the one or more devices and identifies the contact corresponding to the location identified by the event identification unit; a contact unit that transmits a request to the contact identified by the contact identification unit to confirm whether the event has occurred based on the observation results of the one or more devices and receives a response to the request; and a result reflection unit that determines the authenticity of the content of the post based on the confirmation result indicated in the response received by the contact unit and executes a process on the post according to the determination result."

WO2018/216173 A1

With the technology of Patent Document 1, if it takes time to confirm the observation results, it is not possible to decide how to handle the post and the information cannot be used quickly. However, since information posted to SNS by people who happen to be in the area often contains faster and more accurate information about the area, it is desirable to be able to use the information quickly.

Therefore, the present invention provides a post evaluation device that extracts useful posts while simultaneously quickly grasping detailed local information and eliminating erroneous information.

One aspect of the present invention is a data evaluation system including a first acquisition unit that acquires first data including text data, a second acquisition unit that acquires second data, a structuring unit that extracts first information on the presence or absence of a predetermined phenomenon from the text data, an accuracy evaluation unit that extracts second information on the presence or absence of the predetermined phenomenon from the second data, and determines the first information that contradicts the second information to be false if the presence or absence of the predetermined phenomenon has been determined based on the second information, and determines the first information to be true if the presence or absence of the predetermined phenomenon has not been determined based on the second information, thereby calculating an accuracy index of the first data, and a usefulness index calculation unit that calculates an index of usefulness of the first data based on the accuracy index.

Another aspect of the present invention is a method using an information processing system that includes a first acquisition unit and a second acquisition unit and processes information obtained from the first acquisition unit and the second acquisition unit, in which the first acquisition unit acquires first information consisting of text information posted from a first information source, the second acquisition unit acquires second information consisting of items and data for the items from a second information source, and compares the presence or absence of a phenomenon to be judged that is interpreted based on the first information with the presence or absence of a phenomenon to be judged that is interpreted based on the second information to determine whether the first information is correct or not, and This is a posting evaluation method in which a usefulness index of the first information is calculated using the true/false, and when determining the true/false of the first information, if the presence or absence of the phenomenon to be judged interpreted based on the first information matches the presence or absence of the phenomenon to be judged interpreted based on the second information, the first information is judged to be true, if the presence or absence of the phenomenon to be judged interpreted based on the first information does not match the presence or absence of the phenomenon to be judged interpreted based on the second information, the first information is judged to be false, and if the presence or absence of the phenomenon to be judged interpreted based on the second information is uncertain, the first information is judged to be true.

The present invention provides a post evaluation device that extracts useful posts while simultaneously quickly grasping detailed local information and eliminating erroneous information.

Problems, configurations and advantages other than those mentioned above will become clear from the description of the embodiments below.

FIG. 1 is a block diagram showing a configuration of a posting evaluation system. FIG. 2 is a hardware block diagram of the posting evaluation device. A schematic diagram showing part of the railway line that is subject to posted evaluations. 1 is a table showing posted data. 11 is a flowchart showing the process flow of a post structuring unit. A table showing a list of named entity classifications. FIG. 13 is a conceptual diagram showing an example of a named entity extraction result. FIG. 13 is a table showing structured post data. A table showing an example of a train operation schedule. A table showing train location information. 11 is a flowchart showing the process flow of an accuracy evaluation unit. Table diagram of Example 1 of structured individual post data shown on a post-by-post basis. FIG. 11 is a table diagram of Example 2 of structured individual post data shown on a post-by-post basis. FIG. 11 is a table showing an example of delay information extracted from highly reliable information. FIG. 13 is a table showing a judgment table used by the accuracy evaluation unit. 11 is a flowchart showing the process flow of a detail level evaluation unit. 11 is a flowchart showing the process flow of an information source evaluation unit. 13 is a table showing rated post data. FIG. 13 is an image showing an overview of the posting evaluation results. An illustration showing route details based on posted evaluation results. 11 is a flowchart showing the process flow of an accuracy evaluation unit. A table showing highly reliable information regarding equipment inspections within stations input by the accuracy evaluation unit. 11 is a table showing an example of one post among the structured post data input by the accuracy evaluation unit.

The following describes the embodiments with reference to the drawings. However, the present invention should not be interpreted as being limited to the description of the embodiments shown below. Those skilled in the art will easily understand that the specific configuration can be changed without departing from the concept or spirit of the present invention.

In the configurations of the embodiments described below, the same parts or parts having similar functions are designated by the same reference numerals in different drawings, and duplicate descriptions may be omitted.

When there are multiple elements with the same or similar functions, they may be described using the same reference numerals with different subscripts. However, when there is no need to distinguish between multiple elements, the subscripts may be omitted.

The designations "first," "second," "third," and the like in this specification are used to identify components and do not necessarily limit the number, order, or content. Furthermore, numbers for identifying components are used in different contexts, and a number used in one context does not necessarily indicate the same configuration in another context. Furthermore, this does not prevent a component identified by a certain number from also serving the function of a component identified by another number.

The position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings, etc.

The publications, patents and patent applications cited herein are incorporated by reference in their entirety into the present specification.

In this specification, elements expressed in the singular include the plural unless the context clearly indicates otherwise.

In the first embodiment, a post evaluation device is used to evaluate the usefulness of SNS posts regarding traffic information, particularly regarding the presence or absence of delays in train service, so as to quickly extract detailed information while filtering out false information.

Figure 1 is a block diagram showing the configuration of a posting evaluation system 100 according to an embodiment. The posting evaluation system 100 includes a posting evaluation device 101 and an information display device 105.

The post evaluation device 101 includes a post data acquisition unit 111, a highly reliable information acquisition unit 112, a post structuring unit 121, a level of detail evaluation unit 122, an accuracy evaluation unit 123, an information source evaluation unit 124, a usefulness index calculation unit 125, an evaluated post storage unit 131, an evaluation update unit 141, and a distribution unit 151. As described below, the post evaluation device 101 can be configured as an information processing device such as a server. In FIG. 1, the functional blocks are shown, omitting the components that a server naturally has.

The post evaluation device 101 can collect posts posted by an unspecified number of users from a posting terminal 102, such as a portable information terminal, via an SNS server 103. It can also collect information from a highly reliable information distribution server 104. The information processed by the post evaluation device 101 can be output to an information display device 105. The information display device 105 may be a display device such as an image monitor directly connected to the post evaluation device 101, or may be, for example, a portable information terminal connected via a network.

The post data acquisition unit 111 requests SNS post data related to the railway line to be evaluated from the SNS server 103 and receives the SNS post data. In order to extract SNS post data related to the railway line to be evaluated, a publicly known search engine can be used to collect posts that include keywords related to the railway line to be evaluated, such as the name of a railway operator, the name of a line, and the names of stations. The collection is requested, for example, periodically (e.g., every minute), and added posts are received every moment. This makes it possible to constantly monitor SNS posts. In addition to or instead of the constant collection, posts may be collected at any timing. The post data acquisition unit 111 transmits the received post data to the post structuring unit 121. The post data will be described in detail with reference to FIG. 4.

The highly reliable information acquisition unit 112 requests highly reliable information regarding delays on the railway line to be evaluated from the highly reliable information distribution server 104 and receives the highly reliable information. The highly reliable information distribution server 104 is, for example, a server managed and operated by a railway operator that manages and operates the railway to be evaluated, and constitutes an information source different from the SNS server 103. Highly reliable information is information collected by the railway operator, and is generally expected to be more accurate than SNS posted data. Highly reliable information is, for example, managed data that is organized into a database and includes information on location, time, and event.

The highly reliable information acquisition unit 112 transmits the received highly reliable information to the accuracy evaluation unit 123. The highly reliable information is described in detail in Figures 9 and 10.

The post structuring unit 121 receives post data from the post data acquisition unit 111. The post structuring unit 121 performs morphological analysis, named entity extraction, and normalization on each post to extract information on location, time, and events related to train operation delays, etc., and to structure the traffic information contained in the post. The morphological analysis and normalization can be carried out using publicly known techniques for document structuring. Named entity extraction is possible by using a named entity extraction model or by referring to a dictionary that stores named entities.

Location information includes, for example, the line, station, direction (travel direction), etc. Event information includes, for example, the presence or absence of a delay, the extent of the delay, and the cause. The post structuring unit 121 transmits the structured post data to the detail level evaluation unit 122, the accuracy evaluation unit 123, and the information source evaluation unit 124. Details of the post structuring unit 121 will be described with reference to Figures 5 to 7.

The detail level evaluation unit 122 receives structured post data from the post structuring unit 121. The detail level evaluation unit 122 calculates an index of the information detail level of the post based on the number of named entities included in the named entity extraction result. The detail level evaluation unit 122 transmits the calculated index value of the information detail level of the post to the usefulness index calculation unit 125. Details of the detail level evaluation unit 122 are described in FIG. 16.

The accuracy evaluation unit 123 receives structured post data from the post structuring unit 121. The accuracy evaluation unit 123 also receives highly reliable information from the highly reliable information acquisition unit 112. The accuracy evaluation unit 123 compiles actual train operation delay information from the highly reliable information for the location (route/direction) and time mentioned in the structured post data.

If there is no history of delays in the highly reliable information at the time mentioned in the post, the accuracy evaluation unit 123 calculates the scheduled time at which a delay may be confirmed in the future, and if the confirmed scheduled time is in the future than the current time (processing time), the presence or absence of a delay is left unconfirmed.

In addition, the accuracy evaluation unit 123 counts, for example, the latest of the scheduled arrival time of a moving train at the next station and the scheduled departure time of a stopped train as the time when the presence or absence of a delay is confirmed (confirmed time). The scheduled arrival time of a moving train at the next station and the scheduled departure time of a stopped train are made clear, for example, by a railway timetable, as described below. The collected confirmed time determines whether or not a train will arrive at the next station on time or leave the station, so when the confirmed time has passed, the evaluation update unit 141 issues an instruction to update the accuracy evaluation of the post.

The accuracy evaluation unit 123 determines whether the delay mentioned in the post is correct through a comparison with the tabulation results of highly reliable information, and determines an accuracy index. When there is no record of delays and the tabulation results of the delay are unconfirmed, the accuracy evaluation unit 123 provisionally determines that the delay mentioned in the post is correct. The accuracy evaluation unit 123 transmits the calculated index value of the accuracy of the post to the usefulness index calculation unit 125. Details of the accuracy evaluation unit 123 are described in Figures 11 to 15.

In addition, when the accuracy evaluation unit 123 receives an instruction from the evaluation update unit 141 to update the unconfirmed and provisionally calculated index value of the post's accuracy, the presence or absence of a delay has already been determined at that point, so the accuracy evaluation unit 123 performs the same processing as described above to calculate and update the confirmed index value of the accuracy.

The information source evaluation unit 124 receives structured post data from the post structuring unit 121. The information source evaluation unit 124 also receives evaluated post data from the evaluated post storage unit 131. For each post, the information source evaluation unit 124 calculates an index related to the information source of the post content based on the sender's attributes and past post content, and whether the post content is presumed to be hearsay. The information source evaluation unit 124 transmits the calculated index value of the information source of the post to the usefulness index calculation unit 125. Details of the information source evaluation unit 124 will be described in FIG. 17.

The usefulness index calculation unit 125 receives index values of each post from the detail evaluation unit 122, the accuracy evaluation unit 123, and the information source evaluation unit 124. The usefulness index calculation unit 125 calculates a usefulness index based on each index value. The usefulness index calculation unit 125 records the evaluated post data for which the usefulness index has been calculated in the evaluated post storage unit 131. Details of the evaluated post data will be described in FIG. 18.

The rating update unit 141 starts periodically (e.g., once a minute). The rating update unit 141 receives evaluated post data from the evaluated post storage unit 131. From the evaluated post data, the rating update unit 141 extracts posts for which high reliability information is unconfirmed and for which the current time has passed the scheduled confirmation time as posts to be updated. The rating update unit 141 sends an instruction to the accuracy evaluation unit 123 to update (recalculate) the accuracy index for the post to be updated. For posts for which high reliability information is unconfirmed and an accuracy index has been provisionally calculated, the rating update unit 141 updates the accuracy index based on the confirmed high reliability information, thereby improving the accuracy of excluding misinformation.

The distribution unit 151 receives evaluated post data from the evaluated post storage unit 131. The distribution unit 151 extracts posts within a certain period of time, for example, in descending order of usefulness index, and transmits information about the evaluated posts to the information display device 105. The information to be transmitted can include not only the extracted posts, but also text and numerical information that consolidates the information about the extracted posts. The information to be transmitted may also include text summarizing only the important parts from highly useful posts, a word cloud constructed by extracting keywords that are frequently included in multiple posts, and a count of the number of posts. Furthermore, the presence or absence of delays, the duration of delays, and other information that consolidates the operating status of the line from among the highly reliable information can also be transmitted.

Information on evaluated posts can be sent to a destination other than the information display device 105, as well as to an SNS server. For example, it is conceivable to post a post on an SNS that quotes a post with a high usefulness index value, with the post evaluation device 101 as the poster.

The information display device 105 receives information on evaluated posts from the distribution unit 151 of the post evaluation device 101, and displays the text of highly useful posts and their summary information on the screen. Employees of the transportation company, such as crew members and station staff who inform passengers of operation status, can check the screen display and understand the information, which can improve the guidance provided to passengers compared to when they are unaware of the information. In addition, dispatchers who determine the transportation company's operation plans can check the screen display and understand the information, which may enable them to select operation plans that meet passenger expectations.

In addition, passengers of public transport can check the screen display and understand the information, allowing them to select how to use the public transport that best suits their travel purpose. For example, by quickly learning about delays on a route they had planned to use, they can choose to use a different route or postpone their travel time, which is thought to be highly beneficial. Details of the information display device 105 will be described in Figures 20 and 21.

The SNS server 103 receives posts sent from the posting terminal 102 and transmits the post data to the post evaluation system 100 in response to a request.

The highly reliable information distribution server 104 receives and aggregates train operation schedules and train location information from railway train operation management systems and the like, and transmits posting data to the posting evaluation system 100 upon request.

Figure 2 is a hardware block diagram of the posting evaluation device. The hardware configuration of the posting evaluation device 101 will be described with reference to Figure 2. In Figure 2, the posting evaluation device 101 is composed of a CPU (Central Processing Unit) 201, a memory 202, a media input/output unit 203, a communication control unit 204, an input unit 205, a display unit 206, a peripheral device IF (Interface) unit 207, and a bus 210.

The CPU 201 executes programs on the memory 202 to realize the functions of the various functional blocks shown in FIG. 1. The memory 202 temporarily stores programs, tables, etc. The media input/output unit 203 holds the programs, tables, etc.

The input unit 205 is a keyboard, a mouse, etc. The communication control unit 204 is connected to a network 220. The network 220 enables communication with other devices such as the SNS server 103 and the highly reliable information distribution server 104. The display unit 206 is, for example, a display. The peripheral device IF unit 207 is an interface for a printer or the like. The bus 210 interconnects the CPU 201, the memory 202, the media input/output unit 203, the communication control unit 204, the input unit 205, the display unit 206, and the peripheral device IF unit 207.

As is clear from a comparison between FIG. 1 and FIG. 2, the posting evaluation device 101 in FIG. 1 is realized by the CPU 201 executing a program. However, at least a part of each functional block may be configured with hardware. Also, in the example of FIG. 2, the posting evaluation device 101 is configured with a single server, but it is also possible to realize the same function by multiple servers working together.

Figure 3 is a schematic diagram showing a portion of the railway line that is the subject of post evaluation. This route map includes two railway lines, Line X 391 and Line Y 392.

X-line 391 is a double-track line that transports passengers between stations such as A Station 301, B Station 302, C Station 303, and D Station 304, and trains operate in two directions, P and Q. Figure 3 shows that train 311 in the Q direction is running between A Station 301 and B Station 302, and train 312 is stopped at C Station 303. Similarly, trains 321, 322, and 323 are operating in the P direction.

The Y line 392 is a line that can be transferred to the X line 391 at station B 302. From station B 302, the Y line 392 branches off into the S and U directions beyond station K 306.

If a line has mutual through service with lines operated by other railway operators, the line and the other line may be treated as the same line.

Figure 4 is a table showing post data. Post data 400 is an example of post data related to X-rays or Y-rays received from the SNS server 103. Each row of post data 400 represents one post. One row includes a post ID 401 that uniquely identifies the post, a post date and time 402 that indicates, for example, the date and time when the SNS server 103 received the post, post content text 403, a poster ID 404 that uniquely indicates the poster, etc. Post content text 403 may include a URL (Uniform Resource Locator) referenced by the poster. Post date and time 402 is what is known as a timestamp.

Figure 5 shows the processing flow of the post structuring unit 121, which structures the post content.

Step 501 indicates the start of processing. The processing may be real-time processing or batch processing. To process information with immediacy, it is better to be closer to real-time processing.

Step 502 is data reception, and the post structuring unit 121 receives post data such as the example shown in Figure 4 from the post data acquisition unit 111.

Step 503 is named entity extraction, in which the posted content text 403 of the posted data is used as input to extract phrases referring to places, times, and events from the posted text using a named entity extraction model constructed using machine learning technology, for example.

For the named entities that define the phrases to be extracted, a structure of up to three levels, for example, major categories, medium categories, and minor categories, may be defined and used, as described later in Figure 6. The definition may be determined arbitrarily depending on the purpose and application of the system.

In this embodiment, a machine learning model such as CRF (Conditional Random Field) is used as the named entity extraction model. When constructing the named entity extraction model, the accuracy of named entity extraction can be improved by registering words such as line names, station names, facility names, and event names in advance. An example of named entity extraction from a post is shown in Figure 7.

Step 504 is normalization, in which the phrases extracted in the previous step 503 are normalized to create the structured post data shown in FIG. 8. Publicly known techniques can be used as the normalization method, but for example, for locations, if there are variations in the notation of phrases such as transportation company, line, station, direction, and train name, they are unified to the official name. Just as a line name that includes a corresponding station is completed from a station name, an item without a corresponding phrase is completed if possible.

If the post does not contain the relevant phrase, the posting date and time is used as the date and time the post refers to. If there is a past tense phrase such as "just now" or "before," the posting date and time is estimated based on the typical usage of the phrase, starting from the posting date and time. For example, "just now" refers to the period from 30 minutes before the posting date and time to the date and time the posting date and time are being mentioned.

In step 505, the result is sent to the next function block, and the process ends in step 506.

Figure 6 shows a list of named entity classifications identified by the post structuring unit 121. The content, classification, and hierarchical structure of the named entity can be arbitrarily defined in advance by the user. In this example, a maximum three-level structure is defined, consisting of major classification, medium classification, and minor classification.

The major categories are "location," "time," "event," and "other" that does not fall into any of the above categories. Subcategories related to the major category "location" (or "object") include "transportation company name," "route," "station," "direction," "train name," "type" of train (such as local or rapid), and "facilities" such as ticket gates and toilets.

Subcategories of the major category "time" include specific "times" such as "14:00" or "14:15", "past tense" using expressions such as "just now" or "a little while ago", and "time periods" using expressions such as "morning", "afternoon", "evening", and "night".

Subcategories of the major category "event" (or "situation") include "operation status," "delay," "accident," "congestion," and "facilities." For example, subcategories of "delay" include qualitative expressions of the "degree" of the delay, such as "severe" or "slight," and "hours" that quantitatively express the delay, such as "5 minutes." Subcategories of "accident" and "congestion" can also include expressions of qualitative and quantitative degrees. Subcategories of "facilities" include "condition" expressions, such as "breakdown" and "prohibited from use." The above are just a few examples, and users can freely define named expressions according to their purpose and use.

Figure 7 is a conceptual diagram showing an example of the results of extraction of named entities by the post structuring unit 121. In the post content text "The train for Q arrived 5 minutes late at station C on the X line. It's so crowded with sushi, I think I'll let it go," phrases related to places and events extracted as named entities are underlined. In addition, the classification of the corresponding named entities is shown below the underline. For example, "packed with sushi" is classified into the major category "events," the medium category "crowding," and the minor category "degree." Words and phrases that are not underlined fall into the major category "other." For example, "I think I'll let it go" is classified into "other."

Figure 8 is a table showing the structured post data output by the post structuring unit 121. An excerpt of the output obtained when the post structuring unit processes the post data of Figure 4 is shown. One row in the table of structured post data 800 corresponds to one post. One row includes a post ID 801 (same as 401 in Figure 4) that uniquely identifies the post, and named entity classifications 802 to 806 obtained as a result of processing the text of the post, which are assigned according to the classification shown in Figure 6, for example. Named entity extraction result 807 stores the data of the named entity extraction result shown in Figure 7, for example.

For example, in the post with post ID "14371" in Figure 7, the named entity classification "Location: Line" 802 is "X (line)", "Location: Station" 803 is "C (station)", "Date and Time" 804 is "12:04" based on the posting date and time 402, "Event: Delay" 805 is "Delayed", and "Event: Delay: Hours and Minutes" 806 is "5 minutes" based on the content mentioned in the post. In this way, information based on phrases extracted from the post content by the post structuring unit 121 is set for the items of line, station, date and time mentioned in the post, mention of delay, and delay time.

Figure 9 is a table diagram showing an example of a train operation plan timetable 900 from the highly reliable information. In this example, an ID 901 is assigned to each combination of timetable revision date 902, weekday/holiday classification 903, line 904, direction 905, train number 906, and type (local, rapid, etc.) 907, and the departure station 908 and arrival station 909 of the train, as well as departure time 910 from the departure station and arrival time 911 at the arrival station are shown for each station section. Generally, the operation plan timetable 900 is determined by the railway company that operates the train, and each train operates according to the contents of the operation plan timetable 900.

Figure 10 is a table showing train location information 1000 from the highly reliable information. A different ID 1001 is assigned to each train number 1004 and information update date and time 1002. In this example, the line 1003, type 1005, starting station 1006, terminal station 1007, and direction 1008 of the train identified by the train number 1004 are included, but this information is usually fixed data obtained from the operation plan timetable 900 corresponding to the train number 1004.

In the train location information 1000, the train's location at the update date and time 1002 is shown by the station where the train is currently stopped, or by a pair of the last departure station 1009 and the next scheduled arrival station 1010. For example, the row with ID "21" indicates that a train with train number "K8888" is between stations B and C. Also, the row with ID "22" indicates that a train with train number "J4567" is stopped at station B. This data is normally used by railway companies and others who operate trains to control train operations, and can be obtained from sensors installed on the tracks or input from an operator.

Figure 11 is a flow chart showing the processing flow of the accuracy evaluation unit 123. In this embodiment, the accuracy evaluation unit 123 judges the accuracy of information posted on SNS or the like, and extracts posts that are useful. The highly reliable information described in Figures 9 and 10 is data used by, for example, railway operators for the accurate operation of railways, and therefore the content can be expected to be accurate. For example, by comparing the operation plan timetable 900 with the on-track information 1000, the status of train delays can be accurately grasped. However, in terms of the immediacy of information, highly reliable information is generally inferior to information posted on SNS or the like.

Highly reliable information is generally organized and systematized by items and information for each item (symbols, numbers, text, etc.). On the other hand, posted information is generally text information written in a free format. The accuracy evaluation unit 123 evaluates the accuracy of the posted information by comparing it with highly reliable information. Since it is difficult to compare posted text information as it is, the post structuring unit 121 performs processing such as the structuring described above as necessary.

Step 1101 indicates the start of processing. The start timing is arbitrary, but for example, the accuracy evaluation unit 123 is usually activated periodically in a standby state. For example, the accuracy evaluation unit 123 is activated at 10-minute intervals and processes posted information from the last 10 minutes.

Step 1102 is data reception, where the accuracy evaluation unit 123 receives structured post data 800 from the post structuring unit 121. Specific examples of posts are described in Figures 12 and 13. The accuracy evaluation unit 123 also receives highly reliable information 900, 1000 from the highly reliable information acquisition unit 112.

The data received by the accuracy evaluation unit 123 may be, for example, all structured post data posted in the last 10 minutes and all updated highly reliable information, but in order to reduce the amount of processing, information extracted according to the information of interest may be received. In this example, a case will be described in which it is desired to collect information regarding "delays" on "Route X" in relation to train operations.

Figures 12 and 13 are table diagrams of structured individual post data 1200, 1300 showing examples of posts of structured post data 800 obtained from the post structuring unit 121, on a post-by-post basis. The contents 1202, 1302 of items 1201, 1301, such as "post ID", "post content", "post time", and named entity "location", "time", and "event", are citations of post ID 801, named entity classifications 802-806, and named entity extraction result 807 of structured post data 800.

As an example, the posts in Figures 12 and 13 are extracted from all posts using keywords based on the information to be collected, and information posted on "2020-12-17 13:50-14:00" that has the unique expressions "Route X" and "Delay" is extracted. For highly reliable information, for example, all trains on the relevant route "X" whose update dates and times of the on-line information (e.g., Figure 10) fall within the relevant time period "13:50-14:00" are extracted.

Step 1103 is delay information extraction, where delay times for departures and arrivals at stations are calculated by comparing the operation plan timetable 900 with the extracted track information 1000. In other words, the deviation from the target value (usually a delay) is calculated by comparing the planned timetable, which is the target value for the position and time of a specific object (train), with the track information, which is the actual position and time of the specific object. Additionally, the next scheduled arrival and departure times are obtained from the operation plan timetable 900, which is the departure time for trains stopped at a station and the arrival time at the next station for trains traveling between stations.

Figure 14 is a table diagram showing an example of delay information extracted by the accuracy evaluation unit 123 from the highly reliable information in step 1103. Figure 14 shows an example of delay information extracted in this step 1103 for "Route X" on "2020-12-17 13:50-14:00". In this example, the delay information 1400 in Figure 14 is obtained by comparing the states of IDs "21" and "23" that meet the above conditions in Figure 10 with the planned timetable in Figure 10. In this case, there is no delay for either.

In the examples of Figures 12 and 13, since the post does not mention a destination, all trains operating on route X during that time period are extracted. If the post mentions a destination, trains operating in the mentioned destination during that time period are extracted. If the post mentions a specific train, only that train is extracted.

Step 1104 is delay information aggregation, where the delay information extracted in the previous step 1103 is aggregated for each post, and the presence or absence of a delay, the delay duration, whether the information is confirmed or unconfirmed, and the confirmed scheduled time are calculated if the information is unconfirmed. If any of the extracted trains has a delay duration of a specified period (e.g., 1 minute) or more, it is considered to have a delay. The delay duration is considered to be the maximum delay duration among the extracted trains. The information is considered to be confirmed.

If the extracted train does not have a delay of more than the specified time, it is considered to have no actual delay. The delay time is considered to be 0 minutes. If any of the next scheduled arrival/departure times of the extracted train are in the future than the time of this processing by the accuracy evaluation unit, the information is considered to be unconfirmed. If not, the information is considered to be confirmed. In the case of unconfirmed, the confirmed scheduled time is considered to be the latest of the next scheduled arrival/departure times of the extracted trains.

Step 1105 is a determination of accuracy, in which the delay information mentioned in the structured post data is compared with the aggregated results from the previous step 1104 for each post to determine whether it is accurate or not, and an index value for accuracy is determined.

Figure 15 is a table diagram showing a judgment table that the accuracy evaluation unit 123 refers to when judging the accuracy of a post. For each post, "1" is correct, "-1" is incorrect, and "0" is uncertain (no evaluation). Depending on the delay record extracted and aggregated from the highly reliable information, the confirmed or unconfirmed information, and the reference to delay in the structured post data and its content (presence or absence of delay), the corresponding value in the table is used as the accuracy index value. In particular, if the highly reliable information is unconfirmed and the post contains a reference to delay, the index value is provisionally set to 1 (positive).

Based on Figure 15, we will explain the "delay" information for the "X-Ray" train "K8888" for the time period "2020-12-17 13:50-14:00" as an example.

Highly reliable information such as "Delays have been recorded" refers to the case where, for example, train "K8888" is supposed to arrive at a specific station during the above time period according to the planned timetable, but the on-track information indicates that it has not yet arrived.

Highly reliable information such as "No delays" and "Confirmed" refers to the case where, for example, train "K8888" is supposed to arrive at a specific station during the above time period according to the planned timetable, but the on-track information shows that it is arriving on time.

Regarding whether there has been a delay, for example, a condition may be set to ignore delays of less than one minute. In other words, the presence or absence of a phenomenon can be defined arbitrarily.

Highly reliable information such as "No actual delays" and "Unconfirmed" refers to cases where, for example, even when comparing the planned timetable with on-track information, it is not possible to determine whether or not there has been an actual delay. For example, according to the planned timetable, train "K8888" does not depart or arrive at any station during the above time period.

A post's "delay" or "present" refers to a case where the posted information contains a specific expression indicating that there is a delay. For example, "K8888 looks like it will be delayed."

The "delay" and "no delay" in posts are cases where the posted information contains a unique expression indicating no delay. For example, "K8888, right on time."

A post with "no mention of delay" means that there is no mention of a delay in the post information.

When there is a contradiction between highly reliable information and posted information, in principle the highly reliable information is correct and the posted information is incorrect, and the accuracy of the post is evaluated. However, when the highly reliable information is unconfirmed, the posted information is provisionally adopted as correct.

In the above example, posts about the "X-Ray" train "K8888" are evaluated, but posts about the "X-Ray" as a whole may also be evaluated. In that case, for example, the following is an example.

The highly reliable information "record of delays" refers to, for example, a case where, according to the planned timetable, there is a train on the "X-line" that should arrive at the specified station during the above time period, but according to the on-track information, there is at least one train that has not yet arrived.

Highly reliable information such as "No record of delays" and "Confirmed" refers to, for example, trains on the "X-line" during the above time period that should be arriving at the designated station according to the planned timetable, but all trains are arriving on schedule according to the on-track information.

The presence or absence of a delay may be determined based on a condition such as "a delay has occurred" only if the majority of trains are delayed from their scheduled times. Alternatively, a condition may be set such that delays of less than one minute are ignored. In other words, the presence or absence of a phenomenon can be defined arbitrarily.

Highly reliable information such as "No actual delays" and "Unconfirmed" refers to cases where, for example, even when comparing the planned timetable with on-track information, it is not possible to determine whether or not there has been a delay.

The "delayed" and "present" comments in posts are when the posted information contains a specific expression indicating a delay. For example, "X-ray looks like it's going to be delayed."

The "delay" and "no delay" in posts are cases where the posted information contains the proper expression that there is no delay. For example, "X-rays are on time."

A post with "no mention of delay" means that there is no mention of a delay in the post information.

Step 1106 is the transmission of the results, in which the calculated index value of the accuracy of the post, the classification of the information as confirmed or unconfirmed, and the scheduled confirmation time if the information is unconfirmed are transmitted to the usefulness index calculation unit 125.

The process ends in step 1107.

Figure 12 is a diagram showing an example of one of the structured post data input by the accuracy evaluation unit 123. This post mentions an actual delay of eight minutes for an X-ray. If the highly reliable information is also counted as having a delay, the index value of the accuracy of the post will be "1".

Figure 13 is a diagram showing another example of a post among the structured post data input by the accuracy evaluation unit 123. This post mentions that the train stopped between stations and that there may be a delay in the X-ray. In the situation of the post content, there will be no actual delay in the highly reliable information (unconfirmed) until the scheduled arrival time of the train at the next station. In the processing of the accuracy evaluation unit 123 described above, the delay information in the highly reliable information is treated as unconfirmed, and the mention of this post is provisionally determined to be correct, thereby enabling information to be extracted quickly from the SNS post. Alternatively, the provisional correct determination may be distinguished as a value such as "0.8" rather than "1", and updated to "1" when the delay is confirmed by the highly reliable information.

Figure 16 is a flow chart showing the processing flow of the detail evaluation unit 122.

Step 1601 is where processing begins.

Step 1602 is data reception, in which structured post data 800 is received from the post structuring unit 121.

Step 1603 is a count of the number of extracted named entities, and an index of the information detail of the post is calculated for each post based on the number of named entities related to location, time, and event contained in the named entity extraction results. In the example post of FIG. 7, three named entities in "major category: location" and four named entities in "major category: event" are extracted, and the index value of the information detail is set to the number of extracted named entities, 3 + 4 = 7. The index is not limited to the number of named entities, and it is also possible to use the ratio of the number of named entities to the number of words contained in the post content text, the depth of classification, weighting of the types of named entities, or other functions.

Step 1604 is the transmission of the results, in which the calculated index value of the information detail of the post is transmitted to the usefulness index calculation unit 125.

The process ends in step 1605.

Figure 17 is a flow chart showing the processing flow of the information source evaluation unit 124.

Processing begins in step 1701.

In step 1702, data is received. In data reception, structured post data 800 is received from the post structuring unit 121, and rated post data 1800 is received from the rated post storage unit 131.

Step 1703 is poster determination. The poster's attributes and past posting content are evaluated based on the poster ID 801, etc. For example, a list of posters who are thought to post many posts that are not very useful in terms of quickly understanding local detailed information is prepared, and posts from posters included in the list are rated low. Also, by referring to the evaluated post data 1800 recorded in the evaluated post storage unit 131, the average value of the usefulness index 1808 of posts from the same poster is calculated, and posts that are greater than a predetermined value (high usefulness) are rated high, for example.

Step 1704 is hearsay determination. The post is evaluated based on whether the post content is presumed to be hearsay. For example, if the post content text contains links or quotations, it is highly likely that the information was obtained by referring to social media, news, articles, etc., rather than information experienced at the site, so the post is rated low as not containing information from the site. In addition, posts that use hearsay expressions such as "seems to be" or "it seems to be" without indicating a specific source of information (such as in-car announcements or station announcements) are highly likely not information experienced at the site, so they are rated low.

Step 1705 is index calculation. The perspectives listed in steps 1703 and 1704 are combined to calculate an index for the information source. For example, the number of highly rated perspectives is set as the index value. Alternatively, each perspective may be weighted appropriately.

Step 1706 is result transmission. The calculated index value of the post's information source is transmitted to the usefulness index calculation unit 125.

The process ends in step 1707.

Figure 18 is a table diagram showing the evaluated post data output by the usefulness index calculation unit 125. One column of the evaluated post data 1800, identified by a post ID 1801, indicates one post. The post ID 1801 and update date/time 1802 can be obtained by quoting the "post ID" 801 and "date/time" 804 from the structured post data 800.

The usefulness index in column 1808 is calculated based on the level of detail 1803 obtained from the level of detail evaluation unit 122, the accuracy 1804 obtained from the accuracy evaluation unit 123, and the information source index value 1807 obtained from the information source evaluation unit 124. For post ID "14371", the value of the confirmed/unconfirmed category 1805 is "applicable" to unconfirmed, and the accuracy is "1" which is "positive", but the corresponding highly reliable information indicates that the phenomenon is unconfirmed, and that it is scheduled to be confirmed at "14:14", as indicated by the scheduled confirmation time 1806. Tagged post content 1809 is data such as that shown in FIG. 7.

The usefulness index calculation unit 125 receives index values for each post from the detail evaluation unit 122, accuracy evaluation unit 123, and information source evaluation unit 124. In addition, it receives from the accuracy evaluation unit 123 the classification 1805 of the information of each post as confirmed or unconfirmed, and the scheduled confirmation time 1806 in the case of unconfirmed information.

The usefulness index calculation unit 125 calculates a usefulness index 1808 based on each index value. For example, the usefulness index takes a value between 0 and 1, and the more useful a post is to a user who checks the evaluated posts on the information display device 105, the higher the index value is. In this way, the calculation is performed using the standard sigmoid function σ and coefficients 1 to 4 according to the following formula.
(Usefulness index) = σ ((Coefficient 1) × (Detail index) + (Coefficient 2) × (Accuracy index) + (Coefficient 3) × (Information source index) + (Coefficient 4))
Here, the standard sigmoid function is defined as σ(x)=1/(1+exp(x)) where exponential function exp(x) is used.
The above formula is an example, and other functions and weights may be used.

The evaluated post data 1800, which combines the information received for each post and the usefulness index, is recorded in the evaluated post memory unit 131.By the above-mentioned processing of the usefulness index calculation unit 125, the usefulness of the post is evaluated in terms of detail, accuracy, and information source, and integrated into a usefulness index value, it is possible to quickly grasp detailed information while eliminating incorrect information.

Figure 19 is an image diagram showing an overview display of the posting evaluation results displayed on the information display device 105. The overview display window 1901 includes a display 1902 showing the evaluation time and an overview display table 1903.

Each row in the summary display table 1903 shows a summary of the rated posts and highly reliable information for each target route.

Row 1921 indicates route X, and row 1922 indicates route Y. Column 1911 indicates the route name.

Column 1912 is a summary of the operation status of the highly reliable information, showing whether operation is normal or there are delays.

Column 1913 displays a graph of the time series trend of the number of posts. The horizontal axis is time, and the vertical axis is the number of posts. It displays the total number of posts related to the route and the number of filtered posts whose usefulness evaluation index exceeds a specified value.

Column 1914 displays representative posts, such as the post with the highest usefulness index value within a specified period of time, or the post with the most recent posting date and time after filtering.

Column 1915 displays information that aggregates the information of the filtered posts. Specifically, it displays keywords that frequently appear in the filtered posts over a given period of time as a word cloud.

Column 1916 displays a button that transitions to a detailed display of the route shown in FIG. 20.

In the example of FIG. 19, the highly reliable information indicates that operation status 1912 is "operating normally." However, information that suggests delays or congestion is extracted from information posted on SNS. In this way, in this embodiment, information with a relatively high degree of accuracy can be extracted early on from information whose reliability is not guaranteed, such as SNS.

Figure 20 is an image diagram showing route details of the posting evaluation results displayed on the information display device 105. The route details display window 2001 displays the name of the route to be displayed on the panel 2011.

Panel 2012 shows an excerpt of information about the line from the rated post storage unit 131 in table format. The user of the information display device 105 can sort the information by most recent posting time, highest usefulness index, etc. The excerpt can be extracted from the rated post data 1800 using keywords such as the line name, station names and train names that belong to the line. Furthermore, it may be possible to narrow down the time of posting, time of mention, and location, and adjust filters based on the value of the usefulness index, etc.

The posted content text can be displayed as the evaluated posted data 1800 itself, or as appropriate items extracted from the evaluated posted data 1800. As shown in FIG. 7, by highlighting and displaying unique entities related to the extracted location, time, and event, the user of the information display device 105 can quickly grasp the key points of the posted content.

The information display described above in Figures 19 and 20 will enable transportation company employees and passengers to quickly grasp detailed information while eliminating erroneous information, which is believed to improve operation status guidance and route and time selection.

In this embodiment, a post evaluation device is described that evaluates the usefulness of SNS posts about equipment abnormalities so as to quickly extract detailed information while filtering out false information. Using facilities such as platforms and toilets in train stations as concrete examples, the processing of the post evaluation device is described with the main differences from the first embodiment. Portions that are not specifically described may be the same as the first embodiment.

In FIG. 1, the accuracy evaluation unit 123 receives structured post data from the post structuring unit 121, and receives highly reliable information from the highly reliable information acquisition unit 112. In this example, the accuracy evaluation unit 123 is described as receiving all structured post data and filtering out the information required in a later process.

Actual event information is collected from high reliability information for the location (route/direction), time, and event (equipment abnormality) mentioned in the structured posted data. If there is no actual event (equipment abnormality) in the high reliability information, the planned time when the event (equipment abnormality) may be confirmed in the future is calculated, and if the planned confirmation time is in the future than the processing time, the presence or absence of the event (equipment abnormality) is considered unconfirmed.

The time when the presence or absence of an event (equipment abnormality) is determined may be, for example, the scheduled time for the next inspection of the location in question, or the scheduled time for data acquisition from a sensor that detects the abnormality. The accuracy of the presence or absence of the event (equipment abnormality) mentioned in the post is determined through comparison with the aggregated results, and an accuracy index is established. If there is no record of an event (equipment abnormality) and the presence or absence of an event (equipment abnormality) is undetermined, the presence or absence of the event (equipment abnormality) mentioned in the post is provisionally determined to be correct. The calculated index value of the accuracy of the post is sent to the usefulness index calculation unit 125.

The process flow of the accuracy evaluation part 123 in the second embodiment will be described with reference to FIG.
In step 2101, the process begins.
In step 2102, data is received. Structured post data is received from the post structuring unit 121.

Figure 22 shows an example of highly reliable information acquired from the highly reliable information acquisition unit 112. This example is maintenance data 2200 for maintaining and managing the equipment, and includes a data ID 2201, a data update date and time 2202, a station where the equipment is located 2203, an equipment name 2204, the presence or absence of an abnormality 2205, whether or not an action has been taken 2206, and a scheduled time of the next inspection 2207.

FIG. 23 shows an example of a specific post acquired from the post data acquisition unit 111. In FIG.
In step 2103, anomaly information is extracted. For each post, facility anomaly information is extracted from highly reliable information for the location (route/direction) and time mentioned in the structured post data.

In the case of the post in FIG. 23, the maintenance data 2200 is searched based on the named entities contained in the structured post data 2300, and information on facility inspections for the restrooms at the north exit of Station D is extracted. Specifically, the row (ID7) of the record for the restrooms at the north exit of Station D is extracted from the table in FIG. 22. Note that if a single facility cannot be identified from the mentions in the structured post data, the rows of records for multiple possibly applicable facilities are extracted.

In step 2104, the anomaly information is aggregated. For each post, the anomaly information extracted in the previous step 2103 is aggregated, and the presence or absence of an anomaly is calculated from Abnormality Presence 2205, whether the anomaly has been handled or not is calculated from Handled/Not Handled 2206, and the scheduled time for information confirmation is calculated from the next scheduled inspection time 2207.

The distinction between confirmed and unconfirmed information is made as follows: if the extracted anomaly information indicates that there is an anomaly in Abnormality Presence 2205 and there is equipment for which action has not been taken in Action Taken/Not Taken 2206, then there is an anomaly record (confirmed). Anything other than that is basically treated as unconfirmed, and the information is confirmed only at the time of inspection.

In the example post of Figure 23, the row of ID7 of the highly reliable information in Figure 22 is extracted. According to the highly reliable information, there is "no abnormality" in the north exit toilets of Station D as of 11:00, but the presence or absence of an abnormality will not be confirmed until 15:00, the next scheduled inspection time. Therefore, the highly reliable information is unconfirmed, and the information posted that there is an abnormality in the north exit toilets of Station D is provisionally determined to be correct. The provisional determination is updated at 15:00.

In the above method, the highly reliable information is deemed unconfirmed unless "Abnormality history (confirmed)" is indicated in Abnormality presence/absence 2205 and there is equipment that has not yet been addressed in Addressed/not yet 2206. As another method, if the extracted anomaly information indicates an abnormality and there is no equipment that has not yet been addressed, the highly reliable information is deemed "No abnormality history (confirmed)". This is based on the premise that a normal state will continue immediately after an abnormality is addressed. In all other cases, the presence/absence of an abnormality is deemed unconfirmed, and the scheduled confirmation time is set to the latest of the next scheduled inspection times. If any of the next scheduled inspection times is in the future than the time of this process by the accuracy evaluation unit, the information is deemed unconfirmed.

In step 2105, a determination is made as to whether the data is correct or not. For each post, the information on the anomaly mentioned in the structured post data is compared with the aggregated results from the previous step to determine whether the data is correct or not, and an index value for accuracy is determined.

The judgment table in FIG. 15 is used with "delay" replaced with "abnormality", and the same judgment as in Example 1 is made. In particular, if the information is unconfirmed and the post mentions an abnormality, the index value is provisionally set to "1" (positive).

In step 2106, the results are transmitted. The calculated index value of the accuracy of the post, the classification of the information as confirmed or unconfirmed, and the scheduled confirmation time if the information is unconfirmed are transmitted to the usefulness index calculation unit 125.

The process ends in step 2107.

Figure 22 is a table showing highly reliable information on equipment inspections within stations that is input by the accuracy evaluation unit 123. It includes the inspection of each piece of equipment, whether there are any abnormalities, whether the measures have been taken, and the scheduled time of the next inspection.

Figure 23 is a table showing an example of one post from the structured post data input by the accuracy evaluation unit 123.

In the processing of the accuracy evaluation unit 123 described above, by determining the uncertain period from the inspection records of highly reliable information and future inspection plans, it is possible to quickly extract information from SNS posts about abnormal information during the period when highly reliable information is uncertain.

The evaluation update unit 141 starts periodically (e.g., once a minute). The evaluation update unit 141 receives evaluated post data from the evaluated post storage unit 131, and receives high reliability information from the high reliability information acquisition unit 112. Of the evaluated posts, posts for which high reliability information has not been confirmed are extracted as update targets. In particular, posts for which high reliability information of equipment referenced in the accuracy evaluation of the post has been updated are targeted for update. An instruction to update (recalculate) the accuracy index for the post to be updated is sent to the accuracy evaluation unit 123. For posts for which high reliability information has not been confirmed and an accuracy index has been provisionally calculated, the accuracy of excluding erroneous information can be improved by updating the accuracy index based on confirmed high reliability information.

Regarding the equipment abnormality in Example 2, by quickly distributing posts regarding the abnormality information, employees/staff of the business operator managing the equipment can inspect the equipment promptly, i.e., earlier than the next scheduled inspection, and can speed up the response to the abnormality. When an employee/staff inspects the equipment and the information is reflected in the highly reliable information, the accuracy of excluding erroneous information can be improved by the processing of the evaluation update unit 141 described above.

Note that the present invention is not limited to the above-mentioned embodiment, and various modified examples are included. For example, the above-mentioned embodiment has been described in detail to explain the present invention in an easy-to-understand manner, and is not necessarily limited to those having all of the described configurations. In addition, it is possible to replace a part of the configuration of a certain embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of a certain embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. In addition, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware by designing them as integrated circuits, for example, in part or in whole. In addition, each of the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as programs, tables, files, etc. that realize each function can be placed in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

According to the embodiment described above, by evaluating posted information while taking into account uncertain matters in highly reliable information, the range of related information collected is expanded, making it easier to collect meaningful information. In addition, by evaluating the usefulness of posts in terms of level of detail, accuracy, and source of information, it is possible to quickly grasp detailed information while eliminating erroneous information.

Reference Signs List 100 Post evaluation system 101 Post evaluation device 122 Detail level evaluation unit 123 Accuracy evaluation unit 124 Information source evaluation unit 125 Usefulness index calculation unit

Claims (10)

a first acquisition unit that acquires first data including text data;
a second acquisition unit that acquires second data;
a structuring unit that extracts first information regarding the presence or absence of a predetermined phenomenon from the text data;
an accuracy evaluation unit that calculates an accuracy index of the first data by extracting second information on the presence or absence of the predetermined phenomenon and on whether the presence or absence of the phenomenon is confirmed or not confirmed from the second data, and determining that the first information that contradicts the second information is incorrect when the presence or absence of the predetermined phenomenon is confirmed based on the second information, and determining that the first information is correct when the presence or absence of the predetermined phenomenon is not confirmed based on the second information;
a usefulness index calculation unit that calculates an index of usefulness of the first data based on the accuracy index;
A data evaluation system equipped with: the accuracy evaluation unit calculates an expected determination time at which the presence or absence of the predetermined phenomenon will be determined based on the second information, and when the expected determination time has passed , updates the accuracy index of the first data that has been determined to be positive when the presence or absence of the predetermined phenomenon has not been determined based on the second information .
2. The data evaluation system according to claim 1. Further, a detail level evaluation unit is provided,
The structuring unit extracts, as the first information, a unique expression of a place, a time, and an event that describes the predetermined phenomenon;
The detailedness evaluation unit calculates a detailedness index based on the number of named entities;
The usefulness index calculation unit calculates an index of usefulness of the first data based on the accuracy index and the detail index.
2. The data evaluation system according to claim 1. It further comprises an information source evaluation unit,
The information source evaluation unit calculates an information source index based on evaluation information related to an information source of the first data,
The usefulness index calculation unit calculates an index of usefulness of the first data based on the accuracy evaluation unit and the information source index.
2. The data evaluation system according to claim 1. the first data is a post posted by an unspecified user,
The second data is management data including location, time and event information.
2. The data evaluation system according to claim 1. The management data is a transportation operation plan and a train location information,
The accuracy evaluation unit compares the content of the post with the management data regarding the presence or absence of a phenomenon of a delay in the operation of the transportation facility.
6. The data evaluation system according to claim 5. The management data is management information of a facility,
The accuracy evaluation unit compares the content of the post with the management data regarding the presence or absence of a phenomenon of an abnormality in the equipment.
6. The data evaluation system according to claim 5. A method using an information processing system including a first acquisition unit and a second acquisition unit, the information processing system processing information obtained from the first acquisition unit and the second acquisition unit,
The first acquisition unit acquires first information consisting of text information posted from a first information source;
The second acquisition unit acquires second information consisting of items and data for the items from a second information source;
the second information includes information describing the presence or absence of the phenomenon to be judged at a predetermined time and whether the presence or absence of the phenomenon is confirmed or not confirmed;
determining whether the first information is correct or not based on the presence or absence of the phenomenon to be judged interpreted based on the first information, the presence or absence of the phenomenon to be judged interpreted based on the second information, and whether the presence or absence of the phenomenon is confirmed or not ;
Calculating a usefulness index of the first information using the determined correctness of the first information;
When determining whether the first information is correct,
When the presence or absence of the phenomenon to be judged interpreted based on the first information coincides with the confirmed presence or absence of the phenomenon to be judged interpreted based on the second information, the first information is regarded as positive;
When the presence or absence of the phenomenon to be judged interpreted based on the first information does not match the confirmed presence or absence of the phenomenon to be judged interpreted based on the second information, the first information is judged to be erroneous;
When the presence or absence of the phenomenon to be judged based on the second information is undetermined, the first information is regarded as positive.
How to rate posts. The first acquisition unit acquires first information, which is a post including text information with a posting time input from a mobile terminal;
extracting named entities and normalizing the text information to extract information related to the presence or absence of the phenomenon to be determined;
The second acquisition unit acquires items stored in a database and data for the items as second information ,
The case where the presence or absence of the phenomenon to be judged based on the second information is undetermined means a case where the predetermined time is included within a predetermined time based on the posting time, and the phenomenon to be judged is not yet present at the predetermined time.
The post evaluation method according to claim 8. Calculating a scheduled update time for data stored in said database;
After the scheduled update time, the validity of the first information is determined again.
The post evaluation method according to claim 9.

Images