JP6998282B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method, and a program.

Conventionally, a crawler that collects data (documents, images, etc.) from the web and automatically creates a database of the collected data is known (see Patent Document 1). This crawler follows links in web pages and collects data from web pages with various IP addresses. The data collected by the crawler is stored in the web information database.

Japanese Unexamined Patent Publication No. 2012-69171

However, with the above-mentioned conventional technique, it may not be possible to efficiently acquire useful information.

The present invention has been made in consideration of such circumstances, and one of the objects of the present invention is to provide an information processing device, an information processing method, and a program capable of acquiring useful information more efficiently. do.

One aspect of the present invention refers to a knowledge database including an acquisition unit for acquiring information on a web page and information on relationships between a plurality of entities and the entities, and in the web page acquired by the acquisition unit, the knowledge is described. A recognition unit that recognizes a first expression pattern, which is an expression including a first main entity included in the database and a first dependent entity subordinate to the first main entity, and a recognition unit included in the knowledge database on the web page. , An extractor for extracting a second expression pattern that includes a second primary entity that is not associated with a second dependent entity of the same type as the first dependent entity to be associated and that matches the first expression pattern, and the knowledge. It is an information processing apparatus including a providing unit that provides information based on the second expression pattern to the knowledge database in order to expand the database.

According to one aspect of the present invention, useful information can be obtained more efficiently.

It is a figure which shows an example of the functional structure of an information processing system 1. It is a figure which shows a part of the knowledge database 42 schematically. It is a figure which shows an example of the knowledge panel which a knowledge database apparatus 30 provides to a terminal apparatus 10. It is a flowchart which shows an example of the flow of the process executed by the determination part 106 of the collection apparatus 100. It is a figure which shows an example of the information of the sampling web page selected in S10. It is a figure which shows an example of the combination of the entity included in the entity information 134. It is a figure which shows an example of the content of determination information 136. It is a flowchart which shows an example of the flow of the unknown information extraction processing executed by the collecting apparatus 100. FIG. 1 is a diagram (No. 1) for explaining a process in which the recognition unit 110 recognizes a description pattern. FIG. 2 is a diagram (No. 2) for explaining a process in which the recognition unit 110 recognizes a description pattern. It is a figure which shows an example of the outline of processing. It is a figure which shows an example of the contents of the knowledge database 42 before the update. It is a figure which shows an example of the contents of the knowledge database 42 after the update. It is a figure which shows an example of the knowledge panel NP1 generated based on the knowledge database 42 before update. It is a figure which shows an example of the knowledge panel generated based on the knowledge database 42 after the update. It is a figure which shows an example of the functional structure of the collection apparatus 100A of the information processing system 1A of 2nd Embodiment. It is a figure which shows an example of the content of the extraction information 140 with a reliability. It is a figure which shows an example of the tendency of the integrated score for each combination of known ratios.

Hereinafter, embodiments of the information processing apparatus, information processing method, and program of the present invention will be described with reference to the drawings.

[Overview (1)]
The information processing device is realized by one or more processors. The information processing apparatus of the embodiment acquires information on a web page, refers to a knowledge database including information on a plurality of entities and relationships between the entities, and is included in the knowledge database on the web page acquired by the acquisition unit. Recognize a first expression pattern that is an expression including one primary entity and a first dependent entity subordinate to the first primary entity. Then, on the web page, the information processing device includes a second primary entity that is included in the knowledge database and is not associated with a second dependent entity of the same type as the first dependent entity to be associated, and matches the first expression pattern. The second expression pattern to be processed is extracted, and the information based on the second expression pattern is provided to the knowledge database in order to expand the knowledge database. The "expression pattern" is, for example, a description pattern of a language used for generating a web page.

The knowledge database describes information about entities and information about semantic relationships between entities. An entity represents an entity or concept of an object. For example, when a query is entered, if the query corresponds to an entity, it is possible to return a wealth of information to the user rather than a simple keyword search.

The things described in the knowledge database are defined by the ontology. An ontology is a definition of a class and a property of an object, and is a collection of constraints that hold between the class and the property.

A class is information that indicates an attribute of an entity. A class is a group of things that have the same properties in an ontology. What the nature of an object is, that is, which class the object belongs to, is determined by the properties described below.

For example, things that have a beak, an oviparous vertebrate, and forelimbs that are wings are classified in the "bird" class. Also, in the class of "birds", things that have the property of not being able to fly are classified into lower classes such as "penguins" and "ostriches". In this way, the class system may have a hierarchical structure having a higher-lower relationship.

Properties are attributes that describe the nature and characteristics of things and the relationships between classes. For example, a property may be an attribute that has the property of "having ... as a component of the body" or the property of "living in", or "a class is a higher class and a certain class is a lower class". It may be an attribute indicating the relationship between the upper and lower levels of "class". As for the property name for identifying the property, the property name itself may or may not represent the meaning, as in the class name described above.

[Overview (Part 2)]
The information processing device acquires the information of the web page, refers to the knowledge database, and the knowledge database with respect to the web page based on the degree to which the main entity and the dependent entity subordinate to the main entity in the acquired web page are included. In, it is determined whether or not the fourth dependent entity of the same type as the dependent entity to be associated with the fourth main entity is to be extracted.

<First Embodiment>
[Constitution]
FIG. 1 is a diagram showing an example of a functional configuration of the information processing system 1. The information processing system 1 includes, for example, a terminal device 10, one or more hosts 20 (20-1 to 20-3 in the figure), a knowledge database device 30, a search device 50, and a collection device 100. The terminal device 10, the host 20, and the search device 50 communicate with each other via the network NW. Further, the knowledge database 30, the search device 50, and the collection device 100 communicate with each other via the network NW. The network NW includes, for example, WAN (Wide Area Network), LAN (Local Area Network), the Internet, a dedicated line, a wireless base station, a provider, and the like.

The terminal device 10 is a terminal device 10 used by the user. The terminal device 10 is a desktop type terminal device, a portable terminal device such as a notebook computer, a smartphone, a tablet type terminal device, or the like. The host 20 is a web server that provides a so-called web page.

The knowledge database device 30 is, for example, a server that generates a knowledge database 42 based on predetermined data (for example, image or text data) and provides a knowledge panel described later.

The knowledge database 42 is stored in the storage unit 40 of the knowledge database device 30. FIG. 2 is a diagram schematically showing a part of the knowledge database 42. As shown in FIG. 2, the entity includes the entity identification information (for example, "E1 to E7"), the entity name (for example, "A aquarium", etc.), the class (for example, "CL01"), and the entity (not shown). Is associated with information related to. In addition, a property is associated with the edge that indicates the relationship between the entities. In the example of FIG. 2, for example, the official website, the address, the business hours, and the like are associated with the property.

In the present embodiment, E2 to E7 in FIG. 2 are represented as an entity, but these information may be simply information associated with the entity E1.

FIG. 3 is a diagram showing an example of a knowledge panel provided by the knowledge database device 30 to the terminal device 10. For example, when a user operates a terminal device 10 to input a query in a search window such as a portal site and requests the search device 50 to search for information related to the query, the search device 50 refers to the information to be searched. , Search for information according to the query. Further, the search device 50 requests the knowledge database device 30 to provide a knowledge panel related to the query.

The knowledge database device 30 refers to the knowledge database 42, acquires information according to a query, generates a knowledge panel based on the acquired information, and provides the generated knowledge panel to the search device 50. The search device 50 generates the original data of the image including the search result and the knowledge panel, and provides the generated information to the terminal device 10. For example, when the query "A aquarium" is a search query, as shown in FIG. 3, an image including a list of web pages related to the A aquarium and the knowledge panel NP of the A aquarium is a display unit of the user's terminal device 10. Is displayed in.

In the following description, an entity such as Entity E1 "A Aquarium" which is the subject in the knowledge panel is referred to as "Main Entity", and is attached to information or the subject which supplements the subject such as Entity E2 to E7. The entity of the information (business hours, address, official website, etc.) may be referred to as a "subordinate entity".

Returning to the description of FIG. The knowledge database device 30 includes, for example, a communication unit 22, an information management unit 24, an information processing unit 26, and a storage unit 40. The communication unit 22 includes a communication interface such as a network interface card. The information management unit 24 provides the information generated by the knowledge database device 30 to another device, and manages the information provided by the other device. The information processing unit 26 generates a knowledge panel in response to a request from the search device 50, and updates the knowledge database 42 using the information provided by the collection device 100.

[Collection device]
The collection device 100 includes, for example, a communication unit 102, a collection unit 104, a determination unit 106, a target information acquisition unit 108, a recognition unit 110, an extraction unit 112, a specific unit 114, a provision unit 116, and storage. A unit 130 is provided. In the collection unit 104, the determination unit 106, the target information acquisition unit 108, the recognition unit 110, the extraction unit 112, the specific unit 114, and the provision unit 116, a hardware processor such as a CPU (Central Processing Unit) is stored in a storage device. It is realized by executing the program. Further, these functional units may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. It may be realized by the collaboration of software and hardware. Further, the above program may be stored in a storage device in advance, or is stored in a removable storage medium such as a DVD or a CD-ROM, and the storage medium is attached to the drive device of the collection device 100. It may be installed in the storage device.

The storage unit 130 is realized by, for example, a ROM (Read Only Memory), a flash memory, an SD card, a RAM (Random Access Memory), an HDD (Hard Disc Drive), a register, or the like. Further, a part or all of the storage unit 130 may be NAS (Network Attached Storage), an external storage server device, or the like. For example, the collection information 132, the entity information 134, the determination information 136, and the extraction information 138 are stored in the storage unit 130. Details of this information will be described later.

The communication unit 102 communicates with the host 20, the knowledge database device 30, or the search device 50 via the network NW. The communication unit 102 includes, for example, a communication interface such as a NIC (Network Interface Card).

The collecting unit 104 collects information from the host 20 according to a predetermined protocol, and stores the collected information in the storage unit 130 as the collected information 132. Further, for example, the collection unit 104 collects a small amount of pages (hereinafter referred to as sampling web pages) from each host 20, and stores the collected sampling web pages in the storage unit 130 as collection information 132.

The decision unit 106 refers to the knowledge database 42, and based on the degree to which the main entity and the dependent entity subordinate to the main entity are included in the web page acquired by the collection unit 104 (for example, a sampling web page), the web page is concerned. It is determined whether or not the host 20 of the provider of the above is targeted for preferentially collecting uncollected web pages other than the sampling web page.

The target information acquisition unit 108 acquires web page information from the collected information 132.

The recognition unit 110 refers to the knowledge database 42 including a plurality of entities and relationship information between the entities, and in the web page acquired by the target information acquisition unit 108, the first main entity included in the knowledge database 42 and the first main entity. Recognize a first expression pattern that is an expression including a first dependent entity subordinate to one primary entity.

On the web page, the extraction unit 112 includes a second primary entity that is included in the knowledge database 42 and is not associated with a second dependent entity of the same type as the first dependent entity to be associated, and matches the first expression pattern. Extract the second expression pattern.

The identification unit 114 identifies the second dependent entity in the second expression pattern extracted by the extraction unit 112 based on the relative relationship between the first main entity and the first dependent entity in the first expression pattern.

The providing unit 116 provides the knowledge database device 30 with information based on the second expression pattern in order to expand the knowledge database 42.

Hereinafter, the process for expanding the information provided by the knowledge panel will be described.

[Flowchart (1)]
FIG. 4 is a flowchart showing an example of the flow of processing executed by the determination unit 106 of the collection device 100. The details of the processing will be described with reference to FIGS. 5 to 7 described later.

First, the determination unit 106 selects one or more sampling web pages provided by the same host from the collected information 132 (S10). Next, the determination unit 106 refers to the entity information 134 and selects one combination of the entity and the combination of the entity associated with the property (S12). The entity information 134 is the same information as the knowledge database 42, or a part of the information of the knowledge database 42 as shown in FIG. 2 described above.

Next, the determination unit 106 determines whether or not one selected combination is included in the selected sampling web page (S14). Next, the determination unit 106 stores the determination result as the determination information 136 in the storage unit 130 (S16).

Next, the determination unit 106 determines whether or not all the combinations of the entities and the entities have been selected (S18). If all combinations have not been selected, the process returns to step S12.

When all combinations are selected, the determination unit 106 determines whether or not the information on all sampling web pages has been selected (S20). If the information on all sampling web pages has not been selected, the process returns to step S10.

When the information of all the sampling web pages is selected, the determination unit 106 determines the host 20 to be deep-drilled based on the determination information 136 which is the determination result (S22). Next, the collecting unit 104 collects web pages (web pages other than the sampling web page) that have not been collected at the determined host 20 that is the target of deep digging, and collects the collected information as the collected information 132 in the storage unit 130. Remember (S24). That is, the collection unit 104 performs a deep digging process for deep digging a useful host (host to be deep digged) determined from a small amount of collection results. This ends the processing of this flowchart.

The collecting unit 104 collects information possessed by the host 20 and not collected from the host 20 that was not targeted for deep digging at a predetermined timing, and the collected information is used as the collected information 132 in the storage unit 130. To memorize.

Further, in the above-mentioned flowchart example, the determination unit 106 selects one combination of the entities associated with the property and the combination of the entities (S12), and the selected combination is the selected sampling web page. Although it is determined whether or not it is included in the above, instead of this, the processing may be performed as follows.
(1) The determination unit 106 lists the entities to be extracted (for example, the museum C in FIGS. 5 and 6 described later).
(2) The determination unit 106 determines whether or not the sampling web page contains the entity to be extracted.
(3) When the entity to be extracted is included, the determination unit 106 extracts the entity included in the web page (for example, A museum and B museum in FIGS. 5 and 6) included in the knowledge database 42. List the entities (for example, the official websites of Museum A and Museum B in FIGS. 5 and 6) associated with the target property (for example, the official websites of FIGS. 5 and 6).
(4) The determination unit 106 performs a process of determining whether or not the entity associated with the property to be extracted is included in the web page. Then, the determination unit 106 determines whether or not the host 20 of the provider of the web page is targeted for deep digging based on the determination result.

FIG. 5 is a diagram showing an example of information on the sampling web page selected in S10. For example, it is assumed that the name of the tourist spot and the URL of the tourist spot are included in the sampling web page. For example, the sampling web page includes "A Museum", "URL001", "B Museum", "URL002", "C Museum", and "URL003".

FIG. 6 is a diagram showing an example of a combination of entities included in the entity information 134. For example, "A museum" and "URL001" are associated with the property "official site", and "B museum" and "URL002" are associated with the property "official site". The entity information 134 includes the entity "C Museum", but the "C Museum" is not associated with "URL003". When the facility name "A Museum (1st main entity)" is associated with "URL001 (1st subordinate entity)" based on the relationship of facility URL (property), the relationship of facility URL (property) The name "C Museum" of the facility to which "URL002 (second subordinate entity)" is not associated is an example of "second main entity".

FIG. 7 is a diagram showing an example of the contents of the determination information 136. The determination information 136 is information in which the combination of entities, the score, and the information indicating the determination result of whether or not to be the target of deep digging are associated with each other with respect to the host ID. In the processing of S12 to S18 of the flowchart of FIG. 3 described above, the combination of "A museum" and "URL001" and the combination of "B museum" and "URL002" are included in the information of the selected sampling web page. It is determined that it is. For example, when the combination of the two is included in the information on the sampling web page as described above, the determination unit 106 determines the score as “2”. For example, the determination unit 106 determines the host 20 that provided the sampling web page with the score "2" or more as the host to be deep-drilled.

As described above, the host 20 to be deep-drilled is determined, and the deep-drilling process is preferentially performed for the host 20 to be deep-drilled. As a result, the information possessed by the useful host 20 is preferentially collected.

[Flowchart (Part 2)]
FIG. 8 is a flowchart showing an example of the flow of the unknown information extraction process executed by the collection device 100. This flowchart is a process of specifying an entity to be associated with a predetermined property for a specific entity. The specific entity is an entity (second main entity) to which the entity to be associated (second dependent entity) is not associated. In the above example, Museum C corresponds to a specific entity. This is because the entity "URL ***" to be associated with the C museum is not associated. The details of the processing will be described with reference to FIGS. 9 to 11 described later.

First, the target information acquisition unit 108 of the collection device 100 acquires the web page included in the collection information 132 (S100).

Next, the recognition unit 110 recognizes the first description pattern (first expression pattern) including the entity associated with the property and the combination of the entity in the acquired web page (hereinafter referred to as the target web page) (S102). .. A combination of an entity and an entity is, for example, a combination of an entity of the same class as the particular entity (eg, a facility) and an entity having a class of entities to be associated with the particular entity (eg, the URL of the facility).

Next, the recognition unit 110 identifies the relative position of the combination of the entities based on the recognized first description pattern (S104). Next, the recognition unit 110 extracts a second description pattern (second expression pattern) including the specific entity and matching the recognized first description pattern from the target web page acquired in S100 (S106).

Next, the specifying unit 114 specifies a relative position corresponding to the relative position of the first description pattern specified in S104 in the second description pattern (S108). Next, the specific unit 114 extracts the information associated with the position (second position) different from the position (first position) in which the specific entity is described from the information associated with the specified relative position. The extracted information is stored in the storage unit 130 as the extracted information 138 (S110). The extracted information 138 is information in which the specific entity and the entity associated with the specific entity extracted by this process by a predetermined property are associated with each other.

Next, the recognition unit 110 determines whether or not the information of all the web pages to be processed has been selected (S112). If the information of all the web pages to be processed is not selected, the process returns to S100. When all the web page information to be processed is selected, the providing unit 116 transmits the extracted information 138 to the knowledge database device 30 (S114). This ends the processing of one routine in this flowchart.

The web page to be processed may be all the web pages included in the collected information 132 as described above, or may be a set web page. Further, the web page to be processed may be a web page acquired from the host 20 to be deep-drilled. Further, the decision unit 106 refers to the knowledge database 42, and the web page (or the host 20) is based on the degree to which the main entity and the dependent entity subordinate to the main entity are included in the web page acquired by the collection unit 104. May be determined whether or not to be the processing target of the extraction unit 112.

FIG. 9 is a diagram (No. 1) for explaining the process of recognizing the description pattern by the recognition unit 110. FIG. 10 is a diagram (No. 2) for explaining the process of recognizing the description pattern by the recognition unit 110. For example, as shown in FIG. 9, the recognition unit 110 recognizes a source code such as HTML (Hyper Text Markup Language) of a target web page. Then, as shown in FIG. 10, the recognition unit 110 recognizes the description pattern A of the source code including the combination of the entities included in the knowledge database 42.

In the illustrated example, the source code is arranged in the order of "dt", "span", "dd", "a", and the entity "A museum" is associated after "span", and the entity is associated with "a". A "URL" is associated. The entity "A museum" and the entity "URL001" are a combination of the entities associated in the knowledge database 42. The same applies to the entity "B Museum".

In this case, the recognition unit 110 recognizes that the entity "facility name" is associated after "span" and the entity "URL of the facility name" is associated with "a". As a result, the relative position of the combination of entities in the description pattern is specified. The position to which the "facility name" is given is an example of the "first position", and the position to which the "URL of the facility name" is given is an example of the "second position".

The recognition unit 110 extracts a description pattern that matches the description pattern A as described above. The matching description pattern is a pattern in which the source code is arranged in the order of "dt", "span", "dd", and "a", and the entity of the facility included in the knowledge database 42 is associated after "span". Is. For example, the recognition unit 110 recognizes the description pattern A in which the entity “C museum” is associated after the “span”. Then, the specifying unit 114 identifies that the entity "URL003 of the C museum" is associated with "a" based on the relative position of the combination of the entities.

The above processes can be summarized as shown in FIG. The collecting device 100 recognizes the description patterns "dt", "span", "dd", and "a", the entity "facility name" is associated after "span", and the entity "URL of the facility name" is associated with "a". Recognize that is associated. Then, the collecting device 100 recognizes in the knowledge database 42 that the URL of the facility name "C museum", which is an entity to which the entity "URL" is not associated, is associated with "a" of the description pattern A.

In this way, the collecting device 100 can specify the information associated with the specific entity, which is unknown information, based on the description pattern which is the hierarchical structure of the language including the combination of the entities. In other words, the specific unit 114 is a specific path (“dt” → “span”) in the hierarchical structure from a predetermined position (for example, the first main entity or “dt”) in the first expression pattern to the first dependent entity. → “dd” → “a”) can be specified, and the second dependent entity can be specified by following a specific route in the second expression pattern.

FIG. 12 is a diagram showing an example of the contents of the knowledge database 42 before the update. In the knowledge database 42, the URL of "C Museum" is not associated with the entity "C Museum".

FIG. 13 is a diagram showing an example of the contents of the knowledge database 42 after the update. When the knowledge database device 30 acquires the "URL" of the "C museum" from the collection device 100, the knowledge database device 30 associates the entity "C museum" with the URL transmitted from the collection device 100.

FIG. 14 is a diagram showing an example of the knowledge panel NP1 generated based on the knowledge database 42 before the update. When the user inputs the search query "C Museum", the knowledge database device 30 cannot include the URL in the knowledge panel because the URL is not associated with the entity "C Museum".

On the other hand, when the knowledge panel is generated based on the updated knowledge database 42, the knowledge database device 30 has a URL associated with the entity "C museum" as shown in FIG. , The URL can be included in the knowledge panel NP2.

In this way, the knowledge database device 30 can provide more useful information to the user by using the updated knowledge database 42.

In the above example, the processing is described as being performed based on the expression pattern of the source code, but instead (or in addition), it is associated with a specific entity based on the pattern of the image. The information to be used may be specified. For example, the specifying unit 114 may specify the URL of the facility to which the URL information is not associated in the knowledge database 42 based on the position where the facility name is displayed and the position where the URL is displayed in the image.

According to the first embodiment described above, the collection device 100 is an expression including the first main entity included in the knowledge database 42 and the first dependent entity subordinate to the first main entity in the target web page. Recognizing the first expression pattern, the target web page contains a second primary entity that is included in the knowledge database 42 and is not associated with a second dependent entity of the same type as the first dependent entity to be associated, and the first representation. By providing the information based on the second expression pattern matching the pattern to the knowledge database device 30 in order to expand the knowledge database 42, useful information can be acquired more efficiently.

<Second Embodiment>
Hereinafter, the second embodiment will be described. In the second embodiment, the collection device 100A derives the reliability for the extracted entity, and provides the knowledge database device 30 with the entity whose derived reliability is equal to or higher than the threshold value. Hereinafter, the differences from the first embodiment will be described.

FIG. 16 is a diagram showing an example of the functional configuration of the collection device 100A of the information processing system 1A of the second embodiment. The collecting device 100A includes a reliability derivation unit 115 in addition to the functional configuration of the collecting device 100. Further, the collecting device 100A includes a storage unit 130A instead of the storage unit 130. In the storage unit 130A, in addition to the information stored in the storage unit 130, the extraction information 140 with reliability is further stored.

For example, when the same fact is obtained from a plurality of target web pages, the reliability derivation unit 115 derives the reliability of the foct based on the information of the plurality of target web pages. The fact is the fact that it contains a combination of entities in the knowledge database 42. For example, the reliability derivation unit 115 derives the integration score, which is the reliability, based on the ratio of the combination of known entities on the web page, and the derived integration score and the extraction information 138 are combined to extract the information with reliability. Generate 140. Then, the reliability derivation unit 115 determines to provide the knowledge database device 30 with a combination of entities whose integration score is equal to or higher than the threshold value.

FIG. 17 is a diagram showing an example of the contents of the extraction information 140 with reliability. In the extraction information 140 with reliability, the combination of the entities included in the target web page, the information indicating whether the combination is known or unknown in the knowledge database 42, and the combination of the entities are known in the target web page. Percentages and integration scores are information that is associated with each other. For example, the reliability derivation unit 115 determines whether the combination of entities included in the target web page is known or unknown in the knowledge database 42, and based on the determination result, the combination of known entities with respect to the combination of entities. Derivation of the ratio of.

Then, the reliability derivation unit 115 applies a known ratio derived for each target web page to a predetermined model to derive an integrated score. The predetermined model is, for example, the equation (1). "X" in the formula (1) is a known ratio in the first target web page collected from the host 20 of the host ID "001", and "y" is collected from the host 20 of the host ID "002". It is a known ratio in the second target web page. “Α” is an arbitrarily set parameter (for example, “0.1”).

FIG. 18 is a diagram showing an example of the tendency of the integrated score for each combination of known ratios. As shown in FIG. 18, in equation (1), when the known proportions of both "x" and "y" are high, the integration score tends to be high, and both "x" and "y" are known. If the percentage is low, the integration score is a function that tends to be high.

In this way, the reliability deriving unit 115 can derive the integrated score more accurately by deriving the integrated score based on the known ratio between the first web page and the second web page.

According to the second embodiment described above, the collecting device 100 expresses in the first web page so that the main entity and the dependent entities subordinate to the main entity, which are included in the knowledge database 42, have a specific relative relationship. The ratio of the expressed expression pattern to the expression pattern in which the main entity contained in the knowledge database 42 and the non-dependent entity that is not dependent on the main entity in the knowledge database 42 are expressed so as to have a specific relative relationship, and the expression pattern. , In the second web page, the expression pattern in which the main entity and the dependent entities subordinate to the main entity in the knowledge database 42 have a specific relative relationship, and the main entity included in the knowledge database and The non-dependent entity is used as information for expanding the knowledge database 42 based on the ratio of the non-dependent entity that is not dependent on the main entity in the knowledge database to the expression pattern expressed so as to have a specific relative relationship. By determining whether or not it is, the information for expanding the knowledge database can be sorted more accurately.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 Information processing system, 10 Terminal device, 20 Host, 30 Knowledge database device, 42 Knowledge database, 100, 100A Collection device, 102 Communication unit, 104 Collection unit, 106 Decision unit, 108 ... Target information acquisition unit, 110 recognition unit, 112 extraction unit, 114 specific unit, 115 reliability derivation unit, 116 provision unit, 130, 130A storage unit, 134 entity information, 136 judgment information, 138 ‥ Extraction information, 140 ‥ Extraction information with reliability

Claims (12)

The acquisition department that acquires information on the web page,
A knowledge database containing a plurality of entities and relationship information between the entities is referred to, and in the web page acquired by the acquisition unit, the first main entity included in the knowledge database and subordinate to the first main entity. A recognition unit that recognizes the first expression pattern, which is an expression including the first dependent entity,
In the web page, the second dependent entity that is the second primary entity included in the knowledge database and is the same kind of second dependent entity as the first dependent entity and should be associated with the second primary entity is associated with it. An extraction unit that includes the second main entity and extracts a second expression pattern similar to the first expression pattern in which the first main entity is replaced with the second main entity .
A providing unit that provides information based on the second expression pattern to the knowledge database in order to expand the knowledge database.
Information processing device equipped with. The recognition unit recognizes the first expression pattern on one or more web pages provided by the same host.
The information processing apparatus according to claim 1. The first expression pattern and the second expression pattern are description patterns of the language used for generating a web page.
The information processing apparatus according to claim 1 or 2. In the second expression pattern extracted by the extraction unit based on the relative relationship between the first main entity and the first dependent entity in the first expression pattern, the specific unit that specifies the second dependent entity is designated. Further prepare
The information processing device according to any one of claims 1 to 3. The first description pattern of the language used to generate the web page, which is the first expression pattern, and the second description pattern of the language, which is the second expression pattern, are similar.
The specific part is the first position of the first main entity in the first description pattern, the second position of the first dependent entity in the first description pattern, and the second main entity in the second description pattern. Based on the first position, the second position in the second description pattern is specified, and the information described in the second position is specified as the second dependent entity.
The information processing apparatus according to claim 4. The description pattern is a hierarchical structure of the language.
The information processing device according to claim 3. The specific route in the hierarchical structure up to the first dependent entity in the first expression pattern is specified, and in the second expression pattern extracted by the extraction unit, the specific route is traced from the second main entity to the said. It has a specific part that identifies the second dependent entity,
The information processing apparatus according to claim 6. The recognition unit recognizes a third expression pattern, which is an expression including a third main entity included in the knowledge database and a third dependent entity subordinate to the third main entity, on the web page.
In the web page, the extraction unit is the second main entity included in the knowledge database, the second subordinate entity of the same type as the first subordinate entity and the third subordinate entity, and the second main entity. The first main entity and the third main entity among the first expression pattern and the third expression pattern include the second main entity to which the second subordinate entity to be associated with is not attached, and the second is the second . Extract the second expression pattern similar to the one replaced by the main entity ,
The extraction is based on the relative relationship between the first main entity and the first dependent entity in the first expression pattern, and the relative relationship between the third main entity and the third dependent entity in the third expression pattern. In the second expression pattern extracted by the unit, a specific unit that identifies the second dependent entity is further provided.
The information processing apparatus according to any one of claims 1 to 7. The acquisition unit acquires the first web page and the second web page, and obtains them.
In the first web page, the main entity included in the knowledge database and the dependent entity subordinate to the main entity in the knowledge database are expressed so that the main entity and the subordinate entity have a specific relative relationship. The ratio of the expressed expression pattern to the expression pattern in which the main entity included in the knowledge database and the non-dependent entity that is not dependent on the main entity in the knowledge database are expressed so as to have a specific relative relationship. and,
In the second web page, the main entity included in the knowledge database and the dependent entity subordinate to the main entity in the knowledge database are expressed so that the main entity and the dependent entity have a specific relative relationship. Based on the ratio of the expression pattern that is expressed so that the main entity included in the knowledge database and the non-dependent entity that is not dependent on the main entity in the knowledge database have a specific relative relationship. Further, a determination unit for determining whether or not the non-dependent entity is used as information for expanding the knowledge database is further provided.
The information processing apparatus according to any one of claims 1 to 8. Whether to make the web page a processing target of the extraction unit based on the degree to which the main entity and the dependent entity subordinate to the main entity are included in the web page acquired by the acquisition unit with reference to the knowledge database. Further equipped with a decision unit for deciding whether or not to do so,
The information processing apparatus according to any one of claims 1 to 9. The computer
Get the information on the web page and
Refer to the knowledge database containing multiple entities and relationship information between the entities.
In the acquired web page, the first expression pattern, which is an expression including the first main entity included in the knowledge database and the first dependent entity subordinate to the first main entity, is recognized.
In the web page, the second dependent entity that is the second primary entity included in the knowledge database and is the same kind of second dependent entity as the first dependent entity and should be associated with the second primary entity is associated with it. A second expression pattern that includes the second main entity and is similar to the first expression pattern in which the first main entity is replaced with the second main entity is extracted.
Information based on the second expression pattern is provided to the knowledge database in order to expand the knowledge database.
Information processing method. On the computer
Get the information on the web page,
Refer to a knowledge database containing multiple entities and relationship information between the entities.
In the acquired web page, the first expression pattern, which is an expression including the first main entity included in the knowledge database and the first dependent entity subordinate to the first main entity, is recognized.
In the web page, the second dependent entity that is the second primary entity included in the knowledge database and is the same kind of second dependent entity as the first dependent entity and should be associated with the second primary entity is associated with it. A second expression pattern that includes the second main entity and is similar to the first expression pattern in which the first main entity is replaced with the second main entity is extracted.
In order to expand the knowledge database, the knowledge database is provided with information based on the second expression pattern.
program.

Images