JP5153516B2 - Extraction apparatus, extraction system, extraction method and program - Google Patents

Description

  The present invention relates to an extraction device, an extraction system, an extraction method, and a program. In particular, the present invention relates to an extraction device, an extraction system, an extraction method, and a program for extracting a specific portion from an HTML document generated by a Web application.

  In recent years, the need for Web services has increased. Web services are service components that are also used as parts to realize various services, and have been used mainly in the field of B2B. In addition, in recent years, there is a movement to use these also in the ubiquitous field, and Web services are being actively used in B2C and C2C. However, at this stage, the number of Web services that can be used is small, and various services have not been provided. Therefore, it is necessary to create an environment that can realize various services by increasing the number of Web services and combining them.

  There is an idea of using an existing web application as a web service. As a result, a wealth of Web services can be used. There are wrappers that use this. The wrapper has a web service interface and a web application interface. The wrapper converts the request that came to the web service side of the wrapper into a web application request and sends it to the web application. After that, a specific part is extracted from the returned HTML document, converted into SOAP (Simple Object Access Protocol), and returned to the request sender. At this time, a configuration file for protocol conversion is required for each Web application. For example, when implementing a wrapper for making a hotel search Web application a hotel search Web service, the configuration file for the hotel search Web application Necessary.

The configuration file mainly includes rules for converting SOAP requests to Web application requests, and rules for extracting specific parts from HTML documents returned by Web applications and converting the extraction results to SOAP. It is. There are Patent Documents 1 and 2 as technologies for mutual conversion between SOAP and Web application protocols using such rules.
JP 2007-241417 A JP 2008-003848 A

  However, if the HTML document returned by the Web application changes, the correct part cannot be extracted from the preconfigured config file, and the Web application cannot be used as a Web service. In order to solve this problem, it is necessary to rework the config file, which requires a lot of labor.

  When the HTML document returned by the Web application changes, the purpose is to easily extract the correct part so that the Web application can be used as a Web service.

In order to solve the above-described problem, one aspect of the present invention holds content when the extraction request is received from a request transmission / result receiving apparatus that transmits an extraction request for requesting extraction of a specific portion in the content. By transmitting a content request to the content providing apparatus , the content related to the extraction request is received from the content providing apparatus , a specific part of the content is extracted, and the extracted result is transmitted to the request transmission / result receiving apparatus. An extraction device, a rule holding mechanism for holding a rule for responding to the extraction request, and receiving the extraction request from the request transmission / result reception device, and converting the extraction request into the content request according to the rule , Transmitting the content request to the content providing apparatus, and From content received from the extraction mechanism by rules for extracting a specific portion in accordance with the rules, when the extraction of the specific portion by extracting mechanism according to the rule fails, and a rule generation mechanism that generates a new piece of the rules, The extraction mechanism based on the rule has failed to extract a specific part based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and a template of the predetermined extraction result based on the rule. And the rule generation mechanism has a maximum similarity to a template of a predetermined extraction result by the rule among a plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus. Select one extraction result candidate as the optimum extraction result and extract the optimum extraction result. It was generated as a new said rule extraction mechanism by the rule, if the rule generation mechanism to generate a new said rules, characterized by re-extracting the specific portion according to the new the rules.

In order to solve the above-described problem, another aspect of the present invention is that content is received when the extraction request is received from a request transmission / result reception apparatus that transmits an extraction request for requesting extraction of a specific portion in the content. By transmitting the content request to the content providing apparatus that holds the content, receiving the content related to the extraction request from the content providing apparatus , extracting a specific portion of the content, and sending the extracted result to the request transmission / result receiving apparatus. An extraction device for transmitting, a rule holding mechanism for holding a rule for responding to the extraction request, and receiving the extraction request from the request transmission / result receiving device, and converting the extraction request into the content request according to the rule The content request is transmitted to the content providing apparatus, and the content provision is performed. An extraction mechanism based on a rule for extracting a specific part from content received from a device according to the rule; and a rule generation mechanism for generating a new rule when the extraction of the specific part by the extraction mechanism based on the rule fails. When the rule generation mechanism generates a new rule, the rule extraction mechanism re-extracts a specific part according to the new rule , and the rule generation mechanism reads the rule from the rule holding mechanism, A rule holding unit that distributes the content, a content receiving unit that receives the content from the extraction mechanism based on the rule, and a specific part automatic extracting unit that automatically extracts extraction result candidates of the specific part of the content received by the content receiving unit And reading a rule from the rule holding unit, By calculating the similarity between the template of the output result and each of a plurality of extraction result candidates automatically extracted by the specific part automatic extraction unit, the extraction result candidate having the maximum similarity is selected as the optimum extraction result. An extraction result selection unit to select as, a rule generation unit to generate a new rule for extracting the optimum extraction result selected by the extraction result selection unit, and a new generation generated by the rule generation unit It has a rule update part which updates the said rule by transmitting the said rule to the extraction mechanism by the said rule, and a rule holding | maintenance mechanism, It is characterized by the above-mentioned .

In order to solve the above-described problem, another aspect of the present invention is that content is received when the extraction request is received from a request transmission / result reception apparatus that transmits an extraction request for requesting extraction of a specific portion in the content. By transmitting the content request to the content providing apparatus that holds the content, receiving the content related to the extraction request from the content providing apparatus , extracting a specific portion of the content, and sending the extracted result to the request transmission / result receiving apparatus. in the extraction apparatus to transmit, the rule holding mechanism for holding the rules for responding to the retrieval request, receiving the extraction request from the request transmission-result receiving device, converts the extracted request to the content request in accordance with the rules The content request is transmitted to the content providing apparatus, and the content provision is performed. An extraction mechanism based on a rule for extracting a specific part from content received from a device according to the rule; and a rule generation mechanism for generating a new rule when the extraction of the specific part by the extraction mechanism based on the rule fails. The rule extraction mechanism includes a rule holding unit that acquires the rule from the rule holding mechanism, updates and distributes the rule, and a request reception unit that receives the extraction request from the request transmission / result reception device. A request assembly unit that converts the extraction request received by the request reception unit into the content request according to the rule acquired by the rule holding unit, and the content request converted by the request assembly unit. A request transmitter for transmitting to the content providing device; A content holding unit that receives, holds, and distributes content corresponding to the content request from the content providing device, and extracts a specific part from the content received by the content holding unit according to the rules acquired by the rule holding unit And an extraction failure detection unit that determines whether or not extraction of a specific part by the extraction unit based on the rule has failed, and that the extraction failure detection unit determines that the extraction of the specific part has not failed. The extraction result by the extraction unit based on the rule is returned to the request transmission / result reception device, and the extraction failure detection unit determines that the extraction of the specific part has failed The content is acquired from the content holding unit and transmitted to the rule generation mechanism. A content transmission unit, and when the rule generation mechanism generates a new rule based on the content transmitted by the content transmission unit, the rule extraction unit extracts a specific part according to the new rule. An extraction device for re-extraction , wherein the extraction unit according to the rule transmits the extraction result to the extraction failure detection unit when a specific part is extracted, and the extraction failure detection unit detects that extraction has succeeded When the result is received, the extraction result is transmitted to the extraction result transmission unit, and when the detection result indicating that re-extraction is necessary is received from the extraction failure detection unit, the specific part is determined according to the new rule. When the extraction failure detection unit receives the extraction result from the extraction unit based on the rule, the extraction failure detection unit acquires a rule from the rule holding unit, and By calculating the similarity between the template of the predetermined extraction result by the above and the extraction result, when the similarity is equal to or greater than a threshold, it is determined that the extraction of the specific part has not failed, and the extraction has succeeded The result of detection to the effect is returned to the extraction unit by the rule, and when the similarity is less than the threshold, it is determined that the extraction of the specific part has failed, and the content transmission unit is regenerated to regenerate the rule. The content transmission request is transmitted, and a detection result indicating that re-extraction is necessary is transmitted to the extraction unit based on the rule, and the content transmission unit receives the content transmission request from the content holding unit when the content transmission request is received. Content is acquired and transmitted to the rule generation mechanism.

In order to solve the above-described problem, another aspect of the present invention is an extraction system that extracts a specific portion of content, and transmits content held in response to a content request for requesting content. A request transmission / result reception device for transmitting an extraction request for requesting extraction of a specific part in the content and receiving an extraction result for the extraction request; and receiving the extraction request from the request transmission / result reception device. The content request for requesting the content related to the extraction request is transmitted to the content providing apparatus, the specific part of the content received from the content providing apparatus is extracted, and the request is transmitted to the request transmission / result receiving apparatus as an extraction result. And the extraction device holds a rule for responding to the extraction request. And receiving the extraction request from the request holding / result receiving device, converting the extraction request into the content request according to the rule, and transmitting the content request to the content providing device. An extraction mechanism based on a rule that extracts a specific part from content received from the rule, and a rule generation mechanism that generates a new rule when the extraction of the specific part by the extraction mechanism based on the rule fails. The extraction mechanism based on the rule fails to extract a specific part based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and a template of the predetermined extraction result based on the rule. The rule generation mechanism determines whether or not the content providing device Among the plurality of extraction result candidates automatically extracted from the content received from the above, the extraction result candidate having the maximum similarity to the template of the predetermined extraction result by the rule is selected as the optimum extraction result, and the optimum extraction result is selected. A rule for extracting an extraction result is generated as a new rule, and the extraction mechanism based on the rule re-extracts a specific part according to the new rule when the rule generation mechanism generates a new rule. It is characterized by.

In order to solve the above-described problem, another aspect of the present invention is to provide a request transmission / result reception for holding a rule for responding to an extraction request for requesting extraction of a specific portion in content and transmitting the extraction request. when the device has received the extraction request, by sending a content request to the content providing device holding the content, it receives the content related to the retrieval request from the content providing device, and extracts a specific part of the content An extraction method for extracting a specific part in the content by the extraction device that transmits to the request transmission / result reception device as an extraction result, wherein the extraction request is received from the request transmission / result reception device, and the extraction request is received The content request is converted into the content request according to the rule, and the content request is converted into the content providing device. An extraction step based on a rule for extracting a specific part according to the rule from content transmitted and received from the content providing apparatus, and a new rule is generated when extraction of the specific part by the extraction step based on the rule fails A rule generation step, wherein the extraction step based on the rule is based on a similarity between the extraction result extracted according to the rule from the content received from the content providing device and a template of a predetermined extraction result based on the rule. Determining whether or not the extraction of the specific part has failed, and the rule generation step includes a step of extracting a predetermined extraction result by the rule from among a plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus. The extraction result candidate with the maximum degree of similarity with the template is selected as the optimal extraction result. Select Te, a rule for extracting the best extraction results generated as a new said rule extraction step by the rule, if the rule generating step has generated a new said rules, according to the new the rules A specific part is re-extracted.

In order to solve the above-described problem, another aspect of the present invention is to provide a request transmission / result reception for holding a rule for responding to an extraction request for requesting extraction of a specific portion in content and transmitting the extraction request. when the device has received the extraction request, by sending a content request to the content providing device holding the content, it receives the content related to the retrieval request from the content providing device, and extracts a specific part of the content , Receiving the extraction request from the request transmission / result reception device to the computer of the extraction device that transmits to the request transmission / result reception device as an extraction result, converting the extraction request into the content request according to the rule, A content request is transmitted to the content providing apparatus, and the content providing apparatus A program for executing an extraction step based on a rule for extracting a specific part from a received content according to the rule, and a rule generation step for generating a new rule when the extraction of the specific part by the extraction step based on the rule fails In the extraction step based on the rule, based on the similarity between the extraction result extracted from the content received from the content providing apparatus according to the rule and a template of the predetermined extraction result based on the rule, It is determined whether or not extraction has failed, and the rule generation step is similar to a template of a predetermined extraction result by the rule among a plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus. Select the extraction result candidate with the highest degree as the optimal extraction result, The rule for extracting the best extraction results generated as a new said rule extraction step by the rule, if the rule generating step has generated a new said rule, a specific portion according to the new the rules again It is characterized by extracting.

  According to the present invention, when an HTML document returned by a Web application changes and a correct part cannot be extracted, a new rule is automatically generated. Can continue to be used as. In other words, the wrapper detects that the HTML document returned by the Web application has changed, and automatically regenerates the configuration file corresponding to the new HTML document, so that the wrapper responds to changes on the Web application side. Become. In other words, the wrapper automatically follows changes on the Web application side. Therefore, the maintenance cost of the wrapper can be reduced. At the same time, it is possible to reduce the inoperability of the wrapper and reduce the service stop time.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram of an extraction system 1 according to an embodiment of the present invention. As shown in FIG.
The request transmission / result reception device 10, the extraction device 20, and the content providing device 30 are configured.

  The content providing device 30 is, for example, a server that implements a Web application. For example, the content providing apparatus 30 receives an HTTP request and transmits an HTML document. In other words, the content providing apparatus 30 transmits the held content in response to a content request for requesting content (hereinafter referred to as “content request”).

  The request transmission / result reception apparatus 10 is, for example, a client (terminal) of a Web service. Specifically, the request transmission / result reception apparatus 10 is a client on which a mashup application or the like is mounted. The request transmission / result reception apparatus 10 transmits a request to the Web service and receives a response to the request. Specifically, the request transmission / result reception apparatus 10 transmits an extraction request (hereinafter simply referred to as “request”) for requesting extraction of a specific portion in the content, and receives an extraction result for the extraction request.

  The extraction device 20 is, for example, a server. The extraction device 20 receives a request (extraction request) from the request transmission / result reception device 10 by SOAP or the like. Upon receiving the request, the extraction device 20 transmits a content request (content request) such as an HTTP request to the content providing device 30 in order to acquire content corresponding to the request, and transmits content such as an HTML document corresponding to the content request. Received from the content providing apparatus 30. The extraction device 20 that has received the content from the content providing device 30 extracts a specific portion of the content, and transmits (returns) the extraction result to the request transmission / result reception device 10 that is the request transmission source via SOAP or the like. To do.

  FIG. 2 is a diagram for explaining the configuration of the extraction device 20 shown in FIG. As illustrated in FIG. 2, the extraction device 20 includes a rule holding mechanism 200, a rule extraction mechanism 210, and a rule generation mechanism 220.

  The rule holding mechanism 200 is a directory that holds various rules for responding to requests (extraction requests) from the request transmission / result reception apparatus 10. Specifically, the rule holding mechanism 200 includes a rule for converting a request such as SOAP received by the extraction device 20 into a content request (content request) to the content providing device 30, an HTML document received by the extraction device 20, and the like. A specific part is extracted from the contents of, and rules for converting to a form such as SOAP are retained.

  The rule extraction mechanism 210 receives a request (extraction request) from the request transmission / result reception apparatus 10. When receiving a request, the rule extraction mechanism 210 determines which rule should be read from the content of the request. After reading the rule to be read, the rule extraction mechanism 210 assembles a content request (content request) to be transmitted to the content providing device 30 from the request received from the request transmission / result reception device 10 according to the rule.

  The rule extraction mechanism 210 transmits the assembled content request (content request) to the content providing apparatus 30. The rule extraction mechanism 210 receives content corresponding to the content request from the content providing apparatus 30. The extraction mechanism 210 based on the rule that has received the content from the content providing apparatus 30 extracts a specific part from the content according to the read rule, converts it into a form such as SOAP, and transmits a request as a request transmission source as an extraction result Reply to the result receiving device 10 When the rule extraction mechanism 210 fails to extract a specific part from the content, the rule generation mechanism 220 transmits content such as an HTML document to be extracted (corresponding to the content transmission request of the present invention) to a new A specific rule is acquired from the rule generation mechanism 220, and a specific portion is re-extracted from the content according to the new rule.

  When the rule extraction mechanism 210 fails to extract a specific part, the rule generation mechanism 220 acquires the content to be extracted from the rule extraction mechanism 210 and generates a new rule from the content and the rule. The rule generation mechanism 220 that has generated a new rule updates the rule by transmitting the generated new rule to the rule extraction mechanism 210 and the rule holding mechanism 200.

  FIG. 3 is a configuration diagram of the extraction mechanism 210 based on the rules shown in FIG. As shown in FIG. 3, the rule extraction unit 210 includes a rule holding unit 211, a request reception unit 212, a request assembly unit 213, a request transmission unit 214, a content holding unit 215, a rule extraction unit 216, and an extraction failure detection unit 217. And an extraction result transmission unit 218 and a content transmission unit 219.

  The rule holding unit 211 acquires (reads) the rules held by the rule holding mechanism 200 from the rule holding mechanism 200 in advance. Further, the rule holding unit 211 performs rule update and distribution.

  The request reception unit 212 receives a request (extraction request) from the request transmission / result reception apparatus 10. The request reception unit 212 transmits the request received from the request transmission / result reception device 10 to the request assembly unit 213.

  The request assembling unit 213 receives a request (extraction request) from the request receiving unit 212. The request assembling unit 213 acquires (reads) from the rule holding unit 211 a rule for converting the request received from the request receiving unit 212 into a content request to be transmitted to the content providing apparatus 30. The request assembling unit 213 converts the request received from the request receiving unit 212 into a content request (content request) according to the rule acquired from the rule holding unit 211. The request assembly unit 213 transmits a content request to the request transmission unit 214.

  The request transmission unit 214 receives a content request (content request) from the request assembling unit 213 and transmits it to the content providing apparatus 30.

  The content holding unit 215 receives content corresponding to the content request (content request) from the content providing apparatus 30. The content holding unit 215 holds the received content and transmits the content to the rule extraction unit 216. Further, the content holding unit 215 provides (transmits) the content held by the content holding unit 215 according to the request of the content transmission unit 219.

  The rule extraction unit 216 receives content from the content holding unit 215. The rule extraction unit 216 acquires (reads) a rule for extracting a specific portion from the content received from the content holding unit 215 from the rule holding unit 211. The rule extraction unit 216 extracts a specific portion from the content received from the content holding unit 215 according to the rule acquired from the rule holding unit 211. The conversion is, for example, an XSL conversion of an HTML document, and an XSL file may be used as a rule at this time. The rule extraction unit 216 transmits the extraction result to the extraction failure detection unit 217.

  The rule extraction unit 216 determines whether the specific part has been correctly extracted when the specific part is extracted. Specifically, when the extraction unit 216 according to the rule extracts the specific part, the extraction unit 216 transmits the extraction result to the extraction failure detection unit 217 and receives the detection result indicating that the extraction has been successful from the extraction failure detection unit 217. In this case, when it is determined that the specific part has been correctly extracted and a detection result indicating that re-extraction is necessary is received from the extraction failure detection unit 217, it is determined that the specific part has not been correctly extracted.

  When the rule extraction unit 216 determines that the specific portion has been correctly extracted, that is, when the detection result indicating that the extraction has been successful is received from the extraction failure detection unit 217, the extraction result is transmitted to the extraction result transmission unit 218. .

  On the other hand, when the rule extraction unit 216 determines that the specific part has not been correctly extracted, that is, when it receives a detection result indicating that re-extraction is necessary from the extraction failure detection unit 217, it extracts a new one from the rule holding unit 211. The specific part is re-extracted according to the re-obtained rule, and the extraction result by re-extraction is transmitted to the extraction result transmission unit 218.

  The extraction failure detection unit 217 receives the extraction result from the rule extraction unit 216. When the extraction failure detection unit 217 receives the extraction result from the rule extraction unit 216, whether the rule extraction unit 216 has correctly extracted the specific part, that is, the rule extraction unit 216 extracts the specific part. Determine if it failed.

  More specifically, when the extraction failure detection unit 217 receives an extraction result from the rule extraction unit 216, the extraction failure detection unit 217 acquires a rule from the rule holding unit 211, and an example of a portion to be extracted written in the rule; It compares with the actually extracted part, and judges whether similarity is more than a threshold value. In other words, the extraction failure detection unit 217 calculates the similarity between the template of the predetermined extraction result based on the rule and the extraction result, and determines whether the similarity is equal to or greater than a threshold value.

  If the extraction failure detection unit 217 determines that the similarity is equal to or greater than the threshold value, the extraction failure detection unit 217 determines that the extraction of the specific portion has not failed, and transmits a detection result indicating that the extraction has been successful to the rule extraction unit 216 (reply) )

  On the other hand, if the extraction failure detection unit 217 determines that the degree of similarity is less than the threshold, it determines that the extraction of the specific part has failed, transmits a content transmission request to the content transmission unit 219, and requires re-extraction. A detection result to the effect is transmitted to the rule extraction unit 216. The content transmission request is a request for transmitting content to the rule generation mechanism 220 in order to regenerate the rule.

  When the extraction result transmission unit 218 correctly extracts the specific part by the rule extraction unit 216, that is, when the extraction failure detection unit 217 determines that the extraction of the specific part by the extraction unit 216 by the rule has not failed. The result of extraction from the rule extraction unit 216 is received and transmitted (returned) to the request transmission / result reception apparatus 10. The extraction result transmission unit 218 transmits the extraction result as a Web service response in the form of SOAP, for example.

  The content transmission unit 219 performs extraction when the extraction unit 216 according to the rule cannot correctly extract the specific part, that is, when the extraction failure detection unit 217 determines that the extraction of the specific part by the extraction unit 216 according to the rule has failed. A content transmission request is received from the failure detection unit 217. The content transmission unit 219 that has received the content transmission request from the extraction failure detection unit 217 acquires the content from the content holding unit 215 and transmits it to the rule generation mechanism 220. This regenerates the rules.

  FIG. 4 is a configuration diagram of the rule generation mechanism 220 shown in FIG. As shown in FIG. 4, the rule generation mechanism 220 includes a rule holding unit 221, a content reception unit 222, a specific part automatic extraction unit 223, an extraction result selection unit 224, a rule generation unit 225, and a rule update unit 226.

  The rule holding unit 221 acquires (reads) the rules held by the rule holding mechanism 200 from the rule holding mechanism 200 in advance. The rule holding unit 221 also distributes rules.

  The content receiving unit 222 receives content from the rule extraction mechanism 210. The content receiving unit 222 transmits the content received from the rule extraction mechanism 210 to the specific part automatic extraction unit 223.

  The specific part automatic extraction unit 223 receives content from the content reception unit 222. The specific part automatic extraction unit 223 automatically extracts specific part candidates of the content received from the content reception unit 222. Specifically, the specific part automatic extracting unit 223 extracts a plurality of specific part candidates as extraction result candidates from the content received from the content receiving unit 222 without using a rule. The specific part automatic extraction unit 223 automatically extracts specific part candidates using, for example, the characteristics of an HTML tag. The specific part automatic extraction unit 223 transmits the plurality of extraction result candidates automatically extracted to the extraction result selection unit 224.

  The extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223. When the extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223, the extraction result selection unit 224 selects one extraction result candidate from the plurality of extraction result candidates as an optimum extraction result. Specifically, when the extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223, the extraction result selection unit 224 acquires the rule from the rule holding unit 221 and extracts the part to be extracted written in the rule. The model is compared with each extraction result candidate, and the most similar extraction result candidate is selected as the optimum extraction result. In other words, the extraction result selection unit 224 calculates the similarity between the template of a predetermined extraction result based on the rule and each extraction result candidate, and selects the extraction result candidate having the maximum similarity as the optimum extraction result. To do. The extraction result selection unit 224 transmits the optimum extraction result to the rule generation unit 225. The optimum extraction result is the extraction result most likely to be a correct extraction result among a plurality of extraction result candidates, and is the optimum extraction result for generating a new rule.

  The rule generation unit 225 receives an optimum extraction result from the extraction result selection unit 224. When receiving an optimal extraction result from the extraction result selection unit 224, the rule generation unit 225 generates a new rule for extracting the optimal extraction result. In other words, the rule generation unit 225 generates a new rule for extracting the same part as the optimal extraction result. The rule generation unit 225 transmits the generated new rule to the rule update unit 226.

  The rule update unit 226 receives a new rule from the rule generation unit 225. The rule update unit 226 transmits the new rule received from the rule generation unit 225 to the rule extraction mechanism 210 and the rule holding mechanism 200. As a result, the rule is updated.

  FIG. 5 is a sequence diagram illustrating the operation of the extraction device 20. FIG. 5 is a sequence diagram illustrating an operation when extraction of a specific portion fails. The rule extraction mechanism 210 reads a rule from the rule holding mechanism 200 in advance (S10). The rule generation mechanism 220 reads a rule from the rule holding mechanism 200 in advance (S11).

  The request transmission / result reception apparatus 10 transmits a request (extraction request) to the rule extraction mechanism 210 (S12). The rule extraction mechanism 210 converts the request (extraction request) into a content request (content request) and transmits it to the content providing apparatus 30 (S13).

  The content providing apparatus 30 returns the content corresponding to the content request (content request) to the rule extraction mechanism 210 (S14). The rule extraction mechanism 210 extracts a specific part from the received content according to the rule. The rule extraction mechanism 210 determines that the extraction of the specific portion has failed. The rule extraction mechanism 210 transmits the content to the rule generation mechanism 220 to regenerate the rules (S15).

  The rule generation mechanism 220 generates a new rule using the received content. The rule generation mechanism 220 updates the rule by transmitting a new rule to the rule extraction mechanism 210 and the rule holding mechanism 200 (S16).

  The rule extraction mechanism 210 re-extracts the specific part of the content according to the new rule. The rule extraction mechanism 210 returns the re-extraction result to the request transmission / result reception apparatus 10 (S17).

  6 and 7 are sequence diagrams showing the operation of the extraction mechanism 210 based on rules. FIG. 6 is a sequence diagram illustrating an operation when extraction of a specific portion fails. The rule holding unit 211 reads a rule in advance from the rule holding mechanism 200 (S20).

  The request transmission / result reception apparatus 10 transmits a request (extraction request) to the request reception unit 212 (S21). The request reception unit 212 transmits the received request to the request assembly unit 213 (S22). The request assembling unit 213 reads the rule from the rule holding unit 211 (S23).

  The request assembling unit 213 assembles a content request (content request) for acquiring content from the received request (extraction request) according to the read rule. The request assembly unit 213 transmits the assembled content request to the request transmission unit 214 (S24). The request transmission unit 214 transmits the received content request to the content providing apparatus 30 (S25).

  The content providing apparatus 30 transmits content corresponding to the content request (content request) to the content holding unit 215 (S26). The content holding unit 215 holds the received content and transmits the content to the rule extraction unit 216 (S27). The rule extraction unit 216 reads a rule for extracting a specific part from the content from the rule holding unit 211 (S28). The rule extraction unit 216 extracts a specific part from the content according to the rule.

  The rule extraction unit 216 transmits the extraction result to the extraction failure detection unit 217 (S29). The extraction failure detection unit 217 reads the rule from the rule holding unit 211 in order to determine whether or not the rule extraction unit 216 has correctly extracted the specific part (S30).

  The extraction failure detection unit 217 determines whether the extraction unit 216 according to the rule has correctly extracted the specific part according to the rule. Here, the extraction failure detection unit 217 determines that extraction has not been performed correctly, that is, extraction failure. Since the extraction failure detection unit 217 needs to regenerate the rule, the extraction failure detection unit 217 transmits a content transmission request to the content transmission unit 219 (S31). The content transmission unit 219 reads content from the content holding unit 215 (S32). The content transmission unit 219 transmits the read content to the rule generation mechanism 220 (S33).

  The rule generation mechanism 220 generates a new rule using the received content. The rule generation mechanism 220 updates the rule by transmitting a new rule to the rule holding unit 211 and the rule holding mechanism 200 (S34).

  The extraction failure detection unit 217 transmits a detection result indicating that re-extraction is necessary (updated rule) to the rule extraction unit 216 (S35). The rule extraction unit 216 that has received the detection result indicating that re-extraction is necessary reads the rule again (S36). The rule extraction unit 216 again extracts a specific part from the content in accordance with the read rule. The rule extraction unit 216 transmits the re-extraction result to the extraction result transmission unit 218 (S37). The extraction result transmission unit 218 transmits the extraction result to the request transmission / result reception apparatus 10 (S38).

  FIG. 8 is a sequence diagram showing the operation of the rule generation mechanism 220. The rule holding unit 221 reads a rule in advance from the rule holding mechanism 200 (S40).

  When the rule extraction mechanism 210 fails to extract a specific part, the content is transmitted to the content receiver 222 (S41). The content receiving unit 222 transmits the received content to the specific part automatic extraction unit 223 (S42).

  The specific part automatic extraction unit 223 extracts a plurality of specific part candidates as extraction result candidates from the content received from the content reception unit 222 without using rules. For example, the specific part automatic extraction unit 223 automatically extracts specific part candidates using the characteristics of HTML tags. The specific part automatic extraction unit 223 transmits a plurality of extraction result candidates to the extraction result selection unit 224 in order to select an optimum extraction result from the plurality of extraction result candidates (S43).

  The extraction result selection unit 224 reads the rule from the rule holding unit 221 (S44). The extraction result selection unit 224 compares each extraction result candidate with the rule model, and selects the most similar extraction result candidate as the optimum extraction result (described as “correct extraction result selection” in FIG. 8). ). The extraction result selection unit 224 transmits the optimum extraction result (simply described as “extraction result” in FIG. 8) to the rule generation unit 225 (S45).

  The rule generation unit 225 generates a new rule for extracting the received optimal extraction result. The rule generation unit 225 transmits the generated rule to the rule update unit 226 (S46). The rule update unit 226 updates the rule by transmitting the received rule to the rule holding mechanism 200 and the rule extraction mechanism 210 (S47).

  FIG. 9 is a flowchart showing the operation of the extraction failure detection unit 217. The extraction failure detection unit 217 acquires the extraction result from the rule extraction unit 216 (step S100). The extraction failure detection unit 217 reads the rule from the rule holding unit 211 (step S110). The extraction failure detection unit 217 calculates the similarity between the extraction result and the model to be extracted in the rule (step S120). The extraction failure detection unit 217 determines whether or not the calculated similarity is greater than or equal to a threshold value (step S130).

  If the extraction failure detection unit 217 determines that the similarity is greater than or equal to the threshold (step S130: Yes), the extraction failure detection unit 217 determines that the extraction is successful and transmits “extraction success” to the rule extraction unit 216 (S140). That is, the extraction failure detection unit 217 transmits a detection result indicating that the extraction has been successful to the rule extraction unit 216.

  On the other hand, if the extraction failure detection unit 217 determines that the similarity is less than the threshold (No at Step S130), the extraction failure detection unit 217 determines that the extraction has failed and transmits a content transmission request to the content transmission unit 219 for rule generation ( S150). The extraction failure detection unit 217 transmits a detection result indicating that re-extraction is necessary to the rule extraction unit 216.

  FIG. 10 is a flowchart showing the operation of the extraction result selection unit 224. The extraction result selection unit 224 acquires a plurality of extraction result candidates that are automatic extraction results (step S200). The extraction result selection unit 224 reads the rule from the rule holding unit 221 (step S210). The extraction result selection unit 224 calculates the similarity between each extraction result candidate and the model to be extracted in the rule (step S220). The extraction result selection unit 224 determines whether there is an extraction result candidate whose calculated similarity is equal to or greater than a threshold (step S230).

  If the extraction result selection unit 224 determines that there is an extraction result candidate whose similarity is equal to or greater than the threshold (step S230: Yes), the extraction result having a high similarity for rule generation (specifically, an optimal extraction result) Is transmitted to the rule generation unit 225 (S240).

  On the other hand, if the extraction result selection unit 224 determines that there is no extraction result candidate whose similarity is equal to or higher than the threshold (step S230: No), the extraction result selection unit 224 determines that the rule regeneration has failed, and performs rule regeneration and re-extraction. Give up (step S250).

  As described above, according to the present invention, since the wrapper automatically follows the change on the Web application side, the maintenance cost of the wrapper can be reduced. At the same time, it becomes possible to reduce the inoperability of the wrapper and reduce the service stop time.

  It should be noted that a program for realizing each process of the extraction device 20 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed, whereby the extraction device 20 is executed. The various processes described above may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

It is a lineblock diagram of extraction system 1 concerning an embodiment of the present invention. It is a figure for demonstrating the structure of the extracting device 20 shown in FIG. It is a block diagram of the extraction mechanism 210 by the rule shown in FIG. It is a block diagram of the rule production | generation mechanism 220 shown in FIG. FIG. 6 is a sequence diagram showing an operation of the extraction device 20. It is a sequence diagram which shows operation | movement of the extraction mechanism 210 by a rule. It is a sequence diagram which shows operation | movement of the extraction mechanism 210 by a rule. FIG. 6 is a sequence diagram showing an operation of a rule generation mechanism 220. 10 is a flowchart showing the operation of an extraction failure detection unit 217. It is a flowchart which shows operation | movement of the extraction result selection part 224.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Extraction system 10 Request transmission / result reception apparatus 20 Extraction apparatus 30 Content provision apparatus 200 Rule holding mechanism 210 Rule extraction mechanism 211 Rule holding section 212 Request reception section 213 Request assembly section 214 Request transmission section 215 Content holding section 216 Extraction by rule Unit 217 extraction failure detection unit 218 content transmission unit 219 extraction result transmission unit 220 rule generation mechanism 221 rule holding unit 222 content reception unit 223 specific part automatic extraction unit 224 extraction result selection unit 225 rule generation unit 226 rule update unit

Claims (6)

When receiving the extraction request from the request transmission-result receiving unit which transmits the extracted request for extraction of a specific portion of the content by sending a content request to the content providing device holding the content, the content providing An extraction device that receives content related to the extraction request from a device , extracts a specific part of the content, and transmits the extracted result to the request transmission / result reception device,
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism that generates a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
Based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result according to the rule, it is determined whether or not the extraction of the specific part has failed,
The rule generation mechanism is:
Among the plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus, the extraction result candidate having the maximum similarity with the template of the predetermined extraction result by the rule is selected as the optimum extraction result. , Generating a rule for extracting the optimum extraction result as the new rule,
The extraction mechanism by the rule is:
When the rule generation mechanism generates a new rule, the extraction device re-extracts the specific portion according to the new rule. When receiving the extraction request from the request transmission-result receiving unit which transmits the extracted request for extraction of a specific portion of the content by sending a content request to the content providing device holding the content, the content providing An extraction device that receives content related to the extraction request from a device , extracts a specific part of the content, and transmits the extracted result to the request transmission / result reception device,
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism that generates a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
When the rule generation mechanism generates a new rule, the specific part is re-extracted according to the new rule ,
The rule generation mechanism is:
A rule holding unit that reads a rule from the rule holding mechanism and distributes the rule;
A content receiver that receives the content from the extraction mechanism according to the rule;
A specific part automatic extraction unit for automatically extracting extraction result candidates of the specific part of the content received by the content receiving unit;
The similarity is obtained by reading a rule from the rule holding unit and calculating a similarity between a template of a predetermined extraction result based on the rule and each of a plurality of extraction result candidates automatically extracted by the specific part automatic extraction unit. An extraction result selection unit that selects an extraction result candidate having the maximum degree as an optimum extraction result;
A rule generation unit for generating a new rule for extracting the optimum extraction result selected by the extraction result selection unit;
A rule update unit for updating the rule by transmitting the new rule generated by the rule generation unit to an extraction mechanism based on the rule and a rule holding mechanism;
An extraction device comprising: When receiving the extraction request from the request transmission-result receiving unit which transmits the extracted request for extraction of a specific portion of the content by sending a content request to the content providing device holding the content, the content providing In the extraction device that receives the content related to the extraction request from the device , extracts a specific part of the content, and transmits it to the request transmission / result reception device as an extraction result .
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism that generates a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
A rule holding unit that obtains the rule from the rule holding mechanism and updates and distributes the rule;
A request receiving unit that receives the extraction request from the request transmission / result reception device;
A request assembling unit that converts the extraction request received by the request receiving unit into the content request in accordance with the rule acquired by the rule holding unit;
A request transmitting unit that transmits the content request converted by the request assembling unit to the content providing device;
A content holding unit that receives, holds, and distributes content corresponding to the content request from a content providing device;
An extraction unit based on a rule for extracting a specific part from the content received by the content holding unit according to the rule acquired by the rule holding unit;
An extraction failure detection unit that determines whether or not extraction of a specific part by the extraction unit according to the rule has failed;
When the extraction failure detection unit determines that the extraction of the specific part has not failed, an extraction result transmission unit that returns the extraction result by the extraction unit according to the rule to the request transmission / result reception device;
A content transmission unit that acquires the content from the content holding unit and transmits the content to the rule generation mechanism when the extraction failure detection unit determines that the extraction of the specific part has failed;
Have
The extraction part according to the rule is:
When the rule generation mechanism generates a new rule based on the content transmitted by the content transmission unit, the extraction device re-extracts a specific part according to the new rule ,
The extraction part according to the rule is:
When the specific part is extracted, the extraction result is transmitted to the extraction failure detection unit, and when the detection result indicating that the extraction is successful is received from the extraction failure detection unit, the extraction result is transmitted to the extraction result transmission unit. When the detection result indicating that re-extraction is necessary is received from the extraction failure detection unit, the specific part is re-extracted according to the new rule,
The extraction failure detection unit
When the extraction result is received from the extraction unit based on the rule, the similarity is obtained by obtaining a rule from the rule holding unit and calculating a similarity between a template of a predetermined extraction result based on the rule and the extraction result When the degree is equal to or greater than a threshold value, it is determined that the extraction of the specific part has not failed, and the detection result indicating that the extraction is successful is returned to the extraction unit according to the rule, and the similarity is less than the threshold value In this case, it is determined that the extraction of the specific part has failed, the content transmission request is transmitted to the content transmission unit to regenerate the rule, and the detection result indicating that the re-extraction is necessary is extracted by the rule. Send to the department,
The content transmitting unit
When the content transmission request is received, the content is acquired from the content holding unit and transmitted to the rule generation mechanism. An extraction system that extracts a specific part of content,
A content providing device that transmits content held in response to a content request for requesting content;
A request transmission / result reception device for transmitting an extraction request for requesting extraction of a specific part in content and receiving an extraction result for the extraction request;
Receiving the extraction request from the request transmission / result receiving device, transmitting a content request for requesting content related to the extraction request to the content providing device, extracting a specific part of the content received from the content providing device; An extraction device that transmits to the request transmission / result reception device as an extraction result,
The extraction device comprises:
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism for generating a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
Based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result according to the rule, it is determined whether or not the extraction of the specific part has failed,
The rule generation mechanism is:
Among the plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus, the extraction result candidate having the maximum similarity with the template of the predetermined extraction result by the rule is selected as the optimum extraction result. , Generating a rule for extracting the optimum extraction result as the new rule,
The extraction mechanism by the rule is:
An extraction system, wherein when the rule generation mechanism generates a new rule, a specific portion is re-extracted according to the new rule. A content providing apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific part in content and holds the content when the extraction request is received from a request transmission / result receiving apparatus that transmits the extraction request Content by the extraction device that receives the content related to the extraction request from the content providing device , extracts a specific portion of the content, and transmits it to the request transmission / result reception device as an extraction result An extraction method for extracting a specific part of
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus An extraction step according to a rule for extracting a specific part according to the rule;
A rule generation step for generating a new rule when the extraction of the specific part by the extraction step by the rule fails,
The extraction step according to the rule includes:
Based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result according to the rule, it is determined whether or not the extraction of the specific part has failed,
The rule generation step includes:
Among the plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus, the extraction result candidate having the maximum similarity with the template of the predetermined extraction result by the rule is selected as the optimum extraction result. , Generating a rule for extracting the optimum extraction result as the new rule,
The extraction step according to the rule includes:
When the rule generation step generates a new rule, a specific portion is re-extracted according to the new rule. A content providing apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific part in content and holds the content when the extraction request is received from a request transmission / result receiving apparatus that transmits the extraction request A computer of the extraction device that receives the content related to the extraction request from the content providing device , extracts a specific part of the content, and transmits it to the request transmission / result reception device as an extraction result In addition,
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus An extraction step according to a rule for extracting a specific part according to the rule;
When the extraction of the specific part by the extraction step by the rule fails, the rule generation step for generating a new rule is executed,
The extraction step according to the rule includes:
Based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result according to the rule, it is determined whether or not the extraction of the specific part has failed,
The rule generation step includes:
Among the plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus, the extraction result candidate having the maximum similarity with the template of the predetermined extraction result by the rule is selected as the optimum extraction result. , Generating a rule for extracting the optimum extraction result as the new rule,
The extraction step according to the rule includes:
When the rule generation step generates a new rule, a specific part is re-extracted according to the new rule.

Images