JP2010049638A - Apparatus, system, method for extracting and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a program and an apparatus for extracting, a method and an extraction system capable of reducing a service stop time and cutting a maintenance cost by automatically corresponding to a change at a Web application. <P>SOLUTION: An extraction system 1 includes a content provider 30 for transmitting the held content in accordance with a content request for requesting the content, a request transmission and result receiving device 10 for transmitting an extraction request for extracting a specific part within the content and for receiving an extracted result for the extraction request, and an extraction device 20 for receiving the extraction request from the request transmission and result receiving device 10, for transmitting the content request for requesting the content associated with the extraction request to the content provider 30, for extracting the specific part within the content received from the content provider 30, and for transmitting the specific part to a request transmission and result receiving structure as a result of extraction. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an extraction device, an extraction system, an extraction method, and a program. In particular, the present invention relates to an extraction device, an extraction system, an extraction method, and a program for extracting a specific portion from an HTML document generated by a Web application.

  In recent years, the need for Web services has increased. Web services are service components that are also used as parts to realize various services, and have been used mainly in the field of B2B. In addition, in recent years, there is a movement to use these also in the ubiquitous field, and Web services are being actively used in B2C and C2C. However, at this stage, the number of Web services that can be used is small, and various services have not been provided. Therefore, it is necessary to create an environment that can realize various services by increasing the number of Web services and combining them.

  There is an idea of using an existing web application as a web service. As a result, a wealth of Web services can be used. There are wrappers that use this. The wrapper has a web service interface and a web application interface. The wrapper converts the request that came to the web service side of the wrapper into a web application request and sends it to the web application. After that, a specific part is extracted from the returned HTML document, converted into SOAP (Simple Object Access Protocol), and returned to the request sender. At this time, a configuration file for protocol conversion is required for each Web application. For example, when implementing a wrapper for making a hotel search Web application a hotel search Web service, the configuration file for the hotel search Web application Necessary.

The configuration file mainly includes rules for converting SOAP requests to Web application requests, and rules for extracting specific parts from HTML documents returned by Web applications and converting the extraction results to SOAP. It is. There are Patent Documents 1 and 2 as technologies for mutual conversion between SOAP and Web application protocols using such rules.
JP 2007-241417 A JP 2008-003848 A

  However, if the HTML document returned by the Web application changes, the correct part cannot be extracted from the preconfigured config file, and the Web application cannot be used as a Web service. In order to solve this problem, it is necessary to rework the config file, which requires a lot of labor.

  When the HTML document returned by the Web application changes, the purpose is to easily extract the correct part so that the Web application can be used as a Web service.

  In order to solve the above-described problem, an aspect of the present invention is to provide content that retains content when an extraction request is received from a request transmission / result reception device that transmits an extraction request that requests extraction of a specific portion in the content. An extraction device that receives content related to the extraction request from the content transmission device by transmitting a content request to the transmission device, extracts a specific portion of the content, and transmits the extracted result to the request transmission / result reception device. A rule holding mechanism for holding a rule for responding to the extraction request, and receiving the extraction request from the request transmission / result reception device, converting the extraction request into a content request according to the rule, and sending the content request to the content providing device. According to the rules from the content sent and received from the content providing device An extraction mechanism based on a rule that extracts a fixed part and a rule generation mechanism that generates a new rule when extraction of a specific part by the rule extraction mechanism fails. The rule extraction mechanism is a new rule generation mechanism. When a simple rule is generated, a specific part is re-extracted according to a new rule.

  The extraction mechanism based on the rule of the extraction device according to one aspect of the present invention is based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result based on the rule. The rule generation mechanism determines whether the similarity with a template of a predetermined extraction result by the rule from among a plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus. The largest extraction result candidate may be selected as the optimum extraction result, and a rule for extracting the optimum extraction result may be generated as a new rule.

  An extraction mechanism based on a rule of an extraction apparatus according to one aspect of the present invention acquires a rule from a rule holding mechanism, receives a request for extraction from a rule holding unit that updates and distributes the rule, and a request transmission / result reception apparatus A request receiving unit, a request assembling unit that converts an extraction request received by the request receiving unit into a content request according to a rule acquired by the rule holding unit, and a content request converted by the request assembling unit A request transmission unit to transmit, a content holding unit that receives, holds, and distributes content in response to a content request from the content providing device, and a specific part from the content received by the content holding unit according to the rules acquired by the rule holding unit Extraction part by rule to extract The extraction failure detection unit that determines whether or not the extraction of the specific part by the extraction unit by the rule has failed, and the extraction unit by the rule when the extraction failure detection unit determines that the extraction of the specific part has not failed When the extraction result sending unit returns the extraction result to the request transmission / result receiving device and the extraction failure detection unit determines that the extraction of the specific part has failed, the content is acquired from the content holding unit, and a rule is generated. A content transmission unit that transmits to the mechanism, and the rule extraction unit may re-extract the specific portion according to the new rule generated by the rule generation mechanism based on the content transmitted by the content transmission unit. .

  The extraction unit based on the rule of the extraction device according to one aspect of the present invention, when a specific part is extracted, transmits the extraction result to the extraction failure detection unit, and receives the detection result indicating that the extraction has been successful from the extraction failure detection unit When the detection result is transmitted to the extraction result transmission unit and the detection result indicating that re-extraction is necessary is received from the extraction failure detection unit, the specific part is re-extracted according to the new rule to detect the extraction failure. When the unit receives the extraction result from the rule extraction unit, the unit obtains the rule from the rule holding unit, and calculates the similarity between the template of the predetermined extraction result by the rule and the extraction result, whereby the similarity is a threshold value When it is above, it is determined that the extraction of the specific part has not failed, and the detection result indicating that the extraction has been successful is returned to the extraction part according to the rule, and when the similarity is less than the threshold value, Extraction failed The content transmission request is transmitted to the content transmission unit in order to regenerate the rule, and the detection result indicating that re-extraction is necessary is transmitted to the extraction unit based on the rule. May be acquired from the content holding unit and transmitted to the rule generation mechanism.

  A rule generation mechanism of an extraction device according to an aspect of the present invention includes a rule holding unit that reads a rule from a rule holding mechanism and distributes the rule, a content reception unit that receives content from the extraction mechanism based on the rule, and a content reception unit The specific part automatic extraction unit that automatically extracts the extraction result candidates of the specific part of the content received by the rule, the rule is read from the rule holding unit, and a template of a predetermined extraction result based on the rule and the specific part automatic extraction unit are automatically extracted The extraction result selection unit that selects the extraction result candidate having the maximum similarity as the optimum extraction result by calculating the similarity with each of the plurality of extraction result candidates, and the extraction result selection unit selected A rule generation unit that generates a new rule for extracting an optimal extraction result, and a rule generation unit It may have a rule updating unit for updating the rules by sending Tana rule extraction mechanism and rule holding mechanism according to the rules.

  In order to solve the above problem, another aspect of the present invention is an extraction system, a content providing device that transmits content held in response to a content request for requesting content, and identification within the content A request transmission / result reception device that transmits an extraction request for requesting extraction of a portion and receives an extraction result for the extraction request, and a content that receives the extraction request from the request transmission / result reception device and requests content related to the extraction request An extraction device that transmits a request to the content providing device, extracts a specific part of the content received from the content providing device, and transmits the request to the request transmission / result receiving device as an extraction result. Receive the extraction request from the rule holding mechanism and the request sending / result receiving device An extraction mechanism based on a rule that converts the extraction request into a content request according to a rule, transmits the content request to the content providing apparatus, and extracts a specific part from the content received from the content providing apparatus, and an extraction mechanism based on the rule A rule generation mechanism that generates a new rule when the extraction of the specific part fails, and the rule extraction mechanism re-creates the specific part according to the new rule when the rule generation mechanism generates a new rule. It is characterized by extracting.

  In order to solve the above-described problem, another aspect of the present invention provides a request transmission / result reception apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific portion in content and transmits the extraction request. When an extraction request is received, by transmitting the content request to the content transmission device that holds the content, the content related to the extraction request is received from the content transmission device, the specific part of the content is extracted, and the extraction result is An extraction method for extracting a specific part in content by an extraction device that transmits to a request transmission / result reception device, receiving an extraction request from the request transmission / result reception device, and converting the extraction request into a content request according to a rule The content request is transmitted to the content providing device and received from the content providing device. An extraction step by a rule for extracting a specific part from the content according to the rule, and a rule generation step for generating a new rule when the extraction of the specific part by the extraction step by the rule fails. When the rule generation step generates a new rule, the specific part is re-extracted according to the new rule.

  In order to solve the above-described problem, another aspect of the present invention provides a request transmission / result reception apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific portion in content and transmits the extraction request. When an extraction request is received, by transmitting the content request to the content transmission device that holds the content, the content related to the extraction request is received from the content transmission device, the specific part of the content is extracted, and the extraction result is The extraction request is transmitted from the request transmission / result reception device to the request transmission / result reception device, and the extraction request is converted into a content request according to the rule, and the content request is transmitted to the content providing device. From the content received from the content providing device, the specific unit according to the rule Is a program that executes an extraction step based on a rule that extracts a rule, and a rule generation step that generates a new rule if extraction of a specific part by the extraction step based on the rule fails. When the step generates a new rule, the specific part is re-extracted according to the new rule.

  According to the present invention, when an HTML document returned by a Web application changes and a correct part cannot be extracted, a new rule is automatically generated. Can continue to be used as. In other words, the wrapper detects that the HTML document returned by the Web application has changed, and automatically regenerates the configuration file corresponding to the new HTML document, so that the wrapper responds to changes on the Web application side. Become. In other words, the wrapper automatically follows changes on the Web application side. Therefore, the maintenance cost of the wrapper can be reduced. At the same time, it is possible to reduce the inoperability of the wrapper and reduce the service stop time.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram of an extraction system 1 according to an embodiment of the present invention. As shown in FIG.
The request transmission / result reception device 10, the extraction device 20, and the content providing device 30 are configured.

  The content providing device 30 is, for example, a server that implements a Web application. For example, the content providing apparatus 30 receives an HTTP request and transmits an HTML document. In other words, the content providing apparatus 30 transmits the held content in response to a content request for requesting content (hereinafter referred to as “content request”).

  The request transmission / result reception apparatus 10 is, for example, a client (terminal) of a Web service. Specifically, the request transmission / result reception apparatus 10 is a client on which a mashup application or the like is mounted. The request transmission / result reception apparatus 10 transmits a request to the Web service and receives a response to the request. Specifically, the request transmission / result reception apparatus 10 transmits an extraction request (hereinafter simply referred to as “request”) for requesting extraction of a specific portion in the content, and receives an extraction result for the extraction request.

  The extraction device 20 is, for example, a server. The extraction device 20 receives a request (extraction request) from the request transmission / result reception device 10 by SOAP or the like. Upon receiving the request, the extraction device 20 transmits a content request (content request) such as an HTTP request to the content providing device 30 in order to acquire content corresponding to the request, and transmits content such as an HTML document corresponding to the content request. Received from the content providing apparatus 30. The extraction device 20 that has received the content from the content providing device 30 extracts a specific portion of the content, and transmits (returns) the extraction result to the request transmission / result reception device 10 that is the request transmission source via SOAP or the like. To do.

  FIG. 2 is a diagram for explaining the configuration of the extraction device 20 shown in FIG. As illustrated in FIG. 2, the extraction device 20 includes a rule holding mechanism 200, a rule extraction mechanism 210, and a rule generation mechanism 220.

  The rule holding mechanism 200 is a directory that holds various rules for responding to requests (extraction requests) from the request transmission / result reception apparatus 10. Specifically, the rule holding mechanism 200 includes a rule for converting a request such as SOAP received by the extraction device 20 into a content request (content request) to the content providing device 30, an HTML document received by the extraction device 20, and the like. A specific part is extracted from the contents of, and rules for converting to a form such as SOAP are retained.

  The rule extraction mechanism 210 receives a request (extraction request) from the request transmission / result reception apparatus 10. When receiving a request, the rule extraction mechanism 210 determines which rule should be read from the content of the request. After reading the rule to be read, the rule extraction mechanism 210 assembles a content request (content request) to be transmitted to the content providing device 30 from the request received from the request transmission / result reception device 10 according to the rule.

  The rule extraction mechanism 210 transmits the assembled content request (content request) to the content providing apparatus 30. The rule extraction mechanism 210 receives content corresponding to the content request from the content providing apparatus 30. The extraction mechanism 210 based on the rule that has received the content from the content providing apparatus 30 extracts a specific part from the content according to the read rule, converts it into a form such as SOAP, and transmits a request as a request transmission source as an extraction result Reply to the result receiving device 10 When the rule extraction mechanism 210 fails to extract a specific part from the content, the rule generation mechanism 220 transmits content such as an HTML document to be extracted (corresponding to the content transmission request of the present invention) to a new A specific rule is acquired from the rule generation mechanism 220, and a specific portion is re-extracted from the content according to the new rule.

  When the rule extraction mechanism 210 fails to extract a specific part, the rule generation mechanism 220 acquires the content to be extracted from the rule extraction mechanism 210 and generates a new rule from the content and the rule. The rule generation mechanism 220 that has generated a new rule updates the rule by transmitting the generated new rule to the rule extraction mechanism 210 and the rule holding mechanism 200.

  FIG. 3 is a configuration diagram of the extraction mechanism 210 based on the rules shown in FIG. As shown in FIG. 3, the rule extraction unit 210 includes a rule holding unit 211, a request reception unit 212, a request assembly unit 213, a request transmission unit 214, a content holding unit 215, a rule extraction unit 216, and an extraction failure detection unit 217. And an extraction result transmission unit 218 and a content transmission unit 219.

  The rule holding unit 211 acquires (reads) the rules held by the rule holding mechanism 200 from the rule holding mechanism 200 in advance. Further, the rule holding unit 211 performs rule update and distribution.

  The request reception unit 212 receives a request (extraction request) from the request transmission / result reception apparatus 10. The request reception unit 212 transmits the request received from the request transmission / result reception device 10 to the request assembly unit 213.

  The request assembling unit 213 receives a request (extraction request) from the request receiving unit 212. The request assembling unit 213 acquires (reads) from the rule holding unit 211 a rule for converting the request received from the request receiving unit 212 into a content request to be transmitted to the content providing apparatus 30. The request assembling unit 213 converts the request received from the request receiving unit 212 into a content request (content request) according to the rule acquired from the rule holding unit 211. The request assembly unit 213 transmits a content request to the request transmission unit 214.

  The request transmission unit 214 receives a content request (content request) from the request assembling unit 213 and transmits it to the content providing apparatus 30.

  The content holding unit 215 receives content corresponding to the content request (content request) from the content providing apparatus 30. The content holding unit 215 holds the received content and transmits the content to the rule extraction unit 216. Further, the content holding unit 215 provides (transmits) the content held by the content holding unit 215 according to the request of the content transmission unit 219.

  The rule extraction unit 216 receives content from the content holding unit 215. The rule extraction unit 216 acquires (reads) a rule for extracting a specific portion from the content received from the content holding unit 215 from the rule holding unit 211. The rule extraction unit 216 extracts a specific portion from the content received from the content holding unit 215 according to the rule acquired from the rule holding unit 211. The conversion is, for example, an XSL conversion of an HTML document, and an XSL file may be used as a rule at this time. The rule extraction unit 216 transmits the extraction result to the extraction failure detection unit 217.

  The rule extraction unit 216 determines whether the specific part has been correctly extracted when the specific part is extracted. Specifically, when extracting a specific part, the rule extraction unit 216 transmits an extraction result to the extraction failure detection unit 217 and receives a detection result indicating that the extraction has been successful from the extraction failure detection unit 217. In this case, when it is determined that the specific part has been correctly extracted and a detection result indicating that re-extraction is necessary is received from the extraction failure detection unit 217, it is determined that the specific part has not been correctly extracted.

  When the rule extraction unit 216 determines that the specific portion has been correctly extracted, that is, when the detection result indicating that the extraction has been successful is received from the extraction failure detection unit 217, the extraction result is transmitted to the extraction result transmission unit 218. .

  On the other hand, when the rule extraction unit 216 determines that the specific part has not been correctly extracted, that is, when it receives a detection result indicating that re-extraction is necessary from the extraction failure detection unit 217, it extracts a new one from the rule holding unit 211. The specific part is re-extracted according to the re-obtained rule, and the extraction result by re-extraction is transmitted to the extraction result transmission unit 218.

  The extraction failure detection unit 217 receives the extraction result from the rule extraction unit 216. When the extraction failure detection unit 217 receives the extraction result from the rule extraction unit 216, whether the rule extraction unit 216 has correctly extracted the specific part, that is, the rule extraction unit 216 extracts the specific part. Determine if it failed.

  More specifically, when the extraction failure detection unit 217 receives an extraction result from the rule extraction unit 216, the extraction failure detection unit 217 acquires a rule from the rule holding unit 211, and an example of a portion to be extracted written in the rule; It compares with the actually extracted part, and judges whether similarity is more than a threshold value. In other words, the extraction failure detection unit 217 calculates the similarity between the template of the predetermined extraction result based on the rule and the extraction result, and determines whether the similarity is equal to or greater than a threshold value.

  If the extraction failure detection unit 217 determines that the similarity is equal to or greater than the threshold value, the extraction failure detection unit 217 determines that the extraction of the specific portion has not failed, and transmits a detection result indicating that the extraction has been successful to the rule extraction unit 216 (reply) )

  On the other hand, if the extraction failure detection unit 217 determines that the degree of similarity is less than the threshold, it determines that the extraction of the specific part has failed, transmits a content transmission request to the content transmission unit 219, and requires re-extraction. A detection result to the effect is transmitted to the rule extraction unit 216. The content transmission request is a request for transmitting content to the rule generation mechanism 220 in order to regenerate the rule.

  When the extraction result transmission unit 218 correctly extracts the specific part by the rule extraction unit 216, that is, when the extraction failure detection unit 217 determines that the extraction of the specific part by the extraction unit 216 by the rule has not failed. The result of extraction from the rule extraction unit 216 is received and transmitted (returned) to the request transmission / result reception apparatus 10. The extraction result transmission unit 218 transmits the extraction result as a Web service response in the form of SOAP, for example.

  The content transmission unit 219 performs extraction when the extraction unit 216 according to the rule cannot correctly extract the specific part, that is, when the extraction failure detection unit 217 determines that the extraction of the specific part by the extraction unit 216 according to the rule has failed. A content transmission request is received from the failure detection unit 217. The content transmission unit 219 that has received the content transmission request from the extraction failure detection unit 217 acquires the content from the content holding unit 215 and transmits it to the rule generation mechanism 220. This regenerates the rules.

  FIG. 4 is a configuration diagram of the rule generation mechanism 220 shown in FIG. As shown in FIG. 4, the rule generation mechanism 220 includes a rule holding unit 221, a content reception unit 222, a specific part automatic extraction unit 223, an extraction result selection unit 224, a rule generation unit 225, and a rule update unit 226.

  The rule holding unit 221 acquires (reads) the rules held by the rule holding mechanism 200 from the rule holding mechanism 200 in advance. The rule holding unit 221 also distributes rules.

  The content receiving unit 222 receives content from the rule extraction mechanism 210. The content receiving unit 222 transmits the content received from the rule extraction mechanism 210 to the specific part automatic extraction unit 223.

  The specific part automatic extraction unit 223 receives content from the content reception unit 222. The specific part automatic extraction unit 223 automatically extracts specific part candidates of the content received from the content reception unit 222. Specifically, the specific part automatic extracting unit 223 extracts a plurality of specific part candidates as extraction result candidates from the content received from the content receiving unit 222 without using a rule. The specific part automatic extraction unit 223 automatically extracts specific part candidates using, for example, the characteristics of an HTML tag. The specific part automatic extraction unit 223 transmits the plurality of extraction result candidates automatically extracted to the extraction result selection unit 224.

  The extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223. When the extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223, the extraction result selection unit 224 selects one extraction result candidate from the plurality of extraction result candidates as an optimum extraction result. Specifically, when the extraction result selection unit 224 receives a plurality of extraction result candidates from the specific part automatic extraction unit 223, the extraction result selection unit 224 acquires the rule from the rule holding unit 221 and extracts the part to be extracted written in the rule. The model is compared with each extraction result candidate, and the most similar extraction result candidate is selected as the optimum extraction result. In other words, the extraction result selection unit 224 calculates the similarity between the template of a predetermined extraction result based on the rule and each extraction result candidate, and selects the extraction result candidate having the maximum similarity as the optimum extraction result. To do. The extraction result selection unit 224 transmits the optimum extraction result to the rule generation unit 225. The optimum extraction result is the extraction result most likely to be a correct extraction result among a plurality of extraction result candidates, and is the optimum extraction result for generating a new rule.

  The rule generation unit 225 receives an optimum extraction result from the extraction result selection unit 224. When receiving an optimal extraction result from the extraction result selection unit 224, the rule generation unit 225 generates a new rule for extracting the optimal extraction result. In other words, the rule generation unit 225 generates a new rule for extracting the same part as the optimal extraction result. The rule generation unit 225 transmits the generated new rule to the rule update unit 226.

  The rule update unit 226 receives a new rule from the rule generation unit 225. The rule update unit 226 transmits the new rule received from the rule generation unit 225 to the rule extraction mechanism 210 and the rule holding mechanism 200. As a result, the rule is updated.

  FIG. 5 is a sequence diagram illustrating the operation of the extraction device 20. FIG. 5 is a sequence diagram illustrating an operation when extraction of a specific portion fails. The rule extraction mechanism 210 reads a rule from the rule holding mechanism 200 in advance (S10). The rule generation mechanism 220 reads a rule from the rule holding mechanism 200 in advance (S11).

  The request transmission / result reception apparatus 10 transmits a request (extraction request) to the rule extraction mechanism 210 (S12). The rule extraction mechanism 210 converts the request (extraction request) into a content request (content request) and transmits it to the content providing apparatus 30 (S13).

  The content providing apparatus 30 returns the content corresponding to the content request (content request) to the rule extraction mechanism 210 (S14). The rule extraction mechanism 210 extracts a specific part from the received content according to the rule. The rule extraction mechanism 210 determines that the extraction of the specific portion has failed. The rule extraction mechanism 210 transmits the content to the rule generation mechanism 220 to regenerate the rules (S15).

  The rule generation mechanism 220 generates a new rule using the received content. The rule generation mechanism 220 updates the rule by transmitting a new rule to the rule extraction mechanism 210 and the rule holding mechanism 200 (S16).

  The rule extraction mechanism 210 re-extracts the specific part of the content according to the new rule. The rule extraction mechanism 210 returns the re-extraction result to the request transmission / result reception apparatus 10 (S17).

  6 and 7 are sequence diagrams showing the operation of the extraction mechanism 210 based on rules. FIG. 6 is a sequence diagram illustrating an operation when extraction of a specific portion fails. The rule holding unit 211 reads a rule in advance from the rule holding mechanism 200 (S20).

  The request transmission / result reception apparatus 10 transmits a request (extraction request) to the request reception unit 212 (S21). The request reception unit 212 transmits the received request to the request assembly unit 213 (S22). The request assembling unit 213 reads the rule from the rule holding unit 211 (S23).

  The request assembling unit 213 assembles a content request (content request) for acquiring content from the received request (extraction request) according to the read rule. The request assembly unit 213 transmits the assembled content request to the request transmission unit 214 (S24). The request transmission unit 214 transmits the received content request to the content providing apparatus 30 (S25).

  The content providing apparatus 30 transmits content corresponding to the content request (content request) to the content holding unit 215 (S26). The content holding unit 215 holds the received content and transmits the content to the rule extraction unit 216 (S27). The rule extraction unit 216 reads a rule for extracting a specific part from the content from the rule holding unit 211 (S28). The rule extraction unit 216 extracts a specific part from the content according to the rule.

  The rule extraction unit 216 transmits the extraction result to the extraction failure detection unit 217 (S29). The extraction failure detection unit 217 reads the rule from the rule holding unit 211 in order to determine whether or not the rule extraction unit 216 has correctly extracted the specific part (S30).

  The extraction failure detection unit 217 determines whether the extraction unit 216 according to the rule has correctly extracted the specific part according to the rule. Here, the extraction failure detection unit 217 determines that extraction has not been performed correctly, that is, extraction failure. Since the extraction failure detection unit 217 needs to regenerate the rule, the extraction failure detection unit 217 transmits a content transmission request to the content transmission unit 219 (S31). The content transmission unit 219 reads content from the content holding unit 215 (S32). The content transmission unit 219 transmits the read content to the rule generation mechanism 220 (S33).

  The rule generation mechanism 220 generates a new rule using the received content. The rule generation mechanism 220 updates the rule by transmitting a new rule to the rule holding unit 211 and the rule holding mechanism 200 (S34).

  The extraction failure detection unit 217 transmits a detection result indicating that re-extraction is necessary (updated rule) to the rule extraction unit 216 (S35). The rule extraction unit 216 that has received the detection result indicating that re-extraction is necessary re-reads the rule (S36). The rule extraction unit 216 again extracts a specific part from the content in accordance with the read rule. The rule extraction unit 216 transmits the re-extraction result to the extraction result transmission unit 218 (S37). The extraction result transmission unit 218 transmits the extraction result to the request transmission / result reception apparatus 10 (S38).

  FIG. 8 is a sequence diagram showing the operation of the rule generation mechanism 220. The rule holding unit 221 reads a rule in advance from the rule holding mechanism 200 (S40).

  When the rule extraction mechanism 210 fails to extract a specific part, the content is transmitted to the content receiver 222 (S41). The content receiving unit 222 transmits the received content to the specific part automatic extraction unit 223 (S42).

  The specific part automatic extraction unit 223 extracts a plurality of specific part candidates as extraction result candidates from the content received from the content reception unit 222 without using rules. For example, the specific part automatic extraction unit 223 automatically extracts specific part candidates using the characteristics of HTML tags. The specific part automatic extraction unit 223 transmits a plurality of extraction result candidates to the extraction result selection unit 224 in order to select an optimum extraction result from the plurality of extraction result candidates (S43).

  The extraction result selection unit 224 reads the rule from the rule holding unit 221 (S44). The extraction result selection unit 224 compares each extraction result candidate with the rule model, and selects the most similar extraction result candidate as the optimum extraction result (described as “correct extraction result selection” in FIG. 8). ). The extraction result selection unit 224 transmits the optimum extraction result (simply described as “extraction result” in FIG. 8) to the rule generation unit 225 (S45).

  The rule generation unit 225 generates a new rule for extracting the received optimal extraction result. The rule generation unit 225 transmits the generated rule to the rule update unit 226 (S46). The rule update unit 226 updates the rule by transmitting the received rule to the rule holding mechanism 200 and the rule extraction mechanism 210 (S47).

  FIG. 9 is a flowchart showing the operation of the extraction failure detection unit 217. The extraction failure detection unit 217 acquires the extraction result from the rule extraction unit 216 (step S100). The extraction failure detection unit 217 reads the rule from the rule holding unit 211 (step S110). The extraction failure detection unit 217 calculates the similarity between the extraction result and the model to be extracted in the rule (step S120). The extraction failure detection unit 217 determines whether or not the calculated similarity is greater than or equal to a threshold value (step S130).

  If the extraction failure detection unit 217 determines that the similarity is greater than or equal to the threshold (step S130: Yes), the extraction failure detection unit 217 determines that the extraction is successful and transmits “extraction success” to the rule extraction unit 216 (S140). That is, the extraction failure detection unit 217 transmits a detection result indicating that the extraction has been successful to the rule extraction unit 216.

  On the other hand, if the extraction failure detection unit 217 determines that the similarity is less than the threshold (No at Step S130), the extraction failure detection unit 217 determines that the extraction has failed and transmits a content transmission request to the content transmission unit 219 for rule generation ( S150). The extraction failure detection unit 217 transmits a detection result indicating that re-extraction is necessary to the rule extraction unit 216.

  FIG. 10 is a flowchart showing the operation of the extraction result selection unit 224. The extraction result selection unit 224 acquires a plurality of extraction result candidates that are automatic extraction results (step S200). The extraction result selection unit 224 reads the rule from the rule holding unit 221 (step S210). The extraction result selection unit 224 calculates the similarity between each extraction result candidate and the model to be extracted in the rule (step S220). The extraction result selection unit 224 determines whether there is an extraction result candidate whose calculated similarity is equal to or greater than a threshold (step S230).

  If the extraction result selection unit 224 determines that there is an extraction result candidate whose similarity is equal to or greater than the threshold (step S230: Yes), the extraction result having a high similarity for rule generation (specifically, an optimal extraction result) Is transmitted to the rule generation unit 225 (S240).

  On the other hand, if the extraction result selection unit 224 determines that there is no extraction result candidate whose similarity is equal to or higher than the threshold (step S230: No), the extraction result selection unit 224 determines that the rule regeneration has failed, and performs rule regeneration and re-extraction. Give up (step S250).

  As described above, according to the present invention, since the wrapper automatically follows the change on the Web application side, the maintenance cost of the wrapper can be reduced. At the same time, it becomes possible to reduce the inoperability of the wrapper and reduce the service stop time.

  It should be noted that a program for realizing each process of the extraction device 20 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed, whereby the extraction device 20 is executed. The various processes described above may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

It is a lineblock diagram of extraction system 1 concerning an embodiment of the present invention. It is a figure for demonstrating the structure of the extracting device 20 shown in FIG. It is a block diagram of the extraction mechanism 210 by the rule shown in FIG. It is a block diagram of the rule production | generation mechanism 220 shown in FIG. FIG. 6 is a sequence diagram showing an operation of the extraction device 20. It is a sequence diagram which shows operation | movement of the extraction mechanism 210 by a rule. It is a sequence diagram which shows operation | movement of the extraction mechanism 210 by a rule. FIG. 6 is a sequence diagram showing an operation of a rule generation mechanism 220. 10 is a flowchart showing the operation of an extraction failure detection unit 217. It is a flowchart which shows operation | movement of the extraction result selection part 224.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Extraction system 10 Request transmission / result reception apparatus 20 Extraction apparatus 30 Content provision apparatus 200 Rule holding mechanism 210 Rule extraction mechanism 211 Rule holding section 212 Request reception section 213 Request assembly section 214 Request transmission section 215 Content holding section 216 Extraction by rule Unit 217 extraction failure detection unit 218 content transmission unit 219 extraction result transmission unit 220 rule generation mechanism 221 rule holding unit 222 content reception unit 223 specific part automatic extraction unit 224 extraction result selection unit 225 rule generation unit 226 rule update unit

Claims (8)

When the extraction request is received from a request transmission / result receiving apparatus that transmits an extraction request for requesting extraction of a specific part in the content, the content transmission is performed by transmitting the content request to a content transmission apparatus that holds the content. An extraction device that receives content related to the extraction request from a device, extracts a specific part of the content, and transmits the extracted result to the request transmission / result reception device,
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism that generates a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
When the rule generation mechanism generates a new rule, the extraction device re-extracts the specific portion according to the new rule. The extraction mechanism by the rule is:
Based on the similarity between the extraction result extracted from the content received from the content providing device according to the rule and the template of the predetermined extraction result according to the rule, it is determined whether or not the extraction of the specific part has failed,
The rule generation mechanism is:
Among the plurality of extraction result candidates automatically extracted from the content received from the content providing apparatus, the extraction result candidate having the maximum similarity with the template of the predetermined extraction result by the rule is selected as the optimum extraction result. The extraction apparatus according to claim 1, wherein a rule for extracting the optimum extraction result is generated as a new rule. The extraction mechanism by the rule is:
A rule holding unit that obtains the rule from the rule holding mechanism and updates and distributes the rule;
A request receiving unit that receives the extraction request from the request transmission / result reception device;
A request assembling unit that converts the extraction request received by the request receiving unit into the content request in accordance with the rule acquired by the rule holding unit;
A request transmitting unit that transmits the content request converted by the request assembling unit to the content providing device;
A content holding unit that receives, holds, and distributes content corresponding to the content request from a content providing device;
An extraction unit based on a rule for extracting a specific part from the content received by the content holding unit according to the rule acquired by the rule holding unit;
An extraction failure detection unit that determines whether or not extraction of a specific part by the extraction unit according to the rule has failed;
When the extraction failure detection unit determines that the extraction of the specific part has not failed, an extraction result transmission unit that returns the extraction result by the extraction unit according to the rule to the request transmission / result reception device;
A content transmission unit that acquires the content from the content holding unit and transmits it to the rule generation mechanism when the extraction failure detection unit determines that the extraction of the specific part has failed;
The extraction part according to the rule is:
The extraction device according to claim 1, wherein the specific portion is re-extracted according to the new rule generated by the rule generation mechanism based on the content transmitted by the content transmission unit. The extraction part according to the rule is:
When the specific part is extracted, the extraction result is transmitted to the extraction failure detection unit, and when the detection result indicating that the extraction is successful is received from the extraction failure detection unit, the extraction result is transmitted to the extraction result transmission unit. When the detection result indicating that re-extraction is necessary is received from the extraction failure detection unit, the specific part is re-extracted according to the new rule,
The extraction failure detection unit
When the extraction result is received from the extraction unit based on the rule, the similarity is obtained by obtaining a rule from the rule holding unit and calculating a similarity between a template of a predetermined extraction result based on the rule and the extraction result When the degree is equal to or greater than a threshold value, it is determined that the extraction of the specific part has not failed, and the detection result indicating that the extraction is successful is returned to the extraction unit according to the rule, and the similarity is less than the threshold value In this case, it is determined that the extraction of the specific part has failed, the content transmission request is transmitted to the content transmission unit to regenerate the rule, and the detection result indicating that the re-extraction is necessary is extracted by the rule. Send to the department,
The content transmitting unit
The extraction apparatus according to claim 3, wherein when the content transmission request is received, the content is acquired from the content holding unit and transmitted to the rule generation mechanism. The rule generation mechanism is:
A rule holding unit that reads a rule from the rule holding mechanism and distributes the rule;
A content receiver that receives the content from the extraction mechanism according to the rule;
A specific part automatic extraction unit for automatically extracting extraction result candidates of the specific part of the content received by the content receiving unit;
The similarity is obtained by reading a rule from the rule holding unit and calculating a similarity between a template of a predetermined extraction result based on the rule and each of a plurality of extraction result candidates automatically extracted by the specific part automatic extraction unit. An extraction result selection unit that selects an extraction result candidate having the maximum degree as an optimum extraction result;
A rule generation unit that generates a new rule for extracting the optimum extraction result selected by the extraction result selection unit;
The extraction device according to claim 1, further comprising: a rule update unit that updates the rule by transmitting a new rule generated by the rule generation unit to the extraction mechanism based on the rule and a rule holding mechanism. An extraction system that extracts a specific part of content,
A content providing device that transmits content held in response to a content request for requesting content;
A request transmission / result reception device for transmitting an extraction request for requesting extraction of a specific part in content and receiving an extraction result for the extraction request;
Receiving the extraction request from the request transmission / result receiving device, transmitting a content request for requesting content related to the extraction request to the content providing device, extracting a specific part of the content received from the content providing device; An extraction device that transmits to the request transmission / result reception device as an extraction result,
The extraction device comprises:
A rule holding mechanism for holding a rule for responding to the extraction request;
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus , An extraction mechanism based on a rule for extracting a specific part according to the rule,
A rule generation mechanism for generating a new rule when extraction of a specific part by the extraction mechanism by the rule fails;
The extraction mechanism by the rule is:
An extraction system, wherein when the rule generation mechanism generates a new rule, a specific portion is re-extracted according to the new rule. A content transmission apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific part in content, and that holds the content when the extraction request is received from a request transmission / result reception apparatus that transmits the extraction request Content by the extraction device that receives the content related to the extraction request from the content transmission device, extracts a specific part of the content, and transmits it to the request transmission / result reception device as an extraction result An extraction method for extracting a specific part of
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus An extraction step according to a rule for extracting a specific part according to the rule;
A rule generation step for generating a new rule when the extraction of the specific part by the extraction step by the rule fails,
The extraction step according to the rule includes:
When the rule generation step generates a new rule, a specific portion is re-extracted according to the new rule. A content transmission apparatus that holds a rule for responding to an extraction request for requesting extraction of a specific part in content, and that holds the content when the extraction request is received from a request transmission / result reception apparatus that transmits the extraction request A computer of the extraction device that receives the content related to the extraction request from the content transmission device, extracts a specific part of the content, and transmits it as an extraction result to the request transmission / result reception device In addition,
From the content received from the content providing apparatus, receiving the extraction request from the request transmission / result receiving apparatus, converting the extraction request into the content request according to the rule, transmitting the content request to the content providing apparatus An extraction step according to a rule for extracting a specific part according to the rule;
When the extraction of the specific part by the extraction step by the rule fails, the rule generation step for generating a new rule is executed,
The extraction step according to the rule includes:
When the rule generation step generates a new rule, a specific part is re-extracted according to the new rule.

Images