JP2005173671A - Device, method and program for link diagnosis - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically detect the logical inconsistency of a link in a hypertext. <P>SOLUTION: A link feature extracting means 22 extracts, as link feature information showing a logical inconsistence likelihood of a link, (A) link feature information which can be found from the link alone, (B) link feature information which can be obtained based upon the relation between the link and document data at a link destination, or (C) link feature information which can be obtained when links are grouped according to a specified condition. An inconsistency learning means 23 obtains a discrimination function by statistically calculating the relation between contents of respective pieces of link feature information and the rate of links decided as inconsistent links. An inconsistency determination means 25 determines whether a link to be decided is inconsistent by using the link feature information on an undecided link and the discrimination function obtained by the inconsistency learning means 23. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a link diagnosis apparatus, a link diagnosis method, and a link diagnosis program applied to a hypertext management apparatus, and in particular, a link diagnosis apparatus, a link diagnosis method, and a link that detect a logical inconsistency in link source notation and link relation. Regarding diagnostic programs.

  In recent years, with the increase in the number of Internet users, the websites operated by each company have become larger and are being updated rapidly. As the site size increases and the update frequency increases, inconsistencies such as link errors between files tend to occur.

  There are two types of link mismatch: physical mismatch and logical mismatch. A physical inconsistency is an inconsistency in which some error occurs when a linked document is accessed. For example, when a linked document is accessed as a physical inconsistency, an error “404 Not Found”, “403 Forbidden”, or “501 Internal Server Error” from an HTTP (HyperText Transfer Protocol) server may occur. is there. In addition, for example, a physical inconsistency may include a time-out error when the network or server is congested, or a DNS (Domain Name Server) error caused by attempting to access a server name that does not exist. .

  The logical inconsistency is an inconsistency in which an error does not occur when a linked document is accessed, but a document having a content different from the content expected by the user from the link source notation is accessed. Logical inconsistency occurs when linked to incorrect product information or when the link source notation is not consistent.

  As a technique for detecting a hypertext link relation error, Patent Document 1 discloses a WWW server link that hierarchically searches for a document link in a WWW server and detects an error such as a spelling error by accessing a link destination. An information check system is described. In the link information check system described in Patent Document 1, the data processing apparatus hierarchically checks hypertext links and detects an error location depending on whether an error has occurred when the link destination is accessed.

  Patent Document 2 describes a document feature extraction apparatus that generates a feature vector of a link source document based on feature vectors of a plurality of documents. The document feature extraction apparatus described in Patent Document 2 divides a link source document and a plurality of link destination documents into cluster units based on the similarity of feature vectors. Then, the document feature extraction apparatus generates a feature vector of the link source document using only the feature vector of the document included in the cluster including the link source document among the plurality of clusters.

  Patent Document 3 describes an internal structure estimation apparatus for a website that can easily search for information based on the internal structure of the website. The internal structure estimation device described in Patent Literature 3 extracts link meta information from each page. The internal structure estimation device learns based on the set of meta information and link type, and acquires a link classification tree. Then, the internal structure estimation device estimates the internal structure of the website using the acquired link classification tree.

JP-A-11-85801 (page 3-4, FIG. 1-3) JP 2000-181936 A (page 3-8, FIG. 1-6) Japanese Patent Laid-Open No. 2003-186883 (pages 8-11 and 1-9)

  According to the link information check system described in Patent Document 1, it is possible to automatically detect a physical mismatch. However, since link inconsistency is detected based on whether an error occurs when accessing the linked document, logical inconsistency that does not cause an error when accessing the linked document is automatically detected. I can't do it.

  Further, according to the link information check system described in Patent Document 1, it is possible to detect a mismatched portion of the link, but it is not possible to specify the cause of the mismatched link. For this reason, the inconsistent portion is corrected in a coping therapy.

  In addition, the document feature extraction apparatus described in Patent Document 2 merely generates a feature vector of a link source document, classifies the document using the generated feature vector, and there is no link between documents. A match cannot be detected. Furthermore, in the internal structure estimation device described in Patent Document 3, in order to acquire a link classification tree and perform information retrieval for each Web site, the internal structure of the Web site is estimated using the acquired link classification tree. It is just doing and it cannot detect link inconsistencies.

  Accordingly, the present invention has been made to solve the above-described problems, and provides a link diagnosis apparatus, a link diagnosis method, and a link diagnosis program that can automatically detect a logical mismatch of hypertext links. For the purpose. The present invention also provides a link diagnosis apparatus and a link diagnosis that analyze a cause of link mismatch based on a tendency of occurrence of link mismatch and analyze not only a mismatched part but also a site construction problem. It is an object to provide a method and a link diagnostic program.

  The link diagnosis apparatus according to the present invention is the link information of the link information that indicates whether or not inconsistency occurs in the link information indicating the attribute of the hypertext link that associates the designated document with the designated document. Undecided link information storage means for storing decision link information, undecided link information reading means for reading the undetermined link information from the undetermined link information storage means, and undetermined link information read by the undetermined link information reading means Based on the link feature extraction means for extracting link feature information indicating the probability that a mismatch has occurred in the link, and the link feature information of a link for which a determination has been made as to whether or not a mismatch has occurred. Relation information storage means for storing relation information indicating the relation between the judgment result and the judgment result of the judgment, and the link feature extracted by the link feature extraction means By using the broadcast and related information, characterized by comprising a mismatch determination means for determining whether or not an inconsistency link determine undetermined occurs. The link attribute is, for example, a link source notation that is a notation of a link part of a link source document, an address that indicates the location of the link source document, and an address that indicates the location of the link destination document.

  In addition, the link diagnosis apparatus is undecided, which is link information of a link that has not been determined whether or not inconsistency occurs in the link information indicating the attribute of the hypertext link that associates the designated document with the designated document. Undecided link information storage means for storing link information, determined link information that is link information of a link for which inconsistency has already occurred in the link information, a determination result of determination, and Determined link information storage means for storing, undecided link information reading means for reading undetermined link information from the undetermined link information storage means, and determined link information and determination results read from the determined link information storage means Based on the link information reading means and the undetermined link information read by the undetermined link information reading means, the link is inconsistent. Link feature extraction means for extracting link feature information indicating the likelihood of being linked, and extracting link feature information based on the determined link information read by the determined link information reading means; and a link feature extraction means, Link features of links that have already been determined whether or not inconsistency has occurred using the link feature information extracted based on the determined link information and the determination result read by the determined link information reading means. Relation information deriving means for deriving relation information indicating the relation between the information and the determination result of the judgment, relation information storage means for storing the relation information derived by the relation information deriving means, and link feature extraction means for undetermined link information It is not possible to determine whether or not there is a mismatch in an undetermined link using the link feature information and relationship information extracted based on It is those with a focus determination unit may. According to such a configuration, it is possible to obtain relationship information based on the determined link information, and easily determine whether or not there is a mismatch in the determination target link using the obtained relationship information.

  Further, the link diagnosis apparatus includes a feedback unit that stores the determination result for the undecided link by the inconsistency determining unit and the link information of the link in the determined link information storage unit, and the feedback unit Outputs the judgment result of the judgment for the link, prompts the user to confirm whether the judgment result is correct, inputs the confirmation result of confirmation from the user, and determines the link information and judgment result based on the confirmation result The determined link information reading means stores the determined link information and the determination result from the determined link information storage means after the feedback means stores the link information and the determination result in the determined link information storage means. The link feature extraction means reads again by the determined link information reading means. The link feature information is extracted again based on the determined determined link information, and the relation information deriving means re-extracts the link feature information extracted based on the determined link information by the link feature extracting means, and the determined link information reading means. The relationship information may be derived again using the determination result read again. According to such a configuration, the relationship information can be updated by reflecting the user confirmation result, and the logical inconsistency of the link can be automatically detected using the optimum relationship information.

  The link diagnosis apparatus further includes feedback means for storing the determination result for the undecided link by the inconsistency determining means and the link information of the link in the determined link information storage means, and the feedback means is an undetermined link. Each time a determination is made, the determination result is output, prompting the user to confirm whether the determination result is correct, the confirmation result of confirmation is input from the user, and the link information and the determination result are obtained based on the confirmation result. The determined link information storage means stores the determined link information reading means, and the feedback means stores the link information and the determination result in the determined link information storage means, and then determines the determined link information from the determined link information storage means. Then, the determination result is read again, and the link feature extraction means uses the determined link information reading means. The link feature information is extracted again based on the determined link information read again, and the relationship information deriving unit re-extracts the link feature information extracted based on the determined link information by the link feature extracting unit, and the determined link information. Using the determination result read again by the reading means, the relationship information is derived again, and the undetermined link information reading means is updated each time new undetermined link information is added to the undetermined link information storage means. The undetermined link information is read, the link feature extracting means extracts the link feature information based on the new undetermined link information read by the undetermined link information reading means, and the inconsistency determining means includes the link feature extracting means. Using link feature information extracted based on new undecided link information and relationship information derived again by relationship information deriving means , It may be configured to determine whether an inconsistency link undetermined occurs. According to such a configuration, the logical inconsistency of the link can be automatically detected automatically using the relationship information reflecting the updated content every time the determined link information is updated. Further, every time undetermined link information is added, a logical inconsistency of the link can be automatically detected in real time.

  In addition, the link diagnosis apparatus includes an optimum combination determination unit that determines a combination of the link feature information and the relationship information derivation algorithm that maximizes the accuracy of the determination result of whether or not the link is inconsistent. Causes the determined link information reading means to read the determined link information and the determination result from the determined link information storage means, and causes the link feature extraction means to extract a plurality of types of link feature information based on the determined link information. , A plurality of combinations consisting of some or all of the plurality of types of link feature information and one type of the plurality of types of relationship information deriving algorithms are determined, and the relationship information deriving means determines a combination for each individual combination. Link feature information constituting the combination according to the constituting relation information derivation algorithm, and determined link information reading means Therefore, the relation information is derived using the read judgment result, and the inconsistency judgment means derives the relation information derived according to the combination for the link for which the judgment result has already been obtained for each combination. The result of making the inconsistency determination means re-determine by comparing each of the results obtained by making the determination again using the non-match determination means and comparing the determination results already obtained for each combination. May be obtained for each combination, and the combination having the highest accuracy may be specified. According to such a configuration, the user can analyze the cause of the link inconsistency based on the tendency of occurrence of the link inconsistency and analyze not only the inconsistent portion but also the problem in the site construction. .

  In addition, the link feature extraction means, as the link feature information, the link feature information that can be specified from the link that is the target of determination as to whether or not inconsistency occurs, the description content of the link that is the target of determination and the link destination document of the link One or more types of link feature information that can be specified from the above, or link feature information that indicates the size of the group to which the link to be determined belongs when the links are grouped according to a predetermined condition Information may be extracted. According to such a configuration, the link feature information can be easily extracted based on the link source notation, the link source document, and the link destination document, and the logical inconsistency of the link can be automatically detected.

  In addition, the link feature extraction means includes a keyword included in the link source notation corresponding to the link feature information identifiable from the link to be determined, a domain name or directory name included in the link destination address, and a domain included in the link source address Included in the link source notation corresponding to link feature information that can be identified from the name or directory name, the appearance position or display area of the link source notation in the document, or the description contents of the link to be judged and the link destination document The number of times the keyword appears in the title of the linked document, the number of times the keyword included in the link source notation appears in the text of the linked document, and the number of times the keyword included in the link source notation is highlighted in the linked document Or when links are grouped according to predetermined conditions Ratio of links with the same link destination address as the link for deriving the relationship information or the link to be judged in the group of the same link source notation corresponding to the link feature information indicating the size of the group to which the link belongs In the same link destination address group, the ratio of links that have the same link source notation as the link for deriving the relationship information or the link to be judged, in the same link source address and the same link source notation group The ratio of the link for deriving the relationship information or the link having the same link destination address as the link to be judged, the link for deriving the relationship information in the group of the same link source address and the same link destination address Or one of the ratios of links that have the same link source notation as the link to be judged It may be one for extracting a plurality of types of information. According to such a configuration, it is possible to determine whether or not there is a spelling error in the link source notation of the determination target link. Further, the validity of the link destination address can be determined. In addition, it is possible to determine link inconsistency derived from the management status of the user who manages the site. Further, it is possible to determine whether or not there is a mismatch based on the conspicuousness in the document of the link source notation. It is also possible to determine whether or not the correspondence between the link source notation and the link destination document is correct. Further, it is possible to determine inconsistency of a link that refers to an incorrect document with the same name. It is also possible to determine inconsistencies in links that refer to the same document with an incorrect name. Further, it is possible to determine inconsistency of links that refer to different documents with the same name from the same document. Furthermore, it is possible to determine inconsistencies in links that refer to the same document from the same document with different names.

  In the link diagnosis method according to the present invention, the undetermined link information storage unit is undetermined whether or not inconsistency occurs in the link information indicating the hypertext link attribute that associates the designated document with the designated document. The undetermined link information that is the link information of the link is stored, the undetermined link information reading means reads the undetermined link information from the undetermined link information storage means, and the link feature extracting means is read by the undetermined link information reading means. Based on the read undetermined link information, link feature information indicating the probability of inconsistency in the link is extracted, and the relationship information storage means has already determined whether or not inconsistency has occurred. Relationship information indicating the relationship between the link feature information of the link being performed and the determination result of the determination is stored, and the inconsistency determination unit is operated by the link feature extraction unit. With out link feature information and the related information, and judging whether or not an inconsistency link undetermined occurs.

  Further, in the link diagnosis method, the undetermined link information storage means is undetermined whether or not inconsistency occurs in the link information indicating the attribute of the hypertext link that associates the designated document with the designated document. The undetermined link information that is link information of the link is stored, and the determined link information storage means is determined to be link information of a link that has already been determined whether or not inconsistency occurs in the link information. The link information and the determination result of the determination are stored, the undetermined link information reading means reads the undetermined link information from the undetermined link information storage means, and the determined link information reading means is read from the determined link information storage means. The determined link information and the determination result are read, and the link feature extraction means also applies to the undetermined link information read by the undetermined link information reading means. Therefore, link feature information indicating the likelihood of inconsistency in the link is extracted, link feature information is extracted based on the determined link information read by the determined link information reading means, and the relationship information The derivation unit uses the link feature information extracted by the link feature extraction unit based on the determined link information and the determination result read by the determined link information reading unit to determine whether or not there is a mismatch. Deriving relationship information indicating the relationship between the link feature information of the link already performed and the determination result of the determination, the relationship information storage unit stores the relationship information derived by the relationship information deriving unit, and the mismatch determination unit However, using the link feature information extracted by the link feature extraction means based on the undetermined link information and the relationship information, an inconsistency occurs in the undetermined link. Luke may be configured to determine whether. According to such a configuration, it is possible to obtain relationship information based on the determined link information, and easily determine whether or not there is a mismatch in the determination target link using the obtained relationship information.

  In the link diagnosis method, the feedback means outputs the determination result of the undecided link by the inconsistency determination means, prompts the user to check whether the determination result is correct, and confirms the confirmation result from the user. The feedback means stores the determination result of the determination for the undetermined link and the link information of the link in the determined link information storage means based on the confirmation result, and the determined link information reading means returns the feedback. After the means stores the link information and the determination result in the determined link information storage means, the determined link information and the determination result are read again from the determined link information storage means, and the link feature extraction means reads the determined link information reading means. The link feature information is extracted again based on the determined link information read again by The relation information deriving means derives the relation information again using the link feature information extracted again by the link feature extracting means based on the determined link information and the determination result read again by the determined link information reading means. It may be. According to such a configuration, the relationship information can be updated by reflecting the user confirmation result, and the logical inconsistency of the link can be automatically detected using the optimum relationship information.

  In the link diagnosis method, the feedback unit outputs a determination result each time a determination is made on an undecided link by the inconsistency determination unit, and prompts the user to confirm whether the determination result is correct. The confirmation result of confirmation is input, and the feedback means stores the judgment result of judgment for the undecided link and the link information of the link in the judged link information storage means based on the confirmation result, and reads the judged link information. After the feedback means stores the link information and the determination result in the determined link information storage means, the determined link information and the determination result are read again from the determined link information storage means, and the link feature extraction means is determined Link feature information based on the determined link information read again by the link information reading means The relationship information deriving unit extracts the relationship information using the link feature information extracted again by the link feature extracting unit based on the determined link information and the determination result read again by the determined link information reading unit. Derived again, the undetermined link information reading means reads new undetermined link information each time new undetermined link information is added to the undetermined link information storage means, and the link feature extracting means reads the undetermined link information. Link feature information is extracted based on the new undecided link information read by the information reading means, the inconsistency determining means is the link feature information extracted by the link feature extracting means based on the new undetermined link information, and Using the relationship information derived again by the relationship information deriving means, it is determined whether or not an inconsistent link is inconsistent. Good. According to such a configuration, the logical inconsistency of the link can be automatically detected automatically using the relationship information reflecting the updated content every time the determined link information is updated. Further, every time undetermined link information is added, a logical inconsistency of the link can be automatically detected in real time.

  In the link diagnosis method, the optimum combination determination unit causes the determined link information reading unit to read the determined link information and the determination result from the determined link information storage unit, and the link feature extraction unit converts the determined link information into the determined link information. Based on a plurality of types of link feature information extracted, a plurality of combinations consisting of some or all of the plurality of types of link feature information and one type of a plurality of types of relationship information derivation algorithms are determined to derive relationship information. According to the relation information derivation algorithm that constitutes the combination for each individual combination, the means derives the relation information using the link feature information that constitutes the combination and the determination result read by the determined link information reading means, For the inconsistency judgment means, depending on the combination for each link for which a judgment result has already been obtained By making a determination again using the derived relationship information and comparing the result of the determination made again by the inconsistency determination means with the already obtained determination result for each combination, the inconsistency determination means The accuracy of the result determined again may be obtained for each combination, and the combination having the highest accuracy may be specified. According to such a configuration, the user can analyze the cause of link inconsistency based on the tendency of occurrence of link inconsistency, and can analyze not only the inconsistent portion but also problems in site construction.

  Further, in the link diagnosis method, the optimum combination determining means is configured such that each combination including a part or all of a plurality of types of link feature information and one type of a plurality of types of relation information derivation algorithms, A list of the accuracy of the result determined again by the inconsistency determination means may be displayed. According to such a configuration, the user can know a combination of an algorithm with high determination accuracy and link feature information without trial and error in the statistical calculation processing phase.

  In addition, the link diagnosis method is such that the link feature extraction unit can identify link feature information that can be identified from the link that is the target of determination as to whether or not inconsistency has occurred, and the link that is the target of determination and the link destination of the link. One type of link feature information that can be identified from the description content of the document, or link feature information that indicates the size of the group to which the link to be judged belongs when the links are grouped according to a predetermined condition, or A plurality of types of link feature information may be extracted. According to such a configuration, the link feature information can be easily extracted based on the link source notation, the link source document, and the link destination document, and the logical inconsistency of the link can be automatically detected.

  In addition, the link diagnosis method is such that the link feature extraction means includes a keyword included in the link source notation corresponding to the link feature information that can be specified from the link to be determined, the domain name or directory name included in the link destination address, the link source Corresponds to link feature information that can be identified from the domain name or directory name included in the address, the appearance position or display area of the link source notation in the document, or the description contents of the link to be judged and the link destination document. The number of times the keyword included in the link source notation appears in the title of the linked document, the number of times the keyword included in the link source notation appears in the text of the linked document, and the keyword included in the link source notation is highlighted in the link destination document Group links according to the number of times displayed or given conditions In the same link source notation group corresponding to the link feature information indicating the size of the group to which the link to be judged belongs, the same link destination address as the link for deriving the relationship information or the link to be judged Of links with the same link source address, the ratio of links with the same link source notation as the link for deriving the relationship information or the link to be judged within the same link destination address group, the same link source address and the same link source The ratio of links with the same link destination address as the link for deriving the relationship information or the link to be judged within the group indicated, the relationship information within the group with the same link source address and the same link destination address Of the percentage of links that have the same link source notation as the link to be derived or the link to be judged It may be configured to extract either one or more types of information. According to such a configuration, it is possible to determine whether or not there is a spelling error in the link source notation of the determination target link. Further, the validity of the link destination address can be determined. In addition, it is possible to determine link inconsistency derived from the management status of the user who manages the site. Further, it is possible to determine whether or not there is a mismatch based on the conspicuousness in the document of the link source notation. It is also possible to determine whether or not the correspondence between the link source notation and the link destination document is correct. Further, it is possible to determine inconsistency of a link that refers to an incorrect document with the same name. It is also possible to determine inconsistencies in links that refer to the same document with an incorrect name. Further, it is possible to determine inconsistency of links that refer to different documents with the same name from the same document. Furthermore, it is possible to determine inconsistencies in links that refer to the same document from the same document with different names.

  In the link diagnosis method, the relationship information deriving means is designated by the user the type of the link feature information, the type of the relationship information deriving algorithm, and the parameter applied to the relationship information deriving algorithm. The link feature information of the type specified by the user is extracted, and the relationship information deriving means is read by the link feature information specified by the user and extracted based on the determined link information and the determined link information reading means. Using the determination result, the relationship information is derived in accordance with the relationship information derivation algorithm and parameters designated by the user, the relationship information storage means stores the relationship information derived by the relationship information derivation means, and the inconsistency determination means Link characteristic information specified by the user and extracted based on the undetermined link information and related information. There are, may be configured to determine whether an inconsistency link undetermined occurs. According to such a configuration, the user can confirm the determination accuracy when each algorithm is used, and can automatically select an algorithm with high determination accuracy while performing trial and error.

  In the link diagnosis method, the inconsistency determination means determines whether or not there is an inconsistency in an undetermined link using the link feature information and the relationship information, and the reliability indicating the reliability of the determination result. And the determination result and the certainty factor may be displayed for each link for which the determination has been made. According to such a configuration, the user can easily determine which link is inconsistent and which link should be checked by confirming the certainty factor.

  Further, the link diagnosis method may display the link information of each link determined by the inconsistency determination unit in a color-coded manner according to the certainty factor. According to such a configuration, the user can easily determine which link is inconsistent and which link should be checked by identifying the color classification of the link information.

  The link diagnosis method may display the link information of each link for which the inconsistency determination unit has performed the determination by rearranging the certainty factor as a sort key. According to such a configuration, the user can check link information in order of certainty, so that it is easy to determine which link is inconsistent and from which link to check.

  In addition, the link diagnosis method includes a group of links in which the inconsistency determination unit has a common link source notation and the determination target link together with the determination result of the determination target link whether or not inconsistency occurs. A group of links having the same link destination address and a link to be determined, a group of links having the same link source address and link source notation, a link to be determined, a link source address and a link destination address Any one of a group of common links or any of a plurality of groups may be displayed. According to such a configuration, the user can easily determine how to correct the portion determined to be inconsistent.

  In the link diagnosis method, the inconsistency determination means displays a group constituted by each link included in the document that is the link source of the link determined to have inconsistency, and the user designates the document. When the input is made and the designation is inputted, the document may be displayed and the location of the link where the inconsistency is generated may be specified and displayed. According to such a configuration, the user can quickly grasp where inconsistency exists in the document.

  The link diagnosis program according to the present invention is the link information of the link information indicating whether or not there is a mismatch among the link information indicating the hypertext link attribute that associates the specified document with the specified document. The undecided link information storage means for storing the judgment link information and the relation information indicating the relation between the link feature information of the link for which the judgment as to whether or not inconsistency has already been made and the judgment result of judgment are stored The process of reading the undetermined link information from the undetermined link information storage means to the computer provided with the relationship information storage means, and the probability that the link is inconsistent based on the read undetermined link information. Whether there is an inconsistency in an undetermined link using the extracted link feature information and the extracted link feature information and relationship information Characterized in that to execute processing of determining.

  In addition, the link diagnosis program is undecided, which is link information of an undecided link in the link information indicating the hypertext link attribute that associates the designated document with the designated document. Undecided link information storage means for storing link information, determined link information that is link information of a link for which inconsistency has already occurred in the link information, a determination result of determination, and And a relation information storage for storing relation information indicating a relation between link feature information of a link for which a determination as to whether or not inconsistency has already occurred and a determination result of the determination is performed. A process of reading the undetermined link information from the undetermined link information storage means, and the determined link information from the determined link information storage means. And a process of reading the determination result, a process of extracting link feature information indicating the probability that the link is inconsistent based on the read undetermined link information, and based on the read determined link information Processing for extracting link feature information, processing for deriving relationship information using link feature information extracted based on determined link information, and read determination result, and extraction based on undetermined link information Using the link feature information and the relationship information, it may be possible to execute a process for determining whether or not an indeterminate link is inconsistent. According to such a configuration, it is possible to obtain relationship information based on the determined link information, and easily determine whether or not there is a mismatch in the determination target link using the obtained relationship information.

  In addition, the link diagnosis program outputs a determination result of the determination for the undetermined link to the computer, prompts the user to confirm whether or not the determination result is correct, and receives a confirmation result from the user. Processing for storing the determination result of the determination for the undetermined link and the link information of the link in the determined link information storage unit based on the confirmation result, and storing the link information and the determination result in the determined link information storage unit Later, processing for re-reading the determined link information and determination result from the determined link information storage means, processing for re-extracting link feature information based on the re-read determined link information, and again based on the determined link information The relationship information is derived again using the extracted link feature information and the determination result read again. Or it may be to execute the management. According to such a configuration, the relationship information can be updated by reflecting the user confirmation result, and the logical inconsistency of the link can be automatically detected using the optimum relationship information.

  The link diagnosis program outputs a determination result to the computer every time a determination is made for an undetermined link, prompts the user to check whether the determination result is correct, and inputs the confirmation result from the user. Processing, the determination result of the determination for the undetermined link and the link information of the link stored in the determined link information storage unit based on the confirmation result, the link information and the determination result of the determined link information storage unit , The process of re-reading the determined link information and the determination result from the determined link information storage unit, the process of re-extracting the link feature information based on the re-read determined link information, and the determined link information Using the link feature information re-extracted from the original and the determination result read again, the relationship information is derived again. Processing, and whenever new undetermined link information is added to the undetermined link information storage means, new undetermined link information is read, and link feature information is extracted based on the read new undetermined link information. And a process for determining whether or not there is a mismatch in the undetermined link using the link feature information extracted based on the new undetermined link information and the relationship information derived again. It may be allowed. According to such a configuration, the logical inconsistency of the link can be automatically detected automatically using the relationship information reflecting the updated content every time the determined link information is updated. Further, every time undetermined link information is added, a logical inconsistency of the link can be automatically detected in real time.

  Further, the link diagnosis program causes the computer to execute an optimum combination determination process for determining a combination of the link feature information and the relationship information derivation algorithm that gives the highest accuracy of the determination result as to whether or not the link is inconsistent, In the optimum combination determination process, a process of reading the determined link information and the determination result from the determined link information storage unit, a process of extracting a plurality of types of link feature information based on the determined link information, a plurality of types of link feature information A process for determining a plurality of combinations consisting of some or all of them and one of a plurality of types of relationship information deriving algorithms, and a link that configures a combination according to a relationship information deriving algorithm that forms a combination for each individual combination The relationship information is derived using the feature information and the read judgment result. Processing, processing for making a determination again using the relationship information derived according to the combination, and a result of the determination again for each link for which a determination result has already been obtained The result of the determination again is obtained for each combination by performing a comparison with the determination result for each combination, and a process for specifying the combination with the highest accuracy is executed. Good. According to such a configuration, the user can analyze the cause of link inconsistency based on the tendency of occurrence of link inconsistency, and can analyze not only the inconsistent portion but also problems in site construction.

  In addition, the link diagnosis program, on the computer, as link feature information, link feature information that can be specified from the link that is the target of determination as to whether or not inconsistency has occurred, the link that is the target of determination, and the link destination document of the link One or more types of link feature information that can be identified from the description content, or link feature information that indicates the size of the group to which the link to be determined belongs when the links are grouped according to a predetermined condition A process of extracting link feature information may be executed. According to such a configuration, the link feature information can be easily extracted based on the link source notation, the link source document, and the link destination document, and the logical inconsistency of the link can be automatically detected.

  The link diagnosis program is also included in the keyword included in the link source notation corresponding to the link feature information identifiable from the link to be determined, the domain name or directory name included in the link destination address, and the link source address. Link source notation corresponding to link feature information that can be identified from the domain name or directory name to be displayed, the appearance position or display area of the link source notation in the document, or the description contents of the link to be judged and the link destination document The number of times the keyword included in the URL appears in the title of the linked document, the number of times the keyword included in the link source notation appears in the text of the linked document, and the keyword included in the link source notation is highlighted in the linked document. Group links according to the number of times or according to predetermined conditions In the same link source notation group corresponding to the link feature information indicating the size of the group to which the link to be judged belongs, the same link destination address as the link for deriving the relationship information or the link to be judged Of links with the same link source address, the ratio of links with the same link source notation as the link for deriving the relationship information or the link to be judged within the same link destination address group, the same link source address and the same link source The ratio of links with the same link destination address as the link for deriving the relationship information or the link to be judged within the group indicated, the relationship information within the group with the same link source address and the same link destination address Of the percentage of links that have the same link source notation as the link to be derived or the link to be judged Or it may be to execute a process of extracting any one or more types of information. According to such a configuration, it is possible to determine whether or not there is a spelling error in the link source notation of the determination target link. Further, the validity of the link destination address can be determined. In addition, it is possible to determine link inconsistency derived from the management status of the user who manages the site. Further, it is possible to determine whether or not there is a mismatch based on the conspicuousness in the document of the link source notation. It is also possible to determine whether or not the correspondence between the link source notation and the link destination document is correct. Further, it is possible to determine inconsistency of a link that refers to an incorrect document with the same name. It is also possible to determine inconsistencies in links that refer to the same document with an incorrect name. Further, it is possible to determine inconsistency of links that refer to different documents with the same name from the same document. Furthermore, it is possible to determine inconsistencies in links that refer to the same document from the same document with different names.

  According to the present invention, link feature information is extracted based on link information indicating link attributes. Further, paying attention to the extracted link feature information, it is determined whether or not the undetermined link is inconsistent by using relationship information indicating the relationship between the link feature information and the determination result determined as inconsistent. Therefore, it is possible to automatically detect a logical mismatch of hypertext links.

Embodiment 1 FIG.
A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of a link diagnosis apparatus according to the present invention. As shown in FIG. 1, the link diagnosis apparatus includes an input device 11 such as a keyboard and a mouse, an output device 12 such as a display device and a printing device, a data processing device 20 that operates according to a program, and a memory that stores information. Device 30.

  The storage device 30 is realized by a magnetic disk device, for example. As illustrated in FIG. 1, the storage device 30 includes an undetermined link information storage unit 31, a determined link information storage unit 32, a link feature storage unit 33, and a learning result storage unit 34.

  The undetermined link information storage unit 31 is undetermined whether there is an inconsistency in the link between documents among the link information extracted from the hypertext including a plurality of documents correlated by the link (hyperlink). Store link information. The link information is information indicating a hypertext link attribute that associates an access destination (designation destination) document with an access source (designation source) document. In the present embodiment, the undetermined link information storage unit 31 uses, as link information, a link source notation that is a notation of a link part on the link source document, a link source address indicating the location of the link source document, and a link destination document. The link destination address indicating the location is stored. The undetermined link information storage unit 31 obtains, as link information, the link target attribute, style attribute, text of the link source document and link destination document, update date, acquisition date, and whether an error has occurred. Information indicating the state of the above may be stored.

  Further, the undetermined link information storage unit 31 stores a link ID for identifying a link between documents constituting the hypertext in association with the link information. In the present embodiment, a case will be described in which the undetermined link information storage unit 31 stores identification information such as “L131” and “L141” for each link in advance as a link ID. The identification information to which the link ID is assigned is not limited to the information in the format shown in this embodiment. For example, the document ID “xxxx” may be assigned to the document, and the link ID “xxxxyyy” may be assigned to the yyy th link corresponding to the document.

  The determined link information storage unit 32 stores the link information of the link that has already been determined whether or not there is a mismatch between the documents, the link ID, and the determination result whether or not there is a mismatch. . Hereinafter, the determination result of whether or not there is an inconsistency is referred to as an inconsistency determination result. The determined link information storage unit 32 may store a past determination result of a site to be determined as a mismatch determination result, or store a past determination result of a site different from the determination target site. May be.

  The link feature storage unit 33 stores link feature information extracted by a link feature extraction unit 22 described later. Further, the learning result storage unit 34 stores a learning result obtained by the inconsistency learning unit 23 described later.

  The data processing device 20 includes a determined link information access unit 21, a link feature extraction unit 22, an inconsistency learning unit 23, an undetermined link information access unit 24, an inconsistency determination unit 25, and a result reflection unit 26.

  The determined link information access unit 21 reads the determined link information, the link ID, and the inconsistency determination result from the determined link information storage unit 32 and outputs them to the link feature extraction unit 22.

  The link feature extraction unit 22 is characterized by the probability that a logical inconsistency has occurred in the link based on the link information input from the determined link information access unit 21 or the undetermined link information access unit 24. Information (hereinafter referred to as link feature information) is extracted. The link feature extraction unit 22 stores the link ID and the extracted link feature information in the link feature storage unit 33. When link information and an inconsistency determination result are input from the determined link information access unit 21, the link feature extraction unit 22 stores the inconsistency determination result together with the link ID and the extracted link feature information as a link feature. Store in the unit 33.

  The link feature information is information indicating the probability of inconsistency in the links between documents. That is, when the link feature information has a predetermined content (for example, a value in a predetermined range), the probability that the determination target link can be determined to be inconsistent is increased. The link feature information includes “(A) link feature information that can be obtained by a link alone”, “(B) link feature information that can be obtained based on the relationship between a link and its linked document”, and “(C 3) “link feature information that can be obtained by grouping links according to predetermined conditions”.

  (A) Examples of link feature information that can be obtained by a link alone include, for example, “keyword included in link source notation”, “domain name or directory name included in link destination address”, and “link source address” Domain name or directory name ”and“ appearance position or display area of link source notation in document ”.

  The keyword included in the link source notation is a word used as a keyword among the words included in the link source notation. The keyword included in the link source notation is used to determine whether or not there is a spelling error in the link source notation of the link. For example, it is assumed that the cause of inconsistency of a certain link in determined data is a spelling error of a specific link source notation. Then, when the same keyword as the keyword included in the link source notation is used in the determination target link source notation, there is a high possibility that the determination target link is inconsistent.

  In addition, there are keywords that are easily confused as the link source notation and easily lose the consistency of the link source notation. When such a keyword is used in the determination target link source notation, there is a high possibility that the determination target link is inconsistent. For example, “new arrival information” and “event information” are both keywords used when providing new information, and are easily confused. Therefore, when the link source notation includes “new arrival information” or “event information” as keywords, there is a high possibility that the determination target link is inconsistent.

  The domain name or directory name included in the link destination address is used to determine the validity of the link destination address. For example, if a document included in a domain or directory is deleted or moved to another location due to a site renewal, etc., links that refer to the deleted or moved document may become inconsistent. Becomes higher.

  The domain name or directory name included in the link source address is used to determine link inconsistency derived from the management status of the site administrator (user) who manages the site. In general, when managing a large-scale site, a plurality of site managers often divide and perform management work. In this case, each site administrator often divides roles into domain units or directory units, so if some of the site administrators do not have sufficient management of the division area, the documents contained in the domain or directory of that division area The link from is likely to be a link mismatch.

  The appearance position or display area of the link source notation in the document is used to determine whether or not there is a mismatch based on the conspicuousness in the document of the link source notation. For example, when the appearance position of the link source notation is located at the bottom of the document and the size of the link source notation is small, the link source notation is not easily noticeable in the document. For this reason, even if the link is inconsistent, there is a high possibility that the site administrator will overlook the inconsistency and remain in an inconsistent state.

  When obtaining a keyword included in the link source notation as the link feature information, the link feature extracting unit 22 uses the link source notation included in the link information input from the determined link information access unit 21 or the undetermined link information access unit 24. , Extract keywords. When the domain name or directory name is obtained as the link feature information, the link feature extraction unit 22 extracts the domain name or directory name from the link source address or link destination address included in the input link information.

  When the appearance position or display area of the link source notation is obtained as the link feature information, the link feature extraction unit 22 reads the link source document from the link source based on the link source address included in the input link information. Then, the link feature extraction unit 22 obtains the appearance position and the display position of the link source notation in the document based on the link source notation included in the link information and the read link source document. The link feature extraction unit 22 does not read the link source document based on the link source address, but the undetermined link information storage unit 31 and the determined link information storage unit 32 store the link source document as link information in advance. Also good.

  (B) As an example of link feature information that can be obtained based on the relationship between a link and its linked document, for example, “the number of times the keyword included in the link source notation appears in the title of the linked document (hereinafter, title "Number of occurrences"), "Number of times the keyword included in the link source notation appears in the text of the linked document (hereinafter referred to as the number of occurrences of the text)", "Keyword included in the link source notation in the link destination document The number of times of highlighting (hereinafter referred to as the number of highlights). The keyword being highlighted is, for example, that the keyword is displayed in bold or in a large character size in the document.

  The number of title appearances, the number of appearances of text, and the number of highlights are used to determine whether or not the correspondence between the link source notation and the link destination document is correct. That is, as the value of the title appearance count, the text appearance count, and the highlight display count increases, the probability that the link is inconsistent decreases. Note that as the length of the document increases, the values of the title appearance count, the text appearance count, and the highlight display count generally increase. When the length of the document is long, a value obtained by dividing the number of appearances of the title, the number of appearances of the text, and the number of highlights by dividing the length of the document may be used as the link feature information.

  When obtaining the number of title appearances, the number of appearances of text, and the number of highlights, the link feature extraction unit 22 uses the link source notation included in the link information input from the determined link information access unit 21 or the undetermined link information access unit 24. Extract keywords. Further, the link feature extraction unit 22 reads a link destination document from the link destination based on the link destination address included in the input link information. Then, the link feature extraction unit 22 obtains the title appearance count, the text appearance count, and the highlight display count based on the extracted keyword and the read link destination document. The link feature extraction unit 22 does not read the link destination document based on the link destination address, but the undetermined link information storage unit 31 and the determined link information storage unit 32 store the link destination document as link information in advance. Also good.

  (C) As an example of link feature information that can be obtained by grouping links according to a predetermined condition, for example, “a link having the same link destination address as a link to be learned or determined in a group of the same link source notation "Rate", "Percentage of links that have the same link source notation as the link to be learned or judged in the group with the same link destination address", "Learning or judgment target in the group with the same link source address and the same link source notation" The ratio of links having the same link source address as the link to be learned or determined in the group of the same link source address and the same link destination address ”. Note that learning is processing for obtaining an identification function in a phase of learning processing, as will be described later.

  “Percentage of links with the same link destination address as the link to be learned or judged in the same link source notation group” is used to judge inconsistency of links that refer to the wrong document with the same name. It is done. FIG. 2 is an explanatory diagram illustrating an example of a correspondence relationship of links between documents when an inconsistency referring to an incorrect document with the same name occurs. In FIG. 2, a document 111 is a document in which information about the product α is described, and a document 112 is a document in which information about the product β is described.

  In this example, the same link source notation “product α” described in each of the three documents 113, 114, and 115 is correctly linked to the document 111 in which information on the product α is described (link 131, 141, 151). On the other hand, the link source notation “product α” described in the document 116 is erroneously linked to the document 112 in which information on the product β is described (link 161).

  FIG. 3 is an explanatory diagram illustrating an example of a calculation process in the case of obtaining “a ratio of links having the same link destination address as a link to be learned or determined in a group represented by the same link source” as the link feature information. . In this example, a case will be described in which link feature information is obtained when there is a link correspondence between documents shown in FIG. When there is a link correspondence shown in FIG. 2 and each link is grouped with the same link source notation, as shown in FIG. 3, the link feature extraction means 22 has the link source notation “product α”. A group of four links 131, 141, 151, 161 is created. 3, the link ID “L131” corresponds to the link 131 in FIG. 2, the link ID “L141” corresponds to the link 141 in FIG. 2, and the link ID “L151” corresponds to the link 151 in FIG. The ID and the link ID “L161” are IDs corresponding to the link 161 in FIG.

  The link feature extraction unit 22 creates a subgroup of three links 131, 141, 151 whose link destination address is “address of document 111” among the four links 131, 141, 151, 161 grouped. . The link feature extraction unit 22 creates a subgroup including one link 161 whose link destination address is “the address of the document 112”. The link feature extraction means 22 obtains the number of links included in each subgroup. Then, the link feature extraction unit 22 obtains the ratio of the number of links of each subgroup in the entire group.

  As shown in FIG. 3, the ratio of the number of links of the subgroup whose link source notation is “product α” and whose link destination address is “address of document 111” is obtained as 3/4 = 0.75. Further, the ratio of the number of links of the subgroup whose link source notation is “product α” and whose link destination address is “address of document 112” is obtained as 1/4 = 0.25. Therefore, the “ratio of links having the same link destination address as the link to be learned or determined within the group of the same link source notation” obtained as the link feature information is obtained as 0.75 for the links 131, 141, and 151. In the link 161, 0.25 is obtained. Thus, the smaller the “percentage of links that have the same link destination address as the link to be learned or judged in the group of the same link source notation” means that it belongs to the minority subgroup when grouped. And the possibility of inconsistency increases.

  “Percentage of links that have the same link source notation as a link to be learned or judged within a group with the same link destination address” is used to determine inconsistencies in links that refer to the same document with an incorrect name. It is done. FIG. 4 is an explanatory diagram showing an example of a correspondence relationship between links between documents when an inconsistency referring to the same document with an incorrect name has occurred. In FIG. 4, a document 221 is a document including newly arrived information describing a site update history, and a document 222 is a document including event information describing an event schedule such as a seminar or an exhibition. .

  In this example, the link source notation “new arrival information” of the three documents 223, 224, 225 is correctly linked to the document 221 (links 231, 241, 251). On the other hand, the link source notation “event information” of the document 226 should be originally linked to the document 222 but is erroneously linked to the document 221 (link 261).

  FIG. 5 is an explanatory diagram illustrating an example of a calculation process in the case of obtaining “a ratio of links having the same link source notation as a learning or determination target link within a group having the same link destination address” as link feature information. . In this example, a case will be described in which link feature information is obtained when there is a link correspondence between documents shown in FIG. If there is a link correspondence shown in FIG. 4 and grouping with the same link destination address, as shown in FIG. 5, the link feature extraction means 22 has a link destination address “address of document 221”. A group of two links 231, 241, 251, 261 is created. 5, the link ID “L231” corresponds to the ID corresponding to the link 231 in FIG. 4, the link ID “L241” corresponds to the ID corresponding to the link 241 in FIG. 4, and the link ID “L251” corresponds to the link 251 in FIG. The ID and the link ID “L261” are IDs corresponding to the link 261 in FIG.

  Further, the link feature extraction means 22 creates a subgroup of three links 231, 241, 251 whose link source notation is “new arrival information” among the four links 231, 241, 251, 261 grouped. Further, the link feature extraction unit 22 creates a subgroup including one link 261 whose link source notation is “event information”. The link feature extraction means 22 obtains the number of links included in each subgroup. Then, the link feature extraction unit 22 obtains the ratio of the number of links of each subgroup in the entire group.

  As shown in FIG. 5, the ratio of the number of links in the subgroup whose link destination address is “document 221 address” and whose link source notation is “new arrival information” is obtained as 3/4 = 0.75. Further, the ratio of the number of links in the subgroup whose link destination address is “document 221 address” and whose link source notation is “event information” is obtained as 1/4 = 0.25. Therefore, the “ratio of links having the same link source notation as the link to be learned or determined within the group of the same link destination address” obtained as the link feature information is obtained as 0.75 for the links 231, 241, 251. The link 261 requires 0.25. Thus, the smaller the “percentage of links with the same link source notation as the link to be learned or judged within the group with the same link destination address” means that it belongs to the minority subgroup when grouped. And there is a high possibility of inconsistencies.

  “The ratio of links that have the same link destination address as the link to be learned or judged in the group with the same link source address and the same link source notation” refers to links that refer to different documents with the same name from one document. Used to determine inconsistencies. FIG. 6 is an explanatory diagram illustrating an example of a correspondence relationship of links between documents in a case where inconsistency in which different documents with the same name are referred to from one document has occurred. In FIG. 6, a document 631 is a document in which information about the latest version of software X is described, and a document 632 is a document in which information about an old version of software X is described.

  In this example, of the links from the link source notation “software X latest version” of the document 633, two links 331 and 332 correspond to the document 631 correctly. On the other hand, among the links from the link source notation “software X latest version” of the document 633, the link 333 remains corresponding to the document 632 in which information of the old version of the software X is described.

  FIG. 7 shows an example of a calculation process in the case of obtaining “a ratio of links having the same link destination address as a link to be learned or determined in a group having the same link source address and the same link source notation” as the link feature information. It is explanatory drawing shown. In this example, a case will be described in which link feature information is obtained when there is a link correspondence between documents shown in FIG. If there is a link correspondence shown in FIG. 6 and grouping is performed with the same link source address and the same link source notation, the link feature extracting unit 22 sets the link source address to “document 633” as shown in FIG. A group of three links 331, 332, and 333 having the link source notation “software X latest version”. In FIG. 7, the link ID “L331” corresponds to the link 331 in FIG. 6, the link ID “L332” corresponds to the link 332 in FIG. 6, and the link ID “L333” corresponds to the link 333 in FIG. ID.

  Further, the link feature extraction unit 22 creates a subgroup of two links 331 and 332 whose link destination address is “address of document 631” among the three links 331, 332 and 333 grouped. The link feature extraction unit 22 creates a subgroup including one link 333 whose link destination address is “the address of the document 632”. The link feature extraction means 22 obtains the number of links included in each subgroup. Then, the link feature extraction unit 22 obtains the ratio of the number of links of each subgroup in the entire group.

  As shown in FIG. 7, the ratio of the number of links in the subgroup whose link destination address is “document 631 address” is obtained as 2/3 = 0.67. Further, the ratio of the number of links in the subgroup whose link destination address is “document 632 address” is obtained as 1/3 = 0.33. Thus, the smaller the “ratio of links with the same link source address and the same link destination address as the link to be learned or judged in the same link source notation group”, the smaller the subgroup of the minority group when it is grouped. Means that it belongs, and there is a high possibility of inconsistency.

  Note that the grouping condition for calculating the “ratio of links having the same link destination address as the link to be learned or judged within the group of the same link source address and the same link source notation” is “same link source notation”. In this group, the grouping condition in the case of calculating “the ratio of links having the same link destination address as the link to be learned or determined” is made stricter. Therefore, as the link feature information, “the ratio of links having the same link destination address as the link to be learned or determined in the same link source address and the same link source notation group” and “in the same link source notation group The determination accuracy can be further improved by using a combination of “the ratio of links having the same link destination address as the learning or determination target link”.

  "The ratio of links that have the same link source notation as the link to be learned or discriminated in the group of the same link source address and the same link destination address" refers to one document from another document with a different name. Used to determine inconsistencies in existing links. FIG. 8 is an explanatory diagram illustrating an example of a correspondence relationship of links between documents in a case where an inconsistency in which another document is referred to by a different name from one document is generated. In FIG. 8, a document 841 is a document in which an annual report for 2002 is described.

  In this example, the link source notation “annual report 2002” of the document 842 is correctly linked to the document 841 (link 422). On the other hand, for the link 421 corresponding to the same document 841 from the same document 842, the link source notation of the document 842 is “Annual Report 2003”, and the link source notation and the contents of the link destination document are different. Yes.

  FIG. 9 shows an example of a calculation process in the case of obtaining “a ratio of links having the same link source notation as a learning target or a discrimination target within a group of the same link source address and the same link destination address” as the link feature information. It is explanatory drawing shown. In this example, a case will be described in which link feature information is obtained when there is a link correspondence between documents shown in FIG. If there is a link correspondence shown in FIG. 8 and grouping is performed using the same link source address and the same link destination address, the link feature extraction unit 22 sets the link source address to “document 842” as shown in FIG. A group of two links 421 and 422 whose link destination address is “address of document 841”. 9, the link ID “L421” is an ID corresponding to the link 421 in FIG. 8, and the link ID “L422” is an ID corresponding to the link 422 in FIG.

  Further, the link feature extraction means 22 creates a subgroup including one link 421 whose link source notation is “annual report 2003” among the two links 421 and 422 that are grouped. Further, the link feature extraction unit 22 creates a subgroup including one link 422 whose link source notation is “annual report 2002” among the two links 421 and 422 that are grouped. The link feature extraction means 22 obtains the number of links included in each subgroup. Then, the link feature extraction unit 22 obtains the ratio of the number of links of each subgroup in the entire group.

  As shown in FIG. 9, the ratio of the number of links in the subgroup whose link source notation is “annual report 2003” is obtained as 1/2 = 0.5. Further, the ratio of the number of links of the subgroup whose link source notation is “Annual Report 2002” is obtained as 1/2 = 0.5. The smaller the "percentage of links with the same link source notation in the group of the same link source address and the same link destination address", the smaller the subgroup of the minority group when grouped. And there is a high possibility of inconsistency.

  In the example shown in FIG. 9, the ratio of the number of links in any subgroup is 0.5, which is the same. Therefore, using only the ratio of links having the same link source notation as the link to be learned or determined in the group of the same link source address and the same link destination address as the link feature information, any link is inconsistent. It is not possible to immediately determine if there is, but it can be seen that there is an inconsistency in one of the links.

  In this case, the grouping condition for calculating the “ratio of links having the same link source notation as the link to be learned or determined within the group of the same link source address and the same link destination address” is “same link destination”. The grouping condition for calculating “the ratio of links having the same link source notation as the link to be learned or determined in the address group” is made stricter. Therefore, as the link feature information, “the ratio of links having the same link source notation as the link to be learned or determined in the group of the same link source address and the same link destination address” and “in the group of the same link destination address” The determination accuracy can be further increased by using a combination of “the ratio of links having the same link source notation as the link to be learned or determined”.

  FIG. 10 is an explanatory diagram illustrating an example of the data structure of the link feature information, the link ID, and the inconsistency determination result input from the link feature extraction unit 22 to the link feature storage unit 33 and stored in the link feature storage unit 33. FIG. 10A shows an example in which the link feature storage unit 33 stores, as link feature information, a keyword, a title appearance count, and a highlight display count included in the link source notation. As shown in FIG. 10A, the link feature storage unit 33 stores each link feature information, link ID, and inconsistency determination result in association with each other.

  In the example shown in FIG. 10A, it can be seen that the link source notation of the link with the link ID “L11” includes “new arrival” and “information” as keywords. It can also be seen that the keywords “New Arrival” and “Information” appear seven times in the title of the linked document. It can also be seen that the keywords “New Arrival” and “Information” are highlighted eight times in total in the text of the linked document. Further, the link with the link ID “L11” has a mismatch determination result of “◯”, which indicates that the link is determined to be normal.

  It can also be seen that the link source notation of the link with the link ID “L12” includes “event” and “information” as keywords. It can also be seen that the keywords “event” and “information” appear five times in the title of the linked document, and are highlighted three times in the text of the linked document. Furthermore, the link with the link ID “L12” has an inconsistency determination result of “x”, which indicates that the link is determined to be inconsistent.

  In the data structure illustrated in FIG. 10A, the link feature storage unit 33 mixes qualitative data such as keywords (non-quantity data) and quantitative data such as the number of appearances as the link feature information. However, the link feature storage unit 33 may store link feature information including only quantitative data.

  For example, as will be described later, the inconsistency learning unit 23 uses a predetermined learning algorithm to calculate an identification function for determining whether or not the link is inconsistent. When the inconsistency learning unit 23 uses a linear discriminant analysis method or a neural network as a learning algorithm, quantitative data must be input to the inconsistency learning unit 23. In this case, the link feature storage unit 33 may store link feature information including only quantitative data by converting the keyword into numerical data.

  FIG. 10B shows an example in which the link feature storage unit 33 stores link feature information including only quantitative data. Note that the link feature information shown in FIG. 10B corresponds to the link feature information shown in FIG. 10A converted into quantitative data. In the example shown in FIG. 10B, each keyword is converted into a dummy variable and stored. For example, as shown in FIG. 10B, each keyword is converted to “1” if it is included in the link source notation, and is converted to “0” if it is not included in the link source notation. It is stored in the feature storage unit 33.

  The link characteristic information may include only qualitative data. For example, when a decision tree or the like is used as a learning algorithm, qualitative data must be input to the inconsistency learning unit 23. In this case, the link feature storage unit 33 may store link feature information including only qualitative data by categorizing (classifying) each item.

  FIG. 10C shows an example in which the link feature storage unit 33 stores link feature information including only qualitative data. Note that the link feature information shown in FIG. 10C corresponds to the link feature information shown in FIGS. 10A and 10B all converted into qualitative data. In the example shown in FIG. 10C, each keyword is categorized as “include” if it is included in the link source notation, and “not included” if it is not included in the link source notation. 33.

  Since the minimum value of the number of appearances of the title is “5” and the maximum value is “7”, when the section from the number of times “5” to “7” is categorized into three, the number of times “5” is displayed. The case of “small”, the number of times “6” can be categorized as “medium”, and the case of the number of times “7” can be categorized as “many”. In this example, as shown in FIG. 10C, the link feature storage unit 33 stores qualitative data in which the number of title appearances is categorized into “low”, “medium”, and “high”. Yes.

  Further, since the minimum number of highlights is “3” and the maximum value is “8”, when the section from the number of times “3” to “8” is categorized into three, the number of times “3” or “4” ”Can be categorized as“ low ”, the number of times“ 5 ”or“ 6 ”as“ medium ”, and the number of times“ 7 ”or“ 8 ”as“ many ”. In this example, as shown in FIG. 10C, the link feature storage unit 33 stores qualitative data categorized into three categories of “low”, “medium”, and “high” as the number of highlights. Yes.

  In the example shown in FIG. 10C, the case has been described in which the interval from the minimum value to the maximum value is categorized into “low”, “medium”, and “high” based on the data value. The method of categorizing data into three is not limited to the method shown in this embodiment. For example, the data may be categorized into three so that the number of data is equally divided into three based on the number of data. In this example, the case of conversion into three categories of “small”, “medium”, and “many” has been described, but the number of categories to be converted is not limited to the case shown in the present embodiment. For example, it may be classified into any other number of categories such as categorizing into two or categorizing into five.

  In the example illustrated in FIG. 10, the case has been described in which the link feature storage unit 33 stores link feature information of a link that has already been determined, but the link feature storage unit 33 stores link feature information of an undetermined link. The case is the same as that shown in FIG. When storing the link feature information of the undetermined link, the link feature storage unit 33 stores an identifier indicating that the link has not been determined as the inconsistency determination result. For example, the link feature storage unit 33 stores identifiers such as “?”, “−”, “−1” as the inconsistency determination result.

  Based on the link feature information stored in the link feature storage unit 33 and the inconsistency determination result, the inconsistency learning unit 23 determines the content of each link feature information (for example, the magnitude of the value of the link feature information) and the link. Statistically calculate the relationship between the percentage determined to be inconsistent. The inconsistency learning means 23 obtains an identification function used for determining whether or not a certain link is inconsistent by performing statistical calculation. The identification function indicates the relationship between the link feature information of a link for which a determination as to whether or not inconsistency has already been made and the determination result of the determination.

  The inconsistency learning unit 23 stores the obtained discrimination function in the learning result storage unit 34. In this case, the inconsistency learning unit 23 causes the learning result storage unit 34 to store the type of learning algorithm used by the inconsistency learning unit 23 to obtain the discrimination function together with the obtained discrimination function. In the present embodiment, the inconsistency learning unit 23 causes the learning result storage unit 34 to store a parameter for expressing the discriminant function (coefficient included in the discriminant function) as the discriminant function.

  The undetermined link information access unit 24 reads undetermined link information as to whether or not the link is inconsistent from the undetermined link information storage unit 31, and outputs it to the link feature extraction unit 22.

  The inconsistency determination means 25 uses the link feature information of the link that has not been determined whether or not inconsistency among the link feature information stored in the link feature storage unit 33 and the identification function stored in the learning result storage unit 34. It is determined whether or not there is an inconsistency in the undetermined link.

  The result reflection unit 26 has a function of outputting the inconsistency determination result determined by the inconsistency determination unit 25 to the output device 12. The result reflection unit 26 has a function of correcting the document and correcting the determination result according to the operation of the site administrator. For example, when the site manager confirms the inconsistency determination result output to the output device 12, the site administrator operates the input device 11 to input a correction instruction for the inconsistent document. If there is an error in the inconsistency determination result, the site administrator operates the input device 11 to input a correction instruction for the inconsistency determination result. To correct the determination result, when a correction instruction is given for the determination result indicating that there is an inconsistency, the content of the determination result is changed to “no inconsistency” and a determination is made that there is no inconsistency. When a correction instruction is given for the result, it means that the content of the determination result is changed to “inconsistent”.

  When the confirmation result of the site administrator (document correction instruction or determination result correction instruction) is input, the result reflection unit 26 feeds back the site administrator confirmation result to the determined link information access unit 21. In the present embodiment, feedback of the confirmation result means that, when the confirmation result of the site administrator is input, the result reflection means 26 instructs the determined link information access means 21 to update the information, and the determined link information This means that the access means 21 updates the information stored in the determined link information storage unit 32. That is, when the confirmation result is fed back, the determined link information access unit 21 updates the information stored in the determined link information storage unit 32 according to the instruction of the result reflection unit 26. When the information stored in the link feature storage unit 33 is updated, the inconsistency learning unit 23 recalculates the identification function based on the updated data.

  The undetermined link information storage unit is realized by the undetermined link information storage unit 31. The undetermined link information reading unit is realized by the undetermined link information access unit 24. The relationship information storage means is realized by the learning result storage unit 34. The inconsistency determination unit is realized by the inconsistency determination unit 25 and the output device 12. The determined link information storage unit is realized by the determined link information storage unit 32. The determined link information reading unit is realized by the determined link information access unit 21. The relationship information deriving unit is realized by the inconsistency learning unit 23. The feedback means is realized by the determined link information access means 21 and the result reflection means 26.

  In the present embodiment, the relationship information derived by the relationship information deriving unit corresponds to an identification function obtained by the inconsistency learning unit 23. The relation information derivation algorithm corresponds to a learning algorithm used by the inconsistency learning unit 23 to obtain an identification function. Further, the link feature information that can be specified from the link to be determined corresponds to “(A) Link feature information that can be obtained by a link alone”. The link feature information that can be identified from the link to be determined and the description content of the link destination document is “(B) Link feature information that can be obtained based on the relationship between the link and the link destination document”. Is equivalent to. In addition, the link feature information indicating the size of the group to which the determination target link belongs when the links are grouped according to a predetermined condition is “(C) Link feature that can be obtained by grouping links according to a predetermined condition”. Corresponds to “information”.

  Next, the operation will be described. In the present embodiment, the link diagnosis apparatus executes processing of three independent phases of “learning processing”, “determination processing”, and “result confirmation processing”. The “learning process” is a process for obtaining an identification function to be used by the link diagnosis apparatus for determining inconsistency based on the determined link information. The “determination process” is a process in which the link diagnosis apparatus determines whether or not there is a mismatch with an undetermined link. The “result confirmation process” is a process for outputting (displaying) the determination result of link mismatch and correcting the document or correcting the determination result according to the operation of the site administrator who confirmed the determination result. In the present embodiment, it is described that learning to obtain an identification function is learned.

  An operation of learning processing executed by the link diagnosis apparatus will be described. FIG. 11 is a flowchart illustrating an example of the progress of the learning process executed by the link diagnosis apparatus. The inconsistency learning unit 23 of the data processing device 20 accepts input of setting information for learning in accordance with the input operation of the site manager (step S11). In step S11, the inconsistency learning unit 23 receives, for example, input of selection instructions for data corresponding to past determined links used for learning, types of link feature information used for learning, and types of learning algorithm used for learning. .

  FIG. 12 is an explanatory diagram illustrating an example of a learning setting screen for inputting setting information for the site administrator to learn. The inconsistency learning unit 23 displays, for example, a learning setting screen illustrated in FIG. 12 on the output device 12 that is a display device. As shown in FIG. 12, the learning setting screen 1201 includes a determined data list 1211, a link feature information list 1212, a learning algorithm list 1213, and a learning button 1214.

  The determined data list 1211 includes the name, URL, and diagnosis date and time of each site including past determined links. The determined data list 1211 may include, as reference data, the number of documents at each site, the number of links, the ratio of the number of links determined to be inconsistent, and the like. In this example, learning is performed using data having a diagnosis date of “October 24, 2003” and “October 17, 2003” among the data corresponding to the determined link of the site having the site name “AAA Corp.”. The case where it does is demonstrated. As shown in FIG. 12, the inconsistency learning unit 23 displays a learning setting screen 1201 including the determined data list 1211 on the output device 12 to prompt the site administrator to select data. The site administrator selects data to be used for learning by checking the check box 1215 of the displayed determined data list 1211.

  The link feature information list 1212 displays a list of types of link feature information that can be used for learning. The link feature information displayed in the list in the link feature information list 1212 includes the information of “(A) Link feature information that can be obtained by a link alone” and the relationship between “(B) link and its linked document”. Each information of “link feature information that can be obtained based on” and “link feature information that can be obtained by grouping (C) links according to a predetermined condition”. In this example, a case will be described in which learning is performed using the number of title appearances and the number of highlights in the link feature information. As shown in FIG. 12, the inconsistency learning unit 23 displays a learning setting screen 1201 including a link feature information list 1212 on the output device 12, and prompts the site administrator to select the type of link feature information. The site manager selects the type of link feature information used for learning by checking the check box 1216 of the link feature information list 1212.

  In this example, the case where the determined data list 1211 and the link feature information 1212 are displayed on the same learning setting screen 1201 and each setting information can be set at the same time has been described. It is not restricted to the method shown by the form. For example, the inconsistency learning unit 23 may cause the output device 12 to display the determined data list 1211 and the link feature information list 1212 on different screens. Further, for example, an interface that can interactively set link feature information used for learning may be used.

  The learning algorithm list 1213 displays a list of learning algorithms that are used when the inconsistency learning unit 23 obtains an identification function. As a learning algorithm for calculating the discriminant function, a linear discriminant method that is a method generally used in statistics, a decision tree used in pattern recognition or machine learning, a neural network, or the like can be used. In this example, a case will be described in which learning is performed using a linear discriminant method among learning algorithms. As shown in FIG. 12, the inconsistency learning unit 23 displays a learning setting screen 1201 including a learning algorithm list 1213 on the output device 12, and prompts the site administrator to select the type of learning algorithm. The site manager selects the type of learning algorithm used for learning by checking the check box 1217 of the learning algorithm list 1213.

  In addition to selecting a learning algorithm, a parameter value used for each learning algorithm can be input and designated from the learning algorithm list 1213. For example, in the example shown in FIG. 12, the pruning reliability of the decision tree is set to “25%”, but the site administrator changes the reliability value by entering another value. It is also possible to instruct the execution of learning. When a neural network is selected as a learning algorithm for learning, the site manager can select the number of layers and the activation function (linear function or sigmoid function) of neurons.

  In the example shown in FIG. 12, the case where the linear discriminant method, the decision tree, and the neural network are displayed as a list has been described as the learning algorithm. However, the learning algorithm displayed as a list is limited to the one shown in this example. Absent. For example, a k nearest neighbor method, a Bayes method, an SVM (Support Vector Machine) method, or the like used in pattern recognition may be displayed in a list. The parameters that can be set for each learning algorithm are not limited to those shown in the present embodiment. For example, when a neural network is used as a learning algorithm, the number of neurons may be changed and input as a parameter.

  When the site manager selects each setting information from the determined data list 1211, the link feature information list 1212, and the learning algorithm list 1213, the site manager presses a learning button 1214. When the site manager presses the learning button 1214, the inconsistency learning unit 23 receives input of selection instructions for the link information, the type of link feature information, and the type of learning algorithm.

  When receiving the input of each setting information for learning, the determined link information access unit 21 of the data processing device 20 in accordance with the setting content, the link information selected by the site administrator from the determined link information storage unit 32, The link ID and inconsistency determination result are read (step S12). Then, the determined link information access unit 21 outputs the read link information, link ID, and inconsistency determination result to the link feature extraction unit 22.

  The link feature extraction unit 22 extracts the link feature information of the type selected by the site manager in step S11 based on the link information input from the determined link information access unit 21 (step S13). Then, the link feature extraction unit 22 stores the extracted link feature information and the link ID and the mismatch determination result input from the determined link information access unit 21 in the link feature storage unit 33 (step S13).

  The inconsistency learning unit 23 calculates an identification function for the link feature information stored in the link feature storage unit 33 by statistically calculating the relationship between the content of each link feature information and the proportion of links determined to be inconsistent. Obtained (step S14). In step S14, the inconsistency learning unit 23 obtains an identification function using the learning algorithm selected by the site manager in step S11. The inconsistency learning unit 23 stores the obtained discriminant function in the learning result storage unit 34.

  A procedure by which the inconsistency learning unit 23 obtains an identification function will be described. In this example, a case will be described in which the number of title appearances and the number of highlights are used as link feature information, and a discrimination function is obtained using linear discriminant analysis (linear discriminant method) as a learning algorithm.

  13 and 14 are explanatory diagrams illustrating an example of a calculation process when the inconsistency learning unit 23 obtains an identification function. In this example, a case will be described in which the inconsistency learning unit 23 obtains an identification function using information shown in FIG. 13A stored in the link feature storage unit 33. In this example, the determined link information storage unit 32 stores eight pieces of link information corresponding to the link IDs “L51” to “L58” in advance. The link feature extraction unit 22 extracts the number of appearances of the title and the number of highlights from the link information input from the determined link information access unit 21, and together with the link ID and the inconsistency determination result from the determined link information access unit 21. It is stored in the link feature storage unit 33. The link feature storage unit 33 stores the link feature information, the link ID, and the inconsistency determination result shown in FIG. 13A input from the link feature extraction unit 22.

  In FIG. 13A, a link with a mismatch determination result of “◯” means that there is no problem (a link without mismatch). In addition, a link whose inconsistency determination result is “x” means that there is a mismatch.

  FIG. 13B is an explanatory diagram in which each link is plotted and displayed on a two-dimensional plane based on the number of title appearances and the number of highlights shown in FIG. In FIG. 13B, each link is plotted and displayed with the number of title appearances as a variable x_1 and the number of highlights as a variable x_2. The inconsistency learning unit 23 obtains a straight line that divides the link group having the mismatch determination result “◯” and the link group having the mismatch determination result “×” in the two-dimensional plane illustrated in FIG.

  Considering that the determination result of inconsistency can be explained by the variables x_1 and x_2, the discriminant function Z can be expressed by Equation 1. That is, the discriminant function shown in Expression 1 indicates the relationship between the link feature information of a link that has already been determined whether or not inconsistency has occurred and the determination result of the determination.

Z = a_1 · x_1 + a_2 · x_2 + a_0 Equation 1

  As shown in FIG. 14A, the inconsistency learning unit 23 sets each link corresponding to the link ID shown in FIG. 13A to a link group whose inconsistency determination result is “◯”, and the inconsistency determination result. Is divided into “×” links. The inconsistency learning unit 23 obtains a discrimination score obtained by substituting the variables x_1 and x_2 into Equation 1 for each link.

  When the total variation is St and the variation between classes is Sb, a straight line that optimally divides the two link groups can be obtained by obtaining a straight line that maximizes the correlation ratio (Sb / St). The total variation St is a value indicating how much each data is distributed from the total average Z_all of the discrimination scores, and is obtained using Equation 2.

^{St = ΣiΣj (Z_ij-Z_all)} 2 Equation 2

  In Expression 2, i is a dummy index indicating which link group is included. In the case of the data shown in FIG. 14A, i has two values “1” or “2”. In the example shown in FIG. 14A, i is “1” in the link group with the inconsistency determination result “◯”, and i is “2” in the link group with the inconsistency determination result “×”. Moreover, j is a dummy index indicating which data in the link group. In the example shown in FIG. 14A, since any link group having an inconsistency determination result of “◯” or “×” includes four data, i is any value of “1” or “2”. Even so, j has four values from “1” to “4”.

  The inter-class variation Sb is a value indicating how far the link group with the inconsistency determination result “◯” and the link group with the inconsistency determination result “×” are dispersed from the total average Z_all. , Using Equation 3.

Sb = Σi {n_i · (Z_i−Z_all) ² } Equation 3

  In Expression 3, i is a dummy index indicating which link group is the same as Expression 2. In the case of the data shown in FIG. 14A, i has two values “1” or “2”. n_i indicates the number of data of the i-th link group. In the case of the data shown in FIG. 14A, the link group of the i = 1st inconsistency determination result “◯” includes four data with link IDs “L51”, “L53”, “L54”, and “L57”. N_1 = 4 because it is included. In addition, since the link group of the i = 2nd inconsistency determination result “x” includes four pieces of data with link IDs “L52”, “L55”, “L56”, and “L58”, n_2 = 4.

  When the total variation St and the interclass variation Sb are calculated for the data shown in FIG. 14A using Equation 2 and Equation 3, they can be obtained as Equation 4 and Equation 5, respectively.

St = (Z_11−Z_all) ² + (Z_12−Z_all) ² + (Z_13−Z_all) ² + (Z_14−Z_all) ² + (Z_21−Z_all) ² + (Z_22−Z_all) ² + (Z_23−Z_all) ² + (Z — 24−Z_all) ²
= 20 · a_1 ² -2 · a_1 · a_2 + 28 · a_2 ² Formula 4

Sb = 4 · (Z_1−Z_all) ² + 4 · (Z_2−Z_all) ²
= 8 · (a — 1 ² +2 • a — 1 • a — ² + a — ^{2 2} ) Equation 5

  When the correlation ratio (Sb / St) is calculated using the total variation St and the interclass variation Sb obtained by Equation 4 and Equation 5, it can be obtained as Equation 6.

(Sb / St) = (4 · t ² + 8 · t + 4) / (10 · t ² −t + 14) Equation 6

  In Expression 6, t is obtained by replacing (a_1 / a_2). Since the correlation ratio (Sb / St) is maximized when (Sb / St) differentiated by t is 0, the value of t at which the correlation ratio (Sb / St) is maximized is It can be obtained by Expression 7.

(21 · t ² −8t−29) / (10 · t ² −t + 14) ² = 0 Equation 7

  Solving the equation shown in Equation 7, t = 1.381, -1. When the obtained value of t is substituted into Equation 6, the value of correlation ratio (Sb / St) becomes the minimum value 0 when t = −1 is substituted, and the correlation ratio (Sb when t = 1.382 is substituted. / St) is the maximum value of 0.71556. Therefore, it can be seen that the condition of the correlation function to be obtained is (a_1 / a_2) = 1.382. Therefore, if equation 1 is modified and t = 1.381 is substituted, a modified equation such as equation 8 is obtained.

Z = (a_1 / a_2) .x_1 + x_2 + (a_0 / a_2)
= 1.381 · x_1 + x_2 + (a — 0 / a — 2) Equation 8

  In order to divide the link group of the inconsistency determination result “◯” and the link group of the inconsistency determination result “×” into two, the discriminant function Z only has to pass through the total average point (6, 5). Substituting x_1 = 6 and x_2 = 5 into Equation 8 establishes Equation 9.

1.381 · 6 + 5 + (a — 0 / a — 2) = 0 Equation 9

  Using equation 9, (a_0 / a_2) = − 13.286 can be obtained, and the discriminant function Z is obtained as in equation 10.

Z = 1.382.x_1 + x_2-13.286 Equation 10

  Accordingly, if a straight line of 1.381 · x_1 + x_2−13.286 = 0 is drawn on the two-dimensional plane, the link group of the inconsistency determination result “◯” and the link group of the inconsistency determination result “×” are 2 It can be divided into two.

  FIG. 14B is obtained by subtracting the straight line 1.381 · x_1 + x_2−13.286 = 0 from the plot of the link distribution shown in FIG. 13B. In FIG. 14B, in the mismatch determination result “◯” link group located on the upper side of the straight line on the plot diagram, the discriminant function Z of Expression 10 has a positive value, and the mismatch determination result located on the lower side. In the “×” link group, the discriminant function Z in Expression 10 is a negative value.

  When determining whether or not the link is inconsistent with the undetermined link, the link feature extracting unit 22 calculates the title appearance count x_1 and the highlight display count x_2 based on the undetermined link information. Extracted as link feature information. Also, using the link feature information extracted by the link feature extraction unit 22, the inconsistency determination unit 25 substitutes each link feature information into Equation 10 to obtain the value of the identification function Z. Then, the link feature extraction unit 22 can determine whether or not there is a mismatch in the link by determining whether the value of the obtained identification function is a positive value or a negative value.

  When the discriminant function is obtained, the inconsistency learning means 23 uses the type of learning algorithm used “linear discriminant analysis” and parameters “1.381”, “1”, “−13.286” of coefficients of the discriminant function. Is stored in the learning result storage unit 34.

  In this example, the case where the identification function on the two-dimensional plane is obtained using the number of appearances of the title and the number of highlights as the link feature information has been described. However, the method by which the inconsistency learning unit 23 obtains the identification information is It is not limited to the case shown in the embodiment. For example, the inconsistency learning unit 23 may obtain an identification function for multidimensional data using other three or more link feature information.

  In the present embodiment, the case has been described in which the link feature information is extracted and the identification function is obtained after all the determined link information designated by the site administrator is read in step S12. The procedure for obtaining the identification function is as follows. However, the present invention is not limited to this case. For example, instead of processing after all the determined link information is read, the link feature information of each link is extracted while reading the determined link information one by one, and the identification function is calculated each time the link feature information is extracted. You may make it obtain | require by correcting.

  Next, an operation in which the link diagnosis apparatus executes determination processing will be described. FIG. 15 is a flowchart illustrating an example of a process progress of a determination process executed by the link diagnosis apparatus. The inconsistency determination unit 25 of the data processing device 20 outputs (displays) a setting screen for selecting and inputting a link to be determined to be inconsistent to the output device 12, and the site administrator selects a link to be determined. Prompt. The site administrator operates the input device 11 to select a link for which it is determined whether or not there is a mismatch. The undetermined link information access unit 24 of the data processing device 20 reads link information and link ID corresponding to the link selected by the site manager from the undetermined link information storage unit 31 (step S21). Then, the undetermined link information access unit 24 outputs the read undetermined link information and link ID to the link feature extraction unit 22.

  FIG. 16 is an explanatory diagram illustrating an example of a determination setting screen for a site administrator to selectively input a site including a link that is to be determined to be inconsistent. The inconsistency determination unit 25 displays, for example, the determination setting screen illustrated in FIG. 16 on the output device 12 that is a display device. As shown in FIG. 16, the determination setting screen 1602 includes an undetermined data list 1621 and a determination button 1622. In this example, a case will be described in which data of a site having a site name “AAA Corp.” is selected and input by a site administrator among undecided data. As shown in FIG. 16, the inconsistency determination unit 25 displays a determination setting screen 1602 including an undecided data list 1621 on the output device 12 and prompts the site administrator to select a site including a determination target link. The site administrator selects a site to be determined by checking the check box 1623 of the undetermined data list 1621.

  When the site manager selects a site including a link to be determined to be inconsistent from the undecided data list 1621 on the determination setting screen 1602 shown in FIG. 16, the site administrator presses a determination button 1622. When the determination button 1622 is pressed by the site administrator, the undetermined link information access unit 24 reads the undetermined link information and link ID corresponding to the selected site from the undetermined link information storage unit 31, and links Output to the feature extraction means 22.

  When the determination button 1622 is pressed, the undetermined link information access unit 24 may read data from the selected determination target site and extract link information from the read data. In this case, the undetermined link information access unit 24 temporarily stores the extracted link information in the undetermined link information storage unit 31. Then, the undetermined link information access unit 24 reads the link information once stored from the undetermined link information storage unit 31 and outputs it to the link feature extraction unit 22.

  Based on the link information input from the undetermined link information access unit 24, the link feature extraction unit 22 extracts the same type of link feature information as the link feature information used in the “learning process” shown in FIG. S22). Then, the link feature extraction unit 22 stores the link ID and the extracted link feature information in the link feature storage unit 33. For example, in the examples shown in FIGS. 12 to 14, since the number of text appearances and the number of highlights are used as the link feature information in the learning process, the link feature extraction unit 22 is input from the undetermined link information access unit 24. Based on the link information, the number of text appearances and the number of highlights are extracted.

  The inconsistency determination means 25 determines whether or not there is an inconsistency in the determination target link using the link feature information extracted from the link feature storage unit 33 and the identification function stored in the learning result storage unit 34. (Step S23).

  For example, the case where the learning result storage unit 34 stores “linear discriminant analysis” as the type of learning algorithm and “1.381”, “1”, “−13.286” as parameters of the discriminating function will be described. To do. The inconsistency determination unit 25 restores the discriminant function “Z = 1.382 · x_1 + x_2-13.286” based on the discriminant function parameter stored in the learning result storage unit 34. Further, the inconsistency determination unit 25 obtains the value of the identification function for the undetermined link feature information using the restored identification function Z. The inconsistency determination unit 25 determines whether the determined link is “no problem (no inconsistency)” or “inconsistency” by determining whether the obtained value is a positive value or a negative value. Judgment is made.

  Next, the operation of the result confirmation process executed by the link diagnosis apparatus will be described. FIG. 17 is a flowchart illustrating an example of a process progress of a result confirmation process executed by the link diagnosis apparatus. The result reflection unit 26 of the data processing device 20 causes the output device 12 to output the link mismatch determination result determined by the mismatch determination unit 25 (step S31). In step S31, the result reflection unit 26 groups the determination results of each link in units of link source addresses and causes the output device 12 to output them.

  FIG. 18 is an explanatory diagram illustrating an example of a result confirmation screen for displaying the determination result of the link mismatch determined by the mismatch determination unit 25. The result reflection unit 26 displays, for example, a result confirmation screen shown in FIG. 18 on the output device 12 that is a display device. As shown in FIG. 18, the result confirmation screen 1803 includes a determination result list 1831 and a registration button 1832.

  As shown in FIG. 18, the determination result list 1831 includes a link ID, a link source address, a link destination address, a link source anchor (link source notation), a determination result, and a certainty factor as information for each link. In FIG. 18, the determination result “×” indicates that the link corresponding to the determination result is inconsistent, and the determination result “◯” indicates that the link corresponding to the determination result is a correct link without inconsistency. Show. Also, links with inconsistencies and links without inconsistencies may be displayed in different colors. In this case, the result reflecting unit 26 adds a color to each information column of the link determined by the inconsistency determining unit 25 in the determination result list 1831 shown in FIG. May be. By doing so, it is possible to display in an easy-to-understand manner which link is inconsistent to the site administrator.

  The certainty factor is a numerical value indicating how much the determination result is reliable. For example, when linear discriminant analysis is used as the learning algorithm, the distance from the separation plane of the link corresponding to the determination result may be used as the certainty factor.

  In this example, when there is an inconsistency, a case is described in which each piece of information of a corresponding link is displayed with a color. However, the color classification method on the screen is not limited to the case shown in this example. For example, the result reflection unit 26 may display each piece of information on the corresponding link on the output device 12 by changing the brightness or saturation of the color according to the magnitude of the certainty factor. Further, in this example, the case where the inconsistency determination result of “O” or “×” is displayed for all links has been described, but the display method of the determination result is not limited to the case shown in this example. . For example, for a link with a certainty factor equal to or less than a threshold value, it may be determined as undecided without determining whether or not there is a mismatch. In this case, the determination as to whether or not the undetermined link is inconsistent may be left to the site administrator's final determination.

  Further, the result reflection means 26 displays the “link ID”, “link source address”, “link destination address”, and “link source anchor” of each item name in the determination result list 1831 on the result confirmation screen 1803 shown in FIG. , “Judgment result” and “confidence” are displayed on the output device 12 as links. When the item name is clicked with the mouse by the site administrator, the result reflecting means 26 rearranges the information of each link using each item as a sort key and displays it on the output device 12. For example, if the item name of “judgment result” is clicked with the mouse and the information of each link is rearranged in the order of the judgment result, the site administrator can efficiently check only the information of the link with inconsistency. can do.

  If the information of each link is rearranged in ascending order of certainty, the site administrator can check the determination result determined by the inconsistency determination unit 25 from the one with the low reliability. In particular, when learning and inconsistency are determined using the SVM method as a learning algorithm, the SVM method classifies data by focusing only on data in the vicinity of the separation plane rather than looking at variations in the entire data. The data near the separation plane greatly affects the accuracy. For this reason, it is easy to improve the determination accuracy by checking from the low reliability and reflecting the correction result.

  In this example, the case where “link ID”, “link source address”, “link destination address”, “link source anchor”, “judgment result”, “confidence” is used as the sort key has been described. The present invention is not limited to that shown in this example. For example, the number of inconsistencies in a data page or a page view may be used as a sort key. Alternatively, the number of inconsistencies for each directory and the number of pages including inconsistencies may be counted and displayed in the order of inconsistent directories.

  The result reflecting means 26 operates according to an input instruction from the site administrator when an operation input is made by the site administrator when the inconsistency determination result is determined to be correct or when the determination result is determined to be incorrect. It is determined whether an input has been made (step S32). The result reflecting means 26 also displays the document addresses in the link source addresses of the determination result list 1831 on the output device 12 on the result confirmation screen 1803 shown in FIG. When the site administrator wants to confirm the determination result of the link determined to be inconsistent, the site manager clicks the portion of the document address in the corresponding link source address. Then, the result reflection unit 26 calls the document correction screen and displays it on the output device 12.

  FIG. 19 is an explanatory diagram showing an example of a document correction screen for the site administrator to check whether or not the inconsistency determination result is correct and to correct the document. As shown in FIG. 19, the document correction screen 1904 includes a browser display unit 1941, a source code display unit 1942, and a menu unit 1943.

  The browser display unit 1941 is a screen that displays, using a browser, a document that is clicked by the mouse on the result confirmation screen 1803 shown in FIG. The result reflection unit 26 causes the browser display unit 1941 to highlight the link location 1950 corresponding to the line clicked with the mouse on the result confirmation screen 1803 shown in FIG. For example, when the link source address “address of document D41” in the row of the link ID “L412” on the result confirmation screen 1803 shown in FIG. 18 is clicked with the mouse, the browser display unit 1941 corresponds to the link ID “L412”. The link source notation “event information” 1950 is highlighted. By doing so, the site administrator can quickly grasp which link operation should be confirmed on the browser.

  A source code display unit 1942 on the document correction screen 1904 in FIG. 19 is a screen that displays HTML (Hyper Text Markup Language) format source code of the document displayed by the browser display unit 1941. In the source code display unit 1942, the result reflection unit 26 uses a location 1951 in the source code corresponding to the link (link ID “L412”) corresponding to the location 1950 highlighted on the browser display unit 1941. Is highlighted on the output device 12. In this example, for the link (link ID “L412”) corresponding to the location 1950 of the link source notation “event information” highlighted on the browser display unit 1941, the location 1951 in the source code corresponding to the link is It is highlighted. By doing so, the site administrator can quickly grasp which position in the source code needs to be corrected.

  On the menu screen 1943, menu buttons for screen operation are arranged. The menu screen 1943 includes a “next inconsistency” button 1944, a “previous inconsistency” button 1945, a “group display” button 1946, a “document correction” button 1947, and a “judgment result correction” button 1948 as menu buttons. included.

  The “next mismatch” button 1944 is a button for navigating the link mismatch. That is, when the site administrator presses the “next inconsistency” button 1944 when the data selected by the site administrator includes a plurality of link inconsistent portions, the result reflecting means 26 displays the document being viewed. Among the portions corresponding to the existing links, the portion corresponding to the next link determined to be inconsistent by the inconsistency determining means 25 is highlighted on the output device 12. When the “next inconsistency” button 1944 is pressed, if there is no portion corresponding to another inconsistent link in the document being viewed, the result reflecting means 26 is included in the data. The part corresponding to the link determined to be inconsistent in the next document to be displayed is highlighted.

  The “previous inconsistency” button 1945 is a navigation button as well as the “next inconsistency” button 1944. That is, when the site administrator presses the “previous inconsistency” button 1945, the result reflection means 26 is inconsistent in the inconsistency determination means 25 in the portion corresponding to the link existing in the document being browsed. The portion corresponding to the previous link determined to be highlighted on the output device 12. By using the “next inconsistency” button 1944 and the “previous inconsistency” button 1945, the site administrator can efficiently check only the portion corresponding to the link determined to be inconsistent by the inconsistency determination means 25. Can do.

  The result reflection unit 26 prompts the site administrator to confirm the determination result by causing the browser display unit 1941 and the source code display unit 1942 to highlight the portion corresponding to the inconsistent link. The site administrator confirms the document correction screen 1904, and if it is determined that the displayed determination result is correctly inconsistent, the site administrator operates the input device 11 to perform an input operation when the determination result is determined to be correct. Do. In the present embodiment, when the site administrator determines that the determination result is correct, the site administrator operates the input device 11 to input a document correction instruction corresponding to the link determined to have inconsistency. When the result reflection unit 26 determines that the operation input is performed when the site manager determines that the determination result is correct, the result reflection unit 26 causes the output device 12 to output (display) the link group screen (step S33). As will be described later, the link group screen is a screen for confirming grouped information regarding the highlighted link.

  In the example shown in FIG. 19, the “group display” button 1946 has a “link with the same link source notation”, “link with the same link destination address”, “link source address and link source” as the highlighted link. This is a button for grouping “links having the same notation” and “links having the same link source address and link destination address” to display a link group screen. The site administrator can obtain guidelines on how to modify the document by checking the grouped information for the highlighted links.

  When the site manager determines that the displayed determination result is correct and the link is inconsistent, the site manager presses a “group display” button 1946. Then, the result reflecting means 26 determines that the input operation is performed when it is determined that the determination result is correct, and displays the link group screen. Then, the site administrator examines how to correct the document while checking the displayed link group screen.

  FIG. 20 is an explanatory diagram illustrating an example of a link group screen. The result reflection unit 26 displays, for example, the link group screen 2005 shown in FIG. 20 on the output device 12 that is a display device. In the link group screen 2005 shown in FIG. 20, the result reflection unit 26 links the link ID “L412” and the link with the same link source notation as the link ID “L412” (in this example, link IDs “L421”, “L431”). )) Is displayed on the output device 12. As shown in FIG. 20, the link destination addresses of the links with the link IDs “421” and “431” are both “address of document D44”.

  In the link group screen 2005 shown in FIG. 20, the result reflecting unit 26 displays grouping information of links having the same link destination address as the link ID “L412” (in this example, link IDs “L411” and “L425”). It is displayed on the output device 12. As shown in FIG. 20, it can be seen that the link source notations of the links with the link IDs “L411” and “L425” are both “pre-release”. Therefore, if the site administrator corrects the link source notation corresponding to the link with the link ID “L412” to “press release” or corrects the link destination address to “address of document D44”, the inconsistency is resolved. It turns out that there is a possibility.

  In the example shown in FIG. 18, there is no link having the same link source address and link source notation as the link with the link ID “L412”. Therefore, in the link group screen 2005 shown in FIG. "Same link" is not grouped. In addition, since there is no link having the same link source address and link destination address as the link with the link ID “L412”, the link group screen 2005 shown in FIG. 20 has a “link with the same link source address and link destination address”. There is no grouping display.

  The result reflecting means 26 updates the document according to the site manager's correction operation (step S34). The result reflection unit 26 instructs the determined link information access unit 21 to update the information stored in the determined link information storage unit 32. Then, the determined link information access unit 21 updates the information stored in the determined link information storage unit 32 in accordance with the instruction from the result reflection unit 26 (step S35). That is, the determined link information access unit 21 newly stores the link information, link ID, and inconsistency determination result of the link determined by the inconsistency determination unit 25 in the determined link information storage unit 32.

  For example, when the site administrator confirms the link group screen and finishes examining the correction method, the site administrator inputs the correction contents of the source code into the source code display section 1942 of the document correction screen 1904 shown in FIG. "Button 1947 is pressed. When a “document correction” button 1947 is pressed, the result reflection unit 26 updates the source code of the document. The determined link information access unit 21 newly stores the determined link information, the link ID, and the inconsistency determination result in the determined link information storage unit 32. That is, the source code instructed for correction is reflected on the actual site, and at the same time, the confirmation result of the site manager is fed back.

  When the information stored in the determined link information storage unit 32 is updated, the undetermined link information access unit 24 links the link information and link ID corresponding to the updated information (that is, the link for which the determination process has been completed). Link information and link ID) are deleted from the undetermined link information storage unit 31.

  When the information stored in the determined link information storage unit 32 is updated, the learning process from step S11 to step S14 shown in FIG. 11 is executed again using the updated link information (step S36). That is, the inconsistency learning unit 23 performs a statistical calculation using the updated link information to obtain a new identification function.

  In step S32, when an input operation is performed when the site administrator determines that the determination result is not correct, the result reflection unit 26 corrects the determination result in accordance with the operation of the site administrator (step S37). In the present embodiment, when an instruction for correcting the inconsistency determination result is input by the site administrator, the result reflection unit 26 corrects the determination result. The result reflection unit 26 instructs the determined link information access unit 21 to update the information stored in the determined link information storage unit 32. Then, the determined link information access unit 21 updates the information stored in the determined link information storage unit 32 in accordance with the instruction from the result reflection unit 26 (step S35). That is, the determined link information access unit 21 newly stores the link information, the link ID, and the corrected inconsistency determination result of the link determined by the inconsistency determination unit 25 in the determined link information storage unit 32. .

  For example, when the site manager determines that the determination result determined by the inconsistency determination unit 25 is incorrect, the site administrator presses a “determination result correction” button 1948 on the document correction screen 1904 shown in FIG. Then, the result reflection unit 26 corrects the determination result. Further, the determined link information access unit 21 causes the determined link information storage unit 32 to newly store the determined link information, the link ID, and the corrected inconsistency determination result. That is, the determination result instructed to be corrected by the site manager is corrected, and the confirmation result of the site manager is fed back.

  For example, when the result of determination by the inconsistency determination unit 25 as “inconsistent” is an error, when the “correction of determination result” button 1948 is pressed by the site administrator, the determination result becomes “not inconsistent”. Will be corrected. In addition, when the result of the determination by the inconsistency determination unit 25 as “not inconsistent” is an error, when the “correction of determination result” button 1948 is pressed by the site administrator, the determination result becomes “inconsistent”. Will be corrected. When the information stored in the determined link information storage unit 32 is updated, the learning process from step S11 to step S14 is performed again using the updated link information, and the inconsistency learning unit 23 performs statistical calculation. To obtain a new discriminant function.

  When the information stored in the determined link information storage unit 32 is updated, the undetermined link information access unit 24 links the link information and link ID corresponding to the updated information (that is, the link for which the determination process has been completed). Link information and link ID) are deleted from the undetermined link information storage unit 31.

  In the example illustrated in FIG. 19, the case has been described in which only the link determined by the inconsistency determination unit 25 to be “inconsistent” is described. However, the method for the site administrator to check the result of the inconsistency determination is as follows. The present invention is not limited to the case described in this embodiment. For example, links determined as “not inconsistent” in order from the top of the result confirmation screen shown in FIG. 18 may be checked. Alternatively, the links may be displayed in the order of certainty, and the links determined as “not inconsistent” in the certainty order may be checked.

  In this embodiment, inconsistency determination results are output for all links corresponding to the data selected by the site administrator, and the site administrator confirms the result confirmation screen including the determination results for all links. Although the case where the inconsistency learning unit 23 performs relearning later has been described, the procedure from inconsistency determination to relearning is not limited to the case shown in the present embodiment. For example, the inconsistency learning means 23 may determine inconsistencies one by one for the links corresponding to the selected data, and the site administrator's confirmation results for each determination result may be fed back. Then, each time the confirmation result is fed back one by one, the inconsistency learning means 23 re-learns, and the inconsistency determination means 25 uses the discriminant function obtained by re-learning by the inconsistency learning means 23, and the inconsistency determination means 25 receives the next link. It is also possible to repeatedly execute the procedure for determining the above.

  Further, in the present embodiment, the case has been described in which the site administrator confirms and feeds back the determination result of each link in the result confirmation processing, but the procedure for feeding back the determination result is shown in the present embodiment. Not limited to cases. For example, the confirmation information of the determination result is accumulated to some extent, the accumulated confirmation information is collectively reflected in the determined link information storage unit 32, and the updated link information stored in the determined link information storage unit 32 is stored. It is also possible to use the inconsistency learning means 23 for re-learning.

  Further, when the link diagnosis apparatus is used for the first time, the learning process cannot be performed because the determined link information storage unit 32 does not store the determined information. When the link diagnosis apparatus is used for the first time, the learning result may be executed by storing in the determined link information storage unit 32 information on the result of determining whether or not there is an artificial inconsistency in advance.

  When the link diagnostic apparatus is used for the first time, an initial value of the discriminant function is stored in advance in the learning result storage unit 34 so that the inconsistency determination unit 25 determines inconsistency using the initial value of the discriminant function. It may be. In this case, for example, when the linear discriminant method is used as the learning algorithm, the learning result storage unit 34 may store all the values of the coefficient “0” as the parameters of the discrimination function. Further, the inconsistency determination unit 25 may determine that there is inconsistency when the value of the discrimination function calculated for each link feature information is “0” in addition to the negative value. Then, an appropriate identification function may be obtained in accordance with the site manager's correction instruction. Further, the learning result storage unit 34 may store in advance a value selected at random as the initial value of each coefficient of the discrimination function.

  As described above, according to the present embodiment, the link diagnosis apparatus uses “(A” as link feature information indicating the probability that a logical inconsistency has occurred based on the determined link information. ) “Link feature information that can be obtained by a single link”, “(B) Link feature information that can be obtained based on the relationship between a link and its linked document”, and “(C) A group of links according to a predetermined condition” "Link feature information that can be obtained through conversion". The link diagnosis apparatus statistically obtains an identification function indicating the relationship between the contents of each link feature information and the proportion of links determined to be inconsistent. Then, the link diagnosis apparatus determines whether or not the undetermined link is inconsistent using the obtained identification function. Therefore, it is possible to detect a logical inconsistency that does not cause an error when acquiring a document, and it is possible to automatically detect a logical inconsistency of a hypertext link.

  Further, according to the present embodiment, the link diagnosis apparatus outputs a certainty factor of the determination result in addition to the determination result of whether or not the link to be determined is inconsistent. Then, the link diagnosis apparatus displays the links in different colors according to the determination result or the certainty factor. This makes it easier for site managers to determine which links are inconsistent and which links should be checked.

  Further, according to the present embodiment, the link diagnosis apparatus highlights inconsistent portions in the document. Therefore, the site administrator can quickly grasp where inconsistencies exist in the document.

  In addition, according to the present embodiment, the link diagnosis apparatus, the link determined to be inconsistent, “the link with the same link source notation”, “the link with the same link destination address”, “the link source address and the link source notation are the same” By displaying “Link” and “Link with the same link source address and link destination address” as a group, a guideline indicating what kind of link is correct is given. Therefore, the site manager can easily determine how to correct the portion determined to be inconsistent.

  In addition, according to the present embodiment, it is possible to select a learning algorithm that can be used for obtaining a discrimination function and a parameter to be used. Therefore, the site manager can check the determination accuracy when using each learning algorithm, and can select and automatically determine a learning algorithm with high determination accuracy through trial and error.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 21 is a block diagram illustrating another configuration example of the link diagnosis apparatus. As shown in FIG. 21, the data processing device 40 of the link diagnosis device includes a learning method determining unit 27 in addition to the configuration shown in the first embodiment. In the present embodiment, the link diagnosis apparatus selects an optimal learning algorithm from a plurality of learning algorithms selected and input by the site administrator, and automatically selects the optimal combination of the selected learning algorithm and link feature information. To decide. Then, the link diagnosis apparatus executes a learning process, a determination process, and a result confirmation process based on the determined combination of the learning algorithm and the link feature information.

  The learning method determination unit 27 automatically determines the optimal combination of the learning algorithm and the link feature information according to the link generation tendency by performing cross-validation on the determined link information. Note that cross-validation refers to obtaining determination accuracy for each combination by variously combining a learning algorithm and link feature information.

  In the present embodiment, the optimum combination determining unit is realized by the learning method determining unit 27 and the output device 12.

  Next, the operation will be described. In the present embodiment, the link diagnosis apparatus executes “cross-validation processing” in addition to the processing of three independent phases of “learning processing”, “determination processing”, and “result confirmation processing”. The “cross-validation process” is a process for automatically determining an optimal combination of learning algorithm and link feature information by performing cross-validation on the determined link information. The “learning process”, “determination process”, and “result confirmation process” are the same as the processes described in the first embodiment.

  FIG. 22 is a flowchart illustrating an example of the progress of the cross-validation process executed by the link diagnosis apparatus. The learning method determination means 27 of the data processing device 40 accepts input of setting information for performing cross validation in accordance with the input operation of the site manager (step S40). In step S <b> 40, the learning method determination unit 27 receives, for example, input of validated data to be subjected to cross validation and a learning algorithm selection instruction.

  FIG. 23 is an explanatory diagram showing an example of a cross-validation setting screen for inputting setting information for the site administrator to perform cross-validation. The learning method determination means 27 displays, for example, the cross-validation setting screen shown in FIG. 23 on the output device 12 that is a display device. As shown in FIG. 23, the cross-validation setting screen 2306 includes a determined data list 2361, a learning algorithm list 2362, and a cross-validation button 2363.

  The learning method determination means 27 displays the cross-validation setting screen 2306 on the output device 12 and prompts the site administrator to select the determined data and the type of learning algorithm. In the “cross-validation processing” phase, the site administrator selects the determined data to be cross-validated and the type of learning algorithm from the cross-validation setting screen 2306 shown in FIG. In this case, the site administrator selects determined data from the determined data list 2361 included in the cross-validation setting screen 2306, and selects a learning algorithm type from the learning algorithm list 2362 included in the cross-validation setting screen 2306. .

  In this example, as shown in FIG. 23, the data of “AAA Corp.” diagnosed on “October 17, 2003” from the determined data list 2361 was selected as the target of cross-validation by the site administrator. Explain the case. The learning method determination unit 27 displays a cross-validation setting screen 2306 including a determined data list 2361 on the output device 12 and prompts the site administrator to select determined data. The site manager selects the data to be cross-validated by checking the check box 2364 of the determined data list 2361.

  In this example, three types of learning algorithms, linear discriminant analysis, decision tree, and neural network, are selected from the learning algorithm list 2362 by the site administrator. The learning method determination unit 27 displays a cross-validation setting screen 2306 including a learning algorithm list 2362 on the output device 12 and prompts the site administrator to select the type of learning algorithm. The site administrator selects a plurality of learning algorithms by checking the check box 2365 of the learning algorithm list 2362.

  In this example, the decision tree is selected so that the pruning confidence level is performed in three increments of “25%” of “25%”, “50%”, and “75%”. . Furthermore, with respect to the neural network, the number of layers is selected to be performed in three ways, “2”, “3”, and “4”, and the activation function is selected to be performed in two ways: a linear function and a sigmoid function.

  When the site manager selects each setting information from the determined data list 2361 and the learning algorithm list 2362, the site manager presses a cross validation button 2363. When the cross-validation button 2363 is pressed by the site manager, the learning method determination unit 27 receives input of the determined data and the learning algorithm type selected by the site manager.

  When receiving the input of each setting information for cross-validation, the learning method determination means 27 instructs the determined link information access means 21 to read the link information according to the setting contents. The determined link information access means 21 is given to the link information, link ID and link information corresponding to the data selected by the site administrator from the determined link information storage unit 32 according to the instruction of the learning method determination means 27. The inconsistency determination result is read (step S41).

  The learning method determination unit 27 instructs the link feature extraction unit 22 to extract link feature information. The link feature extraction means 22 obtains “(A) link feature information that can be obtained by a link alone” and “(B) the relationship between a link and its linked document according to an instruction from the learning method determination means 27. Link characteristic information "and" (C) link characteristic information that can be obtained by grouping links according to a predetermined condition "are extracted (step S42).

  The learning method determination unit 27 divides n% of the link mismatch determination result and link feature information as training data and (100−n)% of data as test data (step S43). For example, if there are inconsistency determination results and link feature information for 100 links, and n = 80, the learning method determination unit 27 uses the random number or the like to select any 80 links from 100 data. The inconsistency determination result and the link feature information are divided as training data. In this case, the learning method determination unit 27 divides the remaining 20 inconsistency determination results and link characteristic information data as test data.

  The learning method determination means 27 selects one combination for evaluating accuracy among all combinations of the learning algorithm and the link feature information (step S44).

  In the example shown in FIG. 23, three types of linear discriminant analysis, decision tree, and neural network are selected as learning algorithms, and the pruning confidence levels of “25%”, “50%”, and “75%” are selected in the decision tree. Three types are selected, three types of “2”, “3”, and “4” are selected as the number of layers in the neural network, and two types of linear functions and sigmoid functions are selected as activation functions. In this case, the number of combinations of learning algorithms is 1 + 3 + 3 · 2 = 10.

  If there are 11 types of link feature information, there are 11 cases where only one type is used, and 11 cases where only two types are used! / {(11-2)!・ 2! } = 55 ways, only 3 types are used! / {(11-3)!・ 3! } = 165 ways to use only 4 types! / {(11-4)!・ 4! } = 330 streets, only 5 types are used! / {(11-5)!・ 5! } = 462. The total number of cases of using 1 to 11 types is 2 · (11 + 55 + 165 + 330 + 462) + 1 = 2047. This calculation uses the fact that the number of combinations when only x types (x <11) are used is equal to the number of combinations when only (11-x) types are used.

  Therefore, in the example shown in FIG. 23, there are 10 · 2047 = 20470 combinations of learning algorithms and link feature information in total.

  The learning method determination unit 27 instructs the inconsistency learning unit 23 to calculate the discrimination function. The inconsistency learning unit 23 uses the training data inconsistency determination result, the link feature information selected in step S44 and the learning algorithm in accordance with the instruction of the learning method determination unit 27, and the contents of each link feature information (for example, The identification function is obtained by statistically calculating the relationship between the size of the value of the link feature information and the ratio determined to be inconsistent (step S45).

  The learning method determining unit 27 instructs the inconsistency determining unit 25 to determine whether or not each link corresponding to the test data is inconsistent. The inconsistency determination unit 25 uses the link feature information selected in step S44 and the identification function obtained in the learning in step S45 for the test data in accordance with the instruction from the learning method determination unit 27. It is determined whether or not there is a mismatch. Then, the learning method determination unit 27 determines whether or not the output result of the inconsistency determination matches the inconsistency determination result of the test data recorded as the determined link information (counts that match). The accuracy is obtained (step S46).

  The learning method determination unit 27 determines whether there is a combination for which accuracy determination is not performed among combinations of the learning algorithm and the link feature information (step S47). If it is determined that there is a combination for which accuracy determination is not performed, the learning method determination unit 27 returns to step S44, selects another combination, and repeatedly executes the processing from step S44 to step S47.

  If it is determined in step S47 that the accuracy determination has been completed for all the combinations, the learning method determination unit 27 determines the learning algorithm for the combination having the largest determination accuracy value among the combinations subjected to the accuracy determination, and Select link feature information. Then, the learning method determination means 27 outputs the selected combination and the value of determination accuracy (step S48).

  In step S48, the learning method determination means 27 determines the combination of the selected learning algorithm and link feature information as the optimal combination, and outputs the determined combination and the value of determination accuracy. In this case, for example, the learning method determination unit 27 may display a list of correspondence relationships between the determined algorithm and link feature information and the determination accuracy on the output device 12 that is a display device. By doing so, the site administrator can know the combination of the learning algorithm with high determination accuracy and the link feature information without trial and error in the learning process phase.

  In the present embodiment, the training data and the test data are divided into n% and (100-n)%, respectively, and the method of cross-validation is described. However, the method of cross-validation is as follows. The present invention is not limited to the case described in this embodiment. For example, m types of training data of n% of the number of links are created, and the processing of step S45 and step S46 is repeatedly performed m times so that the learning method determination unit 27 calculates an average value of m determination accuracy. It may be.

  Further, the site administrator can estimate the cause of inconsistency at the determination target site from the combination of link feature information with high determination accuracy. For example, if “keywords included in the link source notation” are used as link feature information, and there is a tendency for the determination accuracy to be high, the site administrator may include an error or spelling error in the link source notation on the site to be determined. It can be estimated that there are many inconsistencies as a factor.

  If the domain accuracy or the directory name included in the link destination address is used as the link feature information, if the determination accuracy tends to increase, the site administrator can specify the specified domain in the target site. Alternatively, it can be estimated that the document is deleted from the directory as a cause of inconsistency.

  In addition, if the “domain name or directory name included in the link source address” is used as the link feature information, if there is a tendency for the determination accuracy to increase, the site administrator can select a part of the site administrator of the determination target site. It can be presumed that inconsistent management of these areas is a cause of inconsistency.

  In addition, if there is a tendency for the determination accuracy to increase when “appearance position or display area of the link in the document” is used as the link feature information, the site administrator can display the link source notation in the document at the determination target site. It can be estimated that the inconspicuous link is caused by an overlooked link being overlooked during the check.

  Further, if the determination accuracy tends to be high when the title appearance count, the text appearance count, or the highlight display count is used as the link feature information, the site administrator can use the link source notation and the link destination document in the determination target site. It can be presumed that the cause of the inconsistency is that the correspondence relationship with is broken.

  If the ratio of links that have the same link destination address as the link to be learned or determined in the same link source notation group is used as link feature information, if there is a tendency for the determination accuracy to increase, site management A person can estimate that there are many inconsistencies caused by referring to an incorrect document with the same name at the site to be determined.

  If the ratio of links that have the same link source notation as a link to be learned or determined within a group with the same link destination address is used as link feature information, if there is a tendency for the determination accuracy to increase, site management A person can estimate that there are many inconsistencies caused by referring to the same document with a wrong name in the site to be determined.

  In addition, the determination accuracy tends to be higher when “the ratio of links having the same link destination address as the link to be learned or determined in the same link source address and the same link source notation group” is used as the link feature information. If there is, the site manager can estimate that there are many inconsistencies caused by referring to different documents with the same name from one document in the determination target site.

  In addition, the determination accuracy tends to increase when “the ratio of links having the same link source notation as the link to be learned or determined in the group of the same link source address and the same link destination address” is used as the link feature information. If there is, the site manager can estimate that there are many inconsistencies caused by referring to one document from another document with a different name at the determination target site.

  As described above, according to the present embodiment, the link diagnosis apparatus automatically determines the combination of the learning algorithm with the highest determination accuracy and the link feature information in accordance with the tendency of occurrence of link mismatch. Therefore, the site administrator does not need to perform trial and error in the learning process phase, and can specify a combination of the learning algorithm with the highest determination accuracy and the link feature information.

  Further, according to the present embodiment, the site administrator can estimate the inconsistency tendency at the determination target site from the combination of the link feature information with high determination accuracy. Accordingly, the site administrator can analyze the cause of link inconsistency based on the tendency of occurrence of link inconsistency, and can analyze not only the inconsistent portion but also problems in site construction.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 24 is a block diagram illustrating still another configuration example of the link diagnosis apparatus. As shown in FIG. 24, in the present embodiment, a link diagnosis program 60 for causing the data processing device 50 to execute is included.

  The link diagnosis program 60 is a program for causing the data processing device 50 to execute each process of link diagnosis. The link diagnosis program 60 is read by the data processing device 50. The data processing device 50 stores the read link diagnosis program 60 in a memory (not shown). Then, the data processing device 50 executes each link diagnosis process according to the read link diagnosis program 60. In the present embodiment, the data processing device 50 performs processing executed by the data processing device 20 shown in the first embodiment and the data processing device 40 shown in the second embodiment according to the link diagnosis program 60. The same processing as that executed by is executed.

  In the present embodiment, for example, the data processing device 50 causes the computer to read the undetermined link information from the undetermined link information storage unit, and based on the read undetermined link information, the link is inconsistent. A process of extracting link feature information indicating the probability of occurrence, and a process of determining whether or not a mismatch has occurred in an undetermined link using the extracted link feature information and relationship information The link diagnosis program 60 to be executed is read, and each process of link diagnosis is executed.

  Next, the operation will be described. In the present embodiment, the link diagnosis apparatus executes processing of four independent phases of “learning processing”, “determination processing”, “result confirmation processing”, and “cross validation processing”.

  In the “learning process” phase, when the site administrator specifies past determined data used for learning and the type of link feature information used for learning, the data processing device 50 determines the determined link information storage unit 32. The link information, the link ID, and the inconsistency determination result are read from, and the link feature information of the designated link is extracted. Then, the data processing apparatus 50 statistically calculates the relationship between the content of each link feature information (for example, the size of the value of the link feature information) and the ratio determined to be inconsistent, so that a link is inconsistent. An identification function for determining whether or not is obtained.

  In the “determination process” phase, when the site administrator designates undecided data to be judged, the data processing device 50 reads the undetermined link information from the undetermined link information storage unit 31 to determine whether or not it is inconsistent. Extract link feature information of the same type used. Then, the data processing device 50 determines whether or not the determination target link is inconsistent using the identification function obtained in the learning processing phase with respect to the link characteristic information of the undetermined link.

  In the “result confirmation processing” phase, the data processing device 50 groups the determination results of each link in units of link source addresses and causes the output device 12 to display them. When the site administrator is prompted to check each link and the document is corrected according to the input instruction of the site administrator, the data processing device 50 stores the determined link information indicating that the determination target link is inconsistent. This is reflected in part 32. In addition, when the output determination result is incorrect, the data processing device 50 corrects the determination result and reflects it in the determined link information storage unit 32. In addition, the data processing device 50 recalculates the discrimination function using the reflected result.

  In the “cross-validation process” phase, the data processing device 50 reads the inconsistency determination result given to the link information, the link ID, and the link information from the determined link information storage unit 32. The data processing apparatus 50 determines from the link information “(A) link feature information that can be obtained by a link alone” and “(B) link feature information that can be obtained based on the relationship between the link and its linked document”. , “(C) Link feature information that can be obtained by grouping links according to predetermined conditions” is extracted. The data processing device 50 divides n% as training data and (100−n)% as test data from the data of the link mismatch determination result and the link feature information. Then, the data processing device 50 selects one combination from all combinations of the learning algorithm and the link feature information, and uses the training data to determine the content of each link feature information and the proportion of links determined to be inconsistent. Determine the discriminant function by statistically calculating the relationship.

  The data processing device 50 determines whether or not each link of the test data is inconsistent with respect to the test data, using the link feature information used for learning and the identification function obtained by the learning process. The data processing device 50 determines whether or not the output result of the inconsistency determination matches that recorded as determined link information (counts that match), and obtains determination accuracy.

  The data processing device 50 determines whether there is a combination for which accuracy determination is not performed among combinations of the learning algorithm and the link feature information. If the data processing apparatus 50 determines that there is a combination for which accuracy determination is not performed, the data processing apparatus 50 selects another combination to obtain repeated determination accuracy, and finally outputs the combination with the highest accuracy and the determination accuracy. .

  The link diagnosis apparatus according to the present invention can be applied to a hypertext management apparatus for managing hypertext in a Web site. For example, a website editor may use a hypertext management apparatus to which a link diagnosis apparatus is applied to check and guarantee the quality of hypertext in the range of the website where he is in charge of editing. it can. In addition, when a company outsources the construction of a website, the person in charge of the company uses a hypertext management device to check the quality of the hypertext constituting the website delivered from the outside. Can be used. In addition, a Web site construction consultant uses a hypertext management device to diagnose the quality of hypertext that constitutes a Web site of a company that is a customer or a prospective customer, and proposes improvement of the Web site to the company Can be used.

It is a block diagram which shows an example of a structure of the link diagnostic apparatus by this invention. It is explanatory drawing which shows the example of the correspondence of the link between documents when the inconsistency which refers the wrong document with the same name has generate | occur | produced. It is explanatory drawing which shows the example of the calculation process in the case of calculating | requiring "the ratio of the link which has the same link destination address as the link used as learning or determination within the group of the same link origin description" as link characteristic information. It is explanatory drawing which shows the example of the correspondence of the link between documents when the inconsistency which refers the same document with an incorrect name has generate | occur | produced. It is explanatory drawing which shows the example of the calculation process in the case of calculating | requiring "the ratio of the link which has the same link origin description as the link used as learning or determination within the group of the same link destination address" as link characteristic information. It is explanatory drawing which shows the example of the correspondence of the link between documents when the inconsistency which refers to a different document with the same name from one document has generate | occur | produced. It is explanatory drawing which shows the example of a calculation process in the case of calculating | requiring "the ratio of the link which has the same link destination address as the link used as learning or determination within the group of the same link origin address and the same link origin notation" as link characteristic information. is there. It is explanatory drawing which shows the example of the correspondence of the link between documents when the inconsistency which refers to another one document from one document with a different name has generate | occur | produced. It is explanatory drawing which shows the example of the calculation process in the case of calculating | requiring "the ratio of the link which has the same link origin notation as the learning or discrimination | determination object within the group of the same link source address and the same link destination address" as link characteristic information. is there. It is explanatory drawing which shows the example of the data structure of the link characteristic information which a link characteristic memory | storage part 33 memorize | stores, link ID, and a mismatch determination result. It is a flowchart which shows an example of the process progress of the learning process which a link diagnosis apparatus performs. It is explanatory drawing which shows an example of a learning setting screen. It is explanatory drawing which shows the example of the calculation process in case the mismatching learning means 23 calculates | requires the discrimination function Z. FIG. 14 is a continuation of the explanatory diagram showing an example of a calculation process when the inconsistency learning unit 23 shown in FIG. 13 obtains the discriminant function Z. It is a flowchart which shows an example of the process progress of the determination process which a link diagnostic apparatus performs. It is explanatory drawing which shows an example of the determination setting screen. It is a flowchart which shows an example of the process progress of the result confirmation process which a link diagnosis apparatus performs. It is explanatory drawing which shows an example of a result confirmation screen. It is explanatory drawing which shows an example of a document correction screen. It is explanatory drawing which shows an example of a link group screen. It is a block diagram which shows the other structural example of a link diagnostic apparatus. It is a flowchart which shows an example of the process progress of the cross-validation process which a link diagnostic apparatus performs. It is explanatory drawing which shows an example of a cross-validation setting screen. It is a block diagram which shows the further another structural example of a link diagnostic apparatus.

Explanation of symbols

11 Input device 12 Output device 20 Data processing device 21 Determined link information access means 22 Link feature extraction means 23 Inconsistency learning means 24 Undecided link information access means 25 Inconsistency judgment means 26 Result reflection means 30 Storage device 31 Undecided link Information storage unit 32 Determined link information storage unit 33 Link feature storage unit 34 Learning result storage unit

Claims (28)

Undecided to store undetermined link information, which is link information of undetermined links in the link information indicating the hypertext link attributes that associate the designated document with the designated source document Link information storage means;
Undetermined link information reading means for reading undetermined link information from the undetermined link information storage means;
Link feature extraction means for extracting link feature information indicating the probability of inconsistency in the link based on the undetermined link information read by the undetermined link information reading means;
A relationship information storage unit that stores relationship information indicating a relationship between the link feature information of a link for which determination of whether or not inconsistency has already been performed and the determination result of the determination;
Inconsistency determination means for determining whether or not there is a mismatch in the undecided link using the link feature information extracted by the link feature extraction means and the relation information. Link diagnostic device. Undecided to store undetermined link information, which is link information of undetermined links in the link information indicating the hypertext link attributes that associate the designated document with the designated source document Link information storage means;
A determined link information storage unit that stores determined link information that is link information of a link that has already been determined whether or not inconsistency occurs in the link information, and a determination result of the determination;
Undetermined link information reading means for reading undetermined link information from the undetermined link information storage means;
A determined link information reading unit that reads the determined link information and the determination result from the determined link information storage unit;
Based on the undetermined link information read by the undetermined link information reading means, link feature information indicating the probability of inconsistency in the link is extracted, and the determination read by the determined link information reading means Link feature extraction means for extracting link feature information based on the completed link information;
It is already determined whether or not there is a mismatch using the link feature information extracted by the link feature extraction unit based on the determined link information and the determination result read by the determined link information reading unit. Relationship information deriving means for deriving relationship information indicating the relationship between the link feature information of the existing link and the determination result of the determination;
Relation information storage means for storing relation information derived by the relation information derivation means;
Inconsistency determination means for determining whether or not there is a mismatch in the undetermined link using the link feature information extracted by the link feature extraction means based on the undetermined link information and the relationship information. A link diagnostic device characterized by that. A feedback means for storing the determination result of the undecided link by the inconsistency determining means and the link information of the link in the determined link information storage means;
The feedback means outputs a determination result of the determination for the undetermined link, prompts the user to confirm whether the determination result is correct, receives the confirmation result of the confirmation from the user, and based on the confirmation result The link information and the determination result are stored in the determined link information storage means,
The determined link information reading means reads the determined link information and the determination result again from the determined link information storage means after the feedback means stores the link information and the determination result in the determined link information storage means.
The link feature extraction means re-extracts link feature information based on the determined link information read again by the determined link information reading means,
The relationship information deriving unit derives the relationship information again using the link feature information extracted again by the link feature extracting unit based on the determined link information and the determination result read again by the determined link information reading unit. Item 3. The link diagnosis apparatus according to item 2. A feedback means for storing the determination result for the undecided link by the inconsistency determining means and the link information of the link in the determined link information storage means;
The feedback means outputs a determination result every time a determination is made for the undetermined link, prompts the user to confirm whether the determination result is correct, and receives the confirmation result of the confirmation from the user, Based on the confirmation result, the link information and the determination result are stored in the determined link information storage means,
The determined link information reading means reads the determined link information and the determination result again from the determined link information storage means after the feedback means stores the link information and the determination result in the determined link information storage means.
The link feature extraction means re-extracts link feature information based on the determined link information read again by the determined link information reading means,
The relationship information deriving unit derives the relationship information again using the link feature information extracted again based on the determined link information by the link feature extracting unit and the determination result read again by the determined link information reading unit,
The undetermined link information reading means reads the new undetermined link information each time new undetermined link information is added to the undetermined link information storage means.
The link feature extraction means extracts link feature information based on the new undetermined link information read by the undetermined link information reading means,
The inconsistency determination means uses the link feature information extracted by the link feature extraction means based on the new undetermined link information and the relationship information derived again by the relationship information derivation means, so that the undetermined link is not determined. The link diagnosis apparatus according to claim 2 or 3, wherein it is determined whether or not matching has occurred. An optimum combination determining means for determining a combination of link feature information and a relational information derivation algorithm that gives the highest accuracy of the determination result of whether or not there is a mismatch in the link;
The optimum combination determining means is
Let the determined link information reading means read the determined link information and the determination result from the determined link information storage means,
A link feature extracting unit extracts a plurality of types of link feature information based on the determined link information,
Determining a plurality of combinations consisting of a part or all of the plurality of types of link feature information and one type of a plurality of types of relationship information derivation algorithms;
The relationship information deriving means uses the link feature information constituting the combination and the determination result read by the determined link information reading means in accordance with the relationship information deriving algorithm constituting the combination for each combination. Derived,
For the inconsistency determination means, for each of the individual combinations, for the link for which a determination result has already been obtained, the determination is performed again using the relationship information derived according to the combination,
By comparing the result of the determination by the inconsistency determination unit again with the already obtained determination result for each combination, the accuracy of the result of the determination by the inconsistency determination unit is determined again for each combination, The link diagnosis apparatus according to any one of claims 2 to 4, wherein a combination having the highest accuracy is specified.   The link feature extraction means includes, as link feature information, link feature information that can be specified from a link that is a target of determination as to whether or not inconsistency occurs, a link that is a target of determination, and a description content of a link destination document of the link Link feature information that can be specified from the link, or one or more types of link feature information indicating the size of the group to which the link to be determined belongs when the links are grouped according to a predetermined condition The link diagnosis apparatus according to claim 1, wherein the link diagnosis apparatus extracts one of the following. Link feature extraction means
The keyword included in the link source notation that corresponds to the link feature information that can be identified from the link to be judged,
Domain name or directory name included in the link destination address,
Domain name or directory name included in the link source address,
Appearance position or display area of the link source notation in the document,
Alternatively, the number of times a keyword included in the link source notation corresponding to link feature information that can be specified from the link to be determined and the description content of the link destination document of the link appears in the title of the link destination document,
The number of times a keyword included in the link source notation appears in the text of the linked document,
The number of times a keyword included in the link source notation is highlighted in the linked document,
Or, when links are grouped according to a predetermined condition, a link or determination for deriving relationship information within a group with the same link source notation that corresponds to the link feature information indicating the size of the group to which the determination target link belongs Percentage of links that have the same destination address as the target link,
Percentage of links in the group with the same link destination address that have the same link source notation as the link for deriving the relationship information or the link to be determined,
Percentage of links with the same link source address that have the same link destination address as the link for deriving the relationship information or the link to be judged within the same link source notation group,
One or more types of the same link source address and the ratio of the link for deriving the relationship information or the link source notation same as the link to be judged in the group of the same link destination address The link diagnosis apparatus according to claim 6, wherein the information is extracted. The undetermined link information storage means is the link information of the undetermined link information indicating whether or not there is a mismatch among the link information indicating the hypertext link attribute that associates the designated document with the designated document. Store judgment link information,
The undetermined link information reading means reads the undetermined link information from the undetermined link information storage means,
The link feature extraction means extracts link feature information indicating the probability that a mismatch has occurred in the link based on the undetermined link information read by the undetermined link information reading means,
The relationship information storage means stores relationship information indicating the relationship between the link feature information of a link that has already been determined whether or not inconsistency has occurred and the determination result of the determination,
A link diagnosis characterized in that a mismatch determination means determines whether or not a mismatch occurs in the undetermined link using the link feature information extracted by the link feature extraction means and the relation information. Method. The undetermined link information storage means is the link information of the undetermined link information indicating whether or not there is a mismatch among the link information indicating the hypertext link attribute that associates the designated document with the designated document. Store judgment link information,
The determined link information storage means stores the determined link information that is the link information of the link that has already been determined whether or not inconsistency occurs in the link information, and the determination result of the determination,
The undetermined link information reading means reads the undetermined link information from the undetermined link information storage means,
The determined link information reading means reads the determined link information and the determination result from the determined link information storage means,
Based on the undetermined link information read by the undetermined link information reading means, the link feature extraction means extracts link feature information indicating the probability that the link is inconsistent, and reads the determined link information. Extract link feature information based on the determined link information read by the means,
Whether the relationship information deriving means uses the link feature information extracted by the link feature extracting means based on the determined link information and the determination result read by the determined link information reading means to determine whether or not there is a mismatch. Deriving relationship information indicating the relationship between the link feature information of a link that has already been determined and the determination result of the determination,
The relationship information storage means stores the relationship information derived by the relationship information derivation means,
The inconsistency determining means determines whether or not there is an inconsistency in the undetermined link using the link feature information extracted by the link feature extracting means based on the undetermined link information and the relation information. Link diagnosis method characterized by the above. The feedback means outputs the determination result of the undecided link by the inconsistency determination means, prompts the user to confirm whether the determination result is correct, and receives the confirmation result of the confirmation from the user,
The feedback means stores the determination result of the determination for the undetermined link and the link information of the link in the determined link information storage means based on the confirmation result,
The determined link information reading means reads the determined link information and the determination result again from the determined link information storage means after the feedback means stores the link information and the determination result in the determined link information storage means,
The link feature extraction means extracts link feature information again based on the determined link information read again by the determined link information reading means,
The relationship information deriving unit derives the relationship information again using the link feature information extracted again by the link feature extracting unit based on the determined link information and the determination result read again by the determined link information reading unit. Item 10. The link diagnosis method according to Item 9. The feedback unit outputs a determination result each time a determination is made on an undecided link by the inconsistency determination unit, prompts the user to confirm whether the determination result is correct, and gives the confirmation result of the confirmation from the user. Entered,
The feedback means stores the determination result of the determination for the undetermined link and the link information of the link in the determined link information storage means based on the confirmation result,
The determined link information reading means reads the determined link information and the determination result again from the determined link information storage means after the feedback means stores the link information and the determination result in the determined link information storage means,
The link feature extraction means extracts link feature information again based on the determined link information read again by the determined link information reading means,
The relationship information deriving unit derives the relationship information again using the link feature information extracted again by the link feature extracting unit based on the determined link information and the determination result read again by the determined link information reading unit,
The undetermined link information reading means reads the new undetermined link information each time new undetermined link information is added to the undetermined link information storage means.
Link feature extraction means extracts link feature information based on the new undetermined link information read by the undetermined link information reading means;
The inconsistency determination means uses the link feature information extracted by the link feature extraction means based on the new undetermined link information and the relationship information derived again by the relationship information derivation means, so that the undetermined link is not determined. The link diagnosis method according to claim 9 or 10, wherein it is determined whether or not matching occurs. The optimum combination determining means is
Let the determined link information reading means read the determined link information and the determination result from the determined link information storage means,
A link feature extracting unit extracts a plurality of types of link feature information based on the determined link information,
Determining a plurality of combinations consisting of a part or all of the plurality of types of link feature information and one type of a plurality of types of relationship information derivation algorithms;
The relationship information deriving means uses the link feature information constituting the combination and the determination result read by the determined link information reading means in accordance with the relationship information deriving algorithm constituting the combination for each combination. Derived,
For the inconsistency determination means, for each of the individual combinations, for the link for which a determination result has already been obtained, the determination is performed again using the relationship information derived according to the combination,
By comparing the result of the determination by the inconsistency determination unit again with the already obtained determination result for each combination, the accuracy of the result of the determination by the inconsistency determination unit is determined again for each combination, The link diagnosis method according to any one of claims 9 to 11, wherein a combination having the highest accuracy is specified.   The optimum combination determining unit again sets each combination including a part or all of the plurality of types of link feature information and one type of the plurality of types of relation information derivation algorithms, and the mismatch determination unit in each of the combinations. The link diagnosis method according to claim 12, wherein a list of the accuracy of the determined result is displayed.   The link feature extraction means, as the link feature information, link feature information that can be specified from the link to be judged whether or not inconsistency occurs, the link to be judged and the description content of the link destination document of the link Link feature information that can be specified from the link, or one or more types of link feature information indicating the size of the group to which the link to be determined belongs when the links are grouped according to a predetermined condition The link diagnosis method according to any one of claims 8 to 13, wherein the link is extracted. Link feature extraction means
The keyword included in the link source notation that corresponds to the link feature information that can be identified from the link to be judged,
Domain name or directory name included in the link destination address,
Domain name or directory name included in the link source address,
Appearance position or display area of the link source notation in the document,
Alternatively, the number of times a keyword included in the link source notation corresponding to link feature information that can be specified from the link to be determined and the description content of the link destination document of the link appears in the title of the link destination document,
The number of times a keyword included in the link source notation appears in the text of the linked document,
The number of times a keyword included in the link source notation is highlighted in the linked document,
Or, when links are grouped according to a predetermined condition, a link or determination for deriving relationship information within a group with the same link source notation that corresponds to the link feature information indicating the size of the group to which the determination target link belongs Percentage of links that have the same destination address as the target link,
Percentage of links in the group with the same link destination address that have the same link source notation as the link for deriving the relationship information or the link to be determined,
Percentage of links with the same link source address that have the same link destination address as the link for deriving the relationship information or the link to be judged within the same link source notation group,
One or more types of the same link source address and the ratio of the link for deriving the relationship information or the link source notation same as the link to be judged in the group of the same link destination address The link diagnosis method according to claim 14, wherein the link information is extracted. The relationship information deriving means is designated by the user the type of the link feature information, the type of the relationship information deriving algorithm, and the parameter applied to the relationship information deriving algorithm, and the type specified by the user in the link feature extracting means Link feature information for
The relationship information specified by the user using the link feature information specified by the user and extracted from the determined link information and the determination result read by the determined link information reading unit. Deriving relationship information according to the derivation algorithm and parameters,
The relationship information storage means stores the relationship information derived by the relationship information derivation means,
The inconsistency determining means determines whether or not an inconsistent link is inconsistent using the link feature information specified by the user and extracted based on the undetermined link information and the relation information. The link diagnosis method according to any one of claims 9 to 15.   The inconsistency determination means uses the link feature information and the relationship information to determine whether or not there is an inconsistency in the undetermined link, calculates a certainty factor indicating the reliability of the determination result, and performs the determination. The link diagnosis method according to any one of claims 8 to 16, wherein a determination result and a certainty factor are displayed for each link.   18. The link diagnosis method according to claim 17, wherein the inconsistency determination means displays the link information of each determined link by color coding according to the certainty factor.   19. The link diagnosis method according to claim 17 or 18, wherein the inconsistency determination means rearranges and displays the link information of each determined link by using the certainty factor as a sort key.   A group of links in which the inconsistency determination unit determines whether or not inconsistency is determined, a link group that has the same link source notation as the determination target, and a link to be determined A link group having a common link destination address, a link group having the same link source address and link source notation as the determination target link, and a link having the same link source address and link destination address as the determination target link The link diagnosis method according to any one of claims 8 to 16, wherein any one group or any one of a plurality of groups is displayed.   The inconsistency determining means displays a group constituted by each link included in the document that is the link source of the link determined to have inconsistency, and the user inputs the specification of the document. The link diagnosis method according to any one of claims 8 to 16, wherein, when input, the document is displayed and a location of a link having inconsistency is specified and displayed. Undecided to store undetermined link information, which is link information of undetermined links in the link information indicating the hypertext link attributes that associate the designated document with the designated source document Link information storage means, and relation information storage means for storing relation information indicating the relation between the link feature information of a link for which a determination as to whether or not inconsistency has already occurred and the determination result of the determination is stored. Computer
A process of reading the undetermined link information from the undetermined link information storage means;
Based on the read undetermined link information, using the process of extracting link feature information indicating the probability that the link is inconsistent, and using the extracted link feature information and the relationship information, A link diagnostic program for executing processing to determine whether or not inconsistency occurs in an undetermined link. Undecided to store undetermined link information, which is link information of undetermined links in the link information indicating the hypertext link attributes that associate the designated document with the designated source document A determined link that stores link information storage means, determined link information that is link information of a link for which inconsistency has already been determined in the link information, and the determination result of the determination Information storage means, and relation information storage means for storing relation information indicating the relation between the link feature information of a link for which determination of whether or not inconsistency has already been made and the determination result of the determination are provided On the computer,
A process of reading the undetermined link information from the undetermined link information storage means;
Processing for reading the determined link information and the determination result from the determined link information storage means;
A process of extracting link feature information indicating the probability of inconsistency in the link based on the read undetermined link information;
A process of extracting link feature information based on the read determined link information;
Processing for deriving relationship information using link feature information extracted based on the determined link information and the read determination result, and link feature information extracted based on undecided link information and the relationship information And a link diagnosis program for executing a process of determining whether or not an inconsistency occurs in the undetermined link. On the computer,
A process for outputting a determination result of a determination for an undetermined link, prompting the user to confirm whether the determination result is correct, and inputting the confirmation result of the confirmation from the user;
A process for storing a determination result of determination for the undetermined link and link information of the link in a determined link information storage unit based on the confirmation result;
Processing for re-reading the determined link information and the determination result from the determined link information storage unit after storing the link information and the determination result in the determined link information storage unit;
Using the process of extracting link feature information again based on the determined link information read again, the link feature information extracted again based on the determined link information, and the determination result read again, the relationship information 24. The link diagnosis program according to claim 23, wherein a process for deriving data is executed again. On the computer,
A process of outputting a determination result each time a determination is made on an undetermined link, prompting the user to confirm whether the determination result is correct, and inputting the confirmation result of the confirmation from the user;
A process for storing a determination result of determination for the undetermined link and link information of the link in a determined link information storage unit based on the confirmation result;
Processing for re-reading the determined link information and the determination result from the determined link information storage unit after storing the link information and the determination result in the determined link information storage unit;
Processing to extract link feature information again based on the determined link information read again,
A process of deriving the relationship information again using the link feature information extracted again based on the determined link information and the determination result read again,
A process of reading the new undetermined link information each time new undetermined link information is added to the undetermined link information storage means;
Using the process of extracting link feature information based on the read new undetermined link information, and the link feature information extracted based on the new undetermined link information and the relationship information derived again, The link diagnosis program according to claim 23 or 24, wherein a process for determining whether or not inconsistency occurs in an undetermined link is executed. On the computer,
Executing the optimum combination determination process for determining the combination of the link feature information and the relationship information derivation algorithm that gives the highest accuracy of the determination result of whether or not the link is inconsistent,
In the optimal combination determination process,
Processing for reading the determined link information and the determination result from the determined link information storage means;
Processing for extracting a plurality of types of link feature information based on the determined link information;
A process of determining a plurality of combinations consisting of a part or all of the plurality of types of link feature information and one type of a plurality of types of relationship information derivation algorithms;
For each individual combination, a process for deriving relation information using link feature information constituting the combination and the read determination result in accordance with a relation information derivation algorithm constituting the combination;
For each of the individual combinations, a process for making a determination again using the relation information derived according to the combination for a link for which a determination result has already been obtained, and a result of the determination being already obtained. 24. By performing comparison with the determination result for each individual combination, the accuracy of the result determined again is obtained for each combination, and the processing for specifying the combination with the highest accuracy is executed. The link diagnosis program according to any one of claims 25. On the computer,
As link feature information, link feature information that can be specified from the link that is the target of determination as to whether or not inconsistency occurs, and link features that can be specified from the link to be determined and the description content of the link destination document of the link Executes the process of extracting one or more types of link feature information indicating the size of the group to which the determination target link belongs when the information is grouped according to information or a predetermined condition The link diagnosis program according to any one of claims 22 to 26. On the computer,
The keyword included in the link source notation that corresponds to the link feature information that can be identified from the link to be judged,
Domain name or directory name included in the link destination address,
Domain name or directory name included in the link source address,
Appearance position or display area of the link source notation in the document,
Alternatively, the number of times a keyword included in the link source notation corresponding to link feature information that can be specified from the link to be determined and the description content of the link destination document of the link appears in the title of the link destination document,
The number of times a keyword included in the link source notation appears in the text of the linked document,
The number of times a keyword included in the link source notation is highlighted in the linked document,
Or, when links are grouped according to a predetermined condition, a link or determination for deriving relationship information within a group with the same link source notation that corresponds to the link feature information indicating the size of the group to which the determination target link belongs Percentage of links that have the same destination address as the target link,
Percentage of links in the group with the same link destination address that have the same link source notation as the link for deriving the relationship information or the link to be determined,
Percentage of links with the same link source address that have the same link destination address as the link for deriving the relationship information or the link to be judged within the same link source notation group,
One or more types of the same link source address and the ratio of the link for deriving the relationship information or the link source notation same as the link to be judged in the group of the same link destination address 28. The link diagnosis program according to claim 27, wherein the process for extracting the information is executed.

Images