JP2011022630A - Information processor and information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processor for suppressing the retrieval of not only a document having a similar sentence but also a document having a sentence whose syntax is not similar in an object document. <P>SOLUTION: The document storage means of an information processor stores a document; a sentence extraction means extracts a sentence from an object document; a sentence set generation means generates the set of sentences on the basis of the syntax of the sentence extracted by the sentence extraction means; a similar sentence retrieval means retrieves a second sentence similar to a first sentence in the set of sentences generated by the sentence set generation means from a sentence in the document stored in the document storage means, and a relevant document retrieval means retrieves the document relative to the object document from the document storage means on the basis of the second sentence retrieved by the similar sentence retrieval means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and an information processing program.

  Japanese Patent Laid-Open No. 2004-260688 has an object of obtaining a search result with a small amount of search dust while preventing a search omission corresponding to various expressions, an input step of inputting a search sentence, and an input search sentence as an analysis word A morphological analysis step that divides into words based on a dictionary, a syntax analysis step that analyzes syntactic dependency between words, a primary search step that searches a document database based on the syntax analysis result of the syntax analysis step, and an area It has a semantic collation step that performs semantic collation between the retrieval sentence and the retrieval result of the primary retrieval step based on the conceptual knowledge database storing the dependent conceptual knowledge, and outputs a similar or dissimilar retrieval result. It is disclosed.

JP 2000-242650 A

  The present invention provides an information processing apparatus and an information processing program that suppress a search for a document having a sentence similar to a sentence whose syntax is not similar in a target document. It is an object.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to a first aspect of the present invention, there is provided a document storage unit that stores a document, a sentence extraction unit that extracts a sentence from a target document, and a set of the sentences based on a sentence syntax extracted by the sentence extraction unit. A sentence set generating means to be generated and a second sentence similar to the first sentence in the sentence set generated by the sentence set generating means are searched from sentences in the document stored in the document storage means. A similar sentence search unit; and a related document search unit that searches the document storage unit for a document related to the target document based on the second sentence searched by the similar sentence search unit. Is an information processing apparatus.

  The invention according to claim 2 is the number of the second sentences included in the document searched by the related document search means, the similarity of the first sentence corresponding to the second sentence, and the second sentence. The related document based on the comparison result of the appearance order of the first sentence in the target document corresponding to and the appearance order of the second sentence in the document searched by the related document search means, or a combination thereof The information processing apparatus according to claim 1, further comprising ordering means for ordering documents searched by the search means.

  The invention according to claim 3 is characterized in that the ordering unit performs the ordering based on the similarity between the document searched by the related document search unit and the target document. Information processing apparatus.

  According to a fourth aspect of the present invention, there is provided a computer based on document storage means for storing a document, sentence extraction means for extracting a sentence from a target document, and syntax of the sentence extracted by the sentence extraction means. A sentence set generating means for generating a set of sentences, and a sentence in a document in which a second sentence similar to the first sentence in the sentence set generated by the sentence set generating means is stored in the document storage means Based on the second sentence searched by the similar sentence search means and the related sentence search means for searching from the document storage means for the document related to the target document based on the second sentence searched by the similar sentence search means An information processing program characterized by this.

  According to the information processing apparatus of the first aspect, it is possible to suppress searching for a document having a sentence similar to a sentence whose syntax is not similar in the target document.

  According to the information processing apparatus of the second aspect, documents related to the target document can be ordered.

  According to the information processing apparatus of the third aspect, it is possible to separate a document that is similar to the target document and a document that is a document that is not similar to the target document but has a sentence similar to a sentence in the set.

  According to the information processing program of the fourth aspect, it is possible to suppress searching for a document having a sentence similar to a sentence whose syntax is not similar in the target document.

It is a conceptual module block diagram about the structural example of this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is explanatory drawing which shows the example of the document memorize | stored in the document storage module of this Embodiment. It is explanatory drawing which shows the example of the sentence in a target document. It is explanatory drawing which shows the example of the sentence in the document ranked highly by the related document ranking module of this Embodiment. It is explanatory drawing which shows the example of the sentence in the document ranked highly by the related document ranking module of this Embodiment. It is explanatory drawing which shows the example of the sentence in a target document. It is explanatory drawing which shows the example of the sentence in the document ranked highly by the related document ranking module of this Embodiment. It is explanatory drawing which shows the example of a data structure of a sentence and a document table. It is explanatory drawing which shows the data structure example of a sentence and a group table. It is explanatory drawing which shows the data structure example of a similarity table. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment also serves as an explanation of a computer program, a system, and a method. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. In addition, the modules correspond almost one-to-one with the functions. However, in mounting, one module may be composed of one program, or a plurality of modules may be composed of one program. A plurality of programs may be used. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.).
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point.

  The information processing apparatus according to the present embodiment searches for a document related to a target document. As shown in FIG. 1, a document reception module 110, a document storage module 120, a sentence extraction module 130, and the like It has a syntax set generation module 140, a similar sentence search module 150, a related document search module 160, a related document ranking module 170, and a related document output module 180.

In particular, this information processing apparatus may be applied to the following document management system.
In recent years, there has been a growing need for strict document management against the background of social demands for thorough compliance with companies. For example, in the content of products such as RoHS (Restricting the use of Hazardous Substances), REACH (Registration, Evaluation, Authorization and Restriction of Chemicals) It is indispensable to accurately describe the declaration of conformity to prove that the information is confidential and the confidential information indication given to the document to indicate the confidentiality level and disclosure range of the confidential information. For this purpose, for example, to confirm whether the description of the newly created conformity declaration conforms to standards (definition documents) such as RoHS and REACH to be conformed, conformance declaration of similar products, etc. In addition, it is necessary to refer to them as reference information. In this case, a document management system is used.
Hereinafter, a case where the information processing apparatus is used for such document management will be mainly described as an example.

The document reception module 110 is connected to the sentence extraction module 130. When searching for related documents, a document to be searched is received. Documents are composed of text data, and may include electronic data such as images, moving images, audio, etc., or a combination thereof, and are subject to storage, editing, search, etc. Those that can be exchanged as individual units between persons, including those similar to these. The target document contains a sentence. Further, the target document may be one or a plurality of documents. For example, this is a newly created declaration of conformity.
To accept a document is, for example, reading a document stored in a hard disk (including those connected to a computer in addition to those built in a computer), and reading with a scanner, a camera, etc. This includes recognizing characters in images.

The document storage module 120 is accessed from the sentence extraction module 130, the similar sentence search module 150, the related document search module 160, and the related document ranking module 170. A document stored in the document storage module 120 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of a document stored in the document storage module 120 of the present embodiment. The document storage module 120 stores, for example, documents created in the past, such as a definition document group 310, a conformity declaration group 320 from a supplier, a design specification group 330, and the like, and targets documents in the created document group 340. Remember as a document.
Documents of the definition document group 310 include documents such as “RoHS Directive” and “REACH revision”. Documents of the conformity declaration group 320 from the supplier include documents such as “Declaration of conformity of part a from the supplier”. Yes, as the document of the design specification group 330, there is a document such as “design specification of XX product A”, and the document received by the document reception module 110 and stored in the document storage module 120 is “XX product”. A's XX Declaration of Conformity ”.

  The sentence extraction module 130 is connected to the document reception module 110, the document storage module 120, and the related document ranking module 170. A sentence is extracted from the target document received from the document reception module 110, and the sentence is passed to the similar syntax set generation module 140. In addition, a sentence may be extracted from a document stored in the document storage module 120, and the sentence may be stored in the document storage module 120. The sentence extraction module 130 divides the text data in the document into a plurality of sentences according to the punctuation marks and line feed symbols. In other words, the sentence is cut out with the point where there is a punctuation or a line break as a break.

The similar syntax set generation module 140 is connected to the sentence extraction module 130 and the similar sentence search module 150. Based on the syntax of the sentence extracted by the sentence extraction module 130, a set of the sentences is generated.
The documents related to compliance such as the declaration of conformity and the confidential information display described above have a feature that a plurality of descriptions indicating conformity with the compliance item are described side by side. For example, in the declaration of conformity, a juxtapositional description such as “hexavalent chromium is 500 ppm or less, polybrominated diphenyl ether is 20 ppm or less, etc.” continues. The similar syntax set generation module 140 generates a set of sentences in which a plurality of facts and definitions are described side by side when a conformance declaration document or a document with confidential information display is targeted.

More specifically, for example, the similar syntax set generation module 140 compares the similarities of sentences received from the sentence extraction module 130 in a brute force manner, and selects a group of sentences having a similarity equal to or higher than a predetermined threshold T1. Identify. For example, sentence comparison means disclosed in Patent Document 1 is used for determining the similarity. That is, it has an analysis word dictionary, and a sentence is divided into words based on the analysis word dictionary. Analyze the syntactic dependency between the words. A set of sentences is generated based on the syntax structure as a result of the parsing. That is, sentences whose syntax structures have a similarity (including a case where they match) of the threshold value T1 or more are grouped. Furthermore, it has a thesaurus, and based on the thesaurus, the corresponding words in the syntax are similar on the thesaurus (for example, specifically, the distance on the thesaurus is predetermined) (Threshold value T2 or less) may be added as a grouping condition.
Further, when a plurality of sentence sets are created, only a set in which the number of sentences belonging to the set is larger than a preset threshold value T3 is left, and the other sets are excluded from processing targets.

The similar sentence search module 150 is connected to the document storage module 120, the similar syntax set generation module 140, the related document search module 160, and the related document ranking module 170. A sentence in a document in which a second sentence similar to the first sentence in the sentence set generated by the similar syntax set generation module 140 (hereinafter also referred to as “similar sentence”) is stored in the document storage module 120. Search from.
For example, specifically, a sentence obtained by the sentence extraction module 130 from each document stored in the document storage module 120 for each first sentence belonging to the same set obtained from the similar syntax set generation module 140. A sentence similar to each first sentence is searched for. That is, the similarity between two sentences is calculated, and a sentence whose similarity is larger than a preset threshold T4 is used as a search result. When a plurality of sets are obtained by the similar syntax set generation module 140, this search process is executed for each set.

The similar sentence search method used in the similar sentence search module 150 is different from the sentence set generation method used in the similar syntax set generation module 140. The similar sentence search module 150 uses a search method that places importance on word matching. For example, the word vector method disclosed in “Fundations of Statistical Natural Language Processing, The MIT Press (1999)” may be used.
However, for numerical words, specific numerical values such as “500” and “20” are not used, but are handled as “numerical expression” in a unified manner.

  The related document search module 160 is connected to the document storage module 120, the similar sentence search module 150, and the related document ranking module 170. Based on the similar sentence searched by the similar sentence search module 150, a document related to the target document is searched from the document storage module 120. For example, specifically, among the documents stored in the document storage module 120, a document including a plurality of sentences obtained as a search result from the similar sentence search module 150 is extracted.

The related document ranking module 170 is connected to the document storage module 120, the sentence extraction module 130, the similar sentence search module 150, the related document search module 160, and the related document output module 180. The related document ranking module 170 ranks the documents obtained from the related document search module 160 in descending order of relevance. The information used for ranking is as follows.
(1) The number of similar sentences included in the document searched by the related document search module 160. This is obtained by counting the number of similar sentences retrieved by the similar sentence retrieval module 150 for each document retrieved by the related document retrieval module 160.
(2) Similarity between a similar sentence included in a document searched by the related document search module 160 and a first sentence corresponding to the similar sentence. This uses the similarity calculated by the similar sentence search module 150 for each document searched by the related document search module 160.
(3) A comparison result of the appearance order of the first sentence in the target document corresponding to the similar sentence and the appearance order of the similar sentences in the document searched by the related document search module 160. This is obtained for each document searched by the related document search module 160 by comparing the appearance order of similar sentences in the document and the appearance order of the first sentence corresponding to the similar sentence in the target document. . Note that the value of the comparison result is calculated by a function that sets a high value when the appearance order is the same and a low value when the reverse order is the same.
(4) A combination of two or more of (1), (2) and (3) described above For example, one of the values of (1), (2) and (3), or The documents are arranged in order from the document with the largest combination (for example, the sum of these values, or an average value obtained by multiplying each value by a predetermined weighting factor).

  Further, the related document ranking module 170 may perform the above-described ordering based on the similarity between the document searched by the related document search module 160 and the target document. For example, the similarity between all the words in the document searched by the related document search module 160 and all the words in the target document is obtained by a method equivalent to the similarity calculation used in the similar sentence search module 150. The above-described ordering may be performed only for documents higher than a predetermined threshold value T5. In other words, ordering is performed on documents that are similar to each other as a whole. In addition, the above-described ordering may be performed only for documents lower than a predetermined threshold T6. In other words, the ordering is performed for documents that have sentences that are not similar to each other but that are similar to sentences in the set.

  The related document output module 180 is connected to the related document ranking module 170. The documents ranked by the related document ranking module 170 are output. The output document may be the document itself or a list of attributes (for example, titles) of the document. In addition, output means, for example, displaying on a display device such as a display, printing with a printing device such as a printer, sending an image with an image transmission device such as a fax, or document to a document storage device such as a document database. , Storing in a storage medium such as a memory card, passing to another information processing apparatus, and the like.

FIG. 2 is a flowchart showing an example of processing according to this embodiment. This will be described using a specific example. In this example, the sentence extraction module 130 extracts a sentence from the document in the document storage module 120 in advance. The sentence extraction result is stored in the sentence / document table 900, for example. FIG. 9 is an explanatory diagram showing an example of the data structure of the sentence / document table 900. The sentence / document table 900 includes a sentence ID column 902, a sentence column 904, and a document ID column 906. That is, the sentence is associated with the document.
The sentence ID column 902 stores a sentence ID (IDentification) that uniquely identifies the extracted sentence.
The sentence column 904 stores the extracted sentence.
The document ID column 906 stores a document ID that uniquely identifies the document from which the sentence is extracted.

In step S202, the document reception module 110 receives the target document.
In step S204, the sentence extraction module 130 extracts a sentence from the text data in the received document. FIG. 4 is an explanatory diagram illustrating an example of a sentence in the target document (conformance declaration document), in which the sentence 410 is extracted from the sentence 402. FIG. 7 is an explanatory diagram illustrating an example of a sentence in a target document (a document including confidential information display), and is an example in which sentences of the sentence 708 are extracted from the sentence 702. In other words, a sentence is extracted every time either a punctuation mark or a line feed symbol is found.

In step S206, the similar syntax set generation module 140 calculates the similarity between sentences (based on the similarity of the syntax structure), and classifies the sentences into groups. For example, among the sentences illustrated in FIG. 4, the sentence 402, the sentence 406, and the sentence 408 include juxtaposed description sentences (syntax structure including “<substance name> in the main part and <numeric expression><unit>”. And include them as a group. Sentences 702 to 708 illustrated in FIG. 7 are juxtaposed description sentences, which are specified as a group. The result of grouping the sentences in the target document is stored in the sentence / group table 1000, for example. FIG. 10 is an explanatory diagram showing an example of the data structure of the sentence / group table 1000. The sentence / group table 1000 has a sentence ID column 1002, a sentence column 1004, and a group column 1006.
The sentence ID column 1002 stores a sentence ID that uniquely identifies a sentence in the target document.
The sentence column 1004 stores the extracted sentence.
The group column 1006 stores a group ID that is a result of grouping. In the example of FIG. 10, the sentences “A-005” and “A-007” belong to the same group.

In step S <b> 208, the similar sentence search module 150 searches for similar sentences (based on word similarity) of each sentence belonging to the group for documents created in the past (documents in the document storage module 120). .
For example, the sentence 406 illustrated in FIG. 4 is “the content of polybrominated diphenyl ether is 20 ppm or less.”
As a sentence similar to
“Polybrominated diphenyl ether has a content of 10 ppm or less.”
“Contain polybrominated diphenyl ether to 50 ppm or less.”
Etc. are obtained as search results. That is, since these sentences contain the same words as “polybrominated diphenyl ether”, “contained”, ““ numerical expression ”ppm”, and “below” in the sentence 406, they are searched as similar sentences.
The similar sentence search result is stored in, for example, the similarity table 1100. FIG. 11 is an explanatory diagram showing an example of the data structure of the similarity table 1100. The similarity table 1100 has a target sentence ID column 1102, a similar sentence ID column 1104, a document ID column 1106, and a similarity column 1108.
The target sentence ID column 1102 stores the sentence ID of the sentence in the target document.
The similar sentence ID column 1104 stores sentence IDs of similar sentences that are search results.
The document ID column 1106 stores the document ID including the similar sentence.
The similarity column 1108 stores the similarity between the target sentence and the similar sentence.

In step S210, the related document search module 160 searches the document storage module 120 for a document including the obtained similar sentence. For example, the document ID column 1106 corresponding to the similar sentence ID of the similarity table 1100 may be used.
In step S212, the related document ranking module 170 determines the ranking of the document based on the number of similar sentences in the document. For example, the sentence 402, the sentence 406, and the sentence 408 illustrated in FIG. 4 are similar in word to the sentences, and the appearance order of these sentences is the same, corresponding to the sentences 402, 406, and 408 in FIG. Documents exemplified in sentences 502, 504, and 506) as sentences and sentences illustrated in FIG. 6 (sentences 602, 604, and 606 as sentences corresponding to sentences 402, 406, and 408) are ranked higher. Also, the sentences corresponding to FIG. 8 (sentences 702, 704, 706, and 708 in which the sentences are similar to the sentences in the sentences 702 to 708 illustrated in FIG. 7 and the appearance order of those sentences is the same. Documents 802, 806, 810, and 814) are ranked higher.

  Note that the hardware configuration of the computer on which the program according to the present embodiment is executed is a general computer, specifically a personal computer, a computer that can be a server, and the like, as illustrated in FIG. That is, as a specific example, the CPU 1201 is used as a processing unit (calculation unit), and the RAM 1202, the ROM 1203, and the HD 1204 are used as storage devices. For example, a hard disk may be used as the HD 1204. A CPU 1201 that executes programs such as a sentence extraction module 130, a similar syntax set generation module 140, a similar sentence search module 150, a related document search module 160, a related document ranking module 170, a RAM 1202 that stores the programs and data, and this computer ROM 1203 that stores a program for starting up, an HD 1204 as an auxiliary storage device, an input device 1206 for inputting data such as a keyboard and a mouse, an output device 1205 such as a CRT and a liquid crystal display, and a network interface A communication line interface 1207 for connecting to a communication network such as a card, and a bus 1208 for connecting them to exchange data. A plurality of these computers may be connected to each other via a network.

Among the above-described embodiments, the computer program is a computer program that reads the computer program, which is software, in the hardware configuration system, and the software and hardware resources cooperate with each other. Is realized.
Note that the hardware configuration shown in FIG. 12 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 12, but is a configuration that can execute the modules described in the present embodiment. I just need it. For example, some modules may be configured by dedicated hardware (for example, ASIC), and some modules may be in an external system and connected via a communication line. A plurality of systems shown in FIG. 5 may be connected to each other via communication lines so as to cooperate with each other. In particular, in addition to personal computers, information appliances, copiers, fax machines, scanners, printers, and multifunction machines (image processing apparatuses having two or more functions of scanners, printers, copiers, fax machines, etc.) Etc. may be incorporated.

In the above-described embodiment, the document relating to the restriction rule is shown, but other documents may be targeted. The present invention can be applied to any document in which a plurality of sentences having similar syntax structures are included and these sentences are to be searched.
The technology described as the prior art may be adopted as the processing contents of each module in the above-described embodiment.
Further, in the description of the above-described embodiment, “more than”, “less than”, “greater than”, and “less than (less than)” in a comparison with a predetermined value contradicts the combination. As long as the above does not occur, “larger”, “smaller (less than)”, “more than”, and “less than” may be used.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray Disc (registered trademark), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM), flash Includes memory, random access memory (RAM), etc. .
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 110 ... Document reception module 120 ... Document storage module 130 ... Sentence extraction module 140 ... Similar syntax set generation module 150 ... Similar sentence search module 160 ... Related document search module 170 ... Related document ranking module 180 ... Related document output module

Claims (4)

Document storage means for storing a document;
A sentence extracting means for extracting a sentence from a target document;
A sentence set generation means for generating a set of sentences based on the syntax of the sentence extracted by the sentence extraction means;
Similar sentence search means for searching a sentence in the document stored in the document storage means for a second sentence similar to the first sentence in the sentence set generated by the sentence set generation means;
An information processing apparatus comprising: a related document search unit that searches the document storage unit for a document related to the target document based on the second sentence searched by the similar sentence search unit. The number of the second sentences included in the document searched by the related document search means, the similarity of the first sentence corresponding to the second sentence, and the target document corresponding to the second sentence A document searched by the related document search unit based on a comparison result of the appearance order of the first sentence and the appearance order of the second sentence in the document searched by the related document search unit, or a combination thereof The information processing apparatus according to claim 1, further comprising ordering means for ordering the items. The information processing apparatus according to claim 2, wherein the ordering unit performs the ordering based on a similarity between the document searched by the related document search unit and the target document. Computer
Document storage means for storing a document;
A sentence extracting means for extracting a sentence from a target document;
A sentence set generation means for generating a set of sentences based on the syntax of the sentence extracted by the sentence extraction means;
Similar sentence search means for searching a sentence in the document stored in the document storage means for a second sentence similar to the first sentence in the sentence set generated by the sentence set generation means;
An information processing program for causing a document related to the target document to function as a related document search unit that searches the document storage unit based on a second sentence searched by the similar sentence search unit.

Images