JP5299619B2 - Data inspection apparatus, data inspection method, and data inspection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for high-speed detection of data having a feature to be noted. <P>SOLUTION: A data inspection device includes: a data sequencing means 2 for generating sequenced data by arranging each item value of input data at a predetermined position in a squence; a binary inspection means 3 for inspecting in accordance with whether or not a predetermined item has a value, in regard to the sequenced data; and a detailed inspection means 4 for comparing the sequenced data having the value in the predetermined item with an inspection pattern prepared in advance, and for detecting sequenced data in which each item value of the sequenced data entirely satisfies conditions included in each item of the inspection pattern. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a data inspection apparatus, a data inspection method, and a data inspection program.

  Nowadays, it is called an information processing society, and various services using information processing technology (IT) are provided to people and information processing apparatuses themselves, and logs regarding the use of these services are recorded every day. The large amount of logs recorded in this way is very useful for finding problems or opportunities in people's actions and information processing apparatus operations. For example, by analyzing logs collected from computers and their networks, it is possible to discover actions that cause security problems due to users and computer viruses, etc., and in procurement systems for parts as well, delays in procurement from orders and logistics logs. It is possible to detect problems that lead to problems early and to investigate the causes after the fact.

  However, the log to be recorded has various element configurations and values as well as the type of service, and continues to be generated in large quantities. Furthermore, there are problems and opportunities that can be discovered only by combining multiple logs. Therefore, there is a demand for a means for quickly detecting a remarkable thing from a combination of a large number and a plurality of types of logs.

As a technology related to log analysis, there is an appliance product for integrated log management as in Non-Patent Document 1, and a certain time from logs output from a plurality of security devices such as an IDS (intrusion detection system) and a firewall. It has functions to detect logs whose statistic exceeds the reference value, and to detect anomalies by pattern matching.
RSA enVision (http://japan.rsa.com/node.aspx?id=3170)

  However, with the related technologies as described above, it is necessary to collate with the detection conditions of the problem or chance that you want to detect all the acquired logs, so if the amount of log data increases, the processing time required increases accordingly. However, timely detection processing becomes difficult.

  An object of the present invention has been made in view of the above, and is a data inspection device, a data inspection method, and data that can quickly detect a certain condition from a large amount of data such as a log composed of a plurality of elements. To provide an inspection program.

  In order to solve the above-described problem, a data inspection apparatus according to the present invention is a data inspection apparatus that inspects input data composed of a set of one or more items and values of the items, and the input data is Data permutation means for generating permuted data by placing the values of the respective items at predetermined positions on a permutation prepared in advance, and whether or not a predetermined item has a value for the permuted data The binary inspection means for inspecting, the inspection pattern prepared in advance, the permutation data having a value in the predetermined item is compared with the inspection pattern, the value of each item of the permutation data And a detailed inspection means for detecting the permutation data that satisfies all the conditions of each item of the inspection pattern.

  In order to solve the above-mentioned problem, a data inspection method according to the present invention is a data inspection method for inspecting input data composed of a set of one or more items and values of the items, and the input data is A data permutation procedure for generating permuted data by placing the values of the respective items at predetermined positions on a permutation prepared in advance, and whether or not a predetermined item has a value for the permuted data The binary inspection procedure to be inspected, the inspection pattern prepared in advance, the permutation data having a value in the predetermined item, and the inspection pattern are compared, and the value of each item of the permutation data And a detailed inspection procedure for detecting the permutation data satisfying all the conditions of each item of the inspection pattern.

  In order to solve the above problem, a data inspection program according to the present invention is a data inspection program for inspecting input data composed of a set of one or more items and values of the items, and the input data is A data permutation process for generating permuted data by placing the values of the respective items at predetermined positions on a permutation prepared in advance, and whether or not a predetermined item has a value for the permuted data A binary inspection process to be inspected, a test pattern prepared in advance, the permutation data having a value in the predetermined item, and the inspection pattern are compared, and the value of each item of the permutation data And a detailed inspection process for detecting the permuted data that satisfies all the conditions of each item of the inspection pattern.

  According to the present invention, the data permutation means generates the permutation data by placing the values of the respective items included in the input data at predetermined positions on the permutation, and the binary inspection means generates a predetermined value on the permutation data. To provide a data inspection apparatus, a data inspection method, and a data inspection program capable of detecting at a high speed data having remarkable features from a large amount of data by inspecting whether or not an item has a value. it can.

[First Embodiment]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings. Here, FIG. 1 is a block diagram showing a data inspection apparatus according to the first embodiment of the present invention.

  Referring to FIG. 1, the data inspection apparatus according to the first embodiment of the present invention includes a data input means 1, a data permutation means 2, a binary inspection means 3, a detailed inspection means 4, and an inspection result output means. 5.

  Here, the data input means 1 provides a means for inputting one or more types of data composed of one or more items and values of the items as elements. Each item is further composed of a set of an item name and an item value.

  Typical target data is log data collected by, for example, a computer or communication device, such as a file investigation log, a file access log, and a device connection log as shown in FIG.

  As in the example of FIG. 9, the input data is composed of elements common to each other such as a date and individual elements such as a device type. The element structure of such data is not limited to a one-dimensional item string, and a table or tree structure with a larger number of rows is possible as long as the element can be uniquely converted into a one-dimensional permutation by a certain procedure. May be.

  The data permutation means 2 arranges the value of each item included in the data input by the data input means 1 at a predetermined position on a permutation prepared in advance, so that permutation data consisting of item strings in a predetermined order Convert to FIG. 2 is an explanatory diagram showing an example of converting various logs into permuted data by the data permutation means 2. The structure of the permuted data is determined by an item string in a certain order such as the correspondence table 21. The correspondence table 21 includes all items that may be necessary for the subsequent examination among items of various data input by the data input unit 1.

  In the data permutation means 2, the device connection log 22a is converted as permutation data 22b based on the correspondence table 21, and the file access log 23a is similarly converted as permutation data 23b. Of the items included in the correspondence table 21, items that are not included in the device connection log 22a and the file access log 23b are only items in the permutation data 22b and 23b and have no value.

  The binary inspection unit 3 inspects the permuted data converted into a fixed format by the data permutation unit 2 based on whether or not a predetermined item has a value. FIG. 3 is a block diagram detailing the functional configuration of the binary inspection means 3.

  The binarizing means 31 generates binary permuted data by replacing each item constituting the permuted data with another value indicating only the presence or absence of the value. For example, binary permutation data is generated by setting 1 as the new value of the item when the item has a value and 0 when there is no value.

  On the other hand, the comparison unit 32 compares the generated binary permutation data with the binary inspection pattern stored in the binary inspection pattern storage unit 33, and performs permutation corresponding to the binary permutation data satisfying a predetermined condition. Only the data is passed to the detailed inspection means 4. Here, the binary inspection pattern is composed of a permutation having predetermined items described in the correspondence table 21, and has different values for an item representing a feature to be detected and other items.

  FIG. 5 shows an example of specific processing by the binary inspection means 3. The binarizing means 31 newly sets 1 for the items A, B, F, G, and K having values and 0 for the other items constituting the permutation data 51. Binary permutation data 52 is generated.

  The comparison unit 32 compares the binary inspection pattern 53 stored in the binary inspection pattern storage unit 33 with the binary permutation data 52 and confirms whether or not a predetermined condition is satisfied.

  For example, it is assumed that the condition is satisfied if the value of the item of the binary permutation data 52 is 1 for all items of which the value is 1 in the binary inspection pattern 53. This can be easily determined by making both data into one-dimensional vectors, and confirming the presence or absence of positive or negative bits by subtraction between the vectors. Such a predetermined condition is stored in the binary inspection pattern storage means 33 in principle.

  FIG. 4 is a block diagram showing another functional configuration of the binary inspection means 3. In FIG. 4, the binarizing means 31, the comparing means 32, and the binary check pattern storage means 33 are the same as the means of the same name in FIG. In the configuration of FIG. 4, the binary inspection unit 3 further includes a matching case storage unit 34 and a case matching unit 35.

  The matching case storage means 34 stores binary permutation data 52 that matches the binary inspection pattern by the comparison means 32. If it is the same as any binary permutation data 52 already stored, only one is stored.

  The case collating means 35 compares the matching binary permutation data stored in the matching case storage means 34 with the new binary permutation data 52 acquired from the binarizing means 31, and if they match, the comparison of the next stage is performed. The processing of the means 32 is omitted, and the permutation data corresponding to the new binary permutation data is passed to the detailed inspection means 4 as the inspection result of the binary inspection means 3 as it is.

  The binary inspection unit 3 configured as shown in FIG. 4 is suitable when the comparison unit 32 has more complicated conditions for determining whether or not the permutation data matches the binary inspection pattern. . For example, when any one of two or more specific item sets among the items included in the binary inspection pattern is 1, it is determined that all the item sets are suitable, or the value of a specific item is When it is determined that the matching is performed only when the value of another specific item is different, there is a possibility that complicated comparison can be omitted by performing a simpler matching with a past matching case.

  The detailed inspection unit 4 inspects the values of each item constituting the permuted data with respect to the permuted data passed from the binary inspection unit 3 under more detailed conditions. FIG. 6 shows an example of specific processing by the detailed inspection means 4.

  Upon receiving the permutation data 61, the detailed inspection unit 4 compares this with the inspection pattern 62 prepared in advance. The inspection pattern 62 corresponds to at least one binary inspection pattern used by the binary inspection means 3. In this example, the inspection pattern 62 corresponds to the binary inspection pattern 53 of FIG. 5, the binary inspection pattern 53 defines an item whose value is 1, and the inspection pattern 62 defines specific conditions. On the other hand, no condition is defined in the inspection pattern 62 for items whose value is 0 in the binary inspection pattern 53. If the permuted data 61 to be inspected satisfies all the conditions of each item of the inspection pattern 62, the permuted data 61 is sent to the inspection result output means 5 as a specific type of data represented by the inspection pattern 62. Is output.

  In the example of FIG. 6, the permutation data 61 has values for the items A, B, F, G, and K, the inspection pattern 62 has values for the items A, B, and K (not NULL), and the items. The condition is that the value of G (manufacturing number) is “other than 125135” (≠ 125135). Since the permutation data 61 satisfies all the conditions of the inspection pattern 62, the result is a detailed inspection result 63. Here, for example, when the inspection pattern 62 indicates a problem action “a device other than the permitted serial number is connected to the host machine”, the permutation data 61 is detected as a problem action satisfying this condition.

  With the configuration and processing as described above, the data inspection apparatus according to the first embodiment of the present invention first selects candidates by simple binary inspection based on whether or not necessary items have values for the input data. Since detailed inspection is performed only on the narrowed down and narrowed candidates, a large amount of data can be inspected at high speed. Also, by preparing a plurality of pairs of binary inspection patterns and inspection patterns for detailed inspection, it is possible to inspect while identifying a plurality of types of problems and opportunities.

  FIG. 15 is a block diagram illustrating a hardware configuration of the data inspection apparatus according to the first embodiment.

  Referring to FIG. 15, the data inspection device 15 according to the present embodiment can be realized by a hardware configuration similar to a general computer device, and includes a CPU (Central Processing Unit) 151, a RAM (Random Access Memory), and the like. A main memory 152 used as a data work area and a primary data save area; a data output unit 153 by a liquid crystal display, a printer, a speaker, etc .; a data input unit 154 by a keyboard, mouse, scanner, etc. A communication unit 155 that transmits and receives data by connecting to a peripheral device, a ROM (Read Only Memory), an auxiliary storage unit 156 such as a hard disk device, and a system bus 157 that interconnects the above components are provided.

  The data inspection device 15 according to the embodiment of the present invention operates as a circuit component composed of hardware components such as an LSI (Large Scale Integration) in which a program for realizing such a function is incorporated in the data inspection device 15. As a matter of course, it can be implemented by hardware by executing the program that provides each function of each component described above by the CPU 151 of the computer.

  That is, the CPU 151 loads the program stored in the auxiliary storage unit 156 to the main storage unit 152 and executes it, and controls the operation of the data inspection device 15 to realize each function described above in software. .

According to the series of data inspection methods related to the operation of the data inspection apparatus 15 described above, one or more types of data composed of various sets of items are input, and from these data, first, a binary inspection with a small amount of processing is performed. By narrowing down candidates for problems and opportunities that are desired to be detected and performing detailed inspection only on the narrowed candidates, high-speed inspection can be realized.
[Second Embodiment]
A second embodiment of the present invention will be described. FIG. 7 is a block diagram showing a second embodiment of the data inspection apparatus according to the present invention.

  In FIG. 7, the data input means 1, the data permutation means 2a, the binary inspection means 3a, the detailed inspection means 4a, and the inspection result output means 5 function in the same manner as the means of the same name in the first embodiment.

  Referring to FIG. 7, in the second embodiment, in addition to the first embodiment, the inspection model storage unit 6, the inspection knowledge storage unit 7, the pattern permutation unit 8, the inspection pattern storage unit 9, and the correlation Data integration means 10 and permuted data storage means 11 are further provided.

  The inspection model storage unit 6 stores an inspection model including all types of items of data that can be input by the data input unit 1 as elements. The inspection model can be expressed by a tree structure as shown in the upper part of FIG. The inspection model 81 shown in FIG. 8 includes a root node “log”, a terminal node corresponding to each item, and an intermediate node in which several terminal nodes are collected.

  The data permutation means 2a in the second embodiment refers to terminal nodes representing specific items from the inspection model 81 in a predetermined procedure, and converts the data input by the data input means 1 into permutation data. . Alternatively, a permutation of terminal node names, that is, data item names, collected from the examination model 81 in a predetermined procedure is stored in advance as a correspondence table 82, and the correspondence table is the same as the data permutation means 1 in the first embodiment. May be used to convert the data into permuted data. As this predetermined procedure, for example, a method of searching a tree structure with depth priority and recording when a terminal node is found can be used. The inspection model may be expressed by a structure other than a tree structure as long as a unique permutation of items can be obtained from one static inspection model by a certain procedure.

  The inspection knowledge storage means 7 stores inspection knowledge in which a condition to be satisfied by a specific type of data to be detected is expressed based on the structure of the inspection model. FIG. 10 shows an example of inspection knowledge. The inspection knowledge 101 is expressed as a subtree of the inspection model 81, and includes the date and time of the date, the file path and importance of the file, the device type and serial number, the operation type, the operation type, and the host machine as data items. Includes machine name and IP address.

  In addition, specific conditions are attached to the importance of the file, the device type and serial number, and the operation type. According to FIG. 10, for example, if the inspection knowledge 101 represents a security problem act of “writing an important file to a non-permitted recording device”, the data indicating such an action is an item corresponding to the importance of the file. And the value of the item is 1 or more (“≧ 1”) is one of the necessary conditions.

  The pattern permutation unit 8 converts one or more pieces of inspection knowledge stored in the inspection knowledge storage unit 7 into inspection patterns each including a predetermined item sequence. For example, an inspection pattern 122 is generated from the inspection knowledge 121 as in the example of FIG. Here, the pattern permutation unit 8 first refers to the test model stored in the test model storage unit 6 and collects end nodes from the test model by, for example, depth-first search, as in the data permutation unit 2a described above. And generate a data item string.

  In the example of FIG. 12, the item string of the data is “date, time, file path, importance, type, name, serial number, process name, window title, operation type, machine name, as in the correspondence table 82 of FIG. IP address ", and replacing each item name with alphabets A, B, C, D, E, F, G, H, I, J, K, and L, the second line of the inspection pattern 122 in FIG. become. In addition, when the condition of the inspection knowledge corresponding to each item is acquired from the inspection knowledge 121 and assigned to the inspection pattern 122, the third line of the inspection pattern 122 is obtained.

  Further, the first line of the inspection pattern 122 describes a value representing the relationship between each intermediate node of the inspection knowledge 121 and its lower terminal node by a logical operator. In the inspection knowledge 121, the intermediate node “host machine” has an “OR” relationship in the sense that one of the end nodes “machine name” and “IP address” only needs to have a value. The other intermediate nodes are in an “AND” relationship in the sense that all the lower end nodes need to have values that satisfy a specific condition. When no condition is specified for the terminal node, the condition is that the item represented by the terminal node has some value.

  The inspection pattern 122 generated as described above is stored in the inspection pattern storage unit 9.

  The correlation data integration unit 10 integrates a plurality of correlated permutation data and generates one permutation data. FIG. 9 is an explanatory diagram showing examples of three correlated data, and FIG. 11 is an explanatory diagram showing permutation data obtained from each data of FIG. 9 and an integration example of the three permutation data.

  The determination of whether or not the permuted data correlates is, for example, when the value of a predetermined item is the same or within a certain difference between the permuted data acquired within a certain time. Can be determined. Further, whether or not the correlated permutation data are integrated into one permutation data can be made on condition that the value of a predetermined item is the same or that there is no conflicting item value.

  In the example of FIG. 11, the permutation data 111, the permutation data 112, and the permutation data 113 are items whose items A and B representing the acquisition date and time are within 180 seconds, and the item K representing the host machine. Since both values are “alice”, they are determined as correlation data. Furthermore, since these three permuted data have no different items other than the items A and B representing the date and time, it is determined that they can be integrated and integrated into the integrated permuted data 114. Yes.

  The permuted data storage unit 11 stores permuted data including the integrated permuted data as described above. However, when the integrated permutation data is stored, the individual permutation data that is the basis thereof is not stored. Further, the permutation data referred to by the binary inspection means 3a at the subsequent stage may be deleted from the permutation data storage means 11 after the processing by the detailed inspection means 4a and the inspection result output means 5 is finished.

  The binary inspection unit 3a in the second embodiment generates a binary inspection pattern from the inspection pattern stored in the inspection pattern storage unit 9, and performs the inspection in the same manner as the binary inspection unit 3 in the first embodiment. FIG. 12 shows an example in which a binary inspection pattern 123 is generated from the inspection pattern 122. As with the generation of binary permutation data from permuted data, the binary inspection pattern is generated by setting the value of an item with a condition description to 1 for each item of the inspection pattern and the values of the other items. The result set to 0 is used as a binary inspection pattern.

  FIG. 13 is an explanatory view showing another example of the inspection procedure by the binary inspection means 3 as the inspection procedure by the binary inspection means 3a.

  From the binary inspection pattern 131, it is possible to generate a vector 1 composed of the values of the items [1 1 1 1 1 0 1 0 0 1 1 1], and similarly from two types of binary inspection patterns, for example. Vector 2 and vector 3 as shown in FIG. 13 can be generated. Then, a matrix R having each vector as a row can be constructed from these three vectors.

  On the other hand, in order to detect binary permutation data conforming to the binary inspection pattern 131, the vector 1 generated from the binary inspection pattern 131 and a vector composed of the values of the respective items of the integrated binary permutation data similarly. It is sufficient that the product of VL exceeds the number of items of vector 1 and value 1, that is, the necessary bit total value. However, since the binary check pattern 131 has an OR relationship in which only one of the items K and L needs to have a value, the necessary bit total value is one less than the number of items of the vector 1 and the value of 8 That is all you need.

  Similarly, if the necessary bit total values of the vectors 2 and 3 are 4 and 7, respectively, the threshold vectors VS = [8 4 7] (T) having these as elements (where (T) is a row vector) In the matrix operation result vector obtained by subtracting the threshold vector VS from the product of the matrix R and the vector VL (T) of the binary permutation data. The corresponding row vector corresponds to the matching binary check pattern of the binary permutation data. In the example of FIG. 13, the matrix calculation result is [0 0 −1] (T), and the values in the first and second rows are 0 or more. Therefore, the binary check patterns corresponding to the vectors 1 and 2 are used. Is suitable.

  When one or more binary inspection patterns adapted by the procedure of FIG. 13 are obtained, the binary inspection means 3a converts the inspection pattern corresponding to each of the binary inspection patterns and the binary permutation data (or integrated binary permutation data). A pair with the corresponding original permutation data (or integrated permutation data) is passed to the detailed inspection means 4a.

  In the detailed inspection 4a, for example, as in the example of the procedure shown in FIG. 14, inspection is performed by collating the permuted data (or integrated permuted data) with each inspection pattern. In the example of FIG. 14, among all 12 items of the integrated permutation data 141, items A, B, C, D, E, F, G, J, and K have values. This is collated for each item with the inspection pattern 142 in which conditions are defined for nine items A, B, C, D, E, G, J, K, and L. If both the integrated permutation data 141 and the inspection pattern 142 have values or conditions, they are compared and if the value satisfies the conditions, the item is OK.

  If a value does not satisfy a condition for a certain item, or if there is no value and only a condition, the item is NG. Items that have no value or condition are OK. In the example of FIG. 14, only the item L has no value and the condition is defined, so that it is NG, and all other items are OK.

  However, in the inspection pattern 142, the item K and the item L are in an OR relationship, and if either one of the conditions is satisfied, the upper item of both items is OK, so the result is the detailed inspection result 143, and the integrated permutation It is determined that the digitized data 141 matches the inspection pattern 142. The adapted (integrated) permutation data is passed to the examination result output means 5 together with the examination pattern name to be adapted, for example, and outputted to an output device having a display screen or the like.

  The data inspection apparatus according to the second embodiment can be realized by the same hardware configuration as the data inspection apparatus according to the first embodiment shown in FIG.

  According to the present embodiment, it is possible to find one problem or opportunity even from a combination of a plurality of correlated data as compared with the first embodiment. In addition, it is possible to efficiently narrow down candidate data by collectively performing binary inspection with a plurality of inspection patterns on integrated permutation data obtained by integrating one permutation data or correlated permutation data. Furthermore, with the inspection model and inspection knowledge expressed based on the inspection model, it is possible to easily express to the user what problems and opportunities are detected based on the conditions for which items. Changes are also easier.

  Although the present invention has been described with reference to the preferred embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the technical scope of the present invention.

  The present invention is used to quickly detect, without omission, a large amount of log data collected and accumulated every day, such as security issues and other items that need to be dealt with promptly, or when a product is sold in a specific combination. It is effective for.

It is a block diagram which shows the function structure of the data inspection apparatus by the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the data permutation by the data permutation means of the 1st Embodiment of this invention. It is a block diagram which shows the function structure of the binary test | inspection means by the 1st Embodiment of this invention. It is a block diagram which shows another functional structure of the binary test | inspection means by the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the test | inspection by the binary test | inspection means of the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the test | inspection by the detailed test | inspection means of the 1st Embodiment of this invention. It is a block diagram which shows the function structure of the data inspection apparatus by the 2nd Embodiment of this invention. It is explanatory drawing which shows the example of a test | inspection model and a corresponding | compatible table. It is explanatory drawing which shows the example of the data correlated in the 2nd Embodiment of this invention. It is explanatory drawing which shows the example of the test | inspection knowledge in the 2nd Embodiment of this invention. It is explanatory drawing which shows integration of the correlated data by the 2nd Embodiment of this invention. It is explanatory drawing of the pattern permutation means by the 2nd Embodiment of this invention. It is explanatory drawing of the binary test | inspection means by the 2nd Embodiment of this invention. It is explanatory drawing of the detailed test | inspection means by the 2nd Embodiment of this invention. It is explanatory drawing which shows the hardware constitutions of the data inspection apparatus by the 1st and 2nd embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Data input means 2 Data permutation means 2a Data permutation means 3 Binary inspection means 3a Binary inspection means 4 Detailed inspection means 4a Detailed inspection means 5 Inspection result output means 6 Inspection model storage means 7 Inspection knowledge storage means 8 Pattern permutation means 9 Inspection pattern storage means 10 Correlation data integration means 11 Permutation data storage means 21 Correspondence table 22a Device connection log 22b Permutation data of device connection log 23a File access log 23b Permutation data of file access log 31 Binaryization means 32 Comparison means 33 Binary inspection pattern storage means 34 Applicable case storage means 35 Case collation means 51 Permutation data 52 Binary permutation data 53 Binary inspection pattern 61 Permutation data 62 Inspection pattern 63 Detailed inspection result 81 Inspection model 82 Response table 91 File investigation log 92 File access log 93 Device connection log 101 Inspection knowledge 111 Permutation data 112 Permutation data 113 Permutation data 114 Integrated permutation data 121 Inspection knowledge 122 Inspection pattern 123 Binary inspection pattern 131 Binary inspection pattern 141 Integration Permutation data 142 Inspection pattern 143 Detailed inspection result 15 Data inspection device 151 CPU
152 Main storage unit 153 Output unit 154 Input unit 155 Communication unit 156 Auxiliary storage unit 157 System bus

Claims (39)

A data inspection device for inspecting input data composed of a set of one or more items and values of the items,
Data permutation means for generating permutation data by placing the value of each item of the input data at a predetermined position on a permutation prepared in advance;
About the permuted data, binary inspection means for inspecting whether or not a predetermined item has a value;
The permutation data having a test pattern prepared in advance and having a value in the predetermined item are compared with the test pattern, and the value of each item of the permutation data has each item of the test pattern Detailed inspection means for detecting the permutation data satisfying all conditions;
A data inspection apparatus comprising:   The binary checking means generates binary permutation data by replacing the value of each item of the permutation data with another value that represents only the presence or absence of the value, and the binary permutation data is converted into the predetermined permutation data. Comparing to a binary check pattern that is composed of the permutation having items and has different values for the item representing the feature to be detected and the other items, the binary permutation data satisfying a predetermined condition among the permutation data 2. The data inspection apparatus according to claim 1, wherein only the corresponding permutation data is passed to the detailed inspection unit. The binary inspection means includes
Binarization means for generating the binary permutation data;
Binary inspection pattern storage means for storing the binary inspection pattern;
Each value of the item is compared between the binary permuted data and the binary inspection pattern, and when the comparison result satisfies a predetermined condition, the permuted data corresponding to the binary permuted data is used as a detection result. Comparison means for passing to the detailed inspection means;
The data inspection apparatus according to claim 2, further comprising:   The predetermined condition is that each value of the item is a one-dimensional vector between the binary permutation data and the binary check pattern, and the presence or absence of a positive or negative bit is confirmed by subtraction between the vectors. The data inspection apparatus according to claim 2 or 3, characterized in that   5. The data inspection apparatus according to claim 3, wherein the binary inspection pattern storage unit further stores the predetermined condition used by the comparison unit. The binary inspection means stores matching case storage means for storing the binary permutation data when the comparison means determines that the predetermined condition is satisfied,
The binary permutation data is compared with the binary permutation data stored in the matching case storage means, and if there is matching binary permutation data, the permutation data corresponding to the matched binary permutation data is obtained. A case verification unit that is the inspection result of the binary inspection unit and that omits the processing by the comparison unit;
The data inspection apparatus according to claim 3, further comprising: Inspection knowledge storage means for storing inspection knowledge including conformity conditions for each of the predetermined items for inspecting whether or not the input data is a specific type of data to be detected;
Pattern permutation means for generating an inspection pattern of the inspection knowledge by arranging the matching condition of the inspection knowledge at a predetermined position on the permutation;
Further comprising
The binary inspection unit uses the binary inspection pattern in which the value of each item of the generated inspection pattern is replaced with a value representing only the presence / absence of the matching condition for comparison with the binary permutation data by the comparison unit, The data inspection apparatus according to claim 2, wherein the detailed inspection unit detects the permuted data using the generated inspection pattern. For all items that can be included in the input data received by the data input means, further comprises an inspection model storage means for storing an inspection model composed of element sets each including an element corresponding uniquely.
8. The data inspection apparatus according to claim 7, wherein the data permutation means generates permutation data using a permutation of elements uniquely determined from the inspection model by a predetermined procedure.   The pattern permutation unit arranges each item included in the inspection knowledge stored in the inspection knowledge storage unit with respect to a permutation of elements uniquely determined from the inspection model by the predetermined procedure, thereby performing the inspection. The data inspection apparatus according to claim 8, wherein a pattern is generated.   The binary check means constitutes a matrix in which a column of condition of each item of the plurality of binary check patterns is a row vector or a column vector, respectively, and the column of values of each item of the binary permuted data is the same 10. The method of claim 4, further comprising: determining which binary check pattern the binary permutation data matches from a product operation of the matrix and the vector. Data inspection equipment.   11. The data inspection apparatus according to claim 7, further comprising inspection pattern storage means for storing inspection patterns generated by the pattern permutation means.   Correlation data integration means for integrating the permutation data correlated with each other under a predetermined condition among the plurality of permutation data into one permutation data before processing by the binary checking means is further provided. The data inspection apparatus according to any one of claims 1 to 11.   The permutation data storage means for storing the permutation data generated by the data permutation means or the permutation data integrated by the correlation data integration means is further provided. The data inspection apparatus according to item 1. A data inspection method for inspecting input data composed of a set of one or more items and values of the items,
A data permutation procedure for generating permuted data by placing the values of the items of the input data at predetermined positions on a permutation prepared in advance;
A binary inspection procedure for inspecting whether or not a predetermined item has a value for the permuted data;
The permutation data having a test pattern prepared in advance and having a value in the predetermined item are compared with the test pattern, and the value of each item of the permutation data has each item of the test pattern A detailed inspection procedure for detecting the permutation data satisfying all the conditions;
A data inspection method comprising:   The binary check procedure generates binary permutation data by replacing the value of each item of the permutation data with another value that represents only the presence or absence of the value, and the binary permutation data is converted into the predetermined permutation data. Comparing to a binary check pattern that is composed of the permutation having items and has different values for the item representing the feature to be detected and the other items, the binary permutation data satisfying a predetermined condition among the permutation data 15. The data inspection method according to claim 14, wherein only the corresponding permuted data is passed to the detailed inspection procedure. The binary inspection procedure is:
A binarization procedure for generating the binary permutation data;
A binary inspection pattern storage procedure for storing the binary inspection pattern;
Each value of the item is compared between the binary permuted data and the binary inspection pattern, and when the comparison result satisfies a predetermined condition, the permuted data corresponding to the binary permuted data is used as a detection result. A comparison procedure passed to the detailed inspection procedure;
The data inspection method according to claim 15, further comprising:   The predetermined condition is that each value of the item is a one-dimensional vector between the binary permutation data and the binary check pattern, and the presence or absence of a positive or negative bit is confirmed by subtraction between the vectors. The data inspection method according to claim 15 or 16, characterized in that   18. The data inspection method according to claim 16, wherein the binary inspection pattern storage procedure further stores the predetermined condition used in the comparison procedure. The binary inspection procedure includes a matching case storage procedure for storing the binary permutation data when it is determined that the predetermined condition is satisfied by the comparison procedure;
The binary permutation data is compared with the binary permutation data stored in the conformity case storage procedure. If there is matching binary permutation data, the permutation data corresponding to the matched binary permutation data is obtained. A case verification procedure that is an inspection result of the binary inspection procedure and omits the processing by the comparison procedure;
The data inspection method according to claim 16, further comprising: A test knowledge storage procedure for storing test knowledge including matching conditions for each of the predetermined items for testing whether the input data is a specific type of data to be detected;
A pattern permutation procedure for generating an inspection pattern of the inspection knowledge by arranging the matching condition of the inspection knowledge at a predetermined position on the permutation;
Further comprising
The binary inspection procedure uses the binary inspection pattern in which the value of each item of the generated inspection pattern is replaced with a value indicating only the presence or absence of the matching condition for comparison with the binary permutation data by the comparison procedure, The data inspection method according to claim 15, wherein the detailed inspection procedure detects the permuted data using the generated inspection pattern. A test model storage procedure for storing a test model composed of an element set including elements uniquely corresponding to all items that can be included in the input data received by the data input procedure;
The data inspection method according to claim 20, wherein the data permutation procedure generates permutation data using a permutation of elements uniquely determined from the inspection model by a predetermined procedure.   The pattern permutation procedure arranges each item included in the inspection knowledge stored in the inspection knowledge storage procedure with respect to a permutation of elements that are uniquely determined from the inspection model by the predetermined procedure. The data inspection method according to claim 21, wherein a pattern is generated.   The binary check procedure includes a matrix in which each item condition column of the plurality of binary check patterns is a row vector or a column vector, and the column of each item value included in the binary permuted data is the same. The vector according to any one of claims 17 to 22, wherein it is determined which binary check pattern the binary permutation data matches from a product operation of the matrix and the vector. Data inspection method.   The data inspection method according to any one of claims 20 to 23, further comprising a test pattern storage procedure for storing a test pattern generated by the pattern permutation procedure.   A correlation data integration procedure is further provided for integrating permutation data correlated with each other according to a predetermined condition among the plurality of permutation data into one permutation data before processing by the binary inspection procedure. The data inspection method according to any one of claims 14 to 24.   The permuted data storage procedure for storing the permuted data generated by the data permutation procedure or the permuted data integrated by the correlation data integration procedure is further provided. The data inspection method according to item 1. A data inspection program for inspecting input data composed of a set of one or more items and values of the items,
A data permutation process for generating permuted data by placing the values of the items of the input data at predetermined positions on a permutation prepared in advance;
A binary inspection process for inspecting whether or not a predetermined item has a value for the permuted data;
The permutation data having a test pattern prepared in advance and having a value in the predetermined item are compared with the test pattern, and the value of each item of the permutation data has each item of the test pattern A detailed inspection process for detecting the permuted data satisfying all the conditions;
A data inspection program for causing a computer to execute.   The binary inspection process generates binary permutation data by replacing the value of each item of the permutation data with another value that represents only the presence or absence of the value, and the binary permutation data is converted into the predetermined permutation data. Comparing to a binary check pattern that is composed of the permutation having items and has different values for the item representing the feature to be detected and the other items, the binary permutation data satisfying a predetermined condition among the permutation data 28. The data inspection program according to claim 27, wherein only the corresponding permuted data is passed to the detailed inspection process. The binary inspection process is:
Binarization processing for generating the binary permutation data;
A binary inspection pattern storage process for storing the binary inspection pattern;
Each value of the item is compared between the binary permuted data and the binary inspection pattern, and when the comparison result satisfies a predetermined condition, the permuted data corresponding to the binary permuted data is used as a detection result. A comparison process passed to the detailed inspection process;
29. The data inspection program according to claim 28, wherein the data is executed by a computer.   The predetermined condition is that each value of the item is a one-dimensional vector between the binary permutation data and the binary check pattern, and the presence or absence of a positive or negative bit is confirmed by subtraction between the vectors. 30. The data inspection program according to claim 28 or 29.   The data inspection program according to claim 29 or 30, wherein the binary inspection pattern storage processing further stores the predetermined condition used in the comparison processing. The binary inspection process includes a matching case storage process for storing the binary permutation data when it is determined by the comparison process that the predetermined condition is satisfied;
The binary permutation data is compared with the binary permutation data stored in the matching case storage process, and if there is matching binary permutation data, the permutation data corresponding to the matched binary permutation data is obtained. A case matching process which is an inspection result of the binary inspection process and omits the process by the comparison process;
32. The data inspection program according to claim 29, further causing a computer to execute. A test knowledge storage process for storing test knowledge including a matching condition for each of the predetermined items for testing whether the input data is a specific type of data to be detected;
A pattern permutation process for generating an inspection pattern of the inspection knowledge by arranging the matching condition of the inspection knowledge at a predetermined position on the permutation;
Is further executed on the computer,
The binary inspection process uses the binary inspection pattern in which the value of each item of the generated inspection pattern is replaced with a value indicating only the presence or absence of the matching condition for comparison with the binary permutation data by the comparison process, The data inspection program according to any one of claims 28 to 32, wherein the detailed inspection process detects the permuted data using the generated inspection pattern. For all items that can be included in the input data accepted by the data input process, the computer further executes a test model storage process for storing a test model configured by an element set including elements that uniquely correspond to each item.
34. The data inspection program according to claim 33, wherein the data permutation processing generates permutation data using a permutation of elements uniquely determined from the inspection model by a predetermined processing.   The pattern permutation process arranges each item included in the inspection knowledge stored in the inspection knowledge storage process with respect to a permutation of elements that are uniquely determined from the inspection model by the predetermined process. The data inspection program according to claim 34, wherein a pattern is generated.   The binary check processing is configured as a matrix in which a column of condition of each item of the plurality of binary check patterns is a row vector or a column vector, and the column of the value of each item of the binary permuted data is the same. 36. The method according to claim 30, wherein the binary permutation data is determined to be suitable for the binary permutation pattern based on a product operation of the matrix and the vector. Data inspection program.   37. The data inspection program according to claim 33, further causing a computer to execute an inspection pattern storage process for storing an inspection pattern generated by the pattern permutation process.   Causing the computer to further execute a correlation data integration process of integrating the permutation data correlated with each other according to a predetermined condition into one permutation data before the processing by the binary inspection process among the plurality of permutation data. 38. The data inspection program according to any one of claims 27 to 37, wherein:   The permuted data storage process for storing the permuted data generated by the data permutation process or the permuted data integrated by the correlation data integration process is further executed by the computer. The data inspection program according to any one of the above.

Images