JP2018142074A - Feature quantity calculation device, image similarity determination device, image retrieval device, feature quantity calculation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract such a feature quantity vector as to accurately retrieve image data similar to image data to which one or more processing treatments (for example, of trim, gradation, noise, color change, brightness change, contrast change, overwriting of letters and graphics, and so on) other than those of rotation and scale are applied even when a type and the number of the applied processing treatments are unclear.SOLUTION: A feature quantity calculation device for calculating from input image data a feature quantity vector showing a feature of the data comprises: image acceptance means to accept input of the image data; and feature quantity extraction means to extract from the image data the feature quantity vector, using feature quantity vector extraction methods predetermined for respective types of a plurality of processing treatments for the image data, for each of the extraction methods.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a feature amount calculation apparatus, a feature amount calculation method, and a program for calculating a feature amount vector indicating features of the data from image data. The present invention also relates to an image similarity determination device and an image search device using the calculated feature vector.

  As a means of searching for a target image from a large amount of image data, input the image data to be searched, evaluate the similarity of images based on the feature vector extracted from the image data, and search for similar image data The way to search is known. For extracting the feature vector, several methods have been proposed so far, such as SIFT (Scale Invariant Feature Transformation), SURF (Speed Up Robust Feature), ORB (Oriented BRIEF) (for example, non-patent literature). 1). In addition, a feature vector extraction method that can be executed with high search accuracy even when image data to be searched for has been rotated or scaled is disclosed (for example, Patent Document 1).

Japanese Patent No. 5819158

D. G. Low, “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, Vol. 60, no. 2, 91-110 (204).

  However, for image data to which changes other than rotation or scale, such as trim, contrast or brightness change, blurring, noise, color change, and one or more types of processing such as overwriting of characters and figures are added However, there has been no method for extracting a feature vector that can search for similar image data with high accuracy even if the type and number of the applied processing are unknown.

  Therefore, an object of the present invention is to provide a feature amount calculation device, an image similarity determination device, an image search device, a feature amount calculation method, and a program that can solve the above-described problems.

According to the first aspect of the present invention, the feature amount calculation apparatus includes an image receiving unit that receives input of image data, and a method of extracting a feature amount vector that is predetermined for each of a plurality of types of processing on the image data. And feature amount extraction means for extracting a feature amount vector for each of the extraction methods from the image data received by the image receiving means.
With such a configuration, it is possible to extract a feature quantity vector representing the characteristics of certain image data from the viewpoint of each of the plurality of processing processes.

  According to the second aspect of the present invention, the extraction method determined in advance for each type of processing is an extraction method for extracting a feature vector that does not change by the processing from image data.

According to the third aspect of the present invention, the type of the processing is rotation of image data, change of scale, change of aspect ratio, partial cutout, color change, monochrome conversion, graphic overwriting, character overwriting, trimming , Change in brightness, change in contrast, blur, and noise.
According to the second aspect or the third aspect, it is possible to extract a feature quantity vector that is not easily affected by the processing process for each processing process.

According to the fourth aspect of the present invention, the image similarity determination device is an image similarity determination device that determines the similarity between the input query image data and the reference image data. And a feature amount calculation unit according to any one of the above, and a feature amount extraction unit for reference that extracts a feature amount vector for each of the extraction methods from the reference image data using the feature amount calculation device according to any one of the above. A query feature quantity extraction means for extracting a feature quantity vector for each extraction method from the query image data using a device; a plurality of feature quantity vectors obtained from the query image data; and a plurality of features obtained from the reference image data A similarity calculation means for calculating the similarity between the query image data and the reference image data based on the feature quantity vector, and the query image data based on the calculated similarity. And and a similarity determination means for determining a similarity between the reference image data.
With such a configuration, even if one of the two target image data is image data obtained by performing one or more types of processing on the other image data, the two images The degree of similarity between data can be determined.

According to the fifth aspect of the present invention, the similarity calculation means calculates a similarity between feature quantity vectors extracted from the query image data and the reference image data using the same extraction method among the extraction methods. Then, the similarity between the query image data and the reference image data is calculated based on the similarity corresponding to each of the calculated plurality of extraction methods.
As described above, the similarity calculation unit is configured to calculate the similarity between the image data by combining the similarity between the feature amount vectors for each of the plurality of processing processes. The degree can be calculated.

According to a sixth aspect of the present invention, an image search device is an image search device for searching for reference image data similar to input query image data from a plurality of reference image data, Based on the image similarity determination device according to any one of the above and determining the similarity between each of the reference image data and the query image data, and the similarity determined by the image similarity determination device, Similar image extraction means for extracting image data candidates similar to the query image data from the plurality of reference image data.
With such a configuration, image data similar to the query image data can be extracted from the plurality of reference image data.

  According to the seventh aspect of the present invention, a feature quantity calculation method includes an image receiving step for accepting input of image data, and a feature quantity vector extraction method predetermined for each of a plurality of types of processing on the image data. And a feature quantity extraction step for extracting a feature quantity vector for each of the extraction methods from the received image data.

  According to the eighth aspect of the present invention, a feature amount processing program includes a computer that receives an input of image data, and a feature amount vector extraction method that is predetermined for each of a plurality of types of processing for image data. And functioning as means for extracting a feature vector from the received image data for each of the extraction methods.

  According to the feature amount calculation apparatus of the present invention, it is possible to extract a feature amount vector of image data that can be used without depending on the type and number of processing processes added to the image data.

It is a figure which shows an example of the system configuration | structure of the image search system which concerns on embodiment of this invention. It is a figure which shows an example of the hardware constitutions of the image search system which concerns on embodiment of this invention. It is a figure which shows an example of the various tables which the memory | storage device which concerns on embodiment of this invention memorize | stores. It is a flowchart which shows an example of the flow of a process of the feature-value calculation apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the feature-value vector which concerns on embodiment of this invention. It is a flowchart which shows an example of the flow of a process of the image search system which concerns on embodiment of this invention. It is a figure which shows the other example of the table which the memory | storage device which concerns on embodiment of this invention memorize | stores.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
(System configuration)
FIG. 1 is a diagram showing an example of a system configuration showing a configuration of an image search system according to an embodiment of the present invention.
The image search system 100 searches image data similar to the query image data from a large amount of reference image data. This similarity is determined by comparing feature quantity vectors of image data.
According to FIG. 1, the image search system 100 manages the database on the storage device 300 and the storage device 300 that stores the reference image data and the feature vector of the reference image data in a database, and stores the database of the database. And a management computer 200 that enables searching and the like. A storage device 300 is connected to the management computer 200. Further, the management computer 200 is connected to terminals 400A, 400B, 400C, and 400D via a network such as the Internet. The terminal 400A and the like can access the management computer 200. For example, when the user inputs query image data from the terminal 400A, the management computer 200 extracts image data similar to the query image data from a plurality of reference image data. The user can confirm from the terminal 400A candidates for reference image data similar to the query image data extracted by the management computer 200.

The management computer 200 includes a feature quantity vector extraction unit 210, an image reception unit 220, a similarity calculation unit 230, a similarity determination unit 240, and a result output unit 250.
The image receiving unit 220 inputs reference image data using an image input unit such as an image scanner or a digital camera, or communication input unit via communication such as a network of reference image data stored in an external storage device. Accepts input using. Or the image reception part 220 may receive the input of the reference image data using various input means, such as a mouse | mouth, a keyboard, a touch panel, for example. The image receiving unit 220 stores the received reference image data in the reference data storage unit 310 of the storage device 300.
The feature vector extraction unit 210 uses a plurality of pre-registered feature vector extraction methods to extract feature vectors related to reference image data from reference image data stored in the reference data storage unit 310 of the storage device 300. Extract multiple. Each of the plurality of feature quantity vectors is extracted by a feature quantity vector extraction method corresponding to each feature quantity vector among a plurality of feature quantity vector extraction methods. The feature quantity vector extraction unit 210 stores the extracted feature quantity vectors in the reference data storage unit 310 of the storage device 300.

Further, the image receiving unit 220 receives input of query image data using various input means. The image receiving unit 220 stores the received query image data in the query data storage unit 320 of the storage device 300.
The feature quantity vector extraction unit 210 uses a plurality of pre-registered feature quantity vector extraction methods from the query image data stored in the query data storage unit 320 of the storage device 300, and the feature quantity vector related to the query image data. Extract multiple. Each of the plurality of feature quantity vectors is extracted by a feature quantity vector extraction method corresponding to each feature quantity vector among the plurality of feature quantity vector extraction methods. The feature quantity vector extraction unit 210 stores the extracted feature quantity vectors in the query data storage unit 320 of the storage device 300.

The similarity calculation unit 230 includes a plurality of feature amount vectors of query image data stored in the query data storage unit 320 of the storage device 300 and reference image data stored in the reference data storage unit 310 of the storage device 300. Based on the plurality of feature quantity vectors, the similarity of the reference image data to the query image data is calculated. The similarity calculation unit 230 stores the calculated similarity for each reference image data in the similarity data storage unit 330 of the storage device 300 in association with the identifier of the reference image data.
The similarity determination unit 240 determines the similarity of all reference image data and query image data based on the similarity of the reference image data to the query image data stored in the similarity data storage unit 330 of the storage device 300. To do. The similarity determination unit 240 extracts image data candidates similar to the query image data from all the reference image data based on the determination result, and outputs the extracted image data candidates to the result output unit 250. The result output unit 250 outputs the extracted image data candidates.

(Hardware configuration)
The management computer 200 according to an embodiment can be realized using a general computer 500, for example. FIG. 2 shows an example of the configuration of the computer 500.
FIG. 2 is a diagram illustrating an example of a hardware configuration of the image search system according to the embodiment of the present invention.
The computer 500 includes a CPU (Central Processing Unit) 501, a RAM (Random Access Memory) 502, a ROM (Read Only Memory) 503, a storage device 504, an external I / F (Interface) 505, an input device 506, an output device 507, communication. I / F508 and the like. These devices transmit and receive signals to and from each other via the bus B.

  The CPU 501 is an arithmetic device that implements each function of the computer 500 by reading out programs and data stored in the ROM 503, the storage device 504, and the like onto the RAM 502 and executing the processing. A RAM 502 is a volatile memory used as a work area for the CPU 501. The ROM 503 is a nonvolatile memory that retains programs and data even when the power is turned off. The storage device 504 is realized by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, and stores an OS (Operation System), application programs, various data, and the like. The external I / F 505 is an interface with an external device. Examples of the external device include a recording medium 509. The computer 500 can read and write the recording medium 509 via the external I / F 505. The recording medium 509 includes, for example, an optical disk, a magnetic disk, a memory card, a USB (Universal Serial Bus) memory, and the like.

  The input device 506 includes, for example, a mouse and a keyboard, and inputs various operations and the like to the computer 500 in response to instructions from the operator. The output device 507 is realized by a liquid crystal display, for example, and displays a processing result by the CPU 501. The communication I / F 508 is an interface that connects the computer 500 to a network such as the Internet by wired communication or wireless communication. The bus B is connected to each of the above constituent devices, and transmits and receives various control signals and the like between the control devices.

Next, each table stored in the storage device 300 will be described with reference to FIG.
FIG. 3 is a diagram illustrating an example of various tables stored in the storage device according to the embodiment of the present invention.
The reference data management table 600 stored in the reference data storage unit 310 of the storage device 300 illustrated in FIG. 3A includes each reference image data, an image identifier assigned to the reference image data, and the reference image data. It is a table stored in association with feature quantity vectors.
For example, in the first row data in FIG. 3A, the image identifier of the reference image data “P0001” is “# 00001”, and the feature quantity vector “VA00001” related to the reference image data “P0001” is the feature quantity vector extraction unit. Reference numeral 210 denotes a feature vector extracted by “extraction method a”. Similarly, “VB00001” and “VC00001” in the data in the first row of FIG. 3A are feature quantity vectors extracted by the feature quantity vector extraction unit 210 using “extraction method b” and “extraction method c”, respectively. It shows that there is.

  Here, the extraction method a is, for example, an extraction method corresponding to the processing “trim” for image data. The extraction method a corresponding to the processing process “trim” is a process that is known to be able to extract a feature vector that does not change by the processing process “trim”. The extraction method a may be a known method such as SIFT or CNN (Convolutional Neural Network). In addition, when trimming a certain image data α, it is often considered that only the important part of the image data α is left and the rest is trimmed. Assuming that an important part of the image data α exists in the central part of the image data α, the extraction method a uses, for example, the central part of the image data α for the feature vector of the image data α calculated by a method such as SIFT. An extraction method may be used in which a larger weight is given to the feature quantity vector extracted from the image, and a smaller weight is given to the feature quantity vector extracted from the peripheral portion to obtain the feature quantity vector by the extraction method a. . Note that a feature vector that does not change (is not affected) by a certain processing process is not limited to the meaning of a feature vector that needs to be completely matched before and after the processing process, and is extracted before and after the processing process. It means that the difference between the two feature quantity vectors is included in a predetermined required range.

Further, for example, when the extraction method b is an extraction method corresponding to the processing “add graphic” for the image data α, the feature amount vector that does not change even when the processing “add graphic” is performed. This is a process for extracting. For example, if only the feature vector of a part of the image data changes and the remaining area is not changed is a feature of the processing “add graphic”, the extraction method b The extraction method may be a feature amount vector obtained by the extraction method b by attaching a small weight to the feature amount vector of a part of the region where the change of the feature amount vector is large compared to the region where there is no change.
Similarly, when the extraction method c is an extraction method corresponding to the processing process “change in brightness” for the image data α, the extraction method c is a process for extracting a feature vector that does not change due to a change in brightness. . As described above, in the present embodiment, the feature quantity vector extraction unit 210 has a processing method for extracting a feature quantity vector that is not influenced by the processing process for each processing process, in other words, a feature quantity vector that is resistant to the processing process. .

The feature vector extraction unit 210 extracts a feature vector “VA00001” that is strong against the processing a using the extraction method a corresponding to the processing a for the reference image data “P00001” input to the terminal 400A or the like by the user. To do. The feature vector extraction unit 210 stores “VA00001” in the reference data storage unit 310 of the storage device 300. The same applies to the extraction methods b and c.
As described above, the reference data management table 600 of the reference data storage unit 310 stores feature quantity vectors for each of a plurality of processing processes for the reference image data input by the user.

The query data management table 610 stored in the query data storage unit 320 of the storage device 300 illustrated in FIG. 3B includes query image data, an image identifier assigned to the query image data, and features of the query image data. It is a table stored in association with quantity vectors. For example, this table is a table calculated for each query image.
For example, in the data of FIG. 3B, the image identifier of the query image data “QP0001” is “Q # 00001”, and the feature quantity vector “QVA00001” related to the query image data “QP0001” is “ This indicates that the feature vector is extracted by the extraction method a ”. Similarly, “QVB00001” and “QVC00001” in the data of FIG. 3B indicate that the feature quantity vector extraction unit 210 extracts the feature quantity vectors by “extraction method b” and “extraction method c”, respectively. .

The feature vector extraction unit 210 extracts a feature vector “QVA00001” that is strong against the processing a using the extraction method a corresponding to the processing a for the reference image data “QP00001” input to the terminal 400A or the like by the user. To do. This extraction method a is the same extraction process as the extraction method a described with reference to FIG. The feature vector extraction unit 210 stores “QVA00001” in the query data storage unit 320 of the storage device 300. The same applies to the extraction methods b and c.
In this way, the query data management table 610 of the query data storage unit 320 extracts the feature vector for each of the plurality of processing processes (each of which is extracted by the same extraction process as the reference image data) from the query image data input by the user. ) Is stored.

The similarity data management table 620 stored in the similarity data storage unit 330 of the storage device 300 shown in FIG. 3C includes an image identifier assigned to each reference image data and the reference image data for the query image data. It is a table which memorize | stores similarity with matching. For example, this table is a table calculated for each query image.
The data in FIG. 3C indicates the similarity between the reference image data “P00001” to “P00003” illustrated in FIG. 3A and the query image data “QP00001” illustrated in FIG. For example, the first line data in FIG. 3C indicates that the similarity between the query image data “QP00001” and the reference image data “P00001” (image identifier: # 00001) is “X00001”.
The similarity calculation unit 230 extracts the feature quantity vectors “VA00001”, “VB00001”, “VC00001” for each extraction method of the reference image data “P00001” stored in the reference data management table 600 and the query data management table 610. The similarity “X00001” is calculated using the feature quantity vectors “QVA00001”, “QVB00001”, and “QVC00001” for each method. Specifically, first, the similarity calculation unit 230 compares the feature quantity vectors extracted by the same type of extraction method, and calculates the similarity. For example, the similarity calculation unit 230 calculates the similarity between the feature amount vector “VA00001” of the reference image data obtained by the extraction method a and the feature amount vector “QVA00001” of the query image data obtained by the extraction method a. Calculated based on cosine similarity. The similarity calculated by the cosine similarity takes a value between 0 and 1, and the closer the value is to 1, the more similar the two vectors are. Note that the method of calculating the similarity is not limited to the cosine similarity and may be another method. For example, the Euclidean distance between the vectors may be obtained, and the similarity may be higher as the distance is shorter. The similarity calculation unit 230 calculates the similarity between the feature quantity vector “VB00001” of the reference image data obtained by the extraction method b and the feature quantity vector “QVB00001” of the query image data obtained by the extraction method b, and the extraction method c The similarity between the feature quantity vector “VC00001” of the reference image data and the feature quantity vector “QVC00001” of the query image data obtained by the extraction method c is calculated. Next, the similarity calculation unit 230 calculates the similarity “X00001” between the reference image data “P00001” and the query image data “QP00001” based on the similarity calculated for each of the extraction methods a to c. A method for calculating the similarity will be described later with reference to FIG. The same applies to the reference image data “P00002” and “P00003”.

(Process flow)
Next, the feature vector calculation processing of the reference image data will be described with reference to FIG. 4, taking the configuration of FIG. 1 and various data of FIG. 3 as examples.
FIG. 4 is a flowchart illustrating an example of a processing flow of the feature amount calculation apparatus according to the embodiment of the present invention.
As a premise, it is assumed that the user inputs reference image data “P00001”, “P00002”, and “P00003” to the terminal 400A. Then, the terminal 400A transmits these three image data to the management computer 200. In the management computer 200, the image reception unit 220 receives these three image data and stores them in the storage device 300. More specifically, the image receiving unit 220 stores the reference image data “P00001” in the reference data management table 600 in association with the image identifier. Similarly, the image receiving unit 220 stores the reference image data “P00002” and “P00003” in the reference data management table 600 in association with each image identifier. Note that the image identifier of the reference image data may be calculated by the image receiving unit 220, or may be calculated by a database system operating on the storage device 300 or the like. When the reference image data is stored in the storage device 300 as described above, the management computer 200 starts a feature vector extraction process of the reference image data, for example, according to an instruction from the user.

  First, the feature vector extraction unit 210 takes in the reference image data from the storage device 300 (step S101). More specifically, the feature vector extraction unit 210 reads three reference image data “P00001” to “P00003” stored in the reference data storage unit 310 (reference data management table 600) of the storage device 300. take in.

  Next, the feature vector extraction unit 210 extracts a plurality of feature vectors related to the reference image data from the fetched reference image data using a plurality of feature vector extraction methods registered in advance (step S102). More specifically, first, for the reference image data “P00001” to “P00003”, the feature vector extraction unit 210 extracts “VA00001” to “VA00003” using the extraction method a. Further, the feature vector extraction unit 210 extracts “VB00001” to “VB00003” using the extraction method b. Also, the feature vector extraction unit 210 extracts “VC00001” to “VC00003” using the extraction method c. The feature vector extraction unit 210 stores “VA00001” to “VC00001” in the reference data management table 600 in association with the reference image data “P00001”. The same applies to the reference image data “P00002” and “P00003”. The image search system 100 accepts a large amount of reference image data by the processing of FIG. 4, extracts those feature quantity vectors, and stores the result in the storage device 300.

Next, a method for calculating the similarity between image data will be described with reference to FIG.
FIG. 5 is a diagram showing an example of the feature quantity vector according to the embodiment of the present invention.
The feature vector extraction unit 210 extracts the feature vectors extracted from the reference image data as x _a , x _b, and x _c . Further, the feature quantity vector extraction unit 210, a feature vector extracted for the reference image data and _y a, _{y b} and _{y c.}
Similarity calculation unit 230, by a predetermined similarity calculation method (cosine similarity, etc.), the similarity is calculated _{z a} a _{x a} and _{y a.} Further, the similarity calculation unit 230, by a predetermined similarity calculation method to calculate the similarity _{z b} of _{x b} and _{y b,} likewise calculates the similarity _{z c} of _{x c} and _{y c.} At this time, the similarity calculation methods for calculating the similarities z _a , z _b , and z _c may all be the same calculation method, or may be calculation methods that are all different for each processing process.
Next, the similarity calculation unit 230 calculates the similarity Z between the image data using these similarities z _a , z _b , and z _c . For example, the similarity calculation unit 230 may calculate a total value of the similarities z _a , z _b , and z _c calculated for each extraction process (for each processing process), and use this as the similarity Z. Alternatively, the similarity calculation unit 230 may calculate a weighted average of the similarities z _a , z _b , and z _c to which appropriate weights are assigned for each processing process, and use this as the similarity Z. Further, the similarity calculation unit 230 may select the maximum value among the similarities z _a , z _b , and z _c and determine the selected value as the similarity Z. Alternatively, the similarity calculation unit 230 may calculate a total value, a weighted average, and a maximum value of the similarities z _a , z _b , and z _c , and set the sum as the similarity Z.

  Thus, the similarity Z is calculated by combining similarities between a plurality of feature quantity vectors. Specifically, the similarity Z is image data obtained by adding one or more processing processes among artificial processes such as changes in brightness, color, contrast, trimming, rotation, caption addition, figure addition, etc. On the other hand, it is a value obtained by combining similarities between feature quantity vectors that do not change with each processing. The similarity Z obtained in this way can be used as, for example, an index of similarity between image data that has been subjected to a plurality of processing processes and original image data that has not been processed. . By utilizing this property, it is possible to retrieve image data obtained by performing one or more processing on the image data β from a large amount of image data for a certain image data β.

Next, image data search processing will be described with reference to FIG. 6, taking the configuration of FIG. 1 and various data of FIG. 3 as examples.
FIG. 6 is a flowchart illustrating an example of a processing flow of the image search system according to the embodiment of the present invention.
As a premise, the reference data storage unit 310 of the storage device 300 stores a plurality of reference image data “P00001” to “P00003” and the like by the processing described with reference to FIG. The user inputs the query image data “QP00001” to be searched and search instruction information to the terminal 400A. Here, it is assumed that “QP00001” is an image obtained by performing a plurality of processing processes on the reference image data “QP00001”. The terminal 400A transmits the query image data “QP00001” to the management computer 200. In the management computer 200, the image reception unit 220 receives the query image data “QP00001” and stores it in the storage device 300. More specifically, the image receiving unit 220 stores the query image data “QP00001” in the query data management table 610 in association with the image identifier.

Then, the feature quantity vector extraction unit 210 extracts a feature quantity vector of the query image data “QP00001” (step S201). The flow of this process is the same as steps S101 to S102 described in FIG. The feature quantity vector extraction unit 210 outputs the extracted plurality of feature quantity vectors to the similarity calculation unit 230.
Next, the similarity calculation unit 230 applies the query image data to the query image data based on the plurality of feature amount vectors of the query image data and the feature amount vector of the reference image data stored in the reference data storage unit 310 of the storage device 300. The similarity of reference image data is calculated for each extraction process. Next, the similarity calculation unit 230 calculates the similarity Z between the query image data and the reference image data. The similarity Z is a similarity obtained by considering the similarity between feature quantity vectors that are not affected by one or more types of processing applied to query image data and reference image data. The similarity calculation unit 230 stores the calculated similarity Z in the similarity data storage unit 330 of the storage device 300 in association with the image identifier of the reference image data (step S202).
Next, the similarity determination unit 240, based on the similarity Z of the reference image data to the query image data stored in the similarity data storage unit 330 of the storage device 300, the reference image similar to the query image data “QP00001”. Data candidates are extracted (step S203). As a candidate extraction method, for example, if the calculated similarity Z is equal to or greater than a predetermined threshold, it is determined that the reference image data is similar to the query image data. Alternatively, the similarity determination unit 240 may determine, for example, the top 10 items in descending order of similarity Z, or the top 5% with high similarity Z as similar images. Alternatively, the similarity determination unit 240 may determine the similarity Z between two pieces of image data by dividing them into ranks such as “high”, “medium”, and “low”, or may be determined at a rate of 80% or the like. May be. The similarity determination unit 240 outputs the determination result to the result output unit 250. The result output unit 250 transmits similar image data candidates to the terminal 400A. The user can refer to candidate image data similar to the query image data among the reference image data stored in the storage device 300. In the case of this example, the user can confirm the image data “P00001” before the processing as the image data similar to the query image data “QP00001”. When the user confirms “P00001” before the processing, the user can obtain “P00001” and perform a process such as performing a new processing on “P00001”, which improves user convenience. .

In addition to this, various applications are possible by using the image data search process described in FIG.
FIG. 7 is a diagram showing another example of a table stored in the storage device according to the embodiment of the present invention.
An example of application of the reference data management table 600 is illustrated in FIG. The reference data management table 600 shown in FIG. 7 has items of “usage restriction” and “file path” in addition to the items shown in FIG. In the processing described with reference to the flowchart of FIG. 4, when registering reference image data, information on usage restrictions on the reference image data and information on the path of a document file including the reference image data are associated with the reference image data. sign up. In the image data search process shown in FIG. 6, the result output unit 250 uses the usage restriction information stored in association with the reference image data determined to be similar by the similarity determination unit 240 for the similar image. Output with candidates. Then, the user can grasp whether or not there is a usage restriction on the query image data that he / she is looking for. For example, if the value of “use restriction” is “none”, the user can freely use the query image data or similar reference image data presented as a candidate. Further, if the value of “use restriction” is “in-house only”, the user can use query image data only for in-house documents. If the value of “use restriction” is “prohibited”, the user can grasp that the query image data is prohibited from being used. As described above, the image search process of the present embodiment can be applied for the purpose of preventing unauthorized use of image data.

In the image data search process shown in FIG. 6, the result output unit 250 outputs the file path information stored in association with the similar image data together with the similar image candidate. Then, the user can know and obtain where a file containing image data similar to the query image data that he / she is looking for exists. As described above, the image search process of the present embodiment can be applied for the purpose of file search using the query image data as a key.
In both cases of preventing unauthorized use of image data and file search, when a user inputs a file including query image data to the terminal 400A, an image data extracting means (not shown) of the management computer 200 causes the file to be searched. The image data included in the file may be extracted from the file, and the processing in steps S201 to S203 in FIG. 6 may be performed on each extracted image data.

  Conventionally, when evaluating the coincidence / disagreement between image data that has been subjected to artificial processing and image data that has not been subjected to processing, it is often done manually. In this case, if the number of target image data is large, a large amount of time and cost may be spent on the confirmation work, or the accuracy may vary depending on the skill level of the person performing the confirmation work. In addition, feature quantity vectors are extracted from target image data, and matching / mismatching between image data is evaluated based on the similarity between feature quantity vectors. However, in the case where the target image is processed image data, the feature amount vector fluctuates due to the addition of the processing process, and therefore it is often impossible to correctly determine the similarity with the original image data before the processing process. . Further, there are a plurality of types of processing for image data, but there is no feature amount extraction processing for which no feature vector changes for any image processing. On the other hand, according to the present embodiment, as described above, for example, trim, blur, noise, color tone change, change in contrast and brightness, monochromeization, change in aspect ratio, partial cutout, character and figure It is possible to extract multiple feature vectors for evaluating similarity with high accuracy, regardless of the type of processing, even for images that have undergone processing such as overwriting, rotation, and scale change. It becomes. Also, assuming that there is processed image data that has been subjected to one or more types of processing as exemplified above, what kind of processing has been performed on the image data is actually processed. I can not know other than a user According to the present embodiment, since a feature vector extraction method corresponding to a plurality of types of processing is prepared in advance, there are many image data in which the type and number of added processing are unknown. However, it is possible to extract a plurality of feature quantity vectors from which similar image data can be searched with high accuracy.

  Each process in the management computer 200 described above is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer of the management computer 200 reading and executing this program. Is called. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.
Moreover, the management computer 200 may be comprised by one computer, and may be comprised by the some computer connected so that communication was possible.

  In the description of the embodiment, the image search system 100 has been exemplified. However, for example, the functional units of the feature vector extraction unit 210 and the image reception unit 220 are mounted on one computer and configured as a feature calculation device. Is also possible. According to this feature amount calculation apparatus, when image data is input, it is possible to extract a plurality of feature amount vectors that can be extracted from the image data regardless of whether or not processing is performed on the image data. In a society where image data processed due to the development of the Internet, etc. is widely distributed, feature vectors that are resistant to various types of processing on the original image data are extracted for the purpose of monitoring the distributed image data and preventing unauthorized use. It is effective to keep it.

  In addition, for example, each function unit of the above-described feature amount calculation device and each function unit of the similarity calculation unit 230 and the similarity determination unit 240 may be mounted on one computer to configure as an image similarity determination device. Is possible. The user inputs the query image data and the reference image data to the image similarity determination device. Then, the respective functional units of the above-described feature quantity calculation device (query feature quantity extraction means) included in the image similarity determination apparatus extract a plurality of feature quantity vectors for the query image data. In addition, the respective functional units of the above-described feature amount calculation device (reference feature amount extraction means) included in the image similarity determination device extract a plurality of feature amount vectors for the reference image data. Then, the similarity calculation unit 230 calculates the similarity between the query image data and the reference image data for each of a plurality of feature quantity vectors, and further calculates the similarity Z between the two image data by combining the plurality of similarities. To do. Next, the similarity determination unit 240 determines the similarity between the query image data and the reference image data based on the calculated similarity Z. According to this image similarity determination apparatus, for example, it is used in a scene where the similarity between two image data, in which it is not known at first glance whether there is a relationship between the original image data and the processed image data, or the like. be able to.

In addition, it is possible to appropriately replace the components in the above-described embodiments with known components without departing from the spirit of the present invention. The technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.
The feature vector extraction unit 210 is an example of a feature amount extraction unit, the image reception unit 220 is an example of an image reception unit, the similarity calculation unit 230 is an example of a similarity calculation unit, and the similarity determination unit 240 is a similarity determination unit. The similarity determination unit 240 is an example of a similar image extraction unit. The image search system 100 is an example of an image search device.

DESCRIPTION OF SYMBOLS 100 ... Image retrieval system 200 ... Management computer 210 ... Feature-value vector extraction part 220 ... Image reception part 230 ... Similarity calculation part 240 ... Similarity determination part 250 ... Result Output unit 300 ... storage device 310 ... reference data storage unit 320 ... query data storage unit 330 ... similarity data storage unit 400 ... terminal 500 ... general computer 600 ... Reference data management table 610 ... query data management table 620 ... similarity data management table

Claims (8)

Image receiving means for receiving input of image data;
Feature quantity extraction means for extracting feature quantity vectors for each extraction method from image data received by the image reception means using a feature quantity vector extraction method predetermined for each of a plurality of types of processing for image data ,
A feature amount calculating apparatus. The extraction method defined for each type of the processing process is an extraction method for extracting a feature vector that does not change by the processing process from the image data.
The feature amount calculation apparatus according to claim 1. The processing type is image data rotation, scale change, aspect ratio change, partial cutout, color tone change, monochromeization, figure overwriting, character overwriting, trimming, brightness change, contrast change, blur , Characterized by at least two of the noises,
The feature amount calculation apparatus according to claim 2. An image similarity determination device for determining a similarity between input query image data and reference image data,
A feature extraction unit for reference that extracts a feature vector for each extraction method from the reference image data using the feature calculation device according to any one of claims 1 to 3.
Query feature quantity extraction means for extracting a feature quantity vector for each extraction method from the query image data using the feature quantity calculation device;
Similarity calculation means for calculating a similarity between the query image data and the reference image data based on a plurality of feature vector obtained from the query image data and a plurality of feature vectors obtained from the reference image data; ,
Similarity determination means for determining the similarity between the query image data and the reference image data based on the calculated similarity;
An image similarity determination apparatus comprising: The similarity calculation means calculates a similarity between feature quantity vectors extracted from the query image data and the reference image data using the same extraction method among the extraction methods, and each of the calculated plurality of extraction methods Calculating the similarity between the query image data and the reference image data based on the corresponding similarity;
The image similarity determination apparatus according to claim 4. An image search device for searching for reference image data similar to input query image data from a plurality of reference image data,
The image similarity determination device according to claim 4 or 5, wherein a similarity between each of the plurality of reference image data and the query image data is determined.
Similar image extraction means for extracting image data candidates similar to the query image data from the plurality of reference image data based on the similarity determined by the image similarity determination device;
An image search apparatus comprising: An image receiving process for receiving input of image data;
A feature amount extraction step of extracting a feature amount vector for each of the extraction methods from the received image data using a feature amount vector extraction method determined in advance for each of a plurality of types of processing on the image data;
A method for calculating a feature amount. Computer
Means for receiving input of image data;
Means for extracting a feature vector for each extraction method from the received image data, using a feature vector extraction method predetermined for each of a plurality of types of processing for image data;
Program to function as.

Images