JP4134024B2 - Similar image retrieval apparatus, method and program - Google Patents

Description

  The present invention relates to a similar image search device, method, and program for searching for similar images by comparing a search target image with a plurality of registered images. In particular, the present invention can be found in files and general drawings that contain many similar images, such as mechanical drawings, electrical wiring diagrams, electronic circuit diagrams, semiconductor wiring patterns, logo marks, trademarks, family crests, photographs, paintings, and images. The present invention relates to a similar image search device, method, and program that are effective for searching for a desired image, specifying a location where an image is included, and extracting an entire view having a location similar or coincident with a partial view. .

  In recent years, with the development of multimedia technology, there has been an increasing demand for processing images using a computer including a personal computer, a microcomputer, a dedicated image processing apparatus, and the like, and the progress of software technology therefor is also remarkable. In general, an image is handled as digital data for each pixel in a computer. However, since it is necessary to process a large amount of data, the speed of image processing largely depends on the high-speed calculation processing capability of the computer. Many proposals have been made as image processing techniques to be handled on a computer. Among them, an image search technique is one of important issues.

The image search is to extract a desired image to be searched from a large number of images stored in advance. As a typical image search performed conventionally, a search method using an autocorrelation function of a search target image is known (for example, see Non-Patent Document 1). In this search method, the autocorrelation function obtained from the search target image is divided into three sections, each section is obtained with an exponential function, a sine function, and a similar function represented by a cubic polynomial, and parameters representing these similar functions are set. A registered image similar to the search target image is extracted.
Hideyo Nagashima, 2 others, "Classification of trademark figures using graphs of autocorrelation functions", IEEJ Transactions C, The Institute of Electrical Engineers of Japan, August 21, 2003, Vol. 123, No. 9, p1547 -1554

  By the way, in the search method of Non-Patent Document 1 described above, each of the three sections included in the autocorrelation function is approximated by different types of similar functions. However, a single central section occupying most of the autocorrelation function is used. Therefore, there is a problem that the fine features of the autocorrelation function are not reflected in the search, and the search accuracy is lowered depending on the contents of the search target image.

  The present invention has been made in view of such a point, and an object thereof is to provide a similar image search apparatus, method, and program capable of improving the search accuracy.

  In order to solve the above-described problem, a similar image search apparatus according to the present invention includes an image capturing unit that captures a search target image, and a self-correlation waveform that extracts an autocorrelation waveform of the search target image captured by the image capture unit. Correlation waveform extraction means, joint point extraction means for extracting a joint point where the tendency of the autocorrelation waveform extracted by the autocorrelation waveform extraction means changes, and each segment area of the autocorrelation waveform divided by the joint points Function approximation means for approximating with a function, and image search means for searching for a similar image from a plurality of comparison target images based on feature information related to approximation processing by the function approximation means.

  The similar image search method of the present invention includes an image capturing step for capturing a search target image, an autocorrelation waveform extraction step for extracting an autocorrelation waveform of the search target image captured in the image capturing step, A junction point extraction step for extracting a junction point where the tendency of the autocorrelation waveform extracted in the correlation waveform extraction step changes, and a function approximation step for approximating each segment area of the autocorrelation waveform divided by the junction point with a function. And an image search step for searching for a similar image from a plurality of comparison target images based on the feature information related to the approximation process in the function approximation step.

  Further, the similar image search program of the present invention includes a computer that includes an autocorrelation waveform extraction unit that extracts an autocorrelation waveform of a search target image captured by the image capture unit and a self-correlation waveform extraction unit that extracts the self-correlation waveform extraction unit. For joint point extracting means for extracting joint points where the trend of the correlation waveform changes, function approximating means for approximating each segmented area of the autocorrelation waveform divided by the joint points by function, and approximation processing by the function approximating means Based on the related feature information, it is made to function as an image search means for searching for a similar image from a plurality of comparison target images.

  By functionalizing each segmented area that is divided at each junction point included in the autocorrelation waveform, the entire function including the fine features of the autocorrelation waveform can be accurately functioned and approximated. By using the feature information extracted in step 1, it is possible to improve the search accuracy when searching for similar images.

Further, the information processing apparatus further includes feature information storage means in which feature information created in association with the above-described processing for approximating each segmented region of the autocorrelation waveform corresponding to the comparison target image is stored for each of the plurality of comparison target images. The image search means is similar to the search target image by comparing feature information corresponding to the search target image with feature information corresponding to a plurality of comparison target images stored in the feature information storage means. It is desirable to extract a comparison target image. Alternatively, the information processing apparatus further includes feature information storage means in which feature information created in association with the process of approximating each segmented region of the autocorrelation waveform corresponding to the comparison target image is stored for each of the plurality of comparison target images. The image search step is similar to the search target image by comparing the feature information corresponding to the search target image with the feature information corresponding to a plurality of comparison target images stored in the feature information storage unit. It is desirable to extract a comparison target image. Thus, it is possible to easily search for similar images by sequentially reading out the feature information stored in advance and comparing it with the feature information of the captured image to be searched, and there are many images to be compared. Even in this case, the operation is not complicated, and the operation can be simplified.
Further, when the feature information corresponding to the comparison target image is acquired using the above-described image capturing means, autocorrelation waveform extracting means, junction point extracting means, and function approximating means, this feature information is stored in the feature information storing means. It is desirable to further include characteristic information storage processing means for storing. Alternatively, when the feature information corresponding to the comparison target image is acquired using the image capturing step, autocorrelation waveform extraction step, junction point extraction step, and function approximation step described above, this feature information is stored in the feature information storage unit. It is desirable to further include a feature information storing process step for storing. This makes it possible to add an image to be compared as appropriate.

  Further, it is desirable that the above-described image capturing means is an optical reading device that optically reads the density information or color information of the search target image. Alternatively, it is preferable that the above-described image capturing step is performed using an optical reading device that optically reads the density information or color information of the search target image. Thereby, it is possible to easily capture an image to be searched.

  Further, the above-described image capturing means is a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. It is desirable. Alternatively, the above-described image capturing step uses a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. It is desirable to be done. As a result, when the image data is already stored in the format, the image data can be used, and the image to be searched can be captured more easily.

  Further, the feature information described above includes the order of a plurality of functions corresponding to a plurality of segmented regions constituting the autocorrelation waveform, and the image search means performs a plurality of comparison targets based on the order of the plurality of functions. It is desirable to search for an image similar to the search target image from the images. Alternatively, the feature information described above includes an order of a plurality of functions corresponding to a plurality of segment areas constituting the autocorrelation waveform, and the image search step includes a plurality of comparison targets based on the order of the plurality of functions. It is desirable to search for an image similar to the search target image from the images. The order of a plurality of functions constituting the autocorrelation waveform is considered to be an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by determining similarity of images using the order of functions.

  In addition, the feature information described above includes an array of section lengths of a plurality of functions corresponding to a plurality of partitioned regions constituting the autocorrelation waveform, and the image search means includes a plurality of sections based on the order of the section lengths. It is desirable to search for images similar to the search target image from the comparison target images. Alternatively, the feature information described above includes an array of section lengths of a plurality of functions corresponding to a plurality of segment areas constituting the autocorrelation waveform, and the image search step includes a plurality of sections based on the section lengths. It is desirable to search for images similar to the search target image from the comparison target images. The arrangement of section lengths corresponding to each of a plurality of functions (a plurality of divided regions) constituting the autocorrelation waveform is considered to be an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the arrangement of section lengths.

  Further, the image search means described above calculates the correlation degree of the function corresponding to a plurality of segment areas constituting the respective auto-correlation waveforms of the search target image and the comparison target image, and the search target object in descending order of the correlation degree. It is desirable to search for a comparison target image similar to the image. Alternatively, the image search step described above calculates the degree of correlation of functions corresponding to a plurality of segment areas constituting the respective autocorrelation waveforms of the search target image and the comparison target image, and the search target is calculated in descending order of the correlation degree. It is desirable to search for a comparison target image similar to the image. It is considered that the degree of correlation of each function constituting the autocorrelation waveform is an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the degree of correlation of functions.

  In addition, the above-described image search means focuses on the total number of segment areas included in the autocorrelation waveform corresponding to the search target image, selects a predetermined number of comparison target images as search target candidates, and then based on the feature information. It is desirable to search for a comparison target image similar to the search target image. Alternatively, the above-described image search step focuses on the total number of segment areas included in the autocorrelation waveform corresponding to the search target image, selects a predetermined number of comparison target images as search target candidates, and then based on the feature information. It is desirable to search for a comparison target image similar to the search target image. As a result, the number of images to be searched can be reduced, so that the processing load can be further reduced and the processing time can be shortened.

  Moreover, it is desirable that the above-described autocorrelation waveform extracting unit extracts the autocorrelation waveform along the longitudinal direction of the search target image. Alternatively, the above-described autocorrelation waveform extraction step desirably extracts the autocorrelation waveform along the longitudinal direction of the search target image. Since the autocorrelation waveform extracted along the longitudinal direction well represents the characteristics of the complex shape of the image, it is possible to further improve the search accuracy by using such an autocorrelation waveform.

  Further, it is desirable that the above-described autocorrelation waveform extracting means calculates a rectangle with the smallest area inscribed by the search target image and extracts the autocorrelation waveform in a direction along the long side of the rectangle. In the autocorrelation waveform extraction step, it is desirable to calculate a rectangle with the smallest area inscribed by the search target image and extract the autocorrelation waveform in a direction along the long side of the rectangle. Thereby, reproducibility in the longitudinal direction of the image can be ensured, and the same autocorrelation waveform can always be obtained for the same image.

  Further, it is desirable that the above-described autocorrelation waveform extracting means calculates the centroid position of the image to be searched and extracts the autocorrelation waveform in a predetermined rotation direction with the centroid position as the rotation center. Alternatively, in the above-described autocorrelation waveform extraction step, it is desirable to calculate the centroid position of the search target image and extract the autocorrelation waveform in a predetermined rotation direction with the centroid position as the rotation center. This makes it possible to extract a stable autocorrelation waveform that is not affected by the direction.

  In addition, the autocorrelation waveform extracting unit described above calculates the position of the center of gravity of the rectangle with the smallest area inscribed in the search target image and the search target image, and the ratio of the long side to the short side of the rectangle is greater than or equal to the reference value. The autocorrelation waveform is extracted in the direction along the long side, and when the ratio of the long side to the short side of the rectangle is smaller than the reference value, the autocorrelation waveform can be extracted in a predetermined rotation direction with the center of gravity as the rotation center. desirable. Alternatively, the above-described autocorrelation waveform extraction step calculates the position of the center of gravity of the rectangle with the smallest area inscribed in the search target image and the search target image, and the ratio of the long side to the short side of the rectangle is equal to or greater than the reference value. The autocorrelation waveform is extracted in the direction along the long side, and when the ratio of the long side to the short side of the rectangle is smaller than the reference value, the autocorrelation waveform can be extracted in a predetermined rotation direction with the center of gravity as the rotation center. desirable. By extracting the autocorrelation waveform in the longitudinal direction for images that are long in one direction and extracting the autocorrelation waveform in the rotation direction for other images, it is possible to obtain an autocorrelation waveform that is suitable for the shape of the image become.

  Hereinafter, a similar image search apparatus according to an embodiment to which the present invention is applied will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a similar image search apparatus according to an embodiment. The similar image search apparatus shown in FIG. 1 is for searching a registered image (similar image) similar to this image when a search target image is input, and outputting a search result. A unit 110, an image DB (database) 120, a search processing unit 130, an operation unit 150, a display unit 160, and a printing apparatus 170.

  The image input unit 110 is for inputting the search target image into the search processing unit 130, and inputs image data indicating grayscale information and color information for each pixel constituting the search target image. For example, as the image input unit 110, an optical reading device such as an optical scanner or a digital camera that optically reads the grayscale information and color information of each pixel constituting the search target image is used. Note that as a result, it is only necessary to obtain image data indicating grayscale information for each pixel constituting the search target image. Therefore, image data indicating the grayscale information of each pixel constituting the search target image is a CD, When recorded on various information recording media such as a DVD and a semiconductor memory, a data reading device such as a disk drive device that reads image data from these recording media may be used as the image input unit 110.

  The image DB 120 stores feature information obtained by performing functionalized approximation processing on a plurality of images whose similarity is compared with the search target image (these images are referred to as “comparison target images”). A specific example of the feature information will be described in the function approximation process performed on the search target image. Prior to processing by the search processing unit 130, it is necessary to register feature information of a plurality of comparison target images in the image DB 120. Details of this registration process will be described later.

  The search processing unit 130 extracts feature information of the search target image, and performs a process of searching for a comparison target image similar to the search target image based on the feature information. For this purpose, the search processing unit 130 includes an autocorrelation waveform extraction processing unit 132, a junction point extraction processing unit 134, a function approximation processing unit 136, a similarity determination processing unit 140, and a search result output processing unit 142.

  The autocorrelation waveform extraction processing unit 132 extracts the autocorrelation waveform of the search target image captured by the image input unit 110. 2 and 3 are diagrams showing an outline of processing for extracting an autocorrelation waveform. When extracting an autocorrelation waveform, one search target image G2 is gradually moved in one direction from a state in which two identical search target images G1 and G2 are overlapped (FIG. 2) (FIG. 3). In this process, the inner product of the constituent pixels of the two search target images G1 and G2 corresponding to the same coordinates is calculated and summed to obtain an autocorrelation value. Naturally, as shown in FIG. 2, the total value of the inner products calculated in the state where the same two search target images G1 and G2 are overlapped becomes the largest. The autocorrelation value is normalized using this total value. Thus, the autocorrelation waveform indicates the relationship between the relative movement amount between the two search target images G1 and G2 and the normalized autocorrelation value.

  4 and 5 are diagrams showing specific examples of the autocorrelation waveform. As shown in FIG. 4, an autocorrelation waveform as shown in FIG. 5 is obtained by moving the horizontally written character figure “Fluncy” in the horizontal direction.

  6 and 7 are diagrams showing other specific examples of the autocorrelation waveform. As shown in FIG. 6, an autocorrelation waveform as shown in FIG. 7 is obtained by moving the horizontally written character figure “Fluncy” in the vertical direction.

  As described above, if the direction in which the autocorrelation waveform is extracted is different, the obtained autocorrelation waveform is different. Therefore, in which direction the autocorrelation waveform is extracted is an important problem. Moreover, as shown in FIGS. 6 and 7, when the autocorrelation waveform is extracted in the direction in which the length of the search target image is short (the height is low), the obtained autocorrelation waveform tends to be monotonous. It is desirable to perform preprocessing for extracting the longest direction and obtain an autocorrelation waveform along the extracted direction. For example, it is conceivable that a rectangle having the smallest area that is inscribed by the search target image is obtained and the search target image is moved in a direction in which the long side of the rectangle extends.

  FIG. 8 is a diagram showing an outline of another process for extracting the autocorrelation waveform. As another example of extracting the autocorrelation waveform, one search target image G2 may be rotated in a predetermined direction from a state in which two identical search target images G1 and G2 are overlapped (FIG. 2). (FIG. 8). Although the rotation center at this time can specify an arbitrary position, it is necessary to set the rotation center in the same manner for the comparison target image and the search target image. For example, it is conceivable to perform preprocessing for obtaining the position of the center of gravity of the image and rotate the image around the position of the center of gravity. The processing for obtaining the autocorrelation values of the two search target images G1 and G2 is the same, and in this way, the relative rotation amount between the two search target images G1 and G2 (rotation angle 0 ° to The autocorrelation waveform indicates the relationship between the range of 360 ° and the normalized autocorrelation value.

  Based on the autocorrelation waveform extracted by the autocorrelation waveform extraction processing unit 132, the junction point extraction processing unit 134 extracts a junction point at which the tendency of the autocorrelation waveform changes. For example, a corner point at which the angle of the autocorrelation waveform changes suddenly is extracted as a junction point. The function approximation processing unit 136 converts a partial region (segmented region) divided (divided) at two adjacent joints along the autocorrelation waveform into a function of a straight line, an arc, or a free curve. Using the approximation, the feature information related to the approximation processing is created. For example, when the segmented region can be approximated by a straight line, a straight line is used as the approximation function, and when the segmented area cannot be approximated by a straight line and can be approximated by an arc, an arc is used as the approximation function. When an arc cannot be approximated, a free curve is used as an approximation function. When a straight line is used as the approximate function, the sign indicating that the function used is a straight line and the parameter indicating the shape of the partitioned area approximated by the straight line are characteristic information about the approximate function corresponding to this partitioned area. Created as Similarly, when an arc is used as the approximation function, a sign indicating that the function used is an arc and a parameter indicating the shape of the segment area approximated by the arc are approximate functions corresponding to the segment area. As feature information. When a free curve is used as an approximate function, a sign indicating that the function used is a free curve and a parameter indicating the shape of the segmented region approximated by the free curve is an approximate function corresponding to this segmented region. As feature information.

It should be noted that the determination of which function can approximate the segmented area is based on whether the error between the segmented area and the approximate function (error obtained by the least squares method) is less than a predetermined value. It is done by examining whether. The parameter indicating the shape of the segmented region may be any parameter as long as the shape of the segmented region can be specified. For example, as disclosed in Japanese Patent No. 2646475, the following parameters are used. It may be.
(1) For a straight line: flag indicating a straight line, coordinates of the start point of a segmented area (2) For an arc: a flag indicating a circular arc, the coordinates of the start point of the arc, the coefficient of the central angle between joint points, existing between joint points Number of contour points to be used, coefficients of approximate function (Each coefficient when an arc is expressed by a linear combination of trigonometric functions, for example)
(3) In the case of a free curve: the number of dimensions of an approximate function indicating a free curve between joint points (≧ 3), the number of contour points existing between joint points, the center of variation of the contour point sequence between joint points, the approximation function coefficient.

  The similarity determination processing unit 140 determines the similarity between the search target image and each comparison target image based on the feature information extracted corresponding to the search target image, and performs a comparison similar to the search target image. Determine the target image. The feature information used for the similarity determination includes, in addition to the parameter indicating the shape of the segmented area described above, the order of a plurality of functions corresponding to the plurality of segmented areas constituting the autocorrelation waveform, and each of the plurality of functions. Are included, and the degree of correlation of functions constituting autocorrelation waveforms corresponding to the search target image and the comparison target image, respectively.

  FIG. 9 is a diagram illustrating a function table including feature information extracted by each process of the autocorrelation waveform extraction processing unit 132, the junction point extraction processing unit 134, and the function approximation processing unit 136. The outline included in the following description refers to the outline of the autocorrelation waveform, that is, the autocorrelation waveform itself. The function table shown in FIG. 9 includes the total number of functions, the contour length, the total number of sample points, the number of straight lines, the total length of straight lines, the number of arcs, the total length of arcs, the number of curves, the total length of curves, the section length for each contour line, and the number of sample points. The section length and parameters corresponding to each function are included. The total number of functions is the total number of functions included in the focused contour line and is equal to the number of segmented areas. The contour length is the length of the focused contour line. The number of straight lines is the number of segmented areas approximated by straight lines in the segmented areas constituting the focused contour line. The total length of the straight line is a total value of the lengths of the segmented areas approximated by straight lines in the segmented areas constituting the target contour line. The number of arcs is the number of segmented areas that are approximated by arcs in each segmented area constituting the focused contour line. The total arc length is a total value of the lengths of the segment areas approximated by the arc in the segment areas constituting the target contour line. The number of curves is the number of segmented areas that are approximated by a free curve in each segmented area constituting the target contour line. The total curve length is the total value of the lengths of the segmented regions approximated by the free curve in the segmented regions constituting the target contour line. Further, in FIG. 9, a plurality of functions shown in the “function” section indicate functions that approximate each segmented area constituting the focused autocorrelation waveform, and the arrangement order of each segmented area is As well.

  The similarity determination processing unit 140 refers to the function table shown in FIG. 9 to determine the order of a plurality of functions constituting the autocorrelation waveform, and the section length (the length of the segmented area) corresponding to each of the plurality of functions. ).

  The search result output processing unit 142 displays the search result on the screen of the display unit 160 or instructs the printing apparatus 170 to print the search result when a printing operation using the operation unit 150 is performed.

  Incidentally, the feature information shown in FIG. 9 is created based on the image data of the image to be searched input from the image input unit 110. In order to extract a similar image in comparison with this feature information, there are many features information. It is necessary to previously extract feature information having the same contents for the comparison target images and register them in the image DB 120.

  FIG. 10 is a diagram illustrating a configuration of a database creation apparatus that performs feature information extraction on a comparison target image in advance and registers it in the image DB 120. The configuration shown in FIG. 10 may be provided as part of the similar image search device shown in FIG. 1, but may be constructed separately from the similar image search device shown in FIG. . The database creation device shown in FIG. 10 includes an image input unit 210, a database creation unit 220, an operation unit 230, and a display unit 240 in order to register feature information of many comparison target images in the image DB 120.

  The image input unit 210 inputs image data indicating grayscale information and color information for each pixel constituting the comparison target image. Similar to the image input unit 110 shown in FIG. 1, an optical scanner, a digital camera, a disk drive device, or the like can be used as the image input unit 210.

  The database creation unit 220 performs processing for extracting feature information of the comparison target image and registering it in the image DB 120. For this purpose, the database creation unit 220 includes an autocorrelation waveform extraction processing unit 222, a junction point extraction processing unit 224, a function approximation processing unit 226, and a file creation processing unit 228. Among them, the basic operations of the autocorrelation waveform extraction processing unit 222, the junction point extraction processing unit 224, and the function approximation processing unit 226 except for the file creation processing unit 228 are the same in the search processing unit 130 shown in FIG. This is the same as each name component, and a detailed description of the operation is omitted.

  The file creation processing unit 228 registers the extracted feature information for each comparison target image (feature information included in the function table shown in FIG. 9) in the image DB 120 as a batch file. The operation unit 230 is used for performing extraction of feature information corresponding to the comparison target image, operation instructions necessary for registration, and the like. The display unit 240 is used for confirming the registered content of the feature information and displaying various operation screens necessary for registration.

  By using the database creation device having such a configuration, the feature information of a plurality of comparison target images having basically the same contents as the feature information of the search target image shown in FIG. 9 is registered in the image DB 120.

  The above-described image input units 110 and 210 serve as image capturing means, autocorrelation waveform extraction processing units 132 and 222 serve as autocorrelation waveform extraction means, joint point extraction processing units 134 and 224 serve as joint point extraction means, and function approximation processing. The units 136 and 226 correspond to function approximation means, the similarity determination processing section 140 corresponds to image search means, the image DB 120 corresponds to feature information storage means, and the file creation processing section 228 corresponds to feature information storage processing means. Also, the operation performed by the image input units 110 and 210 is the operation of the image capturing step, the operation performed by the autocorrelation waveform extraction processing units 132 and 222 is the operation of the autocorrelation waveform extraction step, and the junction extraction processing unit 134. 224 is the operation of the junction point extraction step, the operation of the function approximation processing units 136 and 226 is the operation of the function approximation step, and the operation of the similarity determination processing unit 140 is the operation of the image search step. In addition, the operation performed by the file creation processing unit 228 corresponds to the operation of the feature information storage processing step.

  The similar image search apparatus of this embodiment has such a configuration, and the operation thereof will be described next. As described above, the shape of the autocorrelation waveform extracted from the search target image and the comparison target image varies greatly depending on the direction in which the autocorrelation processing is performed. It is necessary to set a characteristic direction as the direction of autocorrelation processing.

  FIG. 11 is a flowchart showing an operation procedure for searching for a similar image in the search processing unit 130. For example, an operation procedure for setting the rotation direction as the direction of autocorrelation processing for an image having an aspect ratio close to 1 and setting the longitudinal direction of the image as the direction of autocorrelation processing in other cases is shown. Has been.

  First, when the autocorrelation waveform extraction processing unit 132 acquires the search target image (image A) input from the image input unit 110 (step 100), the autocorrelation waveform extraction processing unit 132 calculates a rectangle with the smallest area inscribed by the image A (step 100). 101). Next, the autocorrelation waveform extraction processing unit 132 determines whether the ratio (long side / short side) of the extracted long side and short side of the rectangle is equal to or greater than a predetermined reference value (step 102). As described with reference to FIGS. 4 to 7, it is desirable to extract an autocorrelation waveform in the longitudinal direction for a figure that is long in one direction. Therefore, in the present embodiment, the autocorrelation waveform is extracted in the longitudinal direction for a graphic that is long in one direction, and the autocorrelation waveform is extracted in the rotation direction for the other graphic.

  If the ratio of the long side to the short side is greater than or equal to a reference value (for example, 1.5), an affirmative determination is made in step 102, and then the autocorrelation waveform extraction processing unit 132 determines the direction of the long side of the rectangle. The direction of autocorrelation processing is set (step 103). On the other hand, if the ratio of the long side to the short side is smaller than the reference value, a negative determination is made in the determination in step 102, and then the autocorrelation waveform extraction processing unit 132 calculates the center of gravity position of the image A ( In step 104), a counterclockwise direction with the center of gravity as the center of rotation is set as the direction of autocorrelation processing (step 105). The counterclockwise direction is not necessarily required, and the purpose is to unify the rotation direction. Therefore, the clockwise direction may be set as the autocorrelation process direction.

  Next, the autocorrelation waveform extraction processing unit 132 performs autocorrelation processing in the set direction and acquires an autocorrelation waveform (step 106). The joint point extraction processing unit 134 extracts joint points from the extracted autocorrelation waveform (step 107), and the function approximation processing unit 136 sets each segment area divided by each joint point to a straight line or an arc. Then, a functionalization process approximating with any function of the free curve is performed (step 108), and feature information included in the function table shown in FIG. 9 is extracted (step 109).

  When the feature information of the search target image (image A) is thus extracted, the similarity determination processing unit 140 reads the feature information of one comparison target image (image B) from the image DB 120 (step 110). Then, the similarity between these two images A and B is determined (step 111).

  By the way, which feature information in the function table shown in FIG. 9 is used to determine the similarity can be determined as appropriate according to the importance of each feature information. For example, the order of multiple functions corresponding to each segment area constituting the autocorrelation waveform, and feature information (features) such as the total number of functions, the number of straight lines, the number of circular arcs, the number of free straight lines, etc. can be used alone or in combination to determine similarity Used.

  The similarity determination processing unit 140 performs matching processing using the first feature amount corresponding to the autocorrelation waveform of the image A and the second feature amount corresponding to the autocorrelation waveform of the image B, and correlates with the matching distance M. The degree S is calculated using the following formula:

M = ‖f-g‖
S = <f, g> / (‖f‖ × ‖g‖)
Here, ‖ and ‖ indicate norms, and <•> indicates an inner product. Further, f is a vector having the first feature amount as a component. g is a vector having the second feature amount as a component.

  The matching distance M indicates the length of the difference vector between the two vectors f and g. Correlation S indicates a cosine value (cos θ) of an angle θ formed by two vectors f and g. When the first feature amount corresponding to the search target image and the second feature amount corresponding to the comparison target image are very close, the lengths and directions of the two vectors f and g are similar. Therefore, the matching distance M is small and the correlation degree S is close to 1. When there are two or more feature quantities used for similarity determination, the matching distance M and the correlation degree S are calculated using the respective feature quantities, and the calculation results may be summed up for all the feature quantities.

  By the way, since each function corresponding to each divided region constituting the autocorrelation waveform most accurately represents the shape of the autocorrelation waveform, it is desirable to use a parameter that specifies the specific contents of each function as a feature amount. . Specifically, the function parameter corresponding to the first segmented region is extracted as a feature amount from the function table corresponding to each of the images A and B, and the above-described matching distance M and correlation degree S are calculated. The When the types of two functions to be compared are different, the number of parameters may be different. In this case, the calculation is performed by appropriately supplementing the parameter having a value of 0. In this way, the matching distance M and the correlation degree S are calculated for the second, third,... Functions. When the number of functions (number of segmented regions) is different, the correlation degree S and the like are calculated in accordance with the number of functions. All the matching distances M calculated in this way and the correlation degree S are summed to determine the similarity between the two images A and B.

  Next, the similarity determination processing unit 140 determines whether there is another comparison target image for which the similarity determination has not ended (step 112). If another comparison target image exists, an affirmative determination is made, and the processing from step 110 onward is repeated for the next comparison target image. If no other comparison target image exists, a negative determination is made in the determination in step 111, and then the search result output processing unit 142 outputs the result of the similarity determination (step 113). The determination result is output by, for example, displaying the degree of similarity with the comparison target image in order on the display unit 160 in the order in which it is determined that the degree of similarity is large, or printing on the printing paper using the printing device 170. Is called.

  As described above, in the similar image search device according to the present embodiment, the entire region including the fine features of the autocorrelation waveform is obtained by functionalizing each of the divided regions divided by the joint points included in the autocorrelation waveform. Can be accurately approximated by function, and by using the feature information extracted in this approximation process, it is possible to improve the search accuracy when searching for similar images. In addition, when the feature information stored in advance is read in order and compared with the feature information of the captured image to be searched, a similar image can be easily searched, and there are many images to be compared. However, the operation is not complicated, and the operation can be simplified. Further, by using the database creation device shown in FIG. 10, it is possible to add an image to be compared as appropriate.

  In addition, the feature information described above includes the order of a plurality of functions corresponding to a plurality of partitioned regions constituting the autocorrelation waveform. The order of a plurality of functions constituting the autocorrelation waveform is considered to be an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by determining similarity of images using the order of functions.

  In addition, the feature information described above includes an array of section lengths of a plurality of functions corresponding to a plurality of partitioned regions constituting the autocorrelation waveform. The arrangement of section lengths corresponding to each of a plurality of functions (a plurality of divided regions) constituting the autocorrelation waveform is considered to be an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the arrangement of section lengths.

  In addition, the similarity determination processing unit 140 calculates the degree of correlation of functions corresponding to a plurality of segment areas constituting each autocorrelation waveform of the search target image and the comparison target image, and searches the search target in descending order of the correlation degree. A comparison target image similar to the target image is searched. It is considered that the degree of correlation of each function constituting the autocorrelation waveform is an important feature representing the shape of the autocorrelation waveform. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the degree of correlation of functions.

  In addition, since the autocorrelation waveform extracted along the longitudinal direction well represents the characteristics of the complex shape of the image, the search accuracy is further improved by extracting the autocorrelation waveform along the longitudinal direction of the image to be searched. It becomes possible to increase. In particular, it is possible to ensure the reproducibility of the image in the longitudinal direction by calculating the rectangle with the smallest area inscribed by the image to be searched and extracting the autocorrelation waveform in the direction along the long side of the rectangle. Thus, the same autocorrelation waveform can always be obtained for the same image. In addition, by extracting the autocorrelation waveform in a predetermined rotation direction with the center of gravity of the search target image as the rotation center, it is possible to extract a stable autocorrelation waveform that is not influenced by the direction. In addition, an autocorrelation waveform is extracted in the longitudinal direction for images that are long in one direction, and an autocorrelation waveform that is suitable for the shape of the image is acquired by extracting autocorrelation waveforms in the rotation direction for other images Is possible.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention. For example, each operation of the search processing unit 130 and the database creation unit 220 of the above-described embodiment may be performed by a computer including a CPU, a ROM, a RAM, and the like. In this case, a similar image search program (each step shown in FIG. 11 is stored in the ROM, RAM, or other storage device (hard disk device or the like), or each part of the search processing unit 130 and the database creation unit 220 is executed. A program for realizing the function may be executed by the CPU.

  Further, in the embodiment described above, the feature information is read for all the comparison target images and the similarity determination is performed. However, paying attention to the total number M of the segment areas included in the autocorrelation waveform of the search target image, You may make it reduce the comparison object image as a search object candidate. Specifically, only those whose total number N of section areas included in the autocorrelation waveform of the comparison target image coincides with the total number M or within a range of ± several% (for example, 10%) are selectively extracted, and the degree of similarity You may make it perform determination. This eliminates the need for similarity determination for comparison target images with clearly different shapes, thereby reducing the processing load and the processing time.

  In the above-described embodiment, the case where the search target image is moved along the long side direction of the rectangle with the smallest area inscribed by the search target image to extract the autocorrelation waveform is described. However, it is also possible to extract the autocorrelation waveform by moving the image to be searched along the direction perpendicular to the long side direction or the short side direction of the rectangle with the smallest area.

It is a figure which shows the structure of the similar image search device of one Embodiment. It is a figure which shows the outline | summary of the process which extracts an autocorrelation waveform. It is a figure which shows the outline | summary of the process which extracts an autocorrelation waveform. It is a figure which shows the specific example of an autocorrelation waveform. It is a figure which shows the specific example of an autocorrelation waveform. It is a figure which shows the other specific example of an autocorrelation waveform. It is a figure which shows the other specific example of an autocorrelation waveform. It is a figure which shows the outline | summary of the other process which extracts an autocorrelation waveform. It is a figure which shows the function table in which the feature information extracted is contained. It is a figure which shows the structure of a database production apparatus. It is a flowchart which shows the operation | movement procedure which searches for a similar image in a search process part.

Explanation of symbols

110, 210 Image input unit 120 Image DB (database)
130 Search processing unit 132, 222 Autocorrelation waveform extraction processing unit 134, 224 Junction point extraction processing unit 136, 226 Function approximation processing unit 140 Similarity determination processing unit 142 Search result output processing unit 150, 230 Operation unit 160, 240 Display unit 170 Printer 220 Database Creation Unit 228 File Creation Processing Unit

Claims (23)

Image capturing means for capturing a search target image;
Autocorrelation waveform extraction means for extracting an autocorrelation waveform of the search target image captured by the image capture means;
A junction point extracting means for extracting a junction point at which the tendency of the autocorrelation waveform extracted by the autocorrelation waveform extracting means changes;
Function approximating means for approximating each segmented area of the autocorrelation waveform divided by the junction point with a function,
Based on feature information related to the approximation processing by the function approximating means, an image searching means for searching for a similar image from among a plurality of comparison target images;
The autocorrelation waveform extracting means calculates a rectangle having the smallest area inscribed by the search target image and a barycentric position of the search target image, and a ratio of a long side to a short side of the rectangle is equal to or greater than a reference value. When the autocorrelation waveform is extracted in the direction along the long side and the ratio of the long side to the short side of the rectangle is smaller than a reference value, the autocorrelation is performed in a predetermined rotation direction with the center of gravity as the rotation center. similar image retrieval apparatus characterized that you extracted waveform. In claim 1,
The feature information created in association with the process of approximating each segmented area of the autocorrelation waveform corresponding to the comparison target image is further stored for each of the plurality of comparison target images.
The image search means compares the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage means. A similar image retrieval apparatus that extracts the comparison target image similar to an image. In claim 2,
When the feature information corresponding to the comparison target image is acquired using the image capturing unit, the autocorrelation waveform extracting unit, the junction point extracting unit, and the function approximating unit, the feature information is stored in the feature information. A similar image retrieval apparatus, further comprising: feature information storage processing means stored in the means. In any one of Claims 1-3,
The similar image retrieval apparatus, wherein the image capturing means is an optical reading apparatus that optically reads the density information or color information of the image to be retrieved. In any one of Claims 1-3,
The image capturing means is a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. A similar image search device as a feature. In any one of Claims 1-5,
The feature information includes an order of a plurality of functions corresponding to the plurality of segmented regions constituting the autocorrelation waveform,
The image search means searches for a similar image to the search target image from the plurality of comparison target images based on the order of the plurality of functions. In any one of Claims 1-5,
The feature information includes an array of section lengths of a plurality of functions corresponding to the plurality of partitioned regions constituting the autocorrelation waveform,
The similar image search device, wherein the image search means searches for an image similar to the search target image from the plurality of comparison target images based on the sequence of the section lengths. In any one of Claims 1-5,
The image search means calculates a correlation degree of a function corresponding to the plurality of divided regions constituting each autocorrelation waveform of the search target image and the comparison target image, and the search target is in descending order of correlation degree. A similar image search apparatus for searching for a comparison target image similar to a target image. In any one of Claims 1-8,
The image search means focuses on the total number of the divided areas included in the autocorrelation waveform corresponding to the search target image, selects a predetermined number of the comparison target images as search target candidates, and then selects the feature information. The similar image search device, wherein the comparison target image similar to the search target image is searched based on the search target image. An image capture step for capturing an image to be searched;
An autocorrelation waveform extracting step of extracting an autocorrelation waveform of the search target image captured in the image capturing step;
A junction point extraction step for extracting a junction point where the tendency of the autocorrelation waveform extracted in the autocorrelation waveform extraction step changes;
A function approximating step for approximating each of the segmented regions of the autocorrelation waveform divided by the junction points with a function;
An image search step for searching for a similar image from a plurality of comparison target images based on the feature information related to the approximation process in the function approximation step;
The autocorrelation waveform extraction step calculates a rectangle having the smallest area inscribed by the search target image and the barycentric position of the search target image, and the ratio of the long side to the short side of the rectangle is equal to or greater than a reference value. When the autocorrelation waveform is extracted in the direction along the long side and the ratio of the long side to the short side of the rectangle is smaller than a reference value, the autocorrelation is performed in a predetermined rotation direction with the center of gravity as the rotation center. A similar image retrieval method characterized by extracting a waveform . In claim 10,
The feature information created in association with the process of approximating each segmented area of the autocorrelation waveform corresponding to the comparison target image is further stored for each of the plurality of comparison target images.
The image search step includes comparing the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage unit. A similar image search method, wherein the comparison target image similar to an image is extracted. In claim 11,
When the feature information corresponding to the comparison target image is acquired using the image capture step, the autocorrelation waveform extraction step, the junction point extraction step, and the function approximation step, this feature information is stored in the feature information. A similar image search method, further comprising a feature information storage processing step of storing in the means. In any one of Claims 10-12,
The similar image retrieval method, wherein the image capturing step is performed using an optical reading device that optically reads the density information or color information of the image to be retrieved. In any one of Claims 10-12,
The image capturing step is performed using a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. A similar image retrieval method characterized by In any one of Claims 10-14,
The feature information includes an order of a plurality of functions corresponding to the plurality of segmented regions constituting the autocorrelation waveform,
The similar image search method, wherein the image search step searches for a similar image to the search target image from the plurality of comparison target images based on an order of the plurality of functions. In any one of Claims 10-14,
The feature information includes an array of section lengths of a plurality of functions corresponding to the plurality of partitioned regions constituting the autocorrelation waveform,
The similar image search method, wherein the image search step searches for a similar image to the search target image from the plurality of comparison target images based on the sequence of the section lengths. In any one of Claims 10-14,
The image search step calculates a correlation degree of a function corresponding to the plurality of segmented regions constituting each autocorrelation waveform of the search target image and the comparison target image, and the search target is in descending order of correlation degree. A similar image search method, wherein the comparison target image similar to the target image is searched. In any one of Claims 10-17,
The image search step focuses on the total number of the divided areas included in the autocorrelation waveform corresponding to the search target image, selects a predetermined number of the comparison target images as search target candidates, and then selects the feature information. A similar image search method, wherein the comparison target image similar to the search target image is searched based on the method. An autocorrelation waveform extracting means for extracting an autocorrelation waveform of a search target image captured by the image capturing means;
A junction point extracting means for extracting a junction point at which the tendency of the autocorrelation waveform extracted by the autocorrelation waveform extracting means changes;
Function approximating means for approximating each segmented area of the autocorrelation waveform divided by the junction point with a function,
Based on feature information related to the approximation processing by the function approximating means, an image searching means for searching for a similar image from among a plurality of comparison target images;
A similar image search program that functions as
The autocorrelation waveform extracting unit calculates a rectangle having the smallest area inscribed in the search target image and a gravity center position of the search target image, and a ratio of a long side and a short side of the rectangle is a reference value or more. The autocorrelation waveform is extracted in the direction along the long side, and when the ratio of the long side to the short side of the rectangle is smaller than the reference value, the autocorrelation waveform is extracted in a predetermined rotation direction with the center of gravity as the rotation center. similar image search program for. In claim 19,
Feature information created in association with a function that approximates each segmented region of the autocorrelation waveform corresponding to the comparison target image is stored in the feature information storage unit for each of the plurality of comparison target images,
The image search means compares the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage means. A similar image search program that extracts the comparison target image similar to an image. In claim 20,
When the feature information corresponding to the comparison target image is acquired using the computer, the image capturing means, the autocorrelation waveform extracting means, the junction point extracting means, and the function approximating means. A similar image search program that functions as a feature information storage processing means for storing the feature information in the feature information storage means. In any one of Claims 19-21,
The image search means calculates a correlation degree of a function corresponding to the plurality of divided regions constituting each autocorrelation waveform of the search target image and the comparison target image, and the search target is in descending order of correlation degree. A similar image search program for searching for a comparison target image similar to a target image. In any one of Claims 19-21,
The image search means focuses on the total number of the divided areas included in the autocorrelation waveform corresponding to the search target image, selects a predetermined number of the comparison target images as search target candidates, and then selects the feature information. A similar image search program that searches for the comparison target image similar to the search target image based on the search target image.

Images