JP4243577B2 - Similar image retrieval apparatus, method and program - Google Patents

Description

  The present invention relates to a similar image search device, method, and program for searching for similar images by comparing a search target image with a plurality of registered images. In particular, the present invention can be found in files and general drawings that contain many similar images, such as mechanical drawings, electrical wiring diagrams, electronic circuit diagrams, semiconductor wiring patterns, logo marks, trademarks, family crests, photographs, paintings, and images. The present invention relates to a similar image search device, method, and program that are effective for searching for a desired image, specifying a location where an image is included, and extracting an entire view having a location similar or coincident with a partial view. .

  In recent years, with the development of multimedia technology, there has been an increasing demand for processing images using a computer including a personal computer, a microcomputer, a dedicated image processing apparatus, and the like, and the progress of software technology therefor is also remarkable. In general, an image is handled as digital data for each pixel in a computer. However, since it is necessary to process a large amount of data, the speed of image processing largely depends on the high-speed calculation processing capability of the computer. Many proposals have been made as image processing techniques to be handled on a computer. Among them, an image search technique is one of important issues.

  The image search is to extract a desired image to be searched from a large number of images stored in advance. As a typical image search performed conventionally, there is a method (keyword search) in which a name, an index, a number, and the like are added to an image, and a search is performed using the added name, index, number, and the like. Known (for example, refer to Patent Documents 1 and 2).

As other conventional examples of image search, a method using pattern matching (for example, refer to Patent Document 3), a method for performing image comparison using color differences (for example, refer to Patent Document 4), and similarity of exemplary images. A method of performing image comparison by calculating the degree (see, for example, Patent Document 5) is known.
JP-A-2002-259388 (page 3-9, FIG. 1-11) Japanese Patent Laying-Open No. 2003-76694 (page 6-17, FIG. 1-24) JP-A-9-44519 (page 5-8, FIG. 1-7) Japanese Patent Laying-Open No. 2004-94379 (page 5-9, FIG. 1-15) JP 2001-202523 A (page 6-17, FIG. 1-9)

  By the way, in the conventional methods disclosed in Patent Document 1 and Patent Document 2 described above, it is necessary for the user to store the name, index, number, and the like added to the image, or to search for them in a list. There has been a problem that the operation of specifying etc. is complicated. In order to perform this operation without making a mistake, it is necessary to accurately store or write down the name or the like, which requires a more complicated operation accompanying the image search. In addition, each of the conventional methods disclosed in Patent Documents 3 to 5 has a problem that the processing amount is enormous and the search process takes time because it is necessary to perform processing for each pixel constituting the image. .

  The present invention was created in view of the above points, and an object of the present invention is to provide a similar image search device, method, and program that do not require complicated operations and can reduce the time required for search. There is to do.

  In order to solve the above-described problem, a similar image search apparatus according to the present invention includes an image capture unit that captures a search target image, and one or more contours included in the search target image captured by the image capture unit. On the basis of contour information extracting means for extracting a line, function approximating means for approximating the shape of the contour line extracted by the contour extracting means with one or a plurality of functions, and feature information related to the approximation processing by the function approximating means And image search means for searching for a similar image from a plurality of comparison target images.

  The similar image search method of the present invention includes an image capturing step for capturing a search target image, and contour extraction for extracting one or a plurality of contour lines included in the search target image captured in the image capture step. A step of approximating the shape of the contour extracted in the contour extraction step with one or a plurality of functions, and feature information related to the approximation processing in the function approximation step. And an image search step for searching for a similar image from the inside.

  The similar image search program of the present invention extracts a computer by a contour line extraction unit that extracts one or a plurality of contour lines included in the search target image captured by the image capture unit, and a contour line extraction unit. An image that searches for similar images from a plurality of comparison target images based on feature approximation means that approximates the shape of the contour line with one or a plurality of functions and feature information related to the approximation processing by the function approximation means It functions as a search means.

  Similar image retrieval is performed using feature information extracted by approximating the contour of the image as a function, and the user himself / herself remembers the name, index, etc. Since there is no need to search, a complicated operation is unnecessary, and the operation can be simplified. In addition, since the image search is performed using feature information obtained by approximating the contour line, there is no need to perform processing for each pixel constituting the image, so the processing amount can be greatly reduced. The time required for search can be shortened.

  In addition, feature information storage means in which feature information created in association with processing for approximating the shape of the contour line included in the comparison target image described above with one or a plurality of functions is stored for each of the plurality of comparison target images. In addition, the image search unit compares the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage unit, thereby obtaining the search target image. It is desirable to extract similar comparison target images. Alternatively, the feature information created in association with the process of approximating the shape of the contour line included in the comparison target image with one or a plurality of functions is stored in the feature information storage unit for each of the plurality of comparison target images. The image search step is similar to the search target image by comparing the feature information corresponding to the search target image with the feature information corresponding to a plurality of comparison target images stored in the feature information storage unit. It is desirable to extract a comparison target image. Thus, it is possible to easily search for similar images by sequentially reading out the feature information stored in advance and comparing it with the feature information of the captured image to be searched, and there are many images to be compared. Even in this case, the operation is not complicated, and the operation can be simplified.

  Further, when the feature information corresponding to the comparison target image is acquired using the image capturing unit, the contour line extracting unit, and the function approximating unit, the feature information storing process for storing the feature information in the feature information storing unit. It is desirable to further comprise means. Alternatively, when feature information corresponding to the comparison target image is acquired using the above-described image capturing step, contour line extracting step, and function approximating step, feature information storage processing for storing the feature information in the feature information storage unit It is desirable to further comprise steps. This makes it possible to add an image to be compared as appropriate.

  Further, it is desirable that the above-described image capturing means is an optical reading device that optically reads the density information or color information of the search target image. Alternatively, it is preferable that the above-described image capturing step is performed using an optical reading device that optically reads the density information or color information of the search target image. Thereby, it is possible to easily capture an image to be searched.

  Further, the above-described image capturing means is a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. It is desirable. Alternatively, the above-described image capturing step uses a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. It is desirable to be done. As a result, when the image data is already stored in the format, the image data can be used, and the image to be searched can be captured more easily.

  Further, it is desirable that the above-described contour line extracting means extracts a boundary portion between the search target image and the background as a contour line. Alternatively, it is desirable that the contour line extracting step described above extracts a boundary portion between the search target image and the background as a contour line. As a result, a similarity search using the shape of the image can be performed, and in particular, an accurate image search can be performed for an image using only black and white or a single color.

  Further, the contour line extracting means described above can extract the boundary between the search target image and the background and the same shade or the same color edge appearing in the internal region of the search target image as a contour line. desirable. Alternatively, the above-described contour extraction step may extract the boundary between the search target image and the background and the same shade or the same color edge appearing in the internal region of the search target image as the contour. desirable. As a result, not only the shape of the image but also a similar search using the internal halftone portion can be performed. In particular, an accurate image search can be performed for a gray image or a color image having color information.

  In addition, the contour line extracting unit described above separately extracts coordinate values for each of the X coordinate and the Y coordinate of each pixel constituting the contour line, and the function approximating unit extracts the coordinate value extracted by the contour line extracting unit. Based on this, it is desirable to separately perform function approximation processing corresponding to the X coordinate and function approximation processing corresponding to the Y coordinate. Alternatively, in the contour line extraction step described above, coordinate values are separately extracted for each of the X coordinate and Y coordinate of each pixel constituting the contour line, and the function approximation step is performed on the coordinate value extracted in the contour line extraction step. Based on this, it is desirable to separately perform function approximation processing corresponding to the X coordinate and function approximation processing corresponding to the Y coordinate. As a result, the function approximation process can be performed by paying attention to one variable, so that the process can be simplified.

  The feature information described above includes the contour length as the length of the contour line, and the image search means is similar to the search target image from the plurality of comparison target images based on the contour length. It is desirable to search. Alternatively, the feature information described above includes a contour length as the length of the contour line, and the image search step is similar to the search target image from a plurality of comparison target images based on the contour length. It is desirable to search. When an image includes a plurality of contour lines, a combination of contour lengths corresponding to the respective contour lines is considered to be an important feature representing the feature of the image. Therefore, a highly accurate image search can be performed by performing similarity determination of images using the contour length.

  In addition, the feature information described above includes the number of contours as the number of contour lines, and the image search means selects a similar image to the search target image from a plurality of comparison target images based on the number of contours. It is desirable to search. Alternatively, the above-described feature information includes the number of contours as the number of contour lines, and the image search step uses a plurality of comparison target images that are similar to the search target image based on the number of contours. It is desirable to search. When an image includes a plurality of contour lines, the number is considered to be an important feature representing the feature of the image. Therefore, a highly accurate image search can be performed by performing similarity determination of images using the number of contours.

  Further, the feature information described above includes the order of a plurality of functions constituting the contour line, and the image search means selects the search target image from the plurality of comparison target images based on the order of the plurality of functions. It is desirable to search for something similar to. Alternatively, the feature information described above includes the order of a plurality of functions that constitute the contour line, and the image search step includes selecting a search target image from a plurality of comparison target images based on the order of the plurality of functions. It is desirable to search for something similar to. The order of a plurality of functions constituting the contour line is considered to be an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by determining similarity of images using the order of functions.

  In addition, the feature information described above includes an array of section lengths corresponding to each of a plurality of functions constituting the contour line, and the image search means can select a plurality of comparison target images based on the order of the section lengths. It is desirable to search for images similar to the image to be searched. Alternatively, the above-described feature information includes a sequence of section lengths corresponding to each of a plurality of functions constituting the contour line, and the image search step includes a plurality of comparison target images based on the sequence of section lengths. It is desirable to search for images similar to the image to be searched. It is considered that the arrangement of section lengths corresponding to each of a plurality of functions constituting the contour line is an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the arrangement of section lengths.

  Further, the above-described image search means calculates the correlation degree of the functions constituting the contour line included in each of the search target image and the comparison target image, and the comparison target that is similar to the search target image in descending order of the correlation degree. It is desirable to retrieve images. Alternatively, the image search step described above calculates the correlation degree of the functions constituting the contour included in each of the search target image and the comparison target image, and the comparison target that is similar to the search target image in descending order of the correlation degree. It is desirable to retrieve images. When a plurality of contour lines are included in the image, the degree of correlation of each function constituting the contour line is considered to be an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the degree of correlation of functions.

  In addition, the above-described image search means selects a predetermined number of comparison target images as search target candidates while paying attention to the maximum contour length that is the length of the longest contour line, and then selects the search target image based on the feature information. It is desirable to search for similar comparison target images. In addition, the image search unit described above searches for a comparison target image similar to the search target image based on the feature information after selecting a predetermined number of comparison target images as search target candidates by paying attention to the number of contour lines. It is desirable. Alternatively, in the image search step described above, a predetermined number of comparison target images are selected as search target candidates by paying attention to the maximum contour length that is the length of the longest contour line, and then the search target image is determined based on the feature information. It is desirable to search for similar comparison target images. In the above-described image search step, after a predetermined number of comparison target images are selected as search target candidates by paying attention to the number of contour lines, a comparison target image similar to the search target image is searched based on the feature information. It is desirable. As a result, the number of images to be searched can be reduced, so that the processing load can be further reduced and the processing time can be shortened.

  Hereinafter, a similar image search apparatus according to an embodiment to which the present invention is applied will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a similar image search apparatus according to an embodiment. The similar image search apparatus shown in FIG. 1 is for searching a registered image (similar image) similar to this image when a search target image is input, and outputting a search result. A unit 110, an image DB (database) 120, a search processing unit 130, an operation unit 150, a display unit 160, and a printing apparatus 170.

  The image input unit 110 is for inputting the search target image into the search processing unit 130, and inputs image data indicating grayscale information and color information for each pixel constituting the search target image. For example, as the image input unit 110, an optical reading device such as an optical scanner or a digital camera that optically reads the grayscale information and color information of each pixel constituting the search target image is used. Note that as a result, it is only necessary to obtain image data indicating grayscale information for each pixel constituting the search target image. Therefore, image data indicating the grayscale information of each pixel constituting the search target image is a CD, When recorded on various information recording media such as a DVD and a semiconductor memory, a data reading device such as a disk drive device that reads image data from these recording media may be used as the image input unit 110.

  The image DB 120 stores feature information obtained by performing functionalized approximation processing on a plurality of images whose similarity is compared with the search target image (these images are referred to as “comparison target images”). A specific example of the feature information will be described in the function approximation process performed on the search target image. Prior to processing by the search processing unit 130, it is necessary to register feature information of a plurality of comparison target images in the image DB 120. Details of this registration process will be described later.

  The search processing unit 130 extracts feature information of the search target image, and performs a process of searching for a comparison target image similar to the search target image based on the feature information. For this purpose, the search processing unit 130 includes a contour tracking processing unit 132, a joint point extraction processing unit 134, a function approximation processing unit 136, a similarity determination processing unit 140, and a search result output processing unit 142.

  The contour tracking processing unit 132 extracts one or a plurality of contour lines included in the search target image captured by the image input unit 110. Specifically, the contour tracking processing unit 132 tracks the contour line of the search target image in a predetermined direction based on the image data input by the image input unit 110, and forms a pixel string (contour) constituting the contour line. Point sequence). For example, the extracted pixel row is specified separately for each coordinate value of the X coordinate and the Y coordinate. Further, when a plurality of contour lines are included in one search target image, a contour point sequence is extracted for each contour line. It should be noted that it is desirable that the above-described contour line extraction processing be divided depending on whether the search target image is an image using only monochrome or monochrome, or a grayscale image or a color image having color information. That is, in the case of an image using only black and white or a single color, the contour tracking processing unit 132 extracts a boundary portion between the search target image and the background as a contour line. As a result, a similarity search using the shape of the image can be performed, and in particular, an accurate image search can be performed for an image using only black and white or a single color. In the case of a grayscale image or a color image, the contour tracking processing unit 132 also has a boundary between the search target image and the background and an edge of the same grayscale or the same color that appears in the internal region of the search target image. Are extracted as contour lines. As a result, not only the shape of the image but also the similarity search using the internal halftone portion can be performed. In particular, an accurate image search can be performed for a grayscale image and a color image.

  FIG. 2 is a diagram showing an outline of the contour point sequence extracted by the contour tracking processing unit 132. FIG. 3 is a diagram showing a state of change in which the X coordinate of the extracted contour point sequence is separated using a parametric variable. FIG. 4 is a diagram showing a state of change in which the Y coordinate of the extracted contour point sequence is separated using a parametric variable.

  In FIG. 2, circles (◯) indicate pixels constituting the outline, and the numbers attached to the circles indicate the order of the pixels when the outline is traced. Note that the contour line of the actual search target image is composed of a larger number of pixels than in the example shown in FIG. 2, but in FIG. 2, a contour point sequence is composed of a small number of pixels to simplify the description. It is assumed that

  For example, the contour tracking processing unit 132 starts scanning from the position where the X coordinate is the smallest along the Y direction, increases the X coordinate, and assigns the number “1” to the first detected pixel. In the example illustrated in FIG. 2, the contour tracking processing unit 132 detects each pixel constituting the contour line while tracking the contour line in the clockwise direction using the pixel having the number “1” as the tracking start pixel. Each pixel is given a serial number “2”, “3”,. The pixel detection operation along the contour line is performed until the detected pixel makes a round of the tracking start pixel. FIG. 3 is a graph in which the number of each pixel constituting the contour line is plotted on the horizontal axis and the X coordinate value of each pixel is plotted on the vertical axis. FIG. 4 is a graph in which the numbers of the pixels constituting the outline are plotted on the horizontal axis and the Y coordinate values of the pixels are plotted on the vertical axis. As described above, the contour tracking processing unit 132 records the X coordinate value and the Y coordinate value separately using the serial number indicating the detection order given to each pixel constituting the contour line as a parameter, thereby enabling the contour point sequence to be recorded. Perform extraction. In the above description, as an example, the X coordinate value and the Y coordinate value are recorded separately using a parameter, but the combination of the X coordinate value and the Y coordinate value is recorded without using the parameter. You may do it. In general, the search target image includes many contour lines as boundaries between the image and the background, and a contour point sequence is extracted for each contour line.

  The joint point extraction processing unit 134 extracts joint points at which the tendency of the contour line changes based on the contour point sequence extracted by the contour tracking processing unit 132. For example, a corner point where the angle of the contour line changes suddenly is extracted as a joint point. Joint point extraction processing and function approximation processing are performed separately for each of the contour point sequence for the X coordinate shown in FIG. 3 and the contour point sequence for the Y coordinate shown in FIG.

  The function approximation processing unit 136 approximates a partial region (segmented region) divided by two joining points adjacent along the contour line using a function of a straight line, an arc, or a free curve. Create feature information related to the approximation process. For example, when the segmented region can be approximated by a straight line, a straight line is used as the approximation function, and when the segmented area cannot be approximated by a straight line and can be approximated by an arc, an arc is used as the approximation function. When an arc cannot be approximated, a free curve is used as an approximation function. When a straight line is used as the approximate function, the sign indicating that the function used is a straight line and the parameter indicating the shape of the partitioned area approximated by the straight line are characteristic information about the approximate function corresponding to this partitioned area. Created as Similarly, when an arc is used as the approximation function, a sign indicating that the function used is an arc and a parameter indicating the shape of the segment area approximated by the arc are approximate functions corresponding to the segment area. As feature information. When a free curve is used as an approximate function, a sign indicating that the function used is a free curve and a parameter indicating the shape of the segmented region approximated by the free curve is an approximate function corresponding to this segmented region. As feature information.

It should be noted that the determination of which function can approximate the segmented area is based on whether the error between the segmented area and the approximate function (error obtained by the least squares method) is less than a predetermined value. It is done by examining whether. The parameter indicating the shape of the segmented region may be any parameter as long as the shape of the segmented region can be specified. For example, as disclosed in Japanese Patent No. 2646475, the following parameters are used. It may be.
(1) For a straight line: Flag indicating a straight line, coordinates of the start point of a segmented area (2) For an arc: A flag indicating an arc, the coordinates of the start point of the arc, the coefficient of the central angle between the junction points, existing between the junction points Number of contour points to be used, coefficients of approximate function (Each coefficient when an arc is expressed by a linear combination of trigonometric functions, for example)
(3) In the case of a free curve: the number of dimensions of an approximate function indicating a free curve between joint points (≧ 3), the number of contour points existing between joint points, the center of variation of the contour point sequence between joint points, the approximation function coefficient.

  The similarity determination processing unit 140 determines the similarity between the search target image and each comparison target image based on the feature information extracted corresponding to the search target image, and performs a comparison similar to the search target image. Determine the target image. The feature information used for the similarity determination includes, in addition to the parameter indicating the shape of the section area described above, the contour length of each contour line (contour line length), the maximum contour length, and the number of contours (number of contour lines) ), The order of a plurality of functions constituting the contour line, the section length (the length of the segmented area) corresponding to each of the plurality of functions constituting the contour line, and each of the search target image and the comparison target image The correlation degree of the function to be included is included.

  5 and 6 are diagrams showing an outline of feature information extracted by each process of the contour tracking processing unit 132, the junction point extraction processing unit 134, and the function approximation processing unit 136. FIG. A piece of feature information shown in FIG. 5 is extracted for one image to be searched. In the example shown in FIG. 5, the search target image of interest includes a plurality of contour lines indicated by contour lines 1, 2, 3,... (Each contour length is defined as a contour length 1, 2). 3)), the length of the longest contour line among them is indicated by "maximum contour length". Each contour line is associated with an X-axis function table and a Y-axis function table.

  As shown in FIG. 6, in addition to the total number of functions, the contour length, the total number of sample points, the number of straight lines, the total number of straight lines, the number of arcs, the total length of arcs, the number of curves, the total length of curves, The section length, the number of sample points, the start point sample number (the serial number shown in FIGS. 2 and 3), the section length and parameters corresponding to each function are included. The total number of functions is the total number of functions included in the focused contour line and is equal to the number of segmented areas. The contour length is the length of the focused contour line. The number of straight lines is the number of segmented areas approximated by straight lines in the segmented areas constituting the focused contour line. The total length of the straight line is a total value of the lengths of the segmented areas approximated by straight lines in the segmented areas constituting the target contour line. The number of arcs is the number of segmented areas that are approximated by arcs in each segmented area constituting the focused contour line. The total arc length is a total value of the lengths of the segment areas approximated by the arc in the segment areas constituting the target contour line. The number of curves is the number of segmented areas that are approximated by a free curve in each segmented area constituting the target contour line. The total curve length is the total value of the lengths of the segmented regions approximated by the free curve in the segmented regions constituting the target contour line. In FIG. 6, a plurality of functions corresponding to the “X-axis function” indicate functions that approximate each segmented area that constitutes the contour line of interest. It corresponds.

  The similarity determination processing unit 140 can know the number of contours by referring to the table shown in FIG. Further, the similarity determination processing unit 140 refers to the table shown in FIG. 6 to determine the order of a plurality of functions constituting the contour line and the section length (section) corresponding to each of the plurality of functions constituting the contour line. It is possible to know the arrangement of the area length. The Y-axis function table has the same contents.

  The search result output processing unit 142 displays the search result on the screen of the display unit 160 or instructs the printing apparatus 170 to print the search result when a printing operation using the operation unit 150 is performed.

  Incidentally, the feature information shown in FIG. 5 and FIG. 6 is created based on the image data of the image to be searched input from the image input unit 110. In order to extract a similar image in comparison with this feature information. Needs to be extracted in advance and registered in the image DB 120 for feature information of the same content for many images to be compared.

  FIG. 7 is a diagram illustrating a configuration of a database creation apparatus that extracts feature information in advance for a comparison target image and registers the image in the image DB 120. 7 may be provided as a part of the similar image search apparatus illustrated in FIG. 1, but may be configured separately from the similar image search apparatus illustrated in FIG.

  The database creation device shown in FIG. 7 includes an image input unit 210, a database creation unit 220, an operation unit 230, and a display unit 240 in order to register feature information of many comparison target images in the image DB 120.

  The image input unit 210 inputs image data indicating grayscale information and color information for each pixel constituting the comparison target image. Similar to the image input unit 110 shown in FIG. 1, an optical scanner, a digital camera, a disk drive device, or the like can be used as the image input unit 210.

  The database creation unit 220 performs processing for extracting feature information of the comparison target image and registering it in the image DB 120. For this purpose, the database creation unit 220 includes a contour tracking processing unit 222, a joint point extraction processing unit 224, a function approximation processing unit 226, and a file creation processing unit 228. Among these, the basic operations of the contour tracking processing unit 222, the junction point extraction processing unit 224, and the function approximation processing unit 226 except for the file creation processing unit 228 are the same in the search processing unit 130 shown in FIG. This is the same as each component, and a detailed description of the operation is omitted.

  The file creation processing unit 228 registers the extracted feature information for each comparison target image (feature information included in each table shown in FIGS. 5 and 6) in the image DB 120 as a set of files. The operation unit 230 is used for performing extraction of feature information corresponding to the comparison target image, operation instructions necessary for registration, and the like. The display unit 240 is used for confirming the registered content of the feature information and displaying various operation screens necessary for registration.

  By using the database creation device having such a configuration, the feature information of a plurality of comparison target images having basically the same contents as the feature information of the search target images shown in FIGS. 5 and 6 is registered in the image DB 120. Is done.

  The image input units 110 and 210 described above are the image capturing means, the contour tracking processing units 132 and 222 are the contour line extracting means, the function approximation processing units 136 and 226 are the function approximating means, and the similarity determination processing unit 140 is the image. The image DB 120 corresponds to the feature information storage unit, and the file creation processing unit 228 corresponds to the feature information storage processing unit. Further, the functions performed by the image input units 110 and 210 are performed by the function approximation processing units 136 and 226, and the operations performed by the contour tracking processing units 132 and 222 are performed by the contour extracting step. The operation performed by the similarity determination processing unit 140 corresponds to the operation of the image search step, and the operation performed by the file creation processing unit 228 corresponds to the operation of the feature information storage processing step.

  The similar image search apparatus of this embodiment has such a configuration, and the operation thereof will be described next. When the similarity between the search target image and the comparison target image is determined using the various types of feature information illustrated in FIGS. 5 and 6, the similarity determination may be performed using all items. The similarity determination may be performed using a part of the feature information in consideration of reduction of the processing load.

  FIG. 8 is a flowchart showing an operation procedure for searching for similar images. When acquiring the feature information of the search target image (image A) (step 100), the similarity determination processing unit 140 reads the feature information of one comparison target image (image B) from the image DB 120 (step 101). Next, the similarity determination processing unit 140 selects the L-th contour line in descending order of the contour length from the contour lines included in each of the search target image and the comparison target image (step 102). The two contour lines are compared (step 103). Initially, the contour line with the longest contour length is selected from each of the search target image and the comparison target image (L = 1), and the feature information of these two contour lines is compared. This comparison processing can be performed by any one of (1) contour length, (2) order of a plurality of functions constituting the contour line, and (3) arrangement of section lengths corresponding to the plurality of functions constituting the contour line, etc. In addition to the case where these feature quantities are used alone, there may be a case where two or more of these feature quantities are used in combination. By this comparison processing, similarity determination for the Lth contour line is performed.

  Next, the similarity determination processing unit 140 determines whether or not the comparison processing for a predetermined number of contour lines has been completed (step 104). If the number of contour lines subjected to the comparison process has not reached the predetermined number, a negative determination is made, and the processes after step 102 are repeated for the next long contour line. If the number of contour lines subjected to the comparison process reaches a predetermined number, an affirmative determination is made in the determination of step 104, and then the similarity determination processing unit 140 ends the comparison of the contour lines. It is determined whether or not there is another image to be compared (step 105). If another comparison target image exists, an affirmative determination is made, and the processing from step 101 onward is repeated for the next comparison target image. If there is no other comparison target image, a negative determination is made in the determination in step 105, and the similarity determination processing unit 140 uses the comparison result performed for a predetermined number of contour lines of each comparison target image. Then, the degree of similarity between each comparison target image and the search target image is determined (step 106), and the determination result is output (step 107). The determination result is output by, for example, displaying the degree of similarity with the comparison target image in order on the display unit 160 in the order in which it is determined that the degree of similarity is large, or printing on the printing paper using the printing device 170. Is called.

  FIG. 9 is a flowchart showing an operation procedure of a modified example for searching for similar images. When the similarity determination processing unit 140 acquires the feature information of the search target image (image A) (step 200), the similarity determination processing unit 140 extracts the maximum contour length from the feature information (step 201). Next, the similarity determination processing unit 140 selects n comparisons having the maximum contour length close to the maximum contour length extracted in step 201 from all the comparison target images (image B) registered in the image DB 120. A target image is selected (step 202). In the operation procedure shown in FIG. 9, n comparison target images are selected by paying attention to the maximum contour length, but n comparison target images may be selected by paying attention to the number of contours.

  Next, the similarity determination processing unit 140 reads out feature information of one comparison target image from the selected n comparison target images (step 203). The similarity determination processing unit 140 selects the L-th contour line in descending order of the contour length from the contour lines included in the search target image, and the X-axis function table and the Y-axis function table ( A first feature amount used for similarity determination is extracted from among FIGS. 5 and 6 (step 204). In addition, the similarity determination processing unit 140 selects the L-th contour line from the contour lines included in the comparison target image in the descending order of the contour length, and the X-axis function table and the Y-axis function table corresponding to the contour line. A second feature amount used for similarity determination is extracted from the list (step 205). Initially, the contour line having the longest contour length is selected from each of the search target image and the comparison target image (L = 1), and the first and second feature amounts corresponding to the respective contour lines are extracted. Which feature information is used as the first and second feature quantities from among a plurality of feature information included in the X-axis function table and the Y-axis function table can be determined as appropriate according to the importance of the feature information. it can. For example, the contour length, the total number of functions, the number of straight lines, the number of arcs, and the number of curves are used as the first and second feature amounts. Alternatively, since each function is an important feature representing the shape of the contour line, the parameters of each function may be used as the first and second feature amounts.

  Next, the similarity determination processing unit 140 performs a matching process using the first feature quantity and the second feature quantity, and calculates the matching distance M and the correlation degree S using the following expressions (step 206). .

M = ‖f-g‖
S = <f, g> / (‖f‖ × ‖g‖)
Here, ‖ and ‖ indicate norms, and <•> indicates an inner product. Further, f is a vector having the first feature amount as a component. g is a vector having the second feature amount as a component.

  The matching distance M indicates the length of the difference vector between the two vectors f and g. Correlation S indicates a cosine value (cos θ) of an angle θ formed by two vectors f and g. When the first feature amount corresponding to the search target image and the second feature amount corresponding to the comparison target image are very close, the lengths and directions of the two vectors f and g are similar. Therefore, the matching distance M is small and the correlation degree S is close to 1.

  Next, the similarity determination processing unit 140 determines whether or not the comparison processing for a predetermined number of contour lines has been completed (step 207). If the number of contour lines that have undergone the matching process has not reached the predetermined number, a negative determination is made, and the processes after step 204 are repeated for the next long contour line. When the number of contour lines that have undergone the comparison process reaches a predetermined number, an affirmative determination is made in the determination in step 207, and the similarity determination processing unit 140 then ends the comparison of the contour lines. It is determined whether or not there is another image to be compared (step 208). If another comparison target image exists, an affirmative determination is made, and the processing from step 203 onward is repeated for the next comparison target image. If there is no other comparison target image, a negative determination is made in the determination in step 208, and then the similarity determination processing unit 140 uses the comparison result performed for a predetermined number of contour lines of each comparison target image. Then, the similarity between the n comparison target images and the search target image is determined (step 209), and the determination result is output (step 210).

  As described above, in the similar image search apparatus according to the present embodiment, similar images are searched using feature information extracted by function approximation of the contour of the search target image. Since it is not necessary to store an index or the like or search for a corresponding item from the list, a complicated operation is unnecessary, and the operation can be simplified. In addition, since the image search is performed using feature information obtained by approximating the contour line, there is no need to perform processing for each pixel constituting the image, so the processing amount can be greatly reduced. The time required for search can be shortened. In particular, the feature information stored in advance in the image DB 120 is sequentially read out and compared with the feature information of the retrieved image to be searched, whereby a similar image can be easily searched for, and the image to be compared Even if there are many cases, the operation is not complicated, and the operation can be simplified. Further, by using the database creation device shown in FIG. 7, it becomes possible to add an image to be compared as appropriate.

  In addition, attention is paid to one variable by separately extracting coordinate values for each of the X coordinate and Y coordinate of each pixel constituting the contour line and performing processing by the junction point extraction processing unit 134 and the function approximation processing unit 136. Since the function approximation process can be performed, the process can be simplified as compared with the case where the functionalization process is simultaneously performed on the two variables XY.

  The feature information described above includes a contour length as the length of the contour line. When an image includes a plurality of contour lines, a combination of contour lengths corresponding to the respective contour lines is considered to be an important feature representing the feature of the image. Therefore, a highly accurate image search can be performed by performing similarity determination of images using the contour length.

  The feature information described above includes the number of contours as the number of contour lines. When an image includes a plurality of contour lines, the number is considered to be an important feature representing the feature of the image. Therefore, a highly accurate image search can be performed by performing similarity determination of images using the number of contours.

  The feature information described above includes the order of a plurality of functions that constitute the contour line. The order of a plurality of functions constituting the contour line is considered to be an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by determining similarity of images using the order of functions.

  The feature information described above includes an array of section lengths corresponding to each of a plurality of functions constituting the contour line. It is considered that the arrangement of section lengths corresponding to each of a plurality of functions constituting the contour line is an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the arrangement of section lengths.

  Further, in the operation procedure shown in FIG. 9, the similarity determination processing unit 140 described above calculates the degree of correlation of the functions constituting the contours included in each of the search target image and the comparison target image to determine the similarity. It is carried out. When a plurality of contour lines are included in the image, the degree of correlation of each function constituting the contour line is considered to be an important feature representing the shape of the contour line. Therefore, an image search with high accuracy can be performed by performing similarity determination of images using the degree of correlation of functions.

  Further, in the operation procedure illustrated in FIG. 9, the similarity determination processing unit 140 selects a predetermined number of comparison target images based on the maximum contour length (or the number of contour lines) that is the length of the longest contour line. Since the similarity determination is performed for each image to be searched and the number of images to be searched can be reduced, the processing load can be further reduced and the processing time can be shortened.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention. For example, each operation of the search processing unit 130 and the database creation unit 220 of the above-described embodiment may be performed by a computer including a CPU, a ROM, a RAM, and the like. In this case, a similar image search program (each step shown in FIGS. 8 and 9) stored in the ROM, RAM, or other storage device (hard disk device or the like) is executed, or the search processing unit 130 or the database creation unit 220 is executed. The program for realizing the function of each part of the above may be executed by the CPU.

It is a figure which shows the structure of the similar image search device of one Embodiment. It is a figure which shows the outline of the outline point sequence extracted by the outline tracking process part. It is a figure which shows the mode of the change which isolate | separated X coordinate of the extracted outline point row | line | column using the parameter. It is a figure which shows the mode of the change which isolate | separated the Y coordinate of the extracted outline point row | line | column using the parameter. It is a figure which shows the outline | summary of the feature information extracted. It is a figure which shows the outline | summary of the feature information extracted. It is a figure which shows the structure of the database preparation apparatus which extracts characteristic information about a comparison object image beforehand, and registers with respect to image DB. It is a flowchart which shows the operation | movement procedure which searches for a similar image. It is a flowchart which shows the operation | movement procedure of the modification which searches for a similar image.

Explanation of symbols

110, 210 Image input unit 120 Image DB (database)
130 Search processing unit 132, 222 Contour tracking processing unit 134, 224 Junction point extraction processing unit 136, 226 Function approximation processing unit 140 Similarity determination processing unit 142 Search result output processing unit 150, 230 Operation unit 160, 240 Display unit 170 Printing Device 220 Database creation unit 228 File creation processing unit

Claims (23)

Image capturing means for capturing a search target image;
Contour extracting means for extracting one or a plurality of contour lines included in the search target image captured by the image capturing means;
A joint point extracting means for extracting a joint point whose tendency changes with respect to the contour line extracted by the contour line extracting means;
A segmented region as a partial region segmented by two joint points adjacent to each other along the contour line extracted by the contour line extracting unit is used using a function of a straight line, an arc, or a free curve Function approximating means to approximate;
When the function corresponding to each segmented region of the contour line by approximation processing by the function approximation means is determined, based on the order of a plurality of functions constituting the contour line, the from the plurality of comparison target images Image search means for searching for an image similar to the search target image ;
A similar image retrieval apparatus comprising: In claim 1,
Features created in connection with a process of approximating a segmented area divided by two joint points adjacent to each other along a contour line included in the comparison target image using a function of a straight line, an arc, or a free curve Further comprising feature information storage means for storing information for each of the plurality of comparison target images,
The image search means compares the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage means. A similar image retrieval apparatus that extracts the comparison target image similar to an image. In claim 2,
When the feature information corresponding to the comparison target image is acquired using the image capturing unit, the contour line extracting unit, the joint point extracting unit, and the function approximating unit, the feature information is stored in the feature information storing unit. A similar image retrieval apparatus, further comprising: feature information storage processing means for storing the information. In any one of Claims 1-3,
The similar image retrieval apparatus, wherein the image capturing means is an optical reading apparatus that optically reads the density information or color information of the image to be retrieved. In any one of Claims 1-3,
The image capturing means is a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. A similar image search device as a feature. In any one of Claims 1-5,
The similar image retrieval apparatus, wherein the contour line extraction unit extracts a boundary portion between the search target image and a background as a contour line. In any one of Claims 1-5,
The contour line extracting unit extracts a boundary portion between the search target image and a background and an edge portion of the same gradation or the same color appearing in an internal region of the search target image as a contour line. A similar image search device. In any one of Claims 1-7,
The contour line extracting means separately extracts coordinate values for each of the X coordinate and the Y coordinate of each pixel constituting the contour line,
The function approximating unit separately performs a function approximating process corresponding to the X coordinate and a function approximating process corresponding to the Y coordinate based on the coordinate values extracted by the contour line extracting unit. apparatus. In any one of Claims 1-8,
The image search means selects a predetermined number of the comparison target images as search target candidates while paying attention to the maximum contour length that is the length of the longest contour line, and then selects the search target image based on the feature information. A similar image search device for searching for similar images to be compared. In any one of Claims 1-8,
The image search means selects a predetermined number of the comparison target images as search target candidates by paying attention to the number of contour lines, and then searches for the comparison target image similar to the search target image based on the feature information. A similar image search device characterized by: An image capture step for capturing an image to be searched;
A contour extracting step for extracting one or a plurality of contours included in the search target image captured in the image capturing step;
A joint point extracting step for extracting a joint point whose tendency changes with respect to the contour line extracted by the contour line extracting step;
Using a function of one of a straight line, an arc, and a free curve, a segmented area as a partial area segmented by two joint points adjacent to each other along the contour extracted in the contour extracting step A function approximation step to approximate;
When the function corresponding to each segmented region of the contour line by approximation process in the function approximation step is determined, based on the order of a plurality of functions constituting the contour line, the from the plurality of comparison target images An image search step for searching for an image similar to the search target image ;
A similar image retrieval method characterized by comprising: In claim 11,
Features created in connection with a process of approximating a segmented area divided by two joint points adjacent to each other along a contour line included in the comparison target image using a function of a straight line, an arc, or a free curve Information is stored in the feature information storage means for each of the plurality of comparison target images,
The image search step includes comparing the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage unit. A similar image search method, wherein the comparison target image similar to an image is extracted. In claim 12,
When the feature information corresponding to the comparison target image is acquired using the image capture step, the contour line extraction step, the joint point extraction step, and the function approximation step, this feature information is stored in the feature information storage means. The similar image retrieval method further comprising a feature information storage processing step of storing in In any one of Claims 11-13,
The similar image retrieval method, wherein the image capturing step is performed by using an optical reading device that optically reads the density information or color information of the image to be retrieved. In any one of Claims 11-13,
The image capturing step is performed using a data reading device that reads image data from a recording medium in which image data including grayscale information or color information corresponding to each of a plurality of pixels constituting the search target image is stored. A similar image retrieval method characterized by In any one of Claims 11-15,
In the contour line extracting step, a boundary portion between the search target image and the background is extracted as a contour line. In any one of Claims 11-15,
In the contour line extracting step, a boundary portion between the search target image and a background, and an edge portion of the same gradation or color appearing in an internal region of the search target image are extracted as a contour line. Similar image search method. In any one of Claims 11-17,
The contour line extracting step separately extracts coordinate values for each of the X coordinate and the Y coordinate of each pixel constituting the contour line,
The function approximating step separately performs a function approximating process corresponding to the X coordinate and a function approximating process corresponding to the Y coordinate based on the coordinate values extracted in the contour line extracting step. Method. In any one of Claims 11-18,
The image search step focuses on the maximum contour length which is the length of the longest contour line, selects a predetermined number of the comparison target images as search target candidates, and then selects the search target image based on the feature information. A similar image search method, wherein the similar comparison target images are searched. In any one of Claims 11-18,
The image search step searches for the comparison target image similar to the search target image based on the feature information after selecting a predetermined number of the comparison target images as search target candidates by paying attention to the number of contour lines. A similar image search method characterized by: Computer
Contour extracting means for extracting one or a plurality of contours included in the search target image captured by the image capturing means;
A joint point extracting means for extracting a joint point whose tendency changes with respect to the contour line extracted by the contour line extracting means;
A segmented region as a partial region segmented by two joint points adjacent to each other along the contour line extracted by the contour line extracting unit is used using a function of a straight line, an arc, or a free curve Function approximating means to approximate;
When the function corresponding to each segmented region of the contour line by approximation processing by the function approximation means is determined, based on the order of a plurality of functions constituting the contour line, the from the plurality of comparison target images Image search means for searching for an image similar to the search target image ;
A similar image search program that works. In claim 21,
Features created in connection with a process of approximating a segmented area divided by two joint points adjacent to each other along a contour line included in the comparison target image using a function of a straight line, an arc, or a free curve Information is stored in the feature information storage means for each of the plurality of comparison target images,
The image search means compares the feature information corresponding to the search target image with the feature information corresponding to the plurality of comparison target images stored in the feature information storage means. A similar image search program that extracts the comparison target image similar to an image. In claim 22,
When the computer further acquires feature information corresponding to the comparison target image using the image capturing unit, the contour line extracting unit, the junction point extracting unit, and the function approximating unit, the feature information is obtained. A similar image search program that functions as a feature information storage processing means stored in the feature information storage means.

Images