JP2011257963A - Image processing device, and processing method and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology capable of more accurately removing an unneeded local feature amount than a conventional technology.SOLUTION: An image processing device comprises detection means for detecting a plurality of local feature points from an image; selection means for calculating local feature point density at a plurality of areas of the image and for selecting the local feature point which belongs to an area in which the density exceeds a predetermined threshold among the plurality of local feature points; and local feature amount calculation means for calculating the local feature amount corresponding to each selected local feature point.

Description

  The present invention relates to an image processing apparatus, a processing method thereof, and a program.

  Various techniques for retrieving similar images have been proposed. For example, a method of searching for a similar image using the overall feature (overall feature amount) of the image is known. In connection with such a technique, for example, a method has been proposed in which an image is divided into a plurality of blocks, pattern matching is performed using each representative color, and similar images are searched using color position information ( Patent Document 1).

  In addition, the feature amount of each block is calculated by dividing the image into a plurality of blocks, a label matrix with a label according to the feature amount is generated, and this is used as the overall feature amount. A search method is also known (Patent Document 2).

  There has also been proposed a method for searching for a similar image using not a whole image but a local feature amount (local feature amount) of the image. In this method, first, a characteristic point (local feature point) is detected from an image (Non-Patent Document 1). And based on the said feature point and the image information of its periphery, the feature-value (local feature-value) corresponding to the said feature point is calculated (nonpatent literature 2). Image retrieval is performed by matching local feature quantities.

  In the method using the local feature amount, the local feature amount is defined as information including a plurality of elements that are rotation invariant and enlargement / reduction invariant. Thus, a technique has been proposed that enables a search even when an image is rotated or enlarged or reduced (Non-Patent Document 3).

JP-A-8-249349 Japanese Patent Laid-Open No. 10-260983 JP 2006-190201 A JP 2008-257469 A

C. Harris and M.J. Stephens, “A combined corner and edge detector,” In Alvey Vision Conference, pages 147-152, 1988. David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, 60, 2 (2004), pp.91-110. C. Schmid and R. Mohr, “Localgray value invariants for image retrieval,” IEEE Trans. PAMI., Vol.19, No.5, pp530-534, 1997.

  When searching for an image or recognizing an image using local feature values, the local feature values must be stored in a database (DB). The number of local feature amounts is enormous, and reducing the data capacity is an important issue.

  Here, among the detected local feature points, there are also unstable local feature points with low reproducibility that disappear when the image is slightly rotated, enlarged or reduced. Such unstable local feature points (hereinafter referred to as noise feature points) cannot be used when searching for an image (otherwise, for example, when positioning).

  In addition, the noise feature point also causes an erroneous correspondence between local feature points or local feature amounts, and causes a reduction in search accuracy when searching for an image. Still further, such noise feature points are one of the major causes of wasting database capacity.

  Here, in Non-Patent Document 3 described above, a technique is provided in which a threshold is provided for the output of a function value used for detection of local feature points, and local feature points that are equal to or lower than the threshold are discarded, and local feature points are selected. Has also been mentioned. However, noise feature points cannot be sufficiently removed with such a configuration alone.

  In order to solve this problem, Patent Document 3 mentions a technique for selecting local feature points and local feature quantities suitable for image recognition. Specifically, first, local feature points (and local feature amounts) obtained from the learning model image and local feature points (and local features) obtained from the learning input image including the model object included in the learning model image. The amount). As a result of comparison, only local feature points (and local feature amounts) that are frequently paired are registered in the model dictionary as local feature points (and local feature amounts) that are used for recognition processing. Thus, selection of local feature points is performed.

  However, in the technique disclosed in Patent Document 3, it is necessary to prepare a learning model image and a plurality of learning input images for each recognition target. Therefore, for example, when considering application to a system that retrieves images from a database that stores tens of thousands of images, it is not realistic from the viewpoint of the amount of work.

  In addition, as another technique for removing noise feature points, a technique for examining a variation of a local feature amount based on a geometric variation of an image has been proposed (Patent Document 4). In this method, local feature values are tested for reproducible feature points, and only feature points that pass the test are adopted.

  However, local feature points (local feature amounts) that have passed the test by this method tend to be biased toward local feature points (local feature amounts) detected from low-frequency components. Therefore, when a search is performed using local feature points that have passed the test, there is a possibility that a search omission may occur.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique that can remove unnecessary local features more accurately than in the past.

  In order to solve the above problem, an image processing apparatus according to an aspect of the present invention includes a detection unit that detects a plurality of local feature points from an image, calculates a density of the local feature points in a plurality of regions of the image, and Selection means for selecting a local feature point belonging to a region where the density exceeds a predetermined threshold from a plurality of local feature points, and local feature amount calculation for calculating a local feature amount corresponding to each selected local feature point Means.

  According to the present invention, unnecessary local feature amounts can be accurately removed as compared with the case without this configuration. As a result, for example, unnecessary image features (local feature quantities) need not be registered in the database, so the capacity of the database can be reduced, and the search accuracy is improved when searching for image features from the database. Be made.

The figure which shows an example of a structure of the search system concerning one embodiment of this invention. 3 is a flowchart illustrating an example of a processing flow of the image registration apparatus 10 illustrated in FIG. 1. The figure which shows an example of the outline | summary of the selection process of the feature point in the feature point selection part. FIG. 4 is a diagram illustrating an example of a configuration of a search system according to a second embodiment. 9 is a flowchart illustrating an example of a processing flow of the image registration apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of a configuration of a search system according to a third embodiment. FIG. 10 is a diagram illustrating an example of a configuration of a search system according to a fourth embodiment. 10 is a flowchart illustrating an example of a processing flow of the image registration apparatus according to the fourth embodiment.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  In the following embodiment, a description will be given of a case where a local feature point is detected and a local feature amount is calculated from pixel values around the local feature point when calculating a local feature amount. Here, in the embodiment shown below, the case where the technique described in the nonpatent literature 1 is used for the detection of a local feature point is demonstrated, for example. However, any method that is not easily affected by noise may be used, and the technique described in Non-Patent Document 1 is not necessarily used.

  In the embodiment described below, a case where the technique described in Non-Patent Document 2 is used as the local feature amount will be described. In this case as well, the local feature amount that is hardly affected by noise may be used, and the technique described in Non-Patent Document 2 is not necessarily used.

(Embodiment 1)
FIG. 1 is a diagram showing an example of the configuration of a search system according to an embodiment of the present invention.

  The search system includes an image registration device 10, an image search device 40, and an image feature database 30. The image registration device 10 and the image search device 40 are connected to the image feature database 30. The image registration device 10 and the image search device 40 may be connected or may not be connected.

  The image feature database 30 is a database that stores image features. Image features (a set of local feature amounts of the image and accompanying information (for example, registered images and local feature points)) detected from the input image (registered image) 31 by the image registration device 10 are registered.

  The image registration device 10 obtains an image feature for each local region in the image based on an image to be registered in the image feature database 30 (registered image), and registers the result in the image feature database 30.

  The image search device 40 obtains an image feature for each local region in the image based on an image (query image) to be searched from the image feature database 30, and from the image feature database 30 based on the obtained image feature. Perform an image search.

  The image registration device 10 and the image search device 40 have a built-in computer. The computer includes main control means such as a CPU, and storage means such as ROM (Read Only Memory), RAM (Random Access Memory), and HDD (Hard Disk Drive). In addition, the computer may include other input / output means such as buttons, a display, or a touch panel, and communication means such as a network card. These components are connected by a bus or the like, and are controlled by the main control unit executing a program stored in the storage unit. For example, the functional configuration realized in each of the image registration device 10 and the image search device 40 is realized by the CPU reading and executing a program stored in the ROM or the like.

  Here, the image registration device 10 includes an image input unit 11, a local feature point detection unit 12, a feature point selection / selection unit 13, a local feature amount calculation unit 14, and a feature amount registration unit 15. Is done.

  The image input unit 11 inputs a registered image in the apparatus (image registration apparatus 10).

  The local feature point detection unit 12 detects a local feature point (local feature point) from the image input by the image input unit 11. The feature point indicates the calculation position of the local feature amount (local feature amount). Details of the feature point detection method will be described later. In brief, the local feature point detection unit 12 generates a luminance component image from the registered image 31, reduces the image in stages, and performs a blurring process on each reduced image. Then, feature points are detected from the generated blurred image.

  The feature point selection unit 13 plays a role of selecting a highly useful feature point from among a plurality of feature points detected by the local feature point detection unit 12 and outputting it. The feature point sorting and selection unit 13 includes a noise feature point determination unit 21, a block division unit 22, a spatial density calculation unit 23, and a local feature point selection unit 24.

  The noise feature point determination unit 21 determines a local feature point that does not satisfy a predetermined condition from among a plurality of feature points detected by the local feature point detection unit 12 as a noise feature point. Thereby, noise feature points are removed. Note that the noise feature point is a feature point that cannot be used when searching for an image. For example, the noise feature point has a reproducibility that disappears when the image is slightly rotated, enlarged, or reduced. Refers to low unstable local feature points. A method for determining this noise feature point will be described later.

  The block division unit 22 divides the entire area of the blurred image generated by the local feature point detection unit 12 into blocks of a predetermined size.

  The spatial density calculation unit 23 calculates the density of local feature points for each block divided by the block division unit 22. That is, the density of local feature points belonging to each block is calculated. The local feature point selection unit 24 compares the density of each block calculated by the spatial density calculation unit 23 with a predetermined threshold, and selects a block having a density exceeding the threshold as a selected block. Thereby, the local feature points belonging to the selected block are selected as the local feature points after the selection process. The selection result of the local feature points is output to the local feature amount calculation unit 14.

  The local feature amount calculation unit 14 calculates a local feature amount corresponding to each local feature point based on the output result from the local feature point selection unit 24. More specifically, based on local feature points belonging to the selected block and image information within a predetermined range from the local feature points on the image (registered image 31), local feature amounts corresponding to the local feature points are calculated. calculate.

  The feature amount registration unit 15 registers the local feature amount calculated by the local feature amount calculation unit 14 as an image feature in the image feature database 30 in association with the registered image 31.

  Next, an example of the configuration of the image search device 40 will be described. The image search device 40 includes an image input unit 41, a local feature point detection unit 42, a feature point selection / selection unit 43, a local feature amount calculation unit 44, and a feature comparison unit 45. Since the configurations of reference numerals 41 to 44 in the image search apparatus 40 are the same as the reference numerals 11 to 14 in the image registration apparatus 10 described above, description thereof is omitted here.

  Here, the feature comparison unit 45 uses the local feature amount calculated by the local feature amount calculation unit 44 as an image feature and compares it with the local feature amount of each registered image stored in the image feature database 30. As a result of the comparison, an image most similar to the query image 32 is output as the search result 33.

  The above is the description of the configuration of the devices that configure the search system according to the present embodiment. Note that the functional configuration provided in each of these devices (the image registration device 10 and the image search device 40) does not necessarily have to be realized as shown in the figure. Should just be realized. For example, the image registration device 10 and the image search device 40 may be realized as one device.

  Next, the operation of each component in the image registration apparatus 10 will be described with reference to FIG. Here, an example of a processing flow when registering an image feature in the image feature database 30 based on the registered image 31 will be described. Except for the processing of S110, the processing in the image search device 40 is performed in the same flow, and therefore the description of the image search device 40 is omitted. In the case of the image search device 40, a search using the local feature amount calculated in S109 is performed in the process of S110.

  When this process is started, the image registration apparatus 10 first inputs an image (registered image) 31 to be registered in the image input unit 11 (S101). Then, the local feature point detection unit 12 performs the processing of S102 to S104 to detect local feature points from the registered image 31.

  Specifically, the local feature point detection unit 12 first detects a luminance component from the registered image 31 and generates a luminance component image (S102). Next, the local feature point detection unit 12 generates scale space images having different resolutions stepwise from the generated luminance component image. At this time, the luminance component image is reduced stepwise and blurring processing is performed on each reduced image. Thereby, a plurality of blurred images are generated (S103). Note that the blurring process is realized by a convolution process between an image and a blurring filter. In the present embodiment, a Gaussian filter is employed as the blur filter, a parameter in the range of 3σ is generated with σ = 1.5 as a parameter.

  Thereafter, the local feature point detection unit 12 detects a feature point (local feature point) from each blurred image (S104). For example, a Harris operator may be used to detect the feature points (see Non-Patent Document 1).

  When the local feature points are obtained, the image registration apparatus 10 performs the processes of S105 to S108 in the feature point sorting selection unit 13 to select the feature points. Here, the feature point sorting process in the feature point sorting unit 13 will be described with reference to FIG.

  An image 61 shown in FIG. 3 is a blurred image generated by the local feature point detection unit 12, and an image 62 is a blurred image in which the local feature points detected by the local feature point detection unit 12 are set. In this case, the blurred image 62 shows both noise feature points (circles in the figure) and local feature points that are not noise feature points (punishment marks in the figure).

  Here, when the feature point sorting process starts, the feature point sorting unit 13 first selects a noise from among a plurality of local feature points detected by the local feature point detection unit 12 in the noise feature point determination unit 21. A local feature point to be a feature point is determined (S105). In this determination, the noise feature point determination unit 21 first generates an edge image from the blurred image 61 and deletes edges whose pixel values of the edge image are equal to or less than a predetermined threshold (less than the threshold for edge). Then, the remaining edges are bolded to generate an edge mask. This is to select only local feature points detected from the vicinity of the remaining edge (within a predetermined range). Finally, the local feature points outside the edge mask region are labeled as noise feature points, and the local feature points in the edge mask are labeled as non-noise feature points. Thereby, the noise feature point can be determined, and an image from which the noise feature point is removed is obtained as shown in an image 63 in FIG.

  Next, the feature point selection unit 13 divides the entire area of the blurred image 63 into blocks of a predetermined size (for example, 32 pixels × 32 pixels) in the block dividing unit 22 (S106). In the present embodiment, a case where the center of the blurred image 63 is set as a reference will be described in order from the reference position. However, the present invention is not limited to this. For example, the upper left of the blurred image 63 is used as a reference from the reference position. You may make it set in order.

  Here, for example, it is assumed that the size of the blurred image 63 is horizontal × vertical = 320 pixels × 240 pixels and the block size is 32 pixels × 32 pixels. In this case, 10 blocks can be set in the horizontal direction. In the vertical direction, seven blocks can be set in a range of 224 pixels excluding the upper 8 pixels and the lower 8 pixels. As a result, a blurred image 64 in which 70 blocks are set for the blurred image 63 is obtained.

  Next, the feature point sorting and selection unit 13 calculates the density of local feature points for each block divided by the block dividing unit 22 in the spatial density calculating unit 23 (S107). When calculating the spatial density, the spatial density calculation unit 23 firstly coordinates the local feature points that are not labeled with the noise feature points among the local feature points detected from each reduced image obtained in the process of S103. Are mapped to corresponding coordinates on the registered image 31. Next, the number of local feature points mapped inside the block is counted for each block, and each count is divided by the number of pixels in the block. Thereby, the density of local feature points of each block is calculated.

  Here, the feature point sorting and selection unit 13 compares the density calculated in S107 with the predetermined threshold in the local feature point selection unit 24, and selects a block having a density exceeding the threshold (S108). . Thereby, the image 65 is obtained. For the noise local feature points in the selected block, the noise feature point label given by the noise feature point determination unit 21 is removed. Also, all local feature points belonging to blocks other than the selected block are removed. Thereby, an image 66 is obtained. In the image 66 shown in FIG. 3, for the sake of easy understanding, at least one block to which local feature points belong is selected in the block, but it is not always necessary to configure in this way. For example, only blocks to which local feature points exceeding a predetermined threshold belong may be selected.

  Here, the image 67 shown in FIG. 3 is an image obtained by removing the drawing of the block from the image 66 shown in FIG. Here, the remaining local feature points are output to the local feature amount calculation unit 14.

  Next, the local feature quantity calculation unit 14 calculates a local feature quantity corresponding to each local feature point received from the local feature point selection unit 24 (S109). Then, the image registration device 10 registers the calculated local feature amount as an image feature in the feature amount registration unit 15 in association with the registered image 31 in the image feature database 30 (S110). Thus, the image feature registration process in the image registration apparatus 10 is completed.

  As described above, according to this embodiment, after removing noise feature points from a plurality of local feature points, the image is divided into a plurality of blocks. Then, the density of local feature points in each block is calculated, and a block whose density exceeds a predetermined threshold is selected. Finally, a local feature amount is calculated corresponding to each local feature point belonging to the selected block.

  As a result, noise feature points are more accurately removed than in the case without this configuration, and unnecessary image features need not be registered in the database (image feature database 30). Therefore, the capacity of the database can be reduced, and the search accuracy can be improved when searching for image features from the database.

(Embodiment 2)
Next, Embodiment 2 will be described. In the first embodiment, the case has been described in which local feature points outside the edge mask region generated when determining the noise feature points are determined as noise feature points. On the other hand, in the second embodiment, a configuration for determining a noise feature point by a method other than this will be described.

  Specifically, first, a feature amount that acts as noise is determined by prior learning using an image excluded from the search target, and is registered in the database as a noise feature amount. At the time of image registration (or at the time of search, etc.), the noise feature value is determined by comparing the local feature value calculated based on the registered image 31 with the registered noise feature value.

  FIG. 4 is a diagram illustrating an example of a configuration of a search system according to the second embodiment.

  The search system is provided with a noise feature database 34 in addition to the configuration of the first embodiment. The noise feature database 34 stores noise feature amounts. In this database 34, feature quantities (noise feature quantities) that act as noise are registered in advance.

  Here, the difference between the configuration of the image registration device 10 and the image search device 40 according to the first embodiment is the order of processing in the local feature amount calculation units 14 and 44. That is, before the feature point sorting and selection units 13 and 43 perform the feature point sorting process, the local feature amount calculation units 14 and 44 calculate the local feature amount.

  Next, the operation of each component in the image registration apparatus 10 according to the second embodiment will be described with reference to FIG. Here, an example of a processing flow when registering an image feature in the image feature database 30 based on the registered image 31 will be described. Except for the processing of S210, the processing in the image search device 40 is performed in the same flow, and therefore the description of the image search device 40 is omitted. In the case of the image search device 40, a search using the local feature amount calculated in S209 is performed in the process of S210.

  When this processing starts, the image registration apparatus 10 first performs the same processing as S101 to S104 described in the first embodiment. Specifically, first, the registered image 31 is input into the apparatus. Then, a luminance component image is generated from the registered image 31, the image is reduced in stages, and a blurring process is performed on each reduced image. Finally, feature points are detected from the generated blurred image (S201 to S204).

  Next, the image registration apparatus 10 calculates a local feature amount corresponding to each local feature point detected by the local feature point detection unit 12 in the local feature amount calculation unit 14 (S205). Then, the noise feature point determination unit 21 determines the noise feature point. Specifically, the local feature amount calculated by the local feature amount calculation unit 14 and the noise feature amount in the noise feature database 34 are collated (compared), and the local feature point corresponding to the matching local feature amount is determined as noise. Judged as a feature point. The noise feature point determination unit 21 removes local feature points determined as noise feature points based on the determination result (S206).

  Here, a method for creating the noise feature database 34 and a method for collating noise feature amounts (local feature amounts) will be briefly described. First, local feature points are detected and local feature amounts are calculated using a plurality of images (for example, background images) to be excluded from the search target. Next, the calculated local feature is clustered. For clustering, for example, a k-means method may be used. Finally, the clustering center (plurality) is registered in the noise feature database 34 as a noise feature amount. At the time of collation, the distance between the local feature amounts in each local feature amount calculated from the blurred image generated in the process of S203 and each noise feature amount registered in the noise feature database 34 is calculated. When the distance between the local feature amounts is equal to or less than a predetermined threshold, the noise feature point determination unit 21 determines that the local feature point to which the local feature amount at that time belongs is a noise feature point. The creation of the noise feature database 34 and the comparison of noise feature amounts (local feature amounts) may of course be performed using other methods.

  Next, the image registration apparatus 10 uses the block dividing unit 22 to convert the entire area of the blurred image generated by the processing of S203 into a block of a predetermined size (for example, 32 pixels × 32 pixels), as in the first embodiment. Divide (S207). Then, the spatial density calculation unit 23 calculates the density of local feature points for each block divided by the block division unit 22 (S208).

  After calculating the density, the image registration apparatus 10 compares the density calculated in S208 with the predetermined threshold in the local feature point selection unit 24, and selects a block having a density exceeding the threshold (S208). . For the local feature points in the selected block, the local feature points determined as noise feature points by the determination by the noise feature point determination unit 21 and the local feature amounts belonging to the feature points are restored. Further, all local feature points belonging to blocks other than the selected block and all local feature amounts belonging to the local feature points are removed.

Finally, the image registration apparatus 10 causes the feature amount registration unit 15 to register the calculated local feature amount as an image feature in the image feature database 30 in association with the registered image 31 (S210). Thus, the image feature registration process in the image registration apparatus 10 is completed.

As described above, according to the second embodiment, a feature amount (noise feature amount) that acts as noise is registered in advance in a database (noise feature database 34), and noise feature points are removed based on the registered information. In addition, the removed feature points are restored as necessary. Also in this case, as in the first embodiment, the capacity of the database (image feature database 30) can be reduced, and the search accuracy can be improved when searching for image features from the database 30.

(Embodiment 3)
Next, Embodiment 3 will be described. In the second embodiment, the noise feature amount is determined by collating the local feature amount calculated based on the registered image 31 with the noise feature amount registered in the noise feature database 34, thereby determining the noise feature point. Explained when to do. On the other hand, in the third embodiment, a case will be described in which a predetermined important local feature point is determined instead of determining a noise feature point.

  FIG. 6 is a diagram illustrating an example of a configuration of a search system according to the third embodiment.

  The search system is provided with an important feature database 35 in addition to the configuration of the first embodiment. The important feature database 35 stores important features (local feature amounts) determined in advance. That is, important local feature points (hereinafter referred to as important feature points) are registered in the important feature database 35. An important feature point is a feature point of an object to be searched (ie, not a feature point such as a background image), and refers to a local feature amount useful at the time of searching.

  Here, the difference between the configuration of the image registration apparatus 10 and the image search apparatus 40 according to the second embodiment is that, in the third embodiment, an important feature quantity is determined instead of a noise feature quantity. That is, important feature point determination units 25 and 55 are provided instead of the noise feature point determination units 21 and 51 in the second embodiment.

  In this case, when the local feature amount corresponding to each local feature point is calculated by the local feature amount calculating units 14 and 44, the important feature point determining units 25 and 55 determine the important feature point. Specifically, the local feature amount and the local feature amount in the important feature database 35 are collated (compared), and a local feature point corresponding to the matching local feature amount is determined as an important feature point. Thereafter, the important feature point determination units 25 and 55 remove local feature points other than the local feature points determined to be important feature points.

  Here, since the processing flow in the image registration apparatus 10 and the image search apparatus 40 according to the third embodiment is the same as that in the second embodiment (see FIG. 5), the differences will be briefly described here.

  First, a method for creating the important feature database 35 and a method for collating important feature amounts (local feature amounts) will be briefly described. When creating the important feature database 35, first, a local feature point is detected and a local feature amount is calculated using a plurality of images including only the search target object. Next, the calculated local feature is clustered. For clustering, for example, a k-means method may be used. Finally, the clustering center (plurality) is registered in the important feature database 35 as an important local feature amount. At the time of collation, the distance between the local feature amounts in each local feature amount calculated from the blurred image generated when the feature point is detected (processing in S203) and each important local feature amount registered in the important feature database 35 Calculate If the distance between the local feature amounts is equal to or smaller than a predetermined threshold value, the important feature point determination units 25 and 55 determine that the local feature point to which the local feature amount at that time belongs is an important feature point. Of course, the creation of the important feature database 35 and the collation of the important feature amount (local feature amount) may be performed using other methods.

  Further, the local feature point selection units 24 and 54 select a block whose density calculated by the spatial density calculation units 23 and 53 exceeds a predetermined threshold as a selection block. For the local feature points in the selected block, the local feature points that have not been determined as important feature points by the important feature point determination units 25 and 55 and the local feature values belonging to the local feature points are restored. Also, all local feature points belonging to blocks other than the selected block and all local feature points belonging to the local feature points are removed.

  As described above, according to the third embodiment, important feature amounts are registered in the database (important feature database 35) in advance, and unimportant feature amounts (that is, noise feature points) are removed based on the registered information. In addition, the removed feature points are restored as necessary. Also in this case, as in the first embodiment, the capacity of the database (image feature database 30) can be reduced, and the search accuracy can be improved when searching for image features from the database 30.

(Embodiment 4)
Next, Embodiment 4 will be described. In the first embodiment, the spatial density of local feature points calculated based on the registered image 31 is obtained, and local feature points and local feature quantities belonging to a high-density partial region are registered in the image feature database 30. explained. On the other hand, in the fourth embodiment, an edge is detected from the registered image 31, the spatial density of the detected edge is obtained, and local feature points and local feature quantities belonging to a partial region having a high edge density are imaged. A case of registration in the feature database 30 will be described.

  FIG. 7 is a diagram illustrating an example of a configuration of a search system according to the fourth embodiment.

  Here, as a difference in configuration between the image registration apparatus 10 and the image search apparatus 40 according to the first embodiment, edge detection units 26 and 56 that detect edges from the registered image 31 (or the query image 32) are provided. ing.

  Next, the operation of each component in the image registration apparatus 10 according to the fourth embodiment will be described with reference to FIG. Here, an example of a processing flow when registering an image feature in the image feature database 30 based on the registered image 31 will be described. Except for the processing of S310, the processing in the image search device 40 is performed in the same flow, and thus the description of the image search device 40 is omitted. In the case of the image search device 40, the search using the local feature amount calculated in S309 is performed in the process of S310.

  When this processing starts, the image registration apparatus 10 first performs the same processing as S101 to S104 described in the first embodiment. Specifically, first, the registered image 31 is input into the apparatus. Then, a luminance component image is generated from the registered image 31, the image is reduced in stages, and a blurring process is performed on each reduced image. Finally, feature points are detected from the generated blurred image (S301 to S304).

  Next, the image registration apparatus 10 performs the processing of S305 to S308 in the feature point sorting selection unit 13 to select the feature points. Specifically, first, the edge detection unit 26 detects an edge from the blurred image obtained by the process of S303, and generates an edge image. Then, an edge label is given to a pixel whose absolute value of the pixel value of the edge image exceeds a predetermined threshold (edge threshold) (S305).

  Next, the image registration apparatus 10 divides the entire area of the blurred image obtained by the processing of S303 into blocks of a predetermined size (for example, 32 pixels × 32 pixels) in the block dividing unit 22 as in the first embodiment. (S306).

  After the block division, the image registration apparatus 10 calculates the edge density for each block divided by the block division unit 22 in the spatial density calculation unit 23 (S307). In calculating the edge density, first, the coordinates of a pixel (edge pixel) to which an edge label is assigned are mapped to corresponding coordinates on the registered image 31. Then, the number of edge pixels mapped inside the block is counted for each block. Finally, each count number is divided by the number of pixels in the block. Thereby, the density of the edge pixels of each block is calculated.

  When the edge density is calculated, the image registration apparatus 10 compares the density calculated in S307 with a predetermined threshold in the local feature point selection unit 24, and selects a block having a density exceeding the threshold. (S308). Then, all local feature points belonging to blocks other than the selected block are removed from the local feature points detected in the process of S304. The remaining local feature points are output toward the local feature amount calculation unit 14.

  Next, in the image registration apparatus 10, the local feature amount calculation unit 14 calculates a local feature amount corresponding to each local feature point received from the local feature point selection unit 24 (S309). Then, the image registration apparatus 10 registers the calculated local feature amount as an image feature in the feature amount registration unit 15 in association with the registered image 31 in the image feature database 30 (S310). Thus, the image feature registration process in the image registration apparatus 10 is completed.

  As described above, according to the fourth embodiment, noise feature points are removed by selecting local feature points based on edges detected from an image. Also in this case, as in the first embodiment, the capacity of the database (image feature database 30) can be reduced, and the search accuracy can be improved when searching for image features from the database 30.

  The above is an example of a typical embodiment of the present invention, but the present invention is not limited to the embodiment described above and shown in the drawings, and can be appropriately modified and implemented without departing from the scope of the present invention. . Here, some modifications are listed.

(Modified Embodiment 1)
For example, in the above description, the case where an image is divided into fixed-size blocks at the time of block division by the block division units 22 and 42 has been described as an example, but the present invention is not limited thereto. For example, the number of blocks after division (predetermined number of blocks) may be fixed, and the block size may be changed accordingly. Moreover, you may comprise so that the overlap of blocks may be accept | permitted.

(Modified Embodiment 2)
In the above description, the local feature point selection units 24 and 54 select, as selection blocks, blocks whose densities (local feature points or edges) calculated by the spatial density calculation units 23 and 53 exceed a predetermined threshold. Although the case has been described as an example, the present invention is not limited to this. For example, the selected block may be rearranged using local feature points (or edge pixels) in the selected block, and the block may be selected again after the rearrangement. At this time, for the blocks to be rearranged, at least one of position, size, shape, and number may be changed with respect to the selected block before rearrangement. This method will be briefly described. First, blocks adjacent to each other in the selected block before rearrangement are integrated as adjacent blocks. The definition of the adjacency may be 4 neighbors or 8 neighbors. Next, a spatial centroid is obtained for local feature points (or edge pixels) in the adjacent block. Then, a circular area is determined with the center of gravity as the center and the length of the radius as the distance to the local feature point (or edge pixel) in the adjacent block farthest from the center of gravity. As a result, the circular area becomes the selected block after rearrangement.

  Here, with respect to local feature points in the selected block after rearrangement, noise feature points (local feature points other than important feature points) are restored. Also, all local feature points belonging to blocks other than the selected block after rearrangement are removed.

(Modified Embodiment 3)
In the above description, the case where block division is performed when calculating the spatial density of local feature points (or edge pixels) by the spatial density calculation units 23 and 53 has been described as an example, but the present invention is not limited thereto. That is, it is only necessary to calculate the spatial density of local feature points (or edge pixels) on the image, and it is not always necessary to perform such block division. For example, local feature points (or edge pixels) are spatially clustered on the image to generate a cluster. Then, the density of local feature points (or edge pixels) may be obtained in each region formed by local feature points belonging to a cluster. As a method for calculating the density for each cluster, for example, “the number of local feature points (or edge pixels) belonging to the cluster of interest” / “total distance between coordinates between the cluster center and each local feature point (or edge pixel)” Or the like. In this case, the local feature point selection units 24 and 54 select clusters in which the spatial density of local feature points (or edge pixels) exceeds a predetermined threshold value (cluster integration threshold value). The local feature amount calculation units 14 and 44 calculate a local feature amount corresponding to each local feature point in the selected cluster.

  When spatially clustering local feature points (or edge pixels), the distance between the coordinates of the center coordinates of the area formed by the local feature points belonging to the cluster is equal to or less than a predetermined threshold (less than the cluster threshold). There may be a cluster. In such a case, the clusters may be integrated. This improves the possibility that an object in the image can be included in a single cluster. In addition, when spatially clustering local feature points (or edge pixels), density calculation or the like is performed only for clusters in which the number of local feature points (or edge pixels) belonging to a certain cluster is a predetermined number or less (after that) The following process may be performed. As a result, clusters including many noise feature points and the like can be removed.

(Modified Embodiment 4)
In the above description, the density of local feature points (or edge pixels) is calculated for each block in the spatial density calculators 23 and 53, and the selected block is determined based on the threshold value for the density. I can't. For example, instead of the density, the number of local feature points (or edge pixels) in the block may be counted, and the selected block may be determined based on a threshold for the number of local feature points (or edge pixels). . Thereby, calculation can be simplified.

(Modified Embodiment 5)
In the above description, the local feature point detection units 12 and 42 generate a luminance component image, reduce it stepwise, and generate a scale space image by blurring each reduced image. However, the present invention is not limited to this. That is, it is only necessary to generate images having different resolutions in stages as scale space images. For example, a blurred image generated by gradually increasing the blurring degree from the luminance component image may be generated, and the set of the blurred images may be a scale space image. Thereby, it is possible to cope with a search for an image subjected to enlargement and reduction processing.

(Modified Embodiment 6)
In the description of the fourth embodiment described above, the edge detection units 26 and 56 have described the case where the same edge label is assigned to pixels whose edge pixel absolute value exceeds a predetermined threshold value. Not limited to. For example, the connection relationship of pixels whose edge pixel absolute value exceeds a predetermined threshold is examined, a set of connected edge pixels is set as an edge segment, and a different label is assigned to each edge segment. May be. In this case, the spatial density calculators 23 and 53 calculate the density for each edge having the same edge label in each block. Thereby, the influence of a noise edge can be suppressed.

(Modified Embodiment 7)
In the above description, the case where the feature point selection process is performed on all the blurred images generated by the local feature point detection units 12 and 42 has been described as an example. However, the present invention is not limited thereto. . For example, different processing is performed on a local feature amount detected from a blurred image in which the resolution of a blurred image (scale space image) exceeds a predetermined resolution and a local feature amount detected from a blurred image having a predetermined resolution or less. You may carry out. Specifically, when the resolution is defined by the image size and the size of the blurred image exceeds a predetermined size (horizontal × vertical = 320 pixels × 240 pixels), the processing described in the above embodiment (block division to density calculation result) (Calculation of feature amount corresponding to the local feature point selected based on). If the size of the blurred image is equal to or smaller than the predetermined size, either calculate the local feature amount for all the remaining local feature points, or remove the noise feature point and then apply the local feature amount to all the remaining local feature points. The feature amount is calculated. This method is particularly useful in combination with noise feature point removal using an edge mask. That is, as the resolution of the blurred image increases, the fine edges of the object remain, so that there is a high possibility that many local feature points (noise feature points) that do not contribute to the search remain after the application of the edge mask.

  Therefore, by applying this method to a blurred image with high resolution, it is possible to remove local feature points that do not contribute to the search. Thereby, the amount of registered data can be suppressed. In addition, at the time of search, it is possible to suppress erroneous correspondence at the time of comparison in local feature amounts, so that the search accuracy can be improved.

  On the other hand, as the resolution of the blurred image is lowered, the fine edges of the object are less likely to remain, and the edges extracted by the edge mask are more likely to be important edges that constitute the object. Therefore, when this method is applied to a blurred image with a low resolution, there is a possibility that feature points necessary for the search are removed.

(Modified Embodiment 8)
In the above description, the case where the image registration device 10 and the image search device 40 perform the calculation of the local feature points and the selection processing of the local feature values is described as an example, but the present invention is not limited thereto. For example, a simple image processing apparatus may be configured to perform such processing.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (18)

Detecting means for detecting a plurality of local feature points from the image;
Selection means for calculating a density of the local feature points in a plurality of regions of the image, and selecting a local feature point belonging to a region where the density exceeds a predetermined threshold from the plurality of local feature points;
An image processing apparatus comprising: a local feature amount calculating unit that calculates a local feature amount corresponding to each selected local feature point. The selection means includes
Noise feature point determination means for determining a local feature point to be a noise feature point based on a pixel value of each pixel in the image,
After removing the local feature point determined to be the noise feature point from the plurality of local feature points detected by the detection means, the density is calculated using the remaining local feature points and the local feature points The image processing apparatus according to claim 1, wherein a feature point is selected. The noise feature point determination means includes:
The image according to claim 2, wherein an edge is detected from the image, and a local feature point that is not within a predetermined range from an edge whose pixel value of the detected edge exceeds an edge threshold value is determined as a noise feature point. Processing equipment. The selection means includes
Block dividing means for dividing the image into the plurality of regions using blocks;
Spatial density calculating means for calculating the density of local feature points for each block;
2. The image according to claim 1, further comprising: a local feature point selecting unit that selects a block having a density of the local feature points exceeding the predetermined threshold and selects a local feature point belonging to the selected block. Processing equipment. The block dividing means includes
The image processing apparatus according to claim 4, wherein the division is performed while allowing the blocks to overlap. The block dividing means includes
After the block is selected by the local feature point selection unit, again, the block is rearranged by changing at least one of the position, size, shape, and number of the selected block,
The local feature point selection means includes:
5. The block of which density calculated for each of the rearranged blocks by the spatial density calculating unit exceeds the predetermined threshold is selected, and local feature points belonging to the selected block are selected. The image processing apparatus described. The detection means includes
Generating scale space images with different resolutions from the image in stages, and detecting local feature points from images of each resolution of the scale space image;
The selection means includes
The local feature points that are noise feature points based on the pixel values of the image are calculated without calculating the density for the local feature points detected from the scale space image whose resolution is equal to or lower than a predetermined resolution. The image processing apparatus according to claim 1, wherein the local feature point is selected after removal. The selection means includes
Clustering means for clustering local feature points detected by the detection means on an image to generate a plurality of clusters;
Spatial density calculation means for calculating the density of local feature points in each region formed by local feature points belonging to a cluster;
2. The image according to claim 1, further comprising: a local feature point selecting unit that selects a cluster in which the density of the local feature points exceeds the predetermined threshold and selects a local feature point belonging to the selected cluster. Processing equipment. The clustering means includes
9. The image according to claim 8, wherein, when generating the clusters, the clusters in which the distance between the coordinates of the central coordinates of the region formed by the local feature points belonging to the clusters is a distance equal to or less than a cluster threshold value are integrated. Processing equipment. The spatial density calculating means includes
Of the plurality of clusters, the density is calculated only for clusters in which the number of local feature points belonging to the cluster is a predetermined number or less,
The local feature point selection means includes:
The image processing apparatus according to claim 8, wherein the cluster whose density has not been calculated is excluded from the selection target of the local feature points. Detecting means for detecting a plurality of local feature points from the image;
Local feature amount calculating means for calculating a local feature amount corresponding to each local feature point;
A reference is made to a database that registers, as noise feature quantities, local feature quantities obtained by pre-learning using an image of a region where local feature quantities are not calculated on the image, and the noise feature quantities and the detected plurality of detected feature quantities Noise feature point determination means for determining a local feature point having the noise feature amount by collating with a local feature amount;
Spatial density calculation means for calculating the density of local feature points remaining in a plurality of regions of the image after removing the local feature points having the noise feature amount from the detected local feature amounts;
An image processing apparatus comprising: a local feature point selecting unit that selects a local feature point belonging to a region where the calculated density exceeds a predetermined threshold value from the remaining local feature points. Detecting means for detecting a plurality of local feature points from the image;
Local feature amount calculating means for calculating a local feature amount corresponding to each local feature point;
Refer to a database that registers, as an important feature amount, a local feature amount obtained by pre-learning using a region on which the local feature amount should be calculated on the image, and the important feature amount and the plurality of detected local features An important feature point determination means for determining a local feature point having the important feature amount by checking the amount;
Spatial density calculation means for calculating the density of local feature points remaining in a plurality of regions of the image after excluding local feature points other than the local feature points having the important feature amount from the detected local feature amounts When,
An image processing apparatus comprising: a local feature point selecting unit that selects a local feature point belonging to a region where the calculated density exceeds a predetermined threshold value from the remaining local feature points. Detecting means for detecting a plurality of local feature points from the image;
Edge detection means for detecting an edge from the image;
Selecting means for calculating the density of the edge in a plurality of regions of the image and selecting a local feature point belonging to a region where the density exceeds a predetermined threshold from the plurality of local feature points;
An image processing apparatus comprising: a local feature amount calculating unit that calculates a local feature amount corresponding to each selected local feature point. The selection means includes
Block dividing means for dividing the image into the plurality of regions using blocks;
Spatial density calculating means for calculating the density of edges for each block;
The image processing apparatus according to claim 13, further comprising: a local feature point selecting unit that selects a block in which the edge density exceeds the predetermined threshold and selects a local feature point belonging to the selected block. . The spatial density calculating means includes
The image processing apparatus according to claim 14, wherein the density of the edge is calculated using a pixel whose edge pixel value exceeds an edge threshold value among edges detected by the detection unit. The edge detection means includes
Upon detection of edges from the image, the edge connection relationship is examined, and edge labels are assigned to each connected edge as an edge segment,
The spatial density calculating means includes
The image processing apparatus according to claim 14, wherein the density of edges in each block is calculated for each edge having the same edge label. A processing method of an image processing apparatus,
A step of detecting a plurality of local feature points from the image;
Selecting means for calculating a density of the local feature points in a plurality of regions of the image, and selecting a local feature point belonging to a region where the density exceeds a predetermined threshold from the plurality of local feature points;
A local feature amount calculating means including a step of calculating a local feature amount corresponding to each of the selected local feature points. Computer
Detecting means for detecting a plurality of local feature points from the image;
Selecting means for calculating a density of the local feature points in a plurality of regions of the image and selecting a local feature point belonging to a region where the density exceeds a predetermined threshold from the plurality of local feature points;
A program for functioning as a local feature amount calculating means for calculating a local feature amount corresponding to each selected local feature point.

Images