JP2006011622A - Partial image retrieval system, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a partial image retrieval system, method and program, shortening a time taken in retrieval processing when retrieving a partial image similar to a target image from storage images. <P>SOLUTION: In a partial image retrieval system device, a sample characteristic and a distance between the sample characteristics are prepared, an index is attached to a storage partial area characteristic, the storage partial area characteristic similar to a target partial area characteristic is extracted by the index as a retrieval candidate characteristic, a distance between the retrieval candidate characteristic and the target partial area characteristic is calculated by the distance between the sample characteristics corresponding to combination of the retrieval candidate characteristic and the target partial area characteristic, presence of a storage image group of the similar partial image is decided, the same collation window as the target image is set to a storage characteristic inside the storage images decided that the partial image similar to the target image is present, a characteristic distance to a target characteristic is calculated from the distance between the sample characteristics, and presence/absence of the presence of the partial image similar to the target image in a corresponding position inside the storage image group is decided. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a partial image search system, method, and program suitable for identifying an image including a partial image similar to a given image and a position where the partial image is included from a plurality of stored images. .

With the spread of digital cameras, the increase in capacity and price of recording media, and the spread of large-capacity networks, an environment for easily acquiring and storing large-capacity digital images has been established.
For this reason, high-speed image search technology for quickly searching for a desired image from a large number of images is required. In particular, the partial image search is a part of the image in which the image to be searched is stored. In addition, higher speed is required.
This partial image retrieval technique is used for an image detection technique for retrieving an image including a target image as a part thereof from a huge image database at high speed.

For example, a homepage that includes an image in which a face of a specific person is captured from images stored in an image database and presented to the user, or an image in which a specific object or building is captured It is used for search technology that searches for.
Conventionally, with regard to the partial image search method, a high-speed partial image detection method for accurately and rapidly executing a search for an image location similar to an image to be searched as a sample from a larger size image without omission As described above, there is known a high-speed signal matching method that vector-quantizes features extracted by dividing an image into small regions and creates a histogram that is a feature for matching from the entire signal based on the result (for example, (See Patent Documents 1 and 2).
JP 07-262373 A Japanese Patent No. 30653314

However, in the conventional partial image detection method described above, the feature of a small region is vector-quantized and a histogram is created from the quantization result to reduce the amount of data for comparison. In view of the amount of data to be stored, the amount of data is not sufficiently reduced, and the calculation time is also very long for comparison using histograms. Therefore, there is a disadvantage that the time for searching for a partial image in the stored image cannot be significantly reduced. .
Further, in the conventional partial image detection method, since the vector quantization of the image features is converted into a histogram, the positional relationship of the target image in the partial image is accurately taken into consideration at the time of matching. There is a drawback that you can not.
The present invention has been made in view of such circumstances, and when determining the similarity between a target image and a partial image similar to the target image in the stored image, the amount of data to be compared is reduced and search processing is performed. It is an object of the present invention to provide a partial image search system, method, and program for reducing such time and accurately detecting the positional relationship of target images in a partial image at the time of collation.

The partial image search method of the present invention is a partial image search method for detecting an image including a partial image similar to a target image to be searched from a stored image group registered in a database, and a position of the partial image in the image. When detecting partial images, a sample feature creating process for creating a plurality of types of sample features having a typical feature pattern as a sample, and a sample feature for calculating a distance between sample features which is a distance between the sample features An inter-distance calculation process, an accumulated feature extraction process of selecting a plurality of small regions from each image of the accumulated image group, and extracting an accumulated feature that is an image feature in each small region, and the accumulated feature and the sample feature In comparison, an accumulated feature quantization process in which a code corresponding to the sample feature is assigned to each accumulated feature, and an attention window of a predetermined size is arranged at a predetermined interval for the set of codes. A storage sub-region feature extraction process for extracting a storage sub-region feature that is a set of codes within each window of interest, an indexing process for assigning an index to the storage sub-region feature, and a plurality of small regions from the target image are selected. A purpose feature extraction process for extracting each feature of each small region as a target feature, and a purpose of assigning a code corresponding to the sample feature to the target feature by comparing the target feature with the sample feature A target window of a predetermined size is set in the feature quantization process and the set of codes obtained in the target feature quantization process, and the set of codes in each target window is shifted while shifting the target window. A target sub-region feature extraction process for extracting a target sub-region feature, an index search process for extracting a stored sub-region feature similar to a set of target sub-region features by the index, and an index search process. Based on the distance, a feature matching process that calculates the distance between the accumulated partial area feature derived and the target partial area feature derived in the target partial area feature extraction process based on the distance between the sample features, A matching result determination process for determining whether or not there is a possibility that a partial image similar to the target image exists at the location in the stored image group, and a partial image similar to the target image may exist in the matching result determination process. At a location in the stored image that is determined to have a characteristic, a collation window having the same size as the target image is set for the code of the stored feature, and a set of codes in the collation window and the code of the target feature A feature re-matching process for calculating a feature distance with a set based on the distance between sample features, and whether or not a partial image similar to a target image exists at the location in the accumulated image group based on the feature distance Determine A collation result redetermination process; a search result candidate selection process in which a location in the stored image that is determined to have a partial image similar to the target image in the collation result redetermination process; and the target partial region The next target subregion feature selection process that specifies the target subregion feature to be matched next from the feature set, and the next search candidate feature to be matched from the set of search candidate features And a search candidate feature selection process.

As described above, according to the present invention, a set of sample features, which is a typical pattern of image features, is prepared, and the distance between each of the sample features is calculated in advance, and the target feature and the stored feature are stored. By selecting the sample feature corresponding to each and using the distance of this sample feature at the time of matching, the calculation of the distance in the matching is not performed, and the feature for matching is expressed by a code string Compared with the conventional example, the collating process can be speeded up.
In addition, according to the present invention, the positional relationship in the image is maintained by expressing the feature for matching by adding a sign to the feature without performing the quantization process using the histogram. be able to.

<First embodiment>
Hereinafter, a partial image search apparatus according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of the partial image search apparatus of the first embodiment.
In this figure, the partial image search system according to the present embodiment includes a sample feature creation unit 1, a sample feature distance calculation unit 2, an accumulation feature extraction unit 3, an accumulation feature quantization unit 4, and an accumulation partial region feature extraction. Unit 5, index assignment unit 6, target feature extraction unit 7, target feature quantization unit 8, target partial region feature extraction unit 9, index search unit 10, feature matching unit 11, and matching result determination unit 12, a feature re-collation unit 13, a collation result redetermination unit 14, a search result candidate selection unit 15, a next target partial region feature selection determination unit 16, and a next search candidate feature selection unit 17. A target image, that is, a reference image to be searched and a group of stored images stored in a database or the like, that is, a set of stored images that are searched images to be searched, and a partial image similar to the target image The location in the stored image that contains Forces.

The sample feature creation unit 1 creates sample features corresponding to a plurality of image patterns, which are typical feature patterns that serve as samples, for use in searching for images.
The distance between sample features 2 calculates the distance between sample features, which is the distance between the sample features described above. Further, the distance between sample features 2 may store the distance between the sample features in correspondence with each combination of the sample features in the database.
As the distance between the sample features, Euclidean distance, Manhattan distance, inner product, normalized cross-correlation, or the like can be used.
The accumulated feature extraction unit 3 sets a small area of a predetermined size in each image of the accumulated image group, cuts out an image in units of the small area, and stores the characteristics of the images of the plurality of small areas as accumulated characteristics. Extract as

The accumulated feature quantizing unit 4 compares the accumulated feature output from the accumulated feature extracting unit 3 with each sample feature output from the sample feature creating unit 1 and corresponds to a predetermined sample feature that approximates the accumulated feature. Is assigned.
The accumulation partial region feature extraction unit 5 sets a predetermined window of a predetermined size for the set of codes output from the accumulation feature extraction unit 3 at a predetermined interval (predetermined) And the accumulated partial area feature that is a set of the codes included in each window of interest is extracted.
The index assigning unit 6 vector-quantizes each accumulated partial region feature output from the accumulated partial region feature extracting unit 5, and assigns a vector quantization codeword as an index for a sign.

The target feature extraction unit 7 sets a small region of a predetermined size in the target image, cuts out the image in units of the small region, and extracts the features of the images of the plurality of small regions as target features.
The target feature quantizing unit 8 compares the target feature output from the target feature extracting unit 7 with each sample feature output from the sample feature creating unit 1 and compares the target feature with a predetermined sample feature that approximates the accumulated feature. Is assigned.
The target partial region feature extraction unit 9 sets a predetermined window of a predetermined size with respect to the set of codes output from the target feature extraction unit 4 at predetermined predetermined intervals (predetermined The target partial region feature that is a set of the codes included in each window of interest is extracted.

The index search unit 10 extracts similar accumulated partial region features to the set of target partial region features output from the target partial region feature extraction unit 9 using the index output from the index assigning unit 6. I do.
The feature matching unit 11 determines the distance between the accumulated partial region feature extracted by the index search unit 10 and the target partial region feature extracted by the target partial region feature extraction unit 9 by a combination of sample features corresponding to the respective codes. The calculation is performed by extracting the distance between the corresponding sample features from the database.
Based on the distance calculated by the feature matching unit 11, the matching result determination unit 12 determines whether or not there is a possibility that a partial image similar to the target image exists in the location in the stored image group.

When it is determined by the matching result determination unit 12 that there is a possibility that a partial accumulation similar to the target image exists, the feature re-collation unit 13 determines whether the accumulated feature extraction unit 3 A collation window having the same size as the target image is set for the extracted accumulated feature, and a set of codes included in the collation window and a set of codes output from the accumulated feature quantization unit 4 The distance is calculated based on the distance between the sample features as in the feature matching unit 11.
The collation result re-determination unit 14 determines whether or not a partial image similar to the target image exists at the location in the stored image group based on the distance used in the collation result determination unit 13.

The search result candidate selection unit 15 sets a location in the accumulated image determined as having a partial image similar to the target image by the collation result re-determination unit 14 as a search result candidate for performing detailed similarity examination.
The next target partial region feature selection unit 16 designates a target partial region feature to be collated next from the set of target partial region features extracted by the target partial region feature extraction unit 9.
The next search candidate feature selection unit 17 designates a storage partial region to be collated next from a set of storage partial region features output as a search result from the index search unit 10.

Next, the operation of the partial image search apparatus described above will be described with reference to FIGS. FIG. 2 is a flowchart specifically showing an operation example of the partial image search system according to the first embodiment of FIG.
The sample feature creation unit 1 may read a learning signal (image) (step S1) and extract a sample feature from this learning signal, or may read a sample feature from an accumulated image without reading a learning signal as described later. A sample feature may be extracted by selecting an image to be extracted.
As the learning signal, it is preferable to use a plurality of images that can be sufficiently learned, such as a plurality of images, for example, a group of 1000 images. Here, the sample feature creation unit 1 learns feature patterns that typically appear using a clustering algorithm.
Further, in the subsequent processing, when the learning signal is not used for sample feature extraction, it is not necessary to read the learning signal, that is, it is not necessary to perform the processing of step S1.

Next, the sample feature creation unit 1 extracts a feature that can identify this image from the input image of the learning signal by the same process as the accumulated feature extraction unit 3 described later.
Here, the feature extracted from the image of the learning signal is hereinafter referred to as a learning feature.
Further, when the learning signal is not used, the process of step S2 is not performed.
Then, the sample feature creation unit 1 extracts a sample feature that is a typical image feature pattern used in the subsequent processing (step S2).
At this time, when no learning signal is used, the sample feature creation unit 1 performs sample feature generation by randomly generating a sample feature according to a predetermined probability distribution, for example, a multidimensional Gaussian distribution.

In the case of the present embodiment, the sample feature is generated in the sample feature creation unit 1 as described above, and the sample feature is created without using the learning signal.
As another example, the learning feature may be used as a vector, and the vector feature may be subjected to vector quantization and classified (clustering) to extract the sample feature. For example, if the number of words of a vector quantization code is 1024, learning features can be classified into any one of 1024 sets (hereinafter referred to as clusters).
Here, the feature serving as the center of gravity of the learning feature belonging to each cluster is set as a sample feature.
Then, the sample feature creation unit 1 outputs a set of sample features generated by any of the methods described above.

Next, the sample feature distance calculation unit 2 reads the set of sample features (reads the feature existence range corresponding to the sample feature output from the feature existence range determination unit 22 as necessary).
The sample feature distance calculation unit 2 selects all combinations of sample features in the set of sample features, and calculates the distance between sample features that is the distance between the sample features for each combination.
At this time, the sample feature distance calculator 2 calculates, for example, a square error between the sample feature vectors as the sample feature distance, and sets it as the sample feature distance (step S3).
As another embodiment, the distance between sample features can be defined using a feature existence range corresponding to the sample feature.
For example, when the feature existence range is represented by a hypersphere in the feature vector space centered on the sample feature, the minimum distance and the maximum distance between the feature existence ranges are expressed as the distance (square error) d (fi, Using the radius ri and rj of the feature existing range corresponding to fj), the following equation (1) and equation (2) can be used.

The minimum distance dmin between the feature existence ranges obtained by the above equation (1), the maximum distance dmax between the feature existence ranges obtained by the equation (2), or both (minimum distance and maximum distance) are defined as the distance between the sample features. To do.
As a result, the sample feature distance calculation unit 2 outputs a set of distances between sample features for each combination of all sample features (can be stored in a database or the like).

Next, the accumulation feature extraction unit 3 first reads an accumulation image group from a database (not shown) (step S4).
Then, the accumulated feature extraction unit 3 shifts an image from each image of the read accumulated image group by shifting a small area having a size of, for example, “8 × 8” pixels by 1 pixel in the vertical and horizontal directions. Cut out and extract the features of the image included in each small area (step S5).
Here, as an image feature extraction method, for example, RGB values of each small region are used as they are.
For example, when the size of the accumulated image is “384 × 256” pixels, a feature vector of “8 × 8 × 3 = 192” dimension is “(384-8 + 1) × (256-8 + 1) = 93873 ”pieces are obtained.

As another embodiment, the accumulated feature extraction unit 3 can perform a two-dimensional DCT (digital cosine transform) on each of the RGB values of each small region, and characterize the coefficient vector.
Here, for example, assuming that the size of the accumulated image is “384 × 256” pixels, the feature vector of “8 × 8 × 3 = 192” dimension is “(384-8 + 1) × (256” per image. −8 + 1) = 93873 ”is obtained.
The accumulated feature extraction unit 3 outputs a set of accumulated features that are features extracted from the accumulated image by any of the methods described above.

Next, the accumulated feature quantization unit 4 reads the set of sample features output from the sample feature creating unit 1 and the set of accumulated features output from the accumulated feature extracting unit 3.
Then, the accumulated feature quantization unit 4 collates with each sample feature of the set of sample features for each accumulated feature, and assigns one code corresponding to the sample feature to each accumulated feature amount.
At this time, the accumulated feature quantization unit 4 calculates a distance, for example, a square error, between the accumulated feature and each sample feature, and assigns a code corresponding to the sample feature with the closest distance to the accumulated feature (step S6).
For example, when 1024 sample features are prepared, a number from 1 to 1024 is assigned to each sample feature, and the number is used as a code.

As another embodiment, the accumulated feature quantizing unit 4 calculates the distance between the accumulated feature and each sample feature, and when the distance between the nearest sample feature is below an assigned threshold that is a predetermined threshold, When a code corresponding to the sample feature is assigned and the distance exceeds the assignment threshold, another code that does not correspond to any sample feature, for example, in the above-described example, a number such as 1025 may be assigned as the code.
Then, the accumulated feature quantization unit 4 outputs a set of code strings extracted from each accumulated feature using any of the methods described above.

Next, the accumulation partial region feature extraction unit 5 reads a set of code strings from the accumulation feature quantization unit 4 and sets a first target window of a predetermined size in a certain location of the accumulation image. Set.
For example, the first window of interest has a size of 64 × 64 pixels, and is arranged in the upper left corner of the stored image as an initial state.
Then, the accumulation partial region feature extraction unit 5 extracts a set of codes included in the set first attention window, uses the extracted codes in order as vectors, and uses them as features for matching. This feature is used as a storage partial region feature (step S7).
Here, as a method for extracting a set of codes, for example, the small areas are arranged in the first target window in a state where there is no overlap and no gap, and a code corresponding to each small area is extracted. To do.
At this time, a feature vector of “(64/8) × (64/8) × 3 = 64” dimension, that is, one accumulated partial region feature is obtained.

A plurality of the first attention windows are set at predetermined intervals in each accumulated image, and the accumulated partial area features are extracted from the partial areas in each attention window.
That is, the first window of interest (horizontal ωx pixel × vertical ωy pixel) is the predetermined interval (interval between the same sides of each window of interest) in each accumulated image in the vertical and horizontal directions. , The intervals between the upper sides are sequentially shifted by mx pixels in the horizontal direction and my pixels in the vertical direction.

For example, if the first window of interest is arranged at intervals of 16 pixels vertically and horizontally, and the size of the accumulated image is “384 × 256” pixels, the first window of interest is “64 × 64” pixels. For each image, “21 × 13 = 273” first attention windows are set.
In addition, the arrangement interval of the first window of interest is hereinafter referred to as a margin.
Then, the storage partial region feature extraction unit 5 outputs a set of storage partial region features obtained for each first window of interest.

Next, the index assigning unit 6 reads the set of accumulated partial region features extracted by the accumulated partial region feature extracting unit 5 and assigns an index for facilitating the search to each read partial region feature. .
At this time, the index assigning unit 6 reads the sample feature set output from the sample feature creation unit 1 and the sample feature distance output from the sample feature distance calculation unit 2 as necessary.
Here, the index assigning unit 6 assigns an index to the code string constituting the partial region feature (step S8).
For example, a set of accumulated representative features, which are partial region features created in advance, is used to quantize a code string constituting each partial region feature, and use the quantized code word as an index.
That is, the above method will be briefly described. First, the index assigning unit 6 reads a set of accumulated partial region features.

Next, the index assigning unit 6 performs clustering of each partial region feature (code string). That is, the distance between the partial area features is calculated in the same manner as the feature matching unit 11 described later, and an accumulated representative feature having the smallest distance from the accumulated partial area feature is selected from the set of accumulated representative features, This is done by outputting a representative feature number that is a code corresponding to the stored representative feature.
Then, the index assigning unit 6 outputs a representative feature number corresponding to each partial region feature as an index.
As another embodiment, for example, by using a set of read sample features, each code constituting a partial region feature is expanded into a corresponding sample feature vector, and an index is assigned to a vector obtained by connecting them.

For example, as in the embodiment described in “Signal Detection Method, Signal Detection Device, Recording Medium, and Program” (Japanese Patent Application Laid-Open No. 2002-236696), a feature of each partial region image is created using a vector quantization codebook created in advance. The vector is vector-quantized, and the codeword of vector quantization is used as an index.
That is, the above method will be briefly described. First, the index assigning unit 6 reads a set of expansion vectors, which are vectors obtained by expanding a code string of accumulated partial region features.
Next, the index assigning unit 6 performs clustering of each expansion vector. This clustering is performed, for example, by encoding each expansion vector using vector quantization.
For example, if the number of code words of vector quantization is 1024, the expansion vector is classified into one of 1024 clusters.

Here, the vector quantization uses, for example, the square error as a distance function, assigns the expansion vector to a cluster whose distance from the cluster centroid is smaller than the distance from any other cluster centroid, and codeword corresponding to the cluster This is done by outputting
This cluster is created in advance using, for example, a learning vector that is a vector having the same number of dimensions as the expansion vector, which has been facilitated in advance for creating the cluster.
In other words, a cluster is such that the sum of the distances between the learning vector belonging to it and the cluster centroid is minimized, and the distance from the cluster centroid of the cluster to which the cluster belongs is any other cluster. It is comprised so that it may become smaller than the distance with a gravity center.
Then, the index assigning unit 6 outputs a vector quantization code word corresponding to each expansion vector as an index.

As another embodiment, for example, R ^* -Tree (“The R ^* -Tree: an efficient and robust access method for points and rectangles”, (N. Beckman et al., Proc. Of ACM SIGMOD Conference, pp .322-331, 1990)) and SR-Tree ("SR-Tree: Proposal of index structure for nearest neighbor search for high-dimensional data" (Natao Katayama et al., IEICE Transactions DI, Vol. J80-DI, No.8, pp.703-717 (August 1997)) and other methods that are collectively referred to as multidimensional indexes It is also possible to create an index and assign an index by associating the lowest layer area with a feature.

The region including the feature is, for example, a super cuboid corresponding to a rectangular parallelepiped in three-dimensional space if R ^* -Tree, and a super sphere corresponding to a sphere in three-dimensional space if SR-Tree. It becomes an overlap.
Then, the index assigning unit 6 outputs an index for each partial region feature obtained as described above (step S8).

Next, the target feature extraction unit 7 reads the target image (step S9). If the size of the target image is set to be equal to or larger than the sum (total) of the margin and the size of the first window of interest, for example, Since the range in which the feature of the stored image is completely extracted is included, no omission of search occurs.
Then, the target feature extraction unit 7 cuts out the target image while shifting the small region by one pixel in the vertical direction and the horizontal direction, and extracts the features of the image included in each small region in the same manner as the accumulated feature extraction unit 3. A set of target features, which are features extracted from the target image, is output (step S10).

Next, the target feature quantization unit 8 reads the set of sample features output from the sample feature creation unit 1 and the set of target features output from the target feature extraction unit 7, and sets the sample features for each target feature. Each sample feature of the set is collated, and one code corresponding to the sample feature is assigned to each target feature. Here, the procedure in which the target feature quantization unit 8 assigns codes is the same as that in the accumulated feature quantization unit 4.
Then, the target feature quantization unit 8 outputs a set of code strings extracted from each target feature of the set of target features (step S11).

Next, the target partial region feature extraction unit 9 reads a set of code strings extracted by the target feature quantization unit 8 and uses the attention window used by the storage partial region feature extraction unit 5 at one location of the target image. A window of interest having the same size (vertical × horizontal), that is, a first window of interest is used, and a set of codes included in the first window of interest is extracted.
Then, the target partial region feature extraction unit 9 uses the same extraction method as the above-described storage partial region feature extraction unit 5 to sequentially arrange the codes extracted from one place of the target image and use them as vectors for collation. This feature is output as a set of target partial region features. (Step S12).

Here, unlike the case of the feature extraction in the storage partial region feature extraction unit 2, the target partial region feature extraction unit 9 does not overlap the first target window in the target image and has no gap. The target partial region features are extracted at each position while setting them and shifting them one pixel at a time.
At this time, for example, if the first extraction window is shifted at least by the size of the margin, no search omission occurs.
For example, when the size of the target image is “80 × 80” pixels, the size of the first target window is “64 × 64” pixels, and the margin is “16 × 16” pixels, the target image may overlap. Since the number of first target windows that can be set in a state without any gap is one, “16 × 16 = 256” first target windows are set in the entire target image.
Then, the target partial region feature extraction unit 9 outputs a set of target partial region features extracted by the above-described processing.

Next, the index search unit 10 reads the index output from the index assigning unit 6 and the set of target partial region features output from the target partial region feature extraction unit 9.
When the target representative feature extraction unit 19 performs processing for extracting the target representative feature from the target partial region feature, the index search unit 10 uses the target representative feature extraction unit 19 instead of the set of target partial region features. Read a set of target representative features to be output.
The target representative feature extraction unit 19 will be described in detail in another embodiment to be described later.
In addition, the index search unit 10 may read the distance between target partial areas output from the target partial area distance calculation unit 16 as necessary.
Further, when the search threshold update unit 18 performs a process of updating the search threshold and the selection threshold, the index search unit 10 reads the updated selection threshold. The search threshold update unit 18 will be described in detail in another embodiment to be described later.

Then, the index search unit 10 uses the read index to extract an accumulated partial area feature similar to the set of target partial area features as a search candidate feature (step S13).
At this time, the index search unit 10 has a distance dt with respect to any target partial region feature in the set of target partial region features less than a selection threshold θ ₂ that is a predetermined threshold. Such (below) accumulated subregion features are extracted from the index.
The index search unit 10 calculates the distance between the partial region features by the same processing as the feature matching unit 11 (the configuration of the feature matching unit 11 will be described later).
The selection threshold θ ₂ is a value that is automatically determined from parameters described later, and a method for determining the selection threshold will be described later.

As another example, instead of a set of target sub-region features, a representative feature is extracted from the target sub-region feature as a target representative feature, and this target representative feature set is used to distance dt between objects representative feature is the storage part region feature below a selected threshold theta ₂ of, it is also possible to adopt a configuration that extracts from the index (described later).
Further, as another embodiment, the distance between the target partial areas is used by using the distance between the target partial areas, and the distance dt between each target representative area is reduced by the maximum value of the distance between the target partial areas corresponding to the target representative feature. It is also possible to adopt a configuration in which accumulated partial region features whose values are below the selection threshold θ ₂ are extracted from the index (described later).

Here, an extraction method for extracting from the index an accumulated partial region feature whose distance from a certain target partial region feature or target representative feature is below the selection threshold will be described.
This extraction method differs depending on the type of index used in the index assigning unit 3 and can be implemented in the same manner as the method described in the patent specification or the method described in the non-patent document corresponding to the index.
For example, when an index is assigned by quantization using the accumulated representative feature, and an index is assigned using the method described in “Signal detection method, signal detection apparatus, recording medium, and program” (Japanese Patent Laid-Open No. 2002-236696) In this case, a desired accumulated partial region feature can be extracted as follows.

First, the index search unit 10 reads the object partial region feature or objects representative feature, indexes created by the indexing portion 3 (the cluster), and the selection threshold theta _2.
In the following, for the sake of simplicity, the description will be made assuming that the target partial region feature is read in the configuration adopted in this embodiment, not the target representative feature. Even when the target representative feature is read, the same processing can be performed.
Then, the index search unit 10 calculates the distance between the read target partial region feature and each cluster centroid.
Subsequently, index search unit 10, based on the calculated distance dt, selects a potential cluster including storage part region feature where the distance between the objective portion region feature is below a selected threshold theta _2.

The principle of selecting the above-described cluster will be described below.
FIG. 4 is a plane on which three points Q, C1, and C2 are placed, and shows a state in which a feature space in which feature vectors are arranged is cut out.
Here, the point Q is the target partial region feature, C1 is the cluster centroid of the cluster to which the partial region feature Q belongs, C2 is the cluster centroid of the other cluster, and the distances dQ1, dQ2, and d12 are the points. The distance between Q and point C1, the distance between point Q and point C2, and the distance between point C1 and point C2 are shown.

Here, if it is necessary to extract an accumulated partial region feature whose distance dt from the point Q that is the coordinates of the target partial region feature is within the selection threshold θ ₂ , a hypersphere having a radius θ ₂ centered on the point Q storage part region feature in the interior of true thereto (circle of radius d _theta in FIG. 4).
Then, when the radius of the hypersphere centered at the point Q becomes larger than _dθ in FIG. 4, a desired storage partial region feature is included in the storage partial region features belonging to the cluster corresponding to the point C2. There is a possibility.
Therefore, when the selection threshold θ ₂ becomes larger than d _θ in FIG. 4, the cluster is selected.
_dθ is obtained using the following equations (5) to (4).

  From the above equation (3),

Is required.
Finally, when the following expression (5) is established, the index search unit 10 selects a cluster corresponding to the point C2, and extracts all accumulated partial region features belonging to the cluster.
Then, this procedure is performed for all clusters except the cluster to which the target partial region feature Q belongs, and the extracted accumulated partial region features are output as search candidate features.

Next, the feature matching unit 11 reads the set of target partial region features output from the target partial region feature extraction unit 9 and the set of search candidate features output from the index search unit 10.
The feature matching unit 11 reads the target partial region feature when the next target partial region feature selection unit 16 specifies the target partial region feature to be verified next. The next target partial region feature selection unit 16 will be described later in this embodiment.

In addition, when a search candidate feature to be matched next is designated from the next search candidate feature selection unit 17, the feature matching unit 11 reads the search candidate feature. The next search candidate feature selection unit 17 will be described later in this embodiment.
Next, the feature collation unit 11 collates the target partial region feature read from the next target partial region feature selection unit 16 with the set of search candidate features read from the next search candidate feature selection unit 17.
Then, the feature matching unit 11 calculates the distance between the target partial region feature and the search candidate feature using the distance between the sample features.
For example, when the target partial region feature and the search candidate feature are composed of a single code, the feature matching unit 11 obtains the distance between the search candidate feature and the target partial region feature as the distance between the codes, and reads it. The distance between the sample features is output as the distance.

In addition, when the target partial region feature and the search candidate feature are configured by a code string, the feature matching unit 11 corresponds to a combination of codes as a distance between codes that are corresponding elements of the code string, for example. The distance between the sample features is input, and the distance between the sample features of the elements is integrated to output the distance between the search candidate feature and the target partial region feature.
Then, when a code that does not correspond to any sample feature is detected in the set of sample features, the feature matching unit 11 sets the distance between the codes to “0”, for example.
As another example, when a code that does not correspond to any sample feature is detected in the set of sample features, the feature matching unit 11 sets the distance between the codes to a sufficiently large value, for example, all sample features. You may comprise so that it may set to the maximum value in the distance between.

Here, if there is no specified target partial region feature, the feature matching unit 11 specifies, for example, a target partial region feature corresponding to the position of the upper left corner of the target image as the initial target partial region feature, and the search candidate A feature and matching process is performed (step S14).
Further, if there is no designated search candidate feature, the feature matching unit 11 designates, for example, an arbitrarily selected search candidate feature as an initial search candidate feature, and collates it with the target partial region feature.
Then, as a result of the collation calculation (calculation of the distance between the search candidate feature and the target partial region feature), the feature matching unit 11 calculates the distance value d (ω) between the collated search candidate feature and the target partial region feature. Output.

Next, the matching result determination unit 12 reads the distance value d (ω) output from the feature matching unit 11 and determines whether there is a possibility that a partial image similar to the target image exists at the current matching location. That is, the distance value d (ω) is compared with the selection threshold value θ ₂ .
The verification result determining section 12, when the distance value d (omega) is equal to or less than the selected threshold theta _2, it determines that there is a possibility that the partial image similar to the target image exists in the current matching point, outputting the stored partial region feature as the matching portion, whereas, when the distance value d (omega) exceeds the selection threshold theta _2, if there is no possibility of partial image similar to the target image is in the current verification point The determination is made and the process proceeds to step S14 (step S15).

Next, the feature re-collation unit 13 includes a sample feature distance output from the sample feature distance calculation unit 2, a set of accumulated features as a set of codes output from the stored feature quantization unit 4, and a target feature quantum. A set of target features that is a set of codes output from the conversion unit 8 and a verification portion output from the verification result determination unit 12 are read.
Then, the feature re-collation unit 13 sets a second attention window, which is an attention window having the same size (vertical × horizontal) as the target image, in the read accumulated image.

Here, the feature re-collation unit 13 determines the location of the second window of interest corresponding to the collation location output from the collation result determination unit 12.
That is, as shown in the conceptual diagram of FIG. 5, the feature re-collation unit 13 has the same relative position of the accumulation partial region with respect to the location of the second target window as the relative position of the target partial region in the target image. In other words, the second window of interest is arranged at a position where the accumulation partial area and the target partial area at the collation location overlap.

At this time, the feature re-collation unit 13 performs processing similar to that of the storage partial region feature unit 5 and extracts a set of codes of the storage partial region and the target partial region.
Next, the feature re-collation unit 13 collates the set of accumulated feature codes extracted within the second window of interest with the set of target feature codes.
Here, the feature re-collation unit 13 calculates the distance value d between the accumulated feature in the second window of interest and the target feature, for example, by the same calculation process as the feature collation unit 11.
Then, the feature re-collation unit 9 outputs a distance value d that is a collation calculation result (step S16).

Next, the collation result re-determination unit 14 reads the distance value d output from the feature re-collation unit 13, and determines whether or not there is a possibility that a partial image similar to the target image exists in the current collation location. That is, the distance value d is compared with the search threshold value θ.
At this time, when the distance value d is equal to or smaller than the search threshold θ, the matching result re-determination unit 14 determines that a partial image similar to the target image exists at the current matching location, and detects the matching location in the accumulated image as a detected location. The position and determination result (information indicating the presence) of this detection location is output, and when the distance value d exceeds the search threshold θ, a partial image similar to the target image exists at the current verification location It determines with not, and advances a process to step S19 (step S17).

The selection threshold θ ₂ is determined from the search threshold θ, and if the selection threshold θ ₂ is set as shown in the following equation (6), no search omission occurs.

  In the above equation (6), N is the minimum number of first collation windows included in the second collation window arranged at an arbitrary position when it is assumed that the first target window is arranged in the accumulated image without overlap or gap. Yes, given by the following equations (7) to (9).

Here, Nx in equation (8) is the number of first verification windows in the horizontal direction, and Ny in equation (9) is the number of first verification windows in the vertical direction.
However, (qx, qy) is the size of the target image, that is, the size of the second target window, and (ωx, ωy) is the size of the first target window.
Next, the search result candidate selection unit 15 reads the detection location (including position information) output from the collation result re-determination unit 14 and the distance value d output from the feature re-collation unit 13.

Next, the search result candidate selection unit 15 registers the detection location and the distance value d at the detection location as search result candidates.
For example, the search result candidate selection unit 15 registers all sets of the read detection locations and the distance value d regardless of the magnitude of the distance value.
In addition, as another embodiment, the search result candidate selection unit 15 may be configured to register the search result candidates so that the number of search result candidates is equal to or less than a predetermined amount.

That is, the search result candidate selection unit 15 unconditionally registers the search result candidates up to the quantity, and if the number exceeds the quantity, the search result candidate selection unit 15 selects, for example, among the search result candidates already read. The maximum distance value dmax is compared with the distance value d read thereafter, and if the read distance value d falls below the maximum distance value dmax, the candidate corresponding to the maximum distance value dmax is deleted, and a newly read detection location And a set of distance values d are registered as candidates.
As described above, the search result candidate selection unit 15 outputs the search result candidate from the accumulated image by the method described above (step S18).

Next, the next target partial region feature selection unit 16 reads the set of target partial region features output from the target partial region feature extraction unit 9 and, if necessary, the distance value d ( ω), the selection threshold θ ₂ output from the search threshold update unit 18, and the peripheral distance lower limit value output from the peripheral distance lower limit calculation unit 21 are read.
The search threshold update unit 18 and the peripheral distance lower limit calculation unit 21 will be described later in another embodiment.
Then, the next target partial region feature selection unit 16 selects the target partial region feature to be collated next, that is, detects the presence or absence of the target partial region feature to be collated next. The process proceeds to step S20. On the other hand, when it is detected that there is a collation part, the target partial region feature to be collated next is selected, and the process proceeds to step S14 (step S19).

At this time, the next target partial region feature selection unit 16 determines, for example, whether there is a target partial region feature that has not been collated with respect to the storage partial region feature currently focused on, and ends. If there is a target partial area feature that is not yet selected, one arbitrary target partial area feature is selected from these, and when all target partial area features have been collated, attention is paid to the next storage partial area feature. In order to change, the process proceeds to step S20.
As another embodiment, the following objective partial region feature selection unit 16, if read the peripheral distance limit value, the distance limit value, the collation of an object part region feature above a selected threshold theta ₂ read omitted, it is also possible to adopt a configuration in which the distance limit value is any one to choose for the selection threshold theta ₂ following purposes partial region feature.
Then, if there is an unfinished target partial region feature, the next target partial region feature selection unit 16 outputs the target partial region feature for which the collation location is selected and the position in the target image corresponding to the target partial feature. To do.

Next, the next search candidate feature selection unit 17 reads the accumulation partial region feature set output from the index search unit 10 and selects a search candidate feature to be matched next, that is, whether or not there is a search candidate feature to be matched next. If there is no search candidate feature to be matched, the process is terminated. On the other hand, if it is detected that there is a search candidate feature to be matched, the search candidate feature to be matched next is selected and the process is performed. The process proceeds to step S14 (step S20).
At this time, for example, when detecting that there is a search candidate feature to be collated, the next search candidate feature selection unit 17 selects one arbitrarily from the search candidate features that have not been collated.
As a result, the next search candidate feature selection unit 17 outputs the selected search candidate feature to the feature matching unit 11.

Second Embodiment (Corresponding to Claim 3)
FIG. 6 is a block diagram showing a configuration example of a partial image search system according to a second embodiment to which the method according to claim 3 is applied.
The partial image search system according to the second embodiment has the same configuration as the partial image search system according to the first embodiment to which the method according to claim 1 is applied. As a different configuration, as shown in FIG. A search threshold update unit 18 is additionally provided, and an arbitrary target image, that is, an image to be searched as a sample, and a set of stored images, that is, a set of stored images that are a plurality of images to be searched, are input and are similar The location in the stored image of the partial image containing the image to be output is output.

The search threshold update unit 18 detects candidates within a predetermined distance from the search result candidates output from the search result candidate selection unit 15 based on the distance value output from the feature re-collation unit 13. The search threshold θ is newly set based on the detected candidate distance value.
Next, the partial image detection process in the sample feature creation unit 1 to the search threshold update unit 18 described above will be specifically described with reference to FIG. Here, FIG. 7 is a flowchart showing an operation example of the partial image search apparatus shown in FIG.

In the flowchart of FIG. 7, the “search threshold update” process of step S16 is added to the flowchart of the first embodiment shown in FIG. 2 after the “search result candidate selection” process of step S13. Since the other processes are the same, only the process of step S16 will be described.
The search threshold update unit 18 reads the search result candidate output from the search result candidate selection unit 15, selects a search result candidate having the maximum distance value from the read search result candidates, and selects the selected search result candidate. It is determined whether or not the distance value is equal to or smaller than the search threshold value θ. When it is detected that the distance value is equal to or smaller than the search threshold value θ, the search threshold value θ is updated to the same value as the distance value.
Then, the search threshold update unit 18 calculates the selection threshold θ ₂ from the search threshold θ based on the equation (5), and outputs it together with the updated search threshold θ (step S21). Thereafter, the updated search threshold θ and selection threshold θ ₂ are used for determination of the collation result.

<Third embodiment> (corresponding to claim 5)
FIG. 8 is a block diagram showing a configuration example of a partial image search system according to the third embodiment to which the method according to claim 5 is applied.
The partial image search system according to the third embodiment has the same configuration as that of the second embodiment. As a different configuration, the target representative feature extraction unit 19 is different from the partial image search system of the second embodiment. In addition, the target image, that is, a sample image to be searched and a stored image group, that is, a set of stored images that are searched images, are input, and a portion that includes an image similar to the target image The location of the image in the stored image is searched, and the search result is output.
The target representative feature extraction unit 19 classifies (clusters) the target partial region set in the set of target partial region features output from the target partial region feature extraction unit 9 and uses them as features common to the target partial region features. Extract target representative features of.

Next, partial image detection processing in the sample feature creation unit 1 to the target representative feature extraction unit 19 will be described in detail with reference to FIG. Here, FIG. 9 is a flowchart showing an operation example of the partial image search system shown in FIG.
The flowchart of FIG. 9 is different from the flowchart of the second embodiment shown in FIG. 7 in the process of “extraction of target representative feature” in step S22 after the process of “extraction of target partial area feature” in step S12. Since the other processes are the same, only the process of step S22 will be described.

The target representative feature extraction unit 19 reads a set of target partial region features output from the target partial region feature extraction unit 9, and performs clustering processing on the plurality of target partial region features.
At this time, for example, the target representative feature extraction unit 19 prepares only one cluster (extracts a common feature vector), and causes all target partial region features in the set to belong to the cluster (one target representative). Will output features).
Further, as another embodiment, the target representative feature extraction unit 19 can prepare a cluster for the number of read target partial region features and assign a target partial region feature to each cluster one by one ( A set of target representative features will be output).

Furthermore, as another example, the target representative feature extraction unit 19 extracts the positions in the target image corresponding to the target partial region features for all the read target partial region features, and divides the existence range into a grid pattern. It is also possible to adopt a configuration in which each lattice is associated with a cluster.
In addition, as another example, the target representative feature extraction unit 19 uses a cluster so that the maximum value of the distance between target partial region features in the same cluster does not exceed a predetermined classification threshold. Can also be merged.

At this time, the target representative feature extraction unit 19 uses, as an initial state of the cluster, for example, one in which target partial areas are assigned to clusters prepared for the number of read target partial area features.
Next, the target representative feature extraction unit 19 extracts a representative feature that is a partial region feature representing the cluster from each cluster. The target representative feature extraction unit 19 uses, for example, one arbitrarily selected target partial region feature in the cluster as a representative feature.

In addition, as another example, the target representative feature extraction unit 19 can use a target partial region feature corresponding to a specific position as a representative feature.
This specific position is, for example, the position of the upper left corner of the target image.
Further, as another example, the target representative feature extraction unit 19 may be configured such that the target partial feature at the center of gravity of the target partial region feature in the cluster is a representative feature.
Then, the target representative feature extraction unit 19 outputs the representative feature or a set thereof obtained by each embodiment in the third embodiment described above (step S22).
In the next step S <b> 13, the index search unit 10 performs a matching process with the index of the stored image using the target representative feature.

<Fourth embodiment> (corresponding to claim 7)
FIG. 10 is a block diagram showing a configuration example of a partial image search system according to the fourth embodiment to which the method according to claim 7 is applied.
The partial image search system according to the fourth embodiment has the same configuration as that of the third embodiment. As a different configuration, the partial image search system according to the third embodiment is different from the partial image search system of the third embodiment. 20 and a peripheral distance lower limit calculation unit 21 are added, and a target image, that is, a sample image to be searched, and a stored image group, that is, a set of stored images that are searched images, are input. A location in the stored image of a partial image including an image similar to the target image is searched, and a search result is output.

The target partial region distance calculation unit 20 calculates a distance that is the distance between the partial region features for the target partial region feature output from the target partial region feature extraction unit 9.
If the target representative feature extraction unit 15 is present, the distance between the target partial region feature and the corresponding target representative feature is calculated.
The peripheral distance lower limit calculation unit 21 uses the distance value output from the feature matching unit 11 and the numerical value of the distance between partial regions output from the target inter-region distance calculation unit 20 to perform matching in the feature matching unit 11. The lower limit value of the distance between the retrieved candidate feature and the target partial region feature output from the target partial region feature extraction unit 9 is calculated.

Next, the partial image detection processing from the sample feature creation unit 1 to the peripheral distance lower limit calculation unit 21 will be described in detail with reference to FIG. Here, FIG. 11 is a flowchart showing an operation example of the partial image search system shown in FIG.
The flowchart of FIG. 11 is different from the flowchart of the third embodiment shown in FIG. 9 in the “calculation of distance between target partial areas” in step S23 after the process of “extraction of target representative features” in step S22. A process is added, and after the process of “update search threshold” in step S21, only the process of “calculation of lower limit of peripheral distance” in step S24 is added, and the other processes are the same. Therefore, only the processing in steps S23 and S24 will be described.

The target partial region distance calculation unit 20 reads a set of target partial region features output from the target partial region feature extraction unit 9 and calculates the distance between the target partial region features.
Then, the target inter-region distance calculation unit 20 performs distance calculation for all sets of target partial region features, for example.
As another example, when the target representative feature extracting unit 19 is provided, a set of target representative features output from the target representative feature extracting unit 19 is further read, and each target partial region feature, It is also possible to calculate the distance to the corresponding target subregion feature.
As described above, the target inter-region distance calculation unit 20 outputs the distance between the target partial region features or the distance between the target partial region feature and the target representative feature.
Using the distances described above, an index search is performed in step S8.

Next, the peripheral distance lower limit value calculation unit 21 reads the distance value output from the feature matching unit 11 and the inter-subregion distance output from the target inter-region distance calculation unit 20.
Then, the peripheral distance lower limit value calculation unit 21 calculates a distance lower limit value between the accumulated partial area feature and each target partial area feature from the read distance value and the distance between the partial areas.
At this time, the peripheral distance lower limit value calculation unit 21 sets the lower limit value d (f _D ^(W) , f _Q2 ^{(W )} of the distance between the accumulated partial region feature f _D ^(W) and the target partial region feature f _Q2 ^{(W). )} ) Is obtained by the following equation (10) based on the triangular inequality. ,

In the above equation (10), d (f _D ^(W) , f _Q1 ^(W) ) is a distance value read from the feature matching unit 11, and d (f _Q1 ^(W) , f _Q2 ^(W) ) is read. This is the distance between the target partial areas.
As described above, the peripheral distance lower limit value calculation unit 21 outputs the peripheral distance lower limit value that is the lower limit value of the distance between the accumulated partial region feature and the target partial region feature.
The omission in the next step S19, the next objective partial region feature selection unit 16, and inputs the near distance limit value, the distance limit value, the collation of an object part region feature above a selected threshold theta ₂ read and, this distance limit value is any one to choose for the selection threshold theta ₂ following purposes partial region feature.

<Fifth Embodiment> (Corresponding to Claim 13)
FIG. 12 is a block diagram showing a configuration example of a partial image search system according to a fifth embodiment to which the method according to claim 13 is applied.
The partial image search system according to the fifth embodiment has the same configuration as that of the fourth embodiment. As a different configuration, the partial image search system according to the fourth embodiment is different from the partial image search system of the fourth embodiment. In addition, the target image, that is, a sample image to be searched and a stored image group, that is, a set of stored images that are searched images, are input, and a portion that includes an image similar to the target image The location of the image in the stored image is searched, and the search result is output.
The feature existence range determination unit 22 outputs, for each sample feature output from the sample feature creation unit 1, the accumulated feature extraction unit 3 and the target feature extraction unit 7 that may be assigned to the sample feature. A feature existence range that is a range in which each image feature (accumulated feature, target feature) can exist is calculated.

Next, the detection processing of the partial image in the sample feature creation unit 1 to the feature existence range determination unit 22 described above will be specifically described with reference to FIG. Here, FIG. 13 is a flowchart showing an operation example of the partial image search system shown in FIG.
The flowchart of FIG. 13 is different from the flowchart of the fourth embodiment shown in FIG. 11 between the steps of “creating sample features” in step S2 and “calculating distance between sample features” in step S3. Only the process of “determining the feature existence range” of S25 is added and the other processes are the same, so only the process of step S25 will be described.

The feature existence range determination unit 22 reads a set of sample features output from the sample feature creation unit 1, and a range in the feature space in which there is a possibility that an accumulated feature and a target feature assigned to each sample feature of this set may exist. Is set for each sample feature.
At this time, for example, when the accumulated feature quantizing unit 4 and the target feature quantizing unit 8 assign the sample feature closest to the distance below the assignment threshold, the feature existence range determining unit 22 focuses on the sample feature. Then, a hypersphere whose radius is equal to the allocation threshold is set as the feature existence range (step S25).

As another example, the feature existence range determination unit 22 uses the radius of the sample feature as the center when the accumulated feature quantization unit 4 and the index assigning unit 6 perform assignment to the sample feature with the shortest distance. May be configured to set a hypersphere equal to half the maximum value of the distance between the sample features as the feature existence range.
As described above, the feature existence range determination unit 22 outputs a set of feature existence ranges obtained by any of the methods described above.
In step S3, the distance between the sample features is calculated using the feature existence range described above.

<Experiment>
An example of the operation of the apparatus to which the present invention is applied is shown below.
Here, 1000 images with a size of 384 × 256 pixels are used as the stored image group, and 10 images with a size of 80 × 80 pixels are cut out from any location in the stored image group as the target image. It was.
In addition, the partial image was cut out with a size of 64 × 64 pixels and a margin of 16 pixels in both the vertical and horizontal directions.
That is, the number of partial areas per stored image is 273, and the number of partial areas of the target image is 289.

Since the RGB values in a small region having a size of 8 × 8 pixels are used as the features of the accumulated image and the target image, that is, the accumulated feature and the target feature, the dimension of the partial region feature is 192 dimensions.
Representative features were generated by creating a vector quantization codebook using a learning signal, and 64 images were selected and used as learning signals by selecting 10 images from the stored image group.
The index assignment unit 6 and the index search unit 10 are provided with index assignment and index search based on vector quantization as in the embodiments described in “Signal detection method, signal detection device, recording medium, and program” (Japanese Patent Application Laid-Open No. 2002-236496). And the number of clusters was set to 128.
The distance scale used was a square error, and the search threshold was 2000.

In order to confirm the effect of the present invention, the search required time was compared when the present invention was applied and when it was not applied.
As shown in the table of FIG. 14, the result of this experiment was “8.37 seconds” as the search time in the conventional example (not applied), but “3” as the search time in the present invention (applied). .17 seconds ".
As can be seen from FIG. 14, since the sample feature distance can be obtained in advance by using the sample feature, the calculation of the distance between the accumulated feature and the target feature uses the sample feature distance. Thus, the search time can be shortened.

  The program for realizing the function of the partial image search device in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Image search processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system provided with a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is a block diagram which shows the structural example of the partial image search system by 1st Example of this invention. It is a flowchart which shows the operation example of the partial image search system of FIG. It is a conceptual diagram explaining the margin of arrangement when setting a partial image in an accumulated image. It is a conceptual diagram explaining the principle when the index search part 6 selects a cluster. It is a conceptual diagram explaining the setting of the collation window in an accumulation image. It is a block diagram which shows the structural example of the partial image search system by the 2nd Example of this invention. It is a flowchart which shows the operation example of the partial image search system of FIG. It is a block diagram which shows the structural example of the partial image search system by the 3rd Example of this invention. It is a flowchart which shows the operation example of the partial image search system of FIG. It is a block diagram which shows the structural example of the partial image search system by the 4th Example of this invention. It is a flowchart which shows the operation example of the partial image search system of FIG. It is a block diagram which shows the structural example of the partial image search system by the 5th Example of this invention. It is a flowchart which shows the operation example of the partial image search system of FIG. It is a table | surface which shows the comparison of the search required time of a prior art example and this invention when searching an image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Sample feature creation part 2 ... Sample feature distance calculation part 3 ... Accumulated feature extraction part 4 ... Accumulated feature quantization part 5 ... Accumulated partial area feature extraction part 6 ... Index assignment part 7 ... Objective feature extraction part 8 ... Objective feature Quantization unit 9 ... target partial region feature extraction unit 10 ... index search unit 11 ... feature matching unit 12 ... matching result determination unit 13 ... feature re-matching unit 14 ... matching result re-determination unit 15 ... search result candidate selection unit 16 ... next Target partial region feature selection unit 17 ... Next search candidate feature selection unit 18 ... Search threshold update unit 19 ... Target representative feature extraction unit 20 ... Target partial region distance calculation unit 21 ... Peripheral distance lower limit value calculation unit 22 ... Feature existence range determination Part

Claims (17)

A partial image search method for detecting an image including a partial image similar to a target image to be searched from a stored image group registered in a database and a position of the partial image in the image,
A sample feature creation process for creating a plurality of sample features having a typical feature pattern as a sample when detecting a partial image;
A sample feature distance calculation process for calculating a distance between sample features which is a distance between each of the sample features;
An accumulated feature extraction process of selecting a plurality of small regions from each image of the accumulated image group and extracting accumulated features that are image features in each small region;
A stored feature quantization process that compares the stored feature with the sample feature and assigns a code corresponding to the sample feature to each stored feature;
A storage partial region feature extraction process for extracting a storage partial region feature that is a set of codes in each target window by arranging a predetermined size of the target window at a predetermined interval with respect to the set of codes,
An indexing process for indexing the accumulated subregion features;
A target feature extraction process of selecting a plurality of small regions from the target image and extracting features of each of the small regions as a target feature;
A target feature quantization process for assigning a code corresponding to the sample feature to the target feature by comparing the target feature with the sample feature;
A target window having a predetermined size is set in a set of codes obtained in the target feature quantization process, and target subregion features that are sets of codes in each target window are set while shifting the target window. A target partial region feature extraction process to be extracted;
An index search process for extracting accumulated subregion features similar to a set of target subregion features by the index; and
A feature matching process for calculating a distance between the accumulated partial area feature derived in the index search process and the target partial area feature derived in the target partial area feature extraction process based on the distance between the sample features;
Based on the distance, a matching result determination process for determining whether or not there is a possibility that a partial image similar to the target image exists at the location in the accumulated image group;
At a location in the stored image where it is determined that a partial image similar to the target image may exist in the verification result determination process, a verification window having the same size as the target image is set as the code of the stored feature. A feature rematching process for calculating a feature distance between the set of codes in the matching window and the set of codes of the target feature based on the distance between the sample features;
Based on the feature distance, a verification result redetermination process for determining whether or not a partial image similar to the target image exists at the location in the accumulated image group;
A search result candidate selection process in which the location in the stored image determined to have a partial image similar to the target image in the collation result redetermination process is a search result candidate;
A next target subregion feature selection process for designating a target subregion feature to be matched next from the set of target subregion features;
A partial image search method comprising: a next search candidate feature selection step of designating a next search candidate feature to be collated from the set of search candidate features. In the verification result re-determination process, the feature distance is compared with a search threshold that is a threshold corresponding to the feature distance;
It is determined whether or not a partial image similar to the target image exists at the location in the stored image,
In the index search process, the accumulated sub-region feature whose distance from any target sub-region feature derived from the target sub-region feature extraction step is less than a selection threshold determined from the search threshold is used for the index. Extract
In the verification result determination process,
The distance is compared with the selection threshold value, and it is determined whether or not there is a possibility that a partial image similar to the target image exists in the location in the accumulated image. Partial image search method. In the search result candidate selection process,
In the collation result re-determination process, a predetermined number of locations in the stored image that are determined to have a partial image similar to the target image are selected in ascending order of the distance calculated in the feature re-collation process. As search result candidates,
The search threshold value update process of detecting a certain search result candidate from the search result candidate based on the feature distance and setting a new search threshold value from the distance value. Partial image search method. In the index search process,
The search result candidate is extracted using all target partial region features derived in the target partial region feature extraction process with respect to the index. Partial image search method. A target representative feature extraction process for classifying the target partial region features and extracting representative features that are partial region features representing each classification;
4. The partial image search method according to claim 1, wherein the representative feature is used instead of the target partial region feature in the index search process. In the next target partial region feature selection process,
The target partial area feature extracted in the target partial area feature extraction process, in which it is detected that the distance from the accumulated partial area feature that has been verified in the feature verification process exceeds the selection threshold, is then verified. 6. The partial image search method according to claim 1, wherein collation is omitted without specifying as a power partial region feature. In the set of target subregion features, a distance calculation process between target subregions for calculating a distance between each target subregion feature;
Using the distance value and the distance between the partial areas, the lower limit value of the distance between the accumulated partial area collated in the feature matching process and each target partial area feature derived in the target partial area feature extraction process is calculated. And a peripheral distance lower limit calculation process,
In the next target partial region feature selection process, the distance lower limit value and the selection threshold value are compared, and a location in the accumulated signal where the distance lower limit value is lower than the selection threshold value is designated as a location to be checked next. The partial image search method according to claim 6. In the feature matching process and the feature rematching process,
The distance between the partial region feature features is an integrated value obtained by adding the distances between the small regions calculated based on the distance between the sample features. The partial image search method described. In the sample feature creation process,
The partial image search method according to any one of claims 1 to 8, wherein a sample feature is generated by learning a feature pattern that typically appears with a predetermined learning signal prepared in advance. . In the sample feature creation process,
The partial image search method according to claim 9, wherein a feature pattern that typically appears is learned using a clustering algorithm. In the accumulated feature quantization process,
A code corresponding to the sample feature having the smallest distance from the stored feature is assigned to the stored feature,
In the target feature quantization process,
The partial image search method according to any one of claims 1 to 10, wherein a code corresponding to a sample feature having the smallest distance from the target feature is assigned to the target feature. In the accumulated feature quantization process,
A code corresponding to the smallest sample feature whose distance from the accumulated feature is equal to or less than a predetermined allocation threshold is assigned to the accumulated feature, and if there is no corresponding sample feature, Assign other codes that are not supported,
In the target feature quantization process,
A code corresponding to the smallest sample feature whose distance from the target feature is equal to or less than a predetermined allocation threshold is assigned to the target feature, and if there is no corresponding sample feature, any sample feature is supported. The partial code search method according to claim 1, wherein another code that is not to be assigned is assigned. For each of the sample features, a feature existence range determination step for deriving a feature existence range that is a range in which the features derived in the accumulated feature extraction process and the target feature extraction process that can be assigned to the sample feature can exist. ,
In the process of calculating the distance between sample features,
The partial image search method according to claim 1, wherein a distance between each sample feature is calculated using the feature existence range. In the process of calculating the distance between sample features,
Based on the feature existence range, the maximum distance between the feature existence ranges corresponding to each sample feature and / or the minimum distance between the feature existence ranges is calculated, and these are newly calculated between the sample features. The partial image search method according to claim 13, wherein the partial image search method is a distance. A partial image search device for detecting an image including a partial image similar to a target image to be searched from a stored image group registered in a database and a position of the partial image in the image;
When detecting a partial image, a sample feature creation unit that creates a plurality of types of sample features having a typical feature pattern as a sample;
A distance between sample features that calculates a distance between sample features that is a distance between each of the sample features;
A storage feature extraction unit that selects a plurality of small regions from each image of the storage image group and extracts a storage feature that is a feature of the image in each small region;
An accumulated feature quantization unit that compares the accumulated feature with the sample feature and assigns a code corresponding to the sample feature to each accumulated feature;
A storage partial region feature extracting unit that extracts a storage partial region feature that is a set of codes in each target window by arranging a target window of a predetermined size at a predetermined interval with respect to the set of codes,
An indexing unit for indexing the accumulated partial region features;
A target feature extraction unit that selects a plurality of small regions from the target image and extracts features of each of the small regions as a target feature;
A target feature quantization unit that assigns a code corresponding to the sample feature to the target feature by comparing the target feature with the sample feature;
A target window of a predetermined size is set in the set of codes obtained by the target feature quantization unit, and target partial region features that are sets of codes in each target window are extracted while shifting the target window. A target partial area feature extraction unit,
An index search unit for extracting accumulated partial region features similar to a set of target partial region features by the index;
A feature matching unit that calculates the distance between the storage partial region feature derived by the index retrieval unit and the target partial region feature derived by the target partial region feature extraction unit based on the distance between the sample features;
Based on the distance, a matching result determination unit that determines whether or not there is a possibility that a partial image similar to the target image exists in the portion of the accumulated image group;
A collation window having the same size as the target image is set as the sign of the accumulated feature at a location in the accumulated image where it is determined that the partial image similar to the target image may exist by the collation result determining unit. A feature rematching unit that calculates a feature distance between the set of codes in the matching window and the set of codes of the target feature based on the distance between the sample features;
A matching result re-determination unit that determines whether or not a partial image similar to the target image exists in the portion of the accumulated image group based on the feature distance;
A search result candidate selection unit that sets a location in the stored image determined to have a partial image similar to the target image by the matching result re-determination unit;
A next target partial region feature selection unit for designating a target partial region feature to be matched next from the set of target partial region features;
A partial image search system comprising: a next search candidate feature selection unit for designating a next search candidate feature to be collated from the set of search candidate features. A program for causing a computer to execute a partial image search process for detecting an image including a partial image similar to a target image to be searched from a stored image group registered in a database and a position of the partial image in the image,
Sample feature creation processing for creating a plurality of types of sample features having a typical feature pattern as a sample when detecting a partial image;
A sample feature distance calculation process for calculating a distance between sample features which is a distance between the sample features;
A storage feature extraction process for selecting a plurality of small regions from each image of the storage image group and extracting a storage feature that is a feature of the image in each small region;
A stored feature quantization process that compares the stored feature with the sample feature and assigns a code corresponding to the sample feature to each stored feature;
A storage partial region feature extraction process for extracting a storage partial region feature that is a set of codes in each target window by arranging a predetermined size of the target window at a predetermined interval with respect to the set of codes,
An indexing process for indexing the accumulated partial region features;
A target feature extraction process for selecting a plurality of small regions from the target image and extracting the features of each of the small regions as a target feature;
A target feature quantization process for assigning a code corresponding to the sample feature to the target feature by comparing the target feature with the sample feature;
A target window having a predetermined size is set in a set of codes obtained by the target feature quantization process, and target subregion features that are sets of codes in each target window are set while shifting the target window. A target partial region feature extraction process to be extracted;
An index search process for extracting stored partial area features similar to a set of target partial area features by the index; and
A feature matching process for calculating the distance between the storage partial area feature derived in the index search process and the target partial area feature derived in the target partial area feature extraction process based on the distance between the sample features;
Based on the distance, a matching result determination process for determining whether or not there is a possibility that a partial image similar to the target image exists in the location in the accumulated image group;
At a location in the stored image where it is determined that there is a possibility that a partial image similar to the target image exists in the verification result determination process, a verification window having the same size as the target image is set as the code of the stored feature. A feature rematching process for calculating a feature distance between a set of codes in the matching window and a set of codes of the target feature based on the distance between the sample features;
Based on the feature distance, a collation result re-determination process for determining whether or not a partial image similar to the target image exists in the location in the accumulated image group;
In the collation result re-determination process, a search result candidate selection process in which a location in the accumulated image determined to have a partial image similar to the target image is a search result candidate;
A next target partial region feature selection process for designating a target partial region feature to be matched next from the set of target partial region features;
A program for causing a computer to execute a partial image search process including: a next search candidate feature selection process for designating a next search candidate feature to be collated from a set of search candidate features. The computer-readable recording medium which recorded the partial image search program of Claim 16.

Images