JP2016139355A - Image search device, image search method and image search program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a calculation amount when searching similar feature areas between images.SOLUTION: A storage part 2 stores a bit string by which characteristics of a plurality of feature areas respectively configured in a first image and a second image are represented. An operation unit 3 identifies a specific bit position from bit positions of the bit string where number of "1" bits is more than half the number of all feature areas and reverses the value at the specific bit positions in the bit strings of all feature areas of respective images so as to generate conversion bit strings. The operation unit 3 performs search processing in which similar feature area similar to respective feature areas of the first image is searched among the feature areas of the second image on the basis of the Hamming distance of the conversion bit string between the feature areas. In the search processing, the operation unit 3 restricts the feature areas of the second image to be a calculation object of the Hamming distance for respective feature areas of the first image to the feature areas where the norm of the conversion bit string is included within a fixed range from a norm of the conversion bit string for respective feature areas of the first image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image search device, an image search method, and an image search program.

  In recent years, image matching technology has been widely used in various fields. As an example of a matching method between images, a feature amount at a feature point of the first image (hereinafter referred to as “local feature amount”) is compared with a local feature amount at a feature point of the second image, and the first image A method of searching for feature points (hereinafter referred to as “corresponding points”) of the second image corresponding to the feature points is used. By statistically processing the set of corresponding points found by the search, the presence of the first image and the position of the first image in the second image can be recognized.

  In addition, there is a method of expressing a local feature amount used for searching for corresponding points as described above in binary code. A typical example is BRIEF (Binary Robust Independent Elementary Features). BRIEF is expressed by a local feature amount based on a luminance difference between pixels calculated for each of a plurality of pixel pairs set around a feature point. For example, a set of bit values corresponding to the sign (positive / negative) of the luminance difference is used as the local feature amount. As described above, the method of expressing the local feature amount by the binary code has an advantage that the similarity between the feature points can be calculated by high-speed calculation based on the Hamming distance.

  On the other hand, a method called “NOM (Norm Ordering Matching)” that sorts feature values in order of norm and limits the search range to only those with close norms is proposed as a method for performing the nearest neighbor search of multidimensional feature values at high speed. Has been.

  As an example of an image coding technique based on vector quantization, vectors are rearranged in the norm order in the codebook created for each bitmap pattern of the edge portion, and the norm of the vector is used when performing vector quantization matching processing. An image encoding method has been proposed in which only the vicinity of the image is searched.

Japanese Patent Laid-Open No. 11-8848

M. Calonder, V. Lepetit, C. Strecha, and P. Fua., "BRIEF: Binary Robust Independent Elementary Features", In Proceedings of the European Conference on Computer Vision (ECCV), 2010 Mohamed Yousef and Khaled F. Hussain, "Fast exhaustive-search equivalent pattern matching through norm ordering" Journal of Visual Communication and Image Representation, vol. 24, no. 5, pp. 592? 601, 2013

  Here, consider the case where the above-described NOM is applied to image matching processing using local feature amounts represented by binary codes such as BRIEF. In this case, the local feature amounts of the first image and the second image are classified for each norm. However, for any image, the norm of the local feature amount is near the median of the range that the norm can take (for example, When the local feature amount is 128 bits, there is a tendency that feature points having a norm of “64”) are extremely increased. For this reason, between the first image and the second image, the number of combinations of feature points where the norm of the local feature amount is near the median value increases, and the calculation amount of the Hamming distance between these feature points increases. End up. As a result, there is a problem that the effect of improving the calculation efficiency is low in spite of the application of NOM.

  In one aspect, an object of the present invention is to provide an image search device, an image search method, and an image search program that can reduce the amount of calculation when searching for a similar feature region between images.

  In one proposal, the following image processing apparatus is provided. The image processing apparatus includes a storage unit and a calculation unit. The storage unit stores a bit string indicating features of a plurality of feature regions set in each of the first image and the second image. The calculation unit specifies a specific bit position from which the number of predetermined values set is greater than or equal to a predetermined threshold value greater than half of the total number of all feature regions in the first image and the second image, from the bit positions of the bit string. . In addition, the calculation unit inverts the values of the specific bit positions in the bit strings of all the feature areas in the first image and the second image, thereby generating converted bit strings for all these feature areas. In addition, the arithmetic unit executes a search process for searching for a similar feature region similar to each feature region of the first image based on the Hamming distance of the converted bit string between the feature regions from the feature region of the second image. . In this search process, the calculation unit calculates the feature area of the second image for which the Hamming distance is to be calculated for each feature area of the first image, and the norm of the conversion bit string for each feature area of the first image is the norm of the conversion bit string. To a feature region included in a certain range.

In one proposal, an image search method is provided in which processing similar to that of the image processing apparatus is executed.
Furthermore, in one proposal, there is provided an image search program that causes a computer to execute processing similar to that of the image processing apparatus.

  In one aspect, the amount of calculation when searching for similar feature regions between images can be reduced.

1 is a diagram illustrating a configuration example and a processing example of an image processing apparatus according to a first embodiment. It is a figure which shows the hardware structural example of the image processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the 1st comparative example of an image search process. It is a figure which shows the structural example of a pixel pair management table. It is a figure which shows the example of the process which calculates a local feature-value. It is a figure for demonstrating a voting process. It is a figure for demonstrating the determination process of the similar image based on a vote result. It is a flowchart which shows the 2nd comparative example of an image search process. It is a figure which shows the structural example of a feature-value management table. It is a figure which shows the search process example of the corresponding point in a 2nd comparative example. It is a figure which shows the example of the histogram of a norm. It is a figure which shows the example of the bit inversion process of a local feature-value. It is a figure which shows the example of the change of norm distribution by a bit inversion process. It is a block diagram which shows the structural example of the processing function with which an image processing apparatus is provided. It is a flowchart which shows the example of a feature-value calculation process. It is a flowchart (the 1) which shows the example of an image search process. It is a flowchart (the 2) which shows the example of an image search process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating a configuration example and a processing example of the image processing apparatus according to the first embodiment. An image processing apparatus 1 illustrated in FIG. 1 includes a storage unit 2 and a calculation unit 3. The storage unit 2 is realized as a storage device such as a RAM (Random Access Memory) and an HDD (Hard Disk Drive), for example. The calculation unit is realized as a processor, for example.

  The storage unit 2 stores a bit string indicating features of a plurality of feature regions set in each of the first image and the second image. Note that the feature quantities that can be represented by a bit string in this way include, for example, BREF, ORB (Oriented Fast and Rotated BRIEF), and CARD (Compact And Real-time Descriptors).

  In the example of FIG. 1, the storage unit 2 stores a feature amount 10a of the first image and a feature amount 20a of the second image. The feature quantity 10a of the first image includes a bit string for each feature area of the first image, and the feature quantity 20a of the second image includes a bit string for each feature area of the second image.

  The calculation unit 3 identifies a similar feature region similar to each feature region of the first image from the feature regions of the second image. For this specification, the calculation unit 3 executes the following process while referring to the bit string stored in the storage unit 2.

  The computing unit 3 identifies a specific bit position where the number of predetermined values set is equal to or greater than a predetermined threshold from the bit positions of the bit string. The predetermined value is 1 or 0. Hereinafter, the predetermined value is 1 in the first embodiment. Further, the threshold value is set to a value larger than ½ of the total number of all feature regions in the first image and the second image.

  The calculation unit 3 inverts the value of the specific bit position in the bit strings of all feature regions in the first image and the second image. As a result, conversion bit strings corresponding to all of these feature regions are generated.

  In the example of FIG. 1, it is assumed that the value of 1 exceeds half of the whole in the second bit from the top among the bit positions of all the bit strings of the first image and the second image. In this case, the arithmetic unit 3 inverts the value of the second bit in all the bit strings (step S1). Thereby, each bit string is converted into a converted bit string. In FIG. 1, the feature amount 10 b of the first image includes converted bit sequences corresponding to the bit sequences included in the feature amount 10 a of the first image. The feature amount 20b of the second image includes each converted bit string corresponding to each bit string included in the feature amount 20a of the second image.

  Next, the calculation unit 3 searches for a similar feature region similar to each feature region of the first image based on the Hamming distance of the converted bit string between the feature regions from the feature region of the second image. ”Is executed. For example, the calculation unit 3 calculates a Hamming distance between a converted bit string of a certain feature area of the first image and each converted bit string of one or more feature areas included in the second image. And the calculating part 3 specifies a similar feature area | region based on the calculation result of Hamming distance.

  In this search processing, the calculation unit 3 uses the feature region of the second image to be calculated for the Hamming distance for each feature region of the first image, and the norm of the transform bit sequence for the feature region of the first image. The feature area is limited to a certain range from the norm. Here, the norm of the converted bit string indicates the number of 1 included in the converted bit string.

  For example, in FIG. 1, it is assumed that the norm of the converted bit string 11 for a certain feature region (hereinafter referred to as “target feature region”) of the first image is 2. The calculation unit 3 calculates a hamming distance between the converted bit string 11 and a converted bit string corresponding to one or more feature areas of the second image, so that the similarity similar to the target feature area among the feature areas of the second image is calculated. Search for feature regions. At this time, the calculation unit 3 limits the feature region of the second image that is the calculation target of the Hamming distance to a feature region in which the norm of the converted bit string is included in a certain range from 2.

  As an example, this fixed range is set to a range of plus or minus 1, and in the example of FIG. 1, among the converted bit strings corresponding to each feature area of the second image, only the converted bit strings 21 and 22 include norms 1 to 3. And In this case, the arithmetic unit 3 does not calculate the Hamming distance between the converted bit string 11 and all the converted bit strings of the second image, but the Hamming distance between the converted bit string 11 and the converted bit string 21, and the converted bit string 11 and the converted bit string. Only the Hamming distance with 22 is calculated (step S2).

  Here, the Hamming distance between bit strings indicates the number of bits having different values between the bit strings. On the other hand, the norm of the bit string indicates the number of 1 included in the bit string. For this reason, there is a high possibility that the Hamming distance becomes small between bit strings having close norms because the number of 1s included in each bit string is close. On the other hand, bit strings having different norms have a high possibility of increasing the Hamming distance because the number of 1s included in each bit string is different. Therefore, even when the calculation target of the Hamming distance is limited according to the norm as in the above search processing, the possibility that the search accuracy of the similar feature region based on the Hamming distance is low is low. That is, according to the above search processing, the number of times of Hamming distance calculation can be reduced and the time required for the entire processing can be shortened while maintaining the search accuracy of similar feature regions.

  However, the norm has a property that the distribution of norm values tends to be extremely concentrated near the median of the range that the norm can take. From this property, for example, when the above search processing is executed using the bit sequence before the bit inversion processing, the norm of the bit sequence is the center between the first image and the second image. The number of combinations of feature regions near the value increases. For this reason, the number of times of Hamming distance calculation by those combinations increases. In this case, although the calculation target of the Hamming distance is limited according to the norm, the effect of improving the calculation efficiency is low.

  For such a problem, the image processing apparatus 1 according to the first embodiment performs a search process using the converted bit string subjected to the bit inversion process as described above. In the bit inversion processing, a bit position where the number of 1 exceeds at least half of the whole is specified as the specific bit position. Then, the values of specific bit positions in the bit strings of all the feature regions in the first image and the second image are inverted, and a converted bit string is generated.

  By such bit inversion processing, the value of 1 at the specific bit position is decreased, and as a result, the value of 1 in the entire converted bit string is decreased. As a result, the norm distribution based on the converted bit string is distributed in a smaller area from the median value of the norm, and the peak value of the frequency of the norm is reduced. Therefore, in the search process, the number of combinations of feature regions in which the norm of the bit string takes the vicinity of the median value is reduced, and the number of times of Hamming distance calculation by those combinations is reduced.

  Here, by bit inversion processing, the number of combinations of feature regions increases in a range other than the vicinity of the median value of the norm. However, because the norm distribution is dispersed by bit inversion processing, the norm histogram based on the transformed bit string has a frequency increase in each norm with an increased frequency than a frequency decrease in each norm with a decreased frequency. Tends to be larger. In addition, such a change in norm distribution occurs in both the first image and the second image. For this reason, as a whole, there is a high possibility that the number of Hamming distance calculations will be greatly reduced.

  Therefore, according to the first embodiment, it is possible to reduce the amount of calculation in the process of searching for a similar feature region similar to each feature region in the first image from the feature region in the second image. Processing time is shortened and processing efficiency is improved.

  Further, even if the values of the same bit position are inverted for the bit strings corresponding to all the feature regions of the first image and the second image, the calculation result of the Hamming distance using the converted bit string after the bit inversion is the bit inversion This is the same as the calculation result of the Hamming distance using the previous bit string. For this reason, even when the search process is performed using the converted bit string as described above, the accuracy of specifying the similar feature region does not change.

  In the first embodiment described above, the predetermined value counted when specifying the specific bit position is set to 1, but this value can also be set to 0. In this case, the value of 0 in the specific bit position is reduced by the bit inversion processing as described above, and as a result, the value of 0 in all the converted bit strings is reduced from that before the bit inversion processing. As a result, the norm distribution based on the converted bit string is dispersed in a larger region from the median value of the norm, and the peak value of the norm frequency is reduced. Therefore, the same effect as when the predetermined value is 1 can be obtained.

[Second Embodiment]
Next, as a second embodiment, an image processing apparatus that selects a key image from a plurality of captured images and searches a captured image of a scene similar to the key image from captured images other than the key image will be described. In the second embodiment, BRIEF is used as the image feature amount, but other types of binary feature amounts such as ORB and CARD can also be used.

  FIG. 2 is a diagram illustrating a hardware configuration example of the image processing apparatus according to the second embodiment. The image processing apparatus 100 according to the second embodiment is realized as a computer as shown in FIG. 2, for example.

  The entire image processing apparatus 100 is controlled by a processor 101. The processor 101 may be a multiprocessor. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). The processor 101 may be a combination of two or more elements among CPU, MPU, DSP, ASIC, and PLD.

A RAM 102 and a plurality of peripheral devices are connected to the processor 101 via a bus 108.
The RAM 102 is used as a main storage device of the image processing apparatus 100. The RAM 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 101. The RAM 102 stores various data necessary for processing by the processor 101.

  Peripheral devices connected to the bus 108 include an HDD 103, a graphic processing device 104, an input interface 105, a reading device 106, and a communication interface 107.

  The HDD 103 is used as an auxiliary storage device of the image processing apparatus 100. The HDD 103 stores an OS program, application programs, and various data. As the auxiliary storage device, other types of nonvolatile storage devices such as SSD (Solid State Drive) can be used.

  A display device 104 a is connected to the graphic processing device 104. The graphic processing device 104 displays an image on the display device 104a in accordance with an instruction from the processor 101. Examples of the display device include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

  An input device 105 a is connected to the input interface 105. The input interface 105 transmits a signal output from the input device 105a to the processor 101. Examples of the input device 105a include a keyboard and a pointing device. Examples of pointing devices include a mouse, a touch panel, a tablet, a touch pad, and a trackball.

  A portable recording medium 106 a is detached from the reading device 106. The reading device 106 reads the data recorded on the portable recording medium 106 a and transmits it to the processor 101. Examples of the portable recording medium 106a include an optical disk, a magneto-optical disk, and a semiconductor memory.

The communication interface 107 transmits / receives data to / from other devices via the network 107a.
With the hardware configuration described above, the processing functions of the image processing apparatus 100 can be realized.

  Incidentally, a plurality of captured image data are stored in the storage device (for example, the HDD 103) of the image processing apparatus 100. These captured images are images captured by the imaging device. These captured image data may be stored in the storage device of the image processing apparatus 100 using, for example, the portable recording medium 106a, or stored in the storage device of the image processing apparatus 100 via the network 107a. May be.

  In the image processing apparatus 100, the following processing is performed by executing the photo management software. A key image is selected from a plurality of captured images in the storage device by a user input operation. Then, the image processing apparatus 100 extracts a captured image of a scene similar to the key image from a captured image excluding the key image (hereinafter referred to as “target image”) among the plurality of captured images in the storage device. For example, a target image estimated to include the same target object as the target object included in the key image is extracted as a captured image of a scene similar to the key image. Thus, for example, the user can search for an image necessary as a material from the image processing apparatus 100, or can collect and automatically organize photos at the same event. Therefore, convenience and entertainment can be provided to the user.

  Such an image processing device 100 is realized as a terminal device operated by a user such as a personal computer or a smartphone. Further, the image processing apparatus 100 may be realized as a server apparatus on a network. In this case, the captured image data is uploaded from the user terminal device to the image processing apparatus 100 via the network, for example.

  Note that the image search function of the image processing apparatus 100 can be used for managing document content such as presentation materials in addition to management of captured images as described above. For example, data of a plurality of documents is stored in the storage device of the image processing apparatus 100, and a key document is selected from these. For example, the image processing apparatus 100 can extract a document including a sentence that looks similar to a key document when displaying the document from other documents, or the same image, table, graph, or the like as the key document. It is also possible to extract a document including “” from other documents. Thereby, the work time for searching for a document can be reduced. In addition, the reuse of past document assets is promoted, and the efficiency of operations can be improved.

Next, a comparative example of image search processing will be described, and problems in the comparative example will be described. Then, details of the image search process in the second embodiment will be described.
FIG. 3 is a flowchart illustrating a first comparative example of the image search process. In the comparative example shown in FIG. 3, the local feature value of BREF is calculated in steps S101 and S102 of FIG.

  [Step S101] The image processing apparatus sets a plurality of feature points on each captured image. Here, as an example, Dense Sampling that sets feature points at equal intervals (for example, at intervals of 24 pixels) on the captured image is used.

[Step S102] The image processing apparatus calculates a local feature amount for each feature point of each captured image.
BRIEF is calculated as a local feature amount for each fixed region (hereinafter referred to as “feature region”) centered on each feature point. The feature region is, for example, a rectangular region of 48 pixels around the feature point. In addition, a plurality of pixel pairs are set in advance in the feature region. The local feature amount of a certain feature point is calculated as a bit string configured by combining the signs of the luminance difference of each pixel pair in the corresponding feature region.

  Here, FIG. 4 is a diagram illustrating a configuration example of the pixel pair management table. The coordinates of each pixel constituting the pixel pair are registered in advance in the pixel pair management table 112. As shown in FIG. 4, in the pixel pair management table 112, an ID for identifying a pixel pair and the coordinates of the first pixel and the second pixel constituting the pixel pair are registered. Pixel pairs are set at random, for example. The pixel pair information registered in the pixel pair management table 112 is commonly applied to all feature regions.

  FIG. 5 is a diagram illustrating an example of processing for calculating a local feature amount. FIG. 5 shows an example of processing for registering the local feature amount of each feature point in the captured image 200 in the feature amount management table 131. Note that the feature amount management table 131 is created for each captured image.

  For example, the local feature amount of the feature point 201 set in the captured image 200 is calculated as follows. The image processing apparatus calculates the luminance difference of each pixel pair for the feature region 202 corresponding to the feature point 201 (step S102a). The luminance difference between the pixel pairs can be obtained by subtracting the luminance value of the second pixel from the luminance value of the first pixel in the pixel pair management table 112, for example.

  The image processing apparatus generates the bit string 203 by combining the bit values corresponding to the calculated sign of the luminance difference (step S102b). For example, in the pixel pair order, the image processing apparatus adds a bit value “1” to the bit string when the luminance difference is a positive value and a bit value “0” when the luminance difference is 0 or less. When M pixel pairs are set as shown in FIG. 4, an M-bit bit string is generated. The image processing apparatus registers the generated bit string 203 in the feature amount management table 131 as a local feature amount of the feature point 201 (step S102c).

In this way, the local feature amount for each feature point set in the captured image 200 is registered in the feature amount management table 131 corresponding to the captured image 200.
Hereinafter, the description will be returned to FIG.

[Step S103] The image processing apparatus selects a key image from the captured image in response to a user operation input.
[Step S104] The image processing apparatus selects one of the captured images (target images) other than the key image.

[Step S105] The image processing apparatus selects one feature point of the key image.
[Step S106] The image processing apparatus searches for a feature point (corresponding point) similar to the feature point selected from the key image in Step S105 from the target image selected in Step S104. In this processing, the image processing apparatus calculates the Hamming distance between the local feature amount of the feature point selected from the key image and the local feature amount of each feature point of the target image, and the Hamming distance among the feature points of the target image is The smallest feature point is extracted as the corresponding point having the highest similarity.

  [Step S107] When the key image is overlaid on the target image so that the feature point selected in Step S105 matches the corresponding point searched in Step S106, the image processing apparatus performs the center position of the key image in the target image. Is estimated. The image processing device votes for the pixel at the estimated center position among the pixels of the target image. In practice, the image processing apparatus may vote for each pixel included in a predetermined area (for example, a rectangular area of 10 pixels square) centered on the estimated center position, for example.

  [Step S108] The image processing apparatus determines whether all feature points of the key image have been processed. If there is a feature point that has not been processed, the process of step S105 is executed. If all feature points have been processed, the process of step S109 is executed.

  [Step S109] When the maximum number of votes for each pixel of the target image selected in Step S104 exceeds a predetermined threshold, the image processing apparatus determines that the target image is an image similar to the key image. On the other hand, when the maximum number of votes is equal to or less than the threshold value, the image processing apparatus determines that the target image is an image that is not similar to the key image.

  [Step S110] The image processing apparatus determines whether all target images have been processed. If there is a target image that has not been processed, the process of step S104 is executed. On the other hand, if all the target images have been processed, the image processing apparatus outputs the identification information of the target image determined to be similar to the key image in step S109, and ends the image search process.

  Here, FIG. 6 is a diagram for explaining the voting process. FIG. 6 shows an example of processing for searching for corresponding points of the target image 210 similar to the feature points 201 of the key image 200a. For example, the image processing apparatus searches for a corresponding point by calculating a Hamming distance between the local feature amount of the feature point 201 of the key image 200a and the local feature amount of each feature point of the target image 210 (step S106a). .

  It is assumed that the feature point 211 of the target image 210 is extracted as a corresponding point similar to the feature point 201 of the key image 200a. At this time, the image processing apparatus determines the center position 204 of the key image 200a in the target image 210 when the key image 200a is superimposed on the target image 210 so that the feature point 201 and the feature point 211 (corresponding point) match. Estimate (step S107a).

Here, the number of pixels of the horizontal width and height of the target image is wi and hi, respectively, and the number of pixels of the horizontal width and height of the key image are wr and hr, respectively. If the feature point (xi, yi) of the target image is searched as the corresponding point of the target image corresponding to the feature point (xr, yr) of the key image, the position (xv, yv) of the center point of the key image in the target image Is calculated using the following equations (1-1) and (1-2).
xv = xi · xr + (wr / 2) (1-1)
yv = yi · yr + (hr / 2) (1-2)
If the pixel 214 is estimated as the center position of the key image 200a in the target image 210 based on the correspondence relationship between the feature point 201 and the feature point 211 in FIG. Vote for 214. For this voting process, for example, a voting map 114 having an entry corresponding to each pixel of the target image 210 is used. The initial value of each entry in the voting map 114 is 0. In the process of FIG. 6, 1 is added to the entry corresponding to the pixel 214 in the voting map 114 (step S107b).

  In practice, for example, the image processing apparatus may vote for each pixel included in a predetermined area (for example, a rectangular area of 10 pixels square) with the pixel 214 as the center. This makes it possible to perform a certain robust recognition process for the difference between the key image 200a and the target image 210.

  FIG. 7 is a diagram for explaining a similar image determination process based on a vote result. The voting map 114a shown in FIG. 7 shows the state of the voting map 114 after the process shown in FIG. 6 is executed for each feature point of the key image 200a. The image processing apparatus extracts the maximum value of the number of votes for each pixel in the voting map 114a, and determines whether this maximum value exceeds a predetermined threshold.

  Here, when the same object is shown in the key image 200a and the target image 210, the positional relationship between the feature points of the key image 200a and the corresponding points of the target image 210 may be the same between the feature points of the key image. Many. In this case, the number of votes concentrates on entries corresponding to the same pixel in the vote map 114a. On the other hand, when the relevance between the key image 200a and the target image 210 is low, the positional relationship between the feature points of the key image 200a and the corresponding points of the target image 210 is often different between the feature points of the key image. In this case, the number of votes is distributed in the vote map 114a.

  Therefore, when the maximum number of votes in the voting map 114a exceeds the threshold, it is estimated that the number of votes is concentrated on the same pixel, so the same object appears in the key image 200a and the target image 210. It can be judged that there is a high possibility. From this, the image processing apparatus determines that the target image 210 is an image similar to the key image 200a when the maximum value of the number of votes exceeds the threshold value.

  In practice, since the maximum number of votes is affected by the number of feature points in the target image 210, for example, a normalization process such as dividing the number of votes by the number of feature points in the target image 210 is performed. It is desirable that a comparison with a threshold value is performed.

  By the way, in the first comparative example, there is a problem that the calculation time of the Hamming distance between the feature points in step S106 in FIG. 3 is enormous. This is because the Hamming distance is calculated for combinations of all feature points in the key image and all feature points in the target image. For example, when the number of feature points of each image is 1000, the Hamming distance is calculated 1000000 times. Such a Hamming distance calculation time may occupy 80% or more of the time required for the entire image search process. If this calculation time can be shortened, the time required for the entire image search process can be greatly reduced.

  Therefore, in the following second comparative example, the time required for the Hamming distance calculation is shortened by using the above-described NOM. Specifically, the local feature amounts of the feature points of the key image and the target image are classified by the norm of each local feature amount. And the search range of the corresponding point with respect to one feature point of the key image is limited to only one having a close norm. As a result, the number of combinations of feature points when calculating the Hamming distance is reduced, and the calculation time is shortened.

  FIG. 8 is a flowchart illustrating a second comparative example of the image search process. The process of FIG. 8 is a modification of the process of the second comparative example shown in FIG. 3 as follows. In the process of FIG. 8, step S121 is executed between step S102 and step S103 of FIG. Further, step S122 is executed instead of step S106 in FIG. Hereinafter, only steps S121 and S122 will be described, and description of processing steps in which the same processing as that in FIG. 3 is executed will be omitted.

  [Step S121] The image processing apparatus performs the following processing for each captured image. The image processing device calculates the norm of the local feature amount for each feature point in the captured image. The norm of the bit string of binary values is calculated as the number of 1 included in the bit string. The image processing apparatus rearranges the feature points in the captured image in ascending order of the local feature norm.

  [Step S122] From the feature point of the target image selected in Step S104, the image processing apparatus includes a corresponding norm value within a certain range centered on the norm corresponding to the feature point selected in Step S105 in the key image. Identify the feature points The fixed range is, for example, a range of plus or minus 1. The image processing apparatus searches for the corresponding points by limiting the search range of the corresponding points with respect to the feature points of the key image to the specified feature points. That is, the image processing apparatus calculates the Hamming distance between the local feature amount of the feature point of the key image and the local feature amount of each feature point specified from the target image. Then, the image processing apparatus extracts a feature point having the smallest calculated Hamming distance as a corresponding point.

  FIG. 9 is a diagram illustrating a configuration example of the feature amount management table. In the second comparative example, for example, a feature amount management table 113 as shown in FIG. 9 is used. The feature amount management table 113 is prepared for each captured image.

  A record is registered in the feature amount management table 113 for each feature point in the captured image. In each record, an ID, a feature point coordinate, a local feature amount, and a norm are registered. ID indicates an identification number for identifying a feature point in the captured image. The feature point coordinates indicate the coordinates of the feature points. In the item of the local feature amount, a bit string indicating the local feature amount of the feature point is registered. The norm calculated from the local feature is registered in the norm item.

  In step S102 of FIG. 8, the calculated local feature value is registered in the corresponding record in the corresponding feature value management table 113. In step S121, for example, the records in the feature quantity management table 113 are rearranged according to the calculated norm size.

  FIG. 10 is a diagram illustrating an example of corresponding point search processing in the second comparative example. FIG. 10 illustrates a feature quantity management table 113a in which local feature quantities of each feature point in the key image are registered, and a feature quantity management table 113b in which local feature quantities of each feature point in the target image are registered. ing.

  The image processing apparatus calculates the norm of each local feature value registered in the feature value management table 113a. Then, the image processing apparatus sorts the local feature amounts registered in the feature amount management table 113a, for example, in ascending order of the norm value. Similarly, the image processing apparatus calculates the norm of each local feature amount registered in the feature amount management table 113b. Then, the image processing apparatus rearranges the local feature amounts registered in the feature amount management table 113b, for example, in ascending order of the norm value.

  Next, the image processing device searches for a feature region of the target image similar to each feature region of the key image by calculating a Hamming distance between each local feature amount of the key image and each local feature amount of the target image. . At this time, the search range of the feature region of the target image with respect to each feature region of the key image is such that the norm value of the local feature amount of the feature region of the target image is the norm of the local feature amount of the feature region of the key image. Limited to close feature regions. In other words, the local feature quantity selected from the key image and the local feature quantity to be calculated for the Hamming distance are calculated from the local feature quantities of the target image, and the calculated norm value is selected from the key image. It is limited to the local feature amount included in a certain range centered on the norm of the feature amount.

  For example, in FIG. 10, it is assumed that the norm calculated from the local feature amount 251 for the feature region of the key image is 3. Here, assuming that the norm range for determining the search range is plus or minus 1, the calculation target for which the Hamming distance is calculated with respect to the local feature value 251 of the key image is the norm among the local feature values of the target image. The local feature amount is limited to 2 to 4.

  As described above, the calculation target of the Hamming distance is limited according to the norm, so that the Hamming distance is calculated for a combination of each local feature amount of the key image and all the local feature amounts of the target image. Thus, the calculation amount of the Hamming distance can be reduced.

  Further, the Hamming distance between bit strings indicates the number of bits having different values between the bit strings. On the other hand, the norm of the bit string indicates the number of 1 included in the bit string. For this reason, there is a high possibility that the Hamming distance becomes small between bit strings having close norms because the number of 1s included in each bit string is close. On the other hand, bit strings having different norms have a high possibility of increasing the Hamming distance because the number of 1s included in each bit string is different. Therefore, even when the calculation target of the Hamming distance is limited according to the norm as described above, the possibility that the similarity determination accuracy of the feature region based on the Hamming distance is low is low.

Next, problems in the second comparative example will be described.
The norm value of the local feature amount is an integer from 0 to the number of dimensions (maximum value) of the local feature amount. For example, when the local feature amount is expressed as a bit string of 128 bits, the norm can take a value from 0 to 128. Further, the norm value tends to be concentrated and distributed in the vicinity of the median of the range that the norm can take, as in the example of FIG.

  FIG. 11 is a diagram illustrating an example of a norm histogram. For example, when the norm of each local feature amount in a certain captured image is calculated, the number of appearances of the norm value is distributed as shown in FIG. As shown in FIG. 11, the number of occurrences of the norm value is often extremely concentrated around the median value of the norm range (“64” in the example of FIG. 11).

  This is due to the following reason. In a bit string having a small norm value, the number of 0s is greater than the number of 1s. In a bit string having a large norm value, the number of 1s is greater than the number of 0s. On the other hand, in the bit string in which the norm value is near the median value, the number of 1s is almost equal to the number of 0s. In this case, the number of bit string patterns that can be generated by the combination of 1 and 0 is larger than the number of bit string patterns that can be generated when the numbers of 1 and 0 differ greatly. For this reason, the number of bit strings having a norm value near the median value is larger than that of a bit string having a relatively small norm value or a large bit string.

  Usually, the trend of norm distribution as described above is observed for both the key image and the target image. For this reason, even when the calculation target of the Hamming distance is limited according to the norm as in the second comparative example, between the key image and the target image, the local feature amounts whose norm is near the median value The number of combinations increases, and the number of times of Hamming distance calculation by these combinations increases.

  The number of times of Hamming distance calculation is calculated by multiplying the number of local feature quantities having a close norm in both the key image and the target image. For this reason, when the number of local feature values having a certain norm value is doubled, the number of combinations of local feature values having the norm value is quadrupled. In this way, when the norm values are concentrated and distributed near the median of the norm range, the combination of local features whose norm is near the median increases exponentially, and the Hamming distance of those combinations increases. The number of calculations is enormous. As a result, there is a problem that the effect of improving the calculation efficiency is low although the calculation target of the Hamming distance is limited according to the norm.

  As described above, the number of bit string patterns in which the norm value is near the median value is large. For this reason, a combination of local feature values having a large Hamming distance may potentially be included in a combination of local feature values having a norm value near the median value. This is contrary to the purpose of limiting the calculation target of the Hamming distance to those having a small Hamming distance. In that sense, it can be said that the process of calculating the Hamming distance by combining the local feature amounts whose norm is near the median is wasteful.

  With respect to such a problem, in the second embodiment, the calculation result of the Hamming distance does not change even if the bit values at the same position are inverted in all the local feature quantities that are the targets of the Hamming distance calculation. Using the property, the process of the second comparative example is modified as follows. Based on the above properties, the image processing apparatus 100 according to the second embodiment is included in the local feature amount by inverting the bit values at appropriate positions in all the local feature amounts used for the Hamming distance calculation in advance. Decrease the number of 1 bits. As a result, the norm distribution is changed so as to be dispersed in a smaller direction from the median value of the norm, and the degree of concentration of the norm distribution around the median value of the norm is reduced. As a result, the number of combinations of local feature values whose norm takes the vicinity of the median value is reduced, and the number of Hamming distances calculated by the combination is reduced.

  FIG. 12 is a diagram illustrating an example of local feature amount bit inversion processing. FIG. 12 shows an example of the feature amount management tables 113_1, 113_2,..., 113_N corresponding to the first to Nth captured images. In the feature quantity management tables 113_1, 113_2,..., 113_N, the value of the local feature quantity is represented for each bit for easy understanding.

  The image processing apparatus 100 counts the number of 1 for each bit for all local feature values in all captured images. In the example of FIG. 12, a count value of 1 for each bit is registered in the aggregation table 115. The image processing apparatus 100 identifies a bit whose number exceeds one-half of the total number of local feature values (total number of feature points), and inverts the bit value of the specified bit in all local feature values. In the example of FIG. 12, it is assumed that the second bit from the top is specified as a bit whose number exceeds one-half of the number of all local feature values. In this case, the image processing apparatus 100 inverts the bit value of the second highest bit in all local feature values.

  In the local feature amount subjected to such bit inversion, the number of 0 is increased as compared with that before the bit inversion. Therefore, the norm distribution of these local feature amounts is dispersed in a smaller direction from the median value of the norm, and the peak value of the norm frequency is also reduced, compared to before the bit inversion.

  FIG. 13 is a diagram illustrating an example of a change in the norm distribution by the bit inversion process. Note that FIG. 13 shows an example in which the number of bits and the number of feature points of the local feature amount are small for easy understanding.

  The graph 221a shows an example of a norm histogram based on a local feature amount of a certain key image. The graph 222a shows an example of a norm histogram based on the local feature amount of a certain target image. In each of the graphs 221a and 222a, the distribution is concentrated near the median value of the norm. When a similar feature region is searched between the key image and the target image, the number of Hamming distance calculations is 4 × 5 + 8 × 9 + 6 × 4 = 116 (times).

  On the other hand, graphs 221b and 222b show examples of histograms after the bit inversion processing is performed on all the local feature amounts of the key image and the target image in the above procedure. That is, the graph 221b shows an example of the norm histogram based on the local feature after bit inversion processing for the key image, and the graph 222b shows the norm histogram on the key image based on the local feature after bit inversion processing. An example is shown.

  In the graph 221b, compared to the graph 221a, the number of local feature values having a norm having a median value is greatly reduced from 8 to 5, and the norm is distributed and distributed in an area smaller than the median. . Also in the graph 222b, compared to the graph 222a, the number of local feature values having a norm having a median value is greatly reduced from 9 to 6, and the norm is distributed and distributed in an area smaller than the median. .

  Thus, the number of times of Hamming distance calculation using the local feature after bit inversion is 1 × 0 + 2 × 2 + 3 × 3 + 3 × 4 + 5 × 6 + 4 × 3 = 67 (times), which is significantly reduced from before bit inversion. . That is, bit inversion reduces the number of combinations of local feature quantities, thereby reducing the number of Hamming distance calculations. Therefore, the time required for calculating the Hamming distance is shortened and the calculation efficiency is improved.

  As shown in FIGS. 13 and 14, the image processing apparatus 100 according to the present embodiment counts the number of 1 for each bit, but counts the number of 0 instead of the number of 1. May be. In this case, the image processing apparatus 100 specifies bits whose number of 0 exceeds 1/2 of the total number of local feature values (total number of feature points), and inverts the bit values of the specified bits in all local feature values. The norm based on the local feature amount after the bit inversion is performed in this manner is distributed and distributed in a region larger than the median value. As a result, the number of Hamming distance calculations when the local feature after bit inversion is used is reduced in the same way as when the number of 1 is counted.

  FIG. 14 is a block diagram illustrating a configuration example of processing functions included in the image processing apparatus. The image processing apparatus 100 includes a storage unit 110, an image acquisition unit 121, a feature amount calculation unit 122, a feature amount change unit 123, and an image recognition unit 124.

  The storage unit 110 is implemented as a storage device (for example, a storage area of the RAM 102 or the HDD 103) included in the image processing apparatus 100. The storage unit 110 stores the image data 111, the pixel pair management table 112, and the feature amount management table 113. The image data 111 indicates captured image data, and the coordinates of the first pixel and the second pixel constituting each pixel pair are registered in the pixel pair management table 112 as shown in FIG. The feature amount management table 113 is prepared for each captured image, and the feature amount management table 113 includes an ID, a feature point coordinate, and a local feature corresponding to each feature point in the captured image, as shown in FIG. Quantity and norm are registered.

In addition, the voting map 114 shown in FIG. 6 and the tabulation table 115 shown in FIG.
The processing of the image acquisition unit 121, the feature amount calculation unit 122, the feature amount change unit 123, and the image recognition unit 124 is realized, for example, by executing a predetermined program on the processor 101.

  The image acquisition unit 121 acquires the image data 111 of the captured image and stores it in the storage unit 110. For example, the image acquisition unit 121 acquires the image data 111 of the captured image via the portable recording medium 106a or the network 107a.

  The feature amount calculation unit 122 calculates a local feature amount for each feature point in the captured image while referring to the image data 111 and the pixel pair management table 112, and the feature amount management table 113 corresponding to the calculated local feature amount. Register with.

  The feature amount changing unit 123 counts the number of 1 for each bit with respect to the local feature amounts corresponding to all feature points of all captured images, and identifies the bit whose number exceeds 1/2 of the total number of feature points. The feature amount changing unit 123 inverts the bit value of the specified bit in all the local feature amounts. Furthermore, the feature amount changing unit 123 calculates the norm of each local feature amount after the bit inversion processing, and rearranges the local feature amounts in ascending order of the norm for each captured image.

The image recognition unit 124 receives a key image selection operation and searches for a similar image similar to the key image from captured images other than the selected key image.
Next, processing of the image processing apparatus 100 will be described using a flowchart.

FIG. 15 is a flowchart illustrating an example of a feature amount calculation process.
[Step S11] The feature amount calculation unit 122 sets a plurality of feature points on each captured image. For example, Dense Sampling is used in which feature points are set on a captured image at regular intervals (for example, 24 pixel intervals). The feature amount calculation unit 122 creates a record for each set feature point in the feature amount management table 113 corresponding to each captured image, and registers an ID and feature point coordinates in each created record.

[Step S12] The feature amount calculation unit 122 selects one captured image.
[Step S13] The feature amount calculation unit 122 selects one feature point from the captured image selected in step S12.

  [Step S14] The feature amount calculation unit 122 calculates a luminance difference for each pixel pair based on the pixel pair management table 112 in a certain range of feature regions centered on the feature point selected in step S13. The luminance difference is calculated by subtracting the luminance value of the second pixel from the luminance value of the first pixel among the pixels constituting the pixel pair.

  [Step S15] The feature amount calculation unit 122 adds a value corresponding to the calculated sign of the luminance difference of each pixel pair to the bit string in the order of the pixel pair. For example, when the luminance difference is a positive value, a bit value “1” is added, and when the luminance difference is 0 or less, a bit value “0” is added. Thereby, a bit string indicating the local feature amount corresponding to the feature point selected in step S13 is calculated. The feature amount calculation unit 122 registers the calculated bit string in a corresponding record in the feature amount management table 113.

  [Step S16] The feature quantity calculation unit 122 determines whether all feature points in the captured image have been processed. If there is a feature point that has not been processed, the process returns to step S13, and another feature point is selected. On the other hand, when all the feature points have been processed, the process of step S17 is executed.

  [Step S17] The feature amount calculation unit 122 determines whether all captured images have been processed. If there is a captured image that has not been processed, the process returns to step S12, and another captured image is selected. On the other hand, if all captured images have been processed, the processing in FIG. 15 is terminated.

With the processing in FIG. 15 described above, local feature amounts corresponding to each feature point are registered in the feature amount management table 113 corresponding to each captured image.
Note that the processing in FIG. 15 may be executed in an apparatus different from the image processing apparatus 100 in which the processing in FIGS. 16 and 17 is executed. In this case, the image processing apparatus 100 acquires the contents of the feature amount management table 113 from the apparatus that has executed the process of FIG.

16 and 17 are flowcharts showing an example of the image search process.
First, in steps S21 to S26, bit inversion processing of local feature values is executed.
[Step S21] The feature amount changing unit 123 initializes a variable b indicating a bit position in the bit string to 0.

  [Step S22] Let L be the total number of feature points of all captured images. The feature amount changing unit 123 refers to the b-th bit from the top among the bits of the local feature amount corresponding to each of the L feature points. The feature amount changing unit 123 calculates the number S (b) of 1 set in the b-th bit from the top in all the local feature amounts.

  [Step S23] The feature amount changing unit 123 determines whether the calculated number S (b) of 1 is larger than L / 2. When S (b) is greater than L / 2, the process of step S24 is executed, and when S (b) is equal to or less than L / 2, the process of step S25 is executed. Note that the determination threshold value in step S23 may be a value larger than L / 2.

[Step S24] The feature quantity changing unit 123 inverts the b-th bit in all the L local feature quantities in the feature quantity management table 113.
[Step S25] The feature amount changing unit 123 increments the variable b by 1.

  [Step S26] The feature amount changing unit 123 determines whether or not the value of the variable b matches the number of bits (the number of pixel pairs in the feature region) M of the local feature amount. If the value of the variable b is smaller than the number of bits, that is, if there are unprocessed bits, the process of step S22 is executed. On the other hand, when the value of the variable b matches the number of bits, that is, when all the bits have been processed, the process of step S27 is executed.

  Next, in steps S27 to S30, the local feature quantity rearrangement process according to the norm is executed while referring to the feature quantity management table 113 subjected to the bit inversion in step S24.

[Step S27] The feature amount changing unit 123 selects one captured image.
[Step S28] The feature amount changing unit 123 calculates the norm of the local feature amount for each feature point in the selected captured image. The feature amount changing unit 123 registers the calculated norm in the feature amount management table 113 corresponding to the selected captured image.

  [Step S29] The feature amount changing unit 123 rearranges the feature points included in the selected captured image in order of the calculated norm size. Here, it is assumed that the feature points are rearranged in ascending order of norm. Here, it is assumed that the feature amount changing unit 123 rearranges the records of the feature amount management table 113 corresponding to the selected captured image in ascending order of the calculated norm.

  [Step S30] The feature amount changing unit 123 determines whether all captured images have been processed. If there is a captured image that has not been processed, the process returns to step S27, and another captured image is selected. On the other hand, if all captured images have been processed, the process of step S31 in FIG. 17 is executed.

  Next, in steps S31 to S40, processing for specifying a similar image similar to the key image from captured images other than the key image is performed while referring to the feature amount management table 113 in which the records are rearranged in step S29. Is done.

[Step S31] The image recognition unit 124 receives a selection input operation of a key image from the user.
[Step S32] The image recognition unit 124 selects one target image from among captured images (target images) other than the selected key image.

  [Step S33] The image recognition unit 124 selects one feature point of the key image. In this step S33, specifically, a record corresponding to one feature point is selected in order from the top of the feature amount management table 113 corresponding to the key image.

  [Step S34] The image recognizing unit 124 specifies a norm range to be calculated for the Hamming distance. Specifically, the image recognition unit 124 acquires the norm value from the record of the feature point selected in step S33. Here, the acquired norm value is n. The image recognizing unit 124 sets the range from n−d to n + d as the norm range for which the Hamming distance is calculated. Note that d is an integer greater than or equal to 0, for example, 1.

  [Step S35] The image recognizing unit 124 selects, from the records in the feature amount management table 113 corresponding to the target image, records whose registered norms are values from nd to n + d one by one. The image recognition unit 124 calculates the Hamming distance between the local feature amount in the record selected from the feature amount management table 113 corresponding to the target image and the local feature amount in the record selected in step S33. The image recognition unit 124 extracts the feature point corresponding to the record with the smallest Hamming distance among the records selected from the feature amount management table 113 corresponding to the target image as the corresponding point with the highest similarity.

  [Step S36] The image recognizing unit 124 overlays the key image on the target image so that the feature point selected in step S33 matches the corresponding point extracted in step S35. Estimate the position of the center point. In this process, the position of the center point is calculated using the above-described equations (1-1) and (1-2).

  The image recognition unit 124 votes for the pixel corresponding to the calculated center point position among the pixels of the target image. For example, the image recognition unit 124 increments the number of votes of an entry corresponding to the calculated position of the center point among the entries of the voting map 114 mapping each pixel of the target image by one. Note that the voting destination pixel may be not only the pixel corresponding to the position of the center point but also each pixel within a certain range centered on the pixel.

  [Step S37] The image recognition unit 124 determines whether all feature points in the key image have been processed. If there is a feature point that has not been processed, the process returns to step S33, and another feature point is selected. On the other hand, if all feature points have been processed, the process of step S38 is executed.

  [Step S38] The image recognition unit 124 determines whether the maximum number of votes for each pixel of the target image exceeds a predetermined threshold. The image recognition unit 124 determines that the target image is a similar image when the maximum number of votes exceeds a threshold, and determines that the target image is not a similar image when the maximum value of votes is equal to or less than the threshold. To do.

In step S38, the image recognition unit 124 can also calculate the similarity between the key image and the target image based on the number of votes, for example.
[Step S39] The image recognition unit 124 determines whether all target images have been processed. If there is a target image that has not been processed, the process returns to step S32 and another target image is selected. On the other hand, if all the target images have been processed, the process of step S40 is executed.

  [Step S40] The image recognition unit 124 outputs a search result of similar images. For example, the image recognition unit 124 displays the file name and thumbnail image of the similar image searched for on the screen.

  In the second embodiment described above, the bit values of the bits in which the number of 1 exceeds half of the total number of feature points are inverted for all the local feature amounts used for the Hamming distance calculation. As a result, the number of 1s in the local feature bit string decreases and the number of 0s increases. In this way, the distribution of norms of local features that have undergone bit inversion is smaller than the median of the median of the range that the norm can take, compared to the case where no bit inversion is applied. Disperse to range. As a result, when the search range of the corresponding points of the feature points of the key image is limited to the feature points having a near norm among the feature points of the target image, the number of combinations of feature points having a norm near the median decreases, The number of Hamming distance calculations decreases accordingly.

  Here, by bit inversion processing, the number of combinations of feature points increases in a range other than the vicinity of the median value of the norm. However, when bit inversion is not performed, the degree of concentration near the median value of the norm is extremely high. The increase in frequency at each norm in which the frequency of feature points has increased is more likely to be larger than the decrease in frequency in. Moreover, such a change in norm distribution occurs in both the key image and the target image. For this reason, as a whole, there is a high possibility that the number of Hamming distance calculations will be greatly reduced. Therefore, according to the second embodiment, the time required for the process for determining whether the target image is similar to the key image is reduced, and the processing efficiency is improved.

  Further, the processing efficiency is further improved by performing a bit inversion process on all captured images used for the search process in advance, and then searching for similar images similar to the key image from other captured images. Can do.

  For example, when the number of captured images is small, bit inversion processing is performed between the local feature amount of one key image and the local feature amount of one target image as follows. Also good. The image processing apparatus 100 specifies bits whose number of 1 (or 0) exceeds 1/2 of the total number of feature points in the key image based on each local feature amount of the key image. In addition, the image processing apparatus 100 specifies bits whose number of 1 (or 0) exceeds 1/2 of the total number of feature points in the target image based on each local feature amount of the target image. Then, the image processing apparatus 100 identifies a bit in which the number of 1 (or 0) exceeds 1/2 of the total number of feature points in each image in both the key image and the target image, and all local features of both images. Inverts the value of the specified bit in the quantity. As a result, for both the key image and the target image, the norm distribution can be reliably distributed in the direction of the area smaller (or larger) than its median value, and the number of Hamming distance calculations can be reliably reduced. It becomes possible to do.

  The bit inversion processing between the local feature amount of one key image and the local feature amount of one target image as described above is effective when the number of captured images is small. However, as the number of captured images increases, for each combination of images, the processing load of specifying and inverting bits where the number of 1 (or 0) exceeds 1/2 of the total number of feature points in each image Becomes relatively large, and processing efficiency decreases. For this reason, as the number of captured images increases, the processing efficiency increases when the bit inversion processing is performed on local feature points in all captured images at once as in the processing of FIGS.

  The processing functions of the apparatuses (image processing apparatuses 1 and 100) described in the above embodiments can be realized by a computer. In that case, a program describing the processing contents of the functions that each device should have is provided, and the processing functions are realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic storage device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Optical disks include DVD (Digital Versatile Disc), DVD-RAM, CD-ROM (Compact Disc-Read Only Memory), CD-R (Recordable) / RW (ReWritable), and the like. Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, a portable recording medium such as a DVD or a CD-ROM in which the program is recorded is sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing according to the received program.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Memory | storage part 3 Operation part 10a, 10b The feature-value of 1st image 11, 21, 22, 22 Conversion bit string 20a, 20b The feature-value of 2nd image

Claims (5)

A storage unit for storing a bit string indicating features of a plurality of feature regions set in each of the first image and the second image;
An arithmetic unit;
Have
The computing unit is
Specifying a specific bit position from which the number of predetermined values set is greater than or equal to a predetermined threshold greater than half of the total number of all the feature regions in the first image and the second image from the bit positions of the bit string;
Inverting the value of the specific bit position in the bit string of all the feature areas in the first image and the second image to generate a conversion bit string for all the feature areas,
Performing a search process for searching for a similar feature region similar to each feature region of the first image based on a Hamming distance of the converted bit string between the feature regions in the feature region of the second image; In the processing, the feature area of the second image that is the object of calculation of the Hamming distance for each feature area of the first image is determined from the norm of the transform bit string for each feature area of the first image. Limited to feature regions within a certain range,
Image processing device. The storage unit stores the bit string indicating features of a plurality of feature regions set in each of the plurality of second images,
The computing unit is
In specifying the specific bit position, the number of the predetermined value set is equal to or greater than a predetermined determination threshold value that is greater than ½ of the total number of all feature regions in the first image and the plurality of second images. Specifying the position as the specific bit position;
In the generation of the converted bit sequence, the converted bit sequence for all the feature regions is obtained by inverting the values of the specific bit positions in the bit sequences of all the feature regions in the first image and the plurality of second images. Generate
In the search process, the similar feature region similar to each feature region of the first image is searched from the feature regions in each of the plurality of second images.
The image processing apparatus according to claim 1. The calculation unit further includes:
For each feature region of the first image, the position of the first image included in the second image is specified based on a positional relationship between the feature region of the first image and the corresponding similar feature region;
Outputting information based on the similarity between the first image and the second image based on the result of specifying the position;
The image processing apparatus according to claim 1. An image processing apparatus capable of acquiring the bit string from a storage unit that stores bit strings indicating features of a plurality of feature regions set in each of the first image and the second image,
Specifying a specific bit position from which the number of predetermined values set is greater than or equal to a predetermined threshold greater than half of the total number of all the feature regions in the first image and the second image from the bit positions of the bit string;
Inverting the value of the specific bit position in the bit string of all the feature areas in the first image and the second image to generate a conversion bit string for all the feature areas,
Performing a search process for searching for a similar feature region similar to each feature region of the first image based on a Hamming distance of the converted bit string between the feature regions in the feature region of the second image; In the processing, the feature area of the second image that is the object of calculation of the Hamming distance for each feature area of the first image is determined from the norm of the transform bit string for each feature area of the first image. Limited to feature regions within a certain range,
Image search method. A computer capable of acquiring the bit string from a storage unit that stores bit strings indicating features of a plurality of feature regions set in each of the first image and the second image;
Specifying a specific bit position from which the number of predetermined values set is greater than or equal to a predetermined threshold greater than half of the total number of all the feature regions in the first image and the second image from the bit positions of the bit string;
Inverting the value of the specific bit position in the bit string of all the feature areas in the first image and the second image to generate a conversion bit string for all the feature areas,
Performing a search process for searching for a similar feature region similar to each feature region of the first image based on a Hamming distance of the converted bit string between the feature regions in the feature region of the second image; In the processing, the feature area of the second image that is the object of calculation of the Hamming distance for each feature area of the first image is determined from the norm of the transform bit string for each feature area of the first image. Limited to feature regions within a certain range,
An image search program for executing processing.

Images