JP5500404B1 - Image processing apparatus and program thereof - Google Patents

Abstract

A local feature vector having scale invariance and rotation invariance is obtained at a lower calculation cost.
Four feature points in order of decreasing distance from a first feature point are used as second feature points, and pair feature points of the first feature point and each of the second feature points are selected (S1), A distance L between the feature points is calculated, and first and second sampling circles having a radius proportional to the distance L around the first and second feature points are determined (S3), and 16 pixels on each circumference are determined. The average luminances I (Pi), I (Qi), i = 0 to f of the rectangular pixel regions centered on each of the first and second pixels are sequentially sampled with respect to the direction of the straight line L. The difference from the average luminance of the rectangular pixel area centered on the second feature point is arranged in this order as a component of the local feature vector (S4).
[Selection] Figure 2

Description

  The present invention relates to an image processing apparatus having a function for obtaining a scale-invariant and rotation-invariant feature vector of a local region centered on a natural feature point on a luminance image, and a program thereof.

  Markerless AR (Augmented Reality) has become feasible with smartphones as the performance of smartphones equipped with cameras improves and image processing technologies such as FAST (Features From Accelerated Segment) corner detection methods improve.

  According to the FAST corner detection method, a large number of natural feature points in one image can be detected at high speed. By matching these feature points with the feature points in the reference data obtained in advance, the camera parameters for projecting the three-dimensional coordinates onto the two-dimensional coordinates of the camera image can be estimated, and based on these parameters, the 3D model can be estimated. An AR image projected onto the camera image (a CG image superimposed on the camera image) can be generated. In order to perform this matching, it is necessary to describe a local feature vector centered on each feature point.

  Patent Document 1 below discloses a local feature vector calculation method capable of calculating a local feature vector having scale invariance and rotation invariance without depending on contrast.

  Further, Non-Patent Document 1 described below describes that the method called BRIEF (Binary Robust Independent Elementary Features) of Non-Patent Document 2 described below is the fastest as a result of the test.

JP 2012-38290 A

Feature descriptor comparison report: http://computer-vision-talks.com/2011/08/feature-descriptor-comparison-report/ BRIEF: http: //cvlab.epfl.ch/~lepetit/papers/calonder_pami11.pdf Random Forest: http://link.springer.com/article/10.1023%2FA%3A1010933404324?LI=true

  However, the processing for obtaining local feature vectors having scale invariance and rotation invariance has a relatively high calculation cost. In particular, in order to have scale invariance, it is necessary to perform image processing on each of a plurality of image scales. As a result, the calculation cost increases.

More specifically, in Patent Document 1, a concentric circle detection unit that detects pixel data on the circumference of p circles centering on a feature point, and the gradient angle and dominant gradient of the pixel value in each pixel data A weighted difference value calculation unit that calculates a weighted difference value obtained by multiplying the difference value by the square root of the radius of the circle, and a frequency distribution generation unit that generates a frequency distribution having q classes of the weighted difference values And a descriptor vector calculation unit for calculating a p × q-dimensional descriptor vector from a q-dimensional vector having each frequency for each circle as a component, it is necessary to provide a SIFT (Scale Invariant Feature Transform), Although the calculation cost can be reduced as compared with the SURF faster than this, the calculation cost becomes relatively large.

  Further, in BRIEF of Non-Patent Document 2, since the binarized luminance difference of two pixels in a circle centered on a feature point is used as a component of the local feature vector, the number of dimensions of the local feature vector, the bit length, However, in terms of speeding up, it is not sufficient for the following reasons. That is, for example, when generating a 128-dimensional local feature vector, BRIEF can represent a local feature vector with 128 bits, but for example, random sampling of 128 pixels within the circle in an image of 640 × 480 pixels is required. This is because the number of times of caching increases and the processing becomes heavy. Also, BRIEF has no rotation invariance. Furthermore, since the above binarization makes it difficult to distinguish between local feature vectors that are close to each other, the accuracy and stability of matching between the local feature vector acquired from the camera image and the reference local feature vector, that is, the feature point distinguishability is improved. descend.

  Furthermore, when searching for information about images such as characters and symbols in a database, for example, the conventional method has a relatively low discriminability by local feature vectors between feature points, so the image recognition rate is low.

  In view of such problems, an object of the present invention is to provide an image processing apparatus having a configuration for obtaining a local feature vector having scale invariance and rotation invariance at a lower calculation cost, and a program thereof. .

  Another object of the present invention is to provide an image processing apparatus having a configuration for further improving the discriminability of a local feature vector generated from an image and a program thereof.

In a first aspect of the present invention, a processor and a storage device in which data and a program are stored are provided, the data includes a grayscale image, and the program stores a plurality of local feature amounts included in the data with respect to the processor. In an image processing apparatus including a feature vector generation program to be generated,
The feature vector generation program provides the processor with
(A) detecting the coordinates of feature points that are corner points in the grayscale image;
(B) selecting a pair feature point of each of a predetermined number of second feature points in order closer to the first feature point which is each detected feature point and the first feature point;
(C) For each pair feature point, the distance L between the first feature point and the second feature point is obtained,
(D) Pixel regions Pi, i each including n pixels (n ≧ 4) at equal pixel intervals among the pixels on the circumference of the first radius proportional to the distance L with the first feature point as the center. The average first luminance I (Pi) of = 0 to n−1 is sampled in a predetermined order with respect to the line direction of the distance L, and the difference between each and the luminance of the pixel region including the first feature point;
A pixel region Qi, i = 0 to 0 including each of m pixels (m ≧ 4) at equal pixel intervals among the pixels on the circumference of the second radius that is centered on the second feature point and proportional to the distance L The average second luminance I (Qj) of m−1 is sampled in a predetermined order with respect to the line direction of the distance L, and the difference between each and the luminance of the pixel region including the second feature point;
To obtain a normalized local feature vector with
The square root of the number of pixels in the pixel region is substantially proportional to the distance L.

Here, the grayscale image is, for example, a single-color component image of a grayscale image or a color image, and may be a grayscale image of each of one or more channels of R, G, and B channels of an RGB image. The corner point is detected by, for example, a FAST corner detection method or a corner detection method using a Harris operator. The average luminance is a concept including cumulative added luminance. In addition, the local feature vector may be expressed in a predetermined order with respect to the line direction of the distance L, and may be one in which the former luminance difference component and the latter luminance difference component are alternately arranged. .

  In the second aspect of the image processing apparatus according to the present invention, in the first aspect, m and n are each 8, 16, or 32.

  According to the configuration of the first aspect, since the pair feature points are selected and the local feature vector is obtained as described above, the local feature vector having scale invariance and rotation invariance can be obtained at a lower calculation cost than in the past. There is an effect that can be done.

  In addition, since local feature vectors are generated based on the feature points of the image, the discriminability of the local feature vectors is improved even in the case of frame images such as characters and symbols, and as a result, the discriminability of the frame images is improved. There is an effect that it becomes possible.

  According to the configuration of the second aspect, since m and n are both powers of 2, as described in the second embodiment, there is an effect that a local feature vector can be obtained at higher speed.

  Other objects, characteristic configurations and effects of the present invention will become apparent from the following description read in connection with the appended claims and the drawings.

It is a schematic block diagram which shows the hardware constitutions of the image processing apparatus 10 which concerns on Example 1 of this invention. It is a flowchart which shows the process sequence which produces | generates the local feature vector in 1 frame image. (A) is a figure which shows what connected the character image and the feature point of the pair feature point regarding each feature point on it with the straight line on it, (B) is the pair which expanded a part of (A) It is feature point explanatory drawing. 3 is a process explanatory diagram of step S3 of FIG. 2, and is a local feature vector generation process explanatory diagram in which a part of FIG. 3A is enlarged. FIG. It is processing explanatory drawing of step S4 of FIG. (A) to (D) are local area images each showing a feature point pair sharing the first feature point, and (E) to (H) are the features of (A) to (D), respectively. It is a component display figure by the bar graph of the local feature vector regarding a point pair. FIG. 6 is a schematic functional block diagram of an image processing apparatus according to a second embodiment. 8 is a schematic flowchart of a main routine executed by a main processing unit 40 in FIG. It is a schematic flowchart which shows the class ID estimation process with respect to one local feature vector V in the local feature vector memory | storage part M3 in the matching process part 46 in FIG. (A) is explanatory drawing of the local feature vector regarding the same pair feature point labeled with class ID and frame image ID, (B) is the part extracted at random from the whole set of the local feature vector in reference data It is explanatory drawing which shows the discriminator of the random forest which consists of a tree for every set | correspondence with the input / output. FIG. 8 is a diagram including a photographed image of a printed matter including a swan photograph and a character string “Swan” that includes a visualization of the intermediate result obtained by performing the processing of FIG. 7, and the feature points extracted from the image It is a figure which shows a pair and the straight line which connected between the feature points of each pair. FIG. 8 is a diagram including the image obtained by performing the process of FIG. 7 on the printed image including the swan photograph and the character string “Swan”, and visualizing the result of the process, on the reference image obtained by reducing and rotating the image. FIG. 12 is a diagram in which feature points and feature points on the image in FIG. 11 that are recognition targets are matched by a matching unit, and the matched feature points are connected by a straight line. FIG. 8 is a diagram including a photographed image of a printed matter including a photograph of a swan and a character string “Swan”, including a visualization of the result of the processing shown in FIG. 7, the image being reduced, rotated, and projectively converted. FIG. 12 is a diagram in which feature points on the image and feature points on the image in FIG. 11 to be recognized are matched by a matching unit, and the matched feature points are connected by a straight line. FIG. 9 is a diagram including a photographed image of a printed matter including a photograph of a swan and a character string “Swan” that includes a visualization of the result of the process shown in FIG. 7, and the image is rotated and compared with the case of FIG. 12. FIG. 12 is a diagram in which feature points on the reduced reference image and feature points on the image in FIG. 11 to be recognized are matched by a matching unit, and the matched feature points are connected by a straight line.

  FIG. 1 is a schematic block diagram showing a hardware configuration of an image processing apparatus 10 according to the first embodiment of the present invention, and shows only components necessary for the first embodiment. The image processing apparatus 10 is, for example, a mobile terminal device such as a smartphone or PDA provided with a camera, a notebook personal computer, or a desktop personal computer.

  In the main body 20 of the image processing apparatus 10, a processor 21 is coupled to a storage device 23, an input interface 24, a camera interface 25, and a display interface 26 via a bus 22. The processor 21 includes an internal cache memory. An input device 30 is coupled to the input interface 24, a camera 31 is coupled to the camera interface 25, a display device 32 as an output device is coupled to the display interface 26, and a communication unit 27 as another output device is coupled to the display unit 26. An antenna 33 is coupled.

  The input device 30 is an interactive input device and includes a touch panel, a pointing device, a keyboard, or a combination thereof. The communication unit 27 includes an interface for coupling to an external monitor or the Internet via radio waves.

  The storage device 23 stores a program and data. The program accepts a user instruction or setting value selection or input from the input device 30 via the input interface 24 to the processor 21, and according to this input, The application is activated, and the camera 31 captures a subject, for example, a cover or signboard of a library book, and stores the frame image (still image) in the storage device 23, and generates a plurality of local feature vectors from the frame image. Then, based on these and the reference data in the storage device 23, the frame image is identified, and information on the frame image, for example, related book information stored in the library or detailed information on the signboard is stored. Read from device 23 and display on display device 32 via display interface 26 . Alternatively, the product in the store or the mail order catalog is imaged by the camera 31, and information on the product is displayed on the display device 32 in the same manner.

  The feature of the first embodiment is processing for generating a local feature vector in one frame image shown in FIG. In the following, the step identification codes in the figure are shown in parentheses.

  (S0) A feature point is detected by a FAST corner detection method while raster scanning the target pixel in one frame image.

In this FAST corner detection method, if the pixel of interest is at the center and the positive threshold is th, for example, the luminance value of 16 pixels on the circumference of a radius of 3 pixels is smaller than (the luminance value of the pixel of interest) −th. If it is dark, it is bright if it is larger than (the luminance value of the pixel of interest) + th, and if it is between these values, it is ternarized as similar, and when it is determined that, for example, 9 pixels or more are bright or dark continuously, It is determined that the corner is a feature point.

  (S1) Thereafter, the process up to S5 is performed for each feature point (target feature point) detected in step S0.

  (S2) With respect to the feature point of interest (first feature point), a predetermined number n of feature points in order of decreasing distance from the feature point are defined as second feature points, and the first feature point and each of the second feature points n pairs of feature points are selected. n is n ≧ 1, and is a common value for each first feature point.

  In FIG. 3A, n = 4 is set for each feature point, and the first feature point and each second feature point are connected by a straight line (a pair is connected). FIG. 4 is a partially enlarged explanatory view of FIG.

  The processing of step S3 and step S4 is performed for each pair feature point obtained in step S2.

  (S3) The distance L between the feature points of the pair feature points is calculated. For example, the distance L between the first feature point 350 and the second feature point 351 is calculated as shown in FIG. A first sampling circle 352 having a radius proportional to the center distance L and a second sampling circle 353 having a radius proportional to the distance L centering on the second feature point 351 are determined.

  The proportionality constant in FIG. 4 is 1, which is common for each feature point. Different proportional constants for the radii of the first sampling circle 352 and the second sampling circle 353 may be used.

  FIG. 5 shows a first sampling point C1 and a second sampling point C2, which are different from those in FIG.

(S4) Among the pixels on the first sampling circle C1, each of the rectangular pixel regions P0 to P9 and Pa to Pf centered at N pixels with equal pixel intervals (N ≧ 4), for example, 16 pixels, respectively. The average luminance I (Pi), i = 0 to f is deviated from the first feature point 36 toward the second feature point 37 in a predetermined order with reference to the direction vector (or the direction of the straight line L), for example, the direction vector as a reference. Sampling is performed in the clockwise direction. In FIG. 5, the average luminances I (P1), I (P2),..., I (Pf), I (P0) are sampled in this order. Differences from the average luminance I1 of rectangular pixel areas (areas indicated by hatching) centered at 36 are arranged in this order. Similarly, a direction vector (or a straight line L of the straight line L) from the second feature point 37 to the first feature point 36 is arranged. Direction) as a standard For example, sampling is performed in the counterclockwise direction with reference to this direction vector. In FIG. 5, average luminances I (Q9), I (Qa),..., I (Qf), I (Q0),. Q8) is sampled in this order, and the difference between each of these and the average luminance I2 of the rectangular pixel area (area indicated by hatching) centered on the second feature point 37 is arranged in this order and normalized. As a local feature vector for 36 of the pair feature points. That is, this local feature vector V is
V = α (I (P1) -I1, I (P2) -I1,..., I (Pf) -I1, I (P0) -I1, I (Q9), I (Qa) -I2,. ., I (Qf), I (Q0), ..., I (Q8))
Asking. α is a coefficient for normalizing the norm value of the feature vector V to 127 (norm square is 16129) which is the maximum value of a signed 8-bit integer, for example. Also, the sign of each component may be reversed from the above, or may be reversed only with respect to the second sampling circle C2.

  Each of the rectangular pixel areas is a square area, and the length of one side thereof is approximately proportional to the distance L. Here, “approximately proportional” means that a quantization error is included.

  Since all the feature points in one frame are processed in steps S1 to S5, the local feature vector of the above-described second feature point 37 and the first feature point is also calculated.

  The local feature vector thus obtained is constant even if the optical axis and position of the camera 31 are fixed and the camera 31 is rotated around the optical axis, and the camera 31 is slid in this optical axis direction. But it is unchanged. That is, this local feature vector has scale invariance and rotation invariance.

  6 (A) to 6 (D) are local region images that show different feature point pairs sharing the first feature point with dots, and (E) to (H) are (A) to (H), respectively. It is a component display figure by the bar graph of the local feature vector regarding the feature point pair of D).

  The k-th component of the local feature vector V before normalization is V [k], the luminance I (Pi) is R [i], the luminance of the first feature point 36 is I1, and the luminance array element at the calculation start position is R [O] (in the case of FIG. 5, o = 1), and 0x is added before the hexadecimal number, in the case of C language, the component related to the first sampling circle C1 of the vector V is as follows. It can be calculated by loop processing.

for (i = 0; i <16; i ++) {V [i] = S [(i + o) & 0x1f] −I1};
Here, & is a logical product operator. In general, assuming that the mode operator is%, when n is a power of 2, i = (j + o)% n can be calculated by i = (j + o) & (n−1). As in the processing, the index i can be calculated at high speed using the AND operator & without using an extra conditional jump instruction that jumps depending on whether i = n or not in order to determine the value of i.

The components related to the second sampling circle C2 of the vector V are the same as described above.
The local feature vector V is not easily affected by changes in illumination because each component is a difference in luminance value. In addition, since the norm of the local feature vector V is normalized, it is less susceptible to changes in illumination. Further, since each component of the local feature vector V uses an average luminance value of the pixel area (the vector V is normalized later, this may be a cumulative addition value), so the SN of the local feature vector V The ratio can be made relatively large.

  The normalized local feature vector V as described above is used in the following second embodiment.

  FIG. 7 is a schematic functional block diagram of an image processing apparatus according to the second embodiment using the method according to the first embodiment. The hardware configuration of this image processing apparatus is the same as that shown in FIG.

  In FIG. 7, rounded rectangular blocks Mi and buffer areas M0 to M5 are part of the data area in the storage device 23 of FIG.

  The main processing unit 40 is a main routine that performs image processing on the frame image and its luminance image. FIG. 8 is a schematic flowchart showing processing by the main processing unit 40. In steps S4i, S41, S43, and S45 to S48, blocks 4i, 41, 43, and 45 to 48 in FIG. To do.

  In FIG. 7, the image input unit 4i, the buffer region Mi, the gray scale unit 41, the buffer region M0, the feature point detection unit 43, the two-dimensional coordinate storage unit M1, and the local feature vector generation unit 45 are also used in the first embodiment. It is done. That is, the image input unit 4i acquires a color frame image G0 (for example, 640 × 480 pixels) when the shutter is turned on from the camera 31 via the operating system, and stores it in the buffer area Mi. Further, the gray scale conversion unit 41 converts the frame image G0 in the buffer area Mi into a gray scale and converts it into an 8-bit 1-channel luminance image (frame image) G1, and stores this in the buffer area M0. The feature point detection unit 43 performs the same process as step S0 in FIG. 2, acquires the two-dimensional coordinates of each feature point, and stores them in the two-dimensional coordinate storage unit M1. The local feature vector generation unit 45 performs the processing of steps S1 to S5 in FIG. 2 for each feature point in the two-dimensional coordinate storage unit M1, generates a local feature vector, and adds it to the local feature vector storage unit M3.

  Next, reference data used in the search is stored in the reference data storage unit M4 in advance. The reference data storage unit M4 is generated as follows using the reference data generation unit 42, the affine transformation unit 44, the local region image storage unit M2, and the above-described configuration for generating local feature vectors.

  That is, the reference data creation unit 42 cuts out a local region image including the first sampling circle C1 and the second sampling circle C2 as shown in FIG. 5 of each pair feature point from the frame image G1, and sets it as a local region image group G2. The local feature vector (reference local feature vector) of each pair feature point of the local region image group G2 is added to the local region image storage unit M2 and processed by the local feature vector generation unit 45 in steps S1 to S5 in FIG. Obtained and added to the reference data storage unit M4.

  The reference data creation unit 42 also automatically generates a plurality of local area images corresponding to those obtained by changing the depth and orientation of the camera from the respective images of the local area image group G2 by affine transformation, and generates the local area image group G2. In addition, a local feature vector is obtained for each local region image in the same manner as described above and added to the reference data storage unit M4.

  That is, a new local region image is obtained by affine transforming each local region image of the local region image group G2 through each of a plurality of matrices corresponding to changing the optical axis direction without changing the depth via the affine transformation unit 44. A group is generated, these are added to the local region image group G2, and local feature vectors are similarly obtained for each converted image via the local feature vector generation unit 45, and added to the reference data storage unit M4. Further, the reference data creation unit 42 further affine-transforms each local region image group G2 with a plurality of matrices corresponding to lengthening only the depth, that is, reduced local region images, for example, width and height. Each local region image group G3, G4, and G5 is generated by multiplying the length by 1 / √2, further by 1 / √2, and further by 1 / √2, and for each local region image, a local feature vector is generated. Similarly, a local feature vector is obtained through the generation unit 45 and added to the reference data storage unit M4.

In the reference data storage unit M4, the reference data creation unit 42 associates the same class ID (CID) with each local feature vector related to the same pair feature point regardless of the presence or absence of affine transformation. That is, in the reference data storage unit M4, as shown in FIG. 10A, for example, local feature vectors of a plurality of different camera viewpoints for one pair feature point, for example, V0101, V0102, V0103,. Are classified by the same class ID, for example, CID01.

  The reference data storage unit M4 is further associated with a frame image ID (FID) to which each CID belongs. For example, FID01 and FID12 are associated with CID01. This means that CID01 is included in each frame image of FID01 and FID12.

  The reference data storage unit M4 also includes the information associated with each FID, for example, related book information, detailed information on signboards, or product information.

  The matching unit 46 includes a decision tree as a local feature vector classifier (classifier). The classifier receives the local feature vector V as an input and outputs a class ID. In this embodiment, a random forest (Random Forest) using a plurality of trees is used as the decision tree. The reason for this is that it operates at high speed during use, is a multi-class classifier, has a relatively high classification accuracy, and there is a trade-off between identification accuracy and memory usage, but its parameters can be adjusted by the number of trees ( If the tree is small (many), the identification accuracy is low (high) but the memory usage is small (large).

  The reason why each component of the local feature vector is not binarized is that the decision tree is used so that matching can be performed at high speed irrespective of binarization and the discriminating power of the local feature vector is reduced by binarization. This is to avoid it.

  The matching unit 46 learns a random forest classifier. That is, a plurality of subsets (the number of elements of each subset is the same as each other) is randomly determined from the entire set of local feature vectors in the reference data storage unit M4 without considering whether or not the class ID is the same. As shown in FIG. 10B, a division function f (V) for dividing the subset at the branch nodes of the tree and a threshold value t for determining the division boundary are randomly determined, and learning is performed so that the information gain is maximized. Then, the parameter of the division function f (V) and the threshold value t are updated, and the probability Pr for each class ID is associated with each leaf node of each tree (IDs not associated with leaf nodes are The probability is 0).

  The matching unit 46 traces each tree of the random forest for each local feature vector V, obtains the probability for each class ID at the leaf nodes, and determines the ID that maximizes the sum of the probabilities for each class ID in all trees. The output of the random forest classifier.

  That is, the matching unit 46 performs the processing of steps S10 to S15 as shown in FIG. 9 on each local feature vector V in the local feature vector storage unit M3 to estimate the class ID of the local feature vector V.

  (S10) An empty histogram having the horizontal axis as the class ID and the vertical axis as the frequency (more accurately, the cumulative addition value of the probability values) is generated in the frame image ID histogram storage unit M5. Thereafter, the processes in steps S11 to S14 are performed for each tree in the random forest.

  (S12) The tree is traced from top to bottom with respect to the local feature vector V. At this time, at each node of the tree, the corresponding component of the local feature vector V, its threshold value t, and the division function f (V) Based on this, it is determined to which child node to branch, and from the probability distribution of class IDs obtained at the leaf nodes, for example, three class IDs in descending order of probability values are determined.

(S13) The probability values of these three class IDs are added to the histogram of step S10.

  (S15) The mode value on the histogram is estimated as the class ID of the feature point of the local feature vector V (see FIG. 10B).

  (S16) The matching unit 46 stores the frame image ID (FID) corresponding to the estimated class ID (CID), for example, FID01 and FID12 corresponding to ID01 which is the class ID on the left side of FIG. The counter acquired from the part M4 and identified by the frame image ID in the frame image ID histogram storage part M5 is incremented by one.

  After the matching unit 46 performs the processing of FIG. 9 on each local feature vector V in the local feature vector storage unit M3, the frame image ID estimation unit 47 sets the counter value in the frame image ID histogram storage unit M5 to the maximum. The frame image ID is estimated as the ID of the frame image in the buffer area Mi.

  The frame image ID information output unit 48 extracts information corresponding to the frame image ID from the reference data storage unit M4 and outputs the information to the display device 32.

  Next, test results of processing by the matching unit 46 will be described.

  FIGS. 11 to 14 are diagrams including images obtained by performing the processing in FIG. 7 on the input image of the printed matter including the swan photograph and the character string “Swan”, and visualizing the result. FIG. 12 is a view of a visualized image showing the input image, feature point pairs extracted from the image, and straight lines connecting the feature points of each pair. FIGS. 12 to 14 are all on the reference image. The feature points and the feature points on the input image in FIG. 11 are matched by the matching unit 46, and the matched feature points are connected by a straight line. FIG. 12 is a diagram in which the reference image reduces and rotates the input image. FIG. 13 corresponds to the reference image obtained by reducing and rotating the input image and performing projection conversion, and FIG. 14 is obtained by rotating the input image and reducing the reference image as compared with the case of FIG. It is a figure equivalent to a thing. Here, the reference image is an image from which the reference data is obtained.

  There are 137 pairs of feature points on the reference images in FIGS. 12 and 13, among which 111 pairs (81%) succeeded in matching in FIG. 12, and 93 pairs (68%) matched in FIG. succeeded in. The number of feature point pairs on the reference image in FIG. 14 is 36, of which 29 pairs (80%) succeeded in matching.

  In the above, preferred embodiments of the present invention have been described. However, the present invention includes various other modifications, and those using other configurations for realizing the functions of the above-described components can be used by those skilled in the art. Other configurations that would come from these configurations or functions, if any, are also included in the present invention.

  For example, the discriminator used in the matching unit 46 is not limited to a random forest discriminator as long as the matching accuracy can be obtained at a high speed to a certain degree, and is not limited to a random forest discriminator, or an ensemble learning algorithm such as bagging or boosting. It may be a discriminator using one decision tree.

  Further, the reference data storage unit M4 may be configured to automatically generate the reference data as described above by the reference data creation unit 42 after the application is started.

  Furthermore, the present invention can be applied to an augmented reality (AR) display device or the like.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 20 Main body part 21 Processor 22 Bus | bath 23 Memory | storage device 24 Input interface 25 Camera interface 26 Display interface 27 Communication part 30 Input device 31 Camera 32 Display apparatus 33 Antenna 4i Image input part 40 Main processing part 41 Gray scale part 42 Reference data creation unit 43 Feature point detection unit 44 Affine transformation unit 45 Local feature vector generation unit 46 Matching unit 47 Frame image ID estimation unit 48 Frame image ID information output unit 340, 350, 36 First feature points 341-344, 351 37 Second feature point 352, C1 First sampling circle 353, C2 Second sampling circle P0-Pf, Q0-Qf Region Mi, M0 Buffer region M1 Two-dimensional coordinate storage unit M2 Local region image storage unit M3 Local Symptoms vector storage unit M4 reference data storage section M5 frame image ID histogram storage unit

Claims (7)

A processor and a storage device in which the data and the program are stored, wherein the data includes a grayscale image, and the program includes a feature vector generation program causing the processor to generate a plurality of local feature amounts included in the data In the image processing apparatus,
The feature vector generation program provides the processor with
(A) detecting the coordinates of feature points that are corner points in the grayscale image;
(B) selecting a pair feature point of each of a predetermined number of second feature points in order closer to the first feature point which is each detected feature point and the first feature point;
(C) For each pair feature point, the distance L between the first feature point and the second feature point is obtained,
(D) Pixel regions Pi, i each including n pixels (n ≧ 4) at equal pixel intervals among the pixels on the circumference of the first radius proportional to the distance L with the first feature point as the center. The average first luminance I (Pi) of = 0 to n−1 is sampled in a predetermined order with respect to the line direction of the distance L, and the difference between each and the luminance of the pixel region including the first feature point;
A pixel region Qi, i = 0 to 0 including each of m pixels (m ≧ 4) at equal pixel intervals among the pixels on the circumference of the second radius that is centered on the second feature point and proportional to the distance L The average second luminance I (Qj) of m−1 is sampled in a predetermined order with respect to the line direction of the distance L, and the difference between each and the luminance of the pixel region including the second feature point;
To obtain a normalized local feature vector with
An image processing apparatus, wherein the square root of the number of pixels in the pixel region is substantially proportional to the distance L.   The image processing apparatus according to claim 1, wherein each of m and n is 8, 16 or 32. A camera,
The image processing apparatus according to claim 1, wherein the grayscale image is an image obtained by converting a frame image captured by the camera into a gray scale. The data further includes, for each reference gray image, a local feature vector generated by the feature vector generation program associated with a class ID as a reference local feature vector and information on the reference gray image, Including an image search program,
The image search program provides the processor with
(E) For each local feature vector obtained in step (d) for the search grayscale image, the class ID in the reference data corresponding to the local feature vector is set as the local feature vector and the reference local in the reference data. A decision is made by matching with a feature vector, the counter of the reference gray image to which the class ID belongs is incremented,
(F) Information in the reference data regarding the reference grayscale image having the maximum counter value is output as information of the search grayscale image.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The image search program causes the processor to determine the class ID in a step (e) by a discriminator having a local feature vector as an input and a class ID as an output.
The image processing apparatus according to claim 4.   The image processing apparatus according to claim 1, wherein the image processing apparatus is an augmented reality display device.   The program which comprises the image processing apparatus as described in any one of Claims 1 thru | or 6.