TWI592897B - Image Recognition Accelerator System - Google Patents

Description

Image recognition accelerator system

The invention relates to an image recognition accelerator system, in particular to an image recognition system for implementing a SIFT image recognition algorithm on an FPGA.

In recent years, due to the advancement of visual sensors and the maturity of imaging technology, image recognition has become an indispensable part of the field of computer vision. It is widely used in military, industrial, medical fields, such as image stitching, objects. Object recognition, robotic map and navigation, 3D modeling, gesture recognition, and video tracking and match moving.

Image recognition mainly uses the captured image for feature detection. In the past decade, many image feature recognition algorithms have been proposed, and the most well-known one is David G. Lowe's scale feature invariance proposed in the 1999 Computer Vision Conference. Scale-invariant feature transform (SIFT) The SIFT algorithm mainly detects feature points on images and assigns different high-dimensional vector descriptions to each feature point. Thus, images can be matched and similar. The two feature vector points will be compared. It is worth mentioning that the SIFT algorithm takes the direction of each feature point into consideration, so it also successfully solves the problem of Harris corner detection non-rotation-invariant, although SIFT Very good matching results can be obtained under the scale and the rotation of the angle of view. However, the shortcoming of this algorithm is that the amount of computation is very large, which makes the overall operation very time consuming and cannot achieve the effect of real-time operation.

A prior art patent, for example, the Republic of China TW201142718 "Scale Space Normalization Technique for Improving Feature Detection in Uniform and Non-Uniform Illumination Changes", a method and technique for improving the efficiency of an image recognition system . The method comprises: generating a scale spatial image difference by acquiring a difference between two different smoothed versions of an image; and dividing the scale spatial image difference by one of the third smoothed version of the image Generating a normalized scale-space image difference, wherein the third smoothed version of the image is as smooth as or smoother than the smoothest of the two different smoothed versions of the image; and The normalized scale space image difference is used to detect one or more features of the image. Only in the above patent case, the direction of each feature point is not taken into consideration, so that a good matching result cannot be obtained under the change of the viewing angle rotation.

In recent years, some studies have implemented the SIFT algorithm on the FPGA processing platform, mainly through the concept of parallel processing to speed up the computing time. For example, in 2008 Vanderlei Bonato proposed the concept of software and hardware collaborative design, the SIFT part of the algorithm on the FPGA. Using hardware to accelerate the implementation, Jianhui Wang also proposed an architecture based on embedded system feature point detection and matching in 2014. The results show that it can process 60 images per second. Jie Jiang also proposed to use FPGA full hardware. The architecture implements SIFT detection and matching algorithms.

However, when implementing the SIFT algorithm with the FPGA full hardware architecture, it is still necessary to calculate the exponential function, the floating point number and the large use of the divider logic gate, which makes the image recognition consume a lot of computing time, and can not achieve the purpose of instant identification.

An object of the present invention is to provide an image recognition accelerator system, wherein the image pyramid construction module is coupled to the image input module, and is configured to first find a plurality of Gaussian template mask parameters of different scales by software, and then pass through a plurality of The Gaussian filter module performs a plurality of convolution operations in parallel, wherein each of the convolution operations is performed according to the image data and a mask parameter to obtain a plurality of Gaussian images for overcoming the prior art in the Gaussian template. The calculation uses the hardware floating point number generated by the exponential function and the cost of a large amount of computing, so as to effectively improve the system performance.

To achieve the above objective, the present invention provides an image recognition accelerator system, comprising: an image input module for inputting an image data; and an image pyramid construction module coupled to the image input module, which is pre-software Finding a plurality of Gaussian template mask parameters of different scales, and performing a plurality of convolution operations in parallel through the plurality of Gaussian filter modules, wherein each of the convolution operations is performed according to the image data and a mask parameter Obtaining a plurality of Gaussian images, and then inputting the plurality of Gaussian images into a differential image module to perform Gaussian image subtraction; and a SIFT detecting module coupled to the image pyramid construction module The image data outputted by the differential image module is subjected to an extreme value detection and an unstable feature point detection operation via an extreme value detection module and an unstable feature point detection module to determine whether Stabilizing the feature point, and performing the extreme value detection, the result of the unstable feature point detection, and storing the result to a first-in first-out register; and a SIFT description module, and The image pyramid construction module is coupled to calculate the Gaussian images output by the Gaussian filter modules via a first-order partial differential matrix module and a CORDIC module to obtain all image points. Gradient data is further calculated by an image gradient histogram statistical module and a normalized computing module to obtain the descriptor data of the feature point, and combined with the position data of the feature point.俾 to provide a real-time image recognition function.

Another object of the present invention is to provide an image recognition accelerator system, wherein the extreme value detection module includes a maximum value detection circuit and a minimum value detection circuit for simultaneously performing a maximum value detection and a minimum The value detection, and then the output signal is subjected to an OR operation through a gate or a gate to obtain a feature point. The unstable feature point detection module further includes a first-order partial differential matrix module and a Hessian matrix module. a group, a Hessian anti-matrix module, a low-contrast feature detection module and an edge feature detection module, and then outputting the output signals of the low-contrast feature detection module and the edge feature detection module And the gate performs one operation.

The Heisen inverse matrix module calculates the value of the adjoint matrix and the determinant by using the adjoint matrix, and outputs the value to the low-contrast feature detection module, and uses the numerical derivation method to replace the use of the plurality of dividers. In order to achieve effective system performance.

It is still another object of the present invention to provide an image recognition accelerator system, wherein the normalization operation module is configured to calculate a normalized value of a feature point vector, multiply a gain value, and use a right shift operation to greatly reduce the division. The use of the device to achieve effective system performance.

Another object of the present invention is to provide an image recognition accelerator system, wherein the image input module, the image pyramid construction module, the SIFT detection module, and the SIFT description module are all designed and used by a pipeline architecture. The pipeline hold circuit waits for the signal to keep the timing synchronized.

The detailed description of the drawings and the preferred embodiments are set forth in the accompanying drawings.

Please refer to FIG. 1 , which is a schematic diagram of a combination of an image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in FIG. 1 , the image recognition accelerator system of the present invention comprises: an image input module 100; an image pyramid construction module 200; a SIFT detection module 300; and a SIFT description module 400.

The image input module 100 has an image input unit 110 and a first shift register 120, and the image input unit 110 is configured to input an image data; the first shift register 120 and The image input unit 110 is coupled to shift and temporarily store the image data.

The image pyramid construction module 200 is coupled to the image input module 100, and is configured to find a plurality of Gaussian template mask parameters of different scales in advance by software, and then perform multiple volumes in parallel through the plurality of Gaussian filter modules 210. a convolution operation, wherein each of the convolution operations is performed according to the image data and a mask parameter to obtain a plurality of Gauss images, and then input the plurality of Gauss images into one The differential image module 220 performs Gaussian image subtraction. In addition, a plurality of second shift registers 230 are coupled between the image pyramid construction module 200 and the SIFT detection module 300 to output the differential image module 220 output by the image pyramid construction module 200. Shift and temporary storage.

The Gaussian filter modules 210 further each have a Gaussian mask value selection module 211, a multiply accumulator module 212, a parallel adder module 213, and a multiplexer module 214.

The SIFT detection module 300 is coupled to the image pyramid construction module 200, and the differential image outputted by the differential image module 220 is passed through an extreme value detection module 310 and an unstable feature point detection module. The 320 performs an extreme value detection and an unstable feature point detection operation to determine whether the feature point is stable, and outputs the extreme value detection module 310 and the unstable feature point detection module 320. An AND operation is performed via a second AND gate 329, and the result is stored in a first-in first-out register 330.

A third shift register 240 is further coupled between the image pyramid construction module 200 and the SIFT description module 400 to shift and temporarily display the Gauss images output by the Gaussian filter module 210. Save.

The SIFT description module 400 further calculates the received Gauss image through the first-order partial differential matrix module 321 and a CORDIC module 410 to determine the gradient values and directions of all image points. The SIFT description module And a fourth shift register 420 is configured to shift and temporarily store the gradient value of the image point outputted by the CORDIC module 410, and then pass through an image gradient histogram statistical module 430 and a normalization. The operation module 440 performs an operation to obtain the descriptor data of the feature point, and combines with the position data of the feature point to know which pixels are the feature points and the descriptor data of the feature point position, To provide an instant image recognition function. The first in first out buffer 310 is used to delay the SIFT detection module 300, and waits for the SIFT description module 400 to output the completed signal, as shown by the arrow in FIG. 1 to enable the SIFT detection mode. Group 300 is synchronized with the timing of the SIFT description module 400.

The image input module 100, the image pyramid construction module 200, the SIFT detection module 300, and the SIFT description module 400 are all implemented by a field programmable logic gate array (FPGA).

Please refer to FIG. 2 , which is a schematic diagram of a hardware architecture of a Gaussian filter module of an image recognition accelerator system according to a preferred embodiment of the present invention.

The conventional technology continuously uses a Gaussian filter to establish a continuous-scale spatial image. In order to solve the problem of the exponential function and the floating point number generated therein, the present invention uses software to pre-calculate a plurality of Gaussian template mask parameters input with different scales. Body, the equation of the variable-scale Gaussian function (1)         <TABLE border="1" borderColor="#000000" width="_0002"><TBODY><tr><td> <img wi="231" he="70" file="02_image001.jpg" img-format ="jpg"></img></td><td> (1) </td></tr></TBODY></TABLE> Left shift n bits to get equation (2),         <TABLE border="1" borderColor="#000000" width="85%"><TBODY><tr><td><img wi="218" he="69" file="02_image003.jpg" img- Format="jpg"></img></td><td> (2) </td></tr></TBODY></TABLE>

Where ( x , y ) is the coordinate of the pixel in the image and σ is the scale space factor.

Select the parameter to convolve with the original image I ( x , y ), such as equation (3), <TABLE border="1"borderColor="#000000"width="_0003"><TBODY><tr><td><imgwi="247"he="48"file="02_image005.jpg"img-format="jpg"></img></td><td> (3) </td></ Tr></TBODY></TABLE>

The Gaussian image can be calculated by shifting the output result by n bits to the right. Under the consideration of accuracy and resources, the preferred embodiment of the present invention has a mask size of 7×7 and a calculation bit of the Gaussian mask value selection module 211. The number of elements is n=10.

As shown in FIG. 2, the Gaussian filter module 210 first determines the input iGaussian_num signal, and inputs all the mask parameter values required by the Gaussian filter module 210 to the Gaussian mask value selection module 211. The parameter is further convoluted with the image values of the seven lines in the seven multiply accumulator modules 212 and a parallel adder module 213, and then the multiplexer module 214 is used to determine the iRead_en signal. Know if the output is a valid value. When the software is pre-calculated, the calculated Gaussian template has been enlarged by 2 to the power of 10, and an integer is input to the Gaussian filter module 210. Therefore, the result is shifted to the right by 10 bits for output.

Please refer to FIG. 3, which is a schematic diagram of a differential pyramid of an image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in Fig. 3, there are 6 Gaussian images (6 scale values) in each layer, and the corresponding Gaussian template is obtained, and the original image is continuously convoluted with Gauss templates of different scales to obtain a For the continuous Gaussian image, after calculating the first layer, the length and width of the third Gaussian image of the layer are reduced by half, that is, the image area is reduced by a quarter, and then the 6-scale Gaussian template just calculated is used to continue. The Gaussian filter module 210 performs the operation and repeats the action according to the required number of layers, thereby establishing a concept that the image is larger and more blurred and smaller according to the image scale, and constructs a continuous scale image, which is selected in the present invention. For example, the implementation of the image pyramid is not limited to four layers. After the continuous fuzzy Gaussian pyramid is created, the continuous Gaussian images are input to a differential image module 220 for Gaussian image subtraction, if each layer has For 6 Gaussian images, 5 differential images are generated for each layer. After all the differential images are calculated, the differential pyramid is constructed.

Please refer to FIG. 4 , which is a schematic diagram of a hardware architecture of a SIFT detection module of an image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in FIG. 4, the SIFT detection module 300 is coupled to the image pyramid construction module 200, and the differential image data outputted by the differential image module 220 is transmitted through the extreme value detection module 310. The stable feature point detection module 320 performs an extreme value detection and an unstable feature point detection operation to determine whether it is a stable feature point, and then the extreme value detection module 310 and the unstable feature point detection The output of the module 320 is subjected to a sum operation via a second AND gate 329.

The extreme value detecting module 310 further includes a maximum value detecting circuit 312 and a minimum value detecting circuit 311 for simultaneously performing a maximum value detection and a minimum value detection, and then outputting the signal through the maximum value. The OR gate 313 performs an OR operation to know whether the pixel value of the image is the maximum or minimum value of the adjacent points, and then waits for a signal through a first pipeline holding circuit 314.

The unstable feature point detection module 320 further includes a first-order partial differential matrix module 321 , a Hessian matrix module 322 , a Hessian inverse matrix module 324 , a low contrast feature detection module 326 , and a The edge feature detecting module 325, a second pipeline holding circuit 323, and a third pipeline holding circuit 327, and then outputting the output signals of the low contrast feature detecting module 326 and the third pipeline holding circuit 327 through a first The AND gate 328 performs an AND operation to know whether the feature point is a stable feature point.

Because the extreme value detection module 310 and the unstable feature point detection module 320 are all designed by the pipeline architecture, the pipeline holding circuit is used for signal waiting. As shown in FIG. 4, the detection of the extremum module 310 takes 4 clocks (clk) time, and the unstable feature detection module 320 takes 12 clock times. Therefore, the output of the extreme value detection module 310 requires data retention of 8 clock times. After the unstable feature point detection module 320 determines the end, the signals of the two are passed through a second gate 329. By performing an operation, it is known whether the point is really a feature point and is a stable feature.

Please refer to FIG. 5 , which is a schematic diagram of the hardware architecture of the extreme value detection module of the SIFT detection module of the image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in the figure, the extreme value detecting module 310 further includes a maximum value detecting circuit 312 and a minimum value detecting circuit 311 for simultaneously performing a maximum value detection and a minimum value detection. The obig_en and osmall_en signals of one bit are respectively output, and then the obig_en and osmall_en signals are subjected to an OR operation via the OR gate 313 to generate an oextrema_en signal. If the result is 1, the point is a feature point. If the result is 0, the opposite is true, and the valid data is judged by the idval signal and the odval signal. If the input idval is 1, it indicates that the data is a valid data. At the end, an odval signal is output.

Please refer to FIG. 6 , which is a schematic diagram of a hardware architecture of a step partial differential matrix module of a SIFT detection module of an image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in FIG. 6, the hardware architecture of the first-order partial differential matrix module 321 is a function of the operation equation (4).         <TABLE border="1" borderColor="#000000" width="_0005"><TBODY><tr><td><img wi="351" he="144" file="02_image007.jpg" img-format ="jpg"></img></td><td> (4) </td></tr></TBODY></TABLE>

The first-order partial differential matrix module 321 further defines a plurality of first temporary registers 321a, and inputs pixel values of left and right, upper and lower, and front and rear of a feature point in the second shift register 230, and pixels in the same direction. The value is subtracted, and then placed in the first register 321a for temporary storage, and the next clock is shifted to the right by 1 bit, meaning that the value of the first register 321a is divided by 2. The operation of one order partial differential matrix of the feature point can be completed.

Please refer to FIG. 7 , which is a schematic diagram of a hardware architecture of a Hessian matrix module of a SIFT detection module of an image recognition accelerator system according to a preferred embodiment of the present invention.

Equation (5) is the Hessian matrix equation.         <TABLE border="1" borderColor="#000000" width="_0006"><TBODY><tr><td><img wi="210" he="86" file="02_image009.jpg" img-format ="jpg"></img></td><td> (5) </td></tr></TBODY></TABLE>

As shown in FIG. 7, the hardware architecture of the Hessian matrix module 322 is to implement the functions of equations (6) to (11).         <TABLE border="1" borderColor="#000000" width="_0007"><TBODY><tr><td> <img wi="316" he="29" file="02_image011.jpg" img-format ="jpg"></img></td><td> (6) </td></tr><tr><td> <img wi="329" he="29" file="02_image013. Jpg" img-format="jpg"></img></td><td> (7) </td></tr><tr><td> <img wi="334" he="29" File="02_image015.jpg" img-format="jpg"></img></td><td> (8) </td></tr><tr><td> <img wi="441" He="48" file="02_image017.jpg" img-format="jpg"></img></td><td> (9) </td></tr><tr><td> <img Wi="402" he="48" file="02_image019.jpg" img-format="jpg"></img></td><td> (10) </td></tr><tr> <td> <img wi="402" he="48" file="02_image021.jpg" img-format="jpg"></img></td><td> (11) </td></ Tr></TBODY></TABLE>

The Hessian matrix module 322 further defines a plurality of registers 322a, and inputs pixel values in a direction adjacent to a feature point in the second displacement register 230, and calculates x-direction, y-direction, s-direction, xy-direction, and xs direction. And the second-order partial differential of the ys direction, the calculated six operational values are substituted into the Hessian matrix of the equation (5), and then output to a Heisen inverse matrix module 324. The operation result of the equations (9) to (11) needs to be divided by 4, and the value of the register 322a is shifted to the right by 2 bits.

The Heisen inverse matrix module 324 receives the operation result of the Hessian matrix module 322 and inputs it according to the corresponding position. In order to effectively utilize the advantages of hardware parallel processing, the present invention uses the adjoint matrix to perform the inverse matrix operations of equations (12) and (13).         <TABLE border="1" borderColor="#000000" width="_0008"><TBODY><tr><td><img wi="316" he="153" file="02_image023.jpg" img-format ="jpg"></img></td><td> (12) </td></tr><tr><td><img wi="321" he="29" file="02_image025. Jpg" img-format="jpg"></img> <img wi="115" he="144" file="02_image027.jpg" img-format="jpg"></img></td>< Td> (13) </td></tr></TBODY></TABLE>

Where, equation (12) using a parallel processing system part 9 matrix calculation formula used simultaneously within two multipliers and an adder or subtractor be implemented, equation (13) using a parallel processing system to partially calculate d The values from ₁ to d ₆ are subtracted and summed.

In the calculation of the inverse matrix, the conventional technique needs to divide each element in the companion matrix by the determinant of the matrix. In order to avoid the use of the divider in the hardware to reduce the system performance, the present invention will calculate the adjoint matrix and the determinant. The value is output to the low-contrast feature detection module 326, and is calculated by using a numerical derivation formula to replace the use of the divider.

Equation (14) and equation (15) are algorithms for judging low contrast features in the prior art.         <TABLE border="1" borderColor="#000000" width="_0009"><TBODY><tr><td><img wi="270" he="48" file="02_image029.jpg" img-format ="jpg"></img></td><td> (14) </td></tr><tr><td><img wi="224" he="57" file="02_image031. Jpg" img-format="jpg"></img></td><td> (15) </td></tr></TBODY></TABLE>

The present invention squares the left and right equations of equation (14) to obtain equation (16).         <TABLE border="1" borderColor="#000000" width="_0010"><TBODY><tr><td><img wi="421" he="53" file="02_image033.jpg" img-format ="jpg"></img></td><td> (16) </td></tr></TBODY></TABLE>

And replace the inverse matrix of equation (15) with the form of the adjoint matrix, such as equation (17),         <TABLE border="1" borderColor="#000000" width="_0011"><TBODY><tr><td><img wi="226" he="83" file="02_image035.jpg" img-format ="jpg"></img></td><td> (17) </td></tr></TBODY></TABLE>

Substituting equation (17) into equation (16), after finishing, equation (18), equation (19), and equation (20) are obtained.         <TABLE border="1" borderColor="#000000" width="_0012"><TBODY><tr><td><img wi="279" he="29" file="02_image037.jpg" img-format ="jpg"></img></td><td> (18) </td></tr><tr><td><img wi="275" he="83" file="02_image039. Jpg" img-format="jpg"></img></td><td> (19) </td></tr><tr><td><img wi="285" he="90" File="02_image041.jpg" img-format="jpg"></img></td><td> (20) </td></tr></TBODY></TABLE>

Then multiply this equation (18), equation (19), and equation (20) Then, equations (21), equations (22), and equations (23) can be derived, so that it is not necessary to use a divider, and a low contrast judgment can be realized. <TABLE border="1"borderColor="#000000"width="85%"><TBODY><tr><td><imgwi="347"he="44"file="02_image045.jpg" img- Format="jpg"></img></td><td> (21) </td></tr><tr><td><imgwi="347"he="44"file="02_image047.jpg"img-format="jpg"></img></td><td> (22) </td></tr><tr><td><imgwi="283"he="53 File==============

The low-contrast feature detection module 326 of the present invention is connected to the first-order partial differential matrix module 321 and the output matrix of the Heisen inverse matrix module 324 and the value of the determinant, and the equation (21) After multiplying left and right by 1024 (shifting 10 bits to the left), it is judged by comparison with equation (24).         <TABLE border="1" borderColor="#000000" width="_0014"><TBODY><tr><td> <img wi="251" he="48" file="02_image051.jpg" img-format ="jpg"></img></td><td> (24) </td></tr><tr><td> <img wi="182" he="48" file="02_image053. Jpg" img-format="jpg"></img></td><td> </td></tr><tr><td> <img wi="317" he="48" file=" 02_image055.jpg" img-format="jpg"></img></td><td> </td></tr><tr><td> <img wi="290" he="58" file ="02_image057.jpg" img-format="jpg"></img></td><td> </td></tr></TBODY></TABLE>

The edge feature detection module 325 of the present invention performs the operations of the equations (25) and (26) on the operation result of a Hessian matrix to obtain the values of the trace and determinant of the Hessian matrix.         <TABLE border="1" borderColor="#000000" width="_0015"><TBODY><tr><td><img wi="269" he="38" file="02_image059.jpg" img-format ="jpg"></img></td><td> (25) </td></tr><tr><td><img wi="260" he="46" file="02_image061. Jpg" img-format="jpg"></img></td><td> (26) </td></tr></TBODY></TABLE>

Then judge the two values by equation (27).         <TABLE border="1" borderColor="#000000" width="_0016"><TBODY><tr><td><img wi="228" he="40" file="02_image063.jpg" img-format ="jpg"></img></td><td> (27) </td></tr></TBODY></TABLE>

If the determinant is greater than 0 and the value of the right-form is greater than the left-form value, the module outputs a signal that the point is not an edge feature, and conversely, the point is an edge feature.

The SIFT description module 400 of the present invention performs a first-order partial differential matrix operation on the received Gauss image via the first-order partial differential matrix module 321 to calculate the amount of change in the x-direction and the y-direction, and then uses the following equation. (28) Equation (29) calculates its amplitude and direction, and then outputs the result to a CORDIC module 410 for calculation.         <TABLE border="1" borderColor="#000000" width="_0017"><TBODY><tr><td><img wi="422" he="29" file="02_image065.jpg" img-format ="jpg"></img></td><td> (28) </td></tr><tr><td><img wi="259" he="48" file="02_image067. Jpg" img-format="jpg"></img></td><td> (29) </td></tr></TBODY></TABLE>

The function of the CORDIC module 410 is to calculate the square root plus root number and the tan ^-1 function operation, and input two variables x ₀ and y ₀ , and the iterative expression is equation (30), <TABLE border="1"borderColor="#000000"width="_0018"><TBODY><tr><td><imgwi="201"he="106"file="02_image069.jpg"img-format="jpg"></Img></td><td> (30) </td></tr></TBODY></TABLE>

First, the first-order partial differential values D _x and D _{y are} input to x ₀ and y ₀ , and the output of the function is obtained through continuous iteration. The present invention uses 10 iterations to perform the operation. Considering the problem of floating point number, the present invention amplifies the input value and the register by 6 bits, so the square of the output plus the root number and the tan ^-1 function operation are shifted to the right by 6 bits, and the squares are added. The root number is also multiplied by one The value, which is implemented here using a positioning fractional operation.

Referring to FIG. 8 to FIG. 8 , FIG. 8 is a schematic diagram showing the hardware structure of the image gradient histogram statistical module of the SIFT description module of the image recognition accelerator system according to a preferred embodiment of the present invention; A schematic diagram of a hardware architecture of an eight-azimuth statistical container module of a SIFT description module of an image recognition accelerator system according to a preferred embodiment of the present invention.

As shown in FIG. 8 , the image gradient histogram statistic module 430 performs a gradient shifting of the image points calculated by the CORDIC module 410 by using a fourth shift register 420 provided by the SIFT description module 400. Shifting and temporarily storing the feature point description of each image point into 16 sub-regions, and counting the image gradient histogram in the sub-area. The present invention uses a parallel processing architecture to simultaneously use 16 eight-dimensional statistical containers. The module 431 performs 16 sub-area image gradient histogram statistics.

As shown in FIG. 9 , each of 45 degrees is one direction, 360 degrees is 8 directions, and each of the 16 sub-regions needs to count the gradient amplitudes of 8 directions. The eight-azimuth statistical container module 431 of the present invention is simultaneously used. The eight orientation statistical containers 432 to 439 are calculated.

The normalized computing system of the prior art sums up the 128-dimensional descriptor data, and must use 128 multipliers and dividers for parallel processing, which greatly reduces system efficiency and consumes a large number of logic units. The normalization operation module 440 of the present invention is multiplied by a gain value, for example, but not limited to 1023, so that the normalized vector can be represented by an integer type, and then the right shift operation can be used. The value of normalization is used to greatly reduce the use of the divider to achieve effective system performance.

Through the implementation of the image recognition accelerator system of the present invention, the image pyramid construction module is configured to find a plurality of Gaussian template mask parameters of different scales in advance by software, and then perform multiple convolutions in parallel through a plurality of Gaussian filter modules. The operation, wherein each of the convolution operations is performed according to the image data and a mask parameter to obtain a plurality of Gaussian images to overcome the hardware floating point generated by the prior art using the exponential function in the Gaussian template operation The number and the cost of a large amount of computing; the Heisen inverse matrix module computing system uses the accompanying matrix method to output the calculated matrix and determinant values to the low-contrast feature detection module, and uses the numerical derivation method. The calculation replaces the use of a plurality of dividers; when the normalized value of the feature point vector is calculated, the normalization value is multiplied by a gain value, and the right shift operation is used to greatly reduce the use of the divider. By reducing the amount of calculation and improving the correct matching rate of feature points, the system performance is improved to achieve the purpose of instant image recognition. Therefore, it is indeed more advanced than the conventional image recognition system.

The disclosure of the present invention is a preferred embodiment. Any change or modification of the present invention originating from the technical idea of the present invention and being easily inferred by those skilled in the art will not deviate from the scope of patent rights of the present invention.

In summary, this case, regardless of its purpose, means and efficacy, is showing its technical characteristics that are different from the conventional ones, and its first invention is practical and practical, and it is also in compliance with the new patent requirements. I will be granted a patent at an early date.

100‧‧‧Image input module

110‧‧‧Image input unit

120‧‧‧First shift register

200‧‧‧Image Pyramid Construction Module

210‧‧‧Gauss filter module

211‧‧‧Gaussian mask value selection module

212‧‧‧Multiply accumulator module

213‧‧‧Parallel Adder Module

214‧‧‧Multiplexer Module

220‧‧‧Differential Image Module

230‧‧‧Second shift register

240‧‧‧ Third shift register

300‧‧‧SIFT detection module

310‧‧‧Extreme Detection Module

311‧‧‧minimum detection circuit

312‧‧‧Maximum value detection circuit

313‧‧‧ or gate

314‧‧‧First pipeline retention circuit

320‧‧‧Unstable feature point detection module

321‧‧‧first-order partial differential matrix module

321a‧‧‧First register

322‧‧‧Hessen Matrix Module

322a‧‧‧Second register

323‧‧‧Second pipeline retention circuit

324‧‧‧Heisen inverse matrix module

325‧‧‧Edge feature detection module

326‧‧‧Low Contrast Feature Detection Module

327‧‧‧ Third pipeline retention circuit

328‧‧‧First Gate

329‧‧‧Second Gate

330‧‧‧First in, first out register

400‧‧‧SIFT Description Module

410‧‧‧CORDIC module

420‧‧‧4th shift register

430‧‧‧Image Gradient Histogram Statistics Module

431‧‧‧ Eight-dimensional statistical container module

432‧‧‧First orientation statistical container

433‧‧‧Second orientation statistical container

434‧‧‧ Third-party statistical container

435‧‧‧Four orientation statistical container

436‧‧‧ fifth orientation statistical container

437‧‧‧ Sixth orientation statistical container

438‧‧‧ seventh orientation statistical container

439‧‧‧ Eighth orientation statistical container

440‧‧‧Normalized computing module

FIG. 1 is a schematic diagram showing the combination of an image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 2 is a schematic diagram showing a hardware architecture of a Gaussian filter module of an image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 3 is a schematic diagram showing a differential pyramid of an image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 4 is a schematic diagram showing the hardware architecture of the SIFT detection module of the image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 5 is a schematic diagram showing the hardware architecture of the extreme value detection module of the SIFT detection module of the image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 6 is a schematic diagram showing a hardware architecture of a step partial differential matrix module of a SIFT detection module of an image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 7 is a schematic diagram showing the hardware architecture of the Hessian matrix module of the SIFT detection module of the image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 8 is a schematic diagram showing the architecture of an image gradient histogram statistical module of a SIFT description module of an image recognition accelerator system according to a preferred embodiment of the present invention. FIG. 9 is a schematic diagram showing a hardware architecture of an eight-azimuth statistical container module of a SIFT description module of an image recognition accelerator system according to a preferred embodiment of the present invention.

100‧‧‧Image input module

110‧‧‧Image input unit

120‧‧‧First shift register

200‧‧‧Image Pyramid Construction Module

210‧‧‧Gauss filter module

220‧‧‧Differential Image Module

230‧‧‧Second shift register

240‧‧‧ Third shift register

300‧‧‧SIFT detection module

310‧‧‧Extreme Detection Module

320‧‧‧Unstable feature point detection module

321‧‧‧first-order partial differential matrix module

322‧‧‧Hessen Matrix Module

324‧‧‧Heisen inverse matrix module

325‧‧‧Edge feature detection module

326‧‧‧Low Contrast Feature Detection Module

328‧‧‧First Gate

329‧‧‧Second Gate

330‧‧‧First in, first out register

400‧‧‧SIFT Description Module

410‧‧‧CORDIC module

420‧‧‧4th shift register

430‧‧‧Image Gradient Histogram Statistics Module

440‧‧‧Normalized computing module

Claims (9)

An image recognition accelerator system includes: an image input module for inputting an image data; and an image pyramid construction module comprising a plurality of Gaussian filter modules and a plurality of differential image modules, and the image input mode The group coupling is performed by using a software to calculate a plurality of Gaussian template mask parameters of different scales, and then performing a plurality of convolution operations in parallel through a plurality of Gaussian filter modules, wherein each of the convolution operations is based on the image data. Performing a mask parameter to obtain a plurality of Gaussian images, and then inputting the plurality of Gaussian images to the differential image module to perform Gaussian image subtraction; a SIFT detection module, and The image pyramid construction module is coupled to include an extreme value detection module and an unstable feature point detection module, wherein the image data outputted by the differential image module is passed through the extreme value detection module and the The stable feature point detection module performs an extreme value detection and an unstable feature point detection operation to determine whether it is a stable feature point, and the extreme value detection and the unstable feature point detection The result of the measurement is performed and stored in a first-in first-out register; and a SIFT description module includes a first-order partial differential matrix module and a CORDIC module coupled to the image pyramid construction module. The Gaussian images outputted by the Gaussian filter modules are operated by the first-order partial differential matrix module and the CORDIC module to obtain gradient data of all image points, and then an image gradient histogram The graph statistics module and a normalization operation module calculate the gradient data to obtain the descriptor data of the feature point, and combine with the location data of the feature point to provide a real-time image recognition function. . The image recognition accelerator system of claim 1, wherein the image input module has an image input unit and a first shift register, and the first shift register is configured to The output image of the image input module is shifted and temporarily stored; the image pyramid construction module and the SIFT detection module are coupled with a plurality of second shift registers for outputting the differential image module The image data is shifted and temporarily stored; a third shift register is coupled between the image pyramid construction module and the SIFT description module for outputting the Gaussian images to the Gaussian filter module. Shifting and temporarily storing; the SIFT description module is provided with a fourth shift register for computing the image of the CORDIC module The gradient data of the points are shifted and temporarily stored. The image recognition accelerator system according to claim 1, wherein when the software calculation is performed, the Gaussian template is calculated to be enlarged by 10 to the power of 10, and an integer input is taken to overcome the floating point operation problem. The image recognition accelerator system of claim 1, wherein the extreme value detection module performs a maximum value detection and a minimum value detection simultaneously, and then performs an OR operation on the output signal to obtain The feature point detection module further includes a first-order partial differential matrix module, a Hessian matrix module, a Hessian inverse matrix module, a low-contrast feature detection module, and an edge feature. The detection module performs an operation on the output signals of the low-contrast feature detection module and the edge feature detection module. The image recognition accelerator system according to claim 4, wherein the Heisen inverse matrix module calculates the value of the adjoint matrix and the determinant using the adjoint matrix method, and outputs the value to the low contrast feature detection module. It is calculated by numerical derivation to replace the use of a plurality of dividers. The image recognition accelerator system of claim 1, wherein the first-in first-out register is used to delay the SIFT detection module, so that the SIFT detection module and the SIFT description module are The timing is kept in sync. The image recognition accelerator system according to claim 1, wherein the normalization operation module is configured to calculate a normalized value of the feature point vector, multiply a gain value, and use a right shift operation to substantially reduce Use of the divider. The image recognition accelerator system of claim 1, wherein the image input module, the image pyramid construction module, the SIFT detection module, and the SIFT description module are all designed by a pipeline architecture, and Use the pipeline hold circuit for signal waits to keep the timings in sync. The image recognition accelerator system of claim 1, wherein the image input module, the image pyramid construction module, the SIFT detection module, and the SIFT description module are all field programmable logic gate arrays. (FPGA) implementation.