CN110009549B - Computing method of rotational symmetry descriptor and hardware accelerator - Google Patents
Computing method of rotational symmetry descriptor and hardware accelerator Download PDFInfo
- Publication number
- CN110009549B CN110009549B CN201910193892.4A CN201910193892A CN110009549B CN 110009549 B CN110009549 B CN 110009549B CN 201910193892 A CN201910193892 A CN 201910193892A CN 110009549 B CN110009549 B CN 110009549B
- Authority
- CN
- China
- Prior art keywords
- sampling
- descriptor
- calculating
- module
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
- G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
- G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method for calculating a rotational symmetry descriptor, wherein sampling positions of the rotational symmetry descriptor are rotationally symmetrical about the center of a sampling area. The invention can use the characteristic to replace the rotation sampling position with the shift operation to lead the descriptor to have the rotation invariance, thereby greatly reducing the calculation cost and the storage cost. The step of computing a rotational symmetry descriptor on the image comprises: determining a sampling area; calculating a sampling direction; calculating a descriptor and shifting the descriptor. The invention also provides a hardware accelerator adopting the calculation method of the rotational symmetry descriptor.
Description
Technical Field
The invention relates to the field of computer vision and image processing, in particular to a computing method of a rotational symmetry descriptor and a hardware accelerator.
Background
The image feature point extraction is a means for extracting key information from an image, and is a precondition for image analysis and image recognition. Image feature point extraction generally comprises two steps: detecting feature points and calculating descriptors of the feature points. Among the feature descriptors, the brief (binary Robust Independent element features) descriptor is widely applied in the real-time application scene due to low computational complexity and rapid matching.
The BRIEF descriptor is a binary descriptor consisting of a sequence of 0 and 1, and is obtained by comparing the gray value sizes of pixels at sampling positions around a feature point. Because the BRIEF does not have rotation invariance, the BRIEF cannot be directly applied to an actual scene. To achieve rotation invariance for BRIEF descriptors, a common method is to calculate the direction of a feature and then rotate the sample position of the descriptor according to the feature direction to make it consistent with the feature direction (e.g., ORB (organized FAST and rotaTedBEREF) feature). Thus, regardless of the rotation of the feature, the sample positions of the descriptors undergo the same rotation, which leaves the computed descriptors unaffected by the rotation. However, rotating the sampling positions of the descriptors greatly increases the complexity of computing the descriptors, increasing the time and energy consumption overhead.
In some image processing scenarios with high real-time requirements, a special hardware accelerator is often considered to be used to increase the image feature extraction speed. In order to accelerate the rotation operation of the BRIEF description sub-sampling position, a large number of multipliers are needed, which brings large on-chip resource and energy consumption overhead; however, if the sampling positions rotated to the respective directions are calculated in advance, all of them need to be solidified on the chip, and a large amount of on-chip resources also need to be consumed. This is not conducive to the design implementation of the hardware acceleration unit.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a method for computing a rotational symmetry descriptor, and in particular, a method for computing a rotational symmetry binary descriptor based on a BRIFE algorithm, which utilizes the rotational symmetry characteristic of a sampling position to replace the rotation of the sampling position with a shift operation on the descriptor.
Further, another object of the present invention is to provide a hardware accelerator, in particular a hardware accelerator using the above calculation method of the rotational symmetry descriptor.
According to an aspect of the present invention, there is provided a method for calculating a rotational symmetry descriptor, which is a binary descriptor based on BRIEF algorithm, whose sampling positions are rotationally symmetric with respect to the center of a sampling region.
The sampling region is a circular region with the radius of R pixels, the sampling positions are a set of K pairs of sampling points, the sampling positions are rotationally symmetrical, and the minimum rotation angle is
Wherein m is the number of all possible values of the characteristic direction after linear quantization, and the characteristic direction is defined as the direction of a connecting line from the characteristic position to the gray scale centroid position in the sampling region C.
The adopted positions are constructed as follows:
the method comprises the following steps: randomly selecting n pairs of sampling points T in a sampling area according to Gaussian distribution1{(S1,D1),(S2,D2),…,(Sn,Dn) Where n is K/m, (S)i,Di) (i ═ 1,2, …, n) represents a pair of sample points;
step two: will T1Rotate clockwise respectively
To obtain T2,T3,…,Tm；
Step three: finding T1,T2,…,TmUnion T of1∪T2∪T3∪…∪TmAnd obtaining the final sampling position: t { (S)1,D1),(S2,D2),…,(Sm*n,Dm*n)}。
Further, the step of computing the descriptor on the image is as follows:
the method comprises the following steps: extracting features on the image and determining a sampling region for each feature;
step two: calculating a characteristic direction A:
the grayscale centroid position (u, v) within the sampling region C is defined as:
wherein, I (x, y) represents the gray value of the pixel at the coordinate (x, y);
step three: calculating a descriptor sub-sampling direction As：
Wherein FLOOR represents rounding down, and the value range of the characteristic direction A is
Step four: comparing the pixel gray value at the sampling position in the sampling region C to calculate K-bit binary descriptor D [0: K-1]]The calculation method is as follows: for any bit D [ j ] of descriptor D](j-0, 1, …, K-1) if I (S)j)>I(Dj) Then D [ j ]]1, otherwise D [ j ]]0, wherein, I (S)j) And I (D)j) Respectively represent the position SjAnd DjA gray value of the pixel;
step five: according to the descriptor sub-sampling direction AsShifting descriptor D to obtain the first A of DsM shift toAnd finally.
Preferably, the descriptors use hamming distances to represent differences between descriptors.
According to another aspect of the present invention, a hardware accelerator using the above calculation method is provided, and in particular, a hardware accelerator for extracting the above rotational symmetry descriptor is characterized by comprising a buffer, a sampling module, a direction calculation module, and a shift module.
The cache comprises an image cache and a result cache which are respectively used for storing pixels in the sampling area where the characteristic points are located and the descriptors obtained through calculation.
The sampling module is used for comparing the gray value of the pixel at the sampling position. The rotationally symmetric sampling locations are fixed in a sampling module and connected to a set of comparators. The result of each comparator is one bit of the descriptor, and all comparison results are arranged in the order of sampling positions to be the descriptors.
The direction calculation module is used for calculating the sampling direction of the features, and the flow is as follows: firstly, calculating the gray centroid of pixels in a sampling area; then, calculating the ratio of the horizontal and vertical coordinates of the gray centroid; and finally, obtaining the sampling direction through a lookup table.
And the shifting module shifts the descriptor according to the result of the direction calculation module, so that the descriptor has rotation invariance.
Preferably, the direction calculation module comprises a set of multipliers, a set of adders, a divider and a look-up table that solidifies the tangent-to-sample direction mapping relationship.
The invention has the beneficial effects that: a method for calculating binary descriptor based on BRIEF descriptor in rotation symmetry utilizes the rotation symmetry property of sampling position to replace the rotation of sampling position by the shift operation of descriptor. On one hand, the calculation complexity is greatly reduced, on the other hand, the design of a special hardware accelerator is simplified, and the hardware resource overhead is reduced. The hardware accelerator of the rotational symmetry descriptor can accelerate the speed of extracting the descriptor from an image, and has the advantages of low power consumption and high operation speed compared with the traditional computing platform (such as a CPU (Central processing Unit) and a GPU (graphics processing Unit)).
Drawings
FIG. 1 is a schematic diagram of a sampling region of a rotationally symmetric descriptor in accordance with one embodiment of the present invention.
FIG. 2 is a schematic diagram of the sampling locations of a rotational symmetry descriptor in accordance with an embodiment of the present invention.
FIG. 3 is a block diagram of a hardware accelerator that employs a rotational symmetry descriptor computation method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail by combining specific embodiments with the accompanying drawings.
Fig. 1 is a schematic diagram of a sampling region of a rotational symmetry descriptor according to an embodiment of the present invention, in which each square represents a pixel, and there are 31 × 31 pixels. Assuming that the central position of the first map is a feature point, a circular area composed of gray pixels and having a radius of 15 pixels is a sampling area of the feature point.
FIG. 2 is a schematic diagram of the sampling positions of a rotational symmetry descriptor according to an embodiment of the present invention, in which two endpoints of each line segment represent a pair of points, and there are 256 pairs of sampling points { (S)1,D1),(S2,D2),…,(S256,D256)}. The steps for constructing the sampling positions shown in fig. 2 are as follows: firstly, randomly selecting 8 pairs of sampling points in a sampling region according to Gaussian distribution (S)1,D1),(S2,D2),…,(S8,D8). Next, (S) will be1,D1),(S2,D2),…,(S8,D8) Clockwise rotating by 11.25 degrees by taking the center of the sampling region as a rotation center to obtain (S)9,D9),(S10,D10),…,(S16,D16) (ii) a Rotated by 22.5 degrees to obtain (S)17,D17),(S18,D18),…,(S24,D24) (ii) a … …, respectively; by analogy, finally obtaining { (S)1,D1),(S2,D2),…,(S256,D256)}. The 256 pairs of sample points are rotationally symmetric about the sampling region center with a minimum rotation angle of 11.25 degrees.
The process of calculating the descriptor on the image, i.e. extracting the rotationally symmetric descriptor on the picture, comprises the following steps: first, key points (including but not limited to FAST points, Harris points, Shi and Tomasi points, etc.) are extracted on the picture. Then, calculating the characteristic direction of the key point, wherein the direction of the key point is defined as the direction from the position of the key point to the gray scale centroid of all pixels in the sampling area where the key point is located. The coordinate of the grayscale centroid is the ratio of the sum of the pixel coordinates multiplied by the grayscale value to the sum of the pixel grayscale values. If the sampling region is represented by C and the gray value of the pixel at coordinate (x, y) is represented by I (x, y), then the coordinates (u, v) of the gray centroid of all pixels within the sampling region can be defined by:
after calculating the feature direction of the keypoint, discretizing the result into a multiple of 11.25 degrees and mapping to 0 to 31 yields the descriptive sub-sampling direction, such as: a 5.625 degree to 5.625 degree mapping of 0, a 5.625 degree to 16.875 degree mapping of 1, and so on. And after the sampling direction of the descriptor is obtained, calculating the descriptor by comparing the gray value of the pixel at the sampling position. Descriptor D [0:255]Is a 256-bit binary sequence, and is used for any bit D [ j ] of the descriptor D](j-0, 1, …, K-1) if I (S)j)>I(Dj) Then D [ j ]]1, otherwise D [ j ]]0, wherein, I (S)j) And I (D)j) Respectively represent the position SjOr DjThe grey value of the pixel.
Finally, the subsampling direction A is describedsShifting the descriptor D to obtain the first 8A of DsAnd shifting to the end. For example, if the sampling direction is 5, then D [0:255 ]]Becomes D' ═ { D [40:255],D[0:39]}，D' is the final result.
Fig. 3 is a schematic structural diagram of a hardware accelerator adopting the calculation method of the rotational symmetry descriptor according to an embodiment of the present invention, and the hardware accelerator includes a buffer, a sampling module, a direction calculation module, and a shift module. The buffer comprises two parts, one part is used for storing the input pixel matrix, and the other part is used for storing the descriptors obtained through calculation. According to the coordinates of the descriptor to be calculated, a rectangular window 31 x 31 slides on the pixel matrix stored in the cache, and the pixel gray value in the window is input into the sampling module and the direction calculation module. 256 pairs of sampling positions are solidified in the sampling module, and the 256 pairs of sampling positions are divided into 32 groups of 8 pairs each, and are rotationally symmetrical with a minimum rotation angle of 11.25 degrees. This 256 compares the gray values of the pixels at the sample locations by a set of comparators. Each comparator outputs a bit of 0 or 1 as a result, and the results of all comparators are arranged in the order of sampling positions to obtain a 256-bit result of the sampling module. While the sampling module is sampling, the direction calculation module calculates the grayscale centroid of the sampling region with the same 31 × 31 pixel matrix as input. The direction calculation module mainly comprises a group of multipliers, a group of adders, a divider and a lookup table which solidifies the mapping relation of tangent values to sampling directions. The multiplier array is used for calculating the gray value (I) of a pixeli) And the horizontal and vertical coordinates (X) of the pixeli,Yi) The product of (a): u. ofi＝Xi×Ii,vi＝Yi×IiThe adder array is used for summing the results of the adders, calculating u- ∑ ui,v＝∑vi. The divider is used for the ratio of v and u, which is also the tangent value of the gray scale centroid direction, and the obtained result can be used for obtaining the sampling direction through a lookup table. And the shifting module shifts the descriptor according to the sampling direction to ensure the rotation invariance of the descriptor.
The invention is further illustrated above using specific embodiments. It should be noted that the above-mentioned embodiments are only specific embodiments of the present invention, and should not be construed as limiting the present invention. Any modification, replacement, improvement and the like within the idea of the present invention should be within the protection scope of the present invention.
Claims (8)
1. A method for calculating a rotational symmetry descriptor, wherein the descriptor is a binary descriptor based on BRIEF algorithm, the sampling position of the descriptor is rotationally symmetric about the center of a sampling region, and the minimum rotation angle is
Wherein m is the number of all possible values of the characteristic direction after linear quantization, the characteristic direction is defined as the direction of the connecting line from the characteristic position to the gray scale centroid position in the sampling region C,
the sampling region is a circular region with the radius of R pixels, the sampling position is a set of K pairs of sampling points, and the construction mode of the sampling position is as follows:
the method comprises the following steps: randomly selecting n pairs of sampling points T in the sampling area C according to Gaussian distribution1{(S1,D1),(S2,D2),…,(Sn,Dn) And (c) the step of (c) in which,
representing a pair of sample points;
step two: n pairs of sampling points T1Rotate clockwise respectively
To obtain T2,T3,…,Tm；
Step three: finding T1,T2,…,TmUnion T of1∪T2∪T3∪…∪TmAnd obtaining the final sampling position: t { (S)1,D1),(S2,D2),…,(Sm*n,Dm*n)}。
2. The computing method according to claim 1, characterized by comprising the steps of:
the method comprises the following steps: extracting features on the image and determining a sampling region for each feature;
step two: calculating a characteristic direction A:
the grayscale centroid position (u, v) of the sampling region C is defined as:
wherein, I (x, y) represents the gray value of the pixel at the coordinate (x, y);
step three: calculating a descriptor sub-sampling direction As：
Step four: comparing the pixel gray value at the sampling position in the sampling region C, and calculating a K-bit binary descriptor D [0: K-1 ];
step five: according to the descriptor sub-sampling direction AsPerforming a shift operation on the descriptor D to describe the front A of the descriptor DsM shifts to the end.
3. The method of claim 2, wherein the calculation of step four is as follows: for any bit D [ j ] of descriptor D]J is 0,1, …, K-1, if I (S)j)>I(Dj) Then D [ j ]]1, otherwise D [ j ]]0, wherein, I (S)j) And I (D)j) Respectively represent the position SjOr DjThe grey value of the pixel.
4. A method according to one of claims 1-3, characterized in that hamming distances are used to represent the differences between descriptors.
5. A hardware accelerator employing the method of computing a rotational symmetry descriptor according to any one of claims 1 to 4, comprising a buffer, a sampling module, a direction computation module, and a shift module;
the cache comprises an image cache and a result cache, and is respectively used for storing pixels in a sampling area where the characteristic points are located and descriptors obtained through calculation;
the sampling module obtains a descriptor by comparing the gray value of the pixel at the sampling position;
the direction calculation module is used for calculating a characteristic sampling direction;
the shifting module is used for shifting the descriptor to enable the descriptor to have rotation invariance.
6. The hardware accelerator of claim 5 wherein the sampling module compares the gray scale values of the pixels at the sampling locations, the rotationally symmetric sampling locations are fixed in the sampling module and connected to a set of comparators, each comparator has a bit of descriptor, and all comparison results are arranged in the order of the sampling locations to be descriptors.
7. The hardware accelerator according to claim 5 or 6, wherein the flow of the direction calculation module calculating the sampling direction of the feature is: firstly, calculating the gray centroid of pixels in a sampling area; then, calculating the ratio of the horizontal and vertical coordinates of the gray centroid; and finally, obtaining the sampling direction through a lookup table.
8. The hardware accelerator of claim 5 or 6 wherein the direction computation module comprises a set of multipliers, a set of adders, a divider and a lookup table that solidifies the tangent-to-sample direction mapping.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910193892.4A CN110009549B (en) | 2019-03-14 | 2019-03-14 | Computing method of rotational symmetry descriptor and hardware accelerator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910193892.4A CN110009549B (en) | 2019-03-14 | 2019-03-14 | Computing method of rotational symmetry descriptor and hardware accelerator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110009549A CN110009549A (en) | 2019-07-12 |
| CN110009549B true CN110009549B (en) | 2020-08-21 |
Family
ID=67167118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910193892.4A Active CN110009549B (en) | 2019-03-14 | 2019-03-14 | Computing method of rotational symmetry descriptor and hardware accelerator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110009549B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104282001A (en) * | 2014-09-22 | 2015-01-14 | 中国电子科技集团公司第二十研究所 | Method for enhancing image feature two-value descriptor performance |
| CN107239792A (en) * | 2017-05-12 | 2017-10-10 | 大连理工大学 | A kind of workpiece identification method and device based on binary descriptor |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521740A (en) * | 2009-04-01 | 2009-09-02 | 北京航空航天大学 | Real-time athletic estimating method based on multiple dimensioned unchanged characteristic |
| CN103955690B (en) * | 2014-04-15 | 2017-03-01 | 合肥工业大学 | A kind of building method of compact image local feature description |
| CN104851094A (en) * | 2015-05-14 | 2015-08-19 | 西安电子科技大学 | Improved method of RGB-D-based SLAM algorithm |
| US10013765B2 (en) * | 2016-08-19 | 2018-07-03 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for image registrations |
| CN106355577B (en) * | 2016-09-08 | 2019-02-12 | 武汉科技大学 | Rapid image matching method and system based on significant condition and global coherency |
-
2019
- 2019-03-14 CN CN201910193892.4A patent/CN110009549B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104282001A (en) * | 2014-09-22 | 2015-01-14 | 中国电子科技集团公司第二十研究所 | Method for enhancing image feature two-value descriptor performance |
| CN107239792A (en) * | 2017-05-12 | 2017-10-10 | 大连理工大学 | A kind of workpiece identification method and device based on binary descriptor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110009549A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414533B (en) | Feature extraction and matching method for improving ORB | |
| CN106960451B (en) | Method for increasing number of feature points of image weak texture area | |
| US11600059B2 (en) | Systems and methods for providing an image classifier | |
| US9122954B2 (en) | Performing a histogram using an array of addressable registers | |
| US20140226906A1 (en) | Image matching method and apparatus | |
| WO2022033095A1 (en) | Text region positioning method and apparatus | |
| CN111242122A (en) | Lightweight deep neural network rotating target detection method and system | |
| CN109272442B (en) | Method, device and equipment for processing panoramic spherical image and storage medium | |
| CN108960012B (en) | Feature point detection method and device and electronic equipment | |
| CN112364881A (en) | Advanced sampling consistency image matching algorithm | |
| CN112465876A (en) | Stereo matching method and equipment | |
| Kang et al. | Yolo-6d+: single shot 6d pose estimation using privileged silhouette information | |
| CN114937159A (en) | Binocular matching method based on GPU acceleration | |
| Wasala et al. | Real-time HOG+ SVM based object detection using SoC FPGA for a UHD video stream | |
| Sahin et al. | A learning-based variable size part extraction architecture for 6D object pose recovery in depth images | |
| CN110009549B (en) | Computing method of rotational symmetry descriptor and hardware accelerator | |
| JP7265107B2 (en) | Hardware accelerator for computation of histograms of oriented gradients | |
| CN107229935B (en) | Binary description method of triangle features | |
| Cao et al. | FAPP: Extremely fast approach to boosting image matching precision | |
| CN111009004B (en) | Hardware optimization method for accelerating image matching | |
| CN114372993A (en) | Oblique-shooting shelf layered detection method and system based on image correction | |
| Moon et al. | Fast image-matching technique robust to rotation in spherical images | |
| CN113191370A (en) | ORB algorithm based on threshold self-adaptive threshold adjustment | |
| WO2020135602A1 (en) | Image processing method and device, intelligent driving system, and vehicle-mounted computing platform | |
| CN112991448A (en) | Color histogram-based loop detection method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |