CN113792752A - Image feature extraction method and system based on binocular camera and intelligent terminal - Google Patents

Image feature extraction method and system based on binocular camera and intelligent terminal Download PDF

Info

Publication number
CN113792752A
CN113792752A CN202110884275.6A CN202110884275A CN113792752A CN 113792752 A CN113792752 A CN 113792752A CN 202110884275 A CN202110884275 A CN 202110884275A CN 113792752 A CN113792752 A CN 113792752A
Authority
CN
China
Prior art keywords
feature
target
points
constraint condition
descriptor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110884275.6A
Other languages
Chinese (zh)
Other versions
CN113792752B (en
Inventor
朱海涛
杨超
王欣亮
孙钊
裴姗姗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Smarter Eye Technology Co Ltd
Original Assignee
Beijing Smarter Eye Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Smarter Eye Technology Co Ltd filed Critical Beijing Smarter Eye Technology Co Ltd
Priority to CN202110884275.6A priority Critical patent/CN113792752B/en
Publication of CN113792752A publication Critical patent/CN113792752A/en
Application granted granted Critical
Publication of CN113792752B publication Critical patent/CN113792752B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image feature extraction method, system and intelligent terminal based on a binocular camera, wherein the method comprises the following steps: the method comprises the steps of obtaining an original image of a binocular camera, extracting feature points of the original image based on different algorithms, obtaining feature descriptors corresponding to the feature points, fusing the feature points obtained under the different algorithms, obtaining target feature points through screening, fusing the feature descriptors obtained under the different algorithms, obtaining target feature descriptors through screening, constructing constraint conditions according to the target feature descriptors, and constructing a feature point space in a target feature point range based on the constraint conditions to obtain a scene feature set. Therefore, the technical problem that the image feature extraction accuracy of the binocular camera in the prior art is poor is solved, and the accuracy of the image feature extraction is improved.

Description

Image feature extraction method and system based on binocular camera and intelligent terminal
Technical Field
The invention relates to the technical field of visual algorithms, in particular to an image feature extraction method and system based on a binocular camera and an intelligent terminal.
Background
With the development of automatic driving technology, people have increasingly higher requirements on safety and comfort of vehicles for assisting driving. In the automatic driving scene, the feature extraction is used as the basis of other functional algorithms, and the extraction accuracy directly influences the safety of automatic driving. The existing feature extraction scheme mainly focuses on feature extraction of image information, and the actual physical constraint characteristics such as time continuity and space consistency cannot be fully considered, so that the feature extraction accuracy is poor. Therefore, providing an image feature extraction method, system and intelligent terminal based on a binocular camera to improve the accuracy of feature extraction becomes a problem to be solved by those skilled in the art.
Disclosure of Invention
Therefore, the embodiment of the invention provides an image feature extraction method and system based on a binocular camera and an intelligent terminal, and aims to solve the technical problem that the image feature extraction accuracy of the binocular camera in the prior art is poor.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
a binocular camera-based image feature extraction method, the method comprising:
acquiring an original image of a binocular camera;
extracting feature points of the original image based on different algorithms, and obtaining feature descriptors corresponding to the feature points;
fusing the feature points obtained under different algorithms, and obtaining target feature points through screening;
fusing the feature descriptors obtained under different algorithms, and obtaining a target feature descriptor through screening;
constructing a constraint condition according to the target feature descriptor;
and constructing a feature point space in the target feature point range based on the constraint condition to obtain a scene feature set.
Further, the extracting feature points of the original image based on different algorithms and obtaining feature descriptors corresponding to the feature points specifically include:
extracting feature points of the original image based on a first feature extraction algorithm to obtain a first group of feature points and first feature descriptors corresponding to the first group of feature points;
and extracting the feature points of the original image based on a second feature extraction algorithm to obtain a second batch of feature points and a second feature descriptor corresponding to the second batch of feature points.
Further, the first feature extraction algorithm is a SIFT feature extraction algorithm, and the second feature extraction algorithm is an ORB feature extraction algorithm.
Further, fusing the feature points obtained under different algorithms, and obtaining the target feature point through screening, specifically comprising:
calculating ORB features of the first batch of feature points and SIFT features of the second batch of feature points, and fusing the obtained features to obtain fused feature points;
acquiring a fusion feature descriptor corresponding to the fusion feature point;
and if the fusion feature descriptor is judged to be larger than a preset obvious threshold value, reserving the fusion feature point as the target feature point, and using the feature descriptor corresponding to the target feature point as a target feature descriptor.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
calculating three-dimensional coordinates (x, y, z) of the target feature points based on parallax information of a binocular camera;
the constraint condition comprises a first constraint condition which is: l2-norm (| (x1, y1, z1), (x2, y2, z2) of three-dimensional coordinates between adjacent two of the target feature points|L2Is smaller than the preset threshold Dt.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
for the original imageDividing, and counting the number of all characteristic points in different label areas based on the dividing result
Figure BDA0003193430180000021
And the number of feature points satisfying the first constraint condition
Figure BDA0003193430180000022
The constraint conditions further comprise a second constraint condition, wherein the second constraint condition is as follows: the number of target feature points of which the same region does not satisfy the first constraint condition
Figure BDA0003193430180000023
Is less than a preset threshold Nt.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
setting a target characteristic point P at the previous moment of two adjacent frames of original images in the continuous N framest-1Target feature point P at present timetTarget feature point P at the previous timet-1Corresponding object feature descriptor Ft-1And the target feature point P at the current timetCorresponding object feature descriptor Ft；
The constraint conditions further comprise a third constraint condition, wherein the third constraint condition is as follows: the target feature point satisfies the L2-norm F of the target feature descriptor within consecutive N framest-1，Ft|L2Less than a predetermined threshold Tt1And N is a positive integer greater than 1.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
obtaining the target characteristic point P of the left eye view at the same timelAnd target feature point P of right eye viewr；
The constraint conditions further include a fourth constraint condition, and the fourth constraint condition is: the target characteristic point P of the left eye viewlAnd target feature point P of right eye viewrExist simultaneously and meet the target characteristicsL2-norm F of syndrome descriptorl，Fr||L2Less than a predetermined threshold Tt2。
The invention also provides an image feature extraction system based on a binocular camera, which is used for implementing the method, and comprises the following steps:
the image acquisition unit is used for acquiring an original image of the binocular camera;
the feature extraction unit is used for extracting feature points of the original image based on different algorithms and obtaining feature descriptors corresponding to the feature points;
the characteristic fusion unit is used for fusing the characteristic points obtained under different algorithms and obtaining target characteristic points through screening; fusing the feature descriptors obtained under different algorithms, and obtaining a target feature descriptor through screening;
the constraint construction unit is used for constructing constraint conditions according to the target feature descriptors;
and the feature set output unit is used for constructing a feature point space in the target feature point range based on the constraint condition so as to obtain a scene feature set.
The present invention also provides an intelligent terminal, including: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform the method as described above.
The image feature extraction method based on the binocular camera comprises the steps of obtaining original images of the binocular camera, extracting feature points of the original images based on different algorithms, obtaining feature descriptors corresponding to the feature points, fusing the feature points obtained under the different algorithms, obtaining target feature points through screening, fusing the feature descriptors obtained under the different algorithms, obtaining target feature descriptors through screening, constructing constraint conditions according to the target feature descriptors, and constructing a feature point space in a target feature point range based on the constraint conditions to obtain a scene feature set. Therefore, the technical problem that the image feature extraction accuracy of the binocular camera in the prior art is poor is solved, and the accuracy of the image feature extraction is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions that the present invention can be implemented, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the effects and the achievable by the present invention, should still fall within the range that the technical contents disclosed in the present invention can cover.
Fig. 1 is a flowchart of an embodiment of a binocular camera-based image feature extraction method according to the present invention;
fig. 2 is a block diagram of an embodiment of the image feature extraction system based on a binocular camera according to the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In a specific embodiment, as shown in fig. 1, the image feature extraction method based on a binocular camera provided by the invention comprises the following steps:
s1: the original image of the binocular camera is acquired, and obviously, the original image may be a continuous multi-frame image, and the original image includes an image acquired by a left eye and an image acquired by a right eye.
S2: extracting feature points of the original image based on different algorithms, and obtaining feature descriptors corresponding to the feature points; the feature descriptor means that a feature is expressed by using a mathematical vector.
Specifically, feature point extraction is carried out on the original image based on a first feature extraction algorithm to obtain a first group of feature points and first feature descriptors corresponding to the first group of feature points; and extracting the feature points of the original image based on a second feature extraction algorithm to obtain a second batch of feature points and a second feature descriptor corresponding to the second batch of feature points.
In a specific scenario, the first feature extraction algorithm is a SIFT feature extraction algorithm, and the second feature extraction algorithm is an ORB feature extraction algorithm. At the moment, feature point extraction is carried out on the original image based on different algorithms, and feature descriptors corresponding to the feature points are obtained, specifically, feature point extraction is carried out on the image by utilizing an SIFT feature extraction algorithm, and a first batch of feature points p1 and feature descriptors f1s thereof are obtained; and (5) extracting the feature points of the images by using an ORB feature extraction algorithm to obtain a second batch of feature points p2 and a feature descriptor f2o thereof.
It is to be understood that SIFT, a Scale-invariant feature transform (SIFT), is a description used in the field of image processing. The description has scale invariance, can detect key points in the image and is a local feature descriptor. The SIFT feature is based on some locally apparent points of interest on the object, regardless of the size and rotation of the image. The tolerance to light, noise, and micro-viewing angle changes is also quite high. Based on these characteristics, they are highly significant and relatively easy to retrieve, easily identify objects and are rarely misidentified in feature databases with large denominations. The detection rate of partial object occlusion using the SIFT feature description is also quite high, and even more than 3 SIFT object features are enough to calculate the position and orientation. Under the current hardware speed of computer and the condition of small feature database, the recognition speed can approach to real-time operation. The SIFT features have large information quantity and are suitable for quick and accurate matching in a mass database.
ORB is a short for organized Fast and Rotated Brief and can be used to quickly create feature vectors for key points in an image, which can be used to identify objects in the image. Wherein Fast and Brief are the feature detection algorithm and the vector creation algorithm, respectively. The ORB first looks for a special area from the image, called a keypoint. Key points are small areas, such as corners, that stand out in the image, such as they have the characteristic that the pixel values change sharply from light to dark. The ORB will then compute a corresponding feature vector for each keypoint. The feature vector created by the ORB algorithm contains only 1 and 0, called binary feature vector. The order of 1 and 0 will vary depending on the particular keypoint and the pixel area around it. The vector represents the intensity pattern around the keypoint, so multiple feature vectors can be used to identify larger regions, even particular objects in the image. ORB is characterized by being ultra fast and to some extent immune to noise and image transformations, such as rotation and scaling transformations.
S3: fusing the feature points obtained under different algorithms, and obtaining target feature points through screening;
s4: fusing the feature descriptors obtained under different algorithms, and obtaining a target feature descriptor through screening;
specifically, ORB features of the first batch of feature points and SIFT features of the second batch of feature points are calculated, and the obtained features are fused to obtain fused feature points;
acquiring a fusion feature descriptor corresponding to the fusion feature point;
and if the fusion feature descriptor is judged to be larger than a preset obvious threshold value, reserving the fusion feature point as the target feature point, and using the feature descriptor corresponding to the target feature point as a target feature descriptor.
Still taking the above usage scenario as an example, orb feature f1o is calculated for feature point p 1; calculating SIFT feature f2s aiming at the feature point p 2; constructing a fusion characteristic descriptor according to the mode of f1 ═ f1s + f1o and f2 ═ f2s + f2 o; the "+" indicates feature fusion and is not additive in the mathematical sense. Examining the fusion feature descriptors (comprising f1 and f2) aiming at all feature points (comprising p1 and p 2); when the fusion feature descriptor is larger than a preset obvious threshold value, retaining the feature points and the descriptors thereof; otherwise, deleting the feature points and the descriptors thereof. And marking the deleted feature point as point, and marking the corresponding feature descriptor as feature.
S5: constructing a constraint condition according to the target feature descriptor; the constraint condition is a boundary condition when the connection between the point sets is constructed based on the alternative feature points.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
calculating three-dimensional coordinates (x, y, z) of the target feature points based on parallax information of a binocular camera;
the constraint condition comprises a first constraint condition which is: l2-norm (| (x1, y1, z1), (x2, y2, z2) | | of three-dimensional coordinates between two adjacent target feature pointsL2Is smaller than the preset threshold Dt. Wherein, the value range of Dt belongs to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
segmenting the original image, and counting the number of all characteristic points in different label areas based on segmentation results
Figure BDA0003193430180000071
And the number of feature points satisfying the first constraint condition
Figure BDA0003193430180000072
The constraint conditions further comprise a second constraint condition, wherein the second constraint condition is as follows: the number of target feature points of which the same region does not satisfy the first constraint condition
Figure BDA0003193430180000073
Is less than a preset threshold Nt. Wherein the value range of Nt belongs to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
setting a target characteristic point P at the previous moment of two adjacent frames of original images in the continuous N framest-1Target feature point P at present timetTarget feature point P at the previous timet-1Corresponding object feature descriptor Ft-1And the target feature point P at the current timetCorresponding object feature descriptor Ft；
The constraint conditions further comprise a third constraint condition, wherein the third constraint condition is as follows: the target feature point satisfies the L2-norm F of the target feature descriptor within consecutive N framest-1，Ft||L2Less than a predetermined threshold Tt1And N is a positive integer greater than 1. Wherein, Tt1Belong to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
Further, the constructing a constraint condition according to the target feature descriptor specifically includes:
obtaining the target characteristic point P of the left eye view at the same timelAnd target feature point P of right eye viewr；
The constraint conditions further include a fourth constraint condition, and the fourth constraint condition is: the target characteristic point P of the left eye viewlAnd target feature point P of right eye viewrExist simultaneously and satisfy the L2-norm F of the target feature descriptorl，Fr||L2Less than a predetermined threshold Tt2. Wherein, Tt2Fall within the value range ofSet [0,1024]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
In the above usage scenario, the process of constructing the constraint condition according to the target feature descriptor includes:
first, a first constraint C1 is constructed. Based on the parallax information, three-dimensional spatial information calculation (x, y, z) of the feature point is calculated. The first constraint is: the three-dimensional spatial information between feature points should be smooth. The definition of smoothness is: l2-norm (| (x1, y1, z1), (x2, y2, z2) of three-dimensional coordinates between two adjacent feature pointsL2Should be less than the preset threshold Dt.
Second, a second constraint C2 is constructed. Segmenting the image, and counting the number of all characteristic points in different label areas based on segmentation results
Figure BDA0003193430180000081
And the number of feature points satisfying the first constraint condition
Figure BDA0003193430180000082
The second constraint is: the number of points in the same area meeting the first constraint condition is as much as possible. As many definitions as possible are: number of feature points that do not satisfy the L1 constraint
Figure BDA0003193430180000083
Should be less than a preset threshold Nt.
Third, a third constraint C3 is constructed. Repeating the above operations for the adjacent quantity frame data in the time sequence to obtain the characteristic point P of the previous timet-1And the feature point P of the current timet. According to the respective corresponding characteristic descriptor Ft-1And FtMatching, wherein the matching scheme is as follows: the feature descriptors between corresponding feature points should be consistent. The consistent definition of a feature descriptor is: l2-norm | Ft-1, Ft | of feature descriptorL2Less than a predetermined threshold Tt1. The third constraint is: the feature points should have a matching relationship within consecutive N frames; where N is a predetermined threshold, and the value of N is too large or too largeSmall, may cause the constraints to be too tight or too loose, so N should be adjusted empirically for practical use.
Finally, a fourth constraint C4 is constructed. Repeating the above operations for the left and right images at the same time point to obtain the feature point P of the left eye view at the same timelAnd a feature point P of the right eye view. According to the respective corresponding characteristic descriptor F1And FrMatching, wherein the matching scheme is as follows: the feature descriptors between corresponding feature points should be consistent. The consistent definition of a feature descriptor is: l2-norm F of feature descriptorl，Fr||L2Less than a predetermined threshold Tt2. The fourth constraint is: the feature points should instead exist in both the left and right views and satisfy the epipolar constraint.
S6: and constructing a feature point space in the target feature point range based on the constraint condition to obtain a scene feature set. Based on the four constraint conditions, a feature point space Γ ∈ { P ∈ C1, P ∈ C2, P ∈ C3, P ∈ C4} is constructed, that is, the feature point P simultaneously belongs to a feature space composed of 4 constraint conditions, and a space formed by the feature points is an intersection of a feature space set composed of the four constraint conditions.
In a specific embodiment, the image feature extraction method based on the binocular camera provided by the invention obtains an original image of the binocular camera, performs feature point extraction on the original image based on different algorithms, obtains feature descriptors corresponding to the feature points, fuses the feature points obtained under different algorithms, obtains target feature points through screening, fuses the feature descriptors obtained under different algorithms, obtains target feature descriptors through screening, constructs constraint conditions according to the target feature descriptors, and constructs a feature point space within the range of the target feature points based on the constraint conditions to obtain a scene feature set. Therefore, the technical problem that the image feature extraction accuracy of the binocular camera in the prior art is poor is solved, and the accuracy of the image feature extraction is improved.
In addition to the above method, the present invention also provides a binocular camera-based image feature extraction system for implementing the above method, as shown in fig. 2, the system comprising:
an image acquisition unit 100 for acquiring an original image of a binocular camera;
and the feature extraction unit 200 is configured to perform feature point extraction on the original image based on different algorithms, and obtain a feature descriptor corresponding to the feature point.
Specifically, the feature extraction unit 200 is configured to perform feature point extraction on the original image based on a first feature extraction algorithm to obtain a first group of feature points and first feature descriptors corresponding to the first group of feature points; and extracting the feature points of the original image based on a second feature extraction algorithm to obtain a second batch of feature points and a second feature descriptor corresponding to the second batch of feature points.
The first feature extraction algorithm is a SIFT feature extraction algorithm, and the second feature extraction algorithm is an ORB feature extraction algorithm.
A feature fusion unit 300, configured to fuse feature points obtained under different algorithms, and obtain a target feature point through screening; and fusing the feature descriptors obtained under different algorithms, and obtaining the target feature descriptor through screening.
The feature fusion unit 300 is specifically configured to:
calculating ORB features of the first batch of feature points and SIFT features of the second batch of feature points, and fusing the obtained features to obtain fused feature points;
acquiring a fusion feature descriptor corresponding to the fusion feature point;
and if the fusion feature descriptor is judged to be larger than a preset obvious threshold value, reserving the fusion feature point as the target feature point, and using the feature descriptor corresponding to the target feature point as a target feature descriptor.
And a constraint constructing unit 400, configured to construct a constraint condition according to the target feature descriptor.
Calculating three-dimensional coordinates (x, y, z) of the target feature points based on parallax information of a binocular camera;
the constraint conditions compriseA first constraint condition, the first constraint condition being: l2-norm (| (x1, y1, z1), (x2, y2, z2) | | of three-dimensional coordinates between two adjacent target feature pointsL2Is smaller than the preset threshold Dt. Wherein, the value range of Dt belongs to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
The constraint building unit 400 is specifically configured to:
segmenting the original image, and counting the number of all characteristic points in different label areas based on segmentation results
Figure BDA0003193430180000101
And the number of feature points satisfying the first constraint condition
Figure BDA0003193430180000102
The constraint conditions further comprise a second constraint condition, wherein the second constraint condition is as follows: the number of target feature points of which the same region does not satisfy the first constraint condition
Figure BDA0003193430180000103
Is less than a preset threshold Nt. Wherein the value range of Nt belongs to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
The constraint building unit 400 is specifically configured to:
setting a target characteristic point P at the previous moment of two adjacent frames of original images in the continuous N framest-1Target feature point P at present timetTarget feature point P at the previous timet-1Corresponding object feature descriptor Ft-1And the target feature point P at the current timetCorresponding object feature descriptor Ft(ii) a The constraint conditions further comprise a third constraint condition, wherein the third constraint condition is as follows: the target feature point satisfies L2-norm | Ft-1, Ft | of the target feature descriptor in the continuous N framesL2Less than a predetermined threshold Tt1And N is a positive integer greater than 1. Wherein, Tt1Belong to the set[0,1024]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
The constraint building unit 400 is specifically configured to:
obtaining the target characteristic point P of the left eye view at the same timelAnd target feature point P of right eye viewr；
The constraint conditions further include a fourth constraint condition, and the fourth constraint condition is: the target characteristic point P of the left eye viewlAnd target feature point P of right eye viewrExist simultaneously and satisfy the L2-norm F of the target feature descriptorl，Fr||L2Less than a predetermined threshold Tt2. Wherein, Tt2Belong to the set [0,1024 ]]The influence of the deletion result corresponding to different thresholds is different; the smaller the threshold, the more compact the screening conditions.
A feature set output unit 500, configured to construct a feature point space within the target feature point range based on the constraint condition, so as to obtain a scene feature set.
In the foregoing specific embodiment, the image feature extraction system based on a binocular camera provided by the invention obtains an original image of the binocular camera, performs feature point extraction on the original image based on different algorithms, obtains feature descriptors corresponding to the feature points, fuses the feature points obtained under different algorithms, obtains target feature points through screening, fuses the feature descriptors obtained under different algorithms, obtains target feature descriptors through screening, constructs constraint conditions according to the target feature descriptors, and constructs a feature point space within the range of the target feature points based on the constraint conditions, so as to obtain a scene feature set. Therefore, the technical problem that the image feature extraction accuracy of the binocular camera in the prior art is poor is solved, and the accuracy of the image feature extraction is improved.
The present invention also provides an intelligent terminal, including: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform the method as described above.
In correspondence with the above embodiments, embodiments of the present invention also provide a computer storage medium containing one or more program instructions therein. Wherein the one or more program instructions are for executing the method as described above by a binocular camera depth calibration system.
In an embodiment of the invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above embodiments are only for illustrating the embodiments of the present invention and are not to be construed as limiting the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the embodiments of the present invention shall be included in the scope of the present invention.

Claims (10)

1. An image feature extraction method based on a binocular camera is characterized by comprising the following steps:
acquiring an original image of a binocular camera;
extracting feature points of the original image based on different algorithms, and obtaining feature descriptors corresponding to the feature points;
fusing the feature points obtained under different algorithms, and obtaining target feature points through screening;
fusing the feature descriptors obtained under different algorithms, and obtaining a target feature descriptor through screening;
constructing a constraint condition according to the target feature descriptor;
and constructing a feature point space in the target feature point range based on the constraint condition to obtain a scene feature set.
2. The image feature extraction method according to claim 1, wherein the extracting feature points of the original image based on different algorithms and obtaining feature descriptors corresponding to the feature points specifically comprises:
extracting feature points of the original image based on a first feature extraction algorithm to obtain a first group of feature points and first feature descriptors corresponding to the first group of feature points;
and extracting the feature points of the original image based on a second feature extraction algorithm to obtain a second batch of feature points and a second feature descriptor corresponding to the second batch of feature points.
3. The image feature extraction method according to claim 2, wherein the first feature extraction algorithm is a SIFT feature extraction algorithm, and the second feature extraction algorithm is an ORB feature extraction algorithm.
4. The image feature extraction method according to claim 3, wherein feature points obtained under different algorithms are fused, and a target feature point is obtained by screening, and specifically includes:
calculating ORB features of the first batch of feature points and SIFT features of the second batch of feature points, and fusing the obtained features to obtain fused feature points;
acquiring a fusion feature descriptor corresponding to the fusion feature point;
and if the fusion feature descriptor is judged to be larger than a preset obvious threshold value, reserving the fusion feature point as the target feature point, and using the feature descriptor corresponding to the target feature point as a target feature descriptor.
5. The image feature extraction method according to claim 4, wherein the constructing a constraint condition according to the target feature descriptor specifically includes:
calculating three-dimensional coordinates (x, y, z) of the target feature points based on parallax information of a binocular camera;
the constraint condition comprises a first constraint condition which is: l2-norm (| (x1, y1, z1), (x2, y2, z2) | | of three-dimensional coordinates between two adjacent target feature pointsL2Is smaller than the preset threshold Dt.
6. The image feature extraction method according to claim 5, wherein the constructing a constraint condition according to the target feature descriptor specifically includes:
segmenting the original image, and counting the number of all characteristic points in different label areas based on segmentation results
Figure FDA0003193430170000021
And the number of feature points satisfying the first constraint condition
Figure FDA0003193430170000022
The constraint conditions further comprise a second constraint condition, wherein the second constraint condition is as follows: the number of target feature points of which the same region does not satisfy the first constraint condition
Figure FDA0003193430170000023
Is less than a preset threshold Nt.
7. The image feature extraction method according to claim 6, wherein the constructing a constraint condition according to the target feature descriptor specifically includes:
setting a target characteristic point P at the previous moment of two adjacent frames of original images in the continuous N framest-1Target feature point P at present timetTarget feature point P at the previous timet-1Corresponding object feature descriptor Ft-1And the target feature point P at the current timetCorresponding object feature descriptor Ft；
The constraint conditions further comprise a third constraint condition, wherein the third constraint condition is as follows: the target feature point satisfies the L2-norm F of the target feature descriptor within consecutive N framest-1，Ft||L2Less than a predetermined threshold Tt1And N is a positive integer greater than 1.
8. The image feature extraction method according to claim 7, wherein the constructing a constraint condition according to the target feature descriptor specifically includes:
obtaining the target characteristic point P of the left eye view at the same timelAnd target feature point P of right eye viewr；
The constraint conditions further include a fourth constraint condition, and the fourth constraint condition is: the target characteristic point P of the left eye viewlAnd target feature point P of right eye viewrExist simultaneously and satisfy the L2-norm F of the target feature descriptorl，Fr||L2Less than a predetermined threshold Tt2。
9. A binocular camera based image feature extraction system for implementing the method of any one of claims 1 to 8, the system comprising:
the image acquisition unit is used for acquiring an original image of the binocular camera;
the feature extraction unit is used for extracting feature points of the original image based on different algorithms and obtaining feature descriptors corresponding to the feature points;
the characteristic fusion unit is used for fusing the characteristic points obtained under different algorithms and obtaining target characteristic points through screening; fusing the feature descriptors obtained under different algorithms, and obtaining a target feature descriptor through screening;
the constraint construction unit is used for constructing constraint conditions according to the target feature descriptors;
and the feature set output unit is used for constructing a feature point space in the target feature point range based on the constraint condition so as to obtain a scene feature set.
10. An intelligent terminal, characterized in that, intelligent terminal includes: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor, configured to execute one or more program instructions to perform the method of any of claims 1-8.
CN202110884275.6A 2021-08-03 2021-08-03 Binocular camera-based image feature extraction method and system and intelligent terminal Active CN113792752B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110884275.6A CN113792752B (en) 2021-08-03 2021-08-03 Binocular camera-based image feature extraction method and system and intelligent terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110884275.6A CN113792752B (en) 2021-08-03 2021-08-03 Binocular camera-based image feature extraction method and system and intelligent terminal

Publications (2)

Publication Number Publication Date
CN113792752A true CN113792752A (en) 2021-12-14
CN113792752B CN113792752B (en) 2023-12-12

Family

ID=79181330

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110884275.6A Active CN113792752B (en) 2021-08-03 2021-08-03 Binocular camera-based image feature extraction method and system and intelligent terminal

Country Status (1)

Country Link
CN (1) CN113792752B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758162A (en) * 2022-06-14 2022-07-15 北京市农林科学院信息技术研究中心 Commodity anti-counterfeiting identification method and device, electronic equipment and storage medium
CN117253156A (en) * 2023-11-17 2023-12-19 深圳元戎启行科技有限公司 Feature description extraction method, device, terminal and medium based on image segmentation

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105674993A (en) * 2016-01-15 2016-06-15 武汉光庭科技有限公司 Binocular camera-based high-precision visual sense positioning map generation system and method
WO2018177379A1 (en) * 2017-03-31 2018-10-04 北京市商汤科技开发有限公司 Gesture recognition, gesture control and neural network training methods and apparatuses, and electronic device
CN109166149A (en) * 2018-08-13 2019-01-08 武汉大学 A kind of positioning and three-dimensional wire-frame method for reconstructing and system of fusion binocular camera and IMU
CN110132302A (en) * 2019-05-20 2019-08-16 中国科学院自动化研究所 Merge binocular vision speedometer localization method, the system of IMU information
CN111815738A (en) * 2020-06-15 2020-10-23 北京沃东天骏信息技术有限公司 Map construction method and device
CN112001954A (en) * 2020-08-20 2020-11-27 大连海事大学 Polar curve constraint-based underwater PCA-SIFT image matching method
WO2020259365A1 (en) * 2019-06-27 2020-12-30 Oppo广东移动通信有限公司 Image processing method and device, and computer-readable storage medium
CN113192113A (en) * 2021-04-30 2021-07-30 山东产研信息与人工智能融合研究院有限公司 Binocular visual feature point matching method, system, medium and electronic device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105674993A (en) * 2016-01-15 2016-06-15 武汉光庭科技有限公司 Binocular camera-based high-precision visual sense positioning map generation system and method
WO2018177379A1 (en) * 2017-03-31 2018-10-04 北京市商汤科技开发有限公司 Gesture recognition, gesture control and neural network training methods and apparatuses, and electronic device
CN109166149A (en) * 2018-08-13 2019-01-08 武汉大学 A kind of positioning and three-dimensional wire-frame method for reconstructing and system of fusion binocular camera and IMU
CN110132302A (en) * 2019-05-20 2019-08-16 中国科学院自动化研究所 Merge binocular vision speedometer localization method, the system of IMU information
WO2020259365A1 (en) * 2019-06-27 2020-12-30 Oppo广东移动通信有限公司 Image processing method and device, and computer-readable storage medium
CN111815738A (en) * 2020-06-15 2020-10-23 北京沃东天骏信息技术有限公司 Map construction method and device
CN112001954A (en) * 2020-08-20 2020-11-27 大连海事大学 Polar curve constraint-based underwater PCA-SIFT image matching method
CN113192113A (en) * 2021-04-30 2021-07-30 山东产研信息与人工智能融合研究院有限公司 Binocular visual feature point matching method, system, medium and electronic device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王科;黄智;钟志华;: "基于多特征融合的道路理解方法", 中国公路学报, no. 04 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758162A (en) * 2022-06-14 2022-07-15 北京市农林科学院信息技术研究中心 Commodity anti-counterfeiting identification method and device, electronic equipment and storage medium
CN117253156A (en) * 2023-11-17 2023-12-19 深圳元戎启行科技有限公司 Feature description extraction method, device, terminal and medium based on image segmentation
CN117253156B (en) * 2023-11-17 2024-03-29 深圳元戎启行科技有限公司 Feature description extraction method, device, terminal and medium based on image segmentation

Also Published As

Publication number Publication date
CN113792752B (en) 2023-12-12

Similar Documents

Publication Publication Date Title
Wei et al. Toward automatic building footprint delineation from aerial images using CNN and regularization
Chen et al. Improved saliency detection in RGB-D images using two-phase depth estimation and selective deep fusion
Cai et al. Saliency-based pedestrian detection in far infrared images
CN110378837B (en) Target detection method and device based on fish-eye camera and storage medium
CN111160232B (en) Front face reconstruction method, device and system
CN113792752A (en) Image feature extraction method and system based on binocular camera and intelligent terminal
CN113128347B (en) Obstacle target classification method and system based on RGB-D fusion information and intelligent terminal
WO2023185234A1 (en) Image processing method and apparatus, and electronic device and storage medium
CN115082450A (en) Pavement crack detection method and system based on deep learning network
CN112348116A (en) Target detection method and device using spatial context and computer equipment
CN112800978A (en) Attribute recognition method, and training method and device for part attribute extraction network
CN109447023A (en) Determine method, video scene switching recognition methods and the device of image similarity
CN116012432A (en) Stereoscopic panoramic image generation method and device and computer equipment
US9171227B2 (en) Apparatus and method extracting feature information of a source image
CN112036342B (en) Document snapshot method, device and computer storage medium
CN113792583A (en) Obstacle detection method and system based on drivable area and intelligent terminal
Wang et al. LBP-based edge detection method for depth images with low resolutions
CN114972470B (en) Road surface environment obtaining method and system based on binocular vision
CN115205809B (en) Method and system for detecting roughness of road surface
CN111627041B (en) Multi-frame data processing method and device and electronic equipment
KR20160148806A (en) Object Detecter Generation Method Using Direction Information, Object Detection Method and Apparatus using the same
CN114998743A (en) Method, device, equipment and medium for constructing visual map points
CN111144489B (en) Matching pair filtering method and device, electronic equipment and storage medium
CN112907553A (en) High-definition image target detection method based on Yolov3
CN110648388A (en) Scene geometric modeling method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant