CN108230392A - A kind of dysopia analyte detection false-alarm elimination method based on IMU - Google Patents
A kind of dysopia analyte detection false-alarm elimination method based on IMU Download PDFInfo
- Publication number
- CN108230392A CN108230392A CN201810063615.7A CN201810063615A CN108230392A CN 108230392 A CN108230392 A CN 108230392A CN 201810063615 A CN201810063615 A CN 201810063615A CN 108230392 A CN108230392 A CN 108230392A
- Authority
- CN
- China
- Prior art keywords
- obtains
- image
- false
- alarm
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title claims abstract description 38
- 206010047571 Visual impairment Diseases 0.000 title claims abstract description 24
- 230000008030 elimination Effects 0.000 title claims abstract description 18
- 238000003379 elimination reaction Methods 0.000 title claims abstract description 18
- 239000012491 analyte Substances 0.000 title claims abstract description 17
- 230000004888 barrier function Effects 0.000 claims abstract description 45
- 230000004438 eyesight Effects 0.000 claims abstract description 23
- 238000012937 correction Methods 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 238000013519 translation Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 4
- 230000005855 radiation Effects 0.000 claims description 4
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 claims 1
- 239000011800 void material Substances 0.000 claims 1
- 238000001914 filtration Methods 0.000 abstract description 4
- 238000003702 image correction Methods 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 3
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 3
- 235000003140 Panax quinquefolius Nutrition 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 235000008434 ginseng Nutrition 0.000 description 3
- 238000006116 polymerization reaction Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Abstract
The invention discloses a kind of dysopia analyte detection false-alarm elimination method based on IMU, including：First, scene image is acquired based on binocular camera；Then, image correction module includes the distortion correction of image and polar curve corrects；Then, the left and right view after polar curve is corrected carries out Stereo matching, obtains the disparity map of scene；The disparity map is filtered, except denoising obtains dense disparity map, and the difference according to parallax value is divided into a few width disparity maps；Contour detecting is carried out respectively for the disparity map of above-mentioned acquisition, obtains obstacle information；Based on the inside and outside parameter of above-mentioned information and camera, the information such as size, distance and position of barrier in scene are calculated, complete false-alarm filtering.The present invention provides a kind of methods based on binocular stereo vision detection, determine obstacle target, and false-alarm is rejected based on contour detecting, ensure that the accuracy of detection of obstacles.
Description
Technical field
The present invention relates to visual barrier detection fields, and in particular to a kind of dysopia analyte detection false-alarm based on IMU
Elimination method.
Background technology
In recent years, with the rapid development of artificial intelligence technology, unmanned plane, intelligent robot and unmanned technology etc.
The hot spot and difficult point competitively researched and developed and captured as global field.However, only when automobile, unmanned plane or intelligent robot can
Understand all occurred in progress path, when oneself can judging, could really realize it is unmanned, automatic obstacle avoiding with
The functions such as positioning.Therefore, obstacle detection technology becomes one of key technology in the information Perception system.Binocular vision is meter
One important branch of calculation machine vision, the process that binocular vision can be perceived with apish eyes and human stereoscopic vision, is meter
One of core subject of calculation machine vision research.In recent years, binocular vision technology produces, only in detection of obstacles, industrial automation
The fields such as energy safety-protection system are widely used.However, often occur in the obstacle detection method based on binocular vision
Some non-detection of obstacles are accidentally barrier, detection of obstacles mistake occur by the false-alarm problem of barrier.
Although in the prior art, there is the characteristics of motion feature using consecutive frame image, to reduce detection of obstacles
False drop rate, since the frame image of detection is easily by external influence, collected frame image is caused to generate distortion, detection is tied
Fruit has an impact.Even if some technologies can carry out disparity computation to collected frame image, the accuracy of detection is improved, still
It is normally based on U-V parallaxes detection of obstacles and carries out disparity computation, but U-V parallax obstacle detection methods are in contour of object
Or object edge context of detection, also there is certain precision problems, it is impossible to accurately reject false alarm information.
Invention content
The present invention provides a kind of dysopia analyte detection false-alarm elimination method based on IMU, to solve the prior art
Detection of obstacles is susceptible to flase drop and contour of object or object edge accuracy of detection is not high, it is impossible to accurately pick false-alarm
Except the technical issues of.
By above-mentioned technical problem, the present invention provides one kind based on IMU (Inertial measurement unit, Chinese
Inertial Measurement Unit) dysopia analyte detection false-alarm elimination method, which is characterized in that including:
Step 1：Scene image is acquired based on binocular vision system, including left view and right view；
Step 2：Image rectification is carried out to the left view and the right view；
Step 3：The left view after image rectification and the right view are subjected to Stereo matching, obtain dense parallax
Figure, the dense disparity map according to distance is layered, obtains several disparity maps and depth map for different location scene；
Step 4：Barrier calculating is carried out based on the depth map, obtains the first obstacle information；
Step 5：Based on the motion estimation information between the left view and consecutive frame image, from first barrier
False alarm information is rejected in information, obtains the second obstacle information.
Optionally, off-line calibration is further included before the step 1, specially：
Binocular camera may be used in the binocular vision system, and left camera is demarcated, and obtains the inside and outside ginseng of left camera
Number, then right camera is demarcated, the inside and outside parameter of right camera is obtained, finally, binocular camera is demarcated, obtains left and right phase
The rotation translation relation of machine.
Optionally, the step 2 includes：
Distortion correction is carried out to the left view and the right view, it is assumed that the benchmark image not distorted is f (x, y),
Image with larger geometric distortion is f (x ', y '), is distorted by the set between two images coordinate systemIt can obtain the image after distortion correction：
Wherein, n is polynomial coefficient, and i and j represent the specific location of pixel in the picture, aijAnd bijFor each term system
Number；
Polar curve correction is carried out to the left view and the right view, using left and right camera inner parameter matrix and
Bouguet method for correcting polar line makes the left view parallel with the correspondence polar curve of the right view.
Optionally, the dense specific acquisition process of view difference is：
Using the half global Stereo Matching Algorithm for accelerating pattern based on CUDA, to the left view after image rectification and institute
It states right view and carries out disparity computation, obtain preliminary disparity map, the step of half global Stereo Matching Algorithm includes Matching power flow
It calculates, cost polymerize and the left and right consistency check of parallax；
The preliminary disparity map is filtered refinement, obtains dense disparity map.
Optionally, the step 3 includes：
The dense disparity map is subjected to parallax segmentation, using the relationship between parallax and depth and parallax value with away from
Proportionate relationship between obtains several disparity maps for different scenes and the depth map.
Optionally, the step 4 is specially：
Contour detecting is carried out one by one to the depth map, obtains the barrier profile in different levels, returns to the first obstacle
Information { " Contour "：X, y, width, height }, wherein x, y represent the location of pixels coordinate in the profile upper left corner, correspond to respectively
Row and row in place；Width and height represents the width and height of profile respectively, and the unit of value is pixel.
Optionally, the step 5 includes：
Feature point extraction is carried out using Harris Corner Detections device, using KLT (Kannade Lucas Tomasi) algorithm
Feature point tracking is carried out, obtains radiation matrix and the movable informations such as transposed matrix between consecutive frame image；
T moment obstacle information is estimated according to the obstacle information at t-1 moment, judges the false-alarm region of t moment, and is picked
Except false alarm information.
Optionally, it is further included after the step 5：
According to the parameter of second obstacle information and the binocular vision system, position, the ruler of barrier are calculated
Very little and distance, and then obtain the required whole obstacle informations of avoidance.
In conclusion by adopting the above-described technical solution, the beneficial effects of the invention are as follows：
1. the present invention provides a kind of method based on binocular stereo vision detection of obstacles, obtained using binocular solid matching
The disparity map of scene is taken, and parallax refinement is carried out, then scene cut is carried out based on parallax by processing such as later stage filtering, based on wheel
Exterior feature detection determines obstacle target, can carry out three-dimensional reconstruction to obstacle target, obtain size, position and the distance of barrier
Etc. information, efficiently solve the problems, such as the visions avoidances such as pilotless automobile, auxiliary driving, unmanned plane and blind person.
2. CUDA speeding schemes are employed in the present invention in disparity computation, low in energy consumption, small size fast with processing speed
And the features such as portable, suitable for application scenarios such as unmanned, unmanned plane and guides.
3. being detected the present invention is based on dense disparity map to barrier profile or edge, parallax refinement, parallax are employed
Segmentation with contour detecting scheduling algorithm, improve the accuracy of detection of barrier, it is proportional to distance value using parallax value, by scene into
Row layering, obtains several disparity maps, carries out contour detecting to every width disparity map, can detect the obstacle positioned at different location
Object, and return to relevant information.
4. present invention utilizes the correlation between consecutive frame image, the false alarm information in obstacle information is rejected, is dropped significantly
Flase drop problem in low detection of obstacles further improves the correctness and validity of detection of obstacles algorithm.
Description of the drawings
Examples of the present invention will be described by way of reference to the accompanying drawings, wherein：
Fig. 1 is detection of obstacles false-alarm elimination method flow chart of steps in the embodiment of the present application；
Fig. 2 is the flow chart of disparity computation in the embodiment of the present application；
Fig. 3 is the relational graph of parallax and depth in the embodiment of the present application；
Fig. 4 is the flow chart that false-alarm is rejected in the embodiment of the present application.
Specific embodiment
All features or disclosed all methods disclosed in this specification or in the process the step of, in addition to mutually exclusive
Feature and/or step other than, can combine in any way.
It elaborates below in conjunction with the accompanying drawings to the present invention.
A kind of step flow of the dysopia analyte detection false-alarm elimination method based on IMU is provided in the embodiment of the present application
Figure, as shown in Figure 1：
Step 101：Scene image is acquired based on binocular vision system, including left view and right view；
Step 102：Image rectification is carried out to the left view and the right view；
Step 103：The left view after correction and the right view are subjected to Stereo matching, it is poor to obtain dense view,
The dense disparity map according to distance is layered, obtains several disparity maps and depth map for different location scene；
Step 104：Barrier calculating is carried out based on the depth map, obtains the first obstacle information；
Step 105：Based on the motion estimation information between the left view and consecutive frame image, from first barrier
False alarm information is rejected in information, obtains the second obstacle information.
The embodiment of the present application discloses a kind of dysopia analyte detection false-alarm elimination method based on IMU, passes through binocular vision
Feel system acquisition scene image；Then, it carries out distortion correction and polar curve to image to correct, the purpose of distortion correction of image is removal
The influence that optical distortion is brought, polar curve school be based on two image collecting device relative position relations to wherein piece image into
Row transformation so that same object point is located at the correction with a line in the view of left and right；Then, the left and right view after polar curve is corrected carries out
Stereo matching, the view for obtaining scene is poor, and the view difference is filtered, and except denoising, to obtain dense view poor；It is right
Dense view difference carries out contour detecting, and is described with rectangle frame, obtain rectangle frame upper left position and rectangle frame width with
High first obstacle information；Finally, based on the motion estimation information between left view and consecutive frame image, from the first barrier
False alarm information is rejected in information, obtains the second higher obstacle information of accuracy.
Further, due to two image collecting devices of binocular vision system setting when it is difficult to ensure that optical axis is strictly put down
Row, when acquiring scene image there are certain deviation, for example, the optical axis of binocular camera is located at camera internal, in assembling camera
When, very before detection of obstacles, certain adjustment is first carried out to binocular camera, ensures that camera optical axis is parallel.It needs to measure left
Baseline length between right camera optical axis, and the focal length of binocular camera is recorded, while ensure the synchronization of binocular camera acquisition image
Property.After the completion of binocular camera assembling, by gridiron pattern scaling board, using the plane reference algorithm of Zhang Zhengyou, binocular phase is obtained
Outer parameter between the respective intrinsic parameter of machine and two cameras, realizes building for binocular vision system, thus the step 101 it
Before further include off-line calibration：
Binocular camera may be used in the binocular vision system, and left camera is demarcated, and obtains the inside and outside ginseng of left camera
Number, then right camera is demarcated, the inside and outside parameter of right camera is obtained, finally, binocular camera is demarcated, obtains left and right phase
The rotation translation relation of machine.
Detailed process is, it is assumed that any point W=[X, Y, Z] in world coordinate systemT, point corresponding points on the plane of delineation again
For m=[u, v]T, the projection relation between object point and picture point is：
[u, v, 1]T=P [X, Y, Z, 1]T (1)
Wherein P is 3 × 4 projection matrix, can be represented by rotating with translation matrix：
P=A [R t] (2)
Wherein R is 3 × 3 spin matrixs, and t is translation vector, the external parameter of the two matrixes expression binocular vision, one
Represent position, an expression direction thus can determine that each position of the pixel in world coordinate system, wherein A on image
Matrix represents camera internal parameter matrix, can be expressed as：
(u in above formula0, v0) be image center coordinate；fuAnd fvRepresent that horizontal, vertical pixel unit represents respectively
Focal length, β represent obliquity factor.
So far, the intrinsic parameter of camera is obtained, the outer ginseng including focal length, picture centre and distortion parameter etc. and camera
Number includes spin matrix R and translation matrix T.Some parameters obtained during above-mentioned off-line calibration are in image rectification and barrier
There is application during calculating etc., output parameter is transmitted by the system image rectification and barrier computing module.
Further, the step 102 includes：
Distortion correction is carried out to the left view and the right view, it is assumed that the benchmark image not distorted is f
(x, y), the image with larger geometric distortion are g (x ', y '), are distorted by the set between two images coordinate systemIt can obtain the image after distortion correction：
Wherein, n is polynomial coefficient, and i and j represent the specific location of pixel in the picture, aijAnd bijFor each term system
Number；
Polar curve correction is carried out to the left view and the right view, using left and right camera inner parameter matrix and
Bouguet method for correcting polar line makes the left view parallel with the correspondence polar curve of the right view.
In the embodiment of the present application, two functions are included for image rectification：Distortion correction and polar curve correction.Assuming that do not have
The benchmark image of distortion is f (x, y), has the image of larger geometric distortion for g (x ', y '), between two images coordinate system
Set distortion can be expressed as：
Above-mentioned formula is represented with binary polynomial：
Wherein, n is polynomial coefficient, and i and j represent the specific location of pixel in the picture, aijAnd bijFor each term system
Number.The image of distortion correction has been obtained by above formula.
For the polar curve correct operation of image, according to the rotation of the left and right camera obtained in camera off-line correction and translation square
Battle array, it is assumed that left camera rotation translation matrix is R1And t1, the rotation translation matrix of right camera is R2And t2, rotation is with translating square
Battle array can be to obtain in off-line correction.Rotation and translation matrix based on left and right camera utilize the polar curve correction side of Bouguet
Method so that the correspondence polar curve of left and right camera image is parallel.The time complexity of Stereo matching is greatly reduced, simplifies parallaxometer
Calculation process.
Further, it in order to improve the accuracy of contour detecting, after image rectification is carried out, is calculated using half global registration
Method carries out disparity computation, and to obtain dense view poor, and the dense specific acquisition process of view difference is：
Using based on CUDA (computer Unified Device framework, Compute Unified Device Architecture)
The global Stereo Matching Algorithm of the half of acceleration pattern, parallaxometer is carried out to the left view after image rectification and the right view
It calculates, obtains preliminary disparity map, the half global Stereo Matching Algorithm specific steps include Matching power flow calculating, cost polymerization and regard
The left and right consistency check of difference；
The preliminary disparity map is filtered refinement, obtains dense disparity map.
The embodiment of the present application provides the flow chart of disparity computation, as shown in Figure 3：
Step 201：Left and right view, disparity computation are to carry out parallaxometer to having carried out the left and right view after image rectification
It calculates, it is poor to export dense view.
Step 202：Disparity computation.
In the embodiment of the present application, the disparity computation that disparity computation is employed in the present invention accelerated based on CDUA employs base
It, probably can be in three steps based on half global Stereo Matching Algorithm in the half global Stereo Matching Algorithm that CUDA accelerates：Matching
Cost calculates, cost polymerization and the left and right consistency check of parallax.
In half global Stereo Matching Algorithm, two kinds of calculating Matching power flow methods are common are, one kind is BT algorithms, another
Kind is the Matching power flow computational methods based on mutual information.We assume that half global Stereo Matching Algorithm, the cost in a direction
L (p, d) is expressed as, wherein p represents location of pixels, and d represents parallax size.So-called cost polymerization is exactly pixel p point all directions
Matching power flow summation, S (p, d)=∑ L (p, d) can be expressed as.Half global Stereo Matching Algorithm requires to obtain be exactly on
The minimum value of cost aggregate function is stated, corresponding parallax d is the parallax value at this time.Finally, the disparity map of acquisition is carried out
Left and right consistency check is blocked a little and Mismatching point to detect, it is ensured that the constraint of uniqueness, and cause the algorithm to light
According to variation influence it is insensitive, have very strong robustness to noise.
Step 203：Filtering refinement.
Due to consideration that binocular solid matching may due to noise jamming, block, weak texture and repeat the influences such as texture and lead
Error hiding is caused, this causes on initial parallax figure, and there are certain noises, cause the discontinuity and edge blurry of body surface.Cause
This, in the embodiment of the present application, after preliminary parallax is obtained, needs to carry out disparity map certain filtering refinement.To obtaining
Initial parallax figure be filtered refinement, realize reduce noise, smoothed image purpose.
Step 204：Dense disparity map after above-mentioned steps, can obtain dense disparity map.
Further, the step 3 includes：
The dense disparity map is subjected to parallax segmentation, using the relationship between parallax and depth and parallax value with away from
Proportionate relationship between obtains several disparity maps for different scenes and the depth map.
The relational graph of parallax and depth is additionally provided in the embodiment of the present application, the final purpose of binocular stereo vision is to ask for
The depth value z of target point, the parallax value based on acquisition can obtain final depth with the similar triangles relationship shown in upper figure
Value.B represents baseline length, and f represents the focal length of camera lens, and subpoints of the spatial point P on the camera of left and right is PLAnd PR, and 2 points
The coordinate on x is respectively X againLAnd XR, it is similar based on triangle, following relationship can be obtained：
Parallax d is the alternate position spike in P points again left and right view, is denoted as d=XL-XR, therefore above formula can be expressed as：
For binocular camera, what baseline length B and focal length f were to determine, parallax and depth are inversely.General parallax value
Value range in [0,255], represent farthest distance wherein 0 represents nearest distance, 255, therefore in order to by stereo scene
Different distance is divided into according to distance.Assuming that the detectable distance range of Binocular Stereo Vision System is [0.5m, 10.5m],
Corresponding parallax is [255,0].Detection of obstacles is usually that object closer to the distance is focal point, might as well set [0.5m,
4.5m] it is the range detected, corresponding disparity range is [255,151].First, it is dense disparity map intermediate value is complete less than 161
Portion is set as 0, has obtained new depth map D.Next, by the scene foundation in detection range apart from even partition, using 1m as one
A interval, respectively [0.5m, 1.5m], [1.5m, 2.5m], [2.5m, 3.5m] the corresponding depth map point with [3.5m, 4.5m]
It is segmented into [255,229], [229,203], [203,177] and [177,151].0 is all set to less than 229 to depth map D intermediate values,
The first width segmentation depth map D is obtained1；Then, depth map D intermediate values being set as less than 203 0 is obtained into disparity map D '1, by its
With D1It makes the difference, obtains the second width segmentation depth map D2Disparity map D intermediate values are set as 0 less than 177 again, obtain disparity map D '2,
With disparity map D '1It makes the difference, obtains disparity map D3；Finally, by disparity map D and disparity map D '2It makes the difference, obtains disparity map D4。
Further, it after obtaining several depth maps for different location scene by above-mentioned steps three, needs to carry out
Step 4 obtains obstacle information, and the step 4 is specially：
Contour detecting is carried out one by one to the depth map, obtains the barrier profile in different levels, returns to the first obstacle
Information { " Contour "：X, y, width, height }, wherein x, y represent the location of pixels coordinate in the profile upper left corner, correspond to respectively
Row and row in place；Width and height represents the width and height of profile respectively, and the unit of value is pixel.
In the embodiment of the present application, the first obstacle information is obtained by the contour detecting to depth map, it is assumed that pass through
Step 3 obtains the different segmentation depth map D of Same Scene1, D2, D3, D4.Contour detecting is carried out on every width depth map, is obtained
To the profile C in different levels1, C2…Cn.As for profile information, the information lattice that the profile each detected returns are defined
Formula is { " Contour "：X, y, width, height }, wherein x, y represent the location of pixels coordinate in the profile upper left corner, correspond to respectively
Row and row in place；Width and height represents the width and height of profile respectively, and the unit of value is pixel.
Further, it after the first obstacle information is obtained, according to the movement relation of consecutive frame image, will walk
Rapid five reject false-alarm, and the step 5 includes：
Feature point extraction is carried out using Harris Corner Detections device, using KLT (Kannade Lucas Tomasi) algorithm
Feature point tracking is carried out, obtains radiation matrix and the movable informations such as transposed matrix between consecutive frame image；
T moment obstacle information is estimated according to the obstacle information at t-1 moment, judges the false-alarm region of t moment, and is picked
Except false alarm information.
The profile obtained on depth map is looked at as barrier region, however, as influences such as noise, errors,
Cause barrier region detection mistake.The present invention is based on the motion estimation information between left view and consecutive frame image,
False barrier in detection of obstacles is rejected.False-alarm barrier rejects the fortune that the adjacent interframe of image is mainly utilized
Dynamic estimation, and estimation is broadly divided into characteristic point detection and feature point tracking two parts.
The embodiment of the present application provides the flow chart that a kind of false-alarm is rejected, as shown in Figure 4：
Step 201：Barrier calculating is carried out to the dense disparity map of t moment dense disparity map and t-1 moment, when obtaining t
Carve the obstacle information with the t-1 moment.
The detailed process that barrier calculates is described in detail at step 104, repeats no more again.
Step 202：The basic calculatings such as estimation are carried out to the left view of t moment and t-1 moment, according to the t-1 moment
Obstacle information estimates the obstacle information of t moment, believes with reference to t moment barrier estimated information and t moment barrier
Breath rejects barrier during false t.
The feature point extraction algorithm used this feature point detecting method speed and has for Harris Corner Detection devices
Good robustness.After the characteristic point in extracting a frame image, it is necessary to tracking and matching is carried out in subsequent image, is gone back
Potential corresponding points.The tracking and matching algorithm that the present invention uses is KLT (Kannade-Lucas-Tomasi) algorithm.It is assuming that continuous
Two field pictures are that the position in I (x, y) and J (x, y) meets：
J (Ax+d)=I (x) (8)
Assuming thatRepresent 2 × 2 radiation transformation matrix, d=[d1 d2]TFor translation matrix.It is in addition, false
If the characteristic window of I (x, y) is A (x), the characteristic window of J (x, y) is B (x), then has：
∈=∫ ∫W[B(Ax+d)-A(x)]2w(x)dx (9)
When upper formula is minimized, by derivation mode, affine matrix A and transposed matrix d can be solved.
After obtaining the movable informations such as affine matrix and the transposed matrix of adjacent interframe, obtained according to barrier computing module
The obstacle information at t-1 moment estimates the obstacle information of t moment.Assuming that the left view of input and disparity map are for t-1
Moment and the correspondence image of t moment.Take certain barrier region P engraved during t-11(x, y), wherein (x, y) represents a profile left side
The coordinate value at upper angle.Then, it is projected into corresponding positions on t moment image according to affine and transposed matrix calculated above
It puts.It is denoted as P '1(x, y).Take the barrier region P on t moment image2(x, y), and during by the barrier region and t at t-1 moment
All barrier regions carved make the difference, and such as scheme
T=| P '1(x, y)-P2(x, y) |
Wherein, T represents the corresponding threshold value in corresponding region.In addition, we also set a decision threshold Tre, work as calculating
The threshold value T of gained>During Tre, then the region is false-alarm region, is deleted in profile list.Conversely, the region is barrier
Region.The false-alarm problem in detection of obstacles is greatly reduced in this way.
Step 203：Obtain barrier during accurate t.
So far, the corresponding obstacle information of t moment image has all carried out false-alarm and has rejected operation, eliminates false barrier
Information.
Further, by having obtained rejecting the second obstacle information of false alarm information after above-mentioned steps, in the step
It is further included after five：
According to second obstacle information and the parameter of biocular systems, position, size and the distance of barrier are calculated,
And then obtain the required whole obstacle informations of avoidance.
Each differently contoured coordinate and dimension information based on above-mentioned acquisition, the above parameter of biocular systems and acquisition
Dense disparity map D, some positions, size and the range information of barrier are easy to calculate and obtain in scene.With wherein some
For profile C, top left co-ordinate is (x0, y0), width w0, it is highly h0.And then the profile upper right corner and wheel can be obtained
The image coordinate of wide central point is (x0+w0, y0) andFirst according to formula (7) parallax and apart from it
Between position relationship, the parallax value of each point is read from disparity map, and then the upper left of profile can be obtained according to the coordinate of image
The distance between angle scene corresponding with the upper right corner and center and binocular camera Z0,Z1,Z'。
Further according to formula (1) [u, v, 1]T=P [X, Y, Z, 1]T, the variation between world coordinate system and image coordinate system can
To calculate the profile upper left corner and coordinate (X of the upper right corner in world coordinate system respectively0, Y0, Z0) and (X1, Y1Z1).So far, should
Object information in profile all obtains, and is defined as { " Obstacle "：xL, xR, Width, Height, Distance },
In, xLRepresent lateral distance of the barrier leftmost side apart from camera, xL=X0；xRRepresent horizontal stroke of the barrier rightmost side apart from camera
To distance, xR=X1；Width represents the width of barrier, Width=X1-X0；Height represents the height of barrier, Height
=Y0=Y1；Distance represents the axial distance of obstacle distance camera, Distance=Z'.And then required for acquisition avoidance
Whole obstacle informations.
In conclusion the application implements the flase drop problem in view of in the detection of obstacles of binocular stereo vision, the present invention carries
A kind of dysopia analyte detection false-alarm elimination method based on IMU is supplied, this method operation is simple, and real-time is good, can be fine
By in detection of obstacles false-alarm barrier reject.Detection of this method for static-obstacle thing in scene and moving obstacle
It is applicable in, pedestrian, vehicle and barrier can be evaded for the application scenarios such as unmanned, Navigation of Pilotless Aircraft and blind person's guide and provided
Foundation.
The above embodiments merely illustrate the technical concept and features of the present invention, and its object is to allow person skilled in the art
Scholar can understand present disclosure and be implemented, and it is not intended to limit the scope of the present invention, all according to the present invention
The equivalent change or modification that spiritual spirit is made should all cover in the protection domain of invention.
Claims (8)
1. a kind of dysopia analyte detection false-alarm elimination method based on IMU, which is characterized in that including:
Step 1：Scene image is acquired based on binocular vision system, including left view and right view；
Step 2：Image rectification is carried out to the left view and the right view；
Step 3：The left view after image rectification and the right view are subjected to Stereo matching, obtain dense disparity map, it will
The dense disparity map is layered according to distance, obtains several disparity maps and depth map for different location scene；
Step 4：Barrier calculating is carried out based on the depth map, obtains the first obstacle information；
Step 5：Based on the motion estimation information between the left view and consecutive frame image, from first obstacle information
Middle rejecting false alarm information, obtains the second obstacle information.
2. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that described
Off-line calibration is further included before step 1, specially：
Binocular camera may be used in the binocular vision system, and left camera is demarcated, and obtains the inside and outside parameter of left camera, then
Right camera is demarcated, obtains the inside and outside parameter of right camera, finally, binocular camera is demarcated, obtains left and right camera
Rotate translation relation.
3. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that the step
Rapid two include：
Distortion correction is carried out to the left view and the right view, it is assumed that the benchmark image not distorted is f (x, y), is had
The image of larger geometric distortion is g (x ', y '), is distorted by the set between two images coordinate systemIt can
To obtain the image after distortion correction：
Wherein, n is polynomial coefficient, and i and j represent the specific location of pixel in the picture, aijAnd bijFor each term coefficient；
Polar curve correction is carried out to the left view and the right view, utilizes the inner parameter matrix and Bouguet of left and right camera
Method for correcting polar line makes the left view parallel with the correspondence polar curve of the right view.
4. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that described thick
Specifically acquisition process is close view difference：
Using the half global Stereo Matching Algorithm for accelerating pattern based on CUDA, to the left view after image rectification and the right side
The step of view carries out disparity computation, obtains preliminary disparity map, the half global Stereo Matching Algorithm include Matching power flow calculate,
Cost polymerize and the left and right consistency check of parallax；
The preliminary disparity map is filtered refinement, obtains dense disparity map.
5. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that the step
Rapid three include：
The dense disparity map is subjected to parallax segmentation, using the relationship between parallax and depth and parallax value and apart from it
Between proportionate relationship, obtain several disparity maps for different scenes and the depth map.
6. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that the step
Rapid four are specially：
Contour detecting is carried out one by one to the depth map, obtains the barrier profile in different levels, returns to the first complaint message
{“Contour”：X, y, width, height }, wherein x, y represent the location of pixels coordinate in the profile upper left corner, correspond respectively to institute
Row and row；Width and height represents the width and height of profile respectively, and the unit of value is pixel.
7. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that the step
Rapid five include：
Feature point extraction is carried out using Harris Corner Detections device, feature point tracking is carried out using KLT algorithms, obtains consecutive frame figure
The movable informations such as radiation matrix and transposed matrix as between；
T moment obstacle information is estimated according to the obstacle information at t-1 moment, judges the false-alarm region of t moment, and rejects void
Alert information.
8. the dysopia analyte detection false-alarm elimination method based on IMU according to claim 1, which is characterized in that described
It is further included after step 5：
According to the parameter of second obstacle information and the binocular vision system, calculate the position of barrier, size and
Distance, and then obtain the required whole obstacle informations of avoidance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810063615.7A CN108230392A (en) | 2018-01-23 | 2018-01-23 | A kind of dysopia analyte detection false-alarm elimination method based on IMU |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810063615.7A CN108230392A (en) | 2018-01-23 | 2018-01-23 | A kind of dysopia analyte detection false-alarm elimination method based on IMU |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108230392A true CN108230392A (en) | 2018-06-29 |
Family
ID=62668752
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810063615.7A Pending CN108230392A (en) | 2018-01-23 | 2018-01-23 | A kind of dysopia analyte detection false-alarm elimination method based on IMU |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108230392A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034018A (en) * | 2018-07-12 | 2018-12-18 | 北京航空航天大学 | A kind of low latitude small drone method for barrier perception based on binocular vision |
| CN109344687A (en) * | 2018-08-06 | 2019-02-15 | 深圳拓邦股份有限公司 | The obstacle detection method of view-based access control model, device, mobile device |
| CN109410234A (en) * | 2018-10-12 | 2019-03-01 | 南京理工大学 | A kind of control method and control system based on binocular vision avoidance |
| CN110765922A (en) * | 2019-10-18 | 2020-02-07 | 华南理工大学 | AGV is with two mesh vision object detection barrier systems |
| CN110826512A (en) * | 2019-11-12 | 2020-02-21 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| CN110857859A (en) * | 2018-08-23 | 2020-03-03 | 杭州海康机器人技术有限公司 | Obstacle detection method and device |
| CN111145254A (en) * | 2019-12-13 | 2020-05-12 | 上海新时达机器人有限公司 | Door valve blank positioning method based on binocular vision |
| CN111382591A (en) * | 2018-12-27 | 2020-07-07 | 海信集团有限公司 | Binocular camera ranging correction method and vehicle-mounted equipment |
| CN111709938A (en) * | 2020-06-18 | 2020-09-25 | 武汉唯理科技有限公司 | Depth map-based pavement defect and projectile detection method |
| CN111898396A (en) * | 2019-05-06 | 2020-11-06 | 北京四维图新科技股份有限公司 | Obstacle detection method and device |
| CN112116644A (en) * | 2020-08-28 | 2020-12-22 | 辽宁石油化工大学 | Vision-based obstacle detection method and device and obstacle distance calculation method and device |
| CN112254755A (en) * | 2020-11-11 | 2021-01-22 | 北京邮电大学 | Measurement signal processing method and device, electronic equipment and readable storage medium |
| CN112837207A (en) * | 2019-11-25 | 2021-05-25 | 影石创新科技股份有限公司 | Panoramic depth measuring method, four-eye fisheye camera and binocular fisheye camera |
| CN112906479A (en) * | 2021-01-22 | 2021-06-04 | 成都纵横自动化技术股份有限公司 | Unmanned aerial vehicle auxiliary landing method and system |
| CN113157083A (en) * | 2020-01-23 | 2021-07-23 | 海信视像科技股份有限公司 | Obstacle display method and device in virtual reality |
| CN114119700A (en) * | 2021-11-26 | 2022-03-01 | 山东科技大学 | Obstacle ranging method based on U-V disparity map |
| CN114241441A (en) * | 2021-12-03 | 2022-03-25 | 北京工业大学 | Dynamic obstacle detection method based on feature points |
| CN114371693A (en) * | 2021-11-26 | 2022-04-19 | 江苏金晓电子信息股份有限公司 | Traffic cruise controller control method based on multi-sensor data fusion under multiple scenes |
| CN116755562A (en) * | 2023-07-04 | 2023-09-15 | 深圳市仙瞬科技有限公司 | Obstacle avoidance method, device, medium and AR/VR equipment |
| WO2024022337A1 (en) * | 2022-07-29 | 2024-02-01 | 松灵机器人(深圳)有限公司 | Obstacle detection method and apparatus, mowing robot, and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065351A (en) * | 2012-12-16 | 2013-04-24 | 华南理工大学 | Binocular three-dimensional reconstruction method |
| CN103440653A (en) * | 2013-08-27 | 2013-12-11 | 北京航空航天大学 | Binocular vision stereo matching method |
| CN104636763A (en) * | 2014-12-01 | 2015-05-20 | 北京工业大学 | Road and obstacle detecting method based on remotely piloted vehicles |
-
2018
- 2018-01-23 CN CN201810063615.7A patent/CN108230392A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065351A (en) * | 2012-12-16 | 2013-04-24 | 华南理工大学 | Binocular three-dimensional reconstruction method |
| CN103440653A (en) * | 2013-08-27 | 2013-12-11 | 北京航空航天大学 | Binocular vision stereo matching method |
| CN104636763A (en) * | 2014-12-01 | 2015-05-20 | 北京工业大学 | Road and obstacle detecting method based on remotely piloted vehicles |
Non-Patent Citations (1)
| Title |
|---|
| 顾骋等: "基于双目立体视觉的快速人头检测方法", 《中国激光》 * |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034018B (en) * | 2018-07-12 | 2022-01-21 | 北京航空航天大学 | Low-altitude small unmanned aerial vehicle obstacle sensing method based on binocular vision |
| CN109034018A (en) * | 2018-07-12 | 2018-12-18 | 北京航空航天大学 | A kind of low latitude small drone method for barrier perception based on binocular vision |
| CN109344687A (en) * | 2018-08-06 | 2019-02-15 | 深圳拓邦股份有限公司 | The obstacle detection method of view-based access control model, device, mobile device |
| CN109344687B (en) * | 2018-08-06 | 2021-04-16 | 深圳拓邦股份有限公司 | Vision-based obstacle detection method and device and mobile device |
| CN110857859B (en) * | 2018-08-23 | 2022-02-08 | 杭州海康机器人技术有限公司 | Obstacle detection method and device |
| CN110857859A (en) * | 2018-08-23 | 2020-03-03 | 杭州海康机器人技术有限公司 | Obstacle detection method and device |
| CN109410234A (en) * | 2018-10-12 | 2019-03-01 | 南京理工大学 | A kind of control method and control system based on binocular vision avoidance |
| CN111382591B (en) * | 2018-12-27 | 2023-09-29 | 海信集团有限公司 | Binocular camera ranging correction method and vehicle-mounted equipment |
| CN111382591A (en) * | 2018-12-27 | 2020-07-07 | 海信集团有限公司 | Binocular camera ranging correction method and vehicle-mounted equipment |
| CN111898396A (en) * | 2019-05-06 | 2020-11-06 | 北京四维图新科技股份有限公司 | Obstacle detection method and device |
| CN110765922B (en) * | 2019-10-18 | 2023-05-02 | 华南理工大学 | Binocular vision object detection obstacle system for AGV |
| CN110765922A (en) * | 2019-10-18 | 2020-02-07 | 华南理工大学 | AGV is with two mesh vision object detection barrier systems |
| CN110826512B (en) * | 2019-11-12 | 2022-03-08 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| CN110826512A (en) * | 2019-11-12 | 2020-02-21 | 深圳创维数字技术有限公司 | Ground obstacle detection method, ground obstacle detection device, and computer-readable storage medium |
| CN112837207A (en) * | 2019-11-25 | 2021-05-25 | 影石创新科技股份有限公司 | Panoramic depth measuring method, four-eye fisheye camera and binocular fisheye camera |
| WO2021104308A1 (en) * | 2019-11-25 | 2021-06-03 | 影石创新科技股份有限公司 | Panoramic depth measurement method, four-eye fisheye camera, and binocular fisheye camera |
| CN111145254B (en) * | 2019-12-13 | 2023-08-11 | 上海新时达机器人有限公司 | Door valve blank positioning method based on binocular vision |
| CN111145254A (en) * | 2019-12-13 | 2020-05-12 | 上海新时达机器人有限公司 | Door valve blank positioning method based on binocular vision |
| CN113157083A (en) * | 2020-01-23 | 2021-07-23 | 海信视像科技股份有限公司 | Obstacle display method and device in virtual reality |
| CN111709938A (en) * | 2020-06-18 | 2020-09-25 | 武汉唯理科技有限公司 | Depth map-based pavement defect and projectile detection method |
| CN112116644A (en) * | 2020-08-28 | 2020-12-22 | 辽宁石油化工大学 | Vision-based obstacle detection method and device and obstacle distance calculation method and device |
| CN112254755A (en) * | 2020-11-11 | 2021-01-22 | 北京邮电大学 | Measurement signal processing method and device, electronic equipment and readable storage medium |
| CN112906479A (en) * | 2021-01-22 | 2021-06-04 | 成都纵横自动化技术股份有限公司 | Unmanned aerial vehicle auxiliary landing method and system |
| CN112906479B (en) * | 2021-01-22 | 2024-01-26 | 成都纵横自动化技术股份有限公司 | Unmanned aerial vehicle auxiliary landing method and system thereof |
| CN114371693A (en) * | 2021-11-26 | 2022-04-19 | 江苏金晓电子信息股份有限公司 | Traffic cruise controller control method based on multi-sensor data fusion under multiple scenes |
| CN114119700B (en) * | 2021-11-26 | 2024-03-29 | 山东科技大学 | Obstacle ranging method based on U-V disparity map |
| CN114119700A (en) * | 2021-11-26 | 2022-03-01 | 山东科技大学 | Obstacle ranging method based on U-V disparity map |
| CN114241441B (en) * | 2021-12-03 | 2024-03-29 | 北京工业大学 | Dynamic obstacle detection method based on feature points |
| CN114241441A (en) * | 2021-12-03 | 2022-03-25 | 北京工业大学 | Dynamic obstacle detection method based on feature points |
| WO2024022337A1 (en) * | 2022-07-29 | 2024-02-01 | 松灵机器人(深圳)有限公司 | Obstacle detection method and apparatus, mowing robot, and storage medium |
| CN116755562A (en) * | 2023-07-04 | 2023-09-15 | 深圳市仙瞬科技有限公司 | Obstacle avoidance method, device, medium and AR/VR equipment |
| CN116755562B (en) * | 2023-07-04 | 2024-04-05 | 深圳市仙瞬科技有限公司 | Obstacle avoidance method, device, medium and AR/VR equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108230392A (en) | A kind of dysopia analyte detection false-alarm elimination method based on IMU | |
| CN110569704B (en) | Multi-strategy self-adaptive lane line detection method based on stereoscopic vision | |
| CN105225482B (en) | Vehicle detecting system and method based on binocular stereo vision | |
| US8326025B2 (en) | Method for determining a depth map from images, device for determining a depth map | |
| Broggi et al. | The single frame stereo vision system for reliable obstacle detection used during the 2005 DARPA grand challenge on TerraMax | |
| CN106960454B (en) | Depth of field obstacle avoidance method and equipment and unmanned aerial vehicle | |
| Salih et al. | Depth and geometry from a single 2d image using triangulation | |
| CN106033614B (en) | A kind of mobile camera motion object detection method under strong parallax | |
| WO2007142643A1 (en) | Two pass approach to three dimensional reconstruction | |
| CN109447908A (en) | A kind of coil of strip recognition positioning method based on stereoscopic vision | |
| Munoz-Banon et al. | Targetless camera-lidar calibration in unstructured environments | |
| KR20150074544A (en) | Method of tracking vehicle | |
| CN107560592A (en) | A kind of precision ranging method for optronic tracker linkage target | |
| CN112344923A (en) | Robot positioning method and positioning device thereof | |
| Jung et al. | Object detection and tracking-based camera calibration for normalized human height estimation | |
| Jung et al. | A novel 2.5 D pattern for extrinsic calibration of tof and camera fusion system | |
| KR20170014916A (en) | Method for calculating an object's coordinates in an image using single camera and gps | |
| CN108230351A (en) | Sales counter evaluation method and system based on binocular stereo vision pedestrian detection | |
| CN111723778A (en) | Vehicle distance measuring system and method based on MobileNet-SSD | |
| KR100574227B1 (en) | Apparatus and method for separating object motion from camera motion | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera | |
| CN114998532B (en) | Three-dimensional image visual transmission optimization method based on digital image reconstruction | |
| Um et al. | Three-dimensional scene reconstruction using multiview images and depth camera | |
| CN116128966A (en) | Semantic positioning method based on environmental object | |
| Pfeiffer et al. | Ground truth evaluation of the Stixel representation using laser scanners |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180629 |
|
| WD01 | Invention patent application deemed withdrawn after publication |