CN108243623A - Vehicle anticollision method for early warning and system based on binocular stereo vision - Google Patents
Vehicle anticollision method for early warning and system based on binocular stereo vision Download PDFInfo
- Publication number
- CN108243623A CN108243623A CN201680001426.6A CN201680001426A CN108243623A CN 108243623 A CN108243623 A CN 108243623A CN 201680001426 A CN201680001426 A CN 201680001426A CN 108243623 A CN108243623 A CN 108243623A
- Authority
- CN
- China
- Prior art keywords
- straight line
- map
- model
- point
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Vehicle anticollision method for early warning and system based on stereoscopic vision, including：Disparity map is obtained by the binocular camera carried on vehicle body；V disparity maps are obtained from disparity map；To V disparity map binaryzations；Using RANSAC methods segmented linear is obtained to be fitted from the point of V disparity maps；According to multiple image smothing filtering straight line；And the straight line by being extracted obtains the wheeled region in former gray level image；Three-dimensional coordinate of the point for belonging to ground in real world coordinates system is calculated, floor model is fitted using RANSAC；Entire scene is transformed into world coordinates by camera coordinates, when generate plan view, asked for occupying map by plan view；The position of each barrier is obtained, and barrier is calculated to the distance of this vehicle by disparity map from segmentation in map is occupied；Current vehicle distance is alarmed when being less than certain threshold value or further decision.Various road surfaces and road conditions are adapted to, it is low on disparity map required precision, do not depend on caused by data and engineer's feature influence.
Description
Technical field
The present invention relates generally to automatic driving technology, relates more specifically to the automobile based on binocular stereo vision and prevents
Impacting technology.
Background technology
Accurate anticollision early warning in real time has important application value, drives especially in the warning of auxiliary driving safety and automatically
Decisive role is played, such as in automatic Pilot during that sails automatically control, anticollision early warning can reduce thing as much as possible
Therefore avoid the person and property loss；In automatic Pilot, anticollision early warning is more accurate, and safety is higher.
At present, mainly have for anticollision method for early warning, first, based on laser radar sensor or millimetre-wave radar, first
It is demarcated, is ground to the region decision for being less than certain threshold value, laser radar cost is very high needed for this method, it is difficult to which universal make
It is far from laser radar height with, millimeter wave precision；Second is that using monocular colour imagery shot, pass through machine learning and computer vision
Method detect front obstacle, this method depends critically upon trained sample and the feature of engineer, wheeled region
It is multifarious, it encounters situation about being not present in training sample and then can not be detected, autgmentability, versatility be not strong, on the other hand, single
Mesh camera can not accurately obtain depth information, and obtained result does not often meet real scene, last this method real-time
It is difficult to ensure.
In recent years it has been proposed that some automotive safeties for being based on machine vision (including monocular vision and stereoscopic vision) are driven
Sail technology.
Patent document 1CN101135558B discloses a kind of vehicle anti-collision early warning technology based on machine vision, wherein adopting
Front vehicles vehicle license plate characteristic and lane line information are acquired with the method for machine vision, is regarded according to its front vehicles car plate in machine
How much sizes of the projection imaging pixel of feel carry out the shapes such as the calculating with front vehicles distance, speed, steering with reference to this vehicle
State information calculates the transport condition of front truck, according to the relative distance of Ben Che and track line boundary, judges whether traveling in safety
Track within the scope of etc..
The method of the machine learning of patent document 1 is detected, and depends critically upon trained sample and the spy of engineer
It levies, the scene areas encountered in traveling is multifarious, encounters situation about being not present in training sample and then can not be detected, and extends
Property, versatility be not strong；On the other hand, monocular camera can not accurately obtain depth information and velocity information, and obtained result is past
It is past not meet real scene；Last this method real-time is also difficult to ensure.
Patent document 2CN102685516A discloses a kind of active safety formula auxiliary driving side based on stereovision technique
Method.The active safety formula DAS (Driver Assistant System) comprehensively utilizes ray machine teleinformatic technique, by stereoscopic vision subsystem, image quickly
Subsystem and safety assistant driving subsystem composition are managed, is sensed including two high resolution CCD video cameras, ambient light illumination
The quick Processing Algorithm library of device, two-pass video capture card, isochronous controller, data transmission circuit, power supply circuit, image, voice
Reminding module, screen display module and active safe driving control module etc..Under various weather conditions, shunting is identified in real time
The parameters such as relative distance, relative velocity and the relative acceleration of the risk objects such as line, front vehicles, bicycle, pedestrian, pass through language
Sound prompts the counter-measure that driver takes, and realizes automatic retarding and emergency brake under emergency situation, ensures to drive a vehicle round-the-clockly
Safety.
Patent document 2 is also detected with the knowledge method for distinguishing based on machine learning, depend critically upon trained sample and
The feature of engineer, the scene areas encountered in traveling is multifarious, encounters situation about being not present in training sample and then detects
It does not come out, autgmentability, versatility be not strong, but can reach more accurate result by binocular on distance calculates.
Need that versatility is stronger, real-time is stronger, training data and engineer's feature are relied on small is regarded based on machine
The vehicle anticollision early warning technology of feel.
Invention content
In view of the above circumstances, it is proposed that the present invention.
According to an aspect of the invention, there is provided a kind of vehicle anticollision method for early warning based on binocular stereo vision,
It can include：It shoots to obtain the left and right of the vehicle front along automobile direction of travel by the binocular camera carried on body of a motor car
Two gray level images, are calculated disparity map；V disparity maps are converted to from disparity map；Binaryzation is carried out to V disparity maps；It uses
RANSAC methods obtain segmented linear to be fitted from the point of the V disparity maps after binaryzation；
According to multiple image smothing filtering straight line；And the straight line by being extracted obtains the wheeled in former gray level image
Region；According to original image and disparity map, three-dimensional coordinate of the point for belonging to ground in real world coordinates system is calculated, it is assumed that
Ground is areal model, is fitted the plane using RANSAC, obtains ground model；By the entire scene in original-gray image
World coordinates is transformed into, while generate plan view by camera coordinates, is asked for occupying map by plan view；Pass through from occupying in map
Connected component labeling detection algorithm is divided to obtain the position of each barrier, and be transformed into original image and be marked, and pass through
Disparity map calculates barrier to the distance of this vehicle；Decision model is alarmed or be passed to current vehicle distance when being less than certain threshold value
Block participative decision making.
According to above-mentioned vehicle anticollision method for early warning, asking for occupying map by plan view can include：It is first depending on each
The world coordinates of a point, the point that will be above ground first threshold height extract, these points are transformed into earth axes；
It will be above first threshold height and the point less than second threshold height mark in map is occupied；Occupy each pixel in map
Value is that the height of its respective point adds up and thus obtains occupying map.
It is described to be divided by connected component labeling detection algorithm from occupying in map according to above-mentioned vehicle anticollision method for early warning
Obtaining the position of each barrier can include：Size according to each pixel value in map is occupied is converted into colour and carries
The image of label so that the pixel value the big more is biased to red, and pixel value is smaller to be more partial to blue, is detected by connected component labeling
Algorithm is partitioned into different objects, thus obtains the position of each barrier.
According to above-mentioned vehicle anticollision method for early warning, wherein, it is described that V disparity maps progress binaryzation can be included：It asks
The maximum value of every one-line pixel value is taken, by the gray value of pixel is set as 255 only residing for maximum value in every a line, rest of pixels ash
Angle value is set as 0.
According to above-mentioned vehicle anticollision method for early warning, can be included to be fitted one section of segmented linear using RANSAC methods：
Operations described below sequence is performed repeatedly, until reaching predetermined ending standard：One group in maximum of points in selection V disparity maps is random
Subset carries out fitting a straight line, obtains straight line model；It is gone to test all other data with obtained straight line model, if some
Point is suitable for the straight line model of estimation, it is believed that and it is also intra-office point, and intra-office point is classified as if there is exceeding predetermined number point of destination,
The model so estimated is taken as reasonably, then reevaluates model with all intra-office points, and estimate intra-office point and model
Error rate；If the error rate of model substitutes current best model less than current best model with the model；With most
The best model obtained afterwards is as this section of segmented linear.
It is described to be wrapped using RANSAC methods to be fitted multistage segmented linear according to above-mentioned vehicle anticollision method for early warning
It includes：Straight line is extracted first, in accordance with the above method, after extraction, the point of first straight line will be belonged to from V disparity maps
Then removal extracts Article 2 straight line for remaining point method according to the method for claim 5, so goes down repeatedly, directly
Number to remaining point is less than predetermined threshold.
It is described to be included according to multiple image smothing filtering straight line according to above-mentioned vehicle anticollision method for early warning：Setting
One time window, it is assumed that straight line model is expressed as ax+by+c=0, straight line model parameter is obtained to every frame image, for each
Parameter adds up to every frame, when often carrying out the new image of a frame, a frame image being subtracted most since cumulative parametric results
Straight line model parameter along with the straight line model parameter of current frame image, then is averaging the straight line model parameter as this frame.
According to above-mentioned vehicle anticollision method for early warning, it is described obtained by the straight line extracted it is feasible in former gray level image
Region is sailed to include：For every a line in V disparity maps, point that the parallax value on the straight line of selection and withdrawal is d is corresponded in disparity map
Row in, compare the parallax value of each pixel and the difference of d, when difference be less than certain threshold value when, then by artwork corresponding position
It is determined as safe wheeled region.
According to another aspect of the present invention, a kind of vehicle anticollision early warning system based on binocular stereo vision is provided,
It can include：Binocular camera, lasting shooting obtain opening gray level image along the left and right two of the vehicle front of vehicle traveling direction；Meter
Device is calculated, including memory, processor, communication interface, bus, memory, communication interface and processor are all connected to bus, deposit
Computer executable instructions are stored in reservoir, computing device can obtain what binocular camera was shot via communication interface
Two gray level images in left and right when processor performs the computer executable instructions, perform following methods：Based on left and right two
Disparity map is calculated in gray level image；V disparity maps are converted to from disparity map；Binaryzation is carried out to V disparity maps；It uses
RANSAC methods come come from the point of the V disparity maps after binaryzation fitting obtain segmented linear；According to multiple image smothing filtering
Straight line；The wheeled region in former gray level image is obtained by the straight line extracted；According to original image and disparity map, calculate
Belong to three-dimensional coordinate of the point on ground in real world coordinates system, it is assumed that ground is areal model, is fitted using RANSAC
The plane, obtains ground model；Entire scene in original-gray image is transformed into world coordinates by camera coordinates, it is raw simultaneously
Into plan view, asked for occupying map by plan view；From occupy in map divide to obtain by connected component labeling detection algorithm it is each
The position of barrier, and be transformed into original image and be marked, and barrier is calculated to the distance of this vehicle by disparity map,
Decision-making module participative decision making is alarmed or be passed to current vehicle distance when being less than certain threshold value.
According to above-mentioned vehicle anticollision early warning system, asking for occupying map by plan view can include：It is first depending on each
The world coordinates of point, the point that will be above ground first threshold height extract, these points are transformed into earth axes；It will
It is marked in map is occupied higher than first threshold height and less than the point of second threshold height；Occupy the value of each pixel in map
Be its respective point height add up and, thus obtain occupying map.
It is described to be divided by connected component labeling detection algorithm from occupying in map according to above-mentioned vehicle anticollision early warning system
Obtaining the position of each barrier can include：Size according to each pixel value in map is occupied is converted into colour and carries
The image of label so that the pixel value the big more is biased to red, and pixel value is smaller to be more partial to blue, is detected by connected component labeling
Algorithm is partitioned into different objects, thus obtains the position of each barrier.
According to above-mentioned vehicle anticollision early warning system, wherein, it is described that V disparity maps progress binaryzation is included：It asks for each
The maximum value of row pixel value, by the gray value of pixel is set as 255 only residing for maximum value in every a line, rest of pixels gray value is set
It is set to 0.
It is described to be wrapped using RANSAC methods to be fitted one section of segmented linear according to above-mentioned vehicle anticollision early warning system
It includes：Operations described below sequence is performed repeatedly, until reaching predetermined exits standard：Select one group in the maximum of points in V disparity maps
Random subset carries out fitting a straight line, obtains straight line model；It is gone to test all other data with obtained straight line model, if
Some point is suitable for the straight line model of estimation, it is believed that it is also intra-office point, and office is classified as if there is exceeding predetermined number point of destination
Interior point, then the model of estimation is taken as reasonably, then reevaluates model with all intra-office points, and estimate intra-office point with
The error rate of model；If the error rate of model substitutes current best model less than current best model with the model；
Using the best model that finally obtains as this section of segmented linear.
It is described to be wrapped using RANSAC methods to be fitted multistage segmented linear according to above-mentioned vehicle anticollision early warning system
It includes：Straight line is extracted, after extraction, the point for belonging to first straight line is removed from V disparity maps, then for remaining
It puts to extract Article 2 straight line, so go down repeatedly, until the number of remaining point is less than predetermined threshold.
It is described to be included according to multiple image smothing filtering straight line according to above-mentioned vehicle anticollision early warning system：Setting
One time window, it is assumed that straight line model is expressed as ax+by+c=0, straight line model parameter is obtained to every frame image, for each
Parameter adds up to every frame, when often carrying out the new image of a frame, a frame image being subtracted most since cumulative parametric results
Straight line model parameter along with the straight line model parameter of current frame image, then is averaging the straight line model parameter as this frame.
According to above-mentioned vehicle anticollision early warning system, it is described obtained by the straight line extracted it is feasible in former gray level image
Sailing region can include：For every a line in V disparity maps, the parallax value on the straight line of selection and withdrawal is d, right in disparity map
In the row answered, compare the parallax value of each pixel and the difference of d, when difference is less than certain threshold value, then artwork is corresponded into position
It puts and is determined as safe wheeled region.
In accordance with a further aspect of the present invention, a kind of vehicle anticollision early warning system, can include：Binocular camera is configured to
Shooting obtains opening gray level image along the left and right two of the vehicle front of automobile direction of travel；Disparity map calculating unit, from left and right two
Disparity map is calculated in gray level image；V disparity map modular converters are converted to V disparity maps from disparity map；Binarization block, to V
Disparity map carries out binaryzation；RANSAC fitting a straight line modules, using RANSAC methods come from the point of the V disparity maps after binaryzation
Fitting obtains segmented linear；Multiple image filter module, according to multiple image smothing filtering straight line；Original image wheeled region is true
Cover half block obtains the wheeled region in former gray level image by the straight line extracted；Ground model fitting module, according to original
Image and disparity map calculate three-dimensional coordinate of the point for belonging to ground in real world coordinates system, it is assumed that ground is plane mould
Type is fitted the plane using RANSAC, obtains ground model；It occupies map and asks for module, it will be whole in original-gray image
A scene is transformed into world coordinates by camera coordinates, while generates plan view, is asked for occupying map by plan view；Barrier is divided
And distance calculation module, divide to obtain the position of each barrier by connected component labeling detection algorithm from occupying in map, and
It is transformed into original image and is marked, and barrier is calculated to the distance of this vehicle by disparity map；Alarm module, current vehicle
Decision-making module participative decision making is alarmed or be passed to distance when being less than certain threshold value.
Vehicle anticollision method for early warning and system based on stereoscopic vision according to embodiments of the present invention, based on binocular vision
Image asks for ground model, occupies map by what ground model obtained scene, anticollision information is obtained by occupying map, can
To adapt to various road surfaces and road conditions, algorithm is low to disparity map required precisions, reduces front end operand, is anti-dry
Disturb ability is strong, do not depend on data and engineer's feature caused by influence.
Description of the drawings
From the detailed description below in conjunction with the accompanying drawings to the embodiment of the present invention, these and/or other aspects of the invention and
Advantage will become clearer and be easier to understand, wherein：
Fig. 1 shows the schematic diagram of according to embodiments of the present invention, vehicle-mounted vehicle anticollision early warning system 100；
Fig. 2 shows the totality of the vehicle anticollision method for early warning based on binocular stereo vision according to embodiments of the present invention
Flow chart；
Fig. 3 shows the schematic diagram of least square method error extraction straight line situation under there are larger noise situations；
Fig. 4 shows the flow for the method 240 that one section of straight line is fitted in the point according to embodiments of the present invention from V disparity maps
Figure；
Fig. 5 shows the structure diagram of vehicle anticollision early warning system 300 according to another embodiment of the present invention.
Specific embodiment
In order to which those skilled in the art is made to more fully understand the present invention, with reference to the accompanying drawings and detailed description to this hair
It is bright to be described in further detail.
The explanation of term used herein is provided first.
Disparity map：Disparity map is on the basis of image pair appoints piece image, and size is the size of the benchmark image, first
Image of the element value for parallax value.Disparity map contains the range information of scene.Disparity map can be from the left and right that binocular camera is shot
It is calculated in image.Certain point coordinates in ordinary two dimensional disparity map represents that wherein u is abscissa, and v is ordinate with (u, v)；
The pixel value of pixel at point (u, v) represents that pixel value represents the parallax at the point (u, v) with d (u, v).Due to disparity map packet
The range information of scene is contained, therefore has been always binocular vision research from the images match of stereo image pair extraction disparity map
In the most active field.
V disparity maps：V disparity maps are converted to from disparity map, and the gray value of any point (d, v) is corresponding in V disparity maps
The number of point of the parallax value equal to d in the row that the ordinate of disparity map is v.Figuratively, V disparity maps can be considered as disparity map
Side view.It is by the accumulative number with the parallax value identical point of a line that the plane projection in original image is in alignment.
RANSAC:The abbreviation of RANdom Sample Consensus, it is the sample number for including abnormal data according to one group
According to collection, the mathematical model parameter of data is calculated, obtains the algorithm of effective sample data.
Occupy map (occupancy map)：In the vision system of early stage, environmental knowledge is represented by grid, in grid
In, the 2D projection informations on each object to ground in store environment, such representation is referred to as occupying map.
Binocular stereo vision：Binocular stereo vision (Binocular Stereo Vision) is a kind of heavy of machine vision
Form is wanted, it is to pass through based on principle of parallax and using imaging device from the two images of different position acquisition testees
The position deviation between image corresponding points is calculated, to obtain the method for object dimensional geological information.The figure that two eyes of fusion obtain
Picture simultaneously observes the difference between them, us is allow to obtain apparent sense of depth, establishes the correspondence between feature, will be same
Photosites of the space physics point in different images are mapped, this difference, we are referred to as parallax (Disparity) image.
Binocular stereo vision measurement method has many advantages, such as that efficient, precision is suitable, system structure is simple, at low cost.To moving object
During (including animal and human body body) measures, since image acquisition was completed in moment, Stereo Vision is a kind of
More effective measuring method.Binocular Stereo Vision System is one of key technology of computer vision, obtains space three-dimensional scene
Range information be also most basic content in computer vision research.
Fig. 1 shows showing for according to embodiments of the present invention, the vehicle-mounted system 100 for being used to detect automobile wheeled region
It is intended to, including binocular camera 110 and computing device 120.
Persistently shooting obtains opening gray level image along the left and right two of the vehicle front of vehicle traveling direction binocular camera 110.
Binocular camera 110 for example mounted on the top front of vehicle, makes its image pickup scope concentrate on the road of front part of vehicle
Face.
Computing device 120 includes memory 121, processor 122, communication interface 123, bus 124.Memory 121, communication
Interface 123 and processor 122 are all connected to bus 124, are stored with computer executable instructions in memory, computing device via
Communication interface can obtain the left and right two that binocular camera is shot and open gray level image, when the processor execution computer can
During execute instruction, the method for execution vehicle anticollision early warning.
It can also include alarm 125 in computing device 120, for providing alarm signal when finding dangerous or emergency
Number or be sent out notifying.
Structure shown in FIG. 1 is merely illustrative, can be increased as needed, reduced, replaced.
In addition, it is necessary to explanation, a part of in some functions or function can carry out reality by different components as needed
It is existing, such as acquire disparity map from left images to calculate and be described as being realized by computing device in embodiment, but according to
It needs that software, hardware or the firmware for calculating disparity map can also be increased in binocular camera or can also be in the car
The special component for being used to calculate disparity map based on left images of deployment, these are all within the scope of present inventive concept.
With reference to the method in Fig. 2 detailed descriptions real-time detection automobile wheeled region according to embodiments of the present invention.
The technology in the real-time detection automobile wheeled region of the embodiment of the present invention, is obtained left by binocular camera sensor
Right two images obtain disparity map (disparity map) by two images in left and right, V disparity maps (V- are constructed with disparity map
Disparity map), then segmented linear is asked for, and straight line is put down according to multiple image using RANSAC on V disparity maps
Sliding filtering, then the safety zone of wheeled is finally obtained in original image as the parallax corresponding to these straight lines.
Fig. 2 shows the ensemble streams of the method 200 in the wheeled region of real-time detection automobile according to embodiments of the present invention
Cheng Tu.
In step S210, shoot to obtain the vapour along automobile direction of travel by the binocular camera carried on body of a motor car
Gray level image is opened in the left and right two of front side, and disparity map is calculated.
Specifically, for example, matching related algorithm according to binocular solid, the correspondence between each pair of image is first found out, according to
Principle of triangulation obtains the disparity map of current scene.
Here, some denoisings etc. can also be carried out to disparity map.
In step S220, V disparity maps are converted to from disparity map.
Specifically, for example, in disparity map, changed to represent that object is remote relative to the relative distance of camera lens with the gray scale depth
Closely, according to the depth of view information included in disparity map, the parallax on ground is consecutive variations, approximate segmented linear.Assuming that use MdTable
Show the pixel value that certain is put on disparity map, use MvdRepresent the pixel value of corresponding points on V disparity maps.With function f (Md)=MvdTo represent
Transformational relation between disparity map and V disparity maps, function f represent the pixel in the every a line of accumulative disparity map with same disparity
Number Pnum, in this way using parallax as horizontal axis, the longitudinal axis is consistent with disparity map, PnumFor the gray value of respective pixel, one is thus obtained
A gray scale V disparity maps.
In step S230, binaryzation is carried out to V disparity maps.
In one example, binaryzation is carried out with the following method：The principle of binaryzation is first to ask for the maximum of every a line
It is worth, only grey scale pixel value residing for maximum value is set as 255 in every a line, and rest of pixels gray value is set as 0.
In step S240, segmentation is obtained to be fitted from the point of the V disparity maps after binaryzation directly using RANSAC methods
Line.
Under explained later in numerous Algorithm of fitting a straight line, why selection of the embodiment of the present invention uses RANSAC methods
To carry out the fitting a straight line of the point of the V disparity maps after binaryzation.
Data in real life often have certain deviation, in other words noise, this causes difficulty to Mathematical Fitting.
Such as it is understood that two in a linear relationship between variable X and Y, Y=aX+b, we want to determine the occurrence of parameter a and b.It is logical
Experiment is crossed, the test value of one group of X and Y can be obtained.Although theoretically the equation of two unknown numbers only needs two class values true
Recognize, but due to systematic error, the value of a and b of 2 points of calculatings is arbitrarily taken all to be not quite similar.It is desirable that finally
The theoretical model and the error of test value being calculated are minimum.
The usual prior art is using least square method or Hough transformation come fitting a straight line.
The deficiency of Hough transformation is：Detection speed is too slow, can not accomplish to control in real time；Precision is not high enough, desired letter
Breath can't detect makes false judgment instead, and then generates a large amount of redundant data.This is mostly derived from：
1st, a large amount of memory headrooms need to be occupied, take long, real-time it is poor；
2nd, the image in reality is generally all interfered by outside noise, and signal-to-noise ratio is relatively low, conventional H ough transformation at this time
Performance will drastically decline, and since suitable threshold value is difficult to determine when carrying out the search of parameter space maximum, often occur " empty
The problem of peak " and " missing inspection ".
Value when least square method by calculating Minimum Mean Square Error about the partial derivative of parameter a, b is zero.In fact, very
In the case of more, least square method is all the synonym of linear regression.Regrettably, it is smaller to be suitable only for error for least square method
Situation.Just imagine such case, if the extraction model from a noise larger data set is needed (for example there was only 20%
Data when meet model) when, least square method just seems unable to do what one wishes.Such as Fig. 3, visually can easily it find out
Straight line (pattern), but least square method is confused.
The present invention detects road surface by extracting straight line from V disparity maps, has larger noise, such case in disparity map
It is lower to be likely to obtain the fitting of mistake to extract straight line with least square method.
RANSAC algorithms can estimate mathematical modulo from one group of observation data set comprising " point not in the know " by iterative manner
The parameter of type is very suitable for the model parameter estimation of the observation data containing more noise.It gets in practical applications
Data can usually include noise data, these noise datas can make to interfere the structure of model, we claim such make an uproar
Sound data point is outliers (point not in the know), those have a positive effect for model construction we they be referred to as inliers (offices
Interior point), something that RANSAC is done is exactly first random some points of selection, with these points go to obtain a model (if in
If doing fitting a straight line, this so-called model is exactly slope in fact), it is then gone to test remaining point with this model, if surveyed
The data point is then judged to intra-office point, is otherwise judged as point not in the know by the data point of examination in the range of error permission.Intra-office point
If number has reached the threshold value of some setting, these data point sets for illustrating this time to choose have reached acceptable journey
Otherwise degree continues all steps after randomly selecting point set of front, constantly repeats this process, these numbers until finding selection
Until strong point collection has reached acceptable degree, the model obtained at this time can be considered the optimal models structure to data point
It builds.
Fig. 4 shows the flow for the method 240 that one section of straight line is fitted in the point according to embodiments of the present invention from V disparity maps
Figure.This method can be used for the step S240 in Fig. 2.
In step S241, one group of random subset in the point in the V disparity maps after binaryzation is selected to carry out straight line plan
It closes, obtains straight line model.
It is gone with obtained straight line model to test all other data in step S242, if some point is suitable for estimation
Straight line model, it is believed that it is also intra-office point, the number put in statistics bureau.
In step S243, judge intra-office point number whether be more than threshold value, if it is determined that result be yes, then proceed to
Otherwise step S245 proceeds to step S244.
In step S244, judge that the model of estimation is unreasonable, abandon the model, then proceed to step S249.
In step S245, judge that the model of estimation is reasonable, then reevaluate model, and estimate with all intra-office points
The error rate of intra-office point and model is counted, then proceeds to step S246.
In step S246, judge whether the error rate of current estimation model is less than the error rate of best model, if knot
Fruit is affirmative, proceeds to step S247, otherwise proceeds to step S248.
In step S247, best model is substituted with the model currently estimated, i.e., because of the judgement according to step S246, when
The model errors rate of preceding estimation is lower than the error rate of best model, and performance ratio best model is more preferable, therefore replaces best model
As new best model, step S249 is then proceeded to.
In step S248, the model of estimation is abandoned, then proceeds to step S249.
In step S249, determine whether to reach end condition, if reaching end condition, process terminates, and otherwise returns
Step S241 is returned to repeat.Here end condition, such as can be that iterations reach threshold number, error rate is less than
Predetermined threshold etc..
The method for extracting one section of straight line from V disparity maps using RANSAC methods is described above with reference to Fig. 4, ground is not
It is plane, therefore it is the continuous segmented linear of multistage to be reflected in V disparity maps, the segmented linear method for extracting multistage can be such as
It is as follows：Straight line is extracted first, in accordance with the method for example with reference to described in Fig. 4, after extraction, first straight line will be belonged to
Point removed from V disparity maps, then extract Article 2 straight line after the same method for remaining point, so repeatedly under
It goes, until the number of remaining point is less than predetermined threshold.
Fig. 2 is returned to, after step S240 completions, proceeds to step S250.
In step s 250, according to multiple image smothing filtering straight line.
As previously mentioned, in 103489175 B of patent document CN, Kalman filtering has been carried out to the straight line of fitting.
Inventor thinks that variation of the kalman filter method based on process object is Gaussian Profile to carry out through experimental analysis
Filtering, but the variation on actually road surface is not Gaussian Profile, and in addition kalman filter method behaves very slow, can not meet
The requirement of real-time in automatic Pilot field detection automobile wheeled region.
Pavement detection Technology design according to embodiments of the present invention meets smoothly being filtered according to multiple image for real-time requirement
The method of wave straight line.Since great change will not occur for its gradient of road surface that automobile is travelled, variation is uniformly slow, so root
The variation for the straight line being fitted according to the embodiment of the present invention is also even variation.On the other hand it is obtained by binocular camera
Disparity map has many noises, and obtained straight line can generate unnecessary shake.In order to reduce this shake and in view of upper
The straight line that the fitting in face obtains is uniform slow property, and proposition of the embodiment of the present invention carries out smothing filtering with multiple image, comes
Obtain smooth and uniform straight line model.
Specifically, the smothing filtering of straight line can be carried out as follows according to multiple image：Set a time window, it is assumed that straight
Line model is expressed as ax+by+c=0, and straight line model parameter is obtained to every frame image, adds up for each parameter to every frame,
When often carrying out the new image of a frame, the straight line model parameter of a frame image, is added being subtracted most since cumulative parametric results
The straight line model parameter of current frame image, then it is averaging the straight line model parameter as this frame.For example, running car is on road surface
On, current time tc, new shooting obtains present image, at this point, i.e. for stationary window, removes first frame from window, so
New picture frame is added in afterwards, and the straight line mould as new picture frame is averaging to the straight line model parameter of image in window
The mathematical model parameter of shape parameter namely the road surface that estimates in V disparity maps；Then as the progress of time, continue this behaviour
Make, be equivalent to as the time carries out forward slip window.
In step S260, the wheeled region in former gray level image is obtained by the straight line extracted.
In one example, it can be obtained as follows by the straight line extracted in V disparity maps feasible in former gray level image
Sail region：For every a line in V disparity maps, point that the parallax value on the straight line of selection and withdrawal is d, in the corresponding row of disparity map
In, compare the parallax value of each pixel and the difference of d, when difference is less than certain threshold value, then judge artwork corresponding position
Wheeled region for safety.
The information in safe wheeled region is obtained in gray-scale map, can be auxiliary drive, automatic Pilot and nobody drive
It sails and crucial decision information is provided, prevent collision, ensure safety.
Fig. 2 is returned to, in step S270, according to original image and disparity map, calculates the point for belonging to ground in true generation
Three-dimensional coordinate in boundary's coordinate system, it is assumed that ground is areal model, is fitted the plane using RANSAC, obtains ground model.
Specifically, in one example, ground model is obtained by operations described below fitting：According to original image and parallax
Figure, calculates the point for belonging to ground three-dimensional coordinate in real world coordinates system.In real world coordinates system, ground is assumed
For an areal model, ax+by+cz+d=0 is expressed as, is then fitted the plane using RANSAC.RANSAC passes through repeatedly
One group of random subset in the maximum point in the candidate point of ground is selected to carry out fitting a straight line, due to being contained very in disparity map
The acquired point set for belonging to ground after a RANSAC is carried out, is carried out a RANSAC and carrys out fit Plane by more noises again,
Finally obtain ground model.
In step S280, the entire scene in original-gray image is transformed into world coordinates by camera coordinates, simultaneously
Plan view is generated, is asked for occupying map by plan view.
In one example, the process of asking for for occupying map is：The point that will be above ground certain altitude first extracts,
These points are transformed into earth axes；We will be above certain altitude and the point less than certain altitude is occupying map acceptance of the bid
Go out；The value for occupying each pixel in map is that the height of its respective point adds up and thus obtains occupying map.
In step S290, divide to obtain the position of each barrier by connected component labeling detection algorithm from occupying in map
It puts, and is transformed into original image and is marked, and barrier is calculated to the distance of this vehicle by disparity map.
In one example, the markd figure of color ribbon is converted into according to the size for occupying each pixel value in map
Picture, the pixel value the big more is biased to red, and pixel value is smaller to be more partial to blue.Then divided by connected component labeling detection algorithm
Go out different objects (), related connected component labeling detection algorithm can be with bibliography, Di Stefano, Luigi, and
Andrea Bulgarelli."A simple and efficient connected components labeling
algorithm."Image Analysis and Processing,1999.Proceedings.International
Conference on.IEEE,1999..Thus the specific location of each barrier is obtained, and is transformed into original image and marks
Out, and by disparity map barrier is calculated to the distance of this vehicle.Related distance can be based on the pass between parallax and depth
It is to acquire.
In step S2901, current vehicle distance is alarmed or is passed to decision-making module participation and determined when being less than certain threshold value
Plan.
Fig. 5 shows the automobile wheeled in the wheeled region of real-time detection automobile according to another embodiment of the present invention
The structure diagram of region real-time detecting system 300.System 300 is placed on automobile, for detecting the wheeled area of automobile in real time
Domain, the auxiliary for automobile drives, automatic Pilot and unmanned provides critical support.
As shown in figure 5, automobile wheeled region real-time detecting system 300 can include：Binocular camera 310, disparity map meter
Calculate component 320, V disparity maps converting member 330, binaryzation component 340, RANSAC fitting a straight lines component 350, multiple image filtering
Component 360, original image wheeled region determine component 370, ground model fitting module 380, occupy map ask for module 390,
Barrier divides and distance calculation module 391, alarm module 392.
Binocular camera 310 is configured to shooting and obtains opening gray level image along the left and right two of the vehicle front of automobile direction of travel.
Disparity map is calculated from two gray level images in left and right in disparity map calculating unit 320.V disparity maps converting member 330 is from disparity map
It is converted to V disparity maps.Binaryzation component 340 carries out binaryzation to V disparity maps.RANSAC fitting a straight lines component 350 uses
RANSAC methods obtain segmented linear to be fitted from the point of the V disparity maps after binaryzation.360 basis of multiple image filter part
Multiple image smothing filtering straight line.Original image wheeled region determines that component 370 obtains former gray-scale map by the straight line extracted
Wheeled region as in.Ground model fitting module 380 is calculated and belongs to the point on ground and exist according to original image and disparity map
Three-dimensional coordinate in real world coordinates system, it is assumed that ground is areal model, is fitted the plane using RANSAC, obtains ground
Model.It occupies map and asks for module 390 and the entire scene in original-gray image is transformed into world coordinates by camera coordinates, together
Shi Shengcheng plan views are asked for occupying map by plan view.Barrier divides and distance calculation module 391, leads to from occupying in map
Connected component labeling detection algorithm is crossed to divide to obtain the position of each barrier, and be transformed into original image and be marked, and lead to
Disparity map is crossed to calculate barrier to the distance of this vehicle.Alarm module 392, current vehicle distance are alarmed when being less than certain threshold value
Or incoming decision-making module participative decision making.
About disparity map calculating unit 320, V disparity maps converting member 330, binaryzation component 340, RANSAC fitting a straight lines
Component 350, multiple image filter part 360, original image wheeled region determine component 370, ground model fitting module 380,
Occupy map ask for module 390, barrier segmentation and distance calculation module 391, the function of alarm module 392 and specific implementation can
To correspond to the description of step with reference to figure 2, which is not described herein again.
It should be noted that binocular camera herein should be interpreted broadly, it is any to obtain left image and right figure
The camera of picture or the equipment with camera function can be seen as binocular camera herein.
About disparity map calculating unit 320, V disparity maps converting member 330, binaryzation component 340, RANSAC fitting a straight lines
Component 350, multiple image filter part 360, original image wheeled region determine component 370, ground model fitting module 380,
Occupy map ask for module 390, barrier segmentation and distance calculation module 391, alarm module 392 should also be interpreted broadly, this
A little components can with software, firmware or hardware or these combination come realize and all parts can be combined each other,
Secondary combination further carries out splitting etc., these both fall within the scope of the present disclosure.
Automobile wheeled region real-time detection method according to embodiments of the present invention and system, are adapted to various
Road surface and road conditions, it is low to disparity map required precision, reduce front end operand, strong antijamming capability, improve real-time, these for
The automatic safe of automobile drives very crucial.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.In the case of without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes will be apparent from for the those of ordinary skill in art field.Therefore, protection scope of the present invention should
This is subject to the protection scope in claims.
Claims (17)
1. a kind of vehicle anticollision method for early warning based on binocular stereo vision, including：
It shoots to obtain along the left and right two of the vehicle front of automobile direction of travel by the binocular camera carried on body of a motor car and open
Disparity map is calculated in gray level image；
V disparity maps are converted to from disparity map；
Binaryzation is carried out to V disparity maps；
Using RANSAC methods segmented linear is obtained to be fitted from the point of the V disparity maps after binaryzation；
According to multiple image smothing filtering straight line；And
The wheeled region in former gray level image is obtained by the straight line extracted；
According to original image and disparity map, three-dimensional coordinate of the point for belonging to ground in real world coordinates system is calculated, it is assumed that
Ground is areal model, is fitted the plane using RANSAC, obtains ground model；
Entire scene in original-gray image is transformed into world coordinates, while generate plan view by camera coordinates, by plane
Figure is asked for occupying map；
Divide to obtain the position of each barrier, and be transformed into original graph by connected component labeling detection algorithm from occupying in map
It is marked, and barrier is calculated to the distance of this vehicle by disparity map as in；
Decision-making module participative decision making is alarmed or be passed to current vehicle distance when being less than certain threshold value.
2. vehicle anticollision method for early warning according to claim 1, ask for occupying map by plan view and include：
The world coordinates of each point is first depending on, the point that will be above ground first threshold height extracts, these points are converted
Into earth axes；It will be above first threshold height and the point less than second threshold height mark in map is occupied；It occupies
The value of each pixel is that the height of its respective point adds up and thus obtains occupying map in map.
It is described to calculate from occupying to detect by connected component labeling in map 3. vehicle anticollision method for early warning according to claim 1
The position that method divides to obtain each barrier includes：
Size according to each pixel value in map is occupied is converted into the markd image of color ribbon so that pixel value is bigger
Red is more biased to, pixel value is smaller to be more partial to blue, different objects is partitioned by connected component labeling detection algorithm, thus
Obtain the position of each barrier.
4. vehicle anticollision method for early warning according to claim 1, wherein, it is described that V disparity maps progress binaryzation is included：
The maximum value of every one-line pixel value is asked for, by only the gray value of pixel is set as 255 residing for maximum value in every a line, remaining
Grey scale pixel value is set as 0.
5. vehicle anticollision method for early warning according to claim 1 is included using RANSAC methods to be fitted one section of segmented linear：
Operations described below sequence is performed repeatedly, until reaching predetermined ending standard：
One group of random subset in the maximum of points in V disparity maps is selected to carry out fitting a straight line, obtains straight line model；
It is gone to test all other data with obtained straight line model, if some point is suitable for the straight line model of estimation, it is believed that
It is also intra-office point, is classified as intra-office point if there is exceeding predetermined number point of destination, then the model of estimation is taken as rationally
, model then is reevaluated with all intra-office points, and estimate the error rate of intra-office point and model；
If the error rate of model substitutes current best model less than current best model with the model；
Using the best model that finally obtains as this section of segmented linear.
6. vehicle anticollision method for early warning according to claim 5, described to be fitted multistage segmented linear using RANSAC methods
Including：
Straight line is extracted first, in accordance with the method described in claim 5, after extraction, the point of first straight line will be belonged to
It is removed from V disparity maps, then extracts Article 2 straight line for remaining point method according to the method for claim 5, so
Go down repeatedly, until the number of remaining point is less than predetermined threshold.
7. vehicle anticollision method for early warning according to claim 1, described to be included according to multiple image smothing filtering straight line：
Setting a time window, it is assumed that straight line model is expressed as ax+by+c=0, and straight line model parameter is obtained to every frame image,
It adds up for each parameter to every frame, when often carrying out the new image of a frame, one being subtracted most since cumulative parametric results
The straight line model parameter of frame image along with the straight line model parameter of current frame image, then is averaging the straight line as this frame
Model parameter.
8. according to the vehicle anticollision method for early warning of any one of claim 1 to 7, the straight line by being extracted obtains former ash
Wheeled region in degree image includes：
For every a line in V disparity maps, point that the parallax value on the straight line of selection and withdrawal is d, in the corresponding row of disparity map,
Compare the parallax value of each pixel and the difference of d, when difference is less than certain threshold value, then be judged to pacifying by artwork corresponding position
Full wheeled region.
9. a kind of vehicle anticollision early warning system based on binocular stereo vision, including：
Binocular camera, lasting shooting obtain opening gray level image along the left and right two of the vehicle front of vehicle traveling direction；
Computing device, including memory, processor, communication interface, bus, memory, communication interface and processor are all connected to
Bus, computer executable instructions are stored in memory, and computing device can obtain binocular camera via communication interface and clap
Gray level image is opened in the left and right two taken the photograph, and when processor performs the computer executable instructions, performs following methods：
Based on two gray level images in left and right, disparity map is calculated；
V disparity maps are converted to from disparity map；
Binaryzation is carried out to V disparity maps；
Using RANSAC methods segmented linear is obtained to be fitted from the point of the V disparity maps after binaryzation；
According to multiple image smothing filtering straight line；
The wheeled region in former gray level image is obtained by the straight line extracted；
According to original image and disparity map, three-dimensional coordinate of the point for belonging to ground in real world coordinates system is calculated, it is assumed that
Ground is areal model, is fitted the plane using RANSAC, obtains ground model；
Entire scene in original-gray image is transformed into world coordinates, while generate plan view by camera coordinates, by plane
Figure is asked for occupying map；
Divide to obtain the position of each barrier, and be transformed into original graph by connected component labeling detection algorithm from occupying in map
It is marked as in, and barrier is calculated to the distance of this vehicle by disparity map,
Decision-making module participative decision making is alarmed or be passed to current vehicle distance when being less than certain threshold value.
10. vehicle anticollision early warning system according to claim 9, ask for occupying map by plan view and include：
The world coordinates of each point is first depending on, the point that will be above ground first threshold height extracts, these points are converted
Into earth axes；It will be above first threshold height and the point less than second threshold height mark in map is occupied；It occupies
The value of each pixel is that the height of its respective point adds up and thus obtains occupying map in map.
It is described to calculate from occupying to detect by connected component labeling in map 11. vehicle anticollision early warning system according to claim 9
The position that method divides to obtain each barrier includes：
Size according to each pixel value in map is occupied is converted into the markd image of color ribbon so that pixel value is bigger
Red is more biased to, pixel value is smaller to be more partial to blue, different objects is partitioned by connected component labeling detection algorithm, thus
Obtain the position of each barrier.
12. vehicle anticollision early warning system according to claim 9, wherein, it is described that V disparity maps progress binaryzation is included：
The maximum value of every one-line pixel value is asked for, by only the gray value of pixel is set as 255 residing for maximum value in every a line, remaining
Grey scale pixel value is set as 0.
13. vehicle anticollision early warning system according to claim 9, described to be fitted one section of segmented linear using RANSAC methods
Including：
Operations described below sequence is performed repeatedly, until reaching predetermined exits standard：
One group of random subset in the maximum of points in V disparity maps is selected to carry out fitting a straight line, obtains straight line model；
It is gone to test all other data with obtained straight line model, if some point is suitable for the straight line model of estimation, it is believed that
It is also intra-office point, is classified as intra-office point if there is exceeding predetermined number point of destination, then the model of estimation is taken as rationally
, model then is reevaluated with all intra-office points, and estimate the error rate of intra-office point and model；
If the error rate of model substitutes current best model less than current best model with the model；
Using the best model that finally obtains as this section of segmented linear.
14. vehicle anticollision early warning system according to claim 13, described to be fitted multistage segmentation directly using RANSAC methods
Line includes：
Straight line is extracted, after extraction, the point for belonging to first straight line is removed from V disparity maps, then for remaining
Point extract Article 2 straight line, so go down repeatedly, until the number of remaining point is less than predetermined threshold.
15. vehicle anticollision early warning system according to claim 9, described to be included according to multiple image smothing filtering straight line：
Setting a time window, it is assumed that straight line model is expressed as ax+by+c=0, and straight line model parameter is obtained to every frame image,
It adds up for each parameter to every frame, when often carrying out the new image of a frame, one being subtracted most since cumulative parametric results
The straight line model parameter of frame image along with the straight line model parameter of current frame image, then is averaging the straight line as this frame
Model parameter.
16. vehicle anticollision early warning system according to claim 9, described to be obtained in former gray level image by the straight line extracted
Wheeled region include：
For every a line in V disparity maps, the parallax value on the straight line of selection and withdrawal is d, in disparity map in corresponding row, than
Compared with the parallax value of each pixel and the difference of d, when difference is less than certain threshold value, then artwork corresponding position is determined as safety
Wheeled region.
17. a kind of vehicle anticollision early warning system, including：
Binocular camera is configured to shooting and obtains opening gray level image along the left and right two of the vehicle front of automobile direction of travel；
Disparity map calculating unit, from left and right, disparity map is calculated in two gray level images；
V disparity map modular converters are converted to V disparity maps from disparity map；
Binarization block carries out binaryzation to V disparity maps；
RANSAC fitting a straight line modules are segmented using RANSAC methods to be fitted from the point of the V disparity maps after binaryzation
Straight line；
Multiple image filter module, according to multiple image smothing filtering straight line；
Original image wheeled area determination module obtains the wheeled region in former gray level image by the straight line extracted；
Ground model fitting module according to original image and disparity map, calculates the point for belonging to ground in real world coordinates system
In three-dimensional coordinate, it is assumed that ground is areal model, is fitted the plane using RANSAC, obtains ground model；
It occupies map and asks for module, the entire scene in original-gray image is transformed into world coordinates by camera coordinates, simultaneously
Plan view is generated, is asked for occupying map by plan view；
Barrier divides and distance calculation module, divides to obtain each barrier by connected component labeling detection algorithm from occupying in map
Hinder the position of object, and be transformed into original image and be marked, and barrier is calculated to the distance of this vehicle by disparity map；
Decision-making module participative decision making is alarmed or are passed to alarm module, current vehicle distance when being less than certain threshold value.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2016/100521 WO2018058356A1 (en) | 2016-09-28 | 2016-09-28 | Method and system for vehicle anti-collision pre-warning based on binocular stereo vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108243623A true CN108243623A (en) | 2018-07-03 |
| CN108243623B CN108243623B (en) | 2022-06-03 |
Family
ID=61762982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201680001426.6A Active CN108243623B (en) | 2016-09-28 | 2016-09-28 | Automobile anti-collision early warning method and system based on binocular stereo vision |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108243623B (en) |
| WO (1) | WO2018058356A1 (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109601109A (en) * | 2018-12-07 | 2019-04-12 | 江西洪都航空工业集团有限责任公司 | A kind of unmanned grass-cutting vehicle collision-proof method based on binocular vision detection |
| CN110285793A (en) * | 2019-07-08 | 2019-09-27 | 中原工学院 | A kind of Vehicular intelligent survey track approach based on Binocular Stereo Vision System |
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN110909569A (en) * | 2018-09-17 | 2020-03-24 | 深圳市优必选科技有限公司 | Road condition information identification method and terminal equipment |
| CN111469759A (en) * | 2019-01-24 | 2020-07-31 | 海信集团有限公司 | Scratch and rub early warning method for vehicle, vehicle and storage medium |
| CN112233136A (en) * | 2020-11-03 | 2021-01-15 | 上海西井信息科技有限公司 | Method, system, equipment and storage medium for alignment of container trucks based on binocular recognition |
| CN112418103A (en) * | 2020-11-24 | 2021-02-26 | 中国人民解放军火箭军工程大学 | Bridge crane hoisting safety anti-collision system and method based on dynamic binocular vision |
| CN112767818A (en) * | 2019-11-01 | 2021-05-07 | 北京初速度科技有限公司 | Map construction method and device |
| WO2021174539A1 (en) * | 2020-03-06 | 2021-09-10 | 深圳市大疆创新科技有限公司 | Object detection method, mobile platform, device and storage medium |
| CN113658240A (en) * | 2021-07-15 | 2021-11-16 | 北京中科慧眼科技有限公司 | Main obstacle detection method and device and automatic driving system |
| CN113706622A (en) * | 2021-10-29 | 2021-11-26 | 北京中科慧眼科技有限公司 | Road surface fitting method and system based on binocular stereo vision and intelligent terminal |
| CN117672007A (en) * | 2024-02-03 | 2024-03-08 | 福建省高速公路科技创新研究院有限公司 | Road construction area safety precaution system based on thunder fuses |
Families Citing this family (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110969064B (en) * | 2018-09-30 | 2023-10-27 | 北京四维图新科技股份有限公司 | Image detection method and device based on monocular vision and storage equipment |
| CN109444916B (en) * | 2018-10-17 | 2023-07-04 | 上海蔚来汽车有限公司 | Unmanned driving drivable area determining device and method |
| CN109461308B (en) * | 2018-11-22 | 2020-10-16 | 东软睿驰汽车技术（沈阳）有限公司 | Information filtering method and image processing server |
| CN111382591B (en) * | 2018-12-27 | 2023-09-29 | 海信集团有限公司 | Binocular camera ranging correction method and vehicle-mounted equipment |
| CN109993060B (en) * | 2019-03-01 | 2022-11-22 | 长安大学 | Vehicle omnidirectional obstacle detection method of depth camera |
| CN110569704B (en) * | 2019-05-11 | 2022-11-22 | 北京工业大学 | Multi-strategy self-adaptive lane line detection method based on stereoscopic vision |
| CN110110682B (en) * | 2019-05-14 | 2023-04-18 | 西安电子科技大学 | Semantic stereo reconstruction method for remote sensing image |
| CN110472508B (en) * | 2019-07-15 | 2023-04-28 | 天津大学 | Lane line distance measurement method based on deep learning and binocular vision |
| CN111241979B (en) * | 2020-01-07 | 2023-06-23 | 浙江科技学院 | Real-time obstacle detection method based on image feature calibration |
| CN111275698B (en) * | 2020-02-11 | 2023-05-09 | 西安汇智信息科技有限公司 | Method for detecting visibility of road in foggy weather based on unimodal offset maximum entropy threshold segmentation |
| US11961243B2 (en) * | 2020-02-26 | 2024-04-16 | Nvidia Corporation | Object detection using image alignment for autonomous machine applications |
| CN111626095B (en) * | 2020-04-06 | 2023-07-28 | 连云港市港圣开关制造有限公司 | Power distribution inspection system based on Ethernet |
| CN111580131B (en) * | 2020-04-08 | 2023-07-07 | 西安邮电大学 | Method for identifying vehicles on expressway by three-dimensional laser radar intelligent vehicle |
| CN111612760B (en) | 2020-05-20 | 2023-11-17 | 阿波罗智联(北京)科技有限公司 | Method and device for detecting obstacles |
| CN111890358B (en) * | 2020-07-01 | 2022-06-14 | 浙江大华技术股份有限公司 | Binocular obstacle avoidance method and device, storage medium and electronic device |
| CN112097732A (en) * | 2020-08-04 | 2020-12-18 | 北京中科慧眼科技有限公司 | Binocular camera-based three-dimensional distance measurement method, system, equipment and readable storage medium |
| CN111985436B (en) * | 2020-08-29 | 2024-03-12 | 浙江工业大学 | Workshop ground marking recognition fitting method based on LSD |
| CN112200771A (en) * | 2020-09-14 | 2021-01-08 | 浙江大华技术股份有限公司 | Height measuring method, device, equipment and medium |
| CN112288791B (en) * | 2020-11-06 | 2024-04-30 | 浙江中控技术股份有限公司 | Parallax image obtaining method, three-dimensional model obtaining method and device based on fisheye camera |
| CN112669362B (en) * | 2021-01-12 | 2024-03-29 | 四川深瑞视科技有限公司 | Depth information acquisition method, device and system based on speckles |
| CN113370977B (en) * | 2021-05-06 | 2022-11-18 | 上海大学 | Intelligent vehicle forward collision early warning method and system based on vision |
| CN113112553B (en) * | 2021-05-26 | 2022-07-29 | 北京三快在线科技有限公司 | Parameter calibration method and device for binocular camera, electronic equipment and storage medium |
| CN113341391B (en) * | 2021-06-01 | 2022-05-10 | 电子科技大学 | Radar target multi-frame joint detection method in unknown environment based on deep learning |
| CN113900443B (en) * | 2021-09-28 | 2023-07-18 | 合肥工业大学 | Unmanned aerial vehicle obstacle avoidance early warning method and device based on binocular vision |
| CN114119700B (en) * | 2021-11-26 | 2024-03-29 | 山东科技大学 | Obstacle ranging method based on U-V disparity map |
| CN113946154B (en) * | 2021-12-20 | 2022-04-22 | 广东科凯达智能机器人有限公司 | Visual identification method and system for inspection robot |
| CN115116038B (en) * | 2022-08-30 | 2023-03-24 | 北京中科慧眼科技有限公司 | Obstacle identification method and system based on binocular vision |
| CN115205809B (en) * | 2022-09-15 | 2023-03-24 | 北京中科慧眼科技有限公司 | Method and system for detecting roughness of road surface |
| CN115995163B (en) * | 2023-03-23 | 2023-06-27 | 江西通慧科技集团股份有限公司 | Vehicle collision early warning method and system |
| CN117079219B (en) * | 2023-10-08 | 2024-01-09 | 山东车拖车网络科技有限公司 | Vehicle running condition monitoring method and device applied to trailer service |
| CN117930224A (en) * | 2024-03-19 | 2024-04-26 | 山东科技大学 | Vehicle ranging method based on monocular vision depth estimation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102520721A (en) * | 2011-12-08 | 2012-06-27 | 北京控制工程研究所 | Autonomous obstacle-avoiding planning method of tour detector based on binocular stereo vision |
| CN103400392A (en) * | 2013-08-19 | 2013-11-20 | 山东鲁能智能技术有限公司 | Binocular vision navigation system and method based on inspection robot in transformer substation |
| CN103679127A (en) * | 2012-09-24 | 2014-03-26 | 株式会社理光 | Method and device for detecting drivable area of road pavement |
| CN105043350A (en) * | 2015-06-25 | 2015-11-11 | 闽江学院 | Binocular vision measuring method |
| CN105225482A (en) * | 2015-09-02 | 2016-01-06 | 上海大学 | Based on vehicle detecting system and the method for binocular stereo vision |
| CN105550665A (en) * | 2016-01-15 | 2016-05-04 | 北京理工大学 | Method for detecting pilotless automobile through area based on binocular vision |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106651B (en) * | 2012-07-16 | 2015-06-24 | 清华大学深圳研究生院 | Method for obtaining parallax error plane based on three-dimensional hough |
| CN103413313B (en) * | 2013-08-19 | 2016-08-10 | 国家电网公司 | The binocular vision navigation system of electrically-based robot and method |
-
2016
- 2016-09-28 CN CN201680001426.6A patent/CN108243623B/en active Active
- 2016-09-28 WO PCT/CN2016/100521 patent/WO2018058356A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102520721A (en) * | 2011-12-08 | 2012-06-27 | 北京控制工程研究所 | Autonomous obstacle-avoiding planning method of tour detector based on binocular stereo vision |
| CN103679127A (en) * | 2012-09-24 | 2014-03-26 | 株式会社理光 | Method and device for detecting drivable area of road pavement |
| CN103400392A (en) * | 2013-08-19 | 2013-11-20 | 山东鲁能智能技术有限公司 | Binocular vision navigation system and method based on inspection robot in transformer substation |
| CN105043350A (en) * | 2015-06-25 | 2015-11-11 | 闽江学院 | Binocular vision measuring method |
| CN105225482A (en) * | 2015-09-02 | 2016-01-06 | 上海大学 | Based on vehicle detecting system and the method for binocular stereo vision |
| CN105550665A (en) * | 2016-01-15 | 2016-05-04 | 北京理工大学 | Method for detecting pilotless automobile through area based on binocular vision |
Non-Patent Citations (1)
| Title |
|---|
| 张毅: "基于立体视觉的非结构化环境下障碍物检测技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110909569B (en) * | 2018-09-17 | 2022-09-23 | 深圳市优必选科技有限公司 | Road condition information identification method and terminal equipment |
| CN110909569A (en) * | 2018-09-17 | 2020-03-24 | 深圳市优必选科技有限公司 | Road condition information identification method and terminal equipment |
| CN109601109A (en) * | 2018-12-07 | 2019-04-12 | 江西洪都航空工业集团有限责任公司 | A kind of unmanned grass-cutting vehicle collision-proof method based on binocular vision detection |
| CN111469759A (en) * | 2019-01-24 | 2020-07-31 | 海信集团有限公司 | Scratch and rub early warning method for vehicle, vehicle and storage medium |
| CN110298330B (en) * | 2019-07-05 | 2023-07-18 | 东北大学 | Monocular detection and positioning method for power transmission line inspection robot |
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN110285793B (en) * | 2019-07-08 | 2020-05-15 | 中原工学院 | Intelligent vehicle track measuring method based on binocular stereo vision system |
| CN110285793A (en) * | 2019-07-08 | 2019-09-27 | 中原工学院 | A kind of Vehicular intelligent survey track approach based on Binocular Stereo Vision System |
| CN112767818A (en) * | 2019-11-01 | 2021-05-07 | 北京初速度科技有限公司 | Map construction method and device |
| CN112767818B (en) * | 2019-11-01 | 2022-09-27 | 北京初速度科技有限公司 | Map construction method and device |
| WO2021174539A1 (en) * | 2020-03-06 | 2021-09-10 | 深圳市大疆创新科技有限公司 | Object detection method, mobile platform, device and storage medium |
| CN112233136A (en) * | 2020-11-03 | 2021-01-15 | 上海西井信息科技有限公司 | Method, system, equipment and storage medium for alignment of container trucks based on binocular recognition |
| CN112418103A (en) * | 2020-11-24 | 2021-02-26 | 中国人民解放军火箭军工程大学 | Bridge crane hoisting safety anti-collision system and method based on dynamic binocular vision |
| CN113658240A (en) * | 2021-07-15 | 2021-11-16 | 北京中科慧眼科技有限公司 | Main obstacle detection method and device and automatic driving system |
| CN113658240B (en) * | 2021-07-15 | 2024-04-19 | 北京中科慧眼科技有限公司 | Main obstacle detection method and device and automatic driving system |
| CN113706622A (en) * | 2021-10-29 | 2021-11-26 | 北京中科慧眼科技有限公司 | Road surface fitting method and system based on binocular stereo vision and intelligent terminal |
| CN117672007A (en) * | 2024-02-03 | 2024-03-08 | 福建省高速公路科技创新研究院有限公司 | Road construction area safety precaution system based on thunder fuses |
| CN117672007B (en) * | 2024-02-03 | 2024-04-26 | 福建省高速公路科技创新研究院有限公司 | Road construction area safety precaution system based on thunder fuses |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018058356A1 (en) | 2018-04-05 |
| CN108243623B (en) | 2022-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108243623A (en) | Vehicle anticollision method for early warning and system based on binocular stereo vision | |
| CN104573646B (en) | Chinese herbaceous peony pedestrian detection method and system based on laser radar and binocular camera | |
| CN110487562B (en) | Driveway keeping capacity detection system and method for unmanned driving | |
| CN109017570B (en) | Vehicle surrounding scene presenting method and device and vehicle | |
| CN101303735B (en) | Method for detecting moving objects in a blind spot region of a vehicle and blind spot detection device | |
| US9121717B1 (en) | Collision avoidance for vehicle control | |
| CN106379237B (en) | Vehicle lane-changing overall process DAS (Driver Assistant System) based on augmented reality | |
| Bounini et al. | Autonomous vehicle and real time road lanes detection and tracking | |
| EP1796043B1 (en) | Object detection | |
| EP2372308B1 (en) | Image processing system and vehicle control system | |
| CN103797529B (en) | Three-dimensional body detects device | |
| CN108351207A (en) | Stereoscopic camera device | |
| EP2256690B1 (en) | Object motion detection system based on combining 3D warping techniques and a proper object motion detection | |
| KR101891460B1 (en) | Method and apparatus for detecting and assessing road reflections | |
| Labayrade et al. | In-vehicle obstacles detection and characterization by stereovision | |
| JP6574611B2 (en) | Sensor system for obtaining distance information based on stereoscopic images | |
| CN108108750A (en) | Metric space method for reconstructing based on deep learning and monocular vision | |
| CN107517592A (en) | Automobile wheeled region real-time detection method and system | |
| CN112154454A (en) | Target object detection method, system, device and storage medium | |
| KR20090014124A (en) | Method and apparatus for evaluating an image | |
| CN110619674B (en) | Three-dimensional augmented reality equipment and method for accident and alarm scene restoration | |
| US20090052742A1 (en) | Image processing apparatus and method thereof | |
| EP3364336B1 (en) | A method and apparatus for estimating a range of a moving object | |
| CN109791607A (en) | It is detected from a series of images of video camera by homography matrix and identifying object | |
| CN109886064B (en) | Method for determining the boundary of a drivable space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |