CN111762155A - Vehicle distance measuring system and method - Google Patents
Vehicle distance measuring system and method Download PDFInfo
- Publication number
- CN111762155A CN111762155A CN202010518686.9A CN202010518686A CN111762155A CN 111762155 A CN111762155 A CN 111762155A CN 202010518686 A CN202010518686 A CN 202010518686A CN 111762155 A CN111762155 A CN 111762155A
- Authority
- CN
- China
- Prior art keywords
- parallax
- fpga
- result
- view data
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
Abstract
The embodiment of the disclosure discloses a distance measuring system and method for a vehicle. Wherein the system includes: the system comprises an image acquisition device and an FPGA platform; the left view data and the right view data acquired by the image acquisition device are transmitted to the FPGA platform; the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result; performing distance calculation on the matching calculation result to obtain a distance calculation result; performing cost calculation on the distance calculation result to obtain a parallax space diagram; calculating the parallax space image by using a WTA algorithm to obtain a parallax image; performing median filtering on the disparity map to obtain a median filtering result; and carrying out consistency detection on the median filtering result, and outputting a corresponding parallax value if the median filtering result is consistent with the median filtering result. Therefore, real-time binocular ranging is realized, the requirements of accuracy and real-time performance are met, and the requirements of low power consumption and low cost are realized.
Description
Technical Field
The disclosure belongs to the technical field of automatic driving, and particularly relates to a vehicle distance measuring system and method.
Background
In the field of autonomous driving, collision warning is an important function of ADAS. The change of the distance needs to be concerned when the collision early warning is realized, the time about collision is estimated, and the distance measurement is carried out at the bottom. The accuracy and rapidity of obstacle ranging greatly influence the development process of automatic driving.
The binocular camera distance measuring method is used for measuring the distance of a front obstacle by performing parallax calculation on two images. In the ranging process, the requirements on high real-time performance and low power consumption are met; in the prior art, a common ARM embedded platform cannot meet corresponding real-time requirements, and a GPU platform is high in power consumption and price and cost.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a distance measurement system and method for a vehicle, which at least achieve the purposes of high accuracy, strong real-time performance, low power consumption and low cost.
In a first aspect, an embodiment of the present disclosure provides an automotive ranging system, including: the system comprises an image acquisition device and an FPGA platform, wherein the FPGA platform comprises an FPGA;
the left view data and the right view data acquired by the image acquisition device are transmitted to the FPGA;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
the FPGA carries out distance calculation on the matching calculation result to obtain a distance calculation result;
the FPGA carries out cost calculation on the distance calculation result to obtain a parallax space diagram;
the FPGA calculates the parallax space image by using a WTA algorithm to obtain a parallax image;
the FPGA carries out median filtering on the disparity map to obtain a median filtering result;
and the FPGA carries out consistency detection on the median filtering result, and if the median filtering result is consistent with the median filtering result, a corresponding parallax value is output.
Optionally, the FPGA performs consistency detection on the median filtering result, and further includes:
and if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting mean-filtered disparity values.
Optionally, the transmitting the left and right view data acquired by the image acquiring apparatus to the FPGA platform includes:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
and storing the merged data to a cache device.
Optionally, the FPGA performs binocular matching calculation on the left and right view data to obtain a matching calculation result, including:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
and when both FIFOs have data, performing binocular matching calculation.
Optionally, the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM buffers N rows of data, binocular matching calculation is started from N +1 rows of pixel input of the RAM.
Optionally, the binocular matching calculation further includes:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the rest pixels are smaller than the base criterion, the gray values are recorded as 1, otherwise, the gray values are 0, and the results are output in sequence.
Optionally, the distance calculation performed by the FPGA on the matching calculation result to obtain a distance calculation result includes:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
Optionally, the FPGA performs cost calculation on the distance calculation result to obtain a disparity space map, including:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
Optionally, the calculating, by the FPGA, the disparity space map by using a WTA algorithm to obtain the disparity map includes:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
Optionally, performing consistency detection on the median filtering result includes:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
In a second aspect, an embodiment of the present disclosure provides a ranging method for a vehicle, including:
transmitting left and right view data acquired by an image acquisition device to an FPGA platform, wherein the FPGA platform comprises an FPGA;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
performing distance calculation on the matching calculation result to obtain a distance calculation result;
performing cost calculation on the distance calculation result to obtain a parallax space diagram;
calculating the parallax space image by using a WTA algorithm to obtain a parallax image;
performing median filtering on the disparity map to obtain a median filtering result;
and carrying out consistency detection on the median filtering result, and outputting a corresponding parallax value if the median filtering result is consistent with the median filtering result.
Optionally, the step of performing consistency detection on the median filtering result further includes:
and if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting mean-filtered disparity values.
Optionally, the transmitting the left and right view data acquired by the image acquiring apparatus to the FPGA platform includes:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
and storing the merged data to a cache device.
Optionally, the FPGA performs binocular matching calculation on the left and right view data to obtain a matching calculation result, including:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
and when both FIFOs have data, performing binocular matching calculation.
Optionally, the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM buffers N rows of data, binocular matching calculation is started from N +1 rows of pixel input of the RAM.
Optionally, the binocular matching calculation further includes:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the rest pixels are smaller than the base criterion, the gray values are recorded as 1, otherwise, the gray values are 0, and the results are output in sequence.
Optionally, the performing distance calculation on the matching calculation result to obtain a distance calculation result includes:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
Optionally, the performing cost calculation on the distance calculation result to obtain a disparity space map includes:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
Optionally, the calculating the disparity space map by using a WTA algorithm to obtain a disparity map includes:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
Optionally, performing consistency detection on the median filtering result includes:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
The method is based on FPGA to perform parallax calculation, view data are read on an FPGA platform, binocular matching calculation, distance calculation and cost calculation are performed in the FPGA, calculation is performed by using a WTA algorithm, median filtering is performed, then consistency detection is performed on the filtered parallax value, and the parallax value is output according to the detection result, so that real-time binocular ranging is realized, the requirements of accuracy and real-time performance are met, and the requirements of low power consumption and low cost are also realized.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in greater detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.
FIG. 1 illustrates a functional block diagram of a vehicle ranging system according to one embodiment of the present disclosure;
FIG. 2 illustrates a schematic diagram of serial-to-parallel converting and merging data according to one embodiment of the present disclosure;
fig. 3 shows a flow diagram for FPGA to compute disparity according to one embodiment of the present disclosure;
FIG. 4 shows a schematic diagram of an FPGA implementing Census computation according to one embodiment of the present disclosure;
FIG. 5 shows a diagram of median filtering according to one embodiment of the present disclosure;
fig. 6a to 6c are schematic views illustrating detection results output from a ranging system for a vehicle according to an embodiment of the present disclosure.
Detailed Description
Preferred embodiments of the present disclosure will be described in more detail below. While the following describes preferred embodiments of the present disclosure, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein.
Hamming (Hamming theorem);
WTA(Winner Take All)。
as shown in fig. 1, a ranging system for a vehicle includes: the system comprises an image acquisition device and an FPGA platform;
the image acquisition device is a video camera or a binocular camera and the like which are arranged on the vehicle and used for simulating human eyes to acquire binocular images.
The left and right view data acquired by the image acquisition device are transmitted to an FPGA platform, and the FPGA platform comprises an FPGA;
in a specific application scene, a binocular camera acquires a left image and a right image, and the left image and the right image are input into an FPGA (field programmable gate array) through an MIPI (mobile industry processor interface) for calculation. The FPGA platform utilizes the camera to transmit the incoming image.
Optionally, the transmitting the left and right view data acquired by the image acquiring apparatus to the FPGA includes:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
and storing the merged data to a cache device.
In a specific application scenario, after a system is powered on, an MIPI CSI interface of an FPGA platform receives image data, and after the MIPI input data are subjected to serial-parallel conversion, the combined data are stored in onboard DDR for caching.
The FPGA platform carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
optionally, the performing binocular matching calculation on the left view data and the right view data by the FPGA platform to obtain a matching calculation result includes:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
and when both FIFOs have data, performing binocular matching calculation.
In a specific application scenario, after binocular data in the DDR is cached for one frame, parallax calculation marks are enabled, left and right images are read and respectively temporarily stored in the FIFOs, and calculation is started when both the FIFOs have data.
Optionally, the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM buffers N rows of data, binocular matching calculation is started from N +1 rows of pixel input of the RAM.
Optionally, the binocular matching calculation further includes:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the rest pixels are smaller than the base criterion, the gray values are recorded as 1, otherwise, the gray values are 0, and the results are output in sequence.
In a specific application scenario, a Census algorithm is adopted in binocular matching calculation, data in left and right FIFOs are extracted simultaneously, 4 (or 5) dual-port RAMs are utilized to perform line-by-line caching respectively, preparation is made for Census calculation of 5x5, and after 4 (or 5) lines of data are completely cached, the Census algorithm is calculated from the input of pixels in the next line; in Census algorithm 5x5, the gray value of the third row and the third column of pixels is used as a reference, the gray values of the other 24 pixels are compared with the reference, if the gray values are smaller than the reference, the gray values are marked as 1, otherwise, the gray values are 0, and the results are sequentially arranged into 24-bit data output.
The FPGA carries out distance calculation on the matching calculation result to obtain a distance calculation result;
optionally, the performing distance calculation on the matching calculation result to obtain a distance calculation result includes:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
In a specific application scenario, Hamming calculation is adopted for distance calculation, and a Census calculation result is input into a Hamming calculation module for Hamming theorem calculation of 64 parallaxes. Taking the left image frame as a reference, taking the corresponding position and 63 pixels behind the corresponding position in the right image to perform exclusive OR operation, and adding each bit result of each parallax value to obtain a Hamming output result.
The FPGA carries out cost calculation on the distance calculation result to obtain a parallax space diagram;
optionally, the FPGA performs cost calculation on the distance calculation result to obtain a disparity space map, including:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
In a specific application scene, semi-global matching cost calculation is carried out, positive 4-direction cost calculation is carried out through an FPGA, cost values in the positive direction of a line on an RAM cache are used for calculating the cost of the line, and pipeline operation is achieved. Obtaining a parallax space map S (p, d) after summing the 4 direction cost values; meanwhile, the buffer data is used to calculate the disparity map space S _ R (p, d) calculated by taking the right image pixel as the reference.
Calculating the parallax space image by using a WTA algorithm to obtain a parallax image;
optionally, the calculating, by the FPGA, the disparity space map by using a WTA algorithm to obtain the disparity map includes:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
In a specific application scenario, based on a parallax space map S (p, d) and an S _ R (p, d), a wta (winner Take all) algorithm is respectively used for calculation, and in the parallax space map S (p, d), a parallax with the minimum matching cost of each pixel point p is selected, so that a left parallax map and a right parallax map are obtained.
The FPGA carries out median filtering on the disparity map to obtain a median filtering result;
in a specific application scenario, median filtering of 3x3 is performed on the obtained disparity map, fast median filtering is performed on 3 lines of cache by using RAM resources in an FPGA to calculate, and a disparity map median filtering result is output.
And the FPGA carries out consistency detection on the median filtering result, and if the median filtering result is consistent, a corresponding parallax value is output.
Optionally, the FPGA performs consistency detection on the median filtering result, and further includes:
and if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting mean-filtered disparity values.
That is, in the coincidence detection, if all the parallax values of the coincidence detection coincide, the parallax value result is output as the parallax output. If the disparity values are consistent and the disparity values are inconsistent, mean filtering is carried out on the inconsistent disparity values for filling, and the filled disparity values and the disparity values which are detected to be consistent are output as disparity output.
And obtaining the distance between the vehicle and the object according to the output parallax value and the binocular ranging principle.
Optionally, the performing, by the FPGA, consistency detection on the median filtering result includes:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
In a specific application scene, with a left disparity map as a reference, performing left-right consistency detection by using the left-right disparity map, verifying whether a point (x, y) in the left disparity map and the disparity value d thereof are consistent with the disparity of a corresponding point (x + d, y) in the right disparity map, if the difference between the point (x, y) and the disparity value d thereof exceeds 1, determining that the verification fails, and setting the point disparity as 65; if the difference is not more than 1, the original value is retained. And (4) carrying out 3x3 mean filtering on the parallax value (65) which fails to be verified for filling, obtaining the final parallax, and outputting a detection result.
In a specific application scene, the binocular camera inputs data into the FPGA through the MIPI CSI interface, the FPGA caches the data into the DDR through the DDR controller, then left and right images are read from the DDR to perform parallax calculation, and finally the obtained parallax result is output.
As shown in fig. 2, the MIPI CSI interface of the FPGA platform receives the data flow, and the MIPI of the two links are decoded and merged by the serial-parallel merging unit to obtain the DDR pre-stored data.
As shown in fig. 3, which is a flowchart of the parallax calculation unit, after an image is input, the image passes through the Census calculation unit, the Hamming theorem calculation unit, the cost calculation unit, the WTA calculation unit, the median filtering calculation unit, the left-right consistency detection calculation unit, and the mean filtering calculation unit, and then the obtained parallax calculation result is output.
As above formula, it is Census calculation formula, where p refers to the middle pixel in 5x5 pixels, q refers to 24 pixels except the middle pixel in 5x5 pixels, t (p) is Census result of pixel p, ξ is the middle variable after pixel comparison, Np is the rest 24 pixels except the middle pixel in 5x5 pixels, i (p) is the middle pixel value, i (q) is the rest 24 pixel values. By comparison with the value of the middle position, a value less than the middle position takes 1, otherwise it is 0, constituting a Census value.
As shown in fig. 5, in order to implement Census mode for FPGA, 4 lines (or 5 lines) of data are buffered, and calculation is performed by using a template with a size of 5 × 5, so as to obtain a 24-bit data value.
As the formula, Lr is the calculation result of cost aggregation in a certain direction, C is the current cost value, S is the sum of costs in multiple directions, and p-r represents that a point p is calculated in the r direction; k. i is all values of the parallax d; r is the parallax propagation direction; p1 and P2 are both penalty items, which are respectively punished for the case that the parallax difference of the adjacent pixel points is 1 and the parallax difference exceeds 1.
And for the semi-global matching algorithm, calculating the cost of a certain parallax direction according to a formula. And respectively calculating the parallax costs of the positive 4 directions by utilizing a pipeline structure of the FPGA, namely four directions from top to bottom, from left to right, obliquely right to bottom and obliquely left to bottom. And finally, performing summation calculation according to the obtained 4 directional costs to finally obtain the total calculation cost. In the FPGA, a dual-port RAM is utilized to cache cost values of the data in the last line in four directions, and the calculation of the line is carried out, so that the algorithm flow of the pipeline structure is realized.
The parallax map is obtained by using a WTA method, and the specific calculation method is to select the parallax with the minimum matching cost of each pixel point p in the parallax space map S (p, d) to obtain the parallax map. Then, the median filtering is performed as shown in fig. 5, 3 lines of data are cached, calculation is performed according to a 3x3 template, noise is filtered, and a parallax result after the median filtering is obtained.
For the left-right consistency detection algorithm, the FPGA firstly caches the right disparity map for calculation, verifies whether the point (x, y) in the left disparity map and the disparity value d thereof are consistent with the disparity of the corresponding point (x + d, y) in the right disparity map, if the difference exceeds 1, the verification is considered to be failed, and the point disparity is set as 65; if the difference is not more than 1, the original value is retained.
And (3) performing 3x3 mean filtering on the parallax value which fails to be verified to fill up, performing same-median filtering in an FPGA (field programmable gate array), caching three lines of data, judging whether the middle value is an invalid parallax value, if so, performing mean filtering, otherwise, keeping the original value to obtain the final parallax, and outputting a detection result, wherein the steps are shown in fig. 6a to 6 c.
The binocular camera ranging method implemented by the vehicle ranging system adopts a full-pipeline structure, reasonably utilizes the parallelism processing architecture and the internal cache structure of the FPGA to process, can achieve a parallax processing effect of more than 50 frames when the input image of the camera is 1280x800, well meets the requirements of vehicle-mounted real-time performance, and meanwhile, the accuracy of parallax obtaining can also be well met.
The embodiment of the present disclosure further provides a vehicle distance measuring method, including:
transmitting left and right view data acquired by an image acquisition device to an FPGA platform, wherein the FPGA platform comprises an FPGA;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
performing distance calculation on the matching calculation result to obtain a distance calculation result;
performing cost calculation on the distance calculation result to obtain a parallax space diagram;
calculating the parallax space image by using a WTA algorithm to obtain a parallax image;
performing median filtering on the disparity map to obtain a median filtering result;
and carrying out consistency detection on the median filtering result, and outputting a corresponding parallax value if the median filtering result is consistent with the median filtering result.
Optionally, the step of performing consistency detection on the median filtering result further includes:
and if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting mean-filtered disparity values.
Optionally, the transmitting the left and right view data acquired by the image acquiring apparatus to the FPGA platform includes:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
and storing the merged data to a cache device.
Optionally, the performing binocular matching calculation on the left view data and the right view data by the FPGA platform to obtain a matching calculation result includes:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
and when both FIFOs have data, performing binocular matching calculation.
Optionally, the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM buffers N rows of data, binocular matching calculation is started from N +1 rows of pixel input of the RAM.
Optionally, the binocular matching calculation further includes:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the rest pixels are smaller than the base criterion, the gray values are recorded as 1, otherwise, the gray values are 0, and the results are output in sequence.
Optionally, the performing distance calculation on the matching calculation result to obtain a distance calculation result includes:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
Optionally, the performing cost calculation on the distance calculation result to obtain a disparity space map includes:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
Optionally, the calculating the disparity space map by using a WTA algorithm to obtain a disparity map includes:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
Optionally, performing consistency detection on the median filtering result includes:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.
Claims (10)
1. A vehicular distance measuring system, comprising: the system comprises an image acquisition device and an FPGA platform, wherein the FPGA platform comprises an FPGA;
the left view data and the right view data acquired by the image acquisition device are transmitted to the FPGA;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
the FPGA carries out distance calculation on the matching calculation result to obtain a distance calculation result;
the FPGA carries out cost calculation on the distance calculation result to obtain a parallax space diagram;
the FPGA calculates the parallax space image by using a WTA algorithm to obtain a parallax image;
the FPGA carries out median filtering on the disparity map to obtain a median filtering result;
and the FPGA carries out consistency detection on the median filtering result, and if the median filtering result is consistent with the median filtering result, a corresponding parallax value is output.
2. The vehicular ranging system according to claim 1, wherein the FPGA performs consistency detection on the median filtering result, and further comprises:
if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting the disparity values after mean filtering
And/or
The left and right view data acquired by the image acquisition device are transmitted to the FPGA platform, and the method comprises the following steps:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
storing the merged data to a cache device;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result, and the method comprises the following steps:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
when both FIFOs have data, binocular matching calculation is carried out;
the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM caches N rows of data, binocular matching calculation is started from the input of N +1 rows of pixels of the RAM;
the binocular matching calculation further comprises:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the other pixels are smaller than the base criterion and are recorded as 1, otherwise, the gray values are 0, and the results are sequentially arranged and output;
and/or
The FPGA carries out distance calculation on the matching calculation result to obtain a distance calculation result, and the distance calculation result comprises the following steps:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
3. The vehicular distance measuring system according to claim 1, wherein the FPGA performs cost calculation on the distance calculation result to obtain a parallax space map, and the cost calculation comprises:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
4. The vehicular distance measuring system according to claim 3, wherein the FPGA calculates the disparity space map by using a WTA algorithm to obtain a disparity map, and the method comprises the following steps:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
5. The vehicular ranging system according to claim 4, wherein the FPGA performs consistency detection on the median filtering result, and the consistency detection comprises:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
6. A ranging method for a vehicle, comprising:
transmitting left and right view data acquired by an image acquisition device to an FPGA platform, wherein the FPGA platform comprises an FPGA;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result;
performing distance calculation on the matching calculation result to obtain a distance calculation result;
performing cost calculation on the distance calculation result to obtain a parallax space diagram;
calculating the parallax space image by using a WTA algorithm to obtain a parallax image;
performing median filtering on the disparity map to obtain a median filtering result;
and carrying out consistency detection on the median filtering result, and outputting a corresponding parallax value if the median filtering result is consistent with the median filtering result.
7. The vehicular ranging method according to claim 6, wherein the step of performing consistency detection on the median filtering result further comprises:
if the disparity values are inconsistent, performing mean filtering on the corresponding disparity values, and outputting the disparity values after mean filtering
And/or
The left and right view data acquired by the image acquisition device are transmitted to the FPGA platform, and the method comprises the following steps:
performing serial-parallel combination on the left view data and the right view data to obtain combined data;
storing the merged data to a cache device;
the FPGA carries out binocular matching calculation on the left view data and the right view data to obtain a matching calculation result, and the method comprises the following steps:
enabling a parallax computation flag when the merged data stored in the cache device reaches a threshold;
reading the merged data in the cache device and temporarily storing the merged data in the two FIFOs respectively;
when both FIFOs have data, binocular matching calculation is carried out;
the binocular matching calculation includes:
simultaneously extracting data in the two FIFOs, and caching the extracted data line by utilizing N dual-port RAMs;
after the RAM caches N rows of data, binocular matching calculation is started from the input of N +1 rows of pixels of the RAM;
the binocular matching calculation further comprises:
selecting the gray value of the corresponding pixel as a reference;
comparing the gray values of the rest pixels with the reference;
if the gray values of the other pixels are smaller than the base criterion and are recorded as 1, otherwise, the gray values are 0, and the results are sequentially arranged and output;
and/or
The distance calculation of the matching calculation result to obtain a distance calculation result includes:
the matching calculation result comprises left and right image frames;
taking one of the image frames as a reference, and setting a position pixel for exclusive OR operation after taking a corresponding position and a corresponding position from the other image frame;
and adding the result of the exclusive or operation of each bit of each disparity value in the matching calculation result.
8. The vehicular distance measuring method according to claim 6, wherein the calculating the cost of the distance calculation result to obtain the disparity space map comprises:
carrying out cost calculation in multiple directions through the FPGA to obtain cost values in multiple directions;
and summing the cost values in the multiple directions to obtain the parallax space diagram.
9. The vehicular distance measuring method according to claim 8, wherein the calculating the disparity space map by using a WTA algorithm to obtain the disparity map comprises:
and selecting the parallax with the minimum matching cost of each pixel point in the parallax space image to obtain the parallax image.
10. The ranging method for vehicles according to claim 9, wherein the performing consistency detection on the median filtering result comprises:
the parallax graphs are two, and whether the coordinate points of the pixels in the parallax graph serving as the reference and the parallaxes of the corresponding parallax values and the coordinate points of the corresponding pixels in the other parallax graph are consistent or not is verified by taking one of the parallax graphs as the reference;
if the parallax exceeds a set threshold, setting the parallax of the corresponding pixel point as a set value; and if the parallax does not exceed the set threshold value, keeping the original value.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010518686.9A CN111762155B (en) | 2020-06-09 | 2020-06-09 | Vehicle distance measuring system and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010518686.9A CN111762155B (en) | 2020-06-09 | 2020-06-09 | Vehicle distance measuring system and method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111762155A true CN111762155A (en) | 2020-10-13 |
CN111762155B CN111762155B (en) | 2022-06-28 |
Family
ID=72720428
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010518686.9A Active CN111762155B (en) | 2020-06-09 | 2020-06-09 | Vehicle distance measuring system and method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111762155B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112306064A (en) * | 2020-11-04 | 2021-02-02 | 河北省机电一体化中试基地 | RGV control system and method for binocular vision identification |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009134155A1 (en) * | 2008-05-02 | 2009-11-05 | Auckland Uniservices Limited | Real-time stereo image matching system |
CN105719290A (en) * | 2016-01-20 | 2016-06-29 | 天津师范大学 | Binocular stereo depth matching method adopting time domain visual sensor |
CN106355608A (en) * | 2016-09-09 | 2017-01-25 | 南京信息工程大学 | Stereoscopic matching method on basis of variable-weight cost computation and S-census transformation |
CN107403448A (en) * | 2017-07-26 | 2017-11-28 | 海信集团有限公司 | Cost function generation method and cost function generating means |
CN107990878A (en) * | 2017-11-08 | 2018-05-04 | 中国科学院西安光学精密机械研究所 | Range-measurement system and distance measuring method based on low-light binocular camera |
CN109410266A (en) * | 2018-09-18 | 2019-03-01 | 合肥工业大学 | Stereo Matching Algorithm based on four mould Census transformation and discrete disparity search |
CN110110682A (en) * | 2019-05-14 | 2019-08-09 | 西安电子科技大学 | The semantic stereo reconstruction method of remote sensing images |
CN110928318A (en) * | 2019-12-31 | 2020-03-27 | 苏州清研微视电子科技有限公司 | Binocular vision aided driving system based on FPGA |
US20200162719A1 (en) * | 2017-02-07 | 2020-05-21 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision |
US20200177870A1 (en) * | 2017-02-07 | 2020-06-04 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision and tracking |
-
2020
- 2020-06-09 CN CN202010518686.9A patent/CN111762155B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009134155A1 (en) * | 2008-05-02 | 2009-11-05 | Auckland Uniservices Limited | Real-time stereo image matching system |
CN105719290A (en) * | 2016-01-20 | 2016-06-29 | 天津师范大学 | Binocular stereo depth matching method adopting time domain visual sensor |
CN106355608A (en) * | 2016-09-09 | 2017-01-25 | 南京信息工程大学 | Stereoscopic matching method on basis of variable-weight cost computation and S-census transformation |
US20200162719A1 (en) * | 2017-02-07 | 2020-05-21 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision |
US20200177870A1 (en) * | 2017-02-07 | 2020-06-04 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision and tracking |
CN107403448A (en) * | 2017-07-26 | 2017-11-28 | 海信集团有限公司 | Cost function generation method and cost function generating means |
CN107990878A (en) * | 2017-11-08 | 2018-05-04 | 中国科学院西安光学精密机械研究所 | Range-measurement system and distance measuring method based on low-light binocular camera |
CN109410266A (en) * | 2018-09-18 | 2019-03-01 | 合肥工业大学 | Stereo Matching Algorithm based on four mould Census transformation and discrete disparity search |
CN110110682A (en) * | 2019-05-14 | 2019-08-09 | 西安电子科技大学 | The semantic stereo reconstruction method of remote sensing images |
CN110928318A (en) * | 2019-12-31 | 2020-03-27 | 苏州清研微视电子科技有限公司 | Binocular vision aided driving system based on FPGA |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112306064A (en) * | 2020-11-04 | 2021-02-02 | 河北省机电一体化中试基地 | RGV control system and method for binocular vision identification |
Also Published As
Publication number | Publication date |
---|---|
CN111762155B (en) | 2022-06-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10885398B2 (en) | Joint 3D object detection and orientation estimation via multimodal fusion | |
CN110910453B (en) | Vehicle pose estimation method and system based on non-overlapping view field multi-camera system | |
CN104835136B (en) | The system and method for generating depth map by iterated interpolation and distortion | |
US8817079B2 (en) | Image processing apparatus and computer-readable recording medium | |
US10484665B2 (en) | Camera parameter set calculation method, recording medium, and camera parameter set calculation apparatus | |
JP4943034B2 (en) | Stereo image processing device | |
WO2022151664A1 (en) | 3d object detection method based on monocular camera | |
TW200846218A (en) | Device and method for detecting obstacle by stereo computer vision | |
KR20190087258A (en) | Object pose estimating method and apparatus | |
CN112889071A (en) | System and method for determining depth information in two-dimensional images | |
CN111369617A (en) | 3D target detection method of monocular view based on convolutional neural network | |
US11645773B2 (en) | Method for acquiring distance from moving body to at least one object located in any direction of moving body by performing near region sensing and image processing device using the same | |
CN111762155B (en) | Vehicle distance measuring system and method | |
CN214587004U (en) | Stereo matching acceleration circuit, image processor and three-dimensional imaging electronic equipment | |
CN102778684B (en) | Embedded monocular passive target tracking positioning system and method based on FPGA (Field Programmable Gate Array) | |
JP6674959B2 (en) | Parallax calculating device, stereo camera device, vehicle, and parallax calculating method | |
US20230109473A1 (en) | Vehicle, electronic apparatus, and control method thereof | |
Onkarappa et al. | On-board monocular vision system pose estimation through a dense optical flow | |
CN115191928A (en) | Information processing apparatus, information processing method, learning method, and storage medium | |
CN114648639A (en) | Target vehicle detection method, system and device | |
CN113763560B (en) | Method, system, equipment and computer readable storage medium for generating point cloud data | |
CN115063772B (en) | Method for detecting vehicles after formation of vehicles, terminal equipment and storage medium | |
US20240070909A1 (en) | Apparatus and method for distance estimation | |
US20230102186A1 (en) | Apparatus and method for estimating distance and non-transitory computer-readable medium containing computer program for estimating distance | |
Thomas et al. | Architecture and implementation of real-time 3d stereo vision on a xilinx fpga |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |