CN111284490B

CN111284490B - Method for detecting vehicle sliding of front vehicle by vehicle-mounted binocular camera and vehicle-mounted binocular camera

Info

Publication number: CN111284490B
Application number: CN201811489505.3A
Authority: CN
Inventors: 夏克江; 李广琴; 冯谨强
Original assignee: Hisense Co Ltd
Current assignee: Hisense Co Ltd
Priority date: 2018-12-06
Filing date: 2018-12-06
Publication date: 2021-06-04
Anticipated expiration: 2038-12-06
Also published as: CN111284490A

Abstract

The invention provides a method for detecting the sliding of a front vehicle by a vehicle-mounted binocular camera and the vehicle-mounted binocular camera, which can convert a first V disparity map of a first front vehicle image of the shot front vehicle into a first VZ map, calculate the horizontal coordinate variation of the front vehicle in the first VZ map and a second VZ map, wherein the second VZ map is obtained by converting a second V disparity map of a second front vehicle image of the front vehicle shot previously; and if the variation of the horizontal coordinate exceeds a threshold value, determining that the front vehicle slides. Therefore, whether the front vehicle slips or not can be accurately judged by calculating the variation of the abscissa of the front vehicle in the first VZ image and the second VZ image, so that the vehicle slipping early warning is made in advance, and the accuracy of vehicle slipping detection is greatly improved.

Description

Method for detecting vehicle sliding of front vehicle by vehicle-mounted binocular camera and vehicle-mounted binocular camera

Technical Field

The invention relates to the technical field of binocular vision and auxiliary driving, in particular to a method for detecting vehicle sliding of a front vehicle by using a vehicle-mounted binocular camera and the vehicle-mounted binocular camera.

Background

In the traditional method for detecting the sliding of the front vehicle, a distance sensor is generally adopted for detection, such as an infrared sensor, a millimeter wave radar and the like, and the sliding warning can be carried out when the distance is reduced and the movement change reaches a certain threshold value by accurately monitoring the distance between the front vehicle and the vehicle.

However, detecting a rolling motion by sensors generally presents some problems: firstly, when a front vehicle slides, the sliding speed is slow generally, and if the reaction of a sensor is not sensitive, the sliding condition cannot be timely detected; secondly, before the vehicle slides, the two vehicles are in an idle state, the two vehicles can shake to different degrees, and great difficulty is brought to the sensor to detect the vehicle sliding; and thirdly, the purpose of vehicle sliding detection is mainly to perform early warning in time, so that self-adaptive early warning needs to be performed on various conditions such as different distances and different vehicle sliding speeds, and a fixed early warning threshold is not set for different scenes.

In addition, the monocular camera-based method for detecting the fixed characteristic of the front vehicle in the two-dimensional image and judging the vehicle sliding through the coordinate change of the characteristic in the image cannot give an alarm in time because the distance of the front vehicle cannot be accurately estimated; and the vehicle shakes during idling, so that the vehicle slipping is easy to be judged by mistake, and further false alarm is caused.

Disclosure of Invention

In view of the above, the invention provides a method for detecting the vehicle-ahead vehicle sliding by using a vehicle-mounted binocular camera and the vehicle-mounted binocular camera to solve the problem of misjudgment of the vehicle-ahead vehicle sliding in the prior art.

Specifically, the invention is realized by the following technical scheme:

the invention provides a method for detecting vehicle sliding of a front vehicle by using a vehicle-mounted binocular camera, which comprises the following steps:

converting a first V disparity map of a first shot image of a front vehicle into a first VZ map, wherein the abscissa of the first VZ map is a distance value obtained by converting the disparity value of the first V disparity map, and the ordinate is the ordinate of the first V disparity map;

calculating the amount of change of the abscissa of the preceding vehicle in a first VZ map and a second VZ map converted from a second V disparity map of a second preceding vehicle image of the preceding vehicle, which has been captured before;

and if the variation of the horizontal coordinate exceeds a threshold value, determining that the front vehicle slides.

As an embodiment, calculating the amount of change of the abscissa of the preceding vehicle in the first VZ map and the second VZ map includes:

calculating the mean value of the abscissa of the front vehicle in the first VZ image;

calculating the mean value of the abscissa of the front vehicle in a second VZ image;

and the difference between the mean value of the abscissa of the front vehicle in the first VZ diagram and the mean value of the abscissa of the front vehicle in the second VZ diagram is the variation of the abscissa of the front vehicle.

As an embodiment, calculating the abscissa mean of the preceding vehicle in the first VZ map comprises:

searching straight line segments meeting preset conditions in the first VZ image;

matching the straight line segment meeting the preset condition with a preset vehicle tail model;

and calculating the mean value of the abscissa of the front vehicle in the first VZ image according to a preset algorithm corresponding to the matched vehicle tail model.

As an embodiment, the preset vehicle tail model comprises at least a first vehicle tail model, a second vehicle tail model and a third vehicle tail model; the first tail model is a vertical line segment; the second vehicle tail model is a combined line segment formed by connecting the lower end point of an inclined line segment inclining to the left and the upper end point of a vertical line segment; the third tail model is a combined line segment formed by connecting the lower end point of a left inclined oblique line segment with the left end point of a horizontal line segment and connecting the right end point of the horizontal line segment with the upper end point of a vertical line segment.

As an embodiment, finding a straight line segment satisfying a preset condition in the first VZ map includes:

if at least one vertical line segment with an included angle with the abscissa axis within a first angle range is detected in the first VZ diagram;

and selecting the vertical straight line segment with the minimum horizontal coordinate and the maximum vertical coordinate of the upper endpoint from the at least one vertical straight line segment as the straight line segment meeting the preset condition.

As an embodiment, searching for a straight line segment satisfying a preset condition in the first VZ map further includes:

when at least one horizontal line segment with an included angle between the horizontal axis and the first VZ image in a second angle range is detected, if a vertical line segment is detected, calculating a coordinate difference value between the upper end point of the detected vertical line segment and the left end point of the at least one horizontal line segment, and selecting the horizontal line segment with the minimum coordinate difference value from the at least one horizontal line segment as the line segment meeting a preset condition; and if the vertical straight line segment is not detected, selecting the horizontal line segment with the vertical coordinate value of the left endpoint of the horizontal line segment closest to the height of the preset obstacle frame from the at least one horizontal line segment as the straight line segment meeting the preset condition.

when at least one oblique line segment with an included angle between the first VZ image and the abscissa axis in a third angle range is detected, if the first VZ image detects the abscissa line segment, selecting the oblique line segment with the minimum coordinate difference from the at least one oblique line segment as a straight line segment meeting a preset condition according to calculation of the coordinate difference between the right end point of the abscissa line segment and the left end point of the at least one oblique line segment; if the horizontal line segment is not detected but the vertical line segment is detected, selecting the diagonal line segment with the minimum coordinate difference from the at least one horizontal line segment as the line segment meeting the preset condition according to the calculation of the coordinate difference between the upper end point of the vertical line segment and the left end point of the at least one diagonal line segment; and if the horizontal line segment and the vertical line segment are not detected, selecting the diagonal line segment with the vertical coordinate value of the left end point of the diagonal line segment closest to the height of the preset obstacle frame from the at least one diagonal line segment as the straight line segment meeting the preset condition.

Based on the same concept, the invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the computer program realizes any step in the method for detecting the vehicle-mounted binocular camera to roll ahead.

Based on the same conception, the invention also provides a vehicle-mounted binocular camera, which comprises a memory, a processor, a communication interface and a communication bus;

the memory, the processor and the communication interface are communicated with each other through the communication bus;

the memory is used for storing a computer program;

the processor is used for executing the computer program stored in the memory, and any step in the method for detecting the vehicle-mounted binocular camera to slide in the front vehicle is realized when the processor executes the computer program.

Therefore, the invention can convert the first V disparity map of the first front vehicle image of the shot front vehicle into the first VZ map, and calculate the abscissa variation of the front vehicle in the first VZ map and the second VZ map, wherein the second VZ map is obtained by converting the second V disparity map of the second front vehicle image of the previously shot front vehicle; and if the variation of the horizontal coordinate exceeds a threshold value, determining that the front vehicle slides. Therefore, whether the front vehicle slips or not can be accurately judged by calculating the variation of the abscissa of the front vehicle in the first VZ image and the second VZ image, so that the vehicle slipping early warning is made in advance, and the accuracy of vehicle slipping detection is greatly improved.

Drawings

FIG. 1 is a process flow diagram of a method for an on-board binocular camera to detect a vehicle roll in front of a vehicle in an exemplary embodiment of the invention;

2-1 and 2-2 are comparative examples of a V disparity map and a plane-based disparity map, respectively, in an exemplary embodiment of the present invention;

3-1, 3-2, 3-3, 3-4 are comparative example diagrams of a V disparity map and a VZ map in an exemplary embodiment of the invention;

FIG. 4-1 is a first end model illustration in an exemplary embodiment of the invention;

FIG. 4-2 is a second exemplary rear vehicle model in an exemplary embodiment of the invention;

4-3 are exemplary illustrations of a third vehicle tail model in an exemplary embodiment of the invention;

FIG. 5-1 is an exemplary graph of vertical line segment detection results in an exemplary embodiment of the invention;

FIG. 5-2 is an exemplary graph of horizontal line segment detection results in an exemplary embodiment of the invention;

FIGS. 5-3 are exemplary diagrams of diagonal line segment detection results in an exemplary embodiment of the invention;

FIG. 6 is an exemplary illustration of a first tail model calculating a distance active area in an exemplary embodiment of the invention;

FIG. 7 is an exemplary graph of a second tail model a case calculated distance valid region in an exemplary embodiment of the invention;

FIG. 8 is an exemplary plot of a calculated distance effective area for a third vehicle tail model a case in an exemplary embodiment of the invention;

fig. 9 is a logical block diagram of an on-vehicle binocular camera according to an exemplary embodiment of the present invention.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

In order to solve the problems in the prior art, the invention provides a method and a device for detecting rolling of a preceding vehicle, which can convert a first V disparity map of a first preceding vehicle image of the photographed preceding vehicle into a first VZ map, calculate the abscissa variation of the preceding vehicle in the first VZ map and a second VZ map, wherein the second VZ map is obtained by converting a second V disparity map of a second preceding vehicle image of the preceding vehicle, which is photographed before; and if the variation of the horizontal coordinate exceeds a threshold value, determining that the front vehicle slides. Therefore, whether the front vehicle slips or not can be accurately judged by calculating the variation of the abscissa of the front vehicle in the first VZ image and the second VZ image, so that the vehicle slipping early warning is made in advance, and the accuracy of vehicle slipping detection is greatly improved.

Referring to fig. 1, a flowchart of a method for detecting a rolling of a front vehicle by using a vehicle-mounted binocular camera according to an exemplary embodiment of the present invention is shown, where the method includes:

step 101, converting a first V disparity map of a first captured front vehicle image of a front vehicle into a first VZ map, wherein the abscissa of the first VZ map is a distance value obtained by conversion according to the disparity value of the first V disparity map, and the ordinate is the ordinate of the first V disparity map;

in this embodiment, the vehicle-mounted binocular camera may acquire the first forward vehicle image by shooting and obtain the first V disparity map of the first forward vehicle image by image processing. In this embodiment, a V parallax image is used to detect the rolling of the preceding vehicle, the V parallax image is a kind of sparse parallax image, and is characterized in that only a feature point or an edge has a corresponding parallax value, and a non-feature point has an invalid parallax, as shown in fig. 2-1, compared with a plane-based parallax image (as shown in fig. 2-2), the parallax of the feature point in the V parallax image is usually relatively accurate, although parallax noise cannot be avoided, the parallax noise has little influence on the accuracy of the calculated distance. After the first V disparity map of the image of the leading vehicle is acquired, the first V disparity map may be further converted into a first VZ map.

It should be noted that the VZ image of the present invention is similar to the conventional V disparity image, and is different in that the abscissa of the V disparity image is a disparity range (unit is pixel), the ordinate corresponds to the ordinate of the source disparity image, and the origin of the V disparity image is located at the upper left corner of the image; however, the ordinate of the VZ image defined in the present invention is still the ordinate of the image corresponding to the source parallax, but the abscissa of the VZ image does not correspond to the parallax any more, but corresponds to the detection distance value, i.e., the ordinate is the actual distance unit, meter or decimeter, etc., and the origin of the ordinate is located at the upper left corner of the image. The essence of the VZ map is to convert the disparity values corresponding to the V disparity images into corresponding distance values.

The conversion formula for converting the V disparity map into the VZ map is as follows:

wherein, B is the base length of the binocular camera, f is the focal length, d is the parallax value, Δ d is the parallax at infinity, is the calibration value, and z is the distance value corresponding to d.

In the vehicle-rolling detection, because the distance between the front vehicle and the vehicle is short, the distance range in the VZ map of the present invention does not need to be set too large, and because the distance is short, the parallax of the front vehicle is rich, and the matching accuracy is high, therefore in a specific embodiment, the maximum distance in the VZ map is set to be the maximum distance corresponding to the pixel parallax of the front vehicle, for example, the minimum scale in the z direction in the VZ map may be set to be 0.05 m, and the specific embodiment is as follows:

first, based on the parallax belonging to the preceding vehicle in the preceding vehicle obstacle frame, assuming (dmin, dmax), the maximum z value in the VZ map is obtained, that is:

and based on the fact that the minimum scale in the z direction is set to be 0.05 meter, obtaining that the horizontal axis in the VZ diagram has Zmax/0.05 columns, the row number of the vertical axis is equal to the height of the pixel of the obstacle frame, and assuming that the coordinate of the obstacle frame is (u, v, w, h), the vertical axis in the VZ diagram has h rows, so that an image with the height of the pixel of h, the width of the pixel of Zmax/0.05 and the pixel value of all points of zero is generated.

Secondly, the parallax values belonging to the obstacles in the obstacle frame are converted into a VZ diagram one by one, and the implementation method comprises the following steps: traversing the parallax in the obstacle frame (u, v, w, h), if the parallax is within the range of (dmin, dmax), calculating the abscissa x thereof by the following formula (three), that is, calculating the abscissa x

Wherein [ ] is a rounded symbol.

The ordinate y is the value obtained by subtracting v from the coordinate ysrc of the pixel point in the source image, namely

y＝y_src-v formula (four)

For the calculated x and y, a corresponding coordinate point (x, y) in the VZ image can be found, the pixel value corresponding to the coordinate point is added with 1, and after the whole barrier frame is traversed, a VZ image converted based on the disparity map of the front vehicle V in the barrier frame can be obtained.

For example, the effect of the V disparity map and the transformation into the VZ map of the obstacle frame in the same image is as follows:

FIG. 3-1 is a source image with a front vehicle image taken by a vehicle mounted binocular camera; 3-2 are source disparity maps based on the source map, wherein the white frames are barrier frames; 3-3 is a V parallax diagram after parallax conversion of the image in the barrier frame; fig. 3-4 are VZ diagrams of images in the obstacle frame after parallax conversion. As can be seen from the comparison between fig. 3-3 and fig. 3-4, the ordinate of the V disparity map is the disparity value, and the ordinate of the VZ map is the distance value.

102, calculating the variation of the abscissa of the front vehicle in a first VZ image and a second VZ image, wherein the second VZ image is obtained by converting a second V disparity map of a second front vehicle image of the front vehicle which is shot before;

in this embodiment, the amount of change in the abscissa of the preceding vehicle in the first VZ map and the second VZ map obtained by converting the second V disparity map of the second preceding vehicle image of the preceding vehicle, which has been captured before, may be further calculated, and for example, the second VZ map may be obtained by converting the V disparity map of the preceding vehicle image of the previous frame of the first VZ map, may be obtained by converting the V disparity map of the preceding vehicle image of the previous N frames of the first VZ map, or may be obtained by converting the V disparity map of the initial preceding vehicle image.

As an embodiment, calculating the amount of change of the abscissa of the preceding vehicle in the first VZ map and the second VZ map specifically includes: calculating the mean value of the abscissa of the front vehicle in the first VZ image; and calculating the mean value of the abscissa of the front vehicle in the second VZ diagram; and then taking the difference between the mean value of the abscissa of the front vehicle in the first VZ diagram and the mean value of the abscissa of the front vehicle in the second VZ diagram as the variation of the abscissa of the front vehicle.

Calculating an abscissa mean value of the front vehicle on a first VZ image, specifically searching a straight line segment meeting a preset condition in the first VZ image; matching the straight line segment meeting the preset condition with a preset vehicle tail model; and then calculating the mean value of the abscissa of the front vehicle on the first VZ image according to a preset algorithm corresponding to the matched vehicle tail model. The method for calculating the mean value of the abscissa of the front vehicle in the second VZ diagram is similar to the above method, and is not repeated here. The following describes in detail the calculation method of the mean of the abscissa in the first VZ diagram for the preceding vehicle.

As an embodiment, since the tail shapes of different types of vehicles are different, in order to more accurately detect the rolling of the front vehicle, the present invention may further preset a plurality of tail models (the optical axes of the binocular cameras are installed parallel to the road surface) according to the structure and posture of the tail of the vehicle based on the types of vehicles frequently encountered on the road:

the first type: the common vehicle types are buses, trucks, vans and the like, as shown in the left side view (a) of fig. 4-1, and are characterized in that the tail part is basically vertical to the ground in the side view, so that the abstract model of the vehicle is a vertical straight line segment which is used as a first tail model, and the side view of the first tail model is shown in the right side view (b) of fig. 4-1;

the second type: the common vehicle types are SUV, off-road vehicle, etc., as shown in the left side view (a) of fig. 4-2, and it is characterized in that the vehicle tail is a broken line in the side view, i.e. the second vehicle tail model is a combined line segment in which the lower end point of an oblique line segment inclined to the left is connected with the upper end point of a vertical line segment; this is the case of the second rear model a, which is shown in side view in the middle (b) of FIGS. 4-2; such a model, when the front vehicle part is present in the image, the rear of the vehicle may have only one diagonal segment, which is taken as the second rear model b case, i.e. as shown in the right diagram (c) of fig. 4-2;

in the third category: the common vehicle type is a small car and the like, as shown in a first drawing (a) from the left in fig. 4-3, and is characterized in that the tail part of the car in a side view is composed of a plurality of line segments, namely, a combined line segment in which the lower end point of an inclined line segment inclined to the left is connected with the left end point of a horizontal line segment and the right end point of the horizontal line segment is connected with the upper end point of a vertical line segment can be used as the situation of the model a of the third car as shown in a second drawing (b) from the left in fig. 4-3; in such a model, when the front vehicle part is presented in the image, the tail part of the vehicle may have only one oblique line segment or a situation of adding one oblique line segment and one transverse line segment, and the situation of adding one oblique line segment and one transverse line segment to the tail part of the vehicle is taken as the situation of a third tail model b, as shown in a third diagram (c) from the left in fig. 4-3; the situation that the car tail has only one oblique line segment is taken as the situation of the third car tail model c, as shown in the fourth picture (d) from the left of the figure 4-3.

As an embodiment, a hough straight line may be further used in the first VZ diagram to find a straight line segment satisfying a preset condition, and the specific embodiment thereof is as follows:

a first step, if at least one vertical line segment having an angle with the abscissa axis within a first angle range (for example, an angle with the abscissa axis within a range of 80 to 100 degrees) is detected in the first VZ diagram; selecting a vertical straight line segment with the minimum horizontal coordinate and the maximum vertical coordinate of an upper end point from the at least one vertical straight line segment as a straight line segment meeting a preset condition, so that the straight line segment can be reserved, and simultaneously removing residual straight line segments, and simultaneously removing effective pixel points within a certain range of the horizontal coordinate x of the end point on the vertical straight line segment, namely, setting nonzero pixel points within a (x-Tx, x + Tx) range to be zero, thereby avoiding the interference of noise points; if the vertical line segment is not found, the next step is carried out; the actual detection result is shown in fig. 5-1, wherein the left graph (a) shows an initial detection straight line including a plurality of vertical line segments (represented by gray), and the vertical line segments obtained after screening according to the above determination conditions are shown in the middle graph (b); selecting the vertical line segment retained after selection, deleting the vertical line segment and the points around the vertical line segment, and the deleted result is shown in the right side graph (c).

Secondly, when at least one horizontal line segment with an included angle with the abscissa axis in a second angle range (for example, the range with the included angle with the abscissa axis being-10 to 10 degrees) is detected in the first VZ image, if a vertical line segment is detected, calculating a coordinate difference value between the upper end point of the detected vertical line segment and the left end point of the at least one horizontal line segment, and selecting the horizontal line segment with the minimum coordinate difference value from the at least one horizontal line segment as a line segment meeting a preset condition; if the vertical straight line segment is not detected, selecting the horizontal line segment with the vertical coordinate value of the left end point of the horizontal line segment closest to the height of the preset barrier frame from the at least one horizontal line segment as the straight line segment meeting the preset condition, further reserving the horizontal line segment, and simultaneously rejecting effective pixel points in a certain range around the vertical coordinate y of the left end point of the horizontal line segment, namely setting nonzero pixel points in the range of (y-Ty, y + Ty) to be zero, and entering the next step; the actual detection result is shown in fig. 5-2, wherein the left graph (a) shows an initial detection straight line including a plurality of horizontal line segments (indicated by gray), and the horizontal line segments obtained after screening according to the above-mentioned determination conditions are shown in the middle graph (b); the horizontal line segment remaining after the selection and the dots around the horizontal line segment are deleted, and the deleted result is shown in the right diagram (c).

Thirdly, when at least one oblique line segment with an included angle with the abscissa axis in a third angle range (for example, a range with an included angle of 10-80 degrees with the abscissa axis) is detected in the first VZ diagram, if the horizontal line segment is detected, selecting the oblique line segment with the minimum coordinate difference from the at least one oblique line segment as a straight line segment meeting a preset condition according to the calculation of the coordinate difference between the right end point of the horizontal line segment and the left end point of the at least one oblique line segment; if the horizontal line segment is not detected but the vertical line segment is detected, selecting the diagonal line segment with the minimum coordinate difference from the at least one horizontal line segment as the line segment meeting the preset condition according to the calculation of the coordinate difference between the upper end point of the vertical line segment and the left end point of the at least one diagonal line segment; if the horizontal line segment and the vertical line segment are not detected, selecting the diagonal line segment with the vertical coordinate value of the left end point of the diagonal line segment closest to the height of the preset obstacle frame from the at least one diagonal line segment as the straight line segment meeting the preset condition, reserving the diagonal line segment, and simultaneously rejecting other diagonal line segments. An example of the actual detection result is shown in fig. 5-3.

Based on the types and the number of the straight line segments obtained in the steps, matching the detected straight line segments with a preset vehicle tail model, and finding out the vehicle tail model corresponding to the detected straight line segments: for example: one vertical line segment directly corresponds to the first tail model; if one oblique line segment and one vertical line segment correspond to the condition of the second tail model a; if one oblique line segment, one horizontal transverse line segment and one vertical line segment correspond to the condition of the third vehicle tail model a, or if one oblique line segment and one horizontal transverse line segment correspond to the condition of the third vehicle tail model b; if one oblique line segment can correspond to the condition of the second tail model b and also can correspond to the condition of the third tail model c.

Further based on an algorithm corresponding to the matched vehicle tail model, calculating an abscissa mean value of the front vehicle on the first VZ image according to the detected straight line segment, specifically as follows:

for the first tail model, i.e. there is a vertical line segment with the corresponding abscissa in the VZ diagram as x, the mean value Zavg of the abscissas in the range of (x-Tx, x + Tx) is calculated as:

wherein ymin and ymax are longitudinal coordinate values of upper and lower end points of a vertical line segment respectively, P (i, j) is a pixel value of a coordinate (i, j) in a VZ diagram, and k is the accumulated sum of the pixel values of P (i, j) in an abscissa range (x-Tx, x + Tx) and an ordinate range (ymin, ymax). Since the abscissa of the leading vehicle is dispersed, calculation errors can be reduced by calculating the average of the abscissas. And calculating the mean value Zavg of the horizontal coordinates of the obstacle frame of the front vehicle in each frame of image from the starting frame, so as to obtain the variation of the horizontal coordinates of every two frames and the total variation from the starting frame to the current frame. The dashed area of fig. 6 is an effective area for calculating Zavg.

For the second tail model, when the starting frame is a, the horizontal coordinate mean value Zavg in the horizontal coordinate range (x-Tx, x + Tx) and the vertical coordinate range (ymin, ymax) is calculated by using the vertical line segment information, and the calculation mode is the same as that of the first tail model; meanwhile, taking the upper left corner endpoint (xL, yL) of the diagonal line segment as the center, taking the point with the abscissa range (xL-Tx, xL + Tx) and the ordinate range (yL-Ty, yL + Ty), and calculating the abscissa mean value ZavgL in the same way, wherein the calculation formula is as follows:

where k is the cumulative sum of the P (i, j) pixel values within the abscissa range (xL-Tx, xL + Tx) and the ordinate range (yL-Ty, yL + Ty); and then the abscissa changes of two adjacent frames Zavg and ZavgL are monitored, and the changes of the two frames Zavg and ZavgL can be synthesized in a weighting mode to be used as the final abscissa change. The effective areas for calculating Zavg and ZavgL are shown as the dashed areas in fig. 7.

For the second type of model, if the starting frame is b, the horizontal coordinate mean value ZavgL is calculated by taking the point with the horizontal coordinate range of (xL-Tx, xL + Tx) and the vertical coordinate range of (yL-Ty, yL + Ty) as the center, and the calculation method is the same as the above formula (six), thereby monitoring the horizontal coordinate variation of the front vehicle.

For the third type of model, if the initial frame is in the case of a, calculating according to the case of a of the initial frame of the second type of model, namely respectively calculating the horizontal coordinate variation of two adjacent frames Zavg and ZavgL by using the left end point of an oblique line segment and a vertical line segment, and finally integrating the two variations as the final horizontal coordinate variation through a weighting mode; and meanwhile, the length of the horizontal line segment can be recorded, namely the length values of the horizontal line segment of different frames are accumulated, and the accumulated frame number is recorded, so that the average value of the lengths of the previous horizontal line segments is calculated when the situation a is changed into the situation b in the later period. The effective areas for calculating Zavg and ZavgL are shown as the dashed areas in fig. 8.

For the b condition of the third type model, firstly calculating ZavgL according to the b condition of the second type model, then calculating ZavgV based on the length change of each frame horizontal line segment, weighting the ZavgL and the ZavgV, and finally calculating the horizontal coordinate change of the front vehicle according to the weighting result.

For the third type of model, the starting frame is c, and similarly, the processing is performed according to the b condition of the second type of model, that is, ZavgL is calculated to monitor the horizontal coordinate variation of the front vehicle.

Based on the method, the distance change between the front vehicle and the vehicle can be effectively and accurately calculated for all the situations of the first vehicle tail model, the second vehicle tail model and the third vehicle tail model. Different models are established by distinguishing different vehicle tail types, and the horizontal coordinate variation of the front vehicle can be calculated according to the characteristics of the models and corresponding formulas, so that the judgment result of the vehicle sliding of the front vehicle is more accurate.

And 103, if the variation of the abscissa exceeds a threshold value, determining that the front vehicle slides.

In this embodiment, whether the preceding vehicle rolls or not can be determined from the calculated amount of change of the abscissa. Specifically, the abscissa variation of the two adjacent frames of the preceding vehicle and the host vehicle can be obtained, and further the abscissa variation of the current frame and the previous frame can be obtained, at this time, a distance variation threshold Ts can be set, and if the abscissa variation exceeds the variation threshold, the preceding vehicle can be considered to roll, and further a rolling alarm can be triggered.

In an alternative embodiment, while monitoring the abscissa, the rolling relative speed v may also be calculated based on the amount of change in the abscissa, namely:

wherein, s2 is the distance value at t2 moment, and s1 is the distance value at t1 moment, sets up early warning time threshold Ts, if v × Tt > s2, the abscissa variation volume can send the early warning in advance when not exceeding threshold Ts this moment, when the speed of swift current is very fast promptly, can be in time early warning under the prerequisite that does not reach swift current change threshold to can guarantee the timeliness of early warning on the basis of swift current warning stability, further guarantee the early warning.

Based on the same concept, the present invention also provides a vehicle-mounted binocular camera, as shown in fig. 9, including a memory 91, a processor 92, a communication interface 93, and a communication bus 94;

the memory 91, the processor 92 and the communication interface 93 communicate with each other through the communication bus 94;

the memory 91 is used for storing computer programs;

the processor 92 is configured to execute the computer program stored in the memory 91, and when the processor 92 executes the computer program, any step of the method for detecting the rolling of the front vehicle by using the vehicle-mounted binocular camera according to the embodiment of the present invention is implemented.

The invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed by a processor, the computer program realizes any step of the method for detecting the vehicle-mounted binocular camera to slide in the front vehicle provided by the embodiment of the invention.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for embodiments of the computer device and the computer-readable storage medium, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to what is described in the partial description of the method embodiments.

In summary, the present invention can convert the first V disparity map of the first previous vehicle image of the previous vehicle into the first VZ map, and calculate the abscissa variation of the previous vehicle in the first VZ map and the second VZ map, where the second VZ map is obtained by converting the second V disparity map of the second previous vehicle image of the previous vehicle, which has been captured previously; and if the variation of the horizontal coordinate exceeds a threshold value, determining that the front vehicle slides. Therefore, whether the front vehicle slips or not can be accurately judged by calculating the variation of the abscissa of the front vehicle in the first VZ image and the second VZ image, so that the vehicle slipping early warning is made in advance, and the accuracy of vehicle slipping detection is greatly improved.

The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.

For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for detecting vehicle sliding of a front vehicle by using a vehicle-mounted binocular camera is characterized by comprising the following steps:

converting a first V disparity map of a first captured front vehicle image into a first VZ map according to a first expression, wherein the abscissa of the first VZ map is a distance value obtained by conversion according to the disparity value of the first V disparity map, and the ordinate of the first VZ map is the ordinate of the first V disparity map;

the first expression is:

b is the base length of the binocular camera, f is the focal length, d is the parallax value, delta d is the parallax at infinity, and z is the distance value corresponding to d;

2. The method of claim 1, wherein calculating the amount of change in the abscissa of the leading vehicle in the first VZ map and the second VZ map comprises:

3. The method of claim 2, wherein calculating the mean of the abscissa of the leading vehicle in the first VZ map comprises:

4. The method of claim 3,

the preset vehicle tail model at least comprises a first vehicle tail model, a second vehicle tail model and a third vehicle tail model; the first tail model is a vertical line segment; the second vehicle tail model is a combined line segment formed by connecting the lower end point of an inclined line segment inclining to the left and the upper end point of a vertical line segment; the third tail model is a combined line segment formed by connecting the lower end point of a left inclined oblique line segment with the left end point of a horizontal line segment and connecting the right end point of the horizontal line segment with the upper end point of a vertical line segment.

5. The method of claim 4, wherein finding a straight-line segment in the first VZ map that satisfies a predetermined condition comprises:

6. The method of claim 5, wherein finding a straight line segment in the first VZ map that meets a preset condition further comprises:

7. The method of claim 6, wherein finding a straight line segment in the first VZ map that meets a preset condition further comprises:

8. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method of any one of the claims 1-7.

9. The vehicle-mounted binocular camera is characterized by comprising a memory, a processor, a communication interface and a communication bus;

the memory is used for storing a computer program;

the processor for executing the computer program stored on the memory, the processor implementing the method according to any one of claims 1-7 when executing the computer program.