CN118254817A

CN118254817A - Lane departure warning method, electronic device, system and storage medium

Info

Publication number: CN118254817A
Application number: CN202211692971.8A
Authority: CN
Inventors: 刘红
Original assignee: WM Smart Mobility Shanghai Co Ltd
Current assignee: WM Smart Mobility Shanghai Co Ltd
Filing date: 2022-12-28
Publication date: 2024-06-28

Abstract

The invention discloses a lane departure warning method, electronic equipment, a system and a storage medium. The lane departure warning method comprises the following steps: lane departure recognition is performed based on the output information of the vehicle monocular camera device; and if the vehicle is detected to be deviated based on the output information of the monocular image pickup device, checking based on the predicted track coordinates of the vehicle, and if the checking is passed, outputting a lane deviation early warning signal. According to the invention, when the image pickup device recognizes lane departure, the verification of the predicted track coordinates of the vehicle is increased, and after the predicted track coordinates of the vehicle pass the verification, the lane departure warning signal is output, so that the false alarm caused by the recognition mutation of the monocular image pickup device is effectively filtered.

Description

Lane departure warning method, electronic device, system and storage medium

Technical Field

The invention relates to the technical field of vehicles, in particular to a lane departure warning method, electronic equipment, a system and a storage medium.

Background

The current lane departure warning function is mainly realized by a monocular camera, but is limited by the limitation of the monocular camera on lane line perception, and cannot accurately identify the lane lines which are dislocated on the ground, asphalt repair marks and the like. Secondly, on bumpy roads, the identification of lane line attributes by the monocular cameras is affected by pitching, cornering and the like of the vehicle.

Therefore, the existing lane line sensing method based on the monocular camera is easy to cause false triggering of the lane departure warning function.

Disclosure of Invention

Based on the above, it is necessary to provide a lane departure warning method, an electronic device, a system and a storage medium for solving the technical problem that the lane departure warning function is triggered by mistake in the lane line sensing method based on the monocular camera in the prior art.

The invention provides a lane departure warning method, which comprises the following steps:

lane departure recognition is performed based on the output information of the vehicle monocular camera device;

And if the vehicle is detected to be deviated based on the output information of the monocular image pickup device, checking based on the predicted track coordinates of the vehicle, and if the checking is passed, outputting a lane deviation early warning signal.

According to the invention, when the image pickup device recognizes lane departure, the verification of the predicted track coordinates of the vehicle is increased, and after the predicted track coordinates of the vehicle pass the verification, the lane departure warning signal is output, so that the false alarm caused by the recognition mutation of the monocular image pickup device is effectively filtered.

Further, the lane departure recognition based on the output information of the vehicle monocular image pickup device specifically includes:

Acquiring the left shooting transverse distance between the lens center of the image pickup device and the left lane line and the right shooting transverse distance between the lens center of the image pickup device and the right lane line, wherein the lens center of the monocular image pickup device is positioned on the central axis of the vehicle;

and identifying whether the lane is deviated or not according to the left shooting transverse distance and the right shooting transverse distance.

According to the embodiment, based on the shooting transverse distance between the lens center of the imaging device and the left and right lane lines, lane departure recognition is achieved, and the accuracy of lane departure recognition is improved.

Still further, the identifying whether the lane departure is based on the left side photographing lateral distance and the right side photographing lateral distance specifically includes:

And if the difference value of the left shooting transverse distance minus the half vehicle body width is smaller than or equal to a first difference threshold value, or the difference value of the right shooting transverse distance minus the half vehicle body width is smaller than or equal to the first difference threshold value, judging that the lane departure is recognized, otherwise, judging that the lane departure is not recognized.

The embodiment specifically realizes lane departure recognition based on the relationship between the left and right side shooting lateral distance and the vehicle body width.

Further, the verifying based on the predicted track coordinates of the vehicle specifically includes:

Acquiring a predicted track coordinate of a vehicle;

calculating left predicted transverse distance between the vehicle predicted track coordinates and a left lane line and right predicted transverse distance between the lens center and a right lane line;

And checking according to the left predicted transverse distance and the left shooting transverse distance and/or the right predicted transverse distance and the right shooting transverse distance.

According to the embodiment, the vehicle prediction track coordinates, the left and right prediction transverse distances of the left and right lane lines and the left and right shooting transverse distances are used for verification, so that the accuracy of lane departure recognition is improved.

Still further, the verifying according to the left predicted lateral distance and the left shooting lateral distance, and/or the right predicted lateral distance and the right shooting lateral distance specifically includes:

And if the absolute value of the difference between the left predicted transverse distance and the left shooting transverse distance is smaller than or equal to a preset second difference threshold, or the absolute value of the difference between the right predicted transverse distance and the right shooting transverse distance is smaller than or equal to a preset second difference threshold, judging that the verification passes, otherwise, judging that the verification does not pass.

According to the method, the device and the system, the difference value between the left and right predicted lateral distances and the left and right shot lateral distances is used for verification, and the accuracy of lane departure identification is improved.

Further, if the lane departure is identified based on the output information of the monocular image capturing device, checking is performed based on the predicted track coordinates of the vehicle, and if the checking is passed, a lane departure warning signal is output, which specifically includes:

if the lane departure is recognized based on the monocular image pickup device output information, judging whether the motion gesture of the vehicle body is changed or not;

if the motion gesture of the vehicle body is changed, checking based on the predicted track coordinates of the vehicle, and if the checking is passed, outputting a lane departure warning signal; otherwise

And outputting a lane departure warning signal if the motion gesture of the vehicle body is not changed.

In the embodiment, when the monocular image pickup device recognizes lane departure, whether the motion gesture of the vehicle body is changed is judged, and when the motion gesture of the vehicle body is changed, verification is performed based on the predicted track coordinates of the vehicle, otherwise, a lane departure early warning signal is directly output, so that invalid verification is avoided.

Still further, the determining whether the motion gesture of the vehicle body changes specifically includes:

Acquiring steering wheel rotation angle, vehicle pitch angle and/or vehicle yaw angle;

and if the absolute value of the steering wheel rotation angle, the vehicle pitch angle or the change rate of the vehicle yaw angle exceeds the corresponding change rate threshold value, judging that the vehicle body movement posture is changed, otherwise, judging that the vehicle body movement posture is not changed.

According to the embodiment, whether the motion gesture of the vehicle body is changed or not is accurately judged through the steering wheel rotation angle, the vehicle pitch angle and/or the vehicle yaw angle.

Still further, the method further comprises:

if the verification is not passed, lane departure recognition is performed again based on the vehicle monocular camera device.

In the embodiment, when the verification fails, lane departure recognition is performed again based on the vehicle monocular image pickup device so as to avoid false recognition.

The present invention provides an electronic device including:

At least one processor; and

A memory communicatively coupled to at least one of the processors; wherein,

The memory stores instructions executable by at least one of the processors to enable the at least one processor to perform a lane departure warning method as previously described.

The present invention provides a storage medium storing computer instructions that, when executed by a computer, are operable to perform all the steps of a lane departure warning method as previously described.

Drawings

FIG. 1 is a flowchart illustrating a lane departure warning method according to an embodiment of the present invention;

FIG. 2 is a flowchart showing a lane departure warning method according to a preferred embodiment of the present invention;

FIG. 3 is a diagram showing the interaction of system signals in a preferred embodiment of the present invention;

fig. 4 is a schematic hardware structure of an electronic device according to the present invention.

Detailed Description

Specific embodiments of the present invention will be further described below with reference to the accompanying drawings. Wherein like parts are designated by like reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the drawings, and the words "inner" and "outer" refer to directions toward or away from, respectively, the geometric center of a particular component.

Fig. 1 is a flowchart of a lane departure warning method according to an embodiment of the present invention, including:

Step S101, lane departure recognition is carried out based on the output information of the vehicle monocular image pickup device;

And step S102, if the lane departure is identified based on the output information of the monocular image pickup device, checking based on the predicted track coordinates of the vehicle, and if the checking is passed, outputting a lane departure warning signal.

In particular, the invention may be applied to an electronic controller unit (Electronic Control Unit, ECU) of a vehicle. For example by a driving assistance controller (Autonomous Driving Controller, ADC).

First, step S101 is executed to perform lane departure recognition based on the vehicle monocular image pickup device output information. Wherein the vehicle monocular camera device is preferably a monocular camera mounted on the vehicle's front windscreen. Lane departure recognition is performed by an image captured by a monocular image capturing device.

Preferably, when the vehicle speed is higher than a preset vehicle speed threshold, step S101 is performed.

In the case where the lane departure is recognized based on the monocular image pickup device, step S102 is executed to perform further verification based on the vehicle predicted trajectory coordinates, and when the verification passes, a lane departure warning signal is output.

In one embodiment, a warning message of the lane departure warning signal is output at a meter (IC) of the vehicle.

In one embodiment, the lane departure recognition based on the output information of the vehicle monocular image capturing device specifically includes:

Specifically, the lens center of the monocular image pickup device is located on the vehicle central axis, the left side shooting transverse distance between the lens center and the left lane line and the right side shooting transverse distance between the lens center and the right lane line are identified through the existing image identification mode, the left side shooting transverse distance is the distance between the vehicle central axis identified through the image and the left lane line, and the right side shooting transverse distance is the distance between the vehicle central axis identified through the image and the right lane line.

In one embodiment, the identifying whether the lane deviates according to the left shooting lateral distance and the right shooting lateral distance specifically includes:

Specifically, if (LL-body width/2). Ltoreq.L1 or (LR-body width/2). Ltoreq.L1, where L1 is a first difference threshold, preferably 20cm, LL is left side shooting lateral distance, LR is right side shooting lateral distance, and body width/2 is half body width, it is judged that lane departure is recognized. Calculating (LL-width of vehicle body/2) to obtain the distance between the left side of the vehicle body and the left lane line, calculating (LR-width of vehicle body/2) to obtain the distance between the right side of the vehicle body and the lane line, and judging that the vehicle body is too close to the lane line if the distance between the left side of the vehicle body and the left lane line or the distance between the right side of the vehicle body and the right lane line is smaller than a first difference value threshold value, thereby recognizing lane departure.

In one embodiment, the verifying based on the vehicle predicted track coordinates specifically includes:

Acquiring a predicted track coordinate of a vehicle;

The vehicle predicted track coordinates are realized by a driving auxiliary controller in an existing track prediction mode. Specifically:

1. The monocular camera device outputs lane line attributes: the lateral distance LL, LR between the center of the camera lens and the left and right lane lines; left and right lane line types TL, TR; actual road starting points SL, SR of left and right lane lines;

2. A vehicle body posture sensor mounted on the vehicle body outputs yaw angle HB and pitch angle information FY of the vehicle;

3. A steering angle sensor mounted on the steering wheel outputs steering wheel angle information FJ;

4. the wheel speed sensor mounted on the tire outputs wheel speed information VS of the vehicle;

And then the monocular camera inputs the lane line attribute information into a driving auxiliary controller, and the driving auxiliary controller predicts and calculates the movement track of the vehicle body by adopting the existing vehicle track prediction mode according to the steering wheel angle and the wheel speed information of the vehicle input by the vehicle sensor to obtain the vehicle predicted track coordinate. Specifically, a lane line starting point is taken as a calculation starting point, a vehicle motion track is predicted by utilizing time integration based on rotation speed information, a vehicle predicted track coordinate of a vehicle in a lane line is obtained, and then the vehicle motion gesture is modified according to vehicle pitching, yaw and steering wheel rotation angle, so that the calculated vehicle predicted track coordinate is modified.

After the vehicle predicted track coordinates are obtained, the transverse coordinates of the left lane line are obtained, the left predicted transverse distance is calculated to be the absolute value of the difference value between the transverse coordinates of the vehicle predicted track coordinates and the transverse coordinates of the left lane line, and the right predicted transverse distance is calculated to be the absolute value of the difference value between the transverse coordinates of the vehicle predicted track coordinates and the transverse coordinates of the right lane line.

The coordinates of the predicted track of the vehicle, the coordinates of the left lane line and the coordinates of the right lane are all in the same coordinate system.

In one embodiment, the verifying according to the left predicted lateral distance and the left shooting lateral distance, and/or the right predicted lateral distance and the right shooting lateral distance specifically includes:

Specifically, if abs (LL '-LL). Ltoreq.L2, or abs (LR' -LR). Ltoreq.L2, the verification is judged to pass, otherwise the verification is judged not to pass. Wherein L2 is a first difference threshold, preferably 10cm, LL is a left-side shooting lateral distance, LR is a right-side shooting lateral distance, LL 'is a left-side predicted lateral distance, LR' is a right-side predicted lateral distance, and abs is an absolute value function.

In one embodiment, if the lane departure is identified based on the output information of the monocular image capturing device, the calibration is performed based on the predicted track coordinates of the vehicle, and if the calibration is passed, a lane departure warning signal is output, which specifically includes:

Specifically, in the monocular image capturing device identification process, if the motion gesture of the whole vehicle body is changed, false alarm may be caused by abrupt change during the monocular image capturing device identification. Therefore, when the motion gesture of the whole vehicle body is changed, the embodiment does not only take the recognition result of the monocular image pickup device on the lane line as the control input of the lane departure warning, and needs to judge whether to send the lane departure warning signal after the lane line checking logic.

In one embodiment, the determining whether the motion gesture of the vehicle body changes specifically includes:

Specifically, the vehicle body motion profile is corrected by the steering wheel angle, the vehicle pitch angle, and/or the vehicle yaw angle, so that the vehicle body motion profile will be transmitted to change when the steering wheel angle, the vehicle pitch angle, and/or the vehicle yaw angle are abrupt.

Wherein the rate of change threshold comprises: steering wheel angle change rate threshold, vehicle pitch angle change rate threshold, and vehicle yaw angle change rate threshold, steering wheel angle, vehicle pitch angle, and/or vehicle yaw angle are abrupt:

The absolute value of the rate of change of the steering wheel angle exceeds the steering wheel angle rate of change threshold; or alternatively

The absolute value of the change rate of the pitch angle of the vehicle exceeds the change rate threshold value of the pitch angle of the vehicle; or alternatively

The absolute value of the rate of change of the vehicle yaw angle exceeds the vehicle yaw rate threshold.

In one embodiment, the method further comprises:

Specifically, when the verification fails, the vehicle monocular image pickup device performs the secondary recognition again, that is, the lane departure recognition is performed again based on the vehicle monocular image pickup device output information.

Fig. 2 is a flowchart of a lane departure warning method according to a preferred embodiment of the present invention, which includes:

step S201, if the vehicle speed is higher than 60kph and LL+LR is more than or equal to 3.7m, executing step S202, wherein LL is the left side shooting lateral distance and LR is the right side shooting lateral distance;

Step S202, if (LL-body width/2) is less than or equal to 20cm or (LR-body width/2) is less than or equal to 20cm, executing step S203, otherwise, not deviating the departure lane from the early warning signal;

Step S203, if abrupt change occurs in steering wheel angle, vehicle pitch angle, and/or vehicle yaw angle, step S204 is performed, otherwise step S205 is performed;

Step S204, if LL '-LL is less than or equal to 10cm, or LR' -LR is less than or equal to 10cm, executing step S205, otherwise, re-executing step S202, wherein LL 'is left predicted lateral distance, LR' is right predicted lateral distance;

In step S205, the outgoing lane deviates from the pre-warning signal.

Specifically, the embodiment relates to a sensing and control system of a vehicle, which utilizes a vehicle-mounted monocular camera to identify relevant attributes of a ground lane line, and then corrects the lane line attributes by combining the vehicle body posture output by a vehicle body posture sensor, so that lane line abrupt change can be reduced, and the interference of a ground abnormal mark on a lane departure warning function can be reduced.

The implementation of the embodiment needs to depend on sensor support, such as a monocular camera (FCM), a vehicle body posture sensor (SRS), a steering wheel angle sensor (EPS) and a wheel speed sensor (IDB). Meanwhile, the fusion control output is realized by a driving assistance controller (ADC). The lane departure warning prompt is implemented by a meter (IC). The specific implementation scheme is as follows:

1. the monocular camera arranged on the front windshield glass can recognize lane lines in front of the vehicle, and the specific output lane line attributes are as follows: the lateral distance LL, LR between the center of the camera lens and the left and right lane lines; left and right lane line types TL, TR; actual road starting points SL, SR of left and right lane lines;

5. All the information is transmitted to a driving auxiliary controller, the driving auxiliary controller calculates and processes the information, and finally, lane departure early warning and alarming prompt signals are output.

Specifically, as shown in fig. 3, the monocular camera 31 inputs lane line attribute information to the driving assistance controller 35.

When the vehicle speed is higher than 60kph and LL+LR is equal to or higher than 3.7m while the driver is traveling in the lane, the driving support controller 35 executes step S202 to perform lane departure recognition based on the information output from the vehicle monocular image pickup device. If (LL-body width/2) is 20cm or less or (LR-body width/2) is 20cm or less, the lane departure is recognized based on the vehicle monocular camera, and step S203 is performed, otherwise the lane departure warning signal is not issued.

The driving support controller receives the steering angle of the vehicle inputted from the steering angle sensor 32 of the vehicle, the wheel speed inputted from the wheel speed sensor 33, and the pitch and yaw angles of the vehicle inputted from the vehicle body posture sensor 34, and performs step S203 to determine whether or not abrupt changes occur in the steering angle, the pitch angle, and/or the yaw angle of the vehicle. Abrupt changes in steering wheel angle, vehicle pitch angle, and/or vehicle yaw angle indicate changes in vehicle body motion posture, and at this time, the recognition result of the lane line by the monocular camera is not used as the control input of lane departure warning, but step S204 is executed to perform judgment. If no abrupt change occurs in the steering wheel angle, the vehicle pitch angle, and/or the vehicle yaw angle, it indicates that the vehicle body motion posture has not changed, and step S205 is directly performed at this time, and a lane departure warning signal is sent out to the meter 36.

When executing step S204, the driving support controller predicts the vehicle body movement track according to the vehicle steering angle input by the steering angle sensor 32 and the wheel rotation speed input by the wheel speed sensor 33 by using the existing vehicle track prediction method, and obtains the vehicle predicted track coordinates. Specifically, the vehicle motion trajectory is predicted by time integration based on the rotational speed information with the lane line start point as the calculation start point, the vehicle predicted trajectory coordinates of the vehicle in the lane line are obtained, and the vehicle motion gesture is corrected based on the vehicle pitch, yaw angle, and steering wheel angle input from the vehicle body gesture sensor 34, thereby correcting the calculated vehicle predicted trajectory coordinates.

The absolute value of the difference between the abscissa of the vehicle predicted trajectory coordinate output by the driving support controller and the lateral coordinate of the left lane line is the left predicted lateral distance LL ', and the absolute value of the difference between the abscissa of the vehicle predicted trajectory coordinate output by the driving support controller and the lateral coordinate of the right lane line is the right predicted lateral distance LR'.

When LL 'and LR' calculated by the driving assistance controller satisfy abs (LL '-LL) 10cm or abs (LR' -LR) 10cm. And when (LL or LR-vehicle body width/2) is less than or equal to 20cm, the warning logic of triggering the lane departure warning function is met, and lane departure warning signals are normally sent out to the instrument 36; otherwise, the lane line attribute needs to be re-detected, the lane line attribute is updated, and a secondary judgment logic is entered.

In the monocular camera recognition process, if the vehicle body posture of the whole vehicle changes, the recognition result of the monocular camera on the lane line is not used as the control input of lane departure warning, and whether a lane departure warning signal is sent or not is judged after the lane line checking logic is needed. According to the embodiment, for asphalt repairing marks, dislocated lane lines and the like on the ground, the lane lines identified by the monocular camera are compared with track coordinates predicted by driving auxiliary control, false alarms caused by the fact that the monocular camera is suddenly identified can be effectively filtered, so that the technical problem that target attribute output of a single perception sensor cannot be verified or corrected only by controlling the perception result of the sensor is solved, and the accuracy of alarm is improved by adding lane line correction logic. Meanwhile, the pitching of the vehicle body can be caused by the bumping of the road surface, and the recognition effect of the camera can be indirectly influenced by the yaw. The lane line correction logic is introduced in the embodiment, so that the interference of the outside on the identification of the camera can be reduced.

Fig. 4 is a schematic diagram of a hardware structure of an electronic device according to the present invention, including:

at least one processor 401; and

A memory 402 communicatively coupled to at least one of the processors 401; wherein,

The memory 402 stores instructions executable by at least one of the processors to enable the at least one processor to perform a lane departure warning method as previously described.

One processor 401 is illustrated in fig. 4.

The electronic device may further include: an input device 403 and a display device 404.

The processor 401, memory 402, input device 403, and display device 404 may be connected by a bus or other means, which is illustrated as a bus connection.

The memory 402 is used as a non-volatile computer readable storage medium for storing a non-volatile software program, a non-volatile computer executable program, and modules, such as program instructions/modules corresponding to the lane departure warning method in the embodiment of the present application, for example, the method flow shown in fig. 1. The processor 401 executes various functional applications and data processing by running nonvolatile software programs, instructions, and modules stored in the memory 402, that is, implements the lane departure warning method in the above-described embodiment.

Memory 402 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the lane departure warning method, or the like. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 402 may optionally include memory remotely located with respect to the processor 401, which may be connected via a network to a device performing the lane departure warning method. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 403 may receive input user clicks and generate signal inputs related to user settings and function controls of the lane departure warning method. The display 404 may include a display device such as a display screen.

The lane departure warning method in any of the method embodiments described above is performed when the one or more modules are stored in the memory 402 and executed by the one or more processors 401.

An embodiment of the present invention provides a storage medium storing computer instructions that, when executed by a computer, perform all the steps of the lane departure warning method as described above.

The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims

1. A lane departure warning method, comprising:

2. The lane departure warning method according to claim 1, wherein the lane departure recognition based on the output information of the vehicle monocular image pickup device specifically comprises:

3. The lane departure warning method according to claim 2, wherein the identifying whether the lane departure is based on the left side photographing lateral distance and the right side photographing lateral distance specifically includes:

4. The lane departure warning method according to claim 2, wherein the verifying based on the vehicle predicted trajectory coordinates specifically includes:

Acquiring a predicted track coordinate of a vehicle;

5. The lane departure warning method according to claim 4, wherein the verifying according to the left predicted lateral distance and the left photographed lateral distance and/or the right predicted lateral distance and the right photographed lateral distance specifically comprises:

6. The lane departure warning method according to claim 1, wherein if the lane departure is recognized based on the monocular image capturing device output information, the verification is performed based on the vehicle predicted trajectory coordinates, and if the verification is passed, a lane departure warning signal is output, specifically comprising:

7. The lane departure warning method according to claim 6, wherein the determining whether the vehicle body movement posture is changed specifically comprises:

8. The lane departure warning method according to any one of claims 1 to 7, further comprising:

9. An electronic device, comprising:

At least one processor; and

A memory communicatively coupled to at least one of the processors; wherein,

The memory stores instructions executable by at least one of the processors to enable the at least one of the processors to perform the lane departure warning method according to any one of claims 1 to 8.

10. A storage medium storing computer instructions which, when executed by a computer, are adapted to carry out all the steps of the lane departure warning method according to any one of claims 1 to 8.