TWI819928B - Method for detecting skewing of vehicle and related devices - Google Patents

Abstract

The present application provides a method for detecting skewing of vehicle and related devices. The method includes: converting a first foreground image of a vehicle into an aerial view, determining an initial position of a left lane line and an initial position of a right lane line according to a deployment of non-zero pixels in the aerial view, moving a sliding window according to the initial position of the left lane line and the initial position of the right lane line, fitting non-zero pixels in all sliding windows before a current sliding window and non-zero pixels in in the current sliding window into a second curve. Based on corresponding second curves, the left lane line and the right lane line are obtained respectively. A distance between the vehicle and the left lane line and a distance between the vehicle and the right lane line are calculated, for determining whether the vehicle is skewing. The present application can effectively detect lane lines and improve the accuracy of identifying lane lines.

Description

Vehicle offset detection method and related equipment

The present invention relates to the field of artificial intelligence technology, and in particular, to a vehicle offset detection method and related equipment.

Lane line detection is an important technology in driverless or assisted driving scenarios. Lane line detection refers to the detection of traffic indicators (i.e. lane lines) on the road. Through lane line detection, it can be determined whether the vehicle is driving. offset. Current vehicle deviation detection based on lane lines determines whether the vehicle has deviated by comparing the detected lane line width with a preset threshold. Since the widths of different lane lines are inconsistent, comparison based on a single threshold will lead to inaccurate results, making it impossible to effectively detect whether the vehicle has drifted while driving.

In view of the above, it is necessary to provide a vehicle offset detection method and related equipment to solve the problem of inaccurate vehicle offset detection.

The present application provides a vehicle offset detection method. The method includes: acquiring a first foreground image captured while the vehicle is driving; performing distortion correction on the first foreground image to obtain a first corrected image; and performing distortion correction on the first foreground image. The first corrected image undergoes perspective transformation to obtain a bird's-eye view; based on the number of non-zero primitive points in each column of primitive points in the bird's-eye view, a non-zero primitive point distribution map corresponding to the bird's-eye view is generated, so The non-zero primitive point distribution map includes a first peak and a second peak, and the first peak is located to the left of the second peak; the initial position of the left lane line is determined in the bird's-eye view according to the first peak, according to The second peak determines the initial position of the right lane line in the bird's-eye view; the initial position of the left lane line and the initial position of the right lane line are used as the starting positions of the sliding window to start moving in the bird's-eye view. , a first curve is fitted according to the non-zero primitive points in all sliding windows before the current sliding window, and a second curve is fitted according to the first curve and the non-zero primitive points in the current sliding window, where, The movement of the sliding window is dynamically adjusted according to the first curve; the left lane line is obtained according to the second curve fitted with the initial position of the left lane line as the starting position of the sliding window, and the left lane line is obtained according to the right lane line. The initial position is used as the starting position of the sliding window to fit the second curve to obtain the right lane line; calculate the first distance between the vehicle and the left lane line, and calculate the distance between the vehicle and the right lane line a second distance; determine whether the vehicle deviates from the lane according to the first distance and the second distance.

In some optional implementations, determining whether the vehicle deviates from a lane based on the first distance and the second distance includes: calculating the distance based on the first distance and the second distance. The first proportion of the vehicle's deviation to the left lane line and the second proportion of the deviation to the right lane line; calculate the absolute value corresponding to the difference between the first proportion and the second proportion; if If the absolute value corresponding to the difference is less than the preset threshold, it is determined that the vehicle has not deviated from the lane; if the absolute value corresponding to the difference is greater than or equal to the preset threshold, it is determined that the vehicle has deviated from the lane.

In some optional implementations, a first ratio of the vehicle's deviation to the left lane line and a first ratio of the vehicle's deviation to the right lane line are calculated based on the first distance and the second distance. The second ratio includes: calculating the sum of the first distance and the second distance, and taking the sum of the first distance and the second distance as the third distance; calculating the sum of the first distance and the third distance. For the ratio of three distances, use the ratio of the first distance to the third distance as the first ratio; calculate the ratio of the second distance to the third distance, and use the ratio of the second distance to the third distance. The ratio of the third distance serves as the second ratio.

In some optional implementations, the method further includes: obtaining a second foreground image taken at a second moment while the vehicle is driving, the second moment being a moment next to the first moment, and the The first moment is the moment corresponding to the shooting of the first foreground image; the second foreground image is Distortion correction is performed to obtain a second corrected image; the left lane line is expanded to the first direction according to the preset expansion distance to obtain the first boundary; the right lane line is expanded to the second direction according to the preset expansion distance. Expand in the direction to obtain a second boundary; perform area division on the second corrected image according to the first boundary and the second boundary to determine the area where the lane line in the second corrected image is located.

In some optional implementations, performing distortion correction on the first foreground image to obtain a first corrected image includes: establishing an image coordinate system for the first foreground image, and obtaining the first corrected image. The first coordinate of each non-zero primitive point in a foreground image in the image coordinate system; obtain the internal parameters of the camera module that captured the first foreground image, and use the internal parameters to determine the first coordinate according to the internal parameters. Determine the second coordinate corresponding to the first coordinate, wherein the second coordinate is a distortion-free coordinate; determine the first coordinate based on the first coordinate and the center coordinate point of the first foreground image and the center coordinate point; calculate the image complexity of the first foreground image according to the grayscale value of each primitive point in the first foreground image, and calculate the image complexity according to the image The complexity determines the correction parameters of the first foreground image; determines a smoothing coefficient corresponding to the distortion distance and the correction parameter according to a preset smoothing function; and determines a smoothing coefficient corresponding to the distortion distance and the correction parameter according to the smoothing coefficient and the second The coordinates are used to smoothly correct the first coordinates to obtain the first corrected image.

In some optional implementations, the smoothing correction of the first coordinate according to the smoothing coefficient and the second coordinate to obtain the first corrected image includes: according to the smoothing coefficient Determine a first weight of the first coordinate and a second weight of the second coordinate; calculate a first product of the first weight and the first coordinate, and calculate the second weight and the second The second product of the coordinates; perform smooth correction on the first coordinate according to the sum of the first product and the second product to obtain the first corrected image.

In some optional implementations, performing perspective transformation on the first corrected image to obtain a bird's-eye view includes: using each non-zero primitive point in the first corrected image as a target point, The target point is calculated using a coordinate conversion formula to obtain an inverse perspective transformation matrix; based on the inverse perspective transformation matrix, the bird's-eye view is obtained.

In some optional implementations, the first curve and the current sliding Fitting the non-zero primitive points in the moving window into a second curve includes: obtaining the non-zero primitive points corresponding to the first curve; calculating the number of non-zero primitive points in the current sliding window; if The number of non-zero graphic element points in the current sliding window is greater than or equal to the preset threshold, and the non-zero graphic element points corresponding to the first curve are fitted to the non-zero graphic element points in the current sliding window. The element points are fitted to the second curve.

This application also provides an electronic device. The electronic device includes a processor and a storage. The processor is configured to implement the vehicle offset detection method when executing a computer program stored in the storage.

The application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the vehicle offset detection method is implemented.

The vehicle offset detection method and related equipment provided by this application use the first curve to dynamically adjust the sliding window, and simulating the non-zero element points in all sliding windows before the current sliding window with the non-zero element points in the current sliding window. Synthesizing the second curve can effectively detect the position of the lane line and effectively determine whether the vehicle has deviated during driving, thereby improving the safety of vehicle driving.

1: Electronic equipment

11:Storage

12: Processor

13: Communication bus

14: Shooting device

141:Camera module

S21~S29: Steps

A1: first position

A2: The second position

A3: The third position

Figure 1 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention.

Figure 2 is a flow chart of a vehicle deviation detection method provided by a preferred embodiment of the present invention.

Figure 3 is a schematic diagram of fitting the second curve using a sliding window.

Figure 4 is a schematic diagram of the first distance and the second distance.

Figure 5 is a schematic diagram of the area where the lane line is located at the second moment.

To facilitate understanding, some descriptions of concepts related to the embodiments of the present application are exemplarily provided for reference.

It should be noted that “at least one” in this application refers to one or more, and “multiple” means two or more than two. "And/or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can Is singular or plural. The terms "first", "second", "third", "fourth", etc. (if present) in the description, claims and drawings of this application are used to distinguish similar objects, rather than to Describe a specific order or sequence.

In order to better understand the vehicle offset detection method and related equipment provided by the embodiments of the present application, the application scenarios of the vehicle offset detection method of the present application are first described below.

Figure 1 is a schematic structural diagram of an electronic device 1 provided by an embodiment of the present application. Referring to FIG. 1 , the electronic device 1 includes, but is not limited to, a storage 11 , at least one processor 12 and a camera 14 . The storage 11, the processor 12 and the photographing device 14 can be connected through the communication bus 13 or directly. The electronic device 1 is provided on a vehicle. The electronic device 1 may be a vehicle-mounted computer. In some embodiments, the electronic device 1 may include a shooting device 14 (for example, a camera) and a camera inside the shooting device 14 The module 141 is used to shoot multiple images or videos in front of the vehicle. Figure 1 is only an exemplary illustration. In other embodiments, the electronic device 1 may not include a shooting device, but may be externally connected to the shooting device, such as , driving recorder, or one or more shooting devices inside the vehicle, thereby acquiring multiple images or videos directly from external shooting devices. For example, the electronic device 1 can communicate with a driving recorder in the vehicle and obtain corresponding images or videos.

Those skilled in the art should understand that the structure of the electronic device 1 shown in Figure 1 does not constitute a limitation of the embodiment of the present invention. The electronic device 1 may also include more or less other hardware or software than in Figure 1. Or different component configurations.

The processor 12 in the electronic device 1 can implement the vehicle offset detection method that will be described in detail below when executing a computer program. The computer program includes a vehicle offset detection program.

Figure 2 is a flow chart of a vehicle offset detection method provided by an embodiment of the present application. The vehicle offset detection method is applied in electronic equipment (such as electronic equipment 1 in Figure 1), which can improve the accuracy of vehicle offset detection and ensure the safety of vehicle driving. According to different needs, the flow chart steps The order of steps can be changed and some steps can be omitted. In this embodiment, the vehicle offset detection method includes the following steps:

S21: Obtain the first foreground image taken while the vehicle is driving.

The first foreground image is an image of the scene in front of the vehicle, and includes the lane in which the vehicle is located.

In the case where the electronic device includes a photographing device, the first foreground image can be acquired by the photographing device of the electronic device. In the case where the electronic device does not include a photographing device, the first foreground image can be obtained through a photographing device on the vehicle (such as a driving recorder). The image of the scene in front of the vehicle can be captured to obtain the first foreground image. Alternatively, a video of the scene in front of the vehicle can be captured, and the first foreground image can be obtained from the captured video.

S22, perform distortion correction on the first foreground image to obtain a first corrected image.

Due to problems such as angle, rotation, and scaling of the shooting device when shooting, the first foreground image may be distorted (ie, distorted), and the first foreground image needs to be corrected for distortion.

In one embodiment, performing distortion correction on the first foreground image to obtain the first corrected image includes: establishing an image coordinate system for the first foreground image, and obtaining each non-zero value in the first foreground image. The first coordinate of the primitive point in the image coordinate system; obtain the internal parameters of the camera module that captured the first foreground image, and determine the second coordinate corresponding to the first coordinate according to the internal parameter, where, the second The coordinates are distortion-free coordinates; based on the first coordinate and the center coordinate point of the first foreground image, determine the distortion distance between the first coordinate and the center coordinate point; based on the grayscale of each primitive point in the first foreground image value, calculate the image complexity of the first foreground image, and determine the correction parameters of the first foreground image according to the image complexity; determine the smoothing coefficient corresponding to the distortion distance and correction parameter according to the preset smoothing function; Perform smoothing correction on the first coordinate according to the smoothing coefficient and the second coordinate to obtain the first Correct the image.

In one embodiment, performing smooth correction on the first coordinate according to the smoothing coefficient and the second coordinate to obtain the first corrected image includes: determining the first coordinate according to the smoothing coefficient. a first weight of the first coordinate and a second weight of the second coordinate; calculating a first product of the first weight and the first coordinate, and calculating a first product of the second weight and the second coordinate. Two products; perform smooth correction on the first coordinate according to the sum of the first product and the second product to obtain the first corrected image.

In this implementation, since the first foreground image captured on the vehicle is a distorted image, distortion correction is performed on the first foreground image. Establish an image coordinate system of the first foreground image, and obtain the first coordinate corresponding to each non-zero element point in the first foreground image. The first coordinate corresponding to the first foreground image is a coordinate with a certain distortion. Obtain the internal parameters of the camera module that captured the first foreground image. The internal parameters of the camera module are used to determine the degree of distortion of the first coordinate. According to the internal parameters of the first coordinate, obtain the distortion-free corresponding to the first coordinate. coordinate as the second coordinate.

Calculate the grayscale value of each primitive point in the first foreground image, calculate the image complexity of the first foreground image based on the grayscale value of each primitive point, and calculate the grayscale value of each primitive point, In order to calculate the sum of all grayscale values to represent the image complexity. The higher the sum of grayscale values of the first foreground image, the richer the content of the representation image and the higher the complexity of the image. Further, the correction parameters of the first foreground image are determined based on the calculated image complexity. The image complexity can be input into a pre-established deep learning model, and the correction parameters are determined based on the output of the deep learning model.

Based on the camera module shooting mechanism, the closer to the edge of the image, the higher the degree of distortion, and the closer to the center of the image, the smaller the degree of distortion. Therefore, the center coordinate point of the first foreground image can be obtained, and the distortion distance between the first coordinate and the center coordinate point can be calculated. A smoothing coefficient is calculated according to a preset smoothing function and distance, and the smoothing coefficient is used to correct the first foreground image.

Calculate the sum of the distortion distance and correction parameters as the target value, based on the preset smoothing function Number, the positive correlation between the target value and the smoothing coefficient is obtained. That is to say, the closer the area is to the edge of the first foreground image, the higher the complexity of the image corresponding to the first foreground image, and the corresponding target value The larger the smoothing coefficient, the stronger the correction process is required. For non-edge areas of the first foreground image, the smaller the corresponding target value and smoothing coefficient are, the weaker correction processing is required. Based on the positive correlation between the target value and the smoothing coefficient, this embodiment uses the smoothing coefficient and the second coordinate to smooth the first coordinate for different areas, which effectively improves the calculation efficiency.

In order to improve the smoothness of the first corrected image, a first weight corresponding to the first coordinate and a second weight corresponding to the second coordinate are obtained, where the first weight is inversely proportional to the smoothing coefficient, and the second weight is inversely proportional to the smoothing coefficient. Proportional to the relationship, the first coordinate is smoothly corrected using a weighted method to ensure the authenticity of the image.

S23: Perform perspective transformation on the first corrected image to obtain a bird's-eye view.

In one embodiment of the present application, the first corrected image is subjected to image grayscale, gradient threshold, color threshold, saturation threshold preprocessing, etc., to remove irrelevant lane lines in the first corrected image. Information, obtain a binary image, perform perspective transformation on the binary image, and obtain a bird's-eye view. The bird's-eye view is a three-dimensional view drawn based on the perspective principle and using the high viewpoint method to look down at the ground undulations from a high point. It is more realistic than a plan view.

In one embodiment, performing perspective transformation on the first corrected image to obtain a bird's-eye view includes: using each non-zero primitive point in the first corrected image as a target point, and using a coordinate conversion formula to convert the target point Calculation is performed to obtain an inverse perspective transformation matrix; based on the inverse perspective transformation matrix, the bird's-eye view is obtained.

Utilize the approximately parallel characteristics of lanes on the same road surface, use perspective transformation to eliminate the perspective effect, use each non-zero element point in the first corrected image as a target point, use the coordinate conversion formula to calculate the target point, and obtain the inverse transformation Matrix, use the inverse transformation matrix to get a bird's eye view. The bird's-eye view eliminates the interference of the road surrounding environment and the sky, and only retains the road lane information contained in the area of interest in lane line detection, which is used to reduce the amount of complex background calculations and facilitate later lane line detection.

S24, based on the number of non-zero primitive points in each column of primitive points in the bird's-eye view, generate a non-zero primitive point distribution map corresponding to the bird's-eye view. The non-zero primitive point distribution map includes the first peak and the second peak, The first peak is to the left of the second peak.

Since the lane lines extend wirelessly when the vehicle is driving, the lane lines appear as curves with a longitudinal trend, and the lane lines cannot have a large degree of curvature in a short distance, so in a relatively short distance It is approximately a straight line, which appears as a line nearly vertical to the bottom of the image in a bird's-eye view. Considering the above characteristics, a non-zero primitive point distribution map can be established for the lower half of the bird's-eye view image, which can be the area closest to the vehicle.

A non-zero primitive point distribution map is established for the non-zero primitive points corresponding to the lower half of the bird's-eye view, and the first peak sum is obtained by accumulating the number of non-zero primitive points in each column of primitive points. second peak. Wherein, the first peak value may be the peak value corresponding to the left area of the non-zero graphic element point distribution chart, and the second peak value may be the peak value corresponding to the right area of the non-zero graphic element point distribution chart, and the first peak value is between the second peak value and the left area of the non-zero graphic element point distribution chart. On the left, according to the characteristics of the lane line, there is a certain distance between the positions of the first peak and the second peak.

S25: Determine the initial position of the left lane line in the bird's-eye view based on the first peak value, and determine the initial position of the right lane line in the bird's-eye view based on the second peak value.

In order to improve the accuracy of identifying lane lines, the first peak in the bird's-eye view is used as the initial position of the left lane line to search for the left lane line, and the second peak is used as the initial position of the right lane line to search for the right lane line. The direction of searching the lane lines may be to search up and down along the direction of the lane lines.

S26, use the initial position of the left lane line and the initial position of the right lane line as the starting positions of the sliding window and start moving in the bird's-eye view, based on the non-zero elements in all sliding windows before the current sliding window. The points are fitted into a first curve, and the second curve is fitted according to the first curve and the non-zero primitive points in the current sliding window, wherein the movement of the sliding window is dynamically adjusted according to the first curve.

This embodiment uses a sliding window to search for lane lines, and determines the size of the sliding window according to the preset size. In this embodiment, the vertical width of the sliding window can be set to Moving distance, for example, you can use a sliding window with a width of 200 pixels to move, and the distance of each movement can be 200 pixels.

In a specific embodiment, for the initial sliding window (that is, the initial sliding window is the current sliding window), the abscissa of the initial sliding window is determined based on the initial position of the left lane line and the initial position of the right lane line, and the abscissa of the initial sliding window is determined based on the preset movement distance (i.e. ordinate) and the abscissa, obtain the non-zero primitive points in the initial sliding window, obtain the coordinates of each non-zero primitive point, and fit the non-zero primitive points in the initial sliding window , fitted into the first curve corresponding to the initial sliding window, the first curve includes the first curve fitted starting from the initial position of the left lane line, and the first fitted curve starting from the initial position of the right lane line. curve.

According to the first curve corresponding to the initial sliding window and the movement distance, the abscissa coordinate of the sliding window at the second moving position is calculated, that is, the abscissa corresponding to the center of the sliding window. At this time, the sliding window corresponding to the second moving position is the current Sliding window. The movement of the current sliding window is dynamically adjusted according to the first curve. For example: the first curve is y=p(x). The abscissa x corresponding to the center of the sliding window of the current sliding window is determined based on the movement distance y. The second curve is obtained based on the calculation. Move the current sliding window corresponding to the moving position, and fit the non-zero primitive points in the initial sliding window and the non-zero primitive points in the current sliding window (ie, the sliding window corresponding to the second moving position) into a second curve.

According to the first curve and the movement distance, the abscissa coordinate of the sliding window at the current position is calculated. For example, the abscissa coordinate corresponding to the center of the sliding window is used as the abscissa coordinate of the current position of the sliding window. In one embodiment, the movement of the sliding window is dynamically adjusted according to a first curve. Specifically, the first curve is obtained by fitting the non-zero primitive points in all sliding windows before the current position of the sliding window. For The sliding window at the initial position can directly fit the non-zero primitive points in the sliding window. For the sliding window at the non-initial position, the area that the sliding window has moved is used to fit the first curve. Further, the next curve is determined based on the first curve. The position of a sliding window (for details, please refer to the detailed description of Figure 3 below), that is, the moving position of the sliding window can be determined based on the first curve, and the fitting of the first curve is based on the sliding window within the coverage area of the moving process. The number of non-zero primitive points changes dynamically. Therefore, in this embodiment, the movement of the sliding window can be dynamically adjusted according to the first curve. According to Article A curve and the non-zero primitive points covered by the sliding window at the current position can be further fitted into a second curve.

In one embodiment, fitting a second curve based on the first curve and the non-zero primitive points in the current sliding window includes: obtaining the non-zero primitive points fitted to the first curve; calculating The number of non-zero primitive points in the current sliding window; if the number of non-zero primitive points in the current sliding window is greater than or equal to the preset threshold, fit the curve corresponding to the first curve The non-zero primitive points and the non-zero primitive points in the current sliding window are fitted into the second curve.

Obtain the non-zero primitive point corresponding to the first curve fitted, which can be the first curve fitted by the sliding window corresponding to the starting position, or the first curve fitted by multiple moved sliding windows. .

When searching for the current sliding window, obtain the non-zero primitive points corresponding to the first curve previously fitted to the current sliding window, and calculate the number of non-zero primitive points in the current sliding window based on the first curve. If the current sliding window is within The number of non-zero primitive points in the current sliding window is less than the preset threshold, and the non-zero primitive points in the current sliding window will not be fitted. If the number of non-zero primitive points in the current sliding window is greater than or equal to the preset threshold, the fitting will be performed. The non-zero primitive points corresponding to the first curve previously fitted in the current sliding window and the non-zero primitive points in the current sliding window are fitted into the second curve.

The above only shows the fitting between the sliding windows corresponding to the two positions. The first curve described in this embodiment can be fitted by the initial sliding window, or can be fitted by the initial sliding window and the second curve in the above example. The sliding window corresponding to each position is fitted. In this embodiment, the current moving position of the sliding window can be obtained based on the fitted first curve and the preset moving distance to avoid omissions and inaccurate recognition.

Figure 3 is a schematic diagram of using a sliding window to fit the second curve. Figure 3 shows the sliding window corresponding to the first position A1, the sliding window corresponding to the second position A2, and the sliding window corresponding to the third position A3. Establish a coordinate system in the bird's-eye view. The X-axis and Y-axis are the coordinates established in the bird's-eye view respectively. The abscissa and ordinate of the system. In a specific embodiment, the left lane line and the right lane line are fitted in the same way. Taking the left lane line as an example, the least squares method is used to fit the non-zero values in the initial sliding window. Point the primitive and get the curve F1. The second position is calculated according to the ordinate in the sliding window corresponding to the second position (that is, the ordinate of the vertex coordinate corresponding to the sliding window is determined according to the preset movement distance as the ordinate corresponding to the sliding window) and F1 The abscissa corresponding to the center of the corresponding sliding window (i.e., the center position of the sliding window), further calculates the number of non-zero primitive points in the sliding window corresponding to the second position and the coordinates corresponding to each non-zero primitive point, using the minimum The square method fits the non-zero primitive points in the initial sliding window and the sliding window corresponding to the second position to F2. According to the ordinate of the sliding window at the third position and F2, calculate the abscissa corresponding to the center of the sliding window at the third position, and further calculate the number of non-zero primitive points in the sliding window at the third position. Each non-zero For the coordinates corresponding to the primitive points, use the least squares method to fit the non-zero primitive points in the sliding window at the first position, the sliding window at the second position, and the sliding window at the third position to F3, and so on, The second curve Fn is obtained, and n represents the position where the sliding window moves.

The method described in this embodiment makes the lane lines searched more accurate, and the position of the lane lines can be better found when turning, so as to avoid omissions that lead to inaccurate recognition.

S27, obtain the left lane line based on the second curve fitted using the initial position of the left lane line as the starting position of the sliding window, and obtain the right lane based on the second curve fitted using the initial position of the right lane line as the starting position of the sliding window. String.

Using the sliding window to search for lane lines, the first curve fitted based on the non-zero primitive points in all sliding windows before the current sliding window and the location of the center of the sliding window determined based on the first curve improves the search for lane lines accuracy. The second curve obtained by searching based on the initial position of the left lane line as the search starting point is used as the left lane line, and the second curve obtained by searching based on the initial position of the right lane line as the search starting point is used as the right lane line.

S28: Calculate the first distance between the vehicle and the left lane marking, and calculate the second distance between the vehicle and the right lane marking.

Figure 4 is a schematic diagram of the first distance and the second distance. According to the width standard of the motorway, The width of each lane in the highway can be obtained. For example, the width of each motor vehicle lane on a multi-lane highway above level three is 3.5 meters. The unit of graphics can be converted into a length unit based on the lane line fitting results. According to the midpoint of the bottom of the left and right lane lines Comparing with the midpoint position of the first foreground image, the sum of the first distance and the second distance can also be obtained by calculating the width of the vehicle, subtracting the width of the vehicle from the distance between the two lane lines. In the established image coordinate system, the position of the fitted left lane line and the position of the vehicle can be obtained, and then the first distance can be obtained. The position of the inversely synthesized right lane line and the position of the vehicle can also be obtained, and then The second distance can be obtained.

In a specific embodiment, assuming that the calculated left lane is located at 85 pixels (85 pixel represents the number of pixel points in the abscissa direction) and the right lane line is located at 245 pixels, then the width between the left lane line and the right lane line is 160pixel. The position of the pre-corrected vehicle is located in the middle of the first foreground image. For example, the leftmost position of the vehicle is located at 115 pixels, and the rightmost position of the vehicle is located at 205 pixels. That is, the vehicle width can be calculated to be 90 pixels wide. Calculate the remaining distance after excluding the width of the car, the remaining distance = 160pixel-90pixel, the first distance = 115pixel-85pixel, the second distance = 245pixel-205pixel.

S29: Determine whether the vehicle deviates from the lane based on the first distance and the second distance.

In one embodiment, determining whether the vehicle deviates from the lane based on the first distance and the second distance includes: calculating a first proportion of the vehicle's deviation to the left lane line and the right lane line based on the first distance and the second distance. The second proportion of the offset; calculate the absolute value corresponding to the difference between the first proportion and the second proportion; if the absolute value corresponding to the difference is less than the preset threshold, it is determined that the vehicle has not deviated from the lane; if the absolute value corresponding to the difference Greater than or equal to the preset threshold, it is determined that the vehicle has deviated from the lane.

In one embodiment, calculating the first ratio of the vehicle's deviation to the left lane line and the second ratio of the vehicle's deviation to the right lane line based on the first distance and the second distance include: calculating the difference between the first distance and the second distance. and, take the sum of the first distance and the second distance as the third distance; calculate the ratio of the first distance to the third distance, take the ratio of the first distance to the third distance as the first ratio; calculate the ratio of the second distance to the third distance The ratio of the distance is the ratio of the second distance to the third distance as the second ratio.

According to the first distance=115pixel-85pixel and the second distance=245pixel-205pixel, calculate The first distance is calculated to be 30pixel, and the second distance is 40pixel. According to the sum of the first distance and the second distance, the third distance is 70pixel. The first ratio can be calculated to be 30pixel/70pixel=0.43, and the second ratio is 40pixel. /70pixel=0.57.

Based on the above calculation results of the first ratio and the second ratio, the absolute value corresponding to the difference between the first ratio and the second ratio is calculated to be 0.14. If the preset threshold is equal to 0.2, 014<0.2, it can be obtained that the vehicle has no offset. The preset threshold described in this embodiment can be preset according to the actual situation, and the preset threshold can be an offset safe distance.

After this application effectively identifies the lane line, it ensures the safety of the vehicle by judging the degree of vehicle deviation, and then warns the vehicle when the vehicle deviation is large, especially to add effective information to the unmanned driving system. Driving basis.

In one embodiment, a second foreground image captured at a second moment while the vehicle is driving is obtained. The second moment is the next moment after the first moment, and the first moment is the moment corresponding to the first foreground image. Perform distortion correction on the two foreground images to obtain a second corrected image; expand the left lane line to the first direction according to the preset expansion distance to obtain the first boundary; expand the right lane line to the second direction according to the preset expansion distance Expand to obtain the second boundary; perform area division on the second corrected image based on the first boundary and the second boundary to determine the area where the lane line is located in the second corrected image.

Figure 5 is a schematic diagram of the area where the lane line is located at the second moment. The method described in the embodiment of the present application takes into account the continuity of lane changes and does not require a complete window search for each frame of picture. After the processing of a frame of picture at the first moment is completed, it can be based on the image obtained at the first moment. The left lane markings and the right lane markings predict the area where the lane markings are located at the second moment. As shown in Figure 5, using the constant distance between two lane lines, the first boundary is determined based on the left lane line obtained at the first moment, the second boundary is determined based on the right lane line obtained at the second moment, and the third boundary is obtained. In the area between the first boundary and the second boundary, other areas of the captured second foreground image are masked, and the obtained area between the first boundary and the second boundary is used as the location of the lane line at the second moment. area, improving the efficiency of detecting lane lines.

This application calculates the window center position of the current sliding window based on the first curve fitted by the non-zero primitive points in all sliding windows before the current sliding window, which improves the effectiveness of moving the sliding window, and then calculates the current For the non-zero primitive points in the sliding window, fit the non-zero primitive points before the current sliding window and the non-zero primitive points of the current sliding window to generate a second curve, and then obtain the first moment based on the second curve. For left lane markings and right lane markings, based on the characteristics of lane markings, the area where the lane markings are located at the second moment is obtained based on the lane markings obtained at the first moment, which improves the efficiency of detecting lane markings. Based on the obtained left lane markings and right lane markings, the vehicle offset distance can be calculated to ensure the safety of the vehicle during driving.

Please continue to refer to FIG. 1 . In this embodiment, the storage 11 may be an internal storage of the electronic device 1 , that is, a storage built into the electronic device 1 . In other embodiments, the storage 11 may also be an external storage of the electronic device 1 , that is, a storage external to the electronic device 1 .

In some embodiments, the storage 11 is used to store program codes and various data, and realize high-speed and automatic access to programs or data during the operation of the electronic device 1 .

The storage 11 may include random access memory, and may also include non-volatile storage, such as hard disk, memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), Secure Digital (SD) card, memory card (Flash Card), at least one disk storage element, flash memory element, or other volatile solid-state storage element.

In one embodiment, the processor 12 may be a central processing unit (CPU), or other general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), or an application specific integrated circuit (Application Processor). Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic components, discrete gate or transistor logic components, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any other conventional processor, etc.

If the program codes and various data in the storage 11 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, this application implements all or part of the processes in the above embodiment methods, such as vehicle The offset detection method can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above-mentioned functions can be realized. Steps of method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in the form of original program code, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer storage, a read-only memory (ROM, Read- Only Memory) etc.

It can be understood that the module division described above is a logical function division, and there may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application can be integrated in the same processing unit, or each module can exist physically alone, or two or more modules can be integrated in the same unit. The above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified. Modifications or equivalent substitutions may be made without departing from the spirit and scope of the technical solution of the present application.

S21~S29: Steps

Claims (10)

A vehicle offset detection method, applied to electronic equipment, wherein the method includes: acquiring a first foreground image captured while the vehicle is traveling; performing distortion correction on the first foreground image to obtain a first corrected image; Perform perspective transformation on the first corrected image to obtain a bird's-eye view; based on the number of non-zero primitive points in each column of primitive points in the bird's-eye view, generate a distribution of non-zero primitive points corresponding to the bird's-eye view Figure, the non-zero primitive point distribution map includes a first peak and a second peak, the first peak is located to the left of the second peak; the left lane line is determined in the bird's-eye view according to the first peak Initial position, determine the initial position of the right lane line in the bird's-eye view according to the second peak value; use the initial position of the left lane line and the initial position of the right lane line as the starting positions of the sliding window, starting from the Moving in the bird's-eye view, a first curve is fitted according to the non-zero primitive points in all sliding windows before the current sliding window, and a second curve is fitted according to the first curve and the non-zero primitive points in the current sliding window. The curve includes: calculating the abscissa corresponding to the center of the current sliding window based on the movement distance of the current sliding window and the first curve, and determining the current sliding window based on the movement distance and the abscissa. non-zero primitive points within the first curve, fit the non-zero primitive points corresponding to the first curve and the non-zero primitive points within the current sliding window, and obtain the second curve; wherein, the The movement is dynamically adjusted according to the first curve; the left lane line is obtained according to the second curve fitted with the initial position of the left lane line as the starting position of the sliding window, and the left lane line is obtained based on the initial position of the right lane line as the sliding window. The second curve fitted to the starting position obtains the right lane line; calculates the first distance between the vehicle and the left lane line, and calculates the second distance between the vehicle and the right lane line; Based on the first distance and the second distance, it is determined whether the vehicle deviates from the lane. The vehicle deviation detection method according to claim 1, wherein determining whether the vehicle deviates from a lane according to the first distance and the second distance includes: Calculate a first proportion of the vehicle's deviation to the left lane line and a second proportion of its deviation to the right lane line based on the first distance and the second distance; calculate the first proportion and The absolute value corresponding to the difference in the second ratio; if the absolute value corresponding to the difference is less than the preset threshold, it is determined that the vehicle does not deviate from the lane; if the absolute value corresponding to the difference is greater than or equal to the The preset threshold is used to determine that the vehicle deviates from the lane. The vehicle deviation detection method according to claim 2, wherein the first proportion of the vehicle deviation to the left lane line and the deviation to the left lane line are calculated based on the first distance and the second distance. The second proportion of the right lane line offset includes: calculating the sum of the first distance and the second distance, and taking the sum of the first distance and the second distance as the third distance; calculating the third distance. The ratio of a distance to the third distance, taking the ratio of the first distance to the third distance as the first ratio; calculating the ratio of the second distance to the third distance, taking the ratio of the first distance to the third distance as the first ratio; The ratio of the second distance to the third distance serves as the second ratio. The vehicle offset detection method according to claim 1, wherein the method further includes: obtaining a second foreground image taken at a second moment during the driving process of the vehicle, and the second moment is the image of the first moment. At the next moment, the first moment is the moment corresponding to the shooting of the first foreground image; distortion correction is performed on the second foreground image to obtain a second corrected image; and the second foreground image is captured according to the preset extension distance. The left lane line is expanded to the first direction to obtain a first boundary; the right lane line is expanded to the second direction according to the preset expansion distance to obtain a second boundary; according to the first boundary and the second The boundary is divided into areas on the second corrected image to determine the area where the lane line is located in the second corrected image. The vehicle offset detection method according to claim 1, wherein performing distortion correction on the first foreground image to obtain the first corrected image includes: establishing image coordinates for the first foreground image. system, obtain each non-zero image in the first foreground image The first coordinate of the element point in the image coordinate system; obtain the internal parameters of the camera module that captured the first foreground image, and determine the third coordinate corresponding to the first coordinate based on the internal parameters of the first coordinate. Two coordinates, wherein the second coordinate is a distortion-free coordinate; based on the first coordinate and the center coordinate point of the first foreground image, the distortion between the first coordinate and the center coordinate point is determined distance; calculate the image complexity of the first foreground image based on the grayscale value of each primitive point in the first foreground image, and determine the first foreground image based on the image complexity correction parameters; determine the smoothing coefficient corresponding to the distortion distance and the correction parameter according to the preset smoothing function; perform smoothing correction on the first coordinate according to the smoothing coefficient and the second coordinate. , to obtain the first corrected image. The vehicle offset detection method according to claim 5, wherein the smoothing correction of the first coordinate according to the smoothing coefficient and the second coordinate to obtain the first corrected image includes: Determine the first weight of the first coordinate and the second weight of the second coordinate according to the smoothing coefficient; calculate the first product of the first weight and the first coordinate, and calculate the second The second product of the weight and the second coordinate; perform smooth correction on the first coordinate according to the sum of the first product and the second product to obtain the first corrected image. The vehicle offset detection method according to claim 1, wherein said performing perspective transformation on the first corrected image to obtain a bird's-eye view includes: converting each non-zero image in the first corrected image The element point is used as the target point, and the coordinate conversion formula is used to calculate the target point to obtain the inverse perspective transformation matrix; based on the inverse perspective transformation matrix, the bird's-eye view is obtained. The vehicle offset detection method according to claim 1, wherein the method according to the first Fitting a curve and the non-zero primitive points in the current sliding window into a second curve includes: obtaining the non-zero primitive points fitted to the first curve; calculating the non-zero graphic points in the current sliding window The number of primitive points; if the number of non-zero primitive points in the current sliding window is greater than or equal to the preset threshold, fit the non-zero primitive points corresponding to the first curve and the current sliding window Non-zero primitive points in the view window are fitted to the second curve. An electronic device, wherein the electronic device includes a processor and a storage, the processor is used to execute a computer program stored in the storage to implement the vehicle offset detection method in any one of claims 1 to 8 . A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction. When the at least one instruction is executed by a processor, the vehicle offset detection as described in any one of claims 1 to 8 is implemented. method.

Images