WO2020132965A1 - Method and apparatus for determining installation parameters of on-board imaging device, and driving control method and apparatus - Google Patents

Abstract

Embodiments of the present invention provide a method and an apparatus for determining installation parameters of an on-board imaging device, and a driving control method and apparatus. The determining method comprises: detecting N traffic lanes in a road image on the basis of a road image photographed by an on-board imaging device, wherein N is an integer greater than or equal to 2; determining vanishing points of the N traffic lanes; and accurately determining installation parameters of the imaging device on the basis of the vanishing points. During intelligent driving, the detected traffic lanes can be accurately projected into an overlooking view according to the installation parameters of the imaging device determined accurately, so that the intelligent driving can be performed on the basis of the accurately projected traffic lanes, thereby improving the safety and reliability of intelligent driving.

Description

Method and equipment for determining installation parameters and driving control of vehicle-mounted imaging device Technical field

Embodiments of the present invention relate to the field of intelligent driving technology, and in particular, to a method and device for determining installation parameters and driving control of a vehicle-mounted imaging device.

Background technique

With the development of intelligent driving, in order to improve the safety of intelligent driving during road driving, it is necessary to detect the lane lines on the road. The lane line inspection is mainly used for visual navigation systems. The lane line is detected from the road map taken by the imaging device, and the detected lane line is projected into a bird's-eye view (bird view), which is convenient for detecting the degree of deviation of the vehicle from the lane line .

Projecting the detected lane line into a bird's eye view, the current common method is to use the ground plane as the projection screen plane, and project the detected lane line into the bird's eye view based on the pitch angle between the ground plane and the imaging device and the body height under.

A common practice is to default the imaging device to horizontal, or perform a calibration or calibration at the time of shipment or installation to obtain the pitch angle and perform subsequent calculations accordingly. However, due to vibrations during vehicle operation or loosening of fixed installation components, etc., the pitch angle of the imaging device will change, resulting in the inability to accurately project the detected lane line to the bird's eye view if the initial pitch angle is continued From the perspective, which affects the subsequent calculation.

Summary of the invention

Embodiments of the present invention provide a method and equipment for determining installation parameters and driving control of a vehicle-mounted imaging device, so as to achieve accurate determination of the installation parameters of the imaging device.

In a first aspect, an embodiment of the present invention provides a method for determining installation parameters of a vehicle-mounted imaging device, including:

Based on the road map captured by the vehicle-mounted imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2;

Determine the vanishing point of the N road lines;

Based on the vanishing point, the installation parameters of the imaging device are determined.

In a second aspect, an embodiment of the present invention provides a driving control method, including:

Based on the road map captured by the vehicle-mounted imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2;

Determine the vanishing point of the N road lines;

Determine the installation parameters of the imaging device based on the vanishing point;

Determine a top view of the road line based on the installation parameters of the imaging device and the detected road line;

Based on the top view of the road line, the vehicle is controlled to drive intelligently.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

An imaging device for collecting road maps, the imaging device is installed on the vehicle;

Memory, used to store computer programs;

A processor for executing the computer program, specifically for detecting N road lines in the road map based on the road map taken by the imaging device, where N is a positive integer greater than or equal to 2; The vanishing points of the N road lines; based on the vanishing points, the installation parameters of the imaging device are determined.

According to a fourth aspect, an embodiment of the present invention provides a driving control device, including:

An imaging device for collecting road maps, the imaging device is installed on the vehicle;

Memory, used to store computer programs;

A processor for executing the computer program, specifically for detecting N road lines in the road map based on the road map taken by the imaging device, where N is a positive integer greater than or equal to 2; The vanishing points of the N road lines; based on the vanishing points, determining the installation parameters of the imaging device; based on the installation parameters of the imaging device and the detected road lines, determining the top view of the road lines; based on The top view of the road line controls the intelligent driving of the vehicle.

According to a fifth aspect, an embodiment of the present invention provides a vehicle including: a vehicle body on which the electronic device described in the third aspect is installed, or a driving control device described in the fourth aspect is installed on the vehicle body.

According to a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium that stores a computer program, where the computer program includes at least one piece of code that can be executed by a computer to control The computer executes the method for determining the installation parameters of the vehicle-mounted imaging device according to the first aspect of the embodiment of the present invention, or executes the driving control method according to the second aspect of the embodiment of the present invention.

According to a seventh aspect, an embodiment of the present invention provides a computer program which, when executed by a computer, is used to implement the method for determining the installation parameters of the vehicle-mounted imaging device according to the first aspect of the embodiment of the present invention or the second aspect The driving control method described.

The method and device for determining the installation parameters and driving control of the vehicle-mounted imaging device provided by the embodiment of the present invention detect N road lines in the road map based on the road map taken by the vehicle-mounted imaging device, where N is equal to or greater than 2. A positive integer; determine the vanishing points of the N road lines; based on the vanishing points, accurately determine the installation parameters of the imaging device. In the process of intelligent driving, the detected road line can be accurately projected into a bird's eye view according to the installation parameters of the imaging device accurately determined. When intelligent driving based on the accurately projected road line can improve the safety of intelligent driving and reliability.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, without paying any creative labor, other drawings can be obtained based on these drawings.

FIG. 1 is a road map according to an embodiment of the present invention;

FIG. 2 is a road diagram from a bird’s-eye view according to an embodiment of the present invention;

3 is a flowchart of a method for determining an installation parameter of a vehicle-mounted imaging device according to an embodiment of the present invention;

4 is a flowchart of a method for determining an installation parameter of a vehicle-mounted imaging device provided by an embodiment of the present invention;

5 is a plan view of an embodiment of the present invention;

6 is a histogram related to an embodiment of the present invention;

7 is a schematic diagram of the change trend of the histogram shown in FIG. 6;

8 is a schematic diagram of the intersection of multiple road lines involved in an embodiment of the present invention;

9 is a schematic diagram of a first position of an imaging device according to an embodiment of the present invention;

10 is a schematic diagram of the optical center and vanishing point when the imaging device is in the first position;

11 is a schematic diagram of a second position of an imaging device according to an embodiment of the present invention;

12 is a schematic diagram of the optical center and the vanishing point when the imaging device is in the second position;

13 is a schematic diagram of a third position of an imaging device according to an embodiment of the present invention;

14 is a schematic diagram of the optical center and vanishing point when the imaging device is in the third position;

15 is a schematic diagram of a fourth position of an imaging device according to an embodiment of the invention;

16 is a schematic diagram of the optical center and vanishing point when the imaging device is in the fourth position;

17 is a schematic diagram of an imaging principle of an imaging device according to an embodiment of the present invention;

18 is a flowchart of a method for determining installation parameters of a vehicle-mounted imaging device according to an embodiment of the present invention;

19 is a schematic flowchart of a driving control method provided by an embodiment of the present invention;

20 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

21 is a schematic structural diagram of a driving control device provided by an embodiment of the present invention;

22 is a schematic structural diagram of a vehicle provided by an embodiment of the present invention;

23 is a schematic structural diagram of a vehicle provided by an embodiment of the present invention.

detailed description

To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

The method of the embodiment of the present invention is applicable to the fields of computer vision, intelligent driving, etc., where the installation parameters of the imaging device need to be obtained.

Exemplarily, the method of the embodiment of the present invention may be applied to the field of intelligent driving, where intelligent driving includes automatic driving and assisted driving. As shown in Figure 1, in smart driving, the road lines on the road need to be checked, for example, the lane lines on the road are detected, and the vehicle driving is controlled according to the detected lane lines, or based on the detected lane lines Carry out lane lane departure warning to improve the safety and reliability of intelligent driving.

When performing intelligent driving based on the detected lane line, it is necessary to project the detected lane line into a bird's eye view, as shown in FIG. 2. Specifically, the ground is used as the ground plane, and the detected lane line is projected onto the ground plane according to the pitch angle between the imaging device and the ground plane, and the height of the vehicle body. According to the projection of the lane line on the ground plane, the Accurately detect the degree of deviation between the vehicle and the lane line.

However, due to errors in the initial installation of the imaging device, and/or during later use, installation parameters such as the pitch angle between the imaging device and the ground plane may be generated due to vibrations that loosen the mounting components of the fixed imaging device For some changes, at this time, the lane line cannot be accurately projected on the ground level according to the initial pitch angle of the imaging device.

In order to solve the above technical problems, the method of the embodiment of the present invention realizes the accurate determination of the installation parameters of the imaging device by dynamically predicting the current installation parameters of the imaging device in real time.

The technical solutions of the present invention will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

FIG. 3 is a flowchart of a method for determining an installation parameter of a vehicle-mounted imaging device provided by an embodiment of the present invention. As shown in FIG. 3, the method of the embodiment of the present invention may include:

S101. Detect N road lines in the road map based on the road map photographed by the vehicle-mounted imaging device.

Wherein, N is a positive integer greater than or equal to 2.

The on-vehicle imaging device of the embodiment of the present invention is an imaging device mounted on a vehicle, and the imaging device photographs a road on which the vehicle is traveling to obtain a road map. The road map refers to the image including the road captured by the imaging device when the vehicle is traveling on the road. It can be understood that the road map may not include the image acquired when the vehicle is driving in a non-road area, for example, an area with no obvious ground features, such as grass, sand, or the like.

The road map includes at least two road lines, that is, N is greater than or equal to 2.

The road line in the embodiment of the present invention is a longitudinal parallel line in the road map, for example, a lane line, a shoulder line, a fence line, or other parallel lines in the longitudinal direction. For example, the N road lines in the road map may all be lane lines, and may also include lane lines and shoulder lines, or other combinations of longitudinal parallel lines, which are not limited herein.

The execution subject of the embodiment of the present invention is an electronic device, and the electronic device may be, but not limited to, a smart phone, a computer, an in-vehicle device, an in-vehicle system, etc. The execution subject of this embodiment is specifically the processor in the above electronic device.

Optionally, the electronic device of this embodiment is electrically or communicatively connected to the imaging device. The imaging device can take a road map of the scene in front of (or around) the vehicle running. The road map may be a single-frame image or a frame image in the captured video stream.

Optionally, the electronic device and the imaging device of the embodiment of the present invention are integrated, that is, the electronic device of the present application is an imaging device. The imaging device can not only take a road map, but also process the road map to detect the road map. N road lines.

Optionally, the electronic device in this embodiment of the present invention may be a driving recorder, a vehicle-mounted device, an auxiliary driving device, an automatic driving device, or the like.

Optionally, in this embodiment, an edge detection method may be used to detect road lines in the road map.

Optionally, in this embodiment, the support vector machine method may be used to detect the road lines in the road map.

Optionally, in this embodiment, other road line detection methods may be adopted to detect the road lines in the road map from the road map.

S102. Determine the vanishing points of the N road lines.

Road lines are parallel in real-world 3D space, but based on the perspective imaging principle of common cameras, they will eventually intersect at a point in the camera's 2D image, and this intersection point is called the vanishing point of the road line (Vanishing point) .

In this way, based on the detection of the N road lines, the vanishing points of the N road lines can be determined.

S103: Determine installation parameters of the imaging device based on the vanishing point.

In an ideal state, that is, the installation position of the imaging device is horizontal, the vanishing point of the road line coincides with the optical center of the imaging device.

When the installation position of the imaging device changes, that is, when it is not in a horizontal position, the vanishing point determined above does not coincide with the optical center of the imaging device,

Based on the above reasons, the installation parameters of the imaging device at the current moment can be determined based on the determined vanishing point, and thus the installation parameters of the imaging device can be accurately determined in real time.

Optionally, the installation parameters of the imaging device may include at least one of an actual pitch angle and an actual yaw angle.

The method for determining the installation parameters of the on-vehicle imaging device provided by the embodiment of the present invention detects N road lines in the road map based on the road map taken by the on-vehicle imaging device, where N is a positive integer greater than or equal to 2; The vanishing points of the N road lines; based on the vanishing points, accurately determine the installation parameters of the imaging device. In the process of intelligent driving, the detected road line can be accurately projected into a bird's eye view according to the installation parameters of the imaging device accurately determined. When intelligent driving based on the accurately projected road line can improve the safety of intelligent driving and reliability.

FIG. 4 is a flowchart of a method for determining an installation parameter of a vehicle-mounted imaging device provided by an embodiment of the present invention. Based on the foregoing embodiment, the method of the embodiment of the present invention may include:

S201. Determine candidate regions of road lines on the road map based on the deep neural network.

The deep neural network according to the embodiment of the present invention may be FCN (Fully Convolutional Networks, full convolutional network), ResNet (Residual Network, residual network) or convolutional neural network.

Optionally, the deep neural network of this embodiment includes 7 convolutional layers, respectively: the parameters of the first convolutional layer are 145*169*16, and the parameters of the second convolutional layer are 73*85*32 , The parameter of the third convolutional layer is 37*43*64, the parameter of the fourth convolutional layer is 19*22*128, the parameter of the fifth convolutional layer is 73*85*32, and the sixth The parameters of the layer are 145*169*16, and the parameters of the seventh convolution layer are 289*337*5.

The deep neural network in the embodiment of the present invention may be trained in advance, and when the road map collected by the imaging device is input to the neural network, the deep neural network outputs the candidate area of the road line in the road map.

S202. Count the number of vertical pixels belonging to the road line in the candidate area.

S203. If the number of vertical pixels is greater than a preset value, determine that the candidate area is a road line.

During the road line detection according to the above steps, the wrong road line may be detected, or the road line will not intersect at the vanishing point when turning, and these road lines need to be filtered out, and the vertical line is selected from Road line.

Specifically, for each of the candidate regions of the plurality of road lines obtained above, the number of vertical pixel points belonging to the road line in the candidate region is counted. If the number of vertical pixels corresponding to the candidate area is greater than a preset value, the candidate area is determined to be a road line.

In one example, in order to facilitate the statistics of the pixels, the above S202 may include: projecting the candidate area onto the ground plane using the initial pitch angle of the imaging device to obtain a plane projection map of the candidate area; counting the plane The number of vertical pixels belonging to the road line in the projection image.

Specifically, the candidate area is projected onto the ground plane to obtain a planar projection image of the candidate area. For example, as shown in FIG. 5, the number of vertical pixels belonging to the road line in each candidate area in the planar projection image is counted. If the number of vertical pixels corresponding to the candidate area is greater than the preset value, the candidate area is determined to be a road line.

Optionally, the number of vertical pixels belonging to the road line in each candidate area in the plan projection shown in FIG. 5 is counted to obtain the histogram shown in FIG. 6, so that the trend of the histogram can be changed according to To determine the road line. For example, if the change trend of the histogram shown in FIG. 6 is as shown in FIG. 7, that is, the difference between the peak and the adjacent trough is greater than the preset value, it can be determined that the vehicle is driving straight on the road. Determine the candidate area corresponding to the peak as the road line. If the difference between the wave crest and the adjacent wave trough is less than the preset value, it can be determined that the vehicle is turning or turning around on the road, and the candidate regions of these road lines are discarded.

S204. Fit the road line into a straight line.

According to the above steps, after determining the road line, fit the road line to a straight line, for example, use the straight line equation: ax+by=c to fit the pixel points corresponding to the road line to a straight line, so that the obtained road lines are all straight lines To lay the foundation for the subsequent determination of the vanishing point.

S205. Determine the vanishing points of the N road lines.

In an example, if N is 2, the above S205 determining the vanishing points of the N road lines may include: taking the intersection of the two road lines as the vanishing points of the two road lines.

If N is greater than 2, if road line detection is accurate, N road lines can intersect at a vanishing point. At this time, determine the vanishing point of N road lines, which may be the intersection of N road lines as the disappearance of N road lines point.

In practical applications, if N is greater than 2, due to image acquisition errors, road line detection errors, etc., as shown in FIG. 8, the vanishing points that intersect each other in N road lines may be different.

In another example, if N is greater than 2, the above S205 determines the vanishing points of the N road lines, including: taking the point with the smallest sum of distances from the N road lines as the N roads The vanishing point of the route.

Specifically, assuming that the vanishing points of the N road lines are P=(x ₀ , y ₀ ), the vanishing point P can be obtained by finding (x ₀ , y ₀ ) corresponding to an optimal solution through the following formula (1).

In another example, the point where the sum of the distances from the N road lines is the smallest as the vanishing point of the N road lines may include: filtering out the N road lines that do not satisfy the pre- Set an error road line that requires an error, and obtain M road lines that meet the preset error requirements; use a point that has the smallest sum of distances from the M road lines as the vanishing point of the N road lines.

Optionally, based on the random sampling consistent RANSAC algorithm, filter out error road lines among the N road lines that do not meet the preset error requirements.

For example, assuming that N is 4, choose two road lines from them, and record them as road line 1 and road line 2. The intersection of road line 1 and road line 2 is recorded as vanishing point 1. Calculate vanishing point 1 and road line 3. The distance is 1 and the distance between vanishing point 1 and road line 4 is distance 2. If the sum of distance 1 and distance 2 is greater than the preset error requirement a, it is determined that at least one road line exists between road line 1 and road line 2 problem. Road line 1, road line 3, and road line 4 are divided into the first group, and road line 2, road line 3, and road line 4 are divided into the second group. In the first group, determine the distance 3 between the intersection of the road line 1 and the road line 3 from the road line 4, if the distance 3 is greater than the preset error requirement value, determine that there is a problem with the road line 1, delete the road line 1. Optionally, in the first group, the distance 4 between the intersection of the road line 1 and the road line 4 from the road line 3 can also be determined. If the distance 4 is greater than the preset error requirement, it is determined that the road line 1 has a problem and the road line is deleted 1. Similarly, for the second group, determine the distance 5 between the intersection of the road line 2 and the road line 3 from the road line 4, if the distance 5 is greater than the preset error requirement, determine that there is a problem with the road line 2 and delete the road line 2. Optionally, in the second group, the distance 6 between the intersection of the road line 2 and the road line 4 from the road line 3 can also be determined. If the distance 6 is greater than the preset error requirement, it is determined that there is a problem with the road line 2 and the road line is deleted 2.

Referring to the above example, based on the random sampling consistent RANSAC algorithm, the error road lines that do not meet the preset error requirements among N road lines are filtered out, and M road lines that meet the preset error requirements are obtained. Next, bring M road lines into the following formula (2),

Determine the point with the smallest sum of distances from M road lines, and use this point as the vanishing point of N road lines.

S206. Determine the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.

The rotation of the imaging device about the x-axis is referred to as roll deflection, the rotation of the imaging device about the y-axis is referred to as pitch deflection, and the rotation of the imaging device about the z-axis is referred to as yaw deflection.

In practical applications, the location of the imaging device usually includes the following four situations:

Case 1, as shown in Figure 9, when the image plane of the imaging device is perpendicular to the ground and perpendicular to the road line (ideally, a straight road, the body is horizontal, and the imaging device is also installed horizontally), that is, the imaging device is on the x axis , Y-axis and z-axis directions are not deflected. At this time, as shown in Figure 10, the vanishing point will be at the center point O of the image plane. At this time, the installation parameters of the imaging device are the initial installation parameters.

Case 2, as shown in Figure 11, when the image plane of the imaging device is not perpendicular to the ground, for example, when the imaging device is installed, it is downward or upward, that is, the imaging device rotates along the y-axis, and there is a pitch angle (the following is used χ represents, called pitch angle), as shown in FIG. 12, the vanishing point will move along a straight line x=c _x on the image plane, that is, the vanishing point has a vertical displacement relative to the optical center of the imaging device. Based on the vanishing point, optical center and imaging parameters, the pitch angle of the imaging device at the current moment is determined, and the pitch angle is recorded as the actual pitch angle of the imaging device.

Case 3, as shown in Figure 13, when the image plane of the imaging device is not perpendicular to the road surface, for example, the vehicle changes lanes during driving, at this time, the imaging device rotates around the z-axis and has a yaw angle (denoted by φ below, (Called yaw angle), as shown in FIG. 14, at this time, the vanishing point will move along a straight line c=c _y on the image plane, that is, the vanishing point has a horizontal displacement relative to the optical center of the imaging device. At this time, it can be based on The vanishing point, optical center and imaging parameters determine the yaw angle of the imaging device at the current moment, and the yaw angle is recorded as the actual yaw angle of the imaging device.

Case 4, as shown in Figure 15, when the imaging device rotates along the x-axis, there is a roll angle (hereinafter expressed by ψ, called roll angle), that is, the imaging device is not horizontal when installed, one high and one low, At this time, as shown in Fig. 16, the optical center does not change, and coincides with the vanishing point.

It can be seen from the above that the position of the vanishing point in the image plane is related to the pitch angle χ and the yaw angle φ. As shown in Figure 17, the vector OV that passes through the optical center O and is parallel to the road line will also intersect with the road line at the vanishing point, let V = (1,0,0), the vanishing point is marked on the image plane as (u,v).

Any point on the ray OV is denoted as P, and the coordinate of point O in space is X ₀ (x ₀ , y ₀ , z ₀ ), then the coordinates of P can be written as P=X ₀ +λV, λ is a variable, and λ can be determined P, according to the projection formula (3) of the camera aperture imaging model:

Where α _x =fm _x ,α _y =fm _y ,f is the focal length, mx and my are the number of pixels per unit distance in the x and y directions. γ is the distortion parameter between the x and y axes (for example, CCD camera, pixels are not square). μ ₀ , v ₀ is the position of the optical center. The matrix R is a rotation matrix, the matrix T is a displacement matrix, and R and T are external parameters of the camera, expressing the rotation and displacement transformation from the world coordinate system to the camera coordinate system in three-dimensional space.

Bring the coordinates of point P into the above formula (3) to get formula (4)

When P is an infinity point, that is, λ infinity, the above formula (4) can ignore X ₀ , and the approximate formula (5) of formula (4) is obtained

Bring V = (1,0,0) into formula (5) to get formula (6)

Bring the rotation matrix R into formula (6)

Get the formula (7)

Bring the vanishing point determined in the above steps into equation (7), and find the actual pitch angle, χ, and the actual yaw angle, φ.

Optionally, a low-pass filter is used to filter out the actual pitch angle and actual yaw angle obtained above, and smooth the detection result, for example, using a Butterworth filter and a Chebyshev filter (Chebyshev filter), Gaussian filter (Gaussian filter), etc. for filtering.

The method for determining the installation parameters of the vehicle-mounted imaging device provided by the embodiment of the present invention determines the candidate area of the road line on the road map based on the deep neural network, and counts the number of vertical pixels belonging to the road line in the candidate area; if If the number of vertical pixels is greater than the preset value, the candidate area is determined to be a road line; the road line is fitted into a straight line, thereby ensuring that the obtained road lines are all straight lines. At the same time, for different situations, use different methods to determine the vanishing point, to achieve accurate determination of the vanishing point. Further, by analyzing the installation situation of the imaging device, the installation parameters of the imaging device are accurately determined.

FIG. 18 is a flowchart of a method for determining an installation parameter of a vehicle-mounted imaging device provided by an embodiment of the present invention. Based on the foregoing embodiment, the method of the embodiment of the present invention includes:

S301. Determine the running state of the vehicle.

The vehicle of the embodiment of the present invention is a vehicle installed with the imaging device.

In the embodiment of the present invention, before detecting the road line based on the road map captured by the vehicle-mounted imaging device, the driving state of the vehicle needs to be determined first, where the driving state of the vehicle may include straight driving, turning, lane changing, or U-turn. When the determined driving state is straight driving, the subsequent road line detection process is executed.

In an example, the above S301 includes step A and step B;

Step A: Acquire IMU (Inertial Measurement Unit) data of the vehicle at the current sampling time.

The IMU data of the embodiment of the present invention includes vehicle acceleration and gyroscope data. If the IMU acquisition frequency is 400 Hz, the IMU data obtained at the current sampling time is 400 data.

Step B: Determine the driving state of the vehicle based on the IMU data.

Next, based on the IMU data collected above, determine the driving status of the vehicle. For example, the IMU data is the acceleration of the vehicle. When the vehicle travels along a straight line, the acceleration usually remains the same or increases. When the vehicle turns, changes lanes, or makes a U-turn, The acceleration of the vehicle will be significantly reduced, so that the driving state of the vehicle can be determined according to the acceleration of the vehicle.

Optionally, determining the driving state of the vehicle based on the IMU data in step B above may include: determining an average value and variance of the IMU data; if the average value is less than a preset average value, and the If the variance is less than the preset variance, it is determined that the vehicle travels straight.

Specifically, according to formula (8), the average value x of the IMU data is determined,

Among them, x _i is the i-th IMU data, and n is the total number of IMU data.

According to formula (9), determine the variance σ of IMU data

average value

If it is less than the preset average value x _t and the variance σ is less than the preset variance σ _t , it is determined that the vehicle travels straight. Among them, the preset average value and the preset variance are engineering experience values, which can be modified according to the actual situation.

In another example, the above S301 may include: acquiring the rotation angle of the steering wheel of the vehicle; and determining the running state of the vehicle according to the rotation angle of the steering wheel.

Specifically, when driving in a straight line, the rotation angle of the steering wheel of the vehicle is 0 degrees. If the rotation angle of the steering wheel is greater than 0 degrees, it is determined that the vehicle is turning, changing lanes, or turning around. Based on this, the driving state of the vehicle can be determined according to the rotation angle of the steering wheel.

S302. If it is determined that the driving state of the vehicle is straight driving, then based on the road map captured by the on-board imaging device, N road lines in the road map are detected.

Based on the road map captured by the on-board imaging device in S302, the detection of N road lines in the road map may refer to the steps in S202 to S205, and specifically refer to the above description, which will not be repeated here.

S303. Determine the vanishing points of the N road lines.

S304. Determine the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.

The above S303 and S304 can be performed by referring to the above steps of S205 and S206, and specifically refer to the above embodiment, which will not be repeated here.

The method for determining the installation parameters of the vehicle-mounted imaging device provided by the embodiment of the present invention first determines the driving state of the vehicle before detecting the road line, and detects the road line in the road map when the driving state of the vehicle is straight driving The subsequent process avoids the blindness of the determination of the installation parameters of the imaging device, and improves the validity and accuracy of the determination of the installation parameters of the imaging device.

19 is a schematic flowchart of a driving control method provided by an embodiment of the present invention. Based on the foregoing embodiment, an embodiment of the present invention further provides a driving control method, including:

S401. Based on the road map captured by the vehicle-mounted imaging device, detect N road lines in the road map, where N is a positive integer greater than or equal to 2.

S402. Determine the vanishing points of the N road lines.

S403. Determine installation parameters of the imaging device based on the vanishing point.

For the above steps S401 to S403, reference may be made to the steps in the above embodiments, which will not be repeated here.

S404: Determine a top view of the road line based on the installation parameters of the imaging device and the detected road line.

S405: Control the vehicle to drive intelligently based on the top view of the road line.

The execution subject of this embodiment is a driving control device. The driving control device of this embodiment and the electronic device described in the above embodiment may be the same device or separate devices.

Optionally, the driving control device of the embodiment of the present invention is communicatively connected to the imaging device to obtain a road map collected by the imaging device.

Optionally, the driving control device of the embodiment of the present invention includes an imaging device. For example, the driving control device is provided with a camera, and the camera constitutes the imaging device of the embodiment of the present invention.

Specifically, the driving control device determines the installation parameters of the imaging device and detects the road line in the road map according to the method of the foregoing embodiment. The specific process refers to the description of the foregoing embodiment, and details are not described herein again. Next, the driving control device projects the detected road line onto the ground plane according to the installation parameters of the imaging device, and obtains a top view of the road line, and then performs intelligent acceleration of the vehicle based on the top view, for example, outputting prompt information and/or Carry out intelligent driving control.

The prompt information may include a warning warning of lane departure, or a reminder of keeping a lane.

The intelligent driving in this embodiment includes assisted driving and/or automatic driving.

The above-mentioned intelligent driving control may include: braking, changing driving speed, changing driving direction, lane line keeping, changing lamp status, driving mode switching, etc., wherein driving mode switching may be switching between assisted driving and automatic driving, for example To switch from assisted driving to automatic driving.

In the driving control method provided in this embodiment, the driving control device detects N road lines in the road map based on the road map captured by the on-board imaging device, where N is a positive integer greater than or equal to 2; determining the N Vanishing points of road lines; based on the vanishing points, determining the installation parameters of the imaging device; based on the installation parameters of the imaging device and the detected road lines, determining a top view of the road lines; based on the The top view of the road line controls the intelligent driving of the vehicle, thereby improving the safety and reliability of intelligent driving.

FIG. 20 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As shown in FIG. 20, the electronic device 30 of this embodiment includes:

The imaging device 31 is used for collecting road maps, and the imaging device is installed on the vehicle.

Optionally, the imaging device 31 may be a camera or a CCD (Charge-coupled Device).

The memory 32 is used to store computer programs;

The processor 33 is used to execute the computer program, and is specifically used to:

Based on the road map captured by the imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2; the vanishing points of the N road lines are determined; based on the vanishing points To determine the installation parameters of the imaging device.

Optionally, the electronic device in this embodiment of the present invention may be a driving recorder, a vehicle-mounted device, an auxiliary driving device, an automatic driving device, or the like.

The electronic device of the embodiment of the present invention may be used to execute the technical solution of the method embodiment of the method for determining the installation parameters of the vehicle-mounted imaging device shown above. The implementation principles and technical effects are similar, and are not repeated here.

In an implementation manner, the processor 33 is specifically configured to determine a candidate area of a road line on the road map based on a deep neural network; count the number of vertical pixels belonging to the road line in the candidate area; If the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line; the road line is fitted into a straight line.

In another implementation manner, the processor 33 is specifically configured to use the initial pitch angle of the imaging device to project the candidate area onto the ground plane to obtain a planar projection image of the candidate area; The number of vertical pixels belonging to the road line in the plan projection image.

In another implementation, the processor 33 is used to determine the driving state of the vehicle before detecting N road lines in the road map based on the road map captured by the on-board imaging device. The vehicle is a vehicle installed with the imaging device;

The processor 33 is specifically configured to detect N road lines in the road map based on the road map captured by the on-board imaging device if it is determined that the driving state of the vehicle is straight travel.

In another implementation manner, the processor 33 is specifically configured to acquire the IMU data of the inertial measurement unit at the current sampling time of the vehicle; and determine the driving state of the vehicle according to the IMU data.

In another implementation manner, the processor 33 is specifically configured to determine the average value and variance of the IMU data; if the average value is less than the preset average value and the variance is less than the preset variance, determine The vehicle travels straight.

In another implementation manner, the processor 33 is specifically configured to obtain the rotation angle of the steering wheel of the vehicle; and determine the driving state of the vehicle according to the rotation angle of the steering wheel.

Optionally, the road line includes a longitudinal parallel line on the road map.

In another implementation manner, the N is 2, and the processor 33 is specifically configured to use the intersection of two road lines as the vanishing point of the two road lines.

In another implementation manner, the N is greater than 2, and the processor 33 is specifically configured to use a point with the smallest distance from the N road lines as the vanishing point of the N road lines.

In another implementation manner, the processor 33 is specifically configured to filter out error road lines that do not meet the preset error requirements among the N road lines, and obtain M road lines that meet the preset error requirements ; The point with the smallest distance from the M road lines is taken as the vanishing point of the N road lines.

In another implementation manner, the processor 33 is specifically configured to filter out the error road lines among the N road lines that do not meet the preset error requirements based on the random sampling consistent RANSAC algorithm.

Optionally, the installation parameters of the imaging device include at least one of an actual pitch angle and an actual yaw angle of the imaging device.

In another implementation manner, the processor 33 is specifically configured to determine the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.

In another implementation manner, the processor 33 is specifically configured to, if the vanishing point has a vertical displacement relative to the optical center, based on the vanishing point, the optical center, and the imaging parameter, Determine the actual pitch angle of the imaging device.

In another implementation manner, the processor 33 is specifically configured to determine, based on the vanishing point, the optical center, and the imaging parameter, if the vanishing point has a horizontal displacement relative to the optical center The actual yaw angle of the imaging device.

The electronic device of the embodiment of the present invention may be used to execute the technical solution of the method embodiment of the method for determining the installation parameters of the vehicle-mounted imaging device shown above. The implementation principles and technical effects are similar, which will not be repeated here.

FIG. 21 is a schematic structural diagram of a driving control device according to an embodiment of the present invention. As shown in FIG. 21, the driving control device 40 of this embodiment includes:

The imaging device 41 is used for collecting road maps, and the imaging device is installed on the vehicle.

The imaging device 41 may be a camera or a CCD (Charge-coupled Device).

Memory 42, used to store computer programs;

The processor 43 is used to execute the computer program, and is specifically used to:

Based on the road map captured by the imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2; the vanishing points of the N road lines are determined; based on the vanishing points Determine the installation parameters of the imaging device; determine the top view of the road line based on the installation parameters of the imaging device and the detected road line; control the intelligent driving of the vehicle based on the top view of the road line.

The driving control device according to the embodiment of the present invention may be used to execute the technical solutions of the driving control method embodiments shown above, and the implementation principles and technical effects are similar, and are not repeated here.

In an implementation manner, the processor 43 is specifically configured to determine a candidate area of a road line on the road map based on a deep neural network; count the number of vertical pixels belonging to the road line in the candidate area; If the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line; the road line is fitted into a straight line.

In another implementation manner, the processor 43 is specifically configured to use the initial pitch angle of the imaging device to project the candidate area onto the ground plane to obtain a planar projection image of the candidate area; The number of vertical pixels belonging to the road line in the plan projection image.

In another implementation manner, the processor 43 is used to determine the driving state of the vehicle before detecting N road lines in the road map based on the road map captured by the on-board imaging device. The vehicle is a vehicle installed with the imaging device;

The processor 43 is specifically configured to detect N road lines in the road map based on the road map captured by the on-board imaging device if it is determined that the driving state of the vehicle is straight travel.

In another implementation manner, the processor 43 is specifically configured to acquire the IMU data of the inertial measurement unit at the current sampling time of the vehicle; and determine the driving state of the vehicle according to the IMU data.

In another implementation manner, the processor 43 is specifically configured to determine the average value and variance of the IMU data; if the average value is less than the preset average value and the variance is less than the preset variance, determine The vehicle travels straight.

In another implementation manner, the processor 43 is specifically configured to obtain the rotation angle of the steering wheel of the vehicle; and determine the driving state of the vehicle according to the rotation angle of the steering wheel.

Optionally, the road line includes a longitudinal parallel line on the road map.

In another implementation manner, the N is 2, and the processor 43 is specifically configured to use an intersection point of two road lines as a vanishing point of the two road lines.

In another implementation manner, the N is greater than 2, and the processor 43 is specifically configured to use a point with the smallest distance from the N road lines as the vanishing point of the N road lines.

In another implementation manner, the processor 43 is specifically configured to filter out the error road lines that do not meet the preset error requirements among the N road lines, and obtain M road lines that meet the preset error requirements ; The point with the smallest distance from the M road lines is taken as the vanishing point of the N road lines.

In another implementation manner, the processor 43 is specifically configured to filter out the error road lines among the N road lines that do not meet the preset error requirement based on the random sampling consistent RANSAC algorithm.

Optionally, the installation parameters of the imaging device include at least one of an actual pitch angle and an actual yaw angle of the imaging device.

In another implementation manner, the processor 43 is specifically configured to determine the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.

In another implementation manner, the processor 43 is specifically configured to, if the vanishing point has a vertical displacement relative to the optical center, based on the vanishing point, the optical center, and the imaging parameter, Determine the actual pitch angle of the imaging device.

In another implementation manner, the processor 43 is specifically configured to determine, based on the vanishing point, the optical center, and the imaging parameter, if the vanishing point has a horizontal displacement relative to the optical center The actual yaw angle of the imaging device.

The driving control device according to the embodiment of the present invention may be used to execute the technical solutions of the driving control method embodiments shown above, and the implementation principles and technical effects are similar, and are not repeated here.

22 is a schematic structural diagram of a vehicle provided by an embodiment of the present invention. As shown in FIG. 22, the vehicle 50 of this embodiment includes: a body 51 and an electronic device 52 mounted on the body 51.

The electronic device 52 is the electronic device shown in FIG. 21, and the electronic device 52 is used to determine the installation parameters of the imaging device.

Optionally, the electronic device 52 is installed on the roof of the vehicle body 51, and the imaging device in the electronic device 52 may face the front or rear of the vehicle for collecting road maps.

Optionally, the electronic device 52 is installed on the front windshield of the vehicle body 51, or the electronic device 52 is installed on the rear windshield of the vehicle body 51.

Optionally, the electronic device 52 is installed on the front of the vehicle body 51, or the electronic device 52 is installed on the rear of the vehicle body 51.

The embodiment of the present invention does not limit the installation position of the electronic device 52 on the vehicle body 51, and is specifically determined according to actual needs. The imaging device in the electronic device 52 can collect a road map.

The vehicle according to the embodiment of the present invention may be used to execute the technical solution of the method embodiment of the method for determining the installation parameters of the on-vehicle imaging device shown above. The implementation principles and technical effects are similar, and will not be repeated here.

23 is a schematic structural diagram of a vehicle provided by an embodiment of the present invention. As shown in FIG. 23, the vehicle 60 of this embodiment includes a body 61 and a driving control device 62 mounted on the body 61.

Among them, the driving control device 62 is the driving control device shown in FIG. 22, and the driving control device 52 is used to control driving of the vehicle.

Optionally, the driving control device 62 is installed on the roof of the vehicle body 61, and the imaging device in the driving control device 62 may be directed toward the front or rear of the vehicle for collecting road maps.

Alternatively, the driving control device 62 is installed on the front windshield of the vehicle body 51, or the driving control device 62 is installed on the rear windshield of the vehicle body 61.

Optionally, the driving control device 62 is installed on the front of the vehicle body 61, or the driving control device 62 is installed on the rear of the vehicle body 61.

The embodiment of the present invention does not limit the installation position of the driving control device 62 on the vehicle body 61, and is specifically determined according to actual needs, in which the imaging device in the driving control device 62 can collect road maps.

The vehicle according to the embodiment of the present invention may be used to execute the technical solutions of the above-described driving control method embodiments. The implementation principles and technical effects are similar, and are not repeated here.

Those of ordinary skill in the art may understand that all or part of the steps to implement the above method embodiments may be completed by program instructions related hardware. The foregoing program may be stored in a computer-readable storage medium, and when the program is executed, The steps of the above method embodiments are included; and the foregoing storage media include: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store program codes Medium.

An embodiment of the present invention also provides a computer storage medium that stores program instructions, and when the program is executed, it may include some or all of the steps of the method for installing parameters of the imaging device in the foregoing embodiments, Alternatively, the program may include some or all of the steps of the driving control method in the foregoing embodiments when executed.

Those of ordinary skill in the art may understand that all or part of the steps to implement the above method embodiments may be completed by program instructions related hardware. The foregoing program may be stored in a computer-readable storage medium, and when the program is executed, The steps of the above method embodiments are included; and the foregoing storage media include: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store program codes Medium.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than limiting it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features can be equivalently replaced; and these modifications or replacements do not deviate from the essence of the corresponding technical solutions of the technical solutions of the embodiments of the present invention. range.

Claims (66)

1. A method for determining the installation parameters of a vehicle-mounted imaging device is characterized by comprising:

   Based on the road map captured by the vehicle-mounted imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2;

   Determine the vanishing point of the N road lines;

   Based on the vanishing point, the installation parameters of the imaging device are determined.
2. The method according to claim 1, wherein the detecting N road lines in the road map based on the road map captured by the vehicle-mounted imaging device includes:

   Based on the deep neural network, determine the candidate area of the road line on the road map;

   Count the number of vertical pixels belonging to the road line in the candidate area;

   If the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line;

   Fit the road line into a straight line.
3. The method according to claim 2, wherein the counting of the number of vertical pixels belonging to the road line in the candidate area includes:

   Using the initial pitch angle of the imaging device to project the candidate area onto the ground plane to obtain a plan view of the candidate area;

   Count the number of vertical pixels belonging to the road line in the plane projection image.
4. The method according to any one of claims 1 to 3, wherein before detecting N road lines in the road map based on the road map captured by the on-board imaging device, the method further includes:

   Determine the running state of a vehicle, wherein the vehicle is the vehicle on which the imaging device is installed;

   The road map captured based on the vehicle-mounted imaging device detects N road lines in the road map, including:

   If it is determined that the driving state of the vehicle is straight driving, N road lines in the road map are detected based on the road map captured by the on-board imaging device.
5. The method according to claim 4, wherein the determining the driving state of the vehicle comprises:

   Obtain the IMU data of the inertial measurement unit at the current sampling time of the vehicle;

   Based on the IMU data, the driving state of the vehicle is determined.
6. The method according to claim 5, wherein the determining the driving state of the vehicle according to the IMU data includes:

   Determine the average and variance of the IMU data;

   If the average value is less than the preset average value, and the variance is less than the preset variance, it is determined that the vehicle travels straight.
7. The method according to claim 4, wherein the determining the driving state of the vehicle comprises:

   Obtaining the rotation angle of the steering wheel of the vehicle;

   The running state of the vehicle is determined according to the rotation angle of the steering wheel.
8. The method according to any one of claims 1-7, wherein the road line includes a longitudinal parallel line on the road map.
9. The method according to any one of claims 1-8, wherein the N is 2, and the determining the vanishing point of the N road lines includes:

   The intersection point of the two road lines is taken as the vanishing point of the two road lines.
10. The method according to any one of claims 1-8, wherein the N is greater than 2, and the determining the vanishing point of the N road lines includes:

    The point with the smallest sum of distances from the N road lines is taken as the vanishing point of the N road lines.
11. The method according to claim 10, wherein the point where the sum of the distances from the N road lines is the smallest as the vanishing point of the N road lines includes:

    Filtering out error road lines of the N road lines that do not meet the preset error requirements, to obtain M road lines that meet the preset error requirements;

    The point with the smallest sum of distances from the M road lines is taken as the vanishing point of the N road lines.
12. The method according to claim 11, wherein the filtering out the error road lines of the N road lines that do not meet the preset error requirements includes:

    Based on the random sampling consistent RANSAC algorithm, filter out error road lines among the N road lines that do not meet the preset error requirements.
13. The method according to any one of claims 1-12, wherein the installation parameter of the imaging device includes at least one of an actual pitch angle and an actual yaw angle of the imaging device.
14. The method according to claim 13, wherein the determining the installation parameters of the imaging device based on the vanishing point includes:

    The installation parameters of the imaging device are determined based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.
15. The method according to claim 14, wherein the determining the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device includes:

    If the vanishing point has a vertical displacement with respect to the optical center, the actual pitch angle of the imaging device is determined based on the vanishing point, the optical center, and the imaging parameter.
16. The method according to claim 14, wherein the determining the installation parameters of the imaging device based on the vanishing point and the imaging device and the optical center includes:

    If the vanishing point has a horizontal displacement with respect to the optical center, the actual yaw angle of the imaging device is determined based on the vanishing point, the optical center, and the imaging parameters.
17. A driving control method, characterized in that it includes:

    Based on the road map captured by the vehicle-mounted imaging device, N road lines in the road map are detected, where N is a positive integer greater than or equal to 2;

    Determine the vanishing point of the N road lines;

    Determine the installation parameters of the imaging device based on the vanishing point;

    Determine a top view of the road line based on the installation parameters of the imaging device and the detected road line;

    Based on the top view of the road line, the vehicle is controlled to drive intelligently.
18. The method according to claim 16, wherein the detecting N road lines in the road map based on the road map captured by the vehicle-mounted imaging device includes:

    Based on the deep neural network, determine the candidate area of the road line on the road map;

    Count the number of vertical pixels belonging to the road line in the candidate area;

    If the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line;

    Fit the road line into a straight line.
19. The method according to claim 18, wherein the counting of the number of vertical pixels belonging to the road line in the candidate area includes:

    Using the initial pitch angle of the imaging device to project the candidate area onto the ground plane to obtain a plan view of the candidate area;

    Count the number of vertical pixels belonging to the road line in the plane projection image.
20. The method according to any one of claims 17-19, wherein, before detecting the N road lines in the road map based on the road map captured by the vehicle-mounted imaging device, the method further includes:

    Determine the running state of a vehicle, wherein the vehicle is the vehicle on which the imaging device is installed;

    The road map captured based on the vehicle-mounted imaging device detects N road lines in the road map, including:

    If it is determined that the driving state of the vehicle is straight driving, N road lines in the road map are detected based on the road map captured by the on-board imaging device.
21. The method according to claim 20, wherein the determining the driving state of the vehicle comprises:

    Obtain the IMU data of the inertial measurement unit at the current sampling time of the vehicle;

    Based on the IMU data, the driving state of the vehicle is determined.
22. The method according to claim 21, wherein the determining the driving state of the vehicle according to the IMU data includes:

    Determine the average and variance of the IMU data;

    If the average value is less than the preset average value, and the variance is less than the preset variance, it is determined that the vehicle travels straight.
23. The method according to claim 20, wherein the determining the driving state of the vehicle comprises:

    Obtaining the rotation angle of the steering wheel of the vehicle;

    The running state of the vehicle is determined according to the rotation angle of the steering wheel.
24. The method according to any one of claims 17-23, wherein the road line includes a longitudinal parallel line on the road map.
25. The method according to any one of claims 17-24, wherein the N is 2, and the determining the vanishing point of the N road lines includes:

    The intersection point of the two road lines is taken as the vanishing point of the two road lines.
26. The method according to any one of claims 17-24, wherein the N is greater than 2, and the determining the vanishing point of the N road lines includes:

    The point with the smallest sum of distances from the N road lines is taken as the vanishing point of the N road lines.
27. The method according to claim 26, wherein the point where the sum of the distances from the N road lines is the smallest as the vanishing point of the N road lines includes:

    Filtering out error road lines of the N road lines that do not meet the preset error requirements, to obtain M road lines that meet the preset error requirements;

    The point with the smallest sum of distances from the M road lines is taken as the vanishing point of the N road lines.
28. The method according to claim 27, wherein the filtering out the error road lines of the N road lines that do not meet the preset error requirements includes:

    Based on the random sampling consistent RANSAC algorithm, filter out error road lines among the N road lines that do not meet the preset error requirements.
29. The method according to any one of claims 17 to 28, wherein the installation parameter of the imaging device includes at least one of an actual pitch angle and an actual yaw angle of the imaging device.
30. The method according to claim 29, wherein the determining the installation parameters of the imaging device based on the vanishing point includes:

    The installation parameters of the imaging device are determined based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device.
31. The method according to claim 30, wherein the determining the installation parameters of the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameters of the imaging device includes:

    If the vanishing point has a vertical displacement with respect to the optical center, the actual pitch angle of the imaging device is determined based on the vanishing point, the optical center, and the imaging parameter.
32. The method according to claim 30, wherein the determining the installation parameters of the imaging device based on the vanishing point and the imaging device and the optical center includes:

    If the vanishing point has a horizontal displacement with respect to the optical center, the actual yaw angle of the imaging device is determined based on the vanishing point, the optical center, and the imaging parameters.
33. An electronic device, characterized in that it includes:

    An imaging device for collecting road maps, the imaging device is installed on the vehicle;

    Memory, used to store computer programs;

    A processor for executing the computer program, specifically for detecting N road lines in the road map based on the road map taken by the imaging device, where N is a positive integer greater than or equal to 2; The vanishing points of the N road lines; based on the vanishing points, the installation parameters of the imaging device are determined.
34. The electronic device according to claim 33, wherein the processor is specifically configured to determine a candidate area of a road line on the road map based on a deep neural network; to calculate the longitudinal direction of the road line in the candidate area The number of pixels; if the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line; the road line is fitted into a straight line.
35. The electronic device according to claim 34, wherein the processor is specifically configured to use the initial pitch angle of the imaging device to project the candidate area onto the ground plane to obtain a planar projection of the candidate area Figure; Count the number of vertical pixels belonging to the road line in the plane projection.
36. The electronic device according to any one of claims 33 to 35, wherein the processor is used to detect N road lines in the road map before detecting N road lines in the road map based on the road map captured by the on-board imaging device. : Determining the driving state of the vehicle, where the vehicle is the vehicle on which the imaging device is installed;

    The processor is specifically configured to detect N road lines in the road map based on the road map captured by the on-board imaging device if it is determined that the driving state of the vehicle is straight travel.
37. The electronic device according to claim 36, wherein the processor is specifically configured to acquire IMU data of the inertial measurement unit at the current sampling time of the vehicle; and determine the driving state of the vehicle based on the IMU data.
38. The electronic device according to claim 37, wherein the processor is specifically configured to determine an average value and a variance of the IMU data; if the average value is less than a preset average value, and the variance is less than a predetermined Set the variance to determine that the vehicle travels straight.
39. The electronic device according to claim 36, wherein the processor is specifically configured to obtain a rotation angle of a steering wheel of the vehicle; and determine a driving state of the vehicle according to the rotation angle of the steering wheel.
40. The electronic device according to any one of claims 33 to 39, wherein the road line includes a longitudinal parallel line on the road map.
41. The electronic device according to any one of claims 33-40, wherein the N is 2, and the processor is specifically configured to use an intersection of two road lines as a vanishing point of the two road lines .
42. The electronic device according to any one of claims 33-40, wherein the N is greater than 2, and the processor is specifically configured to use a point at which the sum of the distances from the N road lines is the smallest, as The vanishing point of the N road lines.
43. The electronic device according to claim 42, wherein the processor is specifically configured to filter out the error road lines that do not meet the preset error requirements among the N road lines, and obtain the preset error requirements. M road lines; the point with the smallest sum of distances from the M road lines is taken as the vanishing point of the N road lines.
44. The electronic device according to claim 43, wherein the processor is specifically configured to filter out error road lines among the N road lines that do not meet the preset error requirement based on a random sampling consistent RANSAC algorithm .
45. The electronic device according to any one of claims 33 to 44, wherein the installation parameter of the imaging device includes at least one of an actual pitch angle and an actual yaw angle of the imaging device.
46. The electronic device according to claim 45, wherein the processor is specifically configured to determine the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameter of the imaging device Installation parameters.
47. The electronic device according to claim 46, wherein the processor is specifically configured to: based on the vanishing point, the optical center, and the vanishing point, if the vanishing point has a vertical displacement relative to the optical center, The imaging parameter determines the actual pitch angle of the imaging device.
48. The electronic device according to claim 46, wherein the processor is specifically configured to, based on the vanishing point, the optical center, and all positions, if the vanishing point has a horizontal displacement relative to the optical center, The imaging parameters determine the actual yaw angle of the imaging device.
49. A driving control device is characterized by comprising:

    An imaging device for collecting road maps, the imaging device is installed on the vehicle;

    Memory, used to store computer programs;

    A processor for executing the computer program, specifically for detecting N road lines in the road map based on the road map taken by the imaging device, where N is a positive integer greater than or equal to 2; The vanishing points of the N road lines; based on the vanishing points, determining the installation parameters of the imaging device; based on the installation parameters of the imaging device and the detected road lines, determining the top view of the road lines; based on The top view of the road line controls the intelligent driving of the vehicle.
50. The driving control device according to claim 49, wherein the processor is specifically configured to determine a candidate area of a road line on the road map based on a deep neural network; The number of vertical pixels; if the number of vertical pixels is greater than a preset value, it is determined that the candidate area is a road line; the road line is fitted into a straight line.
51. The driving control device according to claim 50, wherein the processor is specifically configured to project the candidate area onto the ground plane using the initial pitch angle of the imaging device to obtain the plane of the candidate area Projection diagram; count the number of vertical pixels belonging to the road line in the plane projection diagram.
52. The driving control device according to any one of claims 49 to 51, wherein the processor is used to detect N road lines in the road map based on the road map captured by the on-board imaging device, and Yu: Determining the driving state of the vehicle, where the vehicle is the vehicle on which the imaging device is installed;

    The processor is specifically configured to detect N road lines in the road map based on the road map captured by the on-board imaging device if it is determined that the driving state of the vehicle is straight travel.
53. The driving control device according to claim 52, wherein the processor is specifically configured to acquire IMU data of the inertial measurement unit at the current sampling time of the vehicle; based on the IMU data, determine the driving state of the vehicle .
54. The driving control device according to claim 53, wherein the processor is specifically configured to determine an average value and variance of the IMU data; if the average value is less than a preset average value and the variance is less than Predetermining the variance, it is determined that the vehicle travels straight.
55. The driving control device according to claim 52, wherein the processor is specifically configured to obtain a rotation angle of a steering wheel of the vehicle; and determine a driving state of the vehicle according to the rotation angle of the steering wheel.
56. The driving control device according to any one of claims 49 to 55, wherein the road line includes a longitudinal parallel line on the road map.
57. The driving control device according to any one of claims 40-56, wherein the N is 2, and the processor is specifically configured to use an intersection of two road lines as the disappearance of the two road lines point.
58. The driving control device according to any one of claims 40-56, wherein the N is greater than 2, and the processor is specifically configured to a point that minimizes the sum of distances from the N road lines, As the vanishing point of the N road lines.
59. The driving control device according to claim 58, wherein the processor is specifically configured to filter out the error road lines that do not meet the preset error requirements among the N road lines, and obtain The required M road lines; the point with the smallest distance from the M road lines is taken as the vanishing point of the N road lines.
60. The driving control device according to claim 59, wherein the processor is specifically configured to filter out the error roads in the N road lines that do not meet the predetermined error requirement based on the random sampling consistent RANSAC algorithm route.
61. The driving control device according to any one of claims 49 to 60, wherein the installation parameter of the imaging device includes at least one of an actual pitch angle and an actual yaw angle of the imaging device.
62. The driving control device according to claim 61, wherein the processor is specifically configured to determine the imaging device based on the vanishing point, the optical center of the imaging device, and the imaging parameter of the imaging device Installation parameters.
63. The driving control device according to claim 62, wherein the processor is specifically configured to: if the vanishing point has a vertical displacement with respect to the optical center, based on the vanishing point, the optical center And the imaging parameter, determine the actual pitch angle of the imaging device.
64. The driving control device according to claim 62, wherein the processor is specifically configured to: if the vanishing point has a horizontal displacement relative to the optical center, based on the vanishing point, the optical center and The imaging parameter determines the actual yaw angle of the imaging device.
65. A vehicle, characterized by comprising: a vehicle body on which the electronic device according to any one of claims 33-48 is mounted, or a vehicle body as described in any of claims 49-64 Driving control equipment.
66. A computer storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed, the method for determining the installation parameters of the vehicle-mounted imaging device according to any one of claims 1-16 is implemented, And the driving control method according to claim 17 is realized.

Images