CN111086452A - Method, device and server for compensating lane line delay - Google Patents
Method, device and server for compensating lane line delay Download PDFInfo
- Publication number
- CN111086452A CN111086452A CN201911379656.8A CN201911379656A CN111086452A CN 111086452 A CN111086452 A CN 111086452A CN 201911379656 A CN201911379656 A CN 201911379656A CN 111086452 A CN111086452 A CN 111086452A
- Authority
- CN
- China
- Prior art keywords
- lane line
- automobile
- matrix
- delay
- display system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
- B60R2300/205—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used using a head-up display
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/804—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for lane monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention relates to the technical field of automobiles, and discloses a method for compensating lane line delay, which is applied to a head-up display system, and comprises the steps of firstly receiving a lane line of a road on which an automobile runs, which is identified by a driving auxiliary system of the automobile, selecting a preset number of calibration points on the lane line, then obtaining initial coordinate values of the calibration points, converting the initial coordinate values of the calibration points into homogeneous coordinates, obtaining delay time between the driving auxiliary system and the head-up display system, obtaining driving information of the automobile, then constructing a delay matrix for compensating the lane line according to the delay time and the driving information, calculating compensated coordinate values of the preset number of calibration points according to the homogeneous coordinates and the delay matrix, and finally drawing the compensated lane line according to the compensated coordinate values of the calibration points by adopting the method provided by the embodiment of the invention, the lane line can be displayed in real time, and the hidden danger of safety accidents is reduced.
Description
Technical Field
The embodiment of the invention relates to the technical field of automobiles, in particular to a method and a device for compensating lane line delay and a server.
Background
Head-up display (HUD) aided driving is a new research field in the automobile industry in recent years, the HUD is limited in popularity at present, is more applied to airplanes or high-grade cars, and is basically absent in the ordinary car models at home, however, with the increasing heat of people on future science and technology concerns, the AR (augmented reality) technology gradually enters the field of view of the public, the automobile HUD has a new market under the combination of the AR, the AR is augmented reality, namely, a 3-dimensional model is superimposed in real scene in real time, the auxiliary explanation is added on the actual scene, so that the human eyes can see more vividly and concretely, the auxiliary explanation is commonly used in teaching, medical treatment and exhibition, the understanding or the operation of the public is convenient, the AR-HUD displays the supplementary description of the AR to the real world on the HUD, and then transmits the supplementary description to the human eye, which is the augmented reality head-up display (AR-HUD).
At present, the application of AR-HUD on the automobile can help the driver to be absorbed in the current road condition and the scene ahead better, and provides guidance for the driver at the same time, so that the probability of traffic accidents is reduced, and the driver has better experience on driving.
In implementing the embodiments of the present invention, the inventors found that at least the following problems exist in the above related art: the method comprises the steps that an auxiliary driving system in an automobile acquires and calculates images and information of a lane line, the auxiliary driving system acquires a current lane line and then sends data information to a head-up display system, so that the head-up display system can display the lane line, and due to the fact that certain delay time usually exists in the processes of data calculation and transmission, the lane line information displayed by the head-up display system is not real-time when the head-up display system displays the lane line, accuracy of the information is poor, and accidents are prone to happening.
Disclosure of Invention
In view of the foregoing defects in the prior art, an object of the embodiments of the present invention is to provide a method, an apparatus, and a server for compensating lane line delay, which can improve the real-time performance of lane line display.
The purpose of the embodiment of the invention is realized by the following technical scheme:
in order to solve the foregoing technical problem, in a first aspect, an embodiment of the present invention provides a method for compensating lane line delay, which is applied to a head-up display system, where the head-up display system is disposed in an automobile, and the automobile further includes a driving assistance system, where the method includes:
receiving a lane line of a road on which the automobile is traveling, the lane line being identified by a driving assistance system of the automobile;
selecting a preset number of calibration points on the lane line;
acquiring an initial coordinate value of the calibration point, and converting the initial coordinate value of the calibration point into a homogeneous coordinate;
acquiring delay time between the driving assistance system and the head-up display system, and acquiring driving information of the automobile;
constructing a delay matrix for compensating the lane line according to the delay time and the driving information;
calculating compensated coordinate values of the preset number of calibration points after compensation according to the homogeneous coordinate and the delay matrix;
and drawing the compensated lane line according to the compensation coordinate value of the calibration point.
In some embodiments, the step of constructing a delay matrix for compensating the lane line according to the delay time and the driving information further includes:
acquiring incremental time for acquiring the image frames by the driving assistance system;
calculating a change matrix according to the increment time and the driving information of the automobile;
inverting the change matrixes to obtain inverted change matrixes;
calculating a multiple of the delay time divided by the increment time and rounded;
and sequentially multiplying the multiple inverse change matrixes to obtain the delay matrix.
In some embodiments, the driving information of the automobile includes a moving speed and an angular speed of a steering wheel of the automobile;
the step of calculating a change matrix according to the incremental time and the driving information of the automobile further comprises:
calculating a translation matrix according to the moving speed and the increment time;
calculating a rotation matrix according to the angular speed and the increment time;
obtaining the scaling matrix;
and multiplying the translation matrix, the rotation matrix and the scaling matrix to obtain the change matrix.
In some embodiments, the calculation formula for calculating the translation matrix is as follows:
Tx＝Vx*deltaTime
Ty＝Vy*deltaTime
Tz＝Vz*deltaTime
wherein, M is1For the translation matrix, (Vx, Vy, Vz) are the components of the moving speed of the car on the X-axis, Y-axis and Z-axis, respectively, and the deltaTime is the incremental time.
In some embodiments, the calculation formula for calculating the rotation matrix is as follows:
θ＝Vyawrate*deltaTime
wherein, M is2For the rotation matrix, the VyawrateThe deltaTime is the incremental time for the angular velocity of the steering wheel of the automobile.
In some embodiments, the step of obtaining the delay time between the driving assistance system and the heads-up display system further comprises:
acquiring a first time stamp of a first data packet sent by the head-up display system to the driving assistance system and a second time stamp of the first data packet received by the driving assistance system;
acquiring a third timestamp of a second data packet sent by the driving assistance system to the head-up display system and a fourth timestamp of the second data packet received by the head-up display system;
and calculating the delay time of the head-up display system and the driving assistance system according to the first time stamp, the second time stamp, the third time stamp and the fourth time stamp.
In some embodiments, the calculation formula for calculating the clock skew and the delay time of the heads-up display system and the driving assistance system is as follows:
T2＝T1+ΔT+D
T4＝T3-ΔT+D
where Δ T is the clock skew, D is the delay time, T1Is the first time stamp, T2Is the second time stamp, T3Is the third timestamp, T4Is the fourth timestamp.
In order to solve the above technical problem, in a second aspect, an embodiment of the present invention provides a device for compensating lane line delay, which is applied to a head-up display system, where the head-up display system is disposed in an automobile, the automobile further includes a driving assistance system, and the device includes:
the receiving module is used for receiving the lane line of the road where the automobile runs, which is identified by the driving assistance system of the automobile;
the selecting module is used for selecting a preset number of calibration points on the lane line;
the conversion module is used for acquiring the initial coordinate value of the calibration point and converting the initial coordinate value of the calibration point into a homogeneous coordinate;
the acquisition module is used for acquiring delay time between the driving assistance system and the head-up display system and acquiring driving information of the automobile;
the compensation module is used for constructing a delay matrix for compensating the lane line according to the delay time and the driving information;
the calculation module is used for calculating the compensated coordinate values of the calibration points with the preset number after compensation according to the homogeneous coordinate and the delay matrix;
and the drawing module is used for drawing the compensated lane line according to the compensation coordinate value of the calibration point.
In order to solve the above technical problem, in a third aspect, an embodiment of the present invention provides a head-up display system, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect as described above.
In order to solve the above technical problem, in a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method according to the first aspect.
In order to solve the above technical problem, in a fifth aspect, the present invention further provides a computer program product, which includes a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute the method according to the first aspect.
Compared with the prior art, the invention has the beneficial effects that: different from the prior art, the embodiment of the present invention provides a method for compensating lane line delay, which is applied to a head-up display system, and includes receiving a lane line of a road on which an automobile travels, the lane line being identified by a driving assistance system of the automobile, selecting a preset number of calibration points on the lane line, then obtaining initial coordinate values of the calibration points, converting the initial coordinate values of the calibration points into homogeneous coordinates, obtaining delay time between the driving assistance system and the head-up display system, and obtaining travel information of the automobile, then constructing a delay matrix for compensating the lane line according to the delay time and the travel information, so as to calculate compensated coordinate values of the preset number of calibration points according to the homogeneous coordinates and the delay matrix, and finally, according to the compensated coordinate values of the calibration points, the lane line after compensation is drawn, and by adopting the method provided by the embodiment of the invention, the lane line can be displayed in real time, and the hidden danger of safety accidents is reduced.
Drawings
One or more embodiments are illustrated by the accompanying figures in the drawings that correspond thereto and are not to be construed as limiting the embodiments, wherein elements/modules and steps having the same reference numerals are represented by like elements/modules and steps, unless otherwise specified, and the drawings are not to scale.
Fig. 1 is a schematic diagram of an application environment of a method for compensating a lane line delay according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for compensating for lane line delay according to an embodiment of the present invention;
FIG. 3 is a sub-flow diagram of step 150 of the method of FIG. 2;
FIG. 4 is a sub-flowchart of step 152 of the method of FIG. 3;
FIG. 5 is a sub-flowchart of step 140 of the method of FIG. 2;
fig. 6 is a schematic structural diagram of an apparatus for compensating lane line delay according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a hardware structure of a head-up display system according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It should be noted that, if not conflicted, the various features of the embodiments of the invention may be combined with each other within the scope of protection of the present application. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. Further, the terms "first," "second," "third," and the like, as used herein, do not limit the data and the execution order, but merely distinguish the same items or similar items having substantially the same functions and actions.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Fig. 1 is a schematic diagram of an application environment of a method for compensating a lane line delay according to an embodiment of the present invention, where the application environment includes: the head-up display system 10 and the driving assistance system 20 are both disposed in the automobile, and the head-up display system 10 and the driving assistance system 20 are in communication connection. The communication connection may be a wired or wireless connection, and when the communication connection is a wireless connection, the heads-up display system 10 and the driving assistance system 20 may be connected through a certain communication protocol, for example, wireless communication is established through bluetooth, so as to achieve data interaction between the heads-up display system 10 and the driving assistance system 20.
The Head-Up display system 10, also called a parallel display system (HUD, Head Up Di sp ay), is used to project important driving information such as moving speed and navigation of a vehicle onto a windshield in front of a driver, and can enable the driver to obtain driving information of the vehicle while keeping direct view of the front, thereby improving the driving safety of the driver. In the embodiment of the present invention, the head-up display system 10 is at least capable of displaying a lane line for navigation, and is connected to the driving assistance system 20 in a communication manner.
The driving assistance system 20 is a system for outputting driving information such as driving speed, navigation information, oil amount information and the like in an automobile, and includes but is not limited to a lane keeping assistance system, an automatic parking assistance system, a brake assistance system, a reverse assistance system, a driving assistance system and the like.
Specifically, the embodiments of the present invention will be further explained below with reference to the drawings.
An embodiment of the present invention provides a method for compensating lane line delay, which may be performed by the above head-up display system 10, where the head-up display system is disposed on the vehicle, and the vehicle further includes a driving assistance system, as shown in fig. 2, which illustrates the method for compensating lane line delay according to the embodiment of the present invention, and the method includes, but is not limited to, the following steps:
step 110: receiving a lane line of a road on which the automobile is traveling, the lane line being identified by a driving assistance system of the automobile.
In the embodiment of the present invention, first, a lane line of a road where an automobile runs is obtained by the driving assistance system, specifically, after an image of the road where the automobile runs is acquired by the driving assistance system through a camera, a feature recognition algorithm is used to recognize the lane line in the image, for example, first, a photographed image is converted into a gray image, then, the image is filtered to filter noise in the image, then, edge detection is realized through binarization processing, and the lane line is extracted according to a feature of the lane line, such as a straight line or a curve that penetrates through at least one edge of the image.
Step 120: and selecting a preset number of calibration points on the lane line.
After the lane line is extracted, the driving assistance system can further convert the lane line into a lane line cubic equation, so that when a preset number of calibration points on the lane line are selected, the specific coordinate information of the calibration points can be acquired. Preferably, the calibration point should include a start point and an end point of the lane line in the image, that is, end points at both ends of the lane line, wherein the start point and the end point can be distinguished by a driving direction of the vehicle. When a preset number of calibration points are selected, preferably, the calibration points are selected at equal intervals, and the preset number can be set according to actual needs.
Step 130: and acquiring an initial coordinate value of the calibration point, and converting the initial coordinate value of the calibration point into a homogeneous coordinate.
After the calibration point is selected, usually, according to the position of the calibration point in the image acquired by the automobile and the distance between the calibration point and the automobile acquired by the sensor, the initial coordinate value of the calibration point is a three-dimensional coordinate value under a rectangular coordinate system, in order to further calculate with the following delay matrix, the initial coordinate of the calibration point needs to be converted into a homogeneous coordinate, a dimension is added, and a four-dimensional coordinate value under a homogeneous coordinate system is obtained, wherein the fourth dimension can be 1.
Step 140: acquiring delay time between the driving assistance system and the head-up display system, and acquiring driving information of the automobile.
Furthermore, each system needs a certain time to process data, and when various data are transmitted between systems, especially when the amount of information is large, the system response is slow, which causes delay. When data is transmitted between the driving assistance system and the head-up display system, delay of a lane line is usually easily caused due to the above reasons, so that the system needs to predict the change situation of the lane line within the delay time, and how the lane line changes depends on the relative position of the automobile and the lane line, and the change situation of the relative position of the automobile and the lane line is related to the current driving information of the automobile, so that the delay time between the driving assistance system and the head-up display system needs to be acquired, and the driving information of the automobile needs to be acquired. The driving information includes, but is not limited to: the moving speed of the automobile, the moving direction of the automobile (depending on the angular velocity of the steering wheel when the driver operates the steering wheel), and the like, it should be noted that the parameters such as the moving speed, the angular velocity, and the like are vector parameters in the embodiment of the present invention, and carry direction information.
Step 150: and constructing a delay matrix for compensating the lane line according to the delay time and the driving information.
After the delay time and the travel information are acquired, a delay matrix for compensating the lane line may be constructed according to the delay time and the travel information, and specifically, a change condition of the lane line within the delay time is determined according to the travel information to construct a delay matrix to compensate the position information of each of the calibration points.
Step 160: and calculating the compensated coordinate values of the preset number of calibration points after compensation according to the homogeneous coordinate and the delay matrix.
Generally, the parameters that influence lane line changes include at least: the time, the moving speed of the automobile, the moving direction of the automobile and the scaling before and after the image acquisition are carried out, therefore, the delay matrix is at least a 4X4 matrix, the homogeneous coordinate value of each calibrated point after compensation can be obtained through calculation by combining the homogeneous coordinate of the calibrated point which also comprises four-dimensional information, and the homogeneous coordinate value is converted into a three-dimensional coordinate value, so that the compensated coordinate value of the preset number of calibrated points can be obtained.
Step 170: and drawing the compensated lane line according to the compensation coordinate value of the calibration point.
And after the compensation coordinate values of the calibration points are obtained, drawing a straight line or a curve of each compensated calibration point, namely the compensated lane line. Preferably, when the lane line is drawn, a cubic equation of the compensated lane line may be calculated according to the compensation coordinate values of the calibration points, and then the lane line is drawn according to the cubic equation, which is also convenient for transmitting lane line information.
The embodiment of the invention provides a method for compensating lane line delay, which is applied to a head-up display system, and comprises the steps of firstly, receiving a lane line of a road on which an automobile runs, which is identified by a driving auxiliary system of the automobile, selecting a preset number of calibration points on the lane line, then obtaining initial coordinate values of the calibration points, converting the initial coordinate values of the calibration points into homogeneous coordinates, obtaining delay time between the driving auxiliary system and the head-up display system, obtaining running information of the automobile, then constructing a delay matrix for compensating the lane line according to the delay time and the running information, calculating compensated coordinate values of the preset number of calibration points according to the homogeneous coordinates and the delay matrix, and finally drawing the compensated lane line according to the compensated coordinate values of the calibration points, by adopting the method provided by the embodiment of the invention, the lane line can be displayed in real time, and the hidden danger of safety accidents is reduced.
In some embodiments, please refer to fig. 3, which illustrates a sub-flowchart of step 150 of the method of fig. 2, wherein step 150 includes, but is not limited to, the following steps:
step 151: acquiring incremental time for acquiring the image frames by the driving assistance system;
step 152: calculating a change matrix according to the increment time and the driving information of the automobile;
step 153: inverting the change matrixes to obtain inverted change matrixes;
step 154: calculating a multiple of the delay time divided by the increment time and rounded;
step 155: and sequentially multiplying the multiple inverse change matrixes to obtain the delay matrix.
In the embodiment of the present invention, when a delay matrix for compensating a lane line is constructed, specifically, when an automobile moves, especially when the automobile moves at a fast speed, a situation that a lane line changes greatly may occur between delta times required for passing through each frame of image acquired by the automobile, for example, when the automobile passes through a road with many curves at a fast speed, and therefore, a parameter affecting the lane line change on each image frame is not a fixed value, for example, a moving direction may be changed continuously, and therefore, a change matrix constructed according to the parameter affecting the lane line change on each frame of image needs to be further acquired, wherein a change matrix is acquired every other image frame, that is, the delta time, and the change matrices within the delay time are sequentially multiplied to obtain the delay matrix. The incremental time is an image acquisition device provided for the automobile, for example, the time interval between every two adjacent frames of images acquired by a camera, and the automobile can acquire a plurality of image frames in the delay time. The increment time is determined by the sampling frequency of an image acquisition device arranged on the automobile, and can also be set by a user or a system.
Specifically, in some embodiments, the driving information of the automobile includes a moving speed and an angular velocity of a steering wheel of the automobile, and referring to fig. 4, a sub-flowchart of step 152 in the method shown in fig. 3 is shown, where the step 152 includes, but is not limited to, the following steps:
step 1521: calculating a translation matrix according to the moving speed and the increment time;
the calculation formula for calculating the translation matrix is as follows:
Tx＝Vx*deltaTime
Ty＝Vy*deltaTime
Tz＝Vz*deltaTime
wherein, M is1For the translation matrix, (Vx, Vy, Vz) are the components of the moving speed of the car on the X-axis, Y-axis and Z-axis, respectively, and the deltaTime is the incremental time.
In the embodiment of the present invention, generally, since the image capturing device disposed on the automobile does not change the angle, the vehicle is always in a fixed position on the image including the lane line on the captured image, and the moving distance of the vehicle in the horizontal and vertical coordinate directions of the image does not change when the vehicle does not change the lane, therefore, preferably, the Tx and Ty are both 0, and only the moving speed of the vehicle in the plane direction of the lane line, that is, the current speed of the automobile, needs to be obtained, and the translation matrix is constructed as described above.
Step 1522: calculating a rotation matrix according to the angular speed and the increment time;
the calculation formula for calculating the rotation matrix is as follows:
θ＝Vyawrate*deltaTime
wherein, M is2For the rotation matrix, the VyawrateThe deltaTime is the incremental time for the angular velocity of the steering wheel of the automobile.
In the embodiment of the present invention, since a vehicle may turn, and the like, at this time, the moving direction of the vehicle changes, the size of the change in the moving direction of the vehicle, that is, the angle at which the vehicle rotates left and right, may be calculated by detecting the angular velocity of the steering wheel, and the rotation matrix as described above may be constructed.
Step 1523: obtaining a scaling matrix;
in the embodiment of the present invention, in order to facilitate data processing, the translation matrix and the rotation matrix are calculated after scaling processing, so that a scaling matrix needs to be obtained to restore the position proportion of the coordinate point on the image. Preferably, the present application does not perform scaling processing on the translation matrix and the rotation matrix, and the scaling matrix is taken
Wherein, M is3Is the scaling matrix.
Step 1524: and multiplying the translation matrix, the rotation matrix and the scaling matrix to obtain the change matrix.
After the translation matrix, the rotation matrix and the scaling matrix are obtained, matching the three matrices to obtain the change matrix, wherein a calculation formula for calculating the change matrix is as follows:
M＝M1*M2*M3
wherein M is the variation matrix, M1For the translation matrix, the M2For the rotation matrix, the M3Is the scaling matrix.
In some embodiments, please refer to fig. 5, which illustrates a sub-flowchart of step 140 of the method of fig. 2, wherein step 140 includes, but is not limited to, the following steps:
step 141: acquiring a first time stamp of a first data packet sent by the head-up display system to the driving assistance system and a second time stamp of the first data packet received by the driving assistance system;
step 142: acquiring a third timestamp of a second data packet sent by the driving assistance system to the head-up display system and a fourth timestamp of the second data packet received by the head-up display system;
step 143: and calculating the delay time of the head-up display system and the driving assistance system according to the first time stamp, the second time stamp, the third time stamp and the fourth time stamp.
In the embodiment of the present invention, the time required for the data packet to travel from the head-up display system to the driving assistance system and the time required for the data packet to travel from the driving assistance system to the head-up display system may be obtained by mutually transmitting the data packet between the head-up display system and the driving assistance system, so as to calculate the delay time that may be brought about when the data packet travels between the two systems. In addition, when the head-up display system and the driving assistance system are not connected to the network, there may be a case where local clocks are not synchronized, and therefore, it is also necessary to obtain clock offsets of the head-up display system and the driving assistance system.
The first data packet and the second data packet may be the same data packet or different data packets, and the transmission times of the first data packet and the second data packet may be the same or different, and may be specifically set according to actual needs. Preferably, the first data packet and/or the second data packet are set to be transmitted every 2 seconds to improve accuracy.
Specifically, the calculation formula for calculating the clock skew and the delay time of the head-up display system and the driving assistance system is as follows:
T2＝T1+ΔT+D
T4＝T3-ΔT+D
where Δ T is the clock skew, D is the delay time, T1Is the first time stamp, T2Is the second time stamp, T3Is the third timestamp, T4Is the fourth timestamp.
An embodiment of the present invention further provides a device for compensating lane line delay, which is applied to a head-up display system, where the head-up display system is disposed on an automobile, and the automobile further includes a driving assistance system, please refer to fig. 6, which shows a structural diagram of the device for compensating lane line delay provided by the embodiment of the present invention, where the device 200 for compensating lane line delay includes: a receiving module 210, a selecting module 220, a converting module 230, an obtaining module 240, a compensating module 250, a calculating module 260, and a rendering module 270.
The receiving module 210 is configured to receive a lane line of a road on which the automobile runs, which is identified by a driving assistance system of the automobile;
the selecting module 220 is configured to select a preset number of calibration points on the lane line;
the conversion module 230 is configured to obtain an initial coordinate value of the calibration point, and convert the initial coordinate value of the calibration point into a homogeneous coordinate;
the obtaining module 240 is configured to obtain a delay time between the driving assistance system and the head-up display system, and obtain driving information of the vehicle;
the compensation module 250 is configured to construct a delay matrix for compensating the lane line according to the delay time and the driving information;
the calculating module 260 is configured to calculate compensated coordinate values of the preset number of calibration points after compensation according to the homogeneous coordinate and the delay matrix;
the drawing module 270 is configured to draw the compensated lane line according to the compensation coordinate value of the calibration point.
In some embodiments, the compensation module 250 is further configured to obtain an incremental time for the driving assistance system to acquire the image frames;
calculating a change matrix according to the increment time and the driving information of the automobile;
inverting the change matrixes to obtain inverted change matrixes;
calculating a multiple of the delay time divided by the increment time and rounded;
and sequentially multiplying the multiple inverse change matrixes to obtain the delay matrix.
In some embodiments, the driving information of the automobile comprises a moving speed and an angular velocity of a steering wheel of the automobile, and the compensation module 250 is further configured to calculate a translation matrix according to the moving speed and the incremental time;
calculating a rotation matrix according to the angular speed and the increment time;
obtaining a scaling matrix;
and multiplying the translation matrix, the rotation matrix and the scaling matrix to obtain the change matrix.
In some embodiments, the calculation formula for calculating the translation matrix is as follows:
Tx＝Vx*deltaTime
Ty＝Vy*deltaTime
Tz＝Vz*deltaTime
wherein, M is1For the translation matrix, (Vx, Vy, Vz) are the components of the moving speed of the car on the X-axis, Y-axis and Z-axis, respectively, and the deltaTime is the incremental time.
In some embodiments, the calculation formula for calculating the rotation matrix is as follows:
θ＝Vyawrate*deltaTime
wherein, M is2For the rotation matrix, the VyawrateThe deltaTime is the incremental time for the angular velocity of the steering wheel of the automobile.
In some embodiments, the obtaining module 240 is further configured to obtain a first timestamp of the first data packet sent by the heads-up display system to the driving assistance system, and a second timestamp of the first data packet received by the driving assistance system;
acquiring a third timestamp of a second data packet sent by the driving assistance system to the head-up display system and a fourth timestamp of the second data packet received by the head-up display system;
and calculating the delay time of the head-up display system and the driving assistance system according to the first time stamp, the second time stamp, the third time stamp and the fourth time stamp.
In some embodiments, the calculation formula for calculating the clock skew and the delay time of the heads-up display system and the driving assistance system is as follows:
T2＝T1+ΔT+D
T4＝T3-ΔT+D
where Δ T is the clock skew, D is the delay time, T1Is the first time stamp, T2Is the second time stamp, T3Is the third timestamp, T4Is the fourth timestamp.
An embodiment of the present invention further provides a head-up display system, please refer to fig. 7, which shows a hardware structure of the head-up display system capable of performing the method for compensating the lane line delay described in fig. 2 to 5. The heads-up display system 10 may be the heads-up display system 10 shown in fig. 1.
The head-up display system 10 includes: at least one processor 11; and a memory 12 communicatively coupled to the at least one processor 11, which is exemplified by one processor 11 in fig. 7. The memory 12 stores instructions executable by the at least one processor 11, the instructions being executable by the at least one processor 11 to enable the at least one processor 11 to perform the method of compensating for lane line delay described above with reference to fig. 2 to 5. The processor 11 and the memory 12 may be connected by a bus or other means, and fig. 7 illustrates the connection by a bus as an example.
The memory 12, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method for compensating lane delay in the embodiment of the present application, for example, the respective modules shown in fig. 6. The processor 11 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 12, namely, implements the method of compensating for lane line delay of the above-described method embodiment.
The memory 12 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of a device that compensates for lane line delay, and the like. Further, the memory 12 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 12 optionally includes a memory remotely located from the processor 11, and these remote memories may be connected over a network to a device that compensates for lane line delays. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 12 and when executed by the one or more processors 11, perform the method of compensating for lane line delay in any of the above-described method embodiments, e.g., perform the method steps of fig. 2-5 described above, implementing the functions of the modules and units in fig. 6.
The product can execute the method provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the methods provided in the embodiments of the present application.
Embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer-executable instructions for execution by one or more processors, for example, to perform the method steps of fig. 2-5 described above to implement the functions of the modules in fig. 6.
Embodiments of the present application further provide a computer program product comprising a computer program stored on a non-volatile computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the method of compensating for lane line delay in any of the above-described method embodiments, e.g., to perform the method steps of fig. 2 to 5 described above, to implement the functions of the respective modules in fig. 6.
The embodiment of the invention provides a method for compensating lane line delay, which is applied to a head-up display system, and comprises the steps of firstly, receiving a lane line of a road on which an automobile runs, which is identified by a driving auxiliary system of the automobile, selecting a preset number of calibration points on the lane line, then obtaining initial coordinate values of the calibration points, converting the initial coordinate values of the calibration points into homogeneous coordinates, obtaining delay time between the driving auxiliary system and the head-up display system, obtaining running information of the automobile, then constructing a delay matrix for compensating the lane line according to the delay time and the running information, calculating compensated coordinate values of the preset number of calibration points according to the homogeneous coordinates and the delay matrix, and finally drawing the compensated lane line according to the compensated coordinate values of the calibration points, by adopting the method provided by the embodiment of the invention, the lane line can be displayed in real time, and the hidden danger of safety accidents is reduced.
It should be noted that the above-described device embodiments are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for compensating lane line delay is applied to a head-up display system, the head-up display system is arranged on an automobile, the automobile further comprises a driving auxiliary system, and the method is characterized by comprising the following steps:
receiving a lane line of a road on which the automobile is traveling, the lane line being identified by a driving assistance system of the automobile;
selecting a preset number of calibration points on the lane line;
acquiring an initial coordinate value of the calibration point, and converting the initial coordinate value of the calibration point into a homogeneous coordinate;
acquiring delay time between the driving assistance system and the head-up display system, and acquiring driving information of the automobile;
constructing a delay matrix for compensating the lane line according to the delay time and the driving information;
calculating compensated coordinate values of the preset number of calibration points after compensation according to the homogeneous coordinate and the delay matrix;
and drawing the compensated lane line according to the compensation coordinate value of the calibration point.
2. The method of claim 1,
the step of constructing a delay matrix for compensating the lane line according to the delay time and the travel information further includes:
acquiring incremental time for acquiring the image frames by the driving assistance system;
calculating a change matrix according to the increment time and the driving information of the automobile;
inverting the change matrixes to obtain inverted change matrixes;
calculating a multiple of the delay time divided by the increment time and rounded;
and sequentially multiplying the multiple inverse change matrixes to obtain the delay matrix.
3. The method according to claim 2, wherein the driving information of the automobile includes a moving speed and an angular speed of a steering wheel of the automobile,
the step of calculating a change matrix according to the incremental time and the driving information of the automobile further comprises:
calculating a translation matrix according to the moving speed and the increment time;
calculating a rotation matrix according to the angular speed and the increment time;
obtaining a scaling matrix;
and multiplying the translation matrix, the rotation matrix and the scaling matrix to obtain the change matrix.
4. The method of claim 3,
the calculation formula for calculating the translation matrix is as follows:
Tx＝Vx*deltaTime
Ty＝Vy*deltaTime
Tz＝Vz*deltaTime
wherein, M is1For the translation matrix, (Vx, Vy, Vz) are the components of the moving speed of the car on the X-axis, Y-axis and Z-axis, respectively, and the deltaTime is the incremental time.
5. The method of claim 3,
the calculation formula for calculating the rotation matrix is as follows:
θ＝Vyawrate*deltaTime
wherein, M is2For the rotation matrix, the VyawrateThe deltaTime is the incremental time for the angular velocity of the steering wheel of the automobile.
6. The method of claim 2,
the step of acquiring the delay time between the driving assistance system and the heads-up display system further includes:
acquiring a first time stamp of a first data packet sent by the head-up display system to the driving assistance system and a second time stamp of the first data packet received by the driving assistance system;
acquiring a third timestamp of a second data packet sent by the driving assistance system to the head-up display system and a fourth timestamp of the second data packet received by the head-up display system;
and calculating the delay time of the head-up display system and the driving assistance system according to the first time stamp, the second time stamp, the third time stamp and the fourth time stamp.
7. The method of claim 6,
the calculation formula for calculating the clock deviation and the delay time of the head-up display system and the driving assistance system is as follows:
T2＝T1+ΔT+D
T4＝T3-ΔT+D
where Δ T is the clock skew, D is the delay time, T1Is the first time stamp, T2Is the second time stamp, T3Is the third timestamp, T4Is the fourth timestamp.
8. The utility model provides a device for compensating lane line delay, is applied to the new line display system, the new line display system set up in the car, the car still includes driving assistance system, its characterized in that, the device includes:
the receiving module is used for receiving the lane line of the road where the automobile runs, which is identified by the driving assistance system of the automobile;
the selecting module is used for selecting a preset number of calibration points on the lane line;
the conversion module is used for acquiring the initial coordinate value of the calibration point and converting the initial coordinate value of the calibration point into a homogeneous coordinate;
the acquisition module is used for acquiring delay time between the driving assistance system and the head-up display system and acquiring driving information of the automobile;
the compensation module is used for constructing a delay matrix for compensating the lane line according to the delay time and the driving information;
the calculation module is used for calculating the compensated coordinate values of the calibration points with the preset number after compensation according to the homogeneous coordinate and the delay matrix;
and the drawing module is used for drawing the compensated lane line according to the compensation coordinate value of the calibration point.
9. A heads-up display system, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A computer-readable storage medium having computer-executable instructions stored thereon for causing a computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911379656.8A CN111086452B (en) | 2019-12-27 | 2019-12-27 | Method, device and server for compensating lane line delay |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911379656.8A CN111086452B (en) | 2019-12-27 | 2019-12-27 | Method, device and server for compensating lane line delay |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111086452A true CN111086452A (en) | 2020-05-01 |
| CN111086452B CN111086452B (en) | 2021-08-06 |
Family
ID=70398366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911379656.8A Active CN111086452B (en) | 2019-12-27 | 2019-12-27 | Method, device and server for compensating lane line delay |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111086452B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115171384A (en) * | 2022-07-04 | 2022-10-11 | 南京四维智联科技有限公司 | Key vehicle position delay compensation method and device in vehicle-mounted display process |
| CN115171384B (en) * | 2022-07-04 | 2024-05-14 | 南京四维智联科技有限公司 | Method and device for compensating position delay of key vehicle in vehicle-mounted display process |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000054008A1 (en) * | 1999-03-11 | 2000-09-14 | Intelligent Technologies International, Inc. | Methods and apparatus for preventing vehicle accidents |
| DE102014217440A1 (en) * | 2013-09-04 | 2015-03-05 | Toyota Jidosha Kabushiki Kaisha | Warning display device and warning display method |
| CN105262989A (en) * | 2015-10-08 | 2016-01-20 | 成都时代星光科技有限公司 | Automatic inspection and real-time image acquisition transmission method of railway line unmanned aerial plane |
| US20160110617A1 (en) * | 2014-10-20 | 2016-04-21 | Denso Corporation | Apparatus for recognizing lane partition lines |
| CN107021015A (en) * | 2015-11-08 | 2017-08-08 | 欧特明电子股份有限公司 | System and method for image procossing |
-
2019
- 2019-12-27 CN CN201911379656.8A patent/CN111086452B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000054008A1 (en) * | 1999-03-11 | 2000-09-14 | Intelligent Technologies International, Inc. | Methods and apparatus for preventing vehicle accidents |
| DE102014217440A1 (en) * | 2013-09-04 | 2015-03-05 | Toyota Jidosha Kabushiki Kaisha | Warning display device and warning display method |
| US20160110617A1 (en) * | 2014-10-20 | 2016-04-21 | Denso Corporation | Apparatus for recognizing lane partition lines |
| CN105262989A (en) * | 2015-10-08 | 2016-01-20 | 成都时代星光科技有限公司 | Automatic inspection and real-time image acquisition transmission method of railway line unmanned aerial plane |
| CN107021015A (en) * | 2015-11-08 | 2017-08-08 | 欧特明电子股份有限公司 | System and method for image procossing |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115171384A (en) * | 2022-07-04 | 2022-10-11 | 南京四维智联科技有限公司 | Key vehicle position delay compensation method and device in vehicle-mounted display process |
| CN115171384B (en) * | 2022-07-04 | 2024-05-14 | 南京四维智联科技有限公司 | Method and device for compensating position delay of key vehicle in vehicle-mounted display process |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111086452B (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7245295B2 (en) | METHOD AND DEVICE FOR DISPLAYING SURROUNDING SCENE OF VEHICLE-TOUCHED VEHICLE COMBINATION | |
| CN109688392B (en) | AR-HUD optical projection system, mapping relation calibration method and distortion correction method | |
| CN107021015B (en) | System and method for image processing | |
| EP2485203B1 (en) | Vehicle-surroundings monitoring device | |
| EP2544449B1 (en) | Vehicle perimeter monitoring device | |
| KR100721560B1 (en) | System and method for provision of 3-dimensional car information with arbitrary viewpoint | |
| CN107211085B (en) | Image pickup apparatus and image pickup method | |
| JP7247173B2 (en) | Image processing method and apparatus | |
| CN110023141B (en) | Method and system for adjusting the orientation of a virtual camera when a vehicle turns | |
| JP2015015527A (en) | Video display system, video synthesis device and video synthesis method | |
| CN107830869B (en) | Information output method and apparatus for vehicle | |
| US9836814B2 (en) | Display control apparatus and method for stepwise deforming of presentation image radially by increasing display ratio | |
| WO2016051447A1 (en) | Information display control system and information display control method | |
| US9849835B2 (en) | Operating a head-up display of a vehicle and image determining system for the head-up display | |
| JP2015505759A (en) | Method for detecting an impending rollover of a vehicle | |
| CN114022565A (en) | Alignment method and alignment device for display equipment and vehicle-mounted display system | |
| CN113483774A (en) | Navigation method, navigation device, electronic equipment and readable storage medium | |
| CN105774657B (en) | Single-camera panoramic reverse image system | |
| CN112129313A (en) | AR navigation compensation system based on inertial measurement unit | |
| CN105721793B (en) | A kind of driving distance bearing calibration and device | |
| CN111086452B (en) | Method, device and server for compensating lane line delay | |
| JP6768554B2 (en) | Calibration device | |
| CN210139817U (en) | Automobile augmented reality display system and automobile | |
| CN111152721B (en) | Method and device for drawing guide line of lane and head-up display system | |
| JP7429246B2 (en) | Methods and systems for identifying objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 230000 a10-01, 13 / F, block a, building J1, phase II, innovation industrial park, No. 2800, innovation Avenue, high tech Zone, Hefei, Anhui Province Applicant after: Hefei Jiangcheng Technology Co.,Ltd. Address before: 518000 1st floor, North Block A, Room 203, 2nd floor, Jinke office building, 8 Qiongyu Road, Science Park community, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: SHENZHEN JIANG CHENG TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |