CN108909625B - Vehicle bottom ground display method based on panoramic all-round viewing system - Google Patents

Vehicle bottom ground display method based on panoramic all-round viewing system Download PDF

Info

Publication number
CN108909625B
CN108909625B CN201810659018.0A CN201810659018A CN108909625B CN 108909625 B CN108909625 B CN 108909625B CN 201810659018 A CN201810659018 A CN 201810659018A CN 108909625 B CN108909625 B CN 108909625B
Authority
CN
China
Prior art keywords
automobile
angle
image
front wheel
gamma
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810659018.0A
Other languages
Chinese (zh)
Other versions
CN108909625A (en
Inventor
张学武
卢鑫
李丽媛
徐晓龙
李敏
范新南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Campus of Hohai University
Original Assignee
Changzhou Campus of Hohai University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Campus of Hohai University filed Critical Changzhou Campus of Hohai University
Priority to CN201810659018.0A priority Critical patent/CN108909625B/en
Publication of CN108909625A publication Critical patent/CN108909625A/en
Application granted granted Critical
Publication of CN108909625B publication Critical patent/CN108909625B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a vehicle bottom ground display method based on a panoramic looking-around system. And secondly, finding out the area to be covered by the automobile in the next frame in the current frame through the displacement information, and extracting the area to be covered by the small automobile in the next frame. Furthermore, the running track of the trolley in a short time can be obtained according to the deflection angle of the steering wheel of the trolley, whether the trolley runs forwards or backwards on the track is judged according to the gear of the trolley, and the running distance of the trolley on the track line after one frame of time can be roughly estimated according to the speed of the trolley. The invention aims to realize the real-time display of the ground image at the bottom of the automobile by matching with an ADAS panoramic all-around viewing system. According to the top view of the vehicle body, the information of the image is extracted from the previous frame, and the ground area covered by the vehicle in the current frame is displayed, so that the function of the ADAS system is further improved.

Description

Vehicle bottom ground display method based on panoramic all-round viewing system
Technical Field
The invention relates to a vehicle bottom ground display method based on a panoramic all-round viewing system, and belongs to the technical field of automatic driving of vehicles.
Background
At present, a panoramic looking-around system collects road conditions around a vehicle body through wide-angle cameras around the vehicle body and synthesizes a complete looking-around image. However, the image of the vehicle bottom part cannot be completely displayed due to the fact that the image is shielded by the automobile, at the moment, the part, which is shielded by the automobile, in the current frame can be extracted through the image information in the video frame collected before, and the part is spliced into the current frame, so that the real-time display of the vehicle bottom image is realized.
By the technology, the automobile can effectively detect the blind area of the automobile bottom in the driving process, avoid being trapped in a hollow or running over sharp objects, and provide useful automobile bottom information for the automobile under special driving conditions such as narrow bridge passing, backing and warehousing, and the like, thereby ensuring the driving safety.
Disclosure of Invention
The purpose is as follows: in order to overcome the defects in the prior art, the invention provides a vehicle bottom ground display method based on a panoramic all-round viewing system.
The technical scheme is as follows: in order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a vehicle bottom ground display method based on a panoramic all-round viewing system specifically comprises the following steps:
step S101, reading a video frame of an automobile panoramic all-around system;
step S102, reading the running parameters of the automobile;
step S103, calculating a running track line of the automobile;
step S104, predicting the position to which the automobile of the next frame is going to move;
step S105, intercepting the image of the position predicted in step S104 from the current frame;
step S106, carrying out affine transformation on the intercepted image to obtain an underbody image;
and S107, reading the panoramic all-around view image of the next frame of automobile, splicing the estimated vehicle bottom image in the previous frame to the part covered by the automobile in the frame, and obtaining the panoramic all-around view image of the next frame of the displayed vehicle bottom image.
As a preferred scheme, the panoramic all-round looking system collects multiple paths of video images through a plurality of wide-angle lenses around a vehicle body, and the multiple paths of video images are spliced after distortion correction, so that each frame is synthesized into a complete overlook image; in the image, the automobile area is positioned in the middle, the surrounding images are the environment where the automobile is located, and the surrounding areas are environment top views which change in real time in the moving process of the automobile; in the panoramic all-round viewing system, at the next frame time, the covered image of the automobile after the movement is obtained from the current image.
Preferably, the step S102 includes: the ADAS system collects information of the driving speed, gear and steering wheel angle of the automobile at regular collection time intervals, and because the frequency of information collection is less than the frequency of displayed video frames, the driving speed, gear and steering wheel angle of the automobile when a frame of image of which the automobile information is not collected is displayed need to be estimated according to the automobile information read for the last few times.
Preferably, the step S103 includes:
step 1: obtaining the steering angle of an automobile tire according to the rotation angle of a steering wheel of the automobile;
step 2: the steering angle of the automobile tyre is mu, the intersection point of the outer circular rail and the front axle is alpha, and the included angle between the connecting line of the alpha and the circle center o of the outer circular rail and the rear axle of the automobile is
Figure BDA0001704210190000021
According to the estimation, the method can obtain,
Figure BDA0001704210190000022
Figure BDA0001704210190000023
and step 3: knowing the distance between the front and rear wheels as L, the large radius of the outer rail is given by:
Figure BDA0001704210190000031
and 4, step 4: according to the vehicle width W and a right triangle formed by the circle center o and the two wheels, the radius of the inner rail is obtained as follows:
Figure BDA0001704210190000032
and 5: since the inner track line is tangent to the inner steerable wheel in the front wheel of the vehicle and the outer track line is tangent to the outer steerable wheel in the rear wheel of the vehicle, the track line on which the vehicle travels in each frame time difference can be approximately regarded as a circle of fixed radius.
Preferably, the step S104 includes:
step 1: performing curve fitting on the change relation of the automobile speed and the steering wheel angle along with time by a least square method to obtain a driving speed and steering wheel angle prediction formula of the automobile;
step 2: assuming that the current motion state of the automobile is only related to the motion state of the first 1s, predicting the current running speed and the current steering wheel angle of the automobile through the first several groups of data;
and step 3: because the time interval between frames of the video image is short, i.e. assuming that the car is moving at a constant speed along the fixed track, the displacement of the car on the trajectory line is calculated to be L'.
Preferably, the step S105 includes:
step 1: because the image is composed of closely arranged pixel points, the image is regarded as a coordinate space according to the sequence of the pixel points from top to bottom and from left to right, the distance between adjacent pixel points in the horizontal direction and the vertical direction is the unit interval in the x-axis direction and the y-axis direction, and the displacement L' of each time of the automobile is reflected as S pixel points moved on the image according to the proportional relation between the image and a real object; establishing a rectangular coordinate system by taking the lower left corner of the panoramic view image as an origin, the horizontal direction as the x-axis direction and the vertical direction as the y-axis direction;
step 2: the coordinates of four vertexes and four tires of the automobile are known, the relative position of the area to which the automobile moves in the next frame in the current frame image is calculated according to the angle and distance information, and then the area covered by the automobile in the next frame is obtained from the current frame, and the specific steps are as follows:
when the automobile runs to the right front, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the size is
Figure BDA0001704210190000041
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000042
At the moment, the right front wheel and the x axis of the automobile are alignedThe included angle of the direction is gamma-lambda; according to the coordinates (x) of the right rear wheel of the automobile0，y0) Obtaining the coordinates (x) of the center of the circle0+R-W，y0) Further, the coordinate of the right front wheel after displacement is obtained as (x)0+R-W+R*cos(γ-λ)，y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda-mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma-lambda-mu;
when the automobile runs to the left front, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000043
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000044
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma + lambda; according to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after displacement is gamma + lambda, and the left front wheel coordinate of the automobile after displacement is further obtained as (x)0-R+W+R*cos(γ+λ)，y0+ R × sin (γ + λ)); the deflection angle of the automobile body is gamma + lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma + lambda-mu;
when the automobile runs to the right rear direction, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure BDA0001704210190000051
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000052
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma + lambda; according to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further can obtainThe coordinate of the right front wheel after displacement is (x)0+R-W+R*cos(γ+λ)，y0+ R × sin (γ + λ)); the angle of the automobile body is gamma + lambda + mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma + lambda + mu;
when the automobile runs towards the left rear part, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000053
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000054
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma-lambda; according to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after the displacement is gamma-lambda, and the left front wheel coordinate of the automobile after the displacement is further obtained as (x)0-R+W+R*cos(γ-λ)，y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma-lambda-mu;
and (3) estimating the position of the displaced automobile in an image coordinate system according to the estimated displacement information which mainly comprises the coordinates of the displaced wheels in the 360 images and the inclination angle of the automobile body, wherein the image in the area is the image area to be covered by the automobile in the next frame.
Preferably, the step S106 includes: and intercepting the image of the area where the automobile is located after the prediction displacement, and mapping the image to the vertical direction to be used as an automobile bottom image.
Has the advantages that: the vehicle bottom ground display method based on the panoramic all-around system provided by the invention displays the vehicle bottom ground in real time by means of image splicing by utilizing useful information in a video sequence of the panoramic all-around system, thereby further improving the function of the ADAS system.
Drawings
FIG. 1 is a top view image of a single frame of a panoramic look-around system;
FIG. 2 is a vehicle travel trajectory line;
FIG. 3 shows the process of displaying images of the vehicle bottom (taking a right turn as an example);
fig. 4 is a flowchart of a method for displaying vehicle bottom images of a look-around system.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
As shown in fig. 4, a vehicle bottom ground display method based on a panoramic all-around viewing system specifically comprises the following steps:
and step S101, reading a video frame of the panoramic all-around system.
As shown in fig. 1, in the panoramic all-round viewing system, multiple paths of video images are collected through multiple wide-angle lenses around a vehicle body, and are spliced after distortion correction, so that each frame is combined into a complete overhead view image. In the image, the automobile area is located in the middle, the surrounding images are the environment where the automobile is located, and the surrounding areas are the environment top view which changes in real time in the moving process of the automobile. In the panoramic all-round system, the distance from the automobile to the four sides of the image is about 1/4 automobile body lengths, so that the image covered by the automobile after the automobile moves can be obtained from the current image at the next frame time. In addition, the video frame frequency of the video image acquisition and display is about 25-30 frames/s, namely the interval time between frames is 30-40 ms.
And step S102, reading parameters such as the speed of the automobile and the like.
Specifically, the ADAS system (advanced driving assistance system) collects information about the driving speed, gear position, and steering wheel angle of the vehicle at regular collection intervals of about 100ms, so the frequency of information collection is less than the frequency of the displayed video frames, about 1: 3, that is, the driving information of the vehicle is collected once within the time of displaying three frames of images. According to the information read for the last time, the driving speed, the gear and the steering wheel angle of the automobile can be estimated when each frame of image is displayed, and the information can be considered to be approximately unchanged from the current frame to the next frame. Since the time interval of image display is short and the acceleration of the vehicle is limited, the error caused by the slight change of the driving state of the vehicle in the time interval of each frame is negligible in the case of normal driving.
Step S103, calculating the running track line of the automobile, wherein the track line is shown in FIG. 2.
Specifically, the tire steering angle of the automobile can be obtained according to the steering wheel angle of the automobile, and the tire is generally turned for 45 degrees by one and a half turns of the steering wheel, namely, the automobile tire rotates for 1 degree every 12 degrees of the steering wheel.
Referring to fig. 2, it can be known through analysis that the steering angle of the automobile tire is μ, the intersection point of the outer circular rail and the front axle is α, and the included angle between the connecting line of α and the circle center o of the outer circular rail and the rear axle of the automobile is α
Figure BDA0001704210190000071
According to the estimation, the method can obtain,
Figure BDA0001704210190000072
knowing the distance between the front and rear wheels as L, it can be obtained that the large radius of the outer rail is:
Figure BDA0001704210190000073
according to the vehicle width W and a right triangle formed by the circle center and the two wheels, the radius of the inner rail is obtained as follows:
Figure BDA0001704210190000081
since the inner track line is tangent to the inner steering wheel of the front wheel of the vehicle and the outer track line is tangent to the outer steering wheel of the rear wheel of the vehicle, the track line of the vehicle driving in each frame time difference can be approximately regarded as a circle with a fixed radius, and the radius of the track circle is related to the deflection angle of the vehicle tire.
Step S104, predicting the position to which the automobile of the next frame is going to move.
Specifically, the reading frequency of the driving state (speed, gear position, tire deflection angle) of the automobile and the display frequency of the video frames are 1: 3, so that the driving information of the automobile needs to be estimated from the automobile information read in the previous times when each frame of image is displayed. By the least square method, curve fitting can be carried out on the change relation of the information such as the automobile speed and the steering wheel angle along with time, and the driving speed and the steering wheel angle of the automobile in the current frame are obtained. Assuming that the current motion state of the vehicle is only related to the motion state of the first 1s, the current driving information of the vehicle can be predicted through 10 sets of data. Because the time interval between frames is short, the change of the speed and the steering wheel angle of the automobile in the time interval can be ignored, namely, the automobile is assumed to move at a constant speed along a fixed track, so that the displacement of the automobile on the track can be calculated to be L', and the area of the automobile after the displacement.
Step S105, the region is cut out from the current frame.
As shown in fig. 3, since the image is composed of closely arranged pixels, all the pixels can be regarded as a coordinate space according to the order of the pixels from top to bottom and from left to right, and the distance between adjacent pixels in the horizontal and vertical directions is the unit interval in the x and y axis directions. According to the proportional relation between the image and the real object, the displacement L' of the automobile each time is reflected on the image as the movement of S pixel points. Because the automobile in the panoramic all-around view image is vertically embedded in the middle of the image all the time, for the convenience of calculation, a rectangular coordinate system is established by taking the lower left corner of the panoramic all-around view image as the origin, the horizontal direction as the x-axis direction and the vertical direction as the y-axis direction. And because the structure of the panoramic all-round view image is fixed, the coordinates of four vertexes and four tires of the automobile are known, so that the relative position of the area to which the automobile moves in the next frame in the current frame image can be calculated according to the angle and distance information, and the area covered by the trolley in the next frame can be obtained from the current frame.
Specifically, when the automobile is driven to the right front, (along the direction represented by the angle and length in FIG. 2) the right front wheel of the automobile has a direction angle γ from the center of the circle, which is the magnitude of
Figure BDA0001704210190000091
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000092
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma-lambda. According to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further, the coordinates of the right front wheel after the displacement can be obtained as (x)0+R-W+R*cos(γ-λ)，y0+ R × sin (γ - λ)). The angle of the automobile body is gamma-lambda-mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma-lambda-mu.
When the automobile runs to the left front, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000093
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000094
At the moment, the included angle between the left front wheel of the automobile and the positive direction of the x axis is 90 degrees + gamma + lambda. According to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after displacement is gamma + lambda, and the left front wheel coordinate of the automobile after displacement is further obtained as (x)0-R+W+R*cos(γ+λ)，y0+ R × sin (γ + λ)). The deflection angle of the automobile body is gamma + lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction angle from the axle to the right front wheel is gamma + lambda-mu.
When the automobile runs to the right rear direction, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure BDA0001704210190000101
After the vehicle moves S pixel points along the track in the image, the vehicle rotates on the track lineAngle lambda is
Figure BDA0001704210190000102
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma + lambda. According to the coordinates (x) of the right rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0+R-W，y0) Further, the coordinates of the right front wheel after the displacement can be obtained as (x)0+R-W+R*cos(γ+λ)，y0+ R × sin (γ + λ)). The angle of the automobile body is gamma + lambda + mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction angle from the axle to the left front wheel is gamma + lambda + mu.
When the automobile runs towards the left rear part, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure BDA0001704210190000103
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure BDA0001704210190000104
At the moment, the included angle between the left front wheel of the automobile and the positive direction of the x axis is 90 degrees + gamma-lambda. According to the coordinates (x) of the left rear wheel of the automobile0，y0) The center coordinates (x) of the circle can be obtained0-R+W，y0) And the direction angle from the center of the circle to the left front wheel of the automobile after the displacement is gamma-lambda, and the left front wheel coordinate of the automobile after the displacement is further obtained as (x)0-R+W+R*cos(γ-λ)，y0+ R × sin (γ - λ)). The angle of the automobile body is gamma-lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma-lambda-mu.
And estimating the position of the displaced automobile in an image coordinate system according to the estimated displacement information which mainly comprises the coordinates of the displaced wheels in the 360 images and the inclination angle of the automobile body, wherein the image in the area is the image area to be covered by the automobile in the next frame.
Step S106, affine transformation.
Specifically, after the image of the area where the automobile is located after the displacement is captured, the image is a pair of inclined rectangular images, and the images need to be mapped to the vertical direction to be spliced to the automobile area of the next frame.
And S107, reading the panoramic all-around view image of the next frame of automobile, splicing the estimated vehicle bottom image in the previous frame to the part covered by the automobile in the frame, and obtaining the panoramic all-around view image of the next frame of the displayed vehicle bottom image.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.

Claims (4)

1. A vehicle bottom ground display method based on a panoramic all-round viewing system is characterized in that: the method specifically comprises the following steps:
step S101, reading a video frame of an automobile panoramic all-around system;
step S102, reading the running parameters of the automobile;
step S103, calculating a running track line of the automobile;
the step S103 includes:
step 1: obtaining the steering angle of an automobile tire according to the rotation angle of a steering wheel of the automobile;
step 2: the steering angle of an automobile tire is mu, the intersection point of the outer circular rail and the front axle is alpha, the included angle between the connecting line of the alpha and the circle center o of the outer circular rail and the rear axle of the automobile is phi, and the mu is phi calculated;
and step 3: knowing the distance between the front and rear wheels as L, the large radius of the outer rail is given by:
Figure FDA0003116602650000011
and 4, step 4: according to the vehicle width W and a right triangle formed by the circle center o and the two wheels, the radius of the inner rail is obtained as follows:
Figure FDA0003116602650000012
and 5: because the inner track line is tangent to the inner steering wheel in the front wheel of the automobile and the outer track line is tangent to the outer steering wheel in the rear wheel of the automobile, the track line of the automobile running in the time difference of each frame can be approximately regarded as a circle with a fixed radius;
step S104, predicting the position to which the automobile of the next frame is going to move;
the step S104 includes:
step 1: performing curve fitting on the change relation of the automobile speed and the steering wheel angle along with time by a least square method to obtain a driving speed and steering wheel angle prediction formula of the automobile;
step 2: assuming that the current motion state of the automobile is only related to the motion state of the first 1s, predicting the current running speed and the current steering wheel angle of the automobile through the first several groups of data;
and step 3: because the time interval between the video image frames is short, namely the automobile is supposed to do uniform motion along the fixed track, the displacement of the automobile on the track line is calculated to be L';
step S105, intercepting the image of the position predicted in step S104 from the current frame; the step S105 includes:
step 1: because the image is composed of closely arranged pixel points, the image is regarded as a coordinate space according to the sequence of the pixel points from top to bottom and from left to right, the distance between adjacent pixel points in the horizontal direction and the vertical direction is the unit interval in the x-axis direction and the y-axis direction, and the displacement L' of each time of the automobile is reflected as S pixel points moved on the image according to the proportional relation between the image and a real object; establishing a rectangular coordinate system by taking the lower left corner of the panoramic view image as an origin, the horizontal direction as the x-axis direction and the vertical direction as the y-axis direction;
step 2: the coordinates of four vertexes and four tires of the automobile are known, the relative position of the area to which the automobile moves in the next frame in the current frame image is calculated according to the angle and distance information, and then the area covered by the automobile in the next frame is obtained from the current frame, and the specific steps are as follows:
when the automobile moves to the right frontWhen driving, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure FDA0003116602650000021
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure FDA0003116602650000022
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma-lambda; according to the coordinates (x) of the right rear wheel of the automobile0,y0) Obtaining the coordinates (x) of the center of the circle0+R-W,y0) Further, the coordinate of the right front wheel after displacement is obtained as (x)0+R-W+R*cos(γ-λ),y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda-mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma-lambda-mu;
when the automobile runs to the left front, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure FDA0003116602650000031
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure FDA0003116602650000032
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma + lambda; according to the coordinates (x) of the left rear wheel of the automobile0,y0) The center coordinates (x) of the circle can be obtained0-R+W,y0) And the direction angle from the center of the circle to the left front wheel of the automobile after displacement is gamma + lambda, and the left front wheel coordinate of the automobile after displacement is further obtained as (x)0-R+W+R*cos(γ+λ),y0+ R × sin (γ + λ)); the deflection angle of the automobile body is gamma + lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma + lambda-mu;
when the automobile runs to the right rear direction, the direction angle from the right front wheel to the circle center of the automobile is gamma, and the magnitude is
Figure FDA0003116602650000033
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure FDA0003116602650000034
At the moment, the positive direction included angle between the right front wheel of the automobile and the x axis is gamma + lambda; according to the coordinates (x) of the right rear wheel of the automobile0,y0) The center coordinates (x) of the circle can be obtained0+R-W,y0) Further, the coordinates of the right front wheel after the displacement can be obtained as (x)0+R-W+R*cos(γ+λ),y0+ R × sin (γ + λ)); the angle of the automobile body is gamma + lambda + mu-90 degrees, and the angle of the right front wheel perpendicular to the automobile body along the direction from the axle to the left front wheel is gamma + lambda + mu;
when the automobile runs towards the left rear part, the direction angle from the initial left front wheel to the circle center of the automobile is gamma, and the magnitude of the direction angle is gamma
Figure FDA0003116602650000041
When the vehicle moves S pixel points along the track in the image, the rotating angle lambda of the vehicle on the track line is
Figure FDA0003116602650000042
At the moment, the positive direction included angle between the left front wheel of the automobile and the x axis is 90 degrees + gamma-lambda; according to the coordinates (x) of the left rear wheel of the automobile0,y0) The center coordinates (x) of the circle can be obtained0-R+W,y0) And the direction angle from the center of the circle to the left front wheel of the automobile after the displacement is gamma-lambda, and the left front wheel coordinate of the automobile after the displacement is further obtained as (x)0-R+W+R*cos(γ-λ),y0+ R × sin (γ - λ)); the angle of the automobile body is gamma-lambda + 90-mu, and the angle of the left front wheel perpendicular to the automobile body along the direction from the axle to the right front wheel is gamma-lambda-mu;
according to the estimated displacement information, the coordinates of the displaced wheels in the 360 images and the inclination angle of the vehicle body are mainly included, so that the position of the displaced vehicle in an image coordinate system can be estimated, and the image in the area is the image area to be covered by the next frame of vehicle;
step S106, carrying out affine transformation on the intercepted image to obtain an underbody image;
and S107, reading the panoramic all-around view image of the next frame of automobile, splicing the estimated vehicle bottom image in the previous frame to the part covered by the automobile in the frame, and obtaining the panoramic all-around view image of the next frame of the displayed vehicle bottom image.
2. The vehicle bottom ground display method based on the panoramic all-round viewing system as claimed in claim 1, characterized in that: the panoramic all-around viewing system collects multiple paths of video images through a plurality of wide-angle lenses around the vehicle body, and the images are spliced after distortion correction, so that each frame is synthesized into a complete overlooking image; in the image, the automobile area is positioned in the middle, the surrounding images are the environment where the automobile is located, and the surrounding areas are environment top views which change in real time in the moving process of the automobile; in the panoramic all-round viewing system, at the next frame time, the covered image of the automobile after the movement is obtained from the current image.
3. The vehicle bottom ground display method based on the panoramic all-round viewing system as claimed in claim 1, characterized in that: the step S102 includes: the ADAS system collects information of the driving speed, gear and steering wheel angle of the automobile at regular collection time intervals, and because the frequency of information collection is less than the frequency of displayed video frames, the driving speed, gear and steering wheel angle of the automobile when a frame of image of which the automobile information is not collected is displayed need to be estimated according to the automobile information read for the last few times.
4. The vehicle bottom ground display method based on the panoramic all-round viewing system as claimed in claim 1, characterized in that: the step S106 includes: and intercepting the image of the area where the automobile is located after the prediction displacement, and mapping the image to the vertical direction to be used as an automobile bottom image.
CN201810659018.0A 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system Active CN108909625B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810659018.0A CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810659018.0A CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Publications (2)

Publication Number Publication Date
CN108909625A CN108909625A (en) 2018-11-30
CN108909625B true CN108909625B (en) 2021-09-17

Family

ID=64422337

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810659018.0A Active CN108909625B (en) 2018-06-22 2018-06-22 Vehicle bottom ground display method based on panoramic all-round viewing system

Country Status (1)

Country Link
CN (1) CN108909625B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111836005A (en) * 2019-04-23 2020-10-27 东莞潜星电子科技有限公司 Vehicle-mounted 3D panoramic all-around driving route display system
CN111845713A (en) * 2019-04-26 2020-10-30 东莞潜星电子科技有限公司 Parking route planning method based on vehicle-mounted 3D panoramic looking around
CN111856871A (en) * 2019-04-26 2020-10-30 东莞潜星电子科技有限公司 Vehicle-mounted 3D panoramic all-around display method
CN112215747A (en) * 2019-07-12 2021-01-12 杭州海康威视数字技术股份有限公司 Method and device for generating vehicle-mounted panoramic picture without vehicle bottom blind area and storage medium
CN110458884A (en) * 2019-08-16 2019-11-15 北京茵沃汽车科技有限公司 Method, apparatus, the medium of vehicle operation state trajectory line are generated in panorama sketch
CN113228135B (en) * 2021-03-29 2022-08-26 华为技术有限公司 Blind area image acquisition method and related terminal device
CN113382294B (en) * 2021-06-04 2023-05-12 广州小鹏汽车科技有限公司 Picture display processing method and device of remote cockpit, cockpit and system
CN113935942A (en) * 2021-09-02 2022-01-14 同致电子科技（厦门）有限公司 Vehicle bottom transparency method, device, equipment, storage medium and computer program product based on nonlinear optimization

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE60207655T2 (en) * 2001-09-07 2006-06-08 Matsushita Electric Industrial Co., Ltd., Kadoma Device for displaying the environment of a vehicle and system for providing images
JP4254887B2 (en) * 2006-07-06 2009-04-15 日産自動車株式会社 Image display system for vehicles
KR100956858B1 (en) * 2009-05-19 2010-05-11 주식회사 이미지넥스트 Sensing method and apparatus of lane departure using vehicle around image
US9187091B2 (en) * 2012-07-30 2015-11-17 Ford Global Technologies, Llc Collision detection system with a plausibiity module
CN203228708U (en) * 2013-04-25 2013-10-09 郑州宇通客车股份有限公司 Vehicular panoramic display system and a bus with same
CN106608220B (en) * 2015-10-22 2019-06-25 比亚迪股份有限公司 Generation method, device and the vehicle of vehicle bottom image
CN108068696B (en) * 2016-11-17 2020-06-19 比亚迪股份有限公司 Display method and display device of panoramic image
CN107066954B (en) * 2017-03-23 2019-07-23 浙江零跑科技有限公司 A kind of vehicle-mounted 360 degree are looked around display methods and system
CN107274342A (en) * 2017-05-22 2017-10-20 纵目科技（上海）股份有限公司 A kind of underbody blind area fill method and system, storage medium, terminal device
CN107745677A (en) * 2017-09-30 2018-03-02 东南（福建）汽车工业有限公司 A kind of method of the 4D underbody transparent systems based on 3D full-view image systems

Also Published As

Publication number Publication date
CN108909625A (en) 2018-11-30

Similar Documents

Publication Publication Date Title
CN108909625B (en) Vehicle bottom ground display method based on panoramic all-round viewing system
US20200401822A1 (en) Monocular cued detection of three-dimensional structures from depth images
CN102745138B (en) Dual view-field dynamic-trajectory reverse image system
US20180307922A1 (en) Method of detecting obstacle around vehicle
CN105678787A (en) Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera
US9792683B2 (en) System, vehicle and method for online calibration of a camera on a vehicle
CN107229908A (en) A kind of method for detecting lane lines
CN102163331A (en) Image-assisting system using calibration method
CN110462682B (en) Object detection device and vehicle
CN102158684A (en) Self-adapting scene image auxiliary system with image enhancement function
EP3803790B1 (en) Motion segmentation in video from non-stationary cameras
CN202716870U (en) Automobile forward track auxiliary device
CN103072528A (en) Vehicle and panoramic parking method and system thereof
CN107284455B (en) A kind of ADAS system based on image procossing
CN103204104B (en) Monitored control system and method are driven in a kind of full visual angle of vehicle
CN108174089B (en) Backing image splicing method and device based on binocular camera
CN107878561A (en) A kind of dynamic reverse trajectory discharge drawing method and its device
CN108198248A (en) A kind of vehicle bottom image 3D display method
CN202847561U (en) Car-backing image system
CN112529935A (en) Target vehicle motion state identification method and device
JP4020071B2 (en) Ambient condition display device
JPH07250268A (en) Vehicle periphery monitoring device
CN115116025B (en) Driving risk assessment system based on underground road vision distance
CN112308786B (en) Method for resolving target vehicle motion in vehicle-mounted video based on photogrammetry
JP7402753B2 (en) Safety support system and in-vehicle camera image analysis method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant