CN107066954B - A kind of vehicle-mounted 360 degree are looked around display methods and system - Google Patents
A kind of vehicle-mounted 360 degree are looked around display methods and system Download PDFInfo
- Publication number
- CN107066954B CN107066954B CN201710180080.7A CN201710180080A CN107066954B CN 107066954 B CN107066954 B CN 107066954B CN 201710180080 A CN201710180080 A CN 201710180080A CN 107066954 B CN107066954 B CN 107066954B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- splicing seams
- seam
- image
- track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
Abstract
The invention discloses a kind of vehicle-mounted 360 degree to look around display methods and system.Method includes the following steps: S1: each camera acquired image is carried out viewpoint change by looking into look-up table by projection module, projects to vehicle；S2: the optical alignment of four camera images is carried out using exposure gain algorithm；S3: being spliced using splicing seams of the stitching algorithm to adjacent cameras acquired image, is obtained vehicle and is overlooked spliced map.The present invention a certain position splicing seams when adjacent time sequence difference is little, previous frame splicing seams are multiplexed, greatly reduce the influence that noise calculates splicing seams, stability is increased with this, in addition the time of splicing seams calculating directly is decreased using previous frame splicing seams, improves efficiency of algorithm.
Description
Technical field
Technical field is looked around the present invention relates to vehicle-mounted, more particularly to a kind of vehicle-mounted 360 degree are looked around display methods and system.
Background technique
Vehicle-mounted 360 degree of viewing systems are that auxiliary drives one of function, this system can be aobvious to driver under speed operation
Show vehicle periphery situation (including part short distance blind spot region).Existing 360 degree of viewing systems generally vehicle body it is forward and backward,
One camera of left and right each installation, each camera acquire data within the vision respectively, then will be adopted through the processing unit
The image of collection is integrated, and lap is merged, and 360 degree of panorama sketch of formation carry out output in display screen and show.
Vehicle-mounted 360 degree of viewing systems are needed when being spliced the image of the camera acquisition of the front, rear, left and right of vehicle body
Calculation processing is carried out to splicing seams.Picture noise can have an impact splicing seams calculating, when camera is larger in certain interference
Environment when (underground garage), there is the noise generated due to indoor light in image, these noises will affect the meter of splicing seams
It calculates, when causing vehicle's surroundings environment unchanged, seam also will appear biggish variation, due to the influence of noise, under Same Scene
The splicing seams of each frame have a very big difference, and stitching image will appear lasting variation, unstable.Meanwhile it is existing most of
This class system splicing seams are fixed splicing seams, under the scene of certain complexity (ground out-of-flatness, splicing seams pass through barrier etc.)
There is a problem of that splicing seams are obvious.
Summary of the invention
Present invention mainly solves the vehicles that when vehicle's surroundings environment is unchanged, existing vehicle-mounted 360 degree of viewing systems are shown
Splicing seams lasting variation affected by noise in vertical view spliced map, unstable technical problem, provide a kind of vehicle-mounted 360 degree of rings
Depending on display methods and system, a certain position splicing seams when adjacent time sequence difference is little, be multiplexed previous frame splicing seams, greatly
Greatly the influence that noise calculates splicing seams is reduced, stability is increased with this, is in addition directly also subtracted using previous frame splicing seams
The time for having lacked splicing seams calculating, improve efficiency of algorithm.
The present invention solves existing vehicle-mounted 360 degree of viewing systems using fixed splicing seams simultaneously, and splicing seams can not cut-through
The technical issues of object, provides a kind of vehicle-mounted 360 degree and looks around display methods and system, bypasses splicing seams in conjunction with trailer-mounted radar
Barrier improves splicing effect from this.
To solve the above-mentioned problems, the present invention is achieved by the following scheme:
Of the invention a kind of vehicle-mounted 360 degree look around display methods, and front side, rear side, left side, the right side of automobile are respectively equipped with one
A camera, comprising the following steps:
S1: each camera acquired image is carried out viewpoint change by looking into look-up table by projection module, projects to vehicle
Top view；
S2: the optical alignment of four camera images is carried out using exposure gain algorithm；
S3: being spliced using splicing seams of the stitching algorithm to adjacent cameras acquired image, is obtained vehicle and is overlooked spelling
Map interlinking；
The method spliced to every splicing seams the following steps are included:
S31: judge that i-th splicing seams whether there is the splicing seams last_seam of previous framei, if there is previous frame
Splicing seams last_seami, then the splicing seams last_seam of previous frame is calculatediOn each pixel value difference color_diffi；
S32: if the splicing seams last_seam of previous frame is not present in i-th splicing seamsiOr color_diffiGreater than threshold
Value color_diff0, then new splicing seams new_seam is calculatediIf color_diffiLess than or equal to threshold value color_
diff0, then by last_seamiIt is assigned to new_seami；
S33: spliced using stitching algorithm along splicing seams.
In the technical scheme, the front side of automobile, rear side, left side, right side camera acquisition image mosaic after form four
A splicing seams.When the splicing seams of a certain position are when adjacent time sequence difference is little, i.e. color_diffiLess than or equal to threshold value
color_diff0, then the newest splicing seams in the position are multiplexed previous frame splicing seams, greatly reduce noise in this way to splicing seams meter
The influence of calculation increases stability with this, in addition directly decreases the time of splicing seams calculating using previous frame splicing seams, mention
High efficiency of algorithm.
Preferably, the step S32 is the following steps are included: if the splicing seams of previous frame are not present in i-th splicing seams
last_seamiOr color_diffiGreater than threshold value color_diff0, then judge whether there is trailer-mounted radar on vehicle, if there is
Trailer-mounted radar then obtains vehicle body surrounding obstacle information by trailer-mounted radar, calculates new splicing seams new_seamiNeed around
The region crossed, new splicing seams new_seamiForbid then calculating new splicing seams new_ across barrier region
seamiIf directly calculating new splicing seams new_seam without trailer-mounted radariIf color_diffiIt is less than or waits
In threshold value color_diff0, then by last_seamiIt is assigned to new_seami.Based on trailer-mounted radar provide Obstacle Position,
This method connects when carrying out the calculating of splicing seams around corresponding region so as to improve splicing effect to avoid what three-dimensional object generated
Stitch geometric alignment error.
Preferably, also executing following steps before being spliced in the step S33 along splicing seams: whether judging car door
It opens, when certain side car door opening, which is overlooked into corresponding part pixel value zero setting in spliced map in vehicle, and will be corresponding
Splicing seams be set as a skew lines.When left and right car door opening, respective camera image is shielded, left and right car door opening is eliminated
When respective camera be subjected to displacement relative to car body, cause stitching image to have the problem of obvious seam.
Preferably, it further includes step S4 that a kind of vehicle-mounted 360 degree, which are looked around display methods: being turned using automobile gear level, steering wheel
Angle λ calculates track of vehicle, overlooks spliced map central area addition vehicle areal model, drafting vehicle in the vehicle of generation
Track, track of vehicle of painting in forward sight or backsight image.Comprehensively consider automobile gear level and steering wheel angle, is based on sideway plane
Vehicle kinematics modeling rendering track of vehicle line, improves the practicability of system.
Preferably, in the step S31 previous frame splicing seams last_seamiOn each pixel value difference
color_diffiIt is calculated using following formula:
Wherein, last_seamiFor i-th splicing seams；
color_diffiFor splicing seams last_seamiThe average value of the value differences of upper adjacent image；
U is splicing seams last_seamiOn a point；
IiIt (u) is point uiPixel value on the image of camera i；
IjIt (u) is point ujPixel value on the image of camera j；
NiFor splicing seams last_seamiThe number of upper pixel.
Preferably, splicing seams new_seam new in the step S32iIt is calculated using following formula:
Wherein, u is splicing seams new_seamiAny one point；
Obstacle is barrier region set；
CostMRF(s, t, Cam1, Cam2)=| | Cam1(s)-Cam2(s)||+||
Cam1(t)-Cam2(t)||；
Wherein, 2 neighbor pixels of s, t2 camera image overlapping regions；
Cam1(s), Cam2(s) brightness value of the same point s in two camera overlapping regions；
Cam1(t), Cam2(t) brightness value of the same point t in two camera overlapping regions.
Preferably, the step S33 splices splicing seams using following formula:
F (x, y)=W1 (x, y) Fl (x, y)+Wr (x, y) Fr (x, y)；
Wherein, Fl (x, y), Fr (x, y) are the images of 2 adjacent cameras of splicing；
X, y are the direction the x and y coordinates an of point, and it is positive direction that coordinate origin is the image upper left corner downwards to the right；
W1, Wr are the weighting function of 2 adjacent cameras of splicing, are determined by splicing seams, using splicing seams as center line, are formed
One width is the overlapping region of T, in this region weighted value by overlapping region one side edge each point along splicing seams normal
Direction is 0 from 1 monotone decreasing to another side edge.
Preferably, track of vehicle is calculated using automobile gear level, steering wheel angle λ in the step S4, in the vehicle of generation
Overlook spliced map central area addition vehicle areal model, draw track of vehicle method the following steps are included:
Front left wheel corner B, front right wheel corner A, the vehicle turning center ICR of vehicle are calculated, coordinate origin O is in rear shaft center
Point, coordinate are backward, to the right positive direction；
ICR=(RICR, 0)；
Wherein, RICRIt is the turning radius of back axle central point；
L is distance between axles；
D is wheelbase；
λ is steering wheel angle, is clockwise positive direction with right ahead for 0；
The movement of left and right wheels can be obtained by vehicle turning center ICR, front left wheel corner B, front right wheel corner A and automobile gear level
Track, if vehicle is in forward, using vehicle turning center ICR as the center of circle, using left and right vehicle wheel front-wheel as starting point, to vehicle
Direction of advance does the arc that length is N；If vehicle, which is in, retreats gear, using vehicle turning center ICR as the center of circle, after left and right vehicle wheel
Wheel is starting point, does the arc that length is N to vehicle rollback direction；If vehicle is in other gears, motion profile is not shown.
Preferably, the method for track of vehicle of painting in forward sight or backsight image in the step S4 includes following step
It is rapid: if vehicle is in forward, the trajectory line in top view to be divided into M-1 parts, takes M endpoint, is looked by the mapping of forward sight
It looks for table to obtain this M point respective coordinates in forward sight image, the track being made of this M point is then drawn in forward sight image；
If vehicle, which is in, retreats gear, the trajectory line in top view is divided into M-1 parts, takes M endpoint, passes through the Map Searching table of backsight
This M point corresponding coordinate in backsight image is obtained, the track being made of this M point is then drawn in backsight image.
Of the invention a kind of vehicle-mounted 360 degree look around display system, look around display side using above-mentioned a kind of vehicle-mounted 360 degree
Method, comprising:
Four cameras, are separately positioned on automobile front side, rear side, left side, right side；
Image mosaic module forms vehicle and bows according to camera acquired image, trailer-mounted radar signal, door switch signal
Depending on spliced map, it is output to post processing of image module；
Track of vehicle computing module calculates track of vehicle according to automobile gear level, steering wheel angle λ, after being output to image
Manage module；
Post processing of image module, the Data Integration that image mosaic module and track of vehicle computing module are exported, is generating
Vehicle overlook spliced map central area addition vehicle areal model, draw track of vehicle, and be output to car-mounted display
Screen.
The beneficial effects of the present invention are: (1) a certain position splicing seams when adjacent time sequence difference is little, multiplexing upper one
Frame splicing seams greatly reduce the influence that noise calculates splicing seams, increase stability with this, in addition directly use previous frame
Splicing seams decrease the time of splicing seams calculating, improve efficiency of algorithm.(2) the obstacle level provided based on trailer-mounted radar
It sets, what this method was generated in the calculating of progress splicing seams around corresponding region so as to improve splicing effect to avoid three-dimensional object
Seam geometric alignment error.(3) when left and right car door opening, respective camera image is shielded, left and right car door opening phase is eliminated
It answers camera to be subjected to displacement relative to car body, stitching image is caused to have the problem of obvious seam.(4) comprehensively consider automobile gear level and side
To disk corner, it is based on sideway plane vehicle kinematics modeling rendering track of vehicle line, improves the practicability of system.
Detailed description of the invention
Fig. 1 is a kind of work flow diagram of the invention；
Fig. 2 is the weighted value curve at the point i of splicing seams along normal direction；
Fig. 3 is that the vehicle when left-hand door is opened overlooks spliced map；
Fig. 4 is that track of vehicle calculates schematic diagram；
Fig. 5 is a kind of structural schematic diagram of the invention；
Fig. 6 is distribution map of four cameras on vehicle.
In figure: 1, camera, 2, image mosaic module, 3, track of vehicle computing module, 4, post processing of image module.
Specific embodiment
Below with reference to the embodiments and with reference to the accompanying drawing the technical solutions of the present invention will be further described.
Embodiment: a kind of vehicle-mounted 360 degree of the present embodiment look around display methods, front side, rear side, left side, the right side of automobile
Be respectively equipped with a camera, as shown in Figure 1 the following steps are included:
S1: each camera acquired image is carried out viewpoint change by looking into look-up table by projection module, projects to vehicle
Top view；
What look-up table saved is to overlook coordinate system to the Nonlinear Mapping relationship of each camera coordinates system, sees formula (1)；It is logical
The pixel coordinate that look-up table obtains the corresponding camera image of each pixel in top view is crossed, then (such as bilinearity is inserted by interpolation
Value) mode, obtain the value of each pixel of top view, the determination of look-up table, which can pass through, carries out the inside and outside mark joined to each camera
Surely (such as flake correction, projective transformation) is obtained.
G (x, y)=f (h (x, y)) (1)；
Wherein, g (x, y) is top view；
F (x, y) is camera original image；
H (x, y) is mapping function, and mapping relations save in a lookup table；
S2: the optical alignment of four camera images is carried out using exposure gain algorithm；
By minimizing following error function error, formula (2) is seen to obtain camera gain；
Wherein, NijThe number of pixels of camera j is overlapped in for camera i；
Gaini, gainjIt is the gain of camera i and j respectively；
It is camera i in the pixel average with the overlapping region camera j；
σNIt is the standard variance of normalized luminance errors；
σgIt is the standard variance of gain；
N is the number of camera；
S3: being spliced using splicing seams of the stitching algorithm to adjacent cameras acquired image, is obtained vehicle and is overlooked spelling
Map interlinking；
The method spliced to every splicing seams the following steps are included:
S31: judge that i-th splicing seams whether there is the splicing seams last_seam of previous framei, if there is previous frame
Splicing seams last_seami, then the splicing seams last_seam of previous frame is calculatediOn each pixel value difference color_diffi,
See formula (3)；
Wherein, last_seamiFor i-th splicing seams；
color_diffiFor splicing seams last_seamiThe average value of the value differences of upper adjacent image；
U is splicing seams last_seamiOn a point；
IiIt (u) is point uiPixel value on the image of camera i；
IjIt (u) is point ujPixel value on the image of camera j；
NiFor splicing seams last_seamiThe number of upper pixel；
S32: if the splicing seams last_seam of previous frame is not present in i-th splicing seamsiOr color_diffiGreater than threshold
Value color_diff0, then judge whether there is trailer-mounted radar on vehicle, if there is trailer-mounted radar, then vehicle obtained by trailer-mounted radar
Body surrounding obstacle information calculates new splicing seams new_seamiThe region for needing to bypass, new splicing seams new_seamiProhibit
Only across barrier region, new splicing seams new_seam is then calculated using graph-cut algorithmiIf do not had
Trailer-mounted radar, the graph-cut algorithm that then be used directly calculate new splicing seams new_seamiIf color_diffiIt is less than
Or it is equal to threshold value color_diff0, then by last_seamiIt is assigned to new_seami；
Forbid new splicing seams new_seamiIt is constrained across barrier region using formula (4):
Wherein, u is splicing seams new_seamiAny one point；
Obstacle is barrier region set；
Use the following cost equation Cost of graph-cut algorithmic minimizingMRF, see formula (5), new splicing be calculated
Stitch new_seami:
CostMRF(s, t, Cam1, Cam2)=| | Cam1(s)-Cam2(s)||+||
Cam1(t)-Cam2(t)||(5)；
Wherein, 2 neighbor pixels of s, t2 camera image overlapping regions；
Cam1(s), Cam2(s) brightness value of the same point s in two camera overlapping regions；
Cam1(t), Cam2(t) brightness value of the same point t in two camera overlapping regions；
S33: judging whether car door is opened, when certain side car door opening, the side camera is right in vehicle vertical view spliced map
Partial pixel value zero setting is answered, and sets a skew lines for corresponding splicing seams；
Spliced using stitching algorithm along splicing seams, formula is as follows:
F (x, y)=Wl (x, y) Fl (x, y)+Wr (x, y) Fr (x, y) (6)；
Wherein, Fl (x, y), Fr (x, y) are the images of 2 adjacent cameras of splicing；
X, y are the direction the x and y coordinates an of point, and it is positive direction that coordinate origin is the image upper left corner downwards to the right；
Wl, Wr are the weighting function of 2 adjacent cameras of splicing, are determined by splicing seams, using splicing seams as center line, are formed
One width is the overlapping region of T, in this region weighted value by overlapping region one side edge each point along splicing seams normal
Direction is the weighted value curve at point i that 0, Fig. 2 is splicing seams along normal direction from 1 monotone decreasing to another side edge；
S4: calculating track of vehicle using automobile gear level, steering wheel angle λ, overlooks spliced map center in the vehicle of generation
It adds vehicle areal model, draw track of vehicle, track of vehicle of painting in forward sight or backsight image, if vehicle in domain
There is trailer-mounted radar, then the obstacle information (orientation, distance) provided according to trailer-mounted radar corresponding region on stitching image marks
Corresponding information warning out.
The front side of automobile, rear side, left side, right side camera acquisition image mosaic after form four splicing seams.When a certain
The splicing seams of position are when adjacent time sequence difference is little, i.e. color_diffiLess than or equal to threshold value color_diff0, then the position
It newest splicing seams multiplexing previous frame splicing seams is set, greatly reduces the influence that noise calculates splicing seams in this way, increased with this
Add stability, in addition directly decreases the time of splicing seams calculating using previous frame splicing seams, improve efficiency of algorithm.
Based on trailer-mounted radar provide Obstacle Position, this method carry out splicing seams calculating when around corresponding region with
This improves the seam geometric alignment error that splicing effect generates to avoid three-dimensional object.
When left and right car door opening, shield respective camera image, when eliminating left and right car door opening respective camera relative to
Car body is subjected to displacement, and stitching image is caused to have the problem of obvious seam.Vehicle when left-hand door is opened overlooks spliced map, such as schemes
Shown in 3.
Step S4 the following steps are included:
S41: track of vehicle is calculated using automobile gear level, steering wheel angle λ, following formula (7), (8), (9), (10) are respectively
Calculate front left wheel corner B, front right wheel corner A, vehicle turning center ICR, coordinate origin O in rear shaft center's point, coordinate backward,
It is to the right positive direction, as shown in figure 4,
ICR=(RICR, 0) and (8)；
Wherein, RICRIt is the turning radius of back axle central point；
L is distance between axles；
D is wheelbase；
λ is steering wheel angle, is clockwise positive direction with right ahead for 0；
S42: spliced map central area addition vehicle areal model, drafting vehicle rail are overlooked in the vehicle of generation
Mark: can be obtained the motion profile of left and right wheels by vehicle turning center ICR, front left wheel corner B, front right wheel corner A and automobile gear level,
If vehicle is in forward, using vehicle turning center ICR as the center of circle, using left and right vehicle wheel front-wheel as starting point, advance to vehicle
Do the arc that length is N in direction；If vehicle, which is in, retreats gear (reverse gear), using vehicle turning center ICR as the center of circle, with left and right vehicle wheel
Rear-wheel is starting point, does the arc that length is N to vehicle rollback direction；If vehicle is in other gears, motion profile is not shown；
S43: track of vehicle of painting in forward sight or backsight image: if vehicle is in forward, by the rail in top view
Trace is divided into M-1 parts, takes M endpoint, obtains this M point respective coordinates in forward sight image by the Map Searching table of forward sight,
Then the track being made of this M point is drawn in forward sight image；If vehicle, which is in, retreats gear, by the trajectory line in top view
It is divided into M-1 parts, takes M endpoint, this M point corresponding coordinate in backsight image is obtained by the Map Searching table of backsight, so
The track being made of this M point is drawn in backsight image afterwards.
S44: it if vehicle has trailer-mounted radar, is being spelled according to the obstacle information (orientation, distance) that trailer-mounted radar provides
Corresponding region marks out corresponding information warning on map interlinking picture.
Comprehensively consider automobile gear level and steering wheel angle, is based on sideway plane vehicle kinematics modeling rendering track of vehicle
Line improves the practicability of system.
A kind of vehicle-mounted 360 degree of the present embodiment look around display system, look around display side using above-mentioned a kind of vehicle-mounted 360 degree
Method, as shown in Figure 5, comprising:
Four cameras 1, are separately positioned on automobile front side, rear side, left side, right side, as shown in Figure 6；
Image mosaic module 2 forms vehicle according to camera acquired image, trailer-mounted radar signal, door switch signal
Spliced map is overlooked, post processing of image module is output to；
Track of vehicle computing module 3 calculates track of vehicle according to automobile gear level, steering wheel angle λ, after being output to image
Manage module；
Post processing of image module 4, the Data Integration that image mosaic module and track of vehicle computing module are exported, is generating
Vehicle overlook spliced map central area addition vehicle areal model, draw track of vehicle, and be output to car-mounted display
Screen.
Claims (8)
1. a kind of vehicle-mounted 360 degree are looked around display methods, front side, rear side, left side, the right side of automobile are respectively equipped with a camera,
It is characterized in that, comprising the following steps:
S1: each camera acquired image is carried out viewpoint change by looking into look-up table by projection module, projects to vehicle vertical view
Figure；
S2: the optical alignment of four camera images is carried out using exposure gain algorithm；
S3: being spliced using splicing seams of the stitching algorithm to adjacent cameras acquired image, is obtained vehicle and is overlooked spliced map；
The method spliced to every splicing seams the following steps are included:
S31: judge that i-th splicing seams whether there is the splicing seams last_seam of previous framei, if there is the splicing seams of previous frame
last_seami, then the splicing seams last_seam of previous frame is calculatediOn each pixel value difference color_diffi；
S32: if the splicing seams last_seam of previous frame is not present in i-th splicing seamsiOr color_diffiGreater than threshold value
color_diff0, then new splicing seams new_seam is calculatediIf color_diffiLess than or equal to threshold value color_
diff0, then by last_seamiIt is assigned to new_seami；
S33: spliced using stitching algorithm along splicing seams；
Following steps are also executed before being spliced in the step S33 along splicing seams: judging whether car door is opened, when certain side vehicle
When door is opened, which is overlooked into corresponding part pixel value zero setting in spliced map in vehicle, and corresponding splicing seams are arranged
For a skew lines.
2. vehicle-mounted 360 degree of one kind according to claim 1 are looked around display methods, which is characterized in that the step S32 includes
Following steps: if the splicing seams last_seam of previous frame is not present in i-th splicing seamsiOr color_diffiGreater than threshold value
color_diff0, then judge whether there is trailer-mounted radar on vehicle, if there is trailer-mounted radar, then vehicle body obtained by trailer-mounted radar
Surrounding obstacle information calculates new splicing seams new_seamiThe region for needing to bypass, new splicing seams new_seamiForbid
Across barrier region, new splicing seams new_seam is then calculatediIf directly calculated without trailer-mounted radar
New splicing seams new_seam outiIf color_diffiLess than or equal to threshold value color_diff0, then by last_seami
It is assigned to new_seami。
3. vehicle-mounted 360 degree of one kind according to claim 1 or 2 are looked around display methods, which is characterized in that further include step
S4: track of vehicle is calculated using automobile gear level, steering wheel angle λ, spliced map central area is overlooked in the vehicle of generation and adds vehicle
Top view plane model draws track of vehicle, track of vehicle of painting in forward sight or backsight image.
4. vehicle-mounted 360 degree of one kind according to claim 1 or 2 are looked around display methods, which is characterized in that the step S31
The splicing seams last_seam of middle previous frameiOn each pixel value difference color_diffiIt is calculated using following formula:
Wherein, last_seamiFor i-th splicing seams；
color_diffiFor splicing seams last_seamiThe average value of the value differences of upper adjacent image；
U is splicing seams last_seamiOn a point；
IiIt (u) is point uiPixel value on the image of camera i；
IjIt (u) is point ujPixel value on the image of camera j；
NiFor splicing seams last_seamiThe number of upper pixel.
5. vehicle-mounted 360 degree of one kind according to claim 1 or 2 are looked around display methods, which is characterized in that the step S33
Splicing seams are spliced using following formula:
F (x, y)=Wl (x, y) Fl (x, y)+Wr (x, y) Fr (x, y)；
Wherein, Fl (x, y), Fr (x, y) are the images of 2 adjacent cameras of splicing；
X, y are the direction the x and y coordinates an of point, and it is positive direction that coordinate origin is the image upper left corner downwards to the right；
Wl, Wr are the weighting function of 2 adjacent cameras of splicing, are determined by splicing seams, using splicing seams as center line, form one
Width is the overlapping region of T, in this region weighted value by overlapping region one side edge each point along splicing seams normal direction
It is 0 from 1 monotone decreasing to another side edge.
6. vehicle-mounted 360 degree of one kind according to claim 3 are looked around display methods, which is characterized in that sharp in the step S4
Track of vehicle is calculated with automobile gear level, steering wheel angle λ, spliced map central area addition vehicle is overlooked in the vehicle of generation and overlooks
Plan model, draw track of vehicle method the following steps are included:
Calculate front left wheel corner B, front right wheel corner A, the vehicle turning center ICR of vehicle, coordinate origin O in rear shaft center's point,
Coordinate is backward, to the right positive direction；
ICR=(RICR, 0)；
Wherein, RICRIt is the turning radius of back axle central point；
L is distance between axles；
D is wheelbase；
λ is steering wheel angle, is clockwise positive direction with right ahead for 0；
The movement rail of left and right wheels can be obtained by vehicle turning center ICR, front left wheel corner B, front right wheel corner A and automobile gear level
Mark, if vehicle is in forward, using vehicle turning center ICR as the center of circle, using left and right vehicle wheel front-wheel as starting point, to before vehicle
The arc that length is N is done into direction；If vehicle, which is in, retreats gear, using vehicle turning center ICR as the center of circle, with left and right vehicle wheel rear-wheel
For starting point, the arc that length is N is done to vehicle rollback direction；If vehicle is in other gears, motion profile is not shown.
7. vehicle-mounted 360 degree of one kind according to claim 6 are looked around display methods, which is characterized in that in the step S4
Paint in forward sight or backsight image track of vehicle method the following steps are included: if vehicle is in forward, will be in top view
Trajectory line be divided into M-1 parts, take M endpoint, obtain that this M point is corresponding in forward sight image to be sat by the Map Searching table of forward sight
Then mark draws the track being made of this M point in forward sight image；If vehicle, which is in, retreats gear, by the rail in top view
Trace is divided into M-1 parts, takes M endpoint, obtains this M point corresponding seat in backsight image by the Map Searching table of backsight
Then mark draws the track being made of this M point in backsight image.
8. a kind of vehicle-mounted 360 degree are looked around display system, vehicle-mounted using one kind described in any claim in claim 1-7
360 degree are looked around display methods characterized by comprising
Four cameras (1), are separately positioned on automobile front side, rear side, left side, right side；
Image mosaic module (2) forms vehicle and bows according to camera acquired image, trailer-mounted radar signal, door switch signal
Depending on spliced map, it is output to post processing of image module；
Track of vehicle computing module (3) calculates track of vehicle according to automobile gear level, steering wheel angle λ, is output to post processing of image
Module；
Post processing of image module (4), the Data Integration that image mosaic module and track of vehicle computing module are exported, in generation
Vehicle overlooks spliced map central area addition vehicle areal model, draws track of vehicle, and is output to vehicle-carrying display screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710180080.7A CN107066954B (en) | 2017-03-23 | 2017-03-23 | A kind of vehicle-mounted 360 degree are looked around display methods and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710180080.7A CN107066954B (en) | 2017-03-23 | 2017-03-23 | A kind of vehicle-mounted 360 degree are looked around display methods and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107066954A CN107066954A (en) | 2017-08-18 |
| CN107066954B true CN107066954B (en) | 2019-07-23 |
Family
ID=59621080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710180080.7A Active CN107066954B (en) | 2017-03-23 | 2017-03-23 | A kind of vehicle-mounted 360 degree are looked around display methods and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107066954B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109525790B (en) * | 2017-09-20 | 2021-06-25 | 杭州海康威视数字技术股份有限公司 | Video file generation method and system, and playing method and device |
| CN109552173A (en) * | 2017-09-26 | 2019-04-02 | 纵目科技（上海）股份有限公司 | Full visual field camera engine control system |
| CN109552174A (en) * | 2017-09-26 | 2019-04-02 | 纵目科技（上海）股份有限公司 | Full visual field camera master machine control unit |
| CN107888894B (en) * | 2017-10-12 | 2019-11-05 | 浙江零跑科技有限公司 | A kind of solid is vehicle-mounted to look around method, system and vehicle-mounted control device |
| CN107945227B (en) * | 2017-11-21 | 2021-10-01 | 北京工业大学 | Method for generating radar display area in infrared panoramic monitoring |
| CN108909625B (en) * | 2018-06-22 | 2021-09-17 | 河海大学常州校区 | Vehicle bottom ground display method based on panoramic all-round viewing system |
| US10864860B2 (en) * | 2018-07-24 | 2020-12-15 | Black Sesame International Holding Limited | Model-based method for 360 degree surround view using cameras and radars mounted around a vehicle |
| CN111064900A (en) * | 2019-12-25 | 2020-04-24 | 宜宾凯翼汽车有限公司 | Self-adaptive white balance method and vehicle-mounted panoramic looking-around system |
| CN114274948A (en) * | 2021-12-15 | 2022-04-05 | 武汉光庭信息技术股份有限公司 | Automatic parking method and device based on 360-degree panorama |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377372A (en) * | 2012-04-23 | 2013-10-30 | 无锡维森智能传感技术有限公司 | Looking-around composite graph overlapping region dividing method and looking-around composite graph representing method |
| CN104392416A (en) * | 2014-11-21 | 2015-03-04 | 中国电子科技集团公司第二十八研究所 | Video stitching method for sports scene |
| CN105539290A (en) * | 2015-12-24 | 2016-05-04 | 科世达（上海）管理有限公司 | System and method for displaying 3D panorama image of vehicle |
| CN105844225A (en) * | 2016-03-18 | 2016-08-10 | 乐卡汽车智能科技（北京）有限公司 | Method and device for processing image based on vehicle |
| CN106469444A (en) * | 2016-09-20 | 2017-03-01 | 天津大学 | Eliminate the rapid image fusion method in splicing gap |
-
2017
- 2017-03-23 CN CN201710180080.7A patent/CN107066954B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377372A (en) * | 2012-04-23 | 2013-10-30 | 无锡维森智能传感技术有限公司 | Looking-around composite graph overlapping region dividing method and looking-around composite graph representing method |
| CN104392416A (en) * | 2014-11-21 | 2015-03-04 | 中国电子科技集团公司第二十八研究所 | Video stitching method for sports scene |
| CN105539290A (en) * | 2015-12-24 | 2016-05-04 | 科世达（上海）管理有限公司 | System and method for displaying 3D panorama image of vehicle |
| CN105844225A (en) * | 2016-03-18 | 2016-08-10 | 乐卡汽车智能科技（北京）有限公司 | Method and device for processing image based on vehicle |
| CN106469444A (en) * | 2016-09-20 | 2017-03-01 | 天津大学 | Eliminate the rapid image fusion method in splicing gap |
Non-Patent Citations (1)
| Title |
|---|
| 全景视图泊车辅助系统中的多视点视频拼接;陈浩;《万方数据知识服务平台》;20160831;1-60 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107066954A (en) | 2017-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107066954B (en) | A kind of vehicle-mounted 360 degree are looked around display methods and system | |
| CN106573577B (en) | Display system and method | |
| Hamada et al. | Surround view based parking lot detection and tracking | |
| CN104851076B (en) | Panoramic looking-around parking assisting system and camera installation method for commercial car | |
| CN105678787A (en) | Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera | |
| JP3645196B2 (en) | Image synthesizer | |
| US20200082185A1 (en) | Periphery monitoring device | |
| CN104185009B (en) | enhanced top-down view generation in a front curb viewing system | |
| US6911997B1 (en) | Monitoring system, camera adjusting method and vehicle monitoring system | |
| US20150130942A1 (en) | Image processing device | |
| CN101442618A (en) | Method for synthesizing 360 DEG ring-shaped video of vehicle assistant drive | |
| CN109624851B (en) | Augmented reality-based driving assistance method and system and readable storage medium | |
| Yu et al. | 360 surround view system with parking guidance | |
| CN107627959A (en) | The panoramic video monitoring method and system of motor vehicle | |
| CN109948398A (en) | The image processing method and panorama parking apparatus that panorama is parked | |
| GB2541906A (en) | Display system and method | |
| CN110363085B (en) | Method for realizing looking around of heavy articulated vehicle based on articulation angle compensation | |
| CN106780344A (en) | 360 ° of the automobile for being shifted based on deviation and being split looks around image split-joint method | |
| US20150098622A1 (en) | Image processing method and system of around view monitoring system | |
| US20220086400A1 (en) | Vehicular display system | |
| JP2007102798A (en) | Vehicle circumference monitoring system | |
| JP2019057800A (en) | Display control device | |
| JP2004147083A (en) | Driving support apparatus | |
| CN111626227B (en) | Implementation method of binocular vision-based vehicle bottom perspective panoramic system | |
| EP2487651A1 (en) | Method for operating a camera system in a motor vehicle, camera system and motor vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200910 Address after: Room 301, building 3, No. 2930, South Ring Road, Puyan street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Dahua Automobile Technology Co.,Ltd. Address before: 310051, Hangzhou, Zhejiang, Binjiang District, 6 street, 451 building, building Patentee before: ZHEJIANG LEAPMOTOR TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 310000 Room 301, building 3, no.2930, South Ring Road, Puyan street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang huaruijie Technology Co.,Ltd. Address before: 310000 Room 301, building 3, no.2930, South Ring Road, Puyan street, Binjiang District, Hangzhou City, Zhejiang Province Patentee before: Zhejiang Dahua Automobile Technology Co.,Ltd. |