CN103377372B - One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing - Google Patents
One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing Download PDFInfo
- Publication number
- CN103377372B CN103377372B CN201210119219.4A CN201210119219A CN103377372B CN 103377372 B CN103377372 B CN 103377372B CN 201210119219 A CN201210119219 A CN 201210119219A CN 103377372 B CN103377372 B CN 103377372B
- Authority
- CN
- China
- Prior art keywords
- pixel
- camera
- overlapping region
- composite diagram
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The present invention is to solve the problems, such as that the lap in composite diagram present in vehicle-mounted viewing system in each camera field of view can not reach most readability, there is provided the subregional division methods in overlapping portion in a kind of composite diagram of vehicle-mounted multi-view camera viewing system in each camera field of view.According to technical scheme provided by the invention, the pixel of all adjacent camera overlapping regions looked around in composite diagram is compared to determine by the pixel back projection to the coordinates of original image coordinates of every camera, embodiment can use coordinates of original image coordinates to be compared apart from the distance of optical axis center, or by pixel space density.The right of eminent domain of that pixel of overlapping region is obtained apart from that camera that optical axis is near or that pixel space density is small.The lap optimal dividing method in composite diagram in each camera field of view is can be obtained by by the above-mentioned processing of pixel progress to all adjacent camera overlapping regions.
Description
Technical field
The present invention relates to vehicle-mounted multi-view camera viewing system, especially a kind of conjunction of vehicle-mounted multi-view camera viewing system
Into the optimal dividing method and image combining method of image overlapping region.
Background technology
Today's society, automobile turn into a kind of essential vehicles.People enjoy automobile belt convenience and
While quick, motor traffic accidents, the problem of environmental pollution and traffic jam that motor vehicle exhaust emission is brought are brought, into
For increasingly serious global social concern.Therefore, traffic is reduced using various advanced automotive safety technologies, equipment and theory
Accident and raising automotive safety have very big market potential.
After the last century nineties, as electronic technology, control technology, sensor technology and new material are in automobile product
In extensive use, automotive safety technology obtained swifter and more violent development.Nowadays, the research of automotive safety technology is by list
The research and development of one safe practice, blend the integrated of collaboration, systematization and intelligent direction to various safe practices and develop.Intelligence
The automobile safety system of change is using modern Detection Techniques, photoelectric sense technology, computer technology and automatic control technology as core, tool
There is specific identification judgement, driver can be assisted automatically under various complex situations or voluntarily control automobile, it is ensured that driving
Safety.
Vehicle environment sensory perceptual system is entered using various sensors to information such as vehicle itself, surrounding environment and driver status
Row detection, by compared with standard set in advance, differentiate whether vehicle in the hole and degree of danger, if necessary
Early warning can be carried out to driver by modes such as sound, light.
The sensor main that vehicle environment sensory perceptual system uses at present will have：1）Monocular or multi-lens camera system, by right
The ambient image that collection comes in real time is handled to obtain the information such as correlation distance, position；2）Laser radar or millimetre-wave radar, lead to
Cross and send and receive infrared laser or electric wave, according to Doppler effect, calculate the information such as distance and the position of periphery barrier；
3）Sonar, sent by orientation and receive ultrasonic wave, calculate the information such as distance and the position of periphery barrier.
Comparatively, laser or millimetre-wave radar can be surveyed, scope is wide, and the ability of anti-extraneous adverse circumstances is strong, but generally only
One or more layers scanning plane, the 3 D stereo information of whole scene can not be obtained, and it is expensive.Sonar is only applicable to low coverage
Measurement（For example reversing is used）, and some information only on direction of illumination.Vehicle-mounted camera system obtains information by vision,
It is one of most widely used at present and most promising sensor.For example backup camera is vehicle-mounted using most common one kind
Vision system.It is typically mounted at vehicle rear, and direction is towards side-lower.When driver moves backward, system turn on automatically is vehicle-mounted aobvious
Display screen, rear wide area scene is provided for driver.
In order to allow driver more accurately to understand the dangerous information such as barrier of vehicle-surroundings, the conversion that system passes through visual angle
Original image with certain angle of depression is transformed to virtual video camera image vertically downward by processing.
Backup camera is used for more complicated reversing environment, but the limited view of a video camera, in vehicle front
In the case that side barrier is more, generally use multiple cameras forms vehicle viewing system, and vehicle 360 is provided for driver
Spend complete environment sensing information.
Vehicle viewing system is mainly the image shot by the multiple cameras installed in vehicle all around, before
The view transformation stated handles to obtain multiple virtually top views centered on vehicle vertically downward, and being spliced into by composition algorithm
One completed top view.
Vehicle-mounted multi-view camera viewing system described in this patent refers to the multi-eye stereo camera shooting machine system that vehicle environment perceives.
（1）Prior art one related to the present invention：Viewpoint change
Prior art one can utilize the inner parameter for being assemblied in 4 cameras of car body all around with external parameter to each
The image absorbed of platform camera, which does single viewpoint change, makes it be for conversion into using car body as the top view with reference to center, Ran Houzai
Splicing one width of generation is done by the seamless spliced top view of 4 camera images by the top view obtained by each camera to this 4 width.
The premise of prior art one is that the inner parameter of camera is known with external parameter, car body parked according to commitment positions or
The posture of person's car body is known.
Patent document 1：Japan Patent JP3625059；
Patent document 2：Japan Patent JPT2002-067158.
（2）Prior art two related to the present invention：The panoramic view picture synthesis of vertical view
It is same as above, after the video camera of vehicle all around passes through viewpoint change, it is virtual that multiple are formed respectively
The overhead view image vertically shot.In initial alignment processing procedure, each video camera can be with relative to the position of vehicle axis system
Obtain.According to the station-keeping data of video camera, these overhead view images can be subjected to synthesis processing, be spliced into a width vehicle ring
Depending on panorama sketch.In the case of the height and focal length identical of each virtual video camera, the dimension scale phase of these overhead view images
Together, splicing can directly be carried out.The schematic top plan view from car is added on spliced circular top view, you can obtain most
Result figure afterwards.
Patent document 3：Japan Patent JPA2002-067158；
Patent document 4：Japan Patent JPA2008-34966.
（3）Prior art three related to the present invention：The solution of virtual view composograph
Although the vehicles of multiple views can be generated in prior art two around top view, such as in slow astern state or
During person's normally travel state, in addition to confirming from car physical location in the scene, it is also necessary to confirm nearby vehicle and its
His barrier.If required effect is only unable to reach by the panoramic view of the virtual view vertically shot.
Prior art three（Japan Patent JP_P3300334）It is proposed to regard by using different points of view, height, the virtual of focal length
Put and including areal model, the model of place such as cylindrical model or simulation cylindrical model, according to vehicle-state, speed, direction turns
Angle generates different virtual ring visible images.Can so reach at the same meet confirm from car state and surrounding enviroment the needs of.
The look-up method corresponding to corresponding different virtual visual point images is also proposed in prior art three simultaneously.
Patent document 5：Japan Patent JP_P3300334.
（4）The shortcomings that prior art
The viewpoint change image of multiple views can be spliced by prior art one and prior art two, obtain ring
Around top plan view；Prior art three can be simulated except plane by the model of place such as cylindrical model or simulation cylindrical model
Stereo scene in addition.Whether top plan view or stereo scene figure are all by being arranged on vehicle diverse location by several
The image of video camera shooting carries out splicing synthesis.Generally, vehicle-mounted processing system of looking around is by more wide angle cameras
Form, every video camera covers certain section, and image is spliced to obtain vehicle and looked around by viewpoint change generation top view
Figure.Due to there may be ken lap between each camera, selection one is just needed in the splicing generating process of panoramic view
Bar line of demarcation is split by lap.As shown in figure 3, composite diagram is divided into the part of several Hu Bu UNICOMs by cut-off rule, often
Effective ken of the corresponding camera of a part.Because how prior art is without reference to selecting the optimal line of demarcation can be with
So that final composite diagram reaches most sharpening, final composite diagram has fuzzy situation in ken lap and occurred.
The content of the invention
The technical scheme is as follows：
The present invention is to solve the lap in composite diagram present in vehicle-mounted viewing system in each camera field of view
A kind of the problem of most readability can not be reached, there is provided each camera field of view in composite diagram of vehicle-mounted multi-view camera viewing system
In lap line of demarcation computational methods.The present invention can provide for driver most clearly looks around composite diagram.
According to technical scheme provided by the invention, the pixels of all adjacent camera overlapping regions looked around in composite diagram by
The pixel back projection is compared to determine to the coordinates of original image coordinates of every camera, and embodiment can use original
Image coordinate or is compared by pixel space density apart from the distance of optical axis center.Apart from optical axis it is near or picture
That small camera of plain control density obtains the right of eminent domain of that pixel of overlapping region.
It can be obtained by by the above-mentioned processing of pixel progress to all adjacent camera overlapping regions each in composite diagram
Lap optimal dividing method in camera field of view.
It is an advantage of the invention that：The present invention is solved in composite diagram present in vehicle-mounted viewing system in each camera field of view
Lap the problem of can not reaching most readability.
Brief description of the drawings
Fig. 1：It is vehicle-mounted to look around camera position and FOV
Fig. 2：Composite diagram subregion
Fig. 3：Composite diagram subregion and cut-off rule
Fig. 4：Recombination region cut-off rule determines method
Fig. 5：Pixel space density algorithm.
Embodiment
The invention will be further described with reference to the accompanying drawings and examples.Vehicle-mounted processing system of looking around is by more wide-angles
Image mechanism is into every video camera covers certain section, and image is spliced to obtain car by viewpoint change generation top view
Panoramic view.Due to there may be ken lap between each camera, just needed in the splicing generating process of panoramic view
One line of demarcation of selection is split by lap.Composite diagram is divided into the part of several Hu Bu UNICOMs, each portion by cut-off rule
Divide effective ken of a corresponding camera.Because how prior art is without reference to selecting the optimal line of demarcation can to cause
Final composite diagram reaches most sharpening, and final composite diagram has fuzzy situation in ken lap and occurred.
The present invention proposes the vehicle-mounted computational methods for looking around ken lap line of demarcation in composite diagram.
As shown in figure 1, vehicle-mounted viewing system all around respectively sets a camera Cam in carF, CamR, CamB,
CamL, each camera has the FOV VF of oneselfF, VFR, VFB, VFL.The FOV VF of preceding cameraFWith right shooting
The FOV VF of headROverlap region VFRF, the FOV VF of preceding cameraFWith the FOV VF of left cameraL
Overlap region VFLF, the FOV VF of rear cameraBWith the FOV VF of right cameraROverlap region
VFRB, the FOV VF of rear cameraBWith the FOV VF of left cameraLOverlap region VFLB.Viewing system base
In video camera intrinsic parameter and outer parameter using accurate viewpoint change technology by the image co-registration obtained by 4 cameras into one
The full cyclogram of width seamless connection, as shown in Figure 2.
Composite diagram Fig. 2 is divided into 8 different regions：IMGF, IMGR, IMGB, IMGL, IMGRF, IMGRB,
IMGLF, IMGLB.Wherein IMGF, IMGR, IMGB, IMGLIt is isolated area：IMGFData entirely from preceding camera
CamF,IMGRData entirely from right camera CamR, IMGBData entirely from rear camera CamB, IMGL's
Data are entirely from left camera CamL。IMGRF, IMGRB, IMGLF, IMGLBIt is recombination region, its data can be taken the photograph from two
Obtained in the image as obtained by head：IMGRFData can be from preceding camera CamFOr right camera CamRObtain, IMGRB's
Data can be from rear camera CamBOr right camera CamRObtain, IMGLFData can be from preceding camera CamFOr
Left camera CamLObtain, IMGLBData can be from rear camera CamBOr left camera CamLObtain.
In recombination region IMGRF, IMGRB, IMGLF, IMGLBIt is interior it needs to be determined that a cut-off rule BRF, BRB, BLF, BLB
（As shown in Figure 3）So as to according to this different camera image data of boundary line options come filled composite region.Such as hard
Split under situation, recombination region IMGRFMiddle cut-off rule BRFArea above preceding camera CamFCorresponding data filling, cut-off rule
BRFThe right camera Cam of region belowRCorresponding data is filled；Or in the case where data integrate situation, recombination region IMGRFIn certain picture
The numerical value of element is by preceding camera CamFCorresponding data in right camera CamRCorresponding data weighting synthesis, its weight proportion by
Apart from cut-off rule BRFFar and near determination.
Therefore how defining boundaries can have an immense impact on to the imaging effect of recombination region.If line of demarcation is chosen not
Distinguish when causing composite diagram to obscure difficulty and will cause great interference to driver.
The present invention proposes a kind of new line of demarcation computational methods, to ensure that composite diagram recombination region part is most clear.Such as Fig. 4
It is shown, with recombination region IMGRFExemplified by, its interior pixel pRFPreceding camera image is corresponded to by projection and coordinate transform respectively
IMGCamFPFPixel and right camera image IMGCamRPRPixel.pFAnd pRIt is at war with, the p that the person of winning is used asRF
Value.In the case where each camera internal parameter is equal, its competition process is：Compare pFDistance IMGCamFThe distance d of origin of optical axisF
With pRDistance IMGCamRThe distance d of origin of optical axisR, won apart from shorter one.In the case of if each camera internal parameter is unequal
Its competition process is：Compare IMGCamFMiddle pFThe pixel space density and IMG at placeCamRMiddle pRThe pixel space density at place, density compared with
Small person wins.
By above-mentioned competition process can in recombination region IMGRF, IMGRB, IMGLF, IMGLBIt is interior it needs to be determined that one
Cut-off rule BRF, BRB, BLF, BLB（As shown in Figure 4）, so as to ensure recombination region IMGRF, IMGRB, IMGLF, IMGLBFigure
As most clear, that is, the information forgiven is most.
As described above, its competition process is in the case of if each camera internal parameter is unequal：Compare IMGCamFMiddle pFPlace
Pixel space density and IMGCamRMiddle pRThe pixel space density at place, density smaller win.Presently describe a kind of calculating pixel
The method of space density：
As shown in Figure 5, recombination region IMGRFExemplified by, its interior pixel pRFCorresponded to respectively by projection and coordinate transform
Preceding camera image IMGCamFPFPixel and right camera image IMGCamRPRPixel.Choose pFAdjacent pixels
pF’（P is selected in Fig. 5FLower adjacent pixel as signal）, can be in IMG by projective transformationCamFIn obtain its corresponding picture
Plain position pRF’, pRF withpRF’Distance can be used as pFThe pixel space density value at place.Space density value is smaller more winning.
Claims (4)
1. overlapping region divides in a kind of composite diagram of viewing system calculating and generation method, it is characterised in that methods described
Including：
The pixel of overlapping region in the composite diagram of vehicle-mounted viewing system, obtained respectively by projection and coordinate transform corresponding
The coordinate of pixel or the space density of coordinate points, the pixel respectively obtained are at war with the original image of adjacent camera,
Competitive relation is：
When each camera inner parameter is identical, the optical axis of the pixel and each self-corresponding adjacent camera respectively obtained
The distance at center, wins apart from shorter one, and now corresponding camera obtains the right of eminent domain of overlapping region pixel；
When each camera inner parameter differs, the pixel space density of the coordinate points for the pixel respectively obtained, as
Plain space density smaller wins, and now corresponding camera obtains the right of eminent domain of overlapping region pixel.
2. overlapping region divides in the composite diagram of viewing system according to claim 1 calculating and generation method, it is special
Sign is, is defined boundaries in the overlapping region of the composite diagram of vehicle-mounted viewing system, and line of demarcation is by overlapping region complete parttion, root
The different camera image data of line options of demarcating accordingly are filled to overlapping region.
3. overlapping region divides in the composite diagram of viewing system according to claim 2 calculating and generation method, it is special
Sign is, is defined boundaries in the overlapping region of the composite diagram of vehicle-mounted viewing system, line of demarcation by overlapping region complete parttion,
In the case that data rub up, the pixel of overlapping region is synthesized by adjacent each camera weighting, and weight proportion is demarcated by distance
The far and near determination of line.
4. overlapping region divides in the composite diagram of viewing system according to claim 1 calculating and generation method, it is special
Sign is, one of pixel of overlapping region in the composite diagram of vehicle-mounted viewing system, passes through projection and coordinate transform point
Two pixel coordinates in the original image of two adjacent cameras are not obtained；
Wherein, the pixel coordinate that two pixel coordinates obtained are adjacent respectively is each being corresponded to by projection and coordinate transform
Original image in obtain corresponding location of pixels, the distance of the pixel of two location of pixels and overlapping region is as obtaining
Two pixel coordinates pixel space density value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210119219.4A CN103377372B (en) | 2012-04-23 | 2012-04-23 | One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210119219.4A CN103377372B (en) | 2012-04-23 | 2012-04-23 | One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103377372A CN103377372A (en) | 2013-10-30 |
| CN103377372B true CN103377372B (en) | 2017-12-22 |
Family
ID=49462456
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210119219.4A Expired - Fee Related CN103377372B (en) | 2012-04-23 | 2012-04-23 | One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103377372B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103985118A (en) * | 2014-04-28 | 2014-08-13 | 无锡观智视觉科技有限公司 | Parameter calibration method for cameras of vehicle-mounted all-round view system |
| CN105844225B (en) * | 2016-03-18 | 2019-06-18 | 法法汽车(中国)有限公司 | A kind of image processing method and device based on vehicle |
| CN107066954B (en) * | 2017-03-23 | 2019-07-23 | 浙江零跑科技有限公司 | A kind of vehicle-mounted 360 degree are looked around display methods and system |
| JP2019074850A (en) * | 2017-10-13 | 2019-05-16 | 株式会社デンソー | Image processing apparatus |
| DE102018201316A1 (en) * | 2018-01-29 | 2019-08-01 | Conti Temic Microelectronic Gmbh | Surroundview system for one vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1168248A2 (en) * | 2000-06-30 | 2002-01-02 | Matsushita Electric Industrial Co., Ltd. | Rendering device |
| CN101179644A (en) * | 2006-11-06 | 2008-05-14 | 索尼株式会社 | Image processing apparatus, camera apparatus, image processing method, and program |
| CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
| CN102142138A (en) * | 2011-03-23 | 2011-08-03 | 深圳市汉华安道科技有限责任公司 | Image processing method and subsystem in vehicle assisted system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4193886B2 (en) * | 2006-07-26 | 2008-12-10 | トヨタ自動車株式会社 | Image display device |
-
2012
- 2012-04-23 CN CN201210119219.4A patent/CN103377372B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1168248A2 (en) * | 2000-06-30 | 2002-01-02 | Matsushita Electric Industrial Co., Ltd. | Rendering device |
| CN101179644A (en) * | 2006-11-06 | 2008-05-14 | 索尼株式会社 | Image processing apparatus, camera apparatus, image processing method, and program |
| CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
| CN102142138A (en) * | 2011-03-23 | 2011-08-03 | 深圳市汉华安道科技有限责任公司 | Image processing method and subsystem in vehicle assisted system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103377372A (en) | 2013-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10992860B2 (en) | Dynamic seam adjustment of image overlap zones from multi-camera source images | |
| EP3554062B1 (en) | Imaging apparatus | |
| KR102493878B1 (en) | controlling host vehicle based on detected spacing between stationary vehicles | |
| KR102344171B1 (en) | Image generating apparatus, image generating method, and program | |
| JP6806156B2 (en) | Peripheral monitoring device | |
| CN108621948A (en) | Vehicle panoramic viewing system and panoramic looking-around image generating method | |
| CA3069114C (en) | Parking assistance method and parking assistance device | |
| JP2021170826A (en) | Method and apparatus for displaying peripheral scene of combination of vehicle and tracked vehicle | |
| US8446268B2 (en) | System for displaying views of vehicle and its surroundings | |
| CN101763640B (en) | Online calibration processing method for vehicle-mounted multi-view camera viewing system | |
| CN103802725B (en) | A kind of new vehicle carried driving assistant images generation method | |
| CN103377372B (en) | One kind looks around composite diagram overlapping region division methods and looks around composite diagram method for expressing | |
| CN202035096U (en) | Mobile operation monitoring system for mobile machine | |
| CN110786004B (en) | Display control device, display control method, and storage medium | |
| JP6007848B2 (en) | Visual confirmation evaluation apparatus, method and program | |
| JP2018531530A5 (en) | ||
| CN115244427A (en) | Simulated laser radar apparatus and system | |
| CN107027329A (en) | The topography of the surrounding environment of traveling instrument is spliced into an image | |
| JP2012086684A (en) | Parking assist device | |
| JP6413477B2 (en) | Image display control device and image display system | |
| JP2005269010A (en) | Image creating device, program and method | |
| CN208715087U (en) | Vehicle panoramic viewing system | |
| JP6781035B2 (en) | Imaging equipment, image processing equipment, display systems, and vehicles | |
| EP3544293B1 (en) | Image processing device, imaging device, and display system | |
| JP7136123B2 (en) | Image processing apparatus, image processing method, program, and information processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 214135 Qingyuan Road, Wuxi New District, Jiangsu, China Science and Technology Park, China Science and Technology Park, A510 Patentee after: Wuxi wisdom Sensor Technology Co., Ltd. Address before: 214135 Qingyuan Road, Wuxi New District, Jiangsu, China Science and Technology Park, China Science and Technology Park, A510 Patentee before: Wuxi Wissen Intelligent Sensing Technology Co., Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20171222 Termination date: 20200423 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |