CN109754363A - Image composition method and device are looked around based on fisheye camera - Google Patents
Image composition method and device are looked around based on fisheye camera Download PDFInfo
- Publication number
- CN109754363A CN109754363A CN201811598527.3A CN201811598527A CN109754363A CN 109754363 A CN109754363 A CN 109754363A CN 201811598527 A CN201811598527 A CN 201811598527A CN 109754363 A CN109754363 A CN 109754363A
- Authority
- CN
- China
- Prior art keywords
- image
- eye images
- fish eye
- fisheye camera
- solid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The present invention, which provides, a kind of looks around image composition method and device based on fisheye camera, this method comprises: obtaining the multiple fish eye images for the multiple fisheye cameras acquisition being arranged on vehicle, according to the corresponding relationship between the multiple fisheye cameras obtained in advance, multiple fish eye images are spliced, obtain the corresponding planar rings visible image of multiple fish eye images, corresponding relationship is used to indicate the positional relationship between the fish eye images of multiple fisheye camera acquisitions, spatial alignment processing is carried out to space object identical in multiple fish eye images, it obtains the corresponding solid of planar rings visible image and looks around image.Blind area when in this way, can obtain vehicle's surroundings outdoor scene looks around image, improving user experience, while eliminating vehicle driving, improves driving safety.
Description
Technical field
The present invention relates to technical field of image processing more particularly to a kind of image composition method is looked around based on fisheye camera
And device.
Background technique
Running car especially in the process of backing up, for driving safety, can clearly illustrate the outdoor scene of vehicle periphery
Image information is most important.
In the prior art, the camera being arranged on vehicle mostly be work independently, these cameras acquisition image usually not into
Row splicing is returned to user, and user experience is bad；In addition, though some high-end vehicles, which have, looks around function, mainly in vehicle
Wide-angle lens is provided on, but since wide-angle lens visual angle is narrow, and what is returned is typically all plan view, user's body
It tests poor.
Summary of the invention
To solve problems of the prior art, the present invention, which provides, a kind of looks around image synthesis side based on fisheye camera
Method and device look around image be supplied to user's vehicle's surroundings outdoor scene, improve user experience, while eliminating vehicle driving
When blind area, improve driving safety.
In a first aspect, the embodiment of the present invention provides and a kind of looks around image composition method based on fisheye camera, comprising:
Obtain the multiple fish eye images for the multiple fisheye cameras acquisition being arranged on vehicle；
According to the corresponding relationship between the multiple fisheye camera obtained in advance, the multiple fish eye images are spelled
It connects, obtains the corresponding planar rings visible image of the multiple fish eye images, the corresponding relationship is for indicating the multiple flake phase
Positional relationship between the fish eye images of machine acquisition；
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the plane panoramic view
As corresponding solid looks around image.
Optionally, the method also includes:
The planar rings visible image is optimized, the planar rings visible image after obtaining optimization.
Optionally, the method also includes:
Multiple scaling board images of the multiple fisheye camera acquisition are obtained, the fish eye images include the scaling board figure
Picture；
According to the multiple scaling board image, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, described that the corresponding relationship of the multiple fisheye camera is obtained according to the multiple scaling board image, packet
It includes:
According to feature point detection algorithm, the characteristic point of the multiple scaling board image is obtained；
According to the characteristic point of the multiple scaling board image, the position coordinates of the characteristic point are obtained；
According to the position coordinates of the characteristic point, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, described that spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtain institute
It states the corresponding solid of planar rings visible image and looks around image, comprising:
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the first three-dimensional panoramic view
Picture；
Image is looked around to first solid and carries out color mixing processing, the second solid is obtained and looks around image；
Image is looked around according to second solid, and the threedimensional model pre-established, obtains the planar rings visible image
Corresponding solid looks around image, and it is that the second solid looks around image in the threedimensional model pre-established that the solid, which looks around image,
On projection.
Optionally, the method also includes:
Show that the corresponding solid of the planar rings visible image looks around image.
Optionally, the quantity of the fisheye camera is four.
Second aspect, the embodiment of the present invention, which provides, a kind of looks around image synthesizer based on fisheye camera, comprising:
Module is obtained, for obtaining multiple fish eye images of the multiple fisheye cameras being arranged on vehicle acquisition；
Processing module, the corresponding relationship between the multiple fisheye camera obtained in advance for basis, to the multiple
Fish eye images are spliced, and obtain the corresponding planar rings visible image of the multiple fish eye images, the corresponding relationship is for indicating
Positional relationship between the fish eye images of the multiple fisheye camera acquisition；
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the plane panoramic view
As corresponding solid looks around image.
Optionally, the processing module is also used to optimize the planar rings visible image, the plane after obtaining optimization
Look around image.
Optionally, the acquisition module is also used to obtain multiple scaling board images of the multiple fisheye camera acquisition, institute
Stating fish eye images includes the scaling board image；
According to the multiple scaling board image, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, the acquisition module is also used to obtain the multiple scaling board image according to feature point detection algorithm
Characteristic point；
According to the characteristic point of the multiple scaling board image, the position coordinates of the characteristic point are obtained；
According to the position coordinates of the characteristic point, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, the processing module is also used to carry out space to space object identical in the multiple fish eye images
Registration process obtains the first solid and looks around image；
Image is looked around to first solid and carries out color mixing processing, the second solid is obtained and looks around image；
The acquisition module is also used to look around image according to second solid, and the threedimensional model pre-established, obtains
The corresponding solid of the planar rings visible image is taken to look around image, it is that the second solid looks around image described that the solid, which looks around image,
The projection on threedimensional model pre-established.
Optionally, described device further include:
Display module, for showing that the corresponding solid of the planar rings visible image looks around image.
Optionally, the quantity of the fisheye camera is four.
The third aspect, the embodiment of the present invention provide a kind of mobile unit, comprising:
Processor；
Memory；And
Computer program；
Wherein, the computer program is stored in the memory, and is configured as being executed by the processor,
The computer program includes the instruction for executing method described in first aspect.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, the computer-readable storage medium
Matter is stored with computer program, and the computer program makes server execute method described in first aspect.
It is provided by the invention that image composition method and device are looked around based on fisheye camera, this method comprises: obtaining setting
Multiple fish eye images of multiple fisheye cameras acquisition on vehicle, according to the correspondence between the multiple fisheye cameras obtained in advance
Relationship splices multiple fish eye images, obtains the corresponding planar rings visible image of multiple fish eye images, and corresponding relationship is used for table
Show the positional relationship between the fish eye images of multiple fisheye camera acquisitions, space object identical in multiple fish eye images is carried out
Spatial alignment processing obtains the corresponding solid of planar rings visible image and looks around image.In this way, vehicle's surroundings reality can be obtained
Scape looks around image, improves user experience, while blind area when eliminating vehicle driving, improves driving safety.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is the flow diagram one provided in an embodiment of the present invention that look around image composition method based on fisheye camera；
Fig. 2 is the schematic diagram of fish eye images provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram that top plan view provided in an embodiment of the present invention looks around image；
Fig. 4 is the flow diagram two provided in an embodiment of the present invention that look around image composition method based on fisheye camera；
Fig. 5 is the schematic diagram of scaling board provided in an embodiment of the present invention；
Fig. 6 is the flow diagram three provided in an embodiment of the present invention that look around image composition method based on fisheye camera；
Fig. 7 is the schematic diagram of splicing fish eye images provided in an embodiment of the present invention；
Fig. 8 is the schematic diagram that solid provided in an embodiment of the present invention looks around image；
Fig. 9 is the structural schematic diagram provided in an embodiment of the present invention that look around image synthesizer based on fisheye camera；
Figure 10 is the structural schematic diagram of mobile unit provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The image composition method of looking around of fisheye camera provided in an embodiment of the present invention can be applied to need to check vehicle four
In the application scenarios of all environment, in vehicle driving or docking process.Vehicle driving or docking process, to guarantee safety, driver
Need to check whether vehicle's surroundings have barrier, the camera being arranged on current most of vehicles is to work independently mostly, these phases
The image of machine acquisition usually gives the unnatural outdoor scene experience of user to user without splicing to be returned to, and user experience is bad；
In addition, though some high-end vehicles, which have, looks around function, wide-angle lens is mainly provided on vehicle, but due to wide-angle lens
Head visual angle is narrow relatively to be fixed, and what is returned is typically all plan view, can not be supplied to user's 3 D stereo panoramic view
Picture, so that user's manual palpation rotated three dimensional tries hard to look around image to any visual angle, and the speed for returning to image is relatively slow, price
Valuableness leads to poor user experience.
The embodiment of the present invention considers the above problem, and propose a kind of fisheye camera looks around image composition method, this method
Middle mobile unit obtains the multiple fish eye images for the multiple fisheye cameras acquisition being arranged on vehicle, multiple according to what is obtained in advance
Corresponding relationship between fisheye camera splices multiple fish eye images, obtains the corresponding plane of multiple fish eye images and looks around
Image, corresponding relationship is used to indicate the positional relationship between the fish eye images of multiple fisheye camera acquisitions, to multiple fish eye images
In identical space object carry out spatial alignment processing, obtain the corresponding solid of planar rings visible image and look around image.Pass through the party
Method, can obtain vehicle's surroundings outdoor scene looks around image, improves user experience, while blind area when eliminating vehicle driving,
Improve driving safety.
Technical solution of the present invention is described in detail with specific embodiment below.These specific implementations below
Example can be combined with each other, and the same or similar concept or process may be repeated no more in some embodiments.
Fig. 1 is the flow diagram one provided in an embodiment of the present invention that look around image composition method based on fisheye camera.
The embodiment of the invention provides a kind of image composition method of looking around of fisheye camera, this method can be by arbitrarily executing fisheye camera
The device for looking around image composition method execute, which can pass through software and or hardware realization.It, should in the present embodiment
Device can integrate in mobile unit.As shown in Figure 1, fisheye camera provided in an embodiment of the present invention looks around image synthesis side
Method includes the following steps:
The multiple fish eye images for multiple fisheye cameras acquisition that S101, acquisition are arranged on vehicle.
Multiple fisheye cameras can be set on vehicle, the surrounding of vehicle can be separately positioned on, for acquiring vehicle not
Equidirectional image.The quantity of fisheye camera is not particularly limited in this programme, as long as the image of fisheye camera acquisition can close
At, and completely show the environment of vehicle's surroundings.
Optionally, the quantity of fisheye camera is 4, can be separately positioned on the headstock, the tailstock, left and right side of vehicle.
The corresponding relationship between multiple fisheye cameras that S102, basis obtain in advance, splices multiple fish eye images,
Obtain the corresponding planar rings visible image of multiple fish eye images.
The corresponding relationship is used to indicate the positional relationship between the fish eye images of multiple fisheye camera acquisitions, if fisheye camera
There is no variations for position, then the corresponding relationship is exactly fixed.
Optionally, which can be geometric transformation relationship.
For being provided with 4 fisheye cameras on vehicle, it is illustrated in conjunction with Fig. 2, Fig. 3, Fig. 2 mentions for the embodiment of the present invention
The schematic diagram of the fish eye images of confession, Fig. 3 are the schematic diagram that top plan view provided in an embodiment of the present invention looks around image.
As shown in Figure 2,4 fisheye cameras being arranged on vehicle are separately positioned on headstock, the tailstock, left side and the right side of vehicle
Side, since the maximum coverage of fisheye camera is 180 °, then the fish eye images and fisheye camera 2 that fisheye camera 1 acquires acquire
Fish eye images have an overlapping region, the fish eye images that fisheye camera 2 and fisheye camera 3 acquire have overlapping region, 3 He of fisheye camera
The fish eye images that fisheye camera 4 acquires have overlapping region, and the fish eye images that fisheye camera 4 and fisheye camera 1 acquire have overlay region
Domain can then acquire each fisheye camera according to the positional relationship between four fish eye images that four fisheye cameras acquire
Each fish eye images spliced, obtain this corresponding planar rings visible image of four fish eye images.
It should be noted that splicing here is exactly will have any two fish eye images of overlapping region to splice, when
Overlapping region is completely coincident, and splicing is completed, and is then carried out the second wheel splicing and is obtained until four fish eye images are stitched together
One complete flat image.
Further, two dimensional model is pre-established, the target that unwanted information shows user is filtered, such as: it is aerial
Object, in conjunction with the two dimensional model, available top plan view shown in Fig. 3 looks around image.
S103, spatial alignment processing is carried out to space object identical in multiple fish eye images, obtains planar rings visible image
Corresponding solid looks around image.
It is substantially aligned (splicing is completed) in the three-dimensional object looked around in the planar rings visible image overlooked in image on ground, and
For object in space in alignment, there is also deviations, therefore the set object that there is alignment deviation to space is needed to carry out space pair
Neat processing.
In one possible implementation, space pair first can be carried out to space object identical in multiple fish eye images
Neat processing obtains the first solid and looks around image, then looks around image to the first solid and carries out color mixing processing, it is vertical to obtain second
Body looks around image, looks around image further according to the second solid, and the threedimensional model pre-established, and it is corresponding to obtain planar rings visible image
Solid look around image, it is that the second solid looks around projection of the image on the threedimensional model pre-established that solid, which looks around image,.
Specifically, the processing of space redemption can according to same object geometry, appearance color in different fish eye images
Deng appearance function being calculated, which can be the change rate of the different fish eye images of a customized reflection, such as: color
Depth change rate etc., with color for example, if the shade change rate is greater than default same object shade change rate
Threshold value can stretch darker regions, or reduce to light areas, so that the same object is showing upper color
It is natural.
The first solid is obtained after spatial alignment processing and looks around image, since the incident ray of each fisheye camera is different, no
It being had differences in brightness with fish eye images, the difference in the same object color in different fish eye images may be bigger, because
This needs to carry out different fish eye images color mixing visually naturally, specifically, can calculate the same object not
With the unit brightness value on fish eye images, then averaged is restained to the object in different fish eye images again,
Image is looked around to obtain the second solid.
Threedimensional model is pre-established, the threedimensional model is for filtering the object for not needing to show user, the threedimensional model
On point and the point of stereo-picture have default corresponding relationship.
Optionally, the threedimensional model can for bowl fill network threedimensional model, this programme for threedimensional model shape not
It is limited.
Then the second obtained solid can be looked around image to project on the threedimensional model, it is corresponding to obtain planar rings visible image
Solid look around image.
Fisheye camera provided in this embodiment looks around image composition method, and mobile unit, which obtains, in this method is arranged in vehicle
Multiple fish eye images of multiple fisheye cameras acquisition on, according to the corresponding pass between the multiple fisheye cameras obtained in advance
System, splices multiple fish eye images, obtains the corresponding planar rings visible image of multiple fish eye images, corresponding relationship is for indicating
Positional relationship between the fish eye images of multiple fisheye camera acquisitions, carries out space object identical in multiple fish eye images empty
Between registration process, obtain the corresponding solid of planar rings visible image and look around image.In this way, vehicle's surroundings outdoor scene can be obtained
Look around image, blind area when improving user experience, while eliminating vehicle driving improves driving safety.
On the basis of above-mentioned Fig. 1 embodiment, Fig. 4 looks around image based on fisheye camera to be provided in an embodiment of the present invention
The flow diagram two of synthetic method, as shown in figure 4, this method is further comprising the steps of:
S201, the multiple scaling board images for obtaining multiple fisheye camera acquisitions.
Scaling board can be set on the ground of the surrounding of vehicle, and the surrounding in vehicle can be set in fisheye camera, be used for
Scaling board image is acquired, fish eye images include scaling board image.
Optionally, scaling board quantity is 4, is separately positioned on the surrounding of vehicle.
The type of scaling board is unlimited in this programme, and the calibration cloth in Fig. 5 is that two squares of black and white are (black in outer ring, Bai
Inner ring), it can also be a square (chequered with black and white).
S202, according to multiple scaling board images, obtain the corresponding relationship of multiple fisheye cameras.
According to multiple scaling board images that fisheye camera acquires, can determine including the same scaling board in different scaling boards
The position of image, can determine the positional relationship of multiple scaling board images, to obtain the corresponding relationship of multiple fisheye cameras.
In one possible implementation, according to feature point detection algorithm, the characteristic point of multiple scaling board images is obtained,
According to the characteristic point of multiple scaling board images, the position coordinates of characteristic point are obtained, according to the position coordinates of characteristic point, are obtained multiple
The corresponding relationship of fisheye camera.
Specifically, this step being illustrated in conjunction with Fig. 5, Fig. 5 is the schematic diagram of scaling board provided in an embodiment of the present invention,
As shown in figure 5, scaling board be arranged in vehicle it is left front, left back, right before, it is right after, vehicle is located at bosom, and fisheye camera is set respectively
The headstock, the tailstock, left and right side in vehicle are set, each fisheye camera can acquire at least two scaling board images.
Characteristic point in Fig. 5 can be the vertex of black squares and white square, and quantity is 8.
Then the upper left angle point A of left front scaling board is initially set as coordinate origin, and measurement obtains each scaling board in advance
Position coordinates relative to coordinate origin.
Further, according to the position coordinates for determining characteristic point, different fish eye images are matched, with two scaling board figures
As for, i.e., the match point N of characteristic point M in scaling board image 2 is found in scaling board image 1, due to two fish eye images it
Between have overlapping region, with reference to the detailed process of Fig. 2, in conjunction with the position coordinates of each characteristic point M, N, obtain scaling board image 1
It similarly can be also sequentially completed in this way and scaling board image 3, scaling board image 4 with the positional relationship of scaling board image 2
Characteristic point matching, to obtain the corresponding relationship of four fisheye cameras.
Fisheye camera provided in this embodiment looks around image composition method, and mobile unit obtains multiple fisheye camera acquisitions
Multiple scaling board images, fish eye images include the scaling board image, according to multiple scaling board images, obtain multiple flake phases
Thus the corresponding relationship for establishing the multiple fisheye cameras being arranged on vehicle is obtained ahead of time, actual in the corresponding relationship of machine
, can be according to the corresponding relationship being obtained ahead of time in vehicle travel process, and fish eye images obtained in real time obtain vehicle's surroundings
Outdoor scene looks around image, improves user experience.
On the basis of above-mentioned Fig. 1, Fig. 4 embodiment, it is illustrated in conjunction with specific implementation process of the Fig. 6 to this programme, Fig. 6
For the flow diagram three provided in an embodiment of the present invention that look around image composition method based on fisheye camera, as shown in fig. 6, should
Method specifically includes the following steps:
The multiple fish eye images for multiple fisheye cameras acquisition that S301, acquisition are arranged on vehicle.
The corresponding relationship between multiple fisheye cameras that S302, basis obtain in advance, splices multiple fish eye images,
Obtain the corresponding planar rings visible image of multiple fish eye images.
S303, planar rings visible image is optimized, the planar rings visible image after obtaining optimization.
Since measurement obtains position coordinates of each scaling board relative to coordinate origin in advance in step S202, due to various
Reason, there is a certain error for the position coordinates measured, and therefore, the obtained corresponding relationship between fisheye camera can also be deposited
In certain error, then there may be the feelings that the same object in different fish eye images is misaligned in the planar rings visible image that obtains
Condition needs to optimize so that spliced image is not so perfection.
Specifically, it is specifically described in conjunction with Fig. 7, Fig. 7 is the signal of splicing fish eye images provided in an embodiment of the present invention
Figure, as shown in fig. 7, fish eye images 1 and fish eye images 2 include overlapping region, according to above so that two fish eye images splice as an example
It after the tentatively splicing of method, fish eye images 1 and fish eye images 2, is not completely coincident, error is region A, no complete when detecting
After coincidence, the overlapping region of fish eye images 1 and fish eye images 2 is stretched towards other side direction respectively, to eliminate error band
A。
S304, spatial alignment processing is carried out to space object identical in multiple fish eye images, obtains planar rings visible image
Corresponding solid looks around image.
Step S301-S302 is similar with the realization process of step S101-S103, the realization of step S304 and step S103
Journey is similar, and details are not described herein.
S305, display plane look around the corresponding solid of image and look around image.
Fig. 8 is that solid provided in an embodiment of the present invention looks around the schematic diagram of image, as shown in figure 8, being presented to the user for
The solid of one vehicle's surroundings outdoor scene looks around image, and intermediate black region is vehicle, specifically, can be in the aobvious of mobile unit
Obtained solid is looked around image in display screen to show, and according to user demand, which can be looked around into image rotation
To any visual angle, so that user is more intuitive and really perceives vehicle's surroundings outdoor scene information.
Fisheye camera provided in this embodiment looks around image composition method, and mobile unit, which obtains, in this method is arranged in vehicle
Multiple fish eye images of multiple fisheye cameras acquisition on, according to the corresponding pass between the multiple fisheye cameras obtained in advance
System, splices multiple fish eye images, obtains the corresponding planar rings visible image of multiple fish eye images, corresponding relationship is for indicating
Positional relationship between the fish eye images of multiple fisheye camera acquisitions, optimizes planar rings visible image, after obtaining optimization
Planar rings visible image carries out spatial alignment processing to space object identical in multiple fish eye images, obtains planar rings visible image
Corresponding solid looks around image.In this way, can obtain vehicle's surroundings outdoor scene looks around image, user experience is improved,
Blind area when vehicle driving is eliminated simultaneously, improves driving safety.
Fig. 9 is the structural schematic diagram provided in an embodiment of the present invention that look around image synthesizer based on fisheye camera, should
Looking around image synthesizer and can be independent mobile unit based on fisheye camera, or be integrated in mobile unit
Device, the device can be realized by way of software, hardware or software and hardware combining.As shown in figure 9, based on fisheye camera
Looking around image synthesizer 10 includes: to obtain module 11, processing module 12, display module 13；
Module 11 is obtained, for obtaining multiple fish eye images of the multiple fisheye cameras being arranged on vehicle acquisition；
Processing module 12, the corresponding relationship between the multiple fisheye camera obtained in advance for basis, to described more
A fish eye images are spliced, and the corresponding planar rings visible image of the multiple fish eye images is obtained, and the corresponding relationship is used for table
Show the positional relationship between the fish eye images of the multiple fisheye camera acquisition；
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the plane panoramic view
As corresponding solid looks around image.
Optionally, the processing module 12 is also used to optimize the planar rings visible image, flat after obtaining optimization
Face ring visible image.
Optionally, the acquisition module 11 is also used to obtain multiple scaling board images of the multiple fisheye camera acquisition,
The fish eye images include the scaling board image；
According to the multiple scaling board image, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, the acquisition module 11 is also used to obtain the multiple scaling board image according to feature point detection algorithm
Characteristic point；
According to the characteristic point of the multiple scaling board image, the position coordinates of the characteristic point are obtained；
According to the position coordinates of the characteristic point, the corresponding relationship of the multiple fisheye camera is obtained.
Optionally, the processing module 12 is also used to carry out space object identical in the multiple fish eye images empty
Between registration process, obtain the first solid look around image；
Image is looked around to first solid and carries out color mixing processing, the second solid is obtained and looks around image；
The acquisition module 11 is also used to look around image according to second solid, and the threedimensional model pre-established,
It obtains the corresponding solid of the planar rings visible image and looks around image, the solid looks around image and looks around image in institute for the second solid
State the projection on the threedimensional model pre-established.
Display module 12, for showing that the corresponding solid of the planar rings visible image looks around image.
Optionally, the quantity of the fisheye camera is four.
Fisheye camera provided in an embodiment of the present invention looks around image synthesizer, can execute above method embodiment,
That the realization principle and technical effect are similar is similar for it, and details are not described herein.
Figure 10 is the hardware structural diagram of mobile unit provided in an embodiment of the present invention, as shown in Figure 10, the present embodiment
Mobile unit 80 include: processor 801 and memory 802；Wherein:
Memory 802, for storing computer executed instructions；
Processor 801 is set for executing the computer executed instructions of memory storage with realizing to receive in above-described embodiment
Standby performed each step.It specifically may refer to the associated description in preceding method embodiment.
Optionally, memory 802 can also be integrated with processor 801 either independent.
When memory 802 is independently arranged, which further includes bus 803, for connecting the memory
802 and processor 801.
The embodiment of the present invention also provides a kind of computer readable storage medium, wherein computer-readable recording medium storage
There is computer program, the computer program makes server execute the ring based on fisheye camera that aforementioned any embodiment provides
Visible image synthetic method.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, apparatus embodiments described above are merely indicative, for example, the division of the module, only
Only a kind of logical function partition, there may be another division manner in actual implementation, for example, multiple modules can combine or
It is desirably integrated into another system, or some features can be ignored or not executed.Another point, it is shown or discussed it is mutual it
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of device or module
It connects, can be electrical property, mechanical or other forms.
The module as illustrated by the separation member may or may not be physically separated, aobvious as module
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of the modules therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in one processing unit
It is that modules physically exist alone, can also be integrated in one unit with two or more modules.Above-mentioned module at
Unit both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated module realized in the form of software function module, can store and computer-readable deposit at one
In storage media.Above-mentioned software function module is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) or processor (English: processor) execute this Shen
Please each embodiment the method part steps.
It should be understood that above-mentioned processor can be central processing unit (English: Central Processing Unit, letter
Claim: CPU), can also be other general processors, digital signal processor (English: Digital Signal Processor,
Referred to as: DSP), specific integrated circuit (English: Application Specific Integrated Circuit, referred to as:
ASIC) etc..General processor can be microprocessor or the processor is also possible to any conventional processor etc..In conjunction with hair
The step of bright disclosed method, can be embodied directly in hardware processor and execute completion, or with hardware in processor and soft
Part block combiner executes completion.
Memory may include high speed RAM memory, it is also possible to and it further include non-volatile memories NVM, for example, at least one
Magnetic disk storage can also be USB flash disk, mobile hard disk, read-only memory, disk or CD etc..
Bus can be industry standard architecture (Industry Standard Architecture, ISA) bus, outer
Portion's apparatus interconnection (Peripheral Component, PCI) bus or extended industry-standard architecture (Extended
Industry Standard Architecture, EISA) bus etc..Bus can be divided into address bus, data/address bus, control
Bus etc..For convenient for indicating, the bus in illustrations does not limit only a bus or a type of bus.
Above-mentioned storage medium can be by any kind of volatibility or non-volatile memory device or their combination
It realizes, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable
Read-only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash memory,
Disk or CD.Storage medium can be any usable medium that general or specialized computer can access.
A kind of illustrative storage medium is coupled to processor, believes to enable a processor to read from the storage medium
Breath, and information can be written to the storage medium.Certainly, storage medium is also possible to the component part of processor.It processor and deposits
Storage media can be located at specific integrated circuit (Application Specific Integrated Circuits, referred to as:
ASIC in).Certainly, pocessor and storage media can also be used as discrete assembly and be present in electronic equipment or main control device.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to
The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey
When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or
The various media that can store program code such as person's CD.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (10)
1. a kind of look around image composition method based on fisheye camera characterized by comprising
Obtain the multiple fish eye images for the multiple fisheye cameras acquisition being arranged on vehicle；
According to the corresponding relationship between the multiple fisheye camera obtained in advance, the multiple fish eye images are spliced,
The corresponding planar rings visible image of the multiple fish eye images is obtained, the corresponding relationship is for indicating that the multiple fisheye camera is adopted
Positional relationship between the fish eye images of collection；
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the planar rings visible image pair
The solid answered looks around image.
2. the method according to claim 1, wherein the method also includes:
The planar rings visible image is optimized, the planar rings visible image after obtaining optimization.
3. the method according to claim 1, wherein the method also includes:
Multiple scaling board images of the multiple fisheye camera acquisition are obtained, the fish eye images include the scaling board image；
According to the multiple scaling board image, the corresponding relationship of the multiple fisheye camera is obtained.
4. according to the method described in claim 3, it is characterized in that, described according to the multiple scaling board image, described in acquisition
The corresponding relationship of multiple fisheye cameras, comprising:
According to feature point detection algorithm, the characteristic point of the multiple scaling board image is obtained；
According to the characteristic point of the multiple scaling board image, the position coordinates of the characteristic point are obtained；
According to the position coordinates of the characteristic point, the corresponding relationship of the multiple fisheye camera is obtained.
5. the method according to claim 1, wherein described to space object identical in the multiple fish eye images
Body carries out spatial alignment processing, obtains the corresponding solid of the planar rings visible image and looks around image, comprising:
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, the first solid is obtained and looks around image；
Image is looked around to first solid and carries out color mixing processing, the second solid is obtained and looks around image；
Image is looked around according to second solid, and the threedimensional model pre-established, it is corresponding to obtain the planar rings visible image
Solid look around image, it is that the second solid looks around image on the threedimensional model pre-established that the solid, which looks around image,
Projection.
6. the method according to claim 1, wherein the method also includes:
Show that the corresponding solid of the planar rings visible image looks around image.
7. method according to any one of claims 1 to 6, which is characterized in that the quantity of the fisheye camera is four.
8. a kind of look around image synthesizer based on fisheye camera characterized by comprising
Module is obtained, for obtaining multiple fish eye images of the multiple fisheye cameras being arranged on vehicle acquisition；
Processing module, the corresponding relationship between the multiple fisheye camera obtained in advance for basis, to the multiple flake
Image is spliced, and obtains the corresponding planar rings visible image of the multiple fish eye images, the corresponding relationship is for indicating described
Positional relationship between the fish eye images of multiple fisheye camera acquisitions；
Spatial alignment processing is carried out to space object identical in the multiple fish eye images, obtains the planar rings visible image pair
The solid answered looks around image.
9. a kind of mobile unit characterized by comprising
Processor；
Memory；And
Computer program；
Wherein, the computer program is stored in the memory, and is configured as being executed by the processor, described
Computer program includes the instruction for executing the method according to claim 1 to 7.
10. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has computer journey
Sequence, the computer program make server perform claim require the described in any item methods of 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811598527.3A CN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811598527.3A CN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109754363A true CN109754363A (en) | 2019-05-14 |
| CN109754363B CN109754363B (en) | 2023-08-15 |
Family
ID=66404056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811598527.3A Active CN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109754363B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110466533A (en) * | 2019-07-25 | 2019-11-19 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of control method for vehicle, apparatus and system |
| CN110580724A (en) * | 2019-08-28 | 2019-12-17 | 贝壳技术有限公司 | method and device for calibrating binocular camera set and storage medium |
| CN110910311A (en) * | 2019-10-30 | 2020-03-24 | 同济大学 | Automatic splicing method for multi-channel panoramic camera based on two-dimensional code |
| CN111754390A (en) * | 2020-05-11 | 2020-10-09 | 上海欧菲智能车联科技有限公司 | Ring view display method and device, computer equipment and storage medium |
| CN113066158A (en) * | 2019-12-16 | 2021-07-02 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN113132708A (en) * | 2021-04-22 | 2021-07-16 | 北京房江湖科技有限公司 | Method and apparatus for acquiring three-dimensional scene image using fisheye camera, device and medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104330074A (en) * | 2014-11-03 | 2015-02-04 | 熊友谊 | Intelligent surveying and mapping platform and realizing method thereof |
| CN106023070A (en) * | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| CN106534670A (en) * | 2016-10-25 | 2017-03-22 | 成都通甲优博科技有限责任公司 | Panoramic video generating method based on fixedly connected fisheye lens camera unit |
| CN106875339A (en) * | 2017-02-22 | 2017-06-20 | 长沙全度影像科技有限公司 | A kind of fish eye images joining method based on strip scaling board |
| CN107256535A (en) * | 2017-06-06 | 2017-10-17 | 斑马信息科技有限公司 | The display methods and device of panoramic looking-around image |
| CN107492125A (en) * | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| WO2018035845A1 (en) * | 2016-08-26 | 2018-03-01 | 深圳市赛亿科技开发有限公司 | Method for eliminating photographic blind spot based on 720 degree panoramic video camera |
| CN108257183A (en) * | 2017-12-20 | 2018-07-06 | 歌尔科技有限公司 | A kind of camera lens axis calibrating method and device |
| CN108629828A (en) * | 2018-04-03 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | Scene rendering transition method in the moving process of three-dimensional large scene |
| CN108846796A (en) * | 2018-06-22 | 2018-11-20 | 北京航空航天大学青岛研究院 | Image split-joint method and electronic equipment |
-
2018
- 2018-12-26 CN CN201811598527.3A patent/CN109754363B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104330074A (en) * | 2014-11-03 | 2015-02-04 | 熊友谊 | Intelligent surveying and mapping platform and realizing method thereof |
| CN106023070A (en) * | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| WO2018035845A1 (en) * | 2016-08-26 | 2018-03-01 | 深圳市赛亿科技开发有限公司 | Method for eliminating photographic blind spot based on 720 degree panoramic video camera |
| CN106534670A (en) * | 2016-10-25 | 2017-03-22 | 成都通甲优博科技有限责任公司 | Panoramic video generating method based on fixedly connected fisheye lens camera unit |
| CN106875339A (en) * | 2017-02-22 | 2017-06-20 | 长沙全度影像科技有限公司 | A kind of fish eye images joining method based on strip scaling board |
| CN107256535A (en) * | 2017-06-06 | 2017-10-17 | 斑马信息科技有限公司 | The display methods and device of panoramic looking-around image |
| CN107492125A (en) * | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| CN108257183A (en) * | 2017-12-20 | 2018-07-06 | 歌尔科技有限公司 | A kind of camera lens axis calibrating method and device |
| CN108629828A (en) * | 2018-04-03 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | Scene rendering transition method in the moving process of three-dimensional large scene |
| CN108846796A (en) * | 2018-06-22 | 2018-11-20 | 北京航空航天大学青岛研究院 | Image split-joint method and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 周芳等: "基于车载多视角的鱼眼图像拼接算法", 《数据通信》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110466533A (en) * | 2019-07-25 | 2019-11-19 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of control method for vehicle, apparatus and system |
| CN110580724A (en) * | 2019-08-28 | 2019-12-17 | 贝壳技术有限公司 | method and device for calibrating binocular camera set and storage medium |
| CN110580724B (en) * | 2019-08-28 | 2022-02-25 | 贝壳技术有限公司 | Method and device for calibrating binocular camera set and storage medium |
| CN110910311A (en) * | 2019-10-30 | 2020-03-24 | 同济大学 | Automatic splicing method for multi-channel panoramic camera based on two-dimensional code |
| CN110910311B (en) * | 2019-10-30 | 2023-09-26 | 同济大学 | Automatic splicing method of multi-path looking-around camera based on two-dimension code |
| CN113066158A (en) * | 2019-12-16 | 2021-07-02 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN113066158B (en) * | 2019-12-16 | 2023-03-10 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted all-round looking method and device |
| CN111754390A (en) * | 2020-05-11 | 2020-10-09 | 上海欧菲智能车联科技有限公司 | Ring view display method and device, computer equipment and storage medium |
| CN113132708A (en) * | 2021-04-22 | 2021-07-16 | 北京房江湖科技有限公司 | Method and apparatus for acquiring three-dimensional scene image using fisheye camera, device and medium |
| CN113132708B (en) * | 2021-04-22 | 2022-02-22 | 贝壳找房(北京)科技有限公司 | Method and apparatus for acquiring three-dimensional scene image using fisheye camera, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109754363B (en) | 2023-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109754363A (en) | Image composition method and device are looked around based on fisheye camera | |
| JP6595726B2 (en) | Transition between binocular and monocular fields | |
| JP6563609B2 (en) | Efficient canvas view generation from intermediate views | |
| WO2021120407A1 (en) | Parallax image stitching and visualization method based on multiple pairs of binocular cameras | |
| US10417829B2 (en) | Method and apparatus for providing realistic 2D/3D AR experience service based on video image | |
| JP2017220923A (en) | Image generating apparatus, image generating method, and program | |
| US20160360104A1 (en) | Systems and methods for producing a combined view from fisheye cameras | |
| US11330172B2 (en) | Panoramic image generating method and apparatus | |
| JP2019079552A (en) | Improvements in and relating to image making | |
| CN106056534A (en) | Obstruction perspective method and device based on smart glasses | |
| CN107451952B (en) | Splicing and fusing method, equipment and system for panoramic video | |
| CN101673395A (en) | Image mosaic method and image mosaic device | |
| CN107462994A (en) | Immersive VR head-wearing display device and immersive VR display methods | |
| CN103813155B (en) | Multi-view image acquisition methods, device and three-dimensional camera, three-dimensional camera | |
| US20220172319A1 (en) | Camera-based Transparent Display | |
| US20200258312A1 (en) | Image processing method and device | |
| CN107317998A (en) | Full-view video image fusion method and device | |
| CN110349077A (en) | A kind of panoramic image synthesis method, device and electronic equipment | |
| CN205787326U (en) | Omnidirectional images collection is combined camera lens | |
| CN110278366A (en) | A kind of panoramic picture weakening method, terminal and computer readable storage medium | |
| CN111815752B (en) | Image processing method and device and electronic equipment | |
| CN114511447A (en) | Image processing method, device, equipment and computer storage medium | |
| WO2024002023A1 (en) | Method and apparatus for generating panoramic stereoscopic image, and electronic device | |
| CA2861212A1 (en) | Image processing device and method, and program | |
| CN113542463A (en) | Video shooting device and method based on folding screen, storage medium and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |