CN106331527B - A kind of image split-joint method and device - Google Patents
A kind of image split-joint method and device Download PDFInfo
- Publication number
- CN106331527B CN106331527B CN201610890008.9A CN201610890008A CN106331527B CN 106331527 B CN106331527 B CN 106331527B CN 201610890008 A CN201610890008 A CN 201610890008A CN 106331527 B CN106331527 B CN 106331527B
- Authority
- CN
- China
- Prior art keywords
- coordinate
- photographic device
- image
- pixel
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000003287 optical effect Effects 0.000 claims abstract description 55
- 238000003384 imaging method Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 20
- 230000006870 function Effects 0.000 description 7
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
This application discloses a kind of image split-joint method and devices.This method comprises: obtaining the image that at least two photographic devices respectively take；For each photographic device, the three-dimensional system of coordinate of the photographic device is constructed using the public optical center of preset at least two photographic device as origin；The each pixel in an image taken for each photographic device executes following processing: the first coordinate of pixel two-dimensional coordinate system in the images is converted to the second coordinate under the three-dimensional system of coordinate；According to the target object point specified in the optical center of the photographic device and the image, the second coordinate is modified, third coordinate is obtained；And spliced according to all images of third coordinate pair of pixel each in all images.Using these technical solutions, it is capable of providing parallax free stitching image, improves the resource utilization of image splicing device.
Description
Technical field
This application involves technical field of image processing more particularly to a kind of image split-joint methods and device.
Background technique
Currently, 360 degree of panoramic videos are increasingly becoming one of main content of field of virtual reality.It is regarded compared to conventional finite
Wild video, this panoramic video be capable of providing to user it is more true to nature immerse viewing experience.Due to acquiring aphorama at present
The single-lens system of frequency is also seldom, is usually formed by the video-splicing of multiple photographic devices or a plurality of lenses system acquisition.
According to the optical perspective geometrical principle of camera lens, the two-dimensional imaging that two lens systems for not being total to optical center capture, at it
Public view sections always there are certain parallax (parallax).Also, in different depth planes, parallax degree is not
Equally, it eventually leads to spliced image and the flaw for being difficult to receive, such as ghost image, ghost, continuous lines mistake visually occurs
Position fracture etc..Therefore, the image effect being spliced into is very poor, affects the viewing experience of user, and reduces imaging device
Resource utilization.
Summary of the invention
In view of this, it is capable of providing parallax free stitching image the present invention provides a kind of image split-joint method and device,
Improve the resource utilization of image splicing device.
The technical scheme of the present invention is realized as follows:
The present invention provides a kind of image split-joint methods, comprising:
Obtain the image that at least two photographic devices respectively take；
For each photographic device, this is constructed as origin using the public optical center of preset at least two photographic device and is taken the photograph
As the three-dimensional system of coordinate of device；
The each pixel in an image taken for each photographic device executes following processing:
First coordinate of pixel two-dimensional coordinate system in the images is converted into the second coordinate under the three-dimensional system of coordinate；
According to the target object point specified in the optical center of the photographic device and the image, second coordinate is modified,
Obtain third coordinate；And
Spliced according to all images of third coordinate pair of pixel each in all images.
The present invention also provides a kind of image splicing devices, comprising:
Obtain module, the image respectively taken for obtaining at least two photographic devices；
Coordinate system constructs module, for being directed to each photographic device, with the public affairs of preset at least two photographic device
Optical center is the three-dimensional system of coordinate that origin constructs the photographic device altogether；
Coordinate processing module, each pixel in an image for taking for each photographic device execute following
Processing: the first coordinate of pixel two-dimensional coordinate system in the images is converted into the second coordinate under the three-dimensional system of coordinate；Root
According to the target object point specified in the optical center and the image of the photographic device, second coordinate is modified, obtains third seat
Mark；And
Splicing module, for being spliced according to all images of third coordinate pair of pixel each in all images.
Compared with prior art, method provided by the invention, the tool with the geometrical property of captured object, photographic device
Body imaging geometry formula, the projection type finally spliced are all unrelated, provide a kind of general technology of no parallax splicing depth plane,
Depth location splices depth plane as no parallax where the main contents selected in scene that can be adaptive, provides parallax free
Stitching image goes parallax to handle, improves the resource utilization of image splicing device without additional.
Detailed description of the invention
For the clearer technical solution illustrated in the embodiment of the present invention, will make below to required in embodiment description
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.Wherein,
Fig. 1 is the exemplary process diagram of the image split-joint method of an embodiment according to the present invention；
Fig. 2 is the schematic diagram of the building cartesian coordinate system of an embodiment according to the present invention；
Fig. 3 is the exemplary process diagram of the optical centre bias compensation method of an embodiment according to the present invention；
Fig. 4 a is the coordinate schematic diagram of an embodiment according to the present invention being modified to the second coordinate；
Fig. 4 b is the coordinate schematic diagram of the determination offset of an embodiment according to the present invention；
Fig. 5 is the exemplary process diagram of the image split-joint method of another embodiment according to the present invention；
Fig. 6 a is the two dimensional image schematic diagram before the splicing of an embodiment according to the present invention；
Fig. 6 b is the spliced two dimensional image schematic diagram of an embodiment according to the present invention；
The structural schematic diagram of Fig. 7 image splicing device of an embodiment according to the present invention；
Fig. 8 is the structural schematic diagram of the image splicing device of another embodiment according to the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Image split-joint method and device in the embodiment of the present invention have taking the photograph at least two photographic devices suitable for any
As system, wherein the visual angle of two adjacent photographic devices has a common portion, i.e., public view sections, captured by the two
Image has lap.Method according to embodiments of the present invention, the image taken respectively for each photographic device into
Row processing, then carries out the splicing of image in entire camera system, can on specified target object point (or depth plane)
Obtain complete parallax free panoramic picture.
Fig. 1 is the exemplary process diagram of the image split-joint method of an embodiment according to the present invention.As shown in Figure 1, this method
May include following steps:
Step 101, the image that at least two photographic devices respectively take is obtained.
This step obtains the image that all photographic devices take in a camera system first.
Step 102, it for each photographic device, is constructed by origin of the public optical center of preset at least two photographic device
The three-dimensional system of coordinate of the photographic device.
Since each photographic device has an optical center of itself camera lens, in this step, a common light is preset first
The heart assumes that all photographic devices all have such a ideal optical center, construct three-dimensional system of coordinate as origin.
If three-dimensional system of coordinate is expressed as (X, Y, Z), the three-dimensional of the photographic device is constructed using preset public optical center as origin
It when coordinate system, specifically includes: using public optical center as origin, establishing two-dimensional coordinate on the parallel surface of the imaging surface of the photographic device
It is (X, Y), Z axis is then determined according to two-dimensional coordinate system (X, Y) and the right-hand rule.
In one embodiment, which is cartesian coordinate system.For the coordinate system of photographic device, this
Kind of cartesian coordinate system is otherwise known as Descartes's world coordinate system.Fig. 2 is that the building Descartes of an embodiment according to the present invention sits
Mark the schematic diagram of system.As shown in Fig. 2, X-axis, Y-axis and Z axis have collectively constituted the cartesian coordinate system of a photographic device A, common light
Heart O is the origin of coordinate system.Incident lightThe lens system for entering photographic device A with the angle θ is imaging after lens reflect
It is imaged on the imaging surface x'o'y' of device A.Wherein, the face XOY is parallel with the face x'o'y'.
Step 103, each pixel in an image taken for each photographic device executes following processing:
Step 1031, the first coordinate of pixel two-dimensional coordinate system in the images is converted under the three-dimensional system of coordinate
Second coordinate；
Step 1032, according to the target object point specified in the optical center of the photographic device and the image, the second coordinate is carried out
Amendment, obtains third coordinate.
Wherein, for step 1031, the first coordinate is converted into the second coordinate, is specifically included: determined according to the first coordinate
The angular coordinate of the pixel determines that incident light and the three-dimensional are sat according to the lens imaging geometric function of the photographic device and the first coordinate
Angle in mark system (X, Y, Z) between Z axis, then calculates the second coordinate according to angle and angular coordinate.
If the first coordinate representation of a pixel is (x1, y1), angular coordinate is expressed asThe pixel is determined according to the first coordinate
Angular coordinate include determineFollowing trigonometric function value:
If the second coordinate representation is (x2, y2, z2), angle is expressed as θ, then calculates in the second coordinate according to following formula
X2、y2And z2:
If the lens imaging geometric function of photographic device is r (θ), when the lens of the photographic device are linear type
(rectilinear) when, there is r (θ)=ftan (θ), then angle
When the lens of the photographic device are isometric type (equidistant), there is r (θ)=f θ, then angle
Wherein, atan () expression negates tangent value function, and pw, ph respectively indicate the width of the pixel and height, f are
The focal length (as shown in Figure 2) of mirror.
It corresponds in Fig. 2, a pixel p of imaging surface x'o'y'1', the first coordinate is (x1, y1), p1' and origin o '
Between line and x ' o ' axis between angle beIt is transformed under cartesian coordinate system (X, Y, Z), corresponding object point P1, thirdly
It ties up shown in coordinate such as formula (2).Wherein, P1P is projected as on XOY two-dimensional surface1, p1Line between origin O and XO axis it
Between angle be also
Above-mentioned public optical center O is unique for all photographic devices, it is contemplated that each camera shooting dress in practice
An optical center O ' for all having oneself is set, therefore, it is necessary to be compensated according to image of the deviation between optical center to imaging, so that
It is consistent for the imaging under origin with O.
In this regard, Fig. 3 is the exemplary process diagram of the optical centre bias compensation method of an embodiment according to the present invention.For step
1032, according to the target object point specified in the optical center of the photographic device and the image, the second coordinate is modified, third is obtained
Coordinate, as shown in figure 3, specifically comprising the following steps:
Step 301, the distance between public optical center and target object point are obtained, that is, obtains the depth of target object point.
In this step, target object point can be referred to by user according to oneself interested object point in taken image
It is fixed, alternatively, can according in scene main target object or content specified.After specifying target object point, estimation
Distance on the face XOZ between public optical center and target object point out.For example, being estimated according to third party software in a specific field
The depth of the target object point is 10m or 20m etc. in scape.
Fig. 4 a is the coordinate schematic diagram of an embodiment according to the present invention being modified to the second coordinate.As shown in fig. 4 a,
Target object point is incident lightOn object point P1, above-mentioned distance is P1It is projected on the face XOZLength, i.e. O to P ' it
Between length, be denoted as R0, the distance also referred to as object point P1Depth.
Step 302, offset of the optical center of the photographic device relative to public optical center is obtained.
In this step, it is contemplated that have between the image captured by two neighboring photographic device in a panoramic shooting system
Standby lap is returned or is imitated according to the sample data of overlapping image and with correspondence/matching relationship of photographic device
True estimation, can determine above-mentioned offset.For example, panorama (i.e. 360 °) video system, is mounted in three dimensions
Multiple cameras, each camera take the image in certain angular field of view.
Fig. 4 b is the coordinate schematic diagram of the determination offset of an embodiment according to the present invention.As shown in Figure 4 b, in three-dimensional ball
In ABC coordinate system constructed by face 400, camera 401 and 402, the tool of image captured by the two are disposed on different location
There is lap.It can be determined according to the sample data of overlapping image inclined between the optical center O ' of each camera and origin O
Shifting amount.It returns in Fig. 4 a, optical center O ' is respectively T relative to offset of the origin O on X-axis, Y-axis and Z axisx, Ty, Tz。
Step 303, third coordinate is calculated according to distance, offset and the second coordinate.
Second coordinate is modified, third coordinate (x can be calculated according to following formula3, y3, z3Each of)
Coordinate value x3、y3And z3:
Wherein,B=2 (Tz·z2+Tx·x2)。
Step 104, spliced according to all images of third coordinate pair of pixel each in all images.
After carrying out above-mentioned processing for each pixel in each image, according to each photographic device in camera system institute
The position at place, according to certain projection type, treated that image splices to all, to obtain locating for the target object point
Panoramic picture without any parallax in depth plane.
In the present embodiment, the image respectively taken by obtaining at least two photographic devices, fills for each camera shooting
It sets, the three-dimensional system of coordinate of the photographic device is constructed using the public optical center of preset at least two photographic device as origin, for every
Each pixel in an image that a photographic device takes executes following processing: by pixel two-dimensional coordinate in the images
First coordinate of system is converted to the second coordinate under the three-dimensional system of coordinate；It is specified according in the optical center of the photographic device and the image
Target object point, the second coordinate is modified, third coordinate is obtained, according to the third coordinate pair of pixel each in all images
All images are spliced, and a kind of technology of no parallax splicing depth plane, the master in selection scene that can be adaptive are provided
Depth location splices depth plane as no parallax where wanting content, so that the spelling of no parallax flaw is presented in the main contents in scene
Connect effect.
In addition, in the above method conversion of coordinate and optical centre bias compensation, it is unrelated with the geometrical property of target object point, no
Dependent on the shape of specific target object point, it is more suitable for the continually changing Video Applications of the content on time dimension.With it is existing
Technology is compared, and the above method is not necessarily to carry out feature detection and characteristic matching to scene content, so as to the basis of fast and flexible
Target object point that user specifies (or specified no parallax splices depth plane), object point or scene content to desired locations into
The complete alignment of row, provides parallax free stitching image.Also, the specific imaging geometry formula of the above method and photographic device, most
The projection type spliced eventually is also unrelated, therefore, has versatility, improves the resource utilization of image splicing device.
Fig. 5 is the exemplary process diagram of the image split-joint method of another embodiment according to the present invention.As shown in figure 5, including
Following steps:
Step 501, the image that at least two photographic devices respectively take is obtained.
Step 502, it for each photographic device, is constructed by origin of the public optical center of preset at least two photographic device
The cartesian coordinate system of the photographic device.
Step 503, each pixel in an image taken for each photographic device executes following processing:
Step 5031, coordinate conversion is carried out:
First coordinate of pixel two-dimensional coordinate system in the images is converted to second under the cartesian coordinate system to sit
Mark；
Step 5032, optical centre bias compensation is carried out:
According to the target object point specified in the optical center of the photographic device and the image, the second coordinate is modified, is obtained
Third coordinate.
The mould of the second coordinate is 1 it can be seen from above-mentioned formula (2), i.e.,Established
Cartesian coordinate system is normalized cartesian coordinate system.Since normalization cartesian coordinate system is free from depth information, institute
In same incident rayThe different object point of upper two depth possesses identical normalization cartesian coordinate value.Such as Fig. 2 institute
Show, by p1' be transformed into normalization cartesian coordinate system (X, Y, Z) under corresponding object point be not only P1, in addition to P1, can also be
Along incident lightOn other object points, such as the P in Fig. 22.Object point P1And P2Depth it is different, i.e., relative to light on the face XOZ
The distance between heart O difference, but the two possesses identical normalization cartesian coordinate value (x2, y2, z2), it both corresponds to be imaged
P on the x'o'y' of face1′。
Step 504, according to each photographic device the location of in panorama system, according to preset projection type by the
Three coordinate projections are into unit panorama spherical surface.
When all photographic devices constitute the camera system of a panorama, by third coordinate projection to a unit panorama
In spherical surface.Preset projection type includes but is not limited to: linear type (rectilinear), fisheye type (fisheye), etc. squares column
Shape projects (equirectangular), orthogonal projection (orthographic), spherical projection (stereographic) etc..
Step 505, all images are spliced in unit panorama spherical surface, obtains panoramic picture.
Through the above steps, in spliced panoramic picture, it can reach on specified object point position and ignore
The splicing depth plane of difference, adjacent image is perfectly aligned, obtains the effect without splicing flaw.It, can be with when showing image to user
Three-dimensional panorama head portrait is reconverted into two-dimensional image.
Fig. 6 a is the two dimensional image schematic diagram before the splicing of an embodiment according to the present invention.Wherein, in left figure 600, mesh
Mark object point is the first flagpole (as shown in arrow 601), corresponds to P1-P ' shown in Fig. 4 a.Before optical centre bias compensation,
Occur above and below as caused by parallax at the flagpole, left images are misaligned phenomenon.It can be clearly seen that in right figure 610,
Also there is extra point 611 ' in the lower left on the top 611 of flagpole, flag be originally used for 612 shown in image, but due to parallax,
Cause to be ultimately imaged is 612 '.
Fig. 6 b is the spliced two dimensional image schematic diagram of an embodiment according to the present invention.Correspondingly, left figure 620 is to pass through
Coordinate transform, the compensated imaging of optical centre bias, at the flagpole above and below image perfection alignment.It can be clear in right figure 630
See there is no the image of alignment all to disappear except top 611 and flag 612, shown clearly flagpole.As it can be seen that realizing
The perfect of main contents object (i.e. flagpole) in scene is aligned, at flagpole position, becomes no parallax splicing depth plane.
It in specific application, can also panorama painting canvas (canvas) by the way of reverse process, i.e., in a blank
On, inversely processing process is executed pixel-by-pixel (successively executes seat described in the compensation of optical centre bias described in step 5032, step 5031
Mark conversion operation), the location of pixels for the photographic device captured images that it is corresponded to is found, then interpolation obtains current panorama picture
The actual value of the pixel on cloth.
The structural schematic diagram of Fig. 7 image splicing device of an embodiment according to the present invention.As shown in fig. 7, image mosaic fills
Setting 700 includes obtaining module 710, coordinate system building module 720, coordinate processing module 730 and splicing module 740, wherein
Obtain module 710, the image respectively taken for obtaining at least two photographic devices；
Coordinate system constructs module 720, for being directed to each photographic device, with the public of preset at least two photographic device
Optical center is the three-dimensional system of coordinate that origin constructs the photographic device；
Coordinate processing module 730, each pixel in an image for taking for each photographic device, execute with
Lower processing: the first coordinate of pixel two-dimensional coordinate system in the images is converted into the second coordinate under the three-dimensional system of coordinate；
According to the target object point specified in the optical center of the photographic device and the image, the second coordinate is modified, third coordinate is obtained；
And
Splicing module 740, for being spliced according to all images of third coordinate pair of pixel each in all images.
In one embodiment, coordinate processing module 730 includes converting unit 731, for determining the picture according to the first coordinate
The angular coordinate of element；Incident light and the three-dimensional system of coordinate are determined according to the lens imaging geometric function of the photographic device and the first coordinate
Angle in (X, Y, Z) between Z axis；Go out the second coordinate according to angular coordinate and angle calcu-lation.
In one embodiment, if the first coordinate representation is (x1, y1), angular coordinate is expressed asConverting unit 731 is used for, really
It is fixed:
The three-dimensional system of coordinate is cartesian coordinate system, if the second coordinate representation is (x2, y2, z2), angle is expressed as θ, conversion
Unit 731 is used for, and x is calculated according to following formula2、y2And z2:
z2=cos (θ)
In one embodiment, coordinate processing module 730 includes amending unit 732, for obtaining public optical center and object
The distance between point；Obtain offset of the optical center of the photographic device relative to public optical center；According to distance, offset and second
Coordinate calculates third coordinate.
In one embodiment, if distance is expressed as R0, offset-lists are shown as (Tx, Ty, Tz), the second coordinate representation is (x2,
y2, z2), third coordinate representation is (x3, y3, z3), amending unit 732 is used for, and x is calculated according to following formula3、y3And z3:
Wherein,B=2 (Tz·z2+Tx·x2)。
In one embodiment, splicing module 740 is used for, according to each photographic device the location of in panorama system,
According to preset projection type by third coordinate projection into unit panorama spherical surface；By all images in unit panorama spherical surface
Spliced, obtains panoramic picture.
Fig. 8 is the structural schematic diagram of the image splicing device of another embodiment according to the present invention.The image splicing device 800
Can include: processor 810, memory 820, port 830 and bus 840.Processor 810 and memory 820 pass through bus 840
Interconnection.Processor 810 can send and receive data by port 830.Wherein,
Processor 810 is used to execute the machine readable instructions module of the storage of memory 820.
Memory 820 is stored with the executable machine readable instructions module of processor 810.The executable finger of processor 810
Enabling module includes: to obtain module 821, coordinate system building module 822, coordinate processing module 823 and splicing module 824.Wherein,
Obtaining can be with when module 821 is executed by processor 810 are as follows: obtains the figure that at least two photographic devices respectively take
Picture；
Coordinate system constructs can be with when module 822 is executed by processor 810 are as follows: be directed to each photographic device, with it is preset extremely
The public optical center of few two photographic devices is the three-dimensional system of coordinate that origin constructs the photographic device；
Coordinate processing module 823 can be with when being executed by processor 810 are as follows: the image taken for each photographic device
In each pixel, execute following processing: the first coordinate of pixel two-dimensional coordinate system in the images be converted into the three-dimensional
The second coordinate under coordinate system；According to the target object point specified in the optical center of the photographic device and the image, to the second coordinate into
Row amendment, obtains third coordinate；And
Splicing module 824 can be with when being executed by processor 810 are as follows: according to the third coordinate pair of pixel each in all images
All images are spliced.
It can thus be seen that when storing the instruction module in memory 820 and being executed by processor 810, it can be achieved that preceding
It states and obtains module, coordinate system building module, the various functions of coordinate processing module and splicing module in each embodiment.
In above-mentioned apparatus and system embodiment, modules and unit realize that the specific method of itself function is implemented in method
It is described in example, which is not described herein again.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in one processing unit
It is that modules physically exist alone, can also be integrated in one unit with two or more modules.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
In addition, each embodiment of the invention can pass through the data processing by data processing equipment such as computer execution
Program is realized.Obviously, data processor constitutes the present invention.In addition, being commonly stored data in one storage medium
Processing routine is by directly reading out storage medium for program or by installing or copying to data processing equipment for program
It stores in equipment (such as hard disk and/or memory) and executes.Therefore, such storage medium also constitutes the present invention.Storage medium can be with
Use any kind of recording mode, such as paper storage medium (such as paper tape), magnetic storage medium (such as floppy disk, hard disk, flash memory
Deng), optical storage media (such as CD-ROM), magnetic-optical storage medium (such as MO) etc..
Therefore, the invention also discloses a kind of storage mediums, wherein it is stored with data processor, the data processor
For executing any embodiment of the above method of the present invention.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the present invention.
Claims (12)
1. a kind of image split-joint method characterized by comprising
Obtain the image that at least two photographic devices respectively take；
For each photographic device, camera shooting dress is constructed by origin of the public optical center of preset at least two photographic device
The three-dimensional system of coordinate set；
The each pixel in an image taken for each photographic device executes following processing:
First coordinate of pixel two-dimensional coordinate system in the images is converted into the second coordinate under the three-dimensional system of coordinate；
According to the target object point specified in the optical center of the photographic device and the image, second coordinate is modified, is obtained
Third coordinate；And
Spliced according to all images of third coordinate pair of pixel each in all images；
Wherein, the optical center according to the photographic device and the target object point specified in the image, carry out second coordinate
Amendment, obtaining third coordinate includes:
Obtain the distance between the public optical center and the target object point；
Obtain offset of the optical center of the photographic device relative to the public optical center；
The third coordinate is calculated according to the distance, the offset and second coordinate.
2. described with preset if the three-dimensional system of coordinate is expressed as (X, Y, Z) according to the method described in claim 1, wherein
The public optical center of at least two photographic device is that origin constructs the three-dimensional system of coordinate of the photographic device and includes:
Using the public optical center as origin, two-dimensional coordinate system (X, Y) is established on the parallel surface of the imaging surface of the photographic device；
Z axis is determined according to the two-dimensional coordinate system (X, Y) and the right-hand rule.
3. described by the pixel if the three-dimensional system of coordinate is expressed as (X, Y, Z) according to the method described in claim 1, wherein
The first coordinate of two-dimensional coordinate system is converted to the second coordinate under the three-dimensional system of coordinate and includes: in the images
The angular coordinate of the pixel is determined according to first coordinate；
According to the lens imaging geometric function of the photographic device and first coordinate determine incident light and the three-dimensional system of coordinate (X,
Y, Z) in angle between Z axis；
Go out second coordinate according to the angular coordinate and the angle calcu-lation.
4. according to the method described in claim 3, wherein, if first coordinate representation is (x1, y1), the angular coordinate indicates
ForThe angular coordinate that the pixel is determined according to the first coordinate includes:
It determinesTrigonometric function value be respectively as follows:
The three-dimensional system of coordinate is cartesian coordinate system, if second coordinate representation is (x2, y2, z2), the angle is expressed as θ,
It is described second coordinate is calculated according to the angle and the angular coordinate to include:
X is calculated according to following formula2、y2And z2:
z2=cos (θ).
5. the method according to claim 3 or 4, wherein if first coordinate representation is (x1, y1), the lens imaging
Geometric function is expressed as r (θ), and the angle is expressed as θ, the lens imaging geometric function according to the photographic device and described
First coordinate determines that the angle in incident light and the three-dimensional system of coordinate (X, Y, Z) between Z axis includes:
When the lens of the photographic device are linear type, there is r (θ)=ftan (θ), then
When the lens of the photographic device are isometric type, there is r (θ)=f θ, then
Wherein, atan () expression negates tangent value function, and pw, ph respectively indicate the width and height of the pixel, and f is lens
Focal length.
6. according to the method described in claim 1, wherein, if the distance is expressed as R0, the offset-lists are shown as (Tx, Ty,
Tz), second coordinate representation is (x2, y2, z2), the third coordinate representation is (x3, y3, z3), it is described according to the distance,
The offset and second coordinate calculate the third coordinate
X is calculated according to following formula3、y3And z3:
Wherein,B=2 (Tz·z2+Tx·x2)。
7. according to claim 1 to method described in any one of 4,6, wherein described according to pixel each in all images
All images of third coordinate pair carry out splicing
According to each photographic device the location of in panorama system, the third coordinate is thrown according to preset projection type
Shadow is into unit panorama spherical surface；
All images are spliced in the unit panorama spherical surface, obtain panoramic picture.
8. a kind of image splicing device characterized by comprising
Obtain module, the image respectively taken for obtaining at least two photographic devices；
Coordinate system constructs module, for being directed to each photographic device, with the common light of preset at least two photographic device
The heart is the three-dimensional system of coordinate that origin constructs the photographic device；
Coordinate processing module, each pixel in an image for taking for each photographic device execute following processing:
First coordinate of pixel two-dimensional coordinate system in the images is converted into the second coordinate under the three-dimensional system of coordinate；It is taken the photograph according to this
As target object point specified in the optical center and the image of device, second coordinate is modified, third coordinate is obtained；And
Splicing module, for being spliced according to all images of third coordinate pair of pixel each in all images；
Wherein, the coordinate processing module includes amending unit, for obtaining between the public optical center and the target object point
Distance；Obtain offset of the optical center of the photographic device relative to the public optical center；According to the distance, the offset
The third coordinate is calculated with second coordinate.
9. device according to claim 8, wherein if the three-dimensional system of coordinate is expressed as (X, Y, Z), the coordinate processing
Module includes converting unit, for determining the angular coordinate of the pixel according to first coordinate；According to the lens of the photographic device
Imaging geometry function and first coordinate determine the angle in incident light and the three-dimensional system of coordinate (X, Y, Z) between Z axis；According to
The angular coordinate and the angle calcu-lation go out second coordinate.
10. device according to claim 9, wherein if first coordinate representation is (x1, y1), the angular coordinate indicates
ForThe converting unit is used for, and is determined:
The three-dimensional system of coordinate is cartesian coordinate system, if second coordinate representation is (x2, y2, z2), the angle is expressed as θ,
The converting unit is used for, and x is calculated according to following formula2、y2And z2:
z2=cos (θ).
11. device according to claim 8, wherein if the distance is expressed as R0, the offset-lists are shown as (Tx, Ty,
Tz), second coordinate representation is (x2, y2, z2), the third coordinate representation is (x3, y3, z3), the amending unit is used for,
X is calculated according to following formula3、y3And z3:
Wherein,B=2 (Tz·z2+Tx·x2)。
12. the device according to any one of claim 8 to 11, wherein the splicing module is used for, according to each camera shooting
Device is the location of in panorama system, according to preset projection type by the third coordinate projection to unit panorama spherical surface
In；All images are spliced in the unit panorama spherical surface, obtain panoramic picture.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610890008.9A CN106331527B (en) | 2016-10-12 | 2016-10-12 | A kind of image split-joint method and device |
| PCT/CN2017/105657 WO2018068719A1 (en) | 2016-10-12 | 2017-10-11 | Image stitching method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610890008.9A CN106331527B (en) | 2016-10-12 | 2016-10-12 | A kind of image split-joint method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106331527A CN106331527A (en) | 2017-01-11 |
| CN106331527B true CN106331527B (en) | 2019-05-17 |
Family
ID=57820319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610890008.9A Active CN106331527B (en) | 2016-10-12 | 2016-10-12 | A kind of image split-joint method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106331527B (en) |
| WO (1) | WO2018068719A1 (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331527B (en) * | 2016-10-12 | 2019-05-17 | 腾讯科技（北京）有限公司 | A kind of image split-joint method and device |
| US10885650B2 (en) | 2017-02-23 | 2021-01-05 | Eys3D Microelectronics, Co. | Image device utilizing non-planar projection images to generate a depth map and related method thereof |
| CN110519774B (en) * | 2018-05-21 | 2023-04-18 | 中国移动通信集团广东有限公司 | Base station investigation method, system and equipment based on VR technology |
| EP3606032B1 (en) * | 2018-07-30 | 2020-10-21 | Axis AB | Method and camera system combining views from plurality of cameras |
| CN109889736B (en) * | 2019-01-10 | 2020-06-19 | 深圳市沃特沃德股份有限公司 | Image acquisition method, device and equipment based on double cameras and multiple cameras |
| CN110072158B (en) * | 2019-05-06 | 2021-06-04 | 复旦大学 | Spherical equator area double-C type panoramic video projection method |
| CN112449100B (en) * | 2019-09-03 | 2023-11-17 | 中国科学院长春光学精密机械与物理研究所 | Aviation camera inclined image splicing method, device, terminal and storage medium |
| CN111142825B (en) * | 2019-12-27 | 2024-04-16 | 杭州拓叭吧科技有限公司 | Multi-screen visual field display method and system and electronic equipment |
| US11645780B2 (en) | 2020-03-16 | 2023-05-09 | Realsee (Beijing) Technology Co., Ltd. | Method and device for collecting images of a scene for generating virtual reality data |
| CN111432119B (en) * | 2020-03-27 | 2021-03-23 | 北京房江湖科技有限公司 | Image shooting method and device, computer readable storage medium and electronic equipment |
| CN112771842A (en) * | 2020-06-02 | 2021-05-07 | 深圳市大疆创新科技有限公司 | Imaging method, imaging apparatus, computer-readable storage medium |
| CN113873220A (en) * | 2020-12-03 | 2021-12-31 | 上海飞机制造有限公司 | Deviation analysis method, device, system, equipment and storage medium |
| CN112669199B (en) * | 2020-12-16 | 2022-06-21 | 影石创新科技股份有限公司 | Image stitching method, computer-readable storage medium and computer device |
| CN114554176A (en) * | 2022-01-24 | 2022-05-27 | 北京有竹居网络技术有限公司 | Depth camera |
| CN115781665B (en) * | 2022-11-01 | 2023-08-08 | 深圳史河机器人科技有限公司 | Mechanical arm control method and device based on monocular camera and storage medium |
| CN116643393B (en) * | 2023-07-27 | 2023-10-27 | 南京木木西里科技有限公司 | Microscopic image deflection-based processing method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521745A (en) * | 2009-04-14 | 2009-09-02 | 王广生 | Multi-lens optical center superposing type omnibearing shooting device and panoramic shooting and retransmitting method |
| CN101710932A (en) * | 2009-12-21 | 2010-05-19 | 深圳华为通信技术有限公司 | Image stitching method and device |
| CN101783883A (en) * | 2009-12-26 | 2010-07-21 | 华为终端有限公司 | Adjusting method in co-optical-center videography and co-optical-center camera system |
| CN102798350A (en) * | 2012-07-10 | 2012-11-28 | 中联重科股份有限公司 | Method, device and system for measuring deflection of arm support |
| CN104506764A (en) * | 2014-11-17 | 2015-04-08 | 南京泓众电子科技有限公司 | An automobile traveling recording system based on a spliced video image |
| CN105812640A (en) * | 2016-05-27 | 2016-07-27 | 北京伟开赛德科技发展有限公司 | Spherical omni-directional camera device and video image transmission method thereof |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE60137655D1 (en) * | 2000-02-29 | 2009-03-26 | Panasonic Corp | Image recording system and vehicle-mounted sensor system |
| KR20020025301A (en) * | 2000-09-28 | 2002-04-04 | 오길록 | Apparatus and Method for Furnishing Augmented-Reality Graphic using Panoramic Image with Supporting Multiuser |
| JP3889650B2 (en) * | 2002-03-28 | 2007-03-07 | 三洋電機株式会社 | Image processing method, image processing apparatus, computer program, and recording medium |
| CN103379267A (en) * | 2012-04-16 | 2013-10-30 | 鸿富锦精密工业（深圳）有限公司 | Three-dimensional space image acquisition system and method |
| US10666860B2 (en) * | 2012-09-11 | 2020-05-26 | Ricoh Company, Ltd. | Image processor, image processing method and program, and imaging system |
| CN106331527B (en) * | 2016-10-12 | 2019-05-17 | 腾讯科技（北京）有限公司 | A kind of image split-joint method and device |
-
2016
- 2016-10-12 CN CN201610890008.9A patent/CN106331527B/en active Active
-
2017
- 2017-10-11 WO PCT/CN2017/105657 patent/WO2018068719A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101521745A (en) * | 2009-04-14 | 2009-09-02 | 王广生 | Multi-lens optical center superposing type omnibearing shooting device and panoramic shooting and retransmitting method |
| CN101710932A (en) * | 2009-12-21 | 2010-05-19 | 深圳华为通信技术有限公司 | Image stitching method and device |
| CN101783883A (en) * | 2009-12-26 | 2010-07-21 | 华为终端有限公司 | Adjusting method in co-optical-center videography and co-optical-center camera system |
| CN102798350A (en) * | 2012-07-10 | 2012-11-28 | 中联重科股份有限公司 | Method, device and system for measuring deflection of arm support |
| CN104506764A (en) * | 2014-11-17 | 2015-04-08 | 南京泓众电子科技有限公司 | An automobile traveling recording system based on a spliced video image |
| CN105812640A (en) * | 2016-05-27 | 2016-07-27 | 北京伟开赛德科技发展有限公司 | Spherical omni-directional camera device and video image transmission method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018068719A1 (en) | 2018-04-19 |
| CN106331527A (en) | 2017-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106331527B (en) | A kind of image split-joint method and device | |
| CN101673395B (en) | Image mosaic method and image mosaic device | |
| CN107316273B (en) | Panoramic image acquisition device and acquisition method | |
| JP5739409B2 (en) | Method for determining the relative position of a first image device and a second image device and these devices | |
| CN109064397B (en) | Image stitching method and system based on camera earphone | |
| CN106709865B (en) | Depth image synthesis method and device | |
| JP6201476B2 (en) | Free viewpoint image capturing apparatus and method | |
| US20190012804A1 (en) | Methods and apparatuses for panoramic image processing | |
| CN110111262A (en) | A kind of projector distortion correction method, device and projector | |
| US20160295108A1 (en) | System and method for panoramic imaging | |
| CN107133918B (en) | Method for generating panorama at any position in three-dimensional scene | |
| JP2018534698A (en) | Method and system for large scale determination of RGBD camera attitude | |
| JPWO2018235163A1 (en) | Calibration apparatus, calibration chart, chart pattern generation apparatus, and calibration method | |
| JP2007257100A (en) | Method for creating panoramic image | |
| TWI669683B (en) | Three dimensional reconstruction method, apparatus and non-transitory computer readable storage medium | |
| Pathak et al. | Dense 3D reconstruction from two spherical images via optical flow-based equirectangular epipolar rectification | |
| KR102200866B1 (en) | 3-dimensional modeling method using 2-dimensional image | |
| Baker et al. | CasualStereo: Casual capture of stereo panoramas with spherical structure-from-motion | |
| JP4778569B2 (en) | Stereo image processing apparatus, stereo image processing method, and stereo image processing program | |
| TWM594322U (en) | Camera configuration system with omnidirectional stereo vision | |
| WO2018150086A2 (en) | Methods and apparatuses for determining positions of multi-directional image capture apparatuses | |
| JP2019527495A (en) | Stereo image capture | |
| CN115147268A (en) | Live view method, panoramic camera, and computer-readable storage medium | |
| Xu et al. | Image rectification for single camera stereo system | |
| Kudinov et al. | The algorithm for a video panorama construction and its software implementation using CUDA technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |