CN101923709A - Image splicing method and equipment - Google Patents
Image splicing method and equipment Download PDFInfo
- Publication number
- CN101923709A CN101923709A CN2009101493467A CN200910149346A CN101923709A CN 101923709 A CN101923709 A CN 101923709A CN 2009101493467 A CN2009101493467 A CN 2009101493467A CN 200910149346 A CN200910149346 A CN 200910149346A CN 101923709 A CN101923709 A CN 101923709A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- piece
- relative distance
- joint position
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses an image splicing method and image splicing equipment which are used for splicing an image rapidly and do not reduce the quality of the spliced image. The method comprises the following steps of: determining a quilt between a first image and a second image; dividing pixels of the first image and the second image into a foreground pixel and a background pixel; splicing the foreground pixel of the first image and the second image based on the quilt; and deforming the background pixel and enabling the background pixel to be fused with the spliced foreground pixel. The method has the characteristics of simple algorithm and high speed. Layered splicing is adopted, and background deformation adapts to the foreground pixel splicing, therefore the inconsistency of vision is eliminated and the quality of image splicing is high.
Description
Technical field
The present invention relates to a kind of image mosaic technology, be specifically related to a kind of image split-joint method and equipment.
Background technology
To seamlessly be stitched together at the image of different angles or position shooting, the technology that forms a panel height resolution panorama sketch is called image mosaic.Study of Image Mosaics Technology is a key areas of computer vision research.This technology has purposes widely, for example the foundation of synthetic, the panoramic virtual scene of the satellite image or the image of taking photo by plane, photo editing etc.Along with the progress of this art, image mosaic has entered into daily life, and for example in digital camera, panorama sketch synthesizes the function that has become an item of digital camera.
Video-splicing is a kind of special image mosaic technology.The purpose of video-splicing is synthetic video image from the different video source, obtains a high-resolution wide-angle video.Along with the develop rapidly of electronics industry, large-sized monitor enters into daily life rapidly.Problem of the thing followed is how to obtain more and more high-resolution video data.Owing to had large-sized monitor, people to wish that the content of seeing is more and more abundanter on a form.This has just caused requiring the visual field of video to want enough big, thereby produces the demand of large scale wide-angle video.In the face of these demands, the video-splicing technology is a possible solution.By video-splicing, can be with the synthetic high resolving power wide-angle video of the video of single low resolution.In field of video monitoring,, the efficient of supervisory system will be improved greatly if can provide the large-scale visual field to cover.In the bigger place of some areas,, all need this video acquisition device that the large scale visual field covers that possesses as big supermarket, street, building inside.This needs the support of video-splicing technology equally.In addition, in video conference, what people's custom was seen is the wide-angle video of similar human eye scope.This also needs video-splicing that corresponding techniques is provided.
Image mosaic will synthesize from the source images of a plurality of viewpoints.Through the image after synthetic just as taking a viewpoint.Therefore, the image mosaic technology at first needs the geometry site between definite source images.According to geometric relationship, stitching algorithm just can be determined the position of pixel in composograph of source images.In general, overlapping owing to existing between the source images in composograph, so lap will be carried out individual processing has vision with the pixel that guarantees composograph consistance.A basic problem of image mosaic is the geometry site that calculates between the source images.Yet accurate geometric relationship is calculated the three-dimensional information that need know scenery.Recovering three-dimensional information from image is an ill-conditioning problem, can not obtain accurate and stable separating.So in image mosaic, geometry site calculates and often adopts approximate treatment, such as scenery being approximately a plane.Owing to adopt approximate treatment, parallax will appear in doubling of the image part.Its direct result is exactly on composograph, and two source images inconsistent situation occurs at seaming position, and then causes occurring in the composograph a tangible piece.Even more serious is that in composograph, object may duplicate or lack near the seam.The inconsistent situation of this serious vision is called as ghost image.A good merging algorithm for images will be eliminated ghost image and tangible piece exactly as much as possible.
Video-splicing can directly use the image mosaic technology on principle.Its basic way is to adopt image mosaic technology composograph frame by frame.Yet in general, high-quality merging algorithm for images often adopts high-precision geometrical calculation method.These methods are more consuming time, are difficult to reach live effect, and the video-splicing that can only be used for off-line is used.Therefore, another problem of video-splicing algorithm is how to provide a kind of not only fast but also have the method for higher joining quality.
In short, the step of image mosaic is that an at first definite appropriate mathematic model is described the geometric relationship between the image, promptly determines the transformation of coordinates of a sub-picture to another width of cloth image.According to this mathematical model, adopt the estimation of method for parameter estimation realization to model parameter.According to the model parameter of estimating, the coordinate that source images is transformed to composograph gets on.At the composograph coordinate, get up in the overlapping region between the adjacent source images seamlessly aliasing.This aliasing is wanted to handle parallax, camera lens distortion, scene motion, exposure difference etc.Document 1 (Richard Szeliski.Image Alignment and Stitching:A Tutorial.Foundations and Trends in Computer Vision, Vol.2, No.1, pp.1-104,2006) provide a complete summary to merging algorithm for images.Describe the geometric relationship between the image except adopting parameter model, also can adopt light stream to describe geometric relationship between the image, referring to document 2 (J.Shade, S.Gortler, L.-W.He, and R.Szeliski.Layered depth images.ACM Trans.Graph., pages 231-242,1998).Specifically, set up the correspondence of Pixel-level between the adjacent image exactly.Precision is not high and unstable as a result owing to optical flow computation, therefore need retrain result of calculation and realize image mosaic by layer mode, referring to document 3 (Ke Colin Zheng, Sing Bing Kang, Michael F.Cohen, Richard Szeliski, " Layered Depth Panoramas, " cvpr, pp.1-8,2007IEEE Conference on Computer Vision and Pattern Recognition, 2007).
Video relatively with the rest image life period on correlativity, therefore, the video-splicing method can utilize the temporal correlation between the image to quicken to splice.In the compressed video that adopts motion compensation, owing to contain preliminary piece match information between the motion macro block, this information just can directly be used for the current model parameter of rough calculation, referring to document 4 (TomoyukiShimizu, Akio Yoneyama, Yasuhiro Takishima.A Fast Video StitchingMethod for Motion-Compensated Frames in Compressed Video Streams.Proceeding of International Conference on Consumer Electronics, pp.173-174,2006).Yet such calculation of parameter and out of true also need an extra piece coupling and an overall motion estimation to improve accuracy of parameter estimation.Document 5 (PatrickPan, Tatsumi Mitsushita, Christine Lin, Benjamin Kuo.Optimized VideoStitching Method.US Patent Application Publication, No.US2007/0211934A1 Sep.2007) then utilizes historical information to come the accelerating video splicing.In the system initialization process, the model parameter between the image goes on record, and frame of video splicing subsequently then utilizes the historical models parameter to realize splicing.
The method that two width of cloth image mosaics get up is the simple example of image mosaic.The most important condition of image mosaic is that this two width of cloth image has common picture material, i.e. overlay region.That is to say that partial content is the partial content that comes from the same scene in this two width of cloth image.Just because of the existence of overlapping region, the overlay region of pressing of two images could be merged, two image mosaics become an image the most at last.So merging algorithm for images is exactly to determine two overlapping regions between the image, and under a unified coordinate system, determine the position of two width of cloth images according to the position of overlay region.Geometric relationship between two width of cloth images can represent that by this geometric transformation, the coordinate that piece image is transformed other piece image (reference picture) gets on a geometric transformation.If the target image coordinate is not the coordinate of reference picture, then spliced image can be carried out a coordinate transform according to the reference picture coordinate to the mapping of target image coordinate more again.Under common image coordinate, the image pixel of overlay region has unique coordinate.When describing the mathematical model of geometric transformation between two width of cloth images, a common practice is that scene is approximately a plane.Like this, the geometric relationship between two width of cloth images just can be described with a Homography matrix.If two width of cloth images are respectively I and I ', X=(x, y, 1) and X '=(x ', y ', 1) is respectively the homogeneous coordinates of a pixel among I and the I ', and H is for being mapped to the pixel coordinate among the I ' coordinate transform of pixel among the I, so
X′＝HX′……(1)
Wherein H is the real number matrix of 3x3.Fig. 1 provides the example of this coordinate transform.As can be seen from Figure 1, I ' is through the conversion of H matrix, and variation has taken place shape.The H matrix is one and protects the line mapping, has 8 parameters.The H matrix has following form:
In case determined the H matrix, then only need with I ' image transformation under the coordinate system of I image, just can realize splicing.At some in particular cases, the H matrix has different number of parameters, such as the situation of having only rotation or translation at camera.Determine that the process of parameter value is called parameter estimation among the H, can adopt direct method or based on the method for unique point, referring to document 1.
Yet, in the process of calculating the H matrix, often need through obtaining valuation comparatively accurately longer computing time.On the other hand, because the H matrix often has more parameter, separating of algorithm relatively is difficult to obtain an optimum solution.So a kind of way of equivalence is directly to calculate the translation of two width of cloth correcting images in X-axis behind the image rectification.By image rectification, the image pixel after two width of cloth are proofreaied and correct is corresponding fully in Y-axis.Therefore, the correspondence between the pixel only occurs on the X-axis.This way of afterwards calculating of proofreading and correct is earlier often used in picture depth is calculated.By image rectification, the pixel matching between two width of cloth images only occurs in X-axis, so can greatly reduce the calculated amount of images match.For image mosaic, the plane of delineation after correction has only the translation on the X-axis between two width of cloth images.Therefore, the geometric transformation between the image deteriorates to a simple translation model.
If I
cAnd I
c' be respectively the image after image I and I ' proofread and correct, p
c=(x, y) and p
c'=(x ', y ') is respectively I
cAnd I
c' in the homogeneous coordinates of a pixel, p then
cAnd p
c' pixel between coordinate transform be
Wherein, d
xBe pixel p
c' relative and pixel p on X-axis
cOff-set value.The plane of delineation after correction, image mosaic can be realized by the image translation on X-axis.Fig. 2 has provided the example in the correcting plane splicing.As shown in Figure 2, the coordinate transform of image rectification still can be represented with a Homography matrix, referring to document 6 (R.Hartley, A.Zisserman, Multiple View Geometry in Computer Vision, CambridgeUniversity Press, Mar.2004), H wherein
cAnd H
c' be to be respectively the correction matrix that affacts I and I '.At stitching image that correcting plane obtains is that coordinate after stitching image that correcting plane obtains is to proofread and correct with reference picture is a reference frame with the difference of the stitching image that directly utilizes the H matrix to obtain.So, the inverse matrix effect of spliced image with correction matrix just can be able to be obtained the splicing result shown in Fig. 1 the right.In Fig. 2, D
xOff-set value d for comprehensive all overlay region pixels
xThe global offset value of being calculated is generally got all d
xAverage.In actual computation, if calculate the d of all pixels
x, equally can be very consuming time.Therefore, can select some representational points to calculate d
x, as the SIFT unique point, referring to document 7 (M.A.Brown, Multi-Image Matching Using InvariantFeatures, Ph.D.Dissertation, the University of British Columbia, 2005).Because the existence of noise needs to adopt the method for robust to calculate D
x, as the RANSAC algorithm, referring to document 6.
It is simple relatively to adopt the image split-joint method of parameter model to have an algorithm, the measured characteristics of matter.Therefore this method is widely studied, and emerges a large amount of achievements in research.Yet this method is relatively more responsive to parallax, and ghost image appears in spliced image easily.Adopt the merging algorithm for images of optic flow technique can effectively handle ghost image, but optical flow computation is very consuming time and unstable.The stitching image quality is not high relatively.The optical flow computation layering can be improved the precision and the stability of optical flow computation, but because image segmentation is an ill-conditioning problem equally.Therefore, coarse optical flow computation can cause new vision uncontinuity,, diplopia clear such as blur margin etc.
From the consideration of calculated amount, the video-splicing algorithm is often derived from the merging algorithm for images based on parameter model, therefore also occurs ghost image and tangible piece easily.
Summary of the invention
The purpose of this invention is to provide a kind of image split-joint method and equipment, so that improve the image mosaic quality.
One aspect of the present invention provides a kind of method that image is spliced, and comprises step: determine the piece between first image and second image; The pixel of first image and second image is divided into foreground pixel and background pixel; Splice based on the foreground pixel of described piece first image and second image; And background pixel is out of shape, itself and spliced foreground pixel are merged.
Another aspect of the present invention provides a kind of equipment of stitching image, comprising: registration apparatus, determine the piece between first image and second image; Classification apparatus is divided into foreground pixel and background pixel with the pixel of first image and second image; Splicing apparatus splices based on the foreground pixel of described piece to first image and second image; And anamorphic attachment for cinemascope, background pixel is out of shape, itself and spliced foreground pixel are merged.
Method of the present invention has simple, the fireballing characteristics of algorithm.Owing to adopt layered splicing, and come the splicing of adaptation prospect with background deformation, eliminated the vision inconsistency, joining quality is higher.
Description of drawings
By below in conjunction with description of drawings the preferred embodiments of the present invention, will make above-mentioned and other purpose of the present invention, feature and advantage clearer, wherein:
Fig. 1 shows the example of coordinate transform;
Fig. 2 shows the example in the correcting plane splicing;
Fig. 3 shows the structured flowchart according to the image mosaic device of the embodiment of the invention;
Fig. 4 shows the process flow diagram according to the image split-joint method of the embodiment of the invention;
Fig. 5 shows the example of foreground pixel and background pixel division;
Fig. 6 shows the example that is out of shape based on to the pixel of piece position;
Fig. 7 shows the example that the image more than two width of cloth is spliced;
Fig. 8 shows the example of video-splicing; And
Fig. 9 shows the example of the concordance list that uses in the process of video-splicing.
Embodiment
To a preferred embodiment of the present invention will be described in detail, having omitted in the description process is unnecessary details and function for the present invention with reference to the accompanying drawings, obscures to prevent that the understanding of the present invention from causing.
As mentioned above, therefore existing image split-joint method the inconsistent of content may occur in the piece position owing to there is not the parallax in processing overlapping zone, and ghost image appears in serious meeting.Therefore, must handle the parallax of the inconsistent pixel of horizontal-shift value that obtains with estimation.Handle parallax, just need calculate its light stream each pixel.With light stream is the best approach of handling parallax, but this can cause calculated amount to increase severely.For this reason, present embodiment proposes a kind of simple disposal route and eliminates the parallax effect.
According to embodiments of the invention, image is divided into prospect and two layers of background.Prospect and background are by row splicing respectively.Spliced background layer seamlessly merges with the foreground layer that has spliced by the mode of distortion again.
According to embodiments of the invention, only handle the parallax of piece position pixel, thereby reduced calculated amount.The thought of this method is that other regional vision inconsistency is unconspicuous as long as the parallax of piece position is eliminated.
Fig. 3 shows the structured flowchart according to the image mosaic device of the embodiment of the invention.As shown in Figure 3, this image mosaic device comprises: memory device 111, feature extraction and matching unit 112, correction matrix computing unit 113, estimation unit 114, calculations of offset unit 115, division unit 116, concatenation unit 117 and deformation unit 118.
Store image to be spliced in the memory device 111, had the overlapping region between at least two width of cloth in these images.
Feature extraction and matching unit 112, correction matrix computing unit 113, estimation unit 114 and calculations of offset unit form registration apparatus, and it is used for determining the position of the piece between the image to be spliced.Division unit 116 is divided into foreground pixel and background pixel with the pixel of image to be spliced.The foreground pixel that stapling unit 117 is treated spliced image based on described piece splices.118 pairs of background pixels of deformation unit are out of shape, and itself and spliced foreground pixel are merged.
Correction matrix computing unit 113 in the registration apparatus calculates the correction matrix of image to be spliced so that according to this correction matrix registration image to be spliced, make image on column direction, align, and estimation unit 114 is estimated image to be spliced relative distance at line direction, i.e. horizontal-shift value D
H, so that determine the position of piece.
According to embodiments of the invention, feature extraction and matching unit 112 are treated spliced image and are carried out feature extraction, obtain feature separately, and coupling is at the obtained feature of image to be spliced.Then, determine piece position between the image based on the distance between the unique point of coupling.
According to embodiments of the invention, division unit 116 is divided into foreground pixel and background pixel based on color or gray-scale value with the pixel of piece position.
According to another embodiment, division unit 116 is judged as foreground pixel with the pixel that the relative distance between the pixel of piece position equals the relative distance between two width of cloth images, and other pixels at stitching position place are judged as background pixel.Concatenation unit 117 splices background image according to the piece position.
According to embodiments of the invention, real offset between the pixel of deviation post computing unit 115 calculating piece positions, and 118 pairs of real offset of crushed element are shunk by row greater than the pixel of the relative distance between two width of cloth images, and the pixel of real offset less than the relative distance between two width of cloth images stretched by row.
Fig. 4 shows the process flow diagram according to the image split-joint method of the embodiment of the invention.
At step S11, feature extraction and matching unit 112 are treated spliced image and are carried out feature extraction, obtain feature separately, and coupling is at the obtained feature of image to be spliced.
Then, at step S12, correction matrix computing unit 113 calculates the correction matrix of image to be spliced so that according to this correction matrix registration image to be spliced, make image align on column direction.
At step S13, estimation unit 114 is estimated image to be spliced relative distance at line direction, i.e. horizontal-shift value between two width of cloth images is so that determine the position of piece.
At step S14, the real offset between the pixel of 115 calculating piece positions, calculations of offset unit, just the respective pixel actual shifts value in the horizontal direction of edge joint position on two width of cloth images.
At step S15, division unit 116 is judged as foreground pixel with the pixel that the relative distance between the pixel of piece position equals the relative distance between two images, and other pixel at stitching position place is judged as background pixel.
At step S16, concatenation unit 117 splices background image according to the piece position.
At step S17, crushed element shrinks by row greater than the pixel of the relative distance between two width of cloth images real offset, and the pixel of real offset less than the relative distance between two width of cloth images is stretched by row.
According to embodiments of the invention, at first by feature extraction and matching unit 112 coupling in two source image pixels of piece position calculation.According to the result of piece pixel matching and the D of calculating
H, the piece pixel is divided into foreground pixel and background pixel two classes.If the same D of the coupling of a piece pixel
HDo not meet, then this pixel is called background pixel, otherwise is called foreground pixel.Fig. 5 has provided the example that prospect background is divided.As shown in Figure 5, if the same D of pixel
HConform to, then the pixel of two width of cloth images just is mapped in the piece position fully.If be not inconsistent, then these two pixels can produce a new side-play amount.Under calibration coordinate system, the coupling of piece pixel can be expressed as:
Wherein F and B represent the set of prospect and background pixel respectively.P is the pixels of two width of cloth source images in the piece position, the coordinate that x and x ' are respectively pixel p on correcting image separately.D
H+ Δ d
HBe the real offset of background pixel, the respective pixel of two width of cloth images actual shifts value in the horizontal direction just, and Δ d
HBe called as two width of cloth images in the deviation of the actual shifts value between the pixel on the piece to the relative distance between two width of cloth images.Therefore, can pass through D
HJudgement come differentiation prospect and background pixel.As shown in Figure 5, the white portion of piece is represented background pixel, and gray area is represented foreground pixel.
A visual interpretation to foreground pixel is exactly on the correcting image plane, and the position of this pixel splicing is positioned at exactly by D
HThe piece position of decision.For background pixel, the true stitching position of its pixel is not positioned on the piece.If we still force to use current piece position, then the pixel in the piece position is exactly unmatched pixel, causes the background parts of edge joint position the inconsistent of vision to occur.Therefore, our strategy is according to D
HSplice foreground pixel.The stitching position of background pixel then navigates to piece by distortion the piece position of prospect.The order ground of this strategy is exactly the Δ d that eliminates in the formula (4)
HBy image zoom by row, can be so that the piece of background pixel transforms to the position of prospect piece.For background pixel, Δ d
HTwo kinds of situations are arranged: a kind of is Δ d
HLess than zero, then presentation video needs to amplify; On the contrary, Δ d
HGreater than zero, image need dwindle.Here, according to the division of prospect and background pixel, source images can be divided into different zones in the pixel of piece position by row, these zones are called as prospect and background area.In the background area, just can eliminate the vision inconsistency of background by the mode of row convergent-divergent in the piece position by image.The mode by the row convergent-divergent of this background is called the distortion of background area.Because each provisional capital has the Δ d of oneself
H, this distortion is non-linear.Fig. 6 has provided the example of background deformation.
In sum, the embodiment of layered image joining method is as follows:
(1) unique point in the extraction source images, and the coupling between the calculated characteristics point.
(2) utilize the result of Feature Points Matching, the correction matrix H of computed image
cAnd H
c'.
(3) the correcting image plane is arrived in the unique point coordinate transform.
(4) utilize the unique point of coupling to estimate D on the correcting image plane
H
(5) on the correcting image plane by having estimated D
HDefinite piece position, the piece position is the centre of two picture centres.
(6), recomputate the Δ d of piece position pixel in the piece position
HΔ d
HCalculating can adopt the method for piece coupling to realize.
(7) according to Δ d
HClassification piece pixel.If Δ d
HNear zero, then this pixel is judged as foreground pixel; Otherwise, be background pixel.
(8) according to Δ d
H, stretch or the contraction source image by row, obtain the stitching image under the correcting image coordinate system.
(9) composograph is used
Carry out coordinate transform, just can obtain the stitching image under the I image coordinate system.
The splicing of multiple image comprises two kinds of situations, and first kind of situation is with piece image as a reference, and other image mosaic is to this width of cloth reference picture.Second kind of situation is the adjacent in twos situation that exists between the multiple image.To first kind of situation, joining method is to recycle two width of cloth image split-joint methods, with one one width of cloth of other image be spliced on the reference picture.The rule of doing to second kind of situation is to utilize the topological relation between adjacent two width of cloth images to realize splicing.At first with piece image as the reference image, image mosaic that will be adjacent with this image is to the plane of this image.Then, will splice good image mosaic again goes to its reference picture corresponding reference image.Fig. 7 has provided the example of three width of cloth image mosaics.Wherein, image I " and the reference picture of I ' be respectively image I ' and I.
As shown in Figure 7, at first with I " image mosaic forms M to the plane of delineation of I '
0Then with M
0According to image I ' mode that is spliced to I is spliced to image I, forms stitching image M
1Therefore, multiple image splicing is a space topological structure by image, with image sequence be spliced to the process on final reference picture plane.
Because to the requirement of real-time, video-splicing needs frame joining method fast.The joining method of the embodiment of the invention is fit to expand to video-splicing and uses very much.At first, the parameter estimation of the correction transformation matrix of image can be finished at system initialization.In addition, the coordinate transform of image can be quickened by index of reference, as shown in Figure 9.The method of present embodiment can be combined closely with index.When forming stitching image, source images only needs a translation just can finish (seeing formula 4) with the renewal of the pixel correspondence between the stitching image.The distortion of background also only can produce a translational movement to each background area pixels.Therefore, algorithm is to be used for actual pixel coordinate conversion with linear additive operation, and this has greatly improved the speed of frame splicing.In addition, feature point detection and Feature Points Matching also are two calculating consuming time.In the video-splicing algorithm, we adopt different prospect background judgment modes, have avoided this two step to calculate, and have further reduced calculated amount.
Under the situation of video, the joining method of the embodiment of the invention is divided into two stages: initialization and frame splicing.At initial phase, as step S21, S22, S23, S24 and S25, used all coordinate transforms of algorithm computation image mosaic, i.e. image rectification conversion and D
HIn the frame splicing stage, as step S31, S32, S33, S34, S35 and S36, at first be, and then divide prospect and background according to the result of pixel matching in original piece position calculation pixel matching.Fig. 8 has provided the block diagram of video-splicing algorithm.
Video-splicing can not done actual splicing at initial phase, and only the data that the subsequent frame splicing need be used obtain just passable.The main difference of video-splicing and image mosaic is in the splicing of subsequent frame.Subsequent frame splicing will utilize operation result in the initialization procedure to reduce the computing of self as far as possible.Therefore, the subsequent frame splicing is just in piece position calculation pixel matching, and the judgement of realization prospect background.The background here is redefined.Background pixel is defined as with initial phase and has consistent D
HThose pixels, foreground pixel then is defined as Δ d
HNon-vanishing pixel.The splicing of subsequent frame is the new D of direct estimation not
HValue D
H'.D
H' be the Δ d that obtains by the foreground pixel coupling
HAverage
Estimation:
Prospect splicing subsequently, background deformation and usefulness
The spliced image of contrary correction all is equal to and previous image split-joint method.
Top description only is used to realize embodiments of the present invention; it should be appreciated by those skilled in the art; the any modification or partial replacement that is not departing from the scope of the present invention; all should belong to claim of the present invention and come restricted portion; therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.
Claims (20)
1. method that image is spliced comprises step:
Determine the piece between first image and second image;
The pixel of first image and second image is divided into foreground pixel and background pixel;
Splice based on the foreground pixel of described piece first image and second image; And
Background pixel is out of shape, itself and spliced foreground pixel are merged.
2. the method for claim 1, determine that wherein the step of the piece between first image and second image comprises:
Registration first image and second image align win image and second image on column direction;
Estimate first image and the relative distance of second image on line direction, to determine the piece of first image and second image.
3. method as claimed in claim 2, wherein said estimating step comprises:
First image and second image are carried out feature extraction, obtain feature separately;
Coupling is at obtained feature of first image and the feature that obtains at described second image;
Determine the piece of first image and second image based on the distance between the unique point of coupling.
4. the method for claim 1, wherein said partiting step comprises:
Based on color or gray-scale value the pixel of edge joint position is divided into foreground pixel and background pixel.
5. method as claimed in claim 2, the pixel that wherein relative distance between the edge joint position pixel is equaled the relative distance between first image and second image is judged as foreground pixel, and other pixels of edge joint position are judged as background pixel.
6. as claim 4 or 5 described methods, wherein said deforming step comprises:
Real offset between the pixel of calculating edge joint position;
Real offset is shunk by row greater than the pixel of described relative distance, and the pixel of real offset less than described relative distance stretched by row.
7. method as claimed in claim 2, wherein
Splice first image image and second image image afterwards afterwards based on described piece.
8. method as claimed in claim 7 also comprises:
At image after first image and the real offset between the image calculation edge joint position pixel after second image;
Upgrade described relative distance based on described real offset.
9. method as claimed in claim 8, wherein said step of updating comprises:
Calculate the average of the real offset between the edge joint position pixel;
With described average and described relative distance addition, upgrade described relative distance.
10. method as claimed in claim 8 also comprises:
The index of relative position of the pixel of edge joint position has been write down in formation;
Come image after first image and the image after second image are spliced by means of described index.
11. the equipment of a stitching image comprises:
Registration apparatus is determined the piece between first image and second image;
Classification apparatus is divided into foreground pixel and background pixel with the pixel of first image and second image;
Splicing apparatus splices based on the foreground pixel of described piece to first image and second image; And
Anamorphic attachment for cinemascope is out of shape background pixel, and itself and spliced foreground pixel are merged.
12. equipment as claimed in claim 11, wherein registration apparatus comprises:
Registration unit, registration first image and second image align win image and second image on column direction;
Estimation unit is estimated first image and the relative distance of second image on line direction, to determine the piece of first image and second image.
13. equipment as claimed in claim 12, wherein said estimation unit comprises:
Feature extraction unit is carried out feature extraction to first image and second image, obtains feature separately;
The characteristic matching unit, coupling is at obtained feature of first image and the feature that obtains at described second image;
Wherein, determine the piece of first image and second image based on the distance between the unique point of coupling.
14. equipment as claimed in claim 11, wherein said classification apparatus is divided into foreground pixel and background pixel based on color or gray-scale value with the pixel of edge joint position.
15. equipment as claimed in claim 12, wherein classification apparatus is judged as foreground pixel with the pixel that the relative distance between the edge joint position pixel equals the relative distance between first image and second image, and other pixels of edge joint position are judged as background pixel.
16. as claim 14 or 15 described equipment, also comprise computing unit, the real offset between the pixel of calculating edge joint position; Wherein, deformation unit shrinks by row greater than the pixel of described relative distance real offset, and the pixel of real offset less than described relative distance stretched by row.
17. equipment as claimed in claim 12 wherein splices first image image and second image image afterwards afterwards based on described piece.
18. equipment as claimed in claim 17 also comprises: computing unit, at image after first image and the real offset between the image calculation edge joint position pixel after second image;
Described equipment also comprises updating device, upgrades described relative distance based on described real offset.
19. equipment as claimed in claim 18, wherein said computing unit calculates the average of the real offset between the edge joint position pixel, and updating block upgrades described relative distance with described average and described relative distance addition.
20. equipment as claimed in claim 18 also comprises: storer, stored record the index of relative position of pixel of edge joint position;
Wherein, come image after first image and the image after second image are spliced by means of described index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910149346 CN101923709B (en) | 2009-06-16 | 2009-06-16 | Image splicing method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910149346 CN101923709B (en) | 2009-06-16 | 2009-06-16 | Image splicing method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101923709A true CN101923709A (en) | 2010-12-22 |
| CN101923709B CN101923709B (en) | 2013-06-26 |
Family
ID=43338619
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910149346 Expired - Fee Related CN101923709B (en) | 2009-06-16 | 2009-06-16 | Image splicing method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101923709B (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102427504A (en) * | 2011-10-08 | 2012-04-25 | 广东威创视讯科技股份有限公司 | Image processing method, device and system based on background splicing wall |
| CN102508628A (en) * | 2011-10-08 | 2012-06-20 | 广东威创视讯科技股份有限公司 | Method and device for eliminating splicing seams of splicing wall as well as image system based on splicing wall |
| CN102915526A (en) * | 2012-09-18 | 2013-02-06 | 合肥工业大学 | Spherical panorama missed information restoring method based on complementary scenario model |
| CN103024416A (en) * | 2011-12-14 | 2013-04-03 | 微软公司 | Parallax compensation |
| CN103903213A (en) * | 2012-12-24 | 2014-07-02 | 联想(北京)有限公司 | Shooting method and electronic device |
| CN104574493A (en) * | 2014-12-30 | 2015-04-29 | 北京像素软件科技股份有限公司 | Smooth distant view fade-out method and device |
| CN104794701A (en) * | 2014-01-21 | 2015-07-22 | 富士通株式会社 | Image splicing device, method and image processing equipment |
| CN104978428A (en) * | 2015-07-10 | 2015-10-14 | 无锡天脉聚源传媒科技有限公司 | Method and device for displaying abstract pictures of resources |
| WO2015165222A1 (en) * | 2014-04-29 | 2015-11-05 | 华为技术有限公司 | Method and device for acquiring panoramic image |
| CN105243119A (en) * | 2015-09-29 | 2016-01-13 | 百度在线网络技术（北京）有限公司 | Determination of to-be-superimposed region of image, superimposition of images and image display method and apparatus |
| CN105381611A (en) * | 2015-11-19 | 2016-03-09 | 网易（杭州）网络有限公司 | Method and device for layered three-dimensional display of 2D game scene |
| US9324184B2 (en) | 2011-12-14 | 2016-04-26 | Microsoft Technology Licensing, Llc | Image three-dimensional (3D) modeling |
| US9406153B2 (en) | 2011-12-14 | 2016-08-02 | Microsoft Technology Licensing, Llc | Point of interest (POI) data positioning in image |
| CN105931188A (en) * | 2016-05-06 | 2016-09-07 | 安徽伟合电子科技有限公司 | Method for image stitching based on mean value duplication removal |
| CN106296577A (en) * | 2015-05-19 | 2017-01-04 | 富士通株式会社 | Image split-joint method and image mosaic device |
| CN106651755A (en) * | 2016-11-17 | 2017-05-10 | 宇龙计算机通信科技(深圳)有限公司 | Panoramic image processing method and device for terminal and terminal |
| WO2017088533A1 (en) * | 2015-11-26 | 2017-06-01 | 华为技术有限公司 | Method and apparatus for merging images |
| CN106934772A (en) * | 2017-03-02 | 2017-07-07 | 深圳岚锋创视网络科技有限公司 | A kind of horizontal alignment method of panoramic picture or video, system and portable terminal |
| CN107645643A (en) * | 2017-10-10 | 2018-01-30 | 成都学知乐科技有限公司 | Suitable for the image recording system of a variety of teaching environments |
| CN108279857A (en) * | 2017-12-29 | 2018-07-13 | 广东创捷智能科技有限公司 | The liquid crystal-spliced screen of zero piece of integrated pixel filling based on HDbaseT signal transmissions |
| US10038842B2 (en) | 2011-11-01 | 2018-07-31 | Microsoft Technology Licensing, Llc | Planar panorama imagery generation |
| WO2018157350A1 (en) * | 2017-03-02 | 2018-09-07 | 深圳岚锋创视网络科技有限公司 | Horizontal calibration method and system for panoramic image or video, and portable terminal |
| CN108921781A (en) * | 2018-05-07 | 2018-11-30 | 清华大学深圳研究生院 | A kind of light field joining method based on depth |
| CN109996010A (en) * | 2017-12-29 | 2019-07-09 | 深圳市优必选科技有限公司 | A kind of method for processing video frequency, device, smart machine and storage medium |
| CN105554441B (en) * | 2014-10-23 | 2019-11-19 | 韩华泰科株式会社 | For being registrated the device and method of image |
| CN110969575A (en) * | 2018-09-30 | 2020-04-07 | 多方科技(广州)有限公司 | Self-adaptive image splicing method and image processing device |
| CN112614051A (en) * | 2020-12-08 | 2021-04-06 | 上海裕芮信息技术有限公司 | Building facade image splicing method, system, equipment and storage medium |
| CN112710698A (en) * | 2020-12-08 | 2021-04-27 | 上海裕芮信息技术有限公司 | Method and system for detecting refinery tower torch equipment based on infrared thermal imaging |
| CN114519671A (en) * | 2022-02-16 | 2022-05-20 | 天津中科无人机应用研究院 | Unmanned aerial vehicle remote sensing image dynamic rapid splicing method |
| CN115829843A (en) * | 2023-01-09 | 2023-03-21 | 深圳思谋信息科技有限公司 | Image splicing method and device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101383002B (en) * | 2008-10-27 | 2010-09-08 | 北京海鑫科金高科技股份有限公司 | Finger print collecting and joining method supporting arbitrary directional rolling |
-
2009
- 2009-06-16 CN CN 200910149346 patent/CN101923709B/en not_active Expired - Fee Related
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102427504B (en) * | 2011-10-08 | 2014-12-24 | 广东威创视讯科技股份有限公司 | Image processing method, device and system based on background splicing wall |
| CN102508628A (en) * | 2011-10-08 | 2012-06-20 | 广东威创视讯科技股份有限公司 | Method and device for eliminating splicing seams of splicing wall as well as image system based on splicing wall |
| CN102427504A (en) * | 2011-10-08 | 2012-04-25 | 广东威创视讯科技股份有限公司 | Image processing method, device and system based on background splicing wall |
| CN102508628B (en) * | 2011-10-08 | 2014-12-24 | 广东威创视讯科技股份有限公司 | Method and device for eliminating splicing seams of splicing wall as well as image system based on splicing wall |
| US10038842B2 (en) | 2011-11-01 | 2018-07-31 | Microsoft Technology Licensing, Llc | Planar panorama imagery generation |
| CN103024416A (en) * | 2011-12-14 | 2013-04-03 | 微软公司 | Parallax compensation |
| CN103024416B (en) * | 2011-12-14 | 2015-06-17 | 微软公司 | Parallax compensation |
| US10008021B2 (en) | 2011-12-14 | 2018-06-26 | Microsoft Technology Licensing, Llc | Parallax compensation |
| US9406153B2 (en) | 2011-12-14 | 2016-08-02 | Microsoft Technology Licensing, Llc | Point of interest (POI) data positioning in image |
| US9324184B2 (en) | 2011-12-14 | 2016-04-26 | Microsoft Technology Licensing, Llc | Image three-dimensional (3D) modeling |
| CN102915526B (en) * | 2012-09-18 | 2014-12-17 | 合肥工业大学 | Spherical panorama missed information restoring method based on complementary scenario model |
| CN102915526A (en) * | 2012-09-18 | 2013-02-06 | 合肥工业大学 | Spherical panorama missed information restoring method based on complementary scenario model |
| CN103903213A (en) * | 2012-12-24 | 2014-07-02 | 联想(北京)有限公司 | Shooting method and electronic device |
| CN104794701A (en) * | 2014-01-21 | 2015-07-22 | 富士通株式会社 | Image splicing device, method and image processing equipment |
| CN105096283A (en) * | 2014-04-29 | 2015-11-25 | 华为技术有限公司 | Panoramic image acquisition method and device |
| WO2015165222A1 (en) * | 2014-04-29 | 2015-11-05 | 华为技术有限公司 | Method and device for acquiring panoramic image |
| CN105096283B (en) * | 2014-04-29 | 2017-12-15 | 华为技术有限公司 | The acquisition methods and device of panoramic picture |
| CN105554441B (en) * | 2014-10-23 | 2019-11-19 | 韩华泰科株式会社 | For being registrated the device and method of image |
| CN104574493B (en) * | 2014-12-30 | 2018-01-30 | 北京像素软件科技股份有限公司 | The method and device that a kind of distant view smoothly fades out |
| CN104574493A (en) * | 2014-12-30 | 2015-04-29 | 北京像素软件科技股份有限公司 | Smooth distant view fade-out method and device |
| CN106296577A (en) * | 2015-05-19 | 2017-01-04 | 富士通株式会社 | Image split-joint method and image mosaic device |
| CN106296577B (en) * | 2015-05-19 | 2019-11-29 | 富士通株式会社 | Image split-joint method and image mosaic device |
| CN104978428A (en) * | 2015-07-10 | 2015-10-14 | 无锡天脉聚源传媒科技有限公司 | Method and device for displaying abstract pictures of resources |
| CN105243119A (en) * | 2015-09-29 | 2016-01-13 | 百度在线网络技术（北京）有限公司 | Determination of to-be-superimposed region of image, superimposition of images and image display method and apparatus |
| CN105243119B (en) * | 2015-09-29 | 2019-05-24 | 百度在线网络技术（北京）有限公司 | Determine region to be superimposed, superimposed image, image presentation method and the device of image |
| CN105381611A (en) * | 2015-11-19 | 2016-03-09 | 网易（杭州）网络有限公司 | Method and device for layered three-dimensional display of 2D game scene |
| WO2017088533A1 (en) * | 2015-11-26 | 2017-06-01 | 华为技术有限公司 | Method and apparatus for merging images |
| CN105931188A (en) * | 2016-05-06 | 2016-09-07 | 安徽伟合电子科技有限公司 | Method for image stitching based on mean value duplication removal |
| CN106651755A (en) * | 2016-11-17 | 2017-05-10 | 宇龙计算机通信科技(深圳)有限公司 | Panoramic image processing method and device for terminal and terminal |
| WO2018090455A1 (en) * | 2016-11-17 | 2018-05-24 | 宇龙计算机通信科技(深圳)有限公司 | Method and device for processing panoramic image of terminal, and terminal |
| CN106934772A (en) * | 2017-03-02 | 2017-07-07 | 深圳岚锋创视网络科技有限公司 | A kind of horizontal alignment method of panoramic picture or video, system and portable terminal |
| US11388336B2 (en) | 2017-03-02 | 2022-07-12 | Arashi Vision Inc. | Horizontal calibration method and system for panoramic image or video, and portable terminal |
| WO2018157350A1 (en) * | 2017-03-02 | 2018-09-07 | 深圳岚锋创视网络科技有限公司 | Horizontal calibration method and system for panoramic image or video, and portable terminal |
| CN106934772B (en) * | 2017-03-02 | 2019-12-20 | 深圳岚锋创视网络科技有限公司 | Horizontal calibration method and system for panoramic image or video and portable terminal |
| CN107645643A (en) * | 2017-10-10 | 2018-01-30 | 成都学知乐科技有限公司 | Suitable for the image recording system of a variety of teaching environments |
| CN109996010A (en) * | 2017-12-29 | 2019-07-09 | 深圳市优必选科技有限公司 | A kind of method for processing video frequency, device, smart machine and storage medium |
| CN108279857A (en) * | 2017-12-29 | 2018-07-13 | 广东创捷智能科技有限公司 | The liquid crystal-spliced screen of zero piece of integrated pixel filling based on HDbaseT signal transmissions |
| CN108921781A (en) * | 2018-05-07 | 2018-11-30 | 清华大学深圳研究生院 | A kind of light field joining method based on depth |
| CN108921781B (en) * | 2018-05-07 | 2020-10-02 | 清华大学深圳研究生院 | Depth-based optical field splicing method |
| CN110969575A (en) * | 2018-09-30 | 2020-04-07 | 多方科技(广州)有限公司 | Self-adaptive image splicing method and image processing device |
| CN110969575B (en) * | 2018-09-30 | 2023-06-27 | 多方科技(广州)有限公司 | Adaptive image stitching method and image processing device |
| CN112710698A (en) * | 2020-12-08 | 2021-04-27 | 上海裕芮信息技术有限公司 | Method and system for detecting refinery tower torch equipment based on infrared thermal imaging |
| CN112614051A (en) * | 2020-12-08 | 2021-04-06 | 上海裕芮信息技术有限公司 | Building facade image splicing method, system, equipment and storage medium |
| CN114519671A (en) * | 2022-02-16 | 2022-05-20 | 天津中科无人机应用研究院 | Unmanned aerial vehicle remote sensing image dynamic rapid splicing method |
| CN115829843A (en) * | 2023-01-09 | 2023-03-21 | 深圳思谋信息科技有限公司 | Image splicing method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101923709B (en) | 2013-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101923709B (en) | Image splicing method and equipment | |
| US11240471B2 (en) | Road vertical contour detection | |
| CN101908230B (en) | Regional depth edge detection and binocular stereo matching-based three-dimensional reconstruction method | |
| Zhu et al. | Parallel-perspective stereo mosaics | |
| CN105005964A (en) | Video sequence image based method for rapidly generating panorama of geographic scene | |
| CN106981078A (en) | Sight bearing calibration, device, intelligent meeting terminal and storage medium | |
| US20080199083A1 (en) | Image filling methods | |
| CN103258321A (en) | Image stitching method | |
| CN102413342A (en) | Geometrical correction method and system of binocular stereoscopic video | |
| Linger et al. | Aerial image registration for tracking | |
| US20140147031A1 (en) | Disparity Estimation for Misaligned Stereo Image Pairs | |
| CN113192183A (en) | Real scene three-dimensional reconstruction method and system based on oblique photography and panoramic video fusion | |
| CN102081796B (en) | Image splicing method and device thereof | |
| CN109600667B (en) | Video redirection method based on grid and frame grouping | |
| Qin et al. | Image stitching by feature positioning and seam elimination | |
| Li et al. | Fast multicamera video stitching for underwater wide field-of-view observation | |
| EP2595393B1 (en) | Rectified stereoscopic 3d panoramic picture | |
| US20120154388A1 (en) | Stereo image processing method, stereo image processing device and display device | |
| CN114913064A (en) | Large parallax image splicing method and device based on structure keeping and many-to-many matching | |
| CN109379577B (en) | Video generation method, device and equipment of virtual viewpoint | |
| KR100434877B1 (en) | Method and apparatus for tracking stereo object using diparity motion vector | |
| Zhu et al. | Theory and practice in making seamless stereo mosaics from airborne video | |
| Zhu et al. | Error characteristics of parallel-perspective stereo mosaics | |
| Uchino et al. | Enhanced integral imaging 3D depth map by using SRCNN | |
| Zhu et al. | Generation and Error Characterization of Pararell-Perspective Stereo Mosaics from Real Video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130626 Termination date: 20180616 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |