CN102186004A - Difference-based multi-viewpoint video image correcting method - Google Patents
Difference-based multi-viewpoint video image correcting method Download PDFInfo
- Publication number
- CN102186004A CN102186004A CN 201110121562 CN201110121562A CN102186004A CN 102186004 A CN102186004 A CN 102186004A CN 201110121562 CN201110121562 CN 201110121562 CN 201110121562 A CN201110121562 A CN 201110121562A CN 102186004 A CN102186004 A CN 102186004A
- Authority
- CN
- China
- Prior art keywords
- viewpoint
- difference
- video image
- correction
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a difference-based multi-viewpoint video image correcting method. The method comprises the following two parts: calculating global offset of a video image between viewpoints from a matching relationship of a characteristic point set; and compensating the video image between the viewpoints according to the global offset, and correcting video images of other viewpoints by utilizing difference iteration. The image correcting method provided by the method avoids errors of the correcting algorithm caused by the global offset because the global offset among the viewpoints is compensated, and the difference optimization can reduce the original relationship of colors and brightness of the images among the viewpoints.
Description
Technical field
The present invention relates to technical field of video processing, particularly relate to multi-viewpoint video image correction method based on difference.
Background technology
Fast development and extensive use along with digital video technology, people are more and more higher to the requirement of video quality and form, traditional two dimensional surface video can not satisfy people to scene demand true and that reproduce naturally, so can provide the solid/multi-view point video technology of third dimension and interactive operation function more and more to be subjected to the attention of industry.In the multi-view image acquisition process, because the electrical characteristic difference of camera, the locational camera collection color of image of different points of view value difference is very not big.At present existing image rectification technology and patented inventions: as application number is that the patent application document of CN200710133272.9 has been introduced a kind of light source colour and calculated and method for correcting image.The method utilizes principle of reflection to add up high light and backlight zone, converts three dimensions to two-dimensional space, utilizes the parameter approximatioss that the point in the contrary Strength Space is carried out fitting a straight line, thereby carry out color correction on normalized image.Application number is that the patent application document of CN200810035370.3 has been introduced image correction apparatus and bearing calibration.The method comprises a level parameter calculating module, row level parameter calculating module, several sections such as Pixel-level parameter calculating module and multiphase filter, the pixel value behind the final output calibration.Application number has been introduced a kind of 3-D view correction, demonstration and reclaim equiment and method and image for the application documents of CN200910166109.1 system is provided.Application number is that the application documents of CN200910212825.9 have been introduced a kind of method that realizes binocular stereo image correction and display optimization.Its advantage is for the multi-view image that adopts amateur equipment or method to take, can be to the binocular image that wherein need show arbitrarily to proofreading and correct.
But in the time of a plurality of camera capture video data, because the electrical characteristic difference of camera can cause the difference of video image in color and brightness.
Summary of the invention
The technical solution used in the present invention is: at first estimate the global offset amount between the video image between different points of view, utilize difference optimization to realize the correction of video image between many viewpoints then.
Concrete steps of the present invention are:
(1) overall video offset estimation;
(2) multi-viewpoint video image of difference optimization is proofreaied and correct.
The overall video offset estimation of described step (1): at first utilize the Harris method to extract the picture frame of reference view
With the picture frame of proofreading and correct viewpoint
Pairing feature point set is then set up the matching relationship of feature point set, utilizes the matching relationship between the feature point set at last, calculates the side-play amount in the 2 parameter translation model.
Reference view is the viewpoint that is positioned at the centre position in a plurality of viewpoints, and other viewpoint is for proofreading and correct viewpoint.
The multi-viewpoint video image that described step (2) difference is optimized is proofreaied and correct, and its concrete steps are: set up the following criterion of proofreading and correct:
Minimize:
Wherein
Expression global motion side-play amount, promptly
, | ▽ I
c|
2This shows the image smoothness constraint spatially after the correction.
This shows the smoothness constraint of the deviation of correction viewpoint and reference view in the space.The correction criterion is carried out Euler find the solution, promptly
Right
Local derviation can get:
Utilize the calculus of variations to get:
Repeatedly adopt said process upgrade to proofread and correct the video image of viewpoint, and the iteration termination condition is the
Inferior iteration correction visual point image frame and
The difference of inferior iteration correction visual point image frame is less than threshold value A, and setting threshold A of the present invention is 0.1, and final correction visual point image frame is exactly the correction chart picture frame after optimizing through difference.
Described matching relationship need be set a match window, and window is long and wide to be respectively 33 pixels and 33 pixels, compares by cross covariance value and threshold value B with feature point set, could determine matching relationship when the cross covariance value greater than threshold value, and threshold value B is set at 16.
Described side-play amount is the translation parameters of 2 parameters, and 2 parameters correspond respectively to the translational movement of the image transverse axis and the longitudinal axis.
Compare with existing video image correction method, the present invention has the following advantages:
(1), the present invention estimates the global offset between the different points of view, avoided correcting algorithm because the caused error of global offset;
(2), difference optimization of the present invention can best reduction viewpoint video image original attribute, thereby make the image between various viewpoints have the most similar color and brightness relationship;
(3), the present invention can provide the signal source with same color and brightness relationship for follow-up multiple view video coding, make that multiple view video coding can more efficient compression multi-view point video data.
Description of drawings
Fig. 1 is a system flow block diagram of the present invention;
Fig. 2 (a) is the original reference view image of video sequence of the present invention;
Fig. 2 (b) is the original visual point image to be corrected of video sequence of the present invention;
Fig. 2 (c) is the visual point image after video sequence of the present invention is proofreaied and correct.
Embodiment
The present invention is described further below in conjunction with accompanying drawing, implementing the used video capture device of the present invention is 8 CCD(Charge-coupled Device of parallel arranged, charge coupled cell) there is global offset in camera between the video image that each camera is taken.Consider that middle CCD camera and the deviation ratio between other cameras are average, and deviation is less, help follow-up estimation of deviation, so the video image that middle selected CCD camera is taken is the video image of reference view, the video image that other CCD camera is taken is for proofreading and correct the video image of viewpoint.
General thought of the present invention is, at first the video image of correction viewpoint and the video image of reference view are carried out the estimation of global offset, then the global offset of the video image of the video image of compensation correction viewpoint and reference view utilizes difference optimization to realize proofreading and correct the color and the luminance difference of viewpoint video image at last.
Because locational difference between the camera, so the image of taking the photograph exists overall deviation on the position, this deviation also is to cause the brightness of institute's images acquired between the camera and the reason of colourity difference, the global offset amount that for this reason needs the video image of the video image of calculation correction viewpoint and reference view, specific implementation is: (concrete Harris method please refer to document Mikolajcyk to utilize the Harris method, K. and Schmid, C. 2002. An affine invariant interest point detector. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada) picture frame of extraction reference view
With the picture frame of proofreading and correct viewpoint
Pairing feature point set (
With
),
It is the picture frame of reference view
The x of middle characteristic point correspondence and the coordinate position of y axle.
It is the picture frame of proofreading and correct viewpoint
The x of middle characteristic point correspondence and the coordinate position of y axle.
Be the characteristic point in the reference view,
It is the characteristic point of proofreading and correct in the viewpoint.In the hunting zone
The middle characteristic point of calculating
And characteristic point
Between cross covariance
, the concrete computing formula of cross covariance is:
Wherein,
With
It is respectively the reference view picture frame
With correction visual point image frame
The average of institute's correspondence position.
In order to guarantee the reliability of matching relationship, must guarantee the cross covariance that calculates
Greater than threshold value B, promptly
Threshold value B of the present invention is set at 16.After setting up Harris Feature Points Matching relation, estimate 2 parameter translation model (
) in side-play amount, wherein
Be the side-play amounts of corresponding points at x axle and y axle.
Through behind the overall deviation compensation, still also there is deviation in image between viewpoint, this deviation is because the difference of the electric property of various cameras causes, in order further to eliminate the deviation that is caused by the camera electric property, design a kind of difference optimization method, specific implementation is: in proofreading and correct visual point image and reference view image, make that the difference of corresponding position is little, and proofread and correct viewpoint and on its space, keep level and smooth, the difference of proofreading and correct viewpoint and reference view also needs spatially to keep level and smooth, based on top consideration, the following criterion of proofreading and correct of design:
Minimize:
Formula (2)
Wherein
Expression global motion side-play amount, promptly
,
Be reference view lens image frame and correction viewpoint lens image frame
,
Show the spatially level and smooth constraint of image after the correction.
Show the spatially level and smooth constraint of difference between the image of proofreading and correct viewpoint and reference view.
It is the gradient computing.The correction criterion is carried out Euler find the solution, promptly
Right
Local derviation can get:
For can be to behind the local derviation
Find the solution, adopt the calculus of variations, convert minimization problem to iterative problem, can obtain:
Wherein
Be
Inferior iteration correction visual point image frame,
Be
Inferior iteration correction visual point image frame, the primary iteration value
Be set at original correction visual point image frame
Adopt above iterative process to upgrade the video image of proofreading and correct viewpoint, until the always
Inferior iteration correction visual point image frame and
The difference of inferior iteration correction visual point image frame is less than threshold value A, and this moment, iterative process finished.Setting threshold A of the present invention is 0.1, and final correction visual point image frame is exactly the correction chart picture frame after optimizing through difference.
The present invention adopts the standard multiview sequence of 3 dimension audio frequency and video expert groups suggestion to verify the performance of the light tone degree correcting algorithm that is proposed.The size of video sequence is 640 * 480.It realizes effect as shown in Figure 2, Fig. 2 (a) be video sequence original be the reference view image, Fig. 2 (b) be video sequence original be visual point image to be corrected, Fig. 2 (c) is the visual point image after video sequence is proofreaied and correct.From calibration result, between visual point image after the correction and the reference view image, aspect tone, be consistent.
Claims (7)
1. based on the multi-viewpoint video image correction method of difference, it is characterized in that: concrete steps are as follows:
(1) overall video offset estimation:
(2) multi-viewpoint video image of difference optimization is proofreaied and correct:
Set up the following criterion of proofreading and correct earlier:
Minimize:
The correction criterion is carried out Euler to be found the solution; Adopt the calculus of variations then, convert minimization problem to iterative problem, obtain calculus of variations formula:
Repeatedly adopt said process upgrade to proofread and correct the video image of viewpoint, until the
Inferior iteration correction visual point image frame and
The difference of inferior iteration correction visual point image frame is less than threshold value A, and iteration finishes, and the visual point image frame that correction of a final proof obtains is exactly the correction chart picture frame after optimizing through difference.
2. the multi-viewpoint video image correction method based on difference according to claim 1 is characterized in that: the detailed process of the overall video offset estimation of described step (1) is: utilize the Harris angular-point detection method to extract the picture frame of reference view
With the picture frame of proofreading and correct viewpoint
Pairing feature point set, set up the matching relationship of feature point set then, utilize the matching relationship between the feature point set again, estimate the side-play amount of the translation model of reference view and correction viewpoint, obtain proofreading and correct the global offset amount between visual point image and the reference view image, all location of pixels deduct the global offset amount on the positive viewpoint of high-ranking officers, just obtain new correction viewpoint location of pixels, thereby finish overall video offset estimation.
3. according to the described multi-viewpoint video image correction method based on difference of claim 2, it is characterized in that: the matching relationship between the described feature point set is in a match window, and cross covariance value and threshold value B by feature point set relatively obtain.
4. according to the described multi-viewpoint video image correction method of claim 3, it is characterized in that based on difference: the length of described match window and wide be respectively 33 pixels and 33 pixels, threshold value B is 16.
5. according to the described multi-viewpoint video image correction method based on difference of claim 4, it is characterized in that: described side-play amount is the translation parameters of 2 parameters, and 2 parameters correspond respectively to the translational movement of the image transverse axis and the longitudinal axis.
6. according to the described multi-viewpoint video image correction method based on difference of claim 5, it is characterized in that: described reference view is the viewpoint that is positioned at the centre position in many viewpoints; Proofreading and correct viewpoint is the viewpoint that is positioned at other position in many viewpoints.
7. according to the described multi-viewpoint video image correction method based on difference of claim 6, it is characterized in that: described threshold value A is 0.1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101215628A CN102186004B (en) | 2011-05-12 | 2011-05-12 | Difference-based multi-viewpoint video image correcting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101215628A CN102186004B (en) | 2011-05-12 | 2011-05-12 | Difference-based multi-viewpoint video image correcting method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102186004A true CN102186004A (en) | 2011-09-14 |
| CN102186004B CN102186004B (en) | 2012-10-31 |
Family
ID=44572069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011101215628A Expired - Fee Related CN102186004B (en) | 2011-05-12 | 2011-05-12 | Difference-based multi-viewpoint video image correcting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102186004B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006011945A2 (en) * | 2004-06-24 | 2006-02-02 | Visiongate, Inc. | Method for correction of relative object-detector motion between successive views |
| CN101026675A (en) * | 2006-02-24 | 2007-08-29 | 佛山市顺德区顺达电脑厂有限公司 | Image correcting system and method for image access device |
| CN101321230A (en) * | 2008-03-31 | 2008-12-10 | 逐点半导体（上海）有限公司 | Image correction device and correction method |
| JP2009244858A (en) * | 2008-03-11 | 2009-10-22 | Canon Inc | Image capturing apparatus and image processing method |
-
2011
- 2011-05-12 CN CN2011101215628A patent/CN102186004B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006011945A2 (en) * | 2004-06-24 | 2006-02-02 | Visiongate, Inc. | Method for correction of relative object-detector motion between successive views |
| CN101026675A (en) * | 2006-02-24 | 2007-08-29 | 佛山市顺德区顺达电脑厂有限公司 | Image correcting system and method for image access device |
| JP2009244858A (en) * | 2008-03-11 | 2009-10-22 | Canon Inc | Image capturing apparatus and image processing method |
| CN101321230A (en) * | 2008-03-31 | 2008-12-10 | 逐点半导体（上海）有限公司 | Image correction device and correction method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102186004B (en) | 2012-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102857739B (en) | Distributed overall view monitoring system and method thereof | |
| Kang et al. | An efficient image rectification method for parallel multi-camera arrangement | |
| CN101933335B (en) | Method and system for converting 2d image data to stereoscopic image data | |
| CN101179745A (en) | Preprocessing method of multi-viewpoint image | |
| WO2011014419A1 (en) | Methods, systems, and computer-readable storage media for creating three-dimensional (3d) images of a scene | |
| CN107545586B (en) | Depth obtaining method and system based on light field polar line plane image local part | |
| CN108886598A (en) | The compression method and device of panoramic stereoscopic video system | |
| CN111027415B (en) | Vehicle detection method based on polarization image | |
| KR101129713B1 (en) | A multi-view image illumination compensation method based on the histogram matching by using image depth and the recording medium thereof | |
| CN202841372U (en) | Distribution type full-view monitoring system | |
| CN104185004A (en) | Image processing method and image processing system | |
| CN114697623A (en) | Projection surface selection and projection image correction method and device, projector and medium | |
| CN102223545B (en) | Rapid multi-view video color correction method | |
| US9208549B2 (en) | Method and apparatus for color transfer between images | |
| US20120133735A1 (en) | Occlusion layer extension | |
| CN105791795A (en) | Three-dimensional image processing method and device and three-dimensional video display device | |
| CN107330856B (en) | Panoramic imaging method based on projective transformation and thin plate spline | |
| CN101945299A (en) | Camera-equipment-array based dynamic scene depth restoring method | |
| KR20130135606A (en) | A feature based pre-processing method to compensate color mismatching for multi-view video | |
| KR101281003B1 (en) | Image processing system and method using multi view image | |
| US11758101B2 (en) | Restoration of the FOV of images for stereoscopic rendering | |
| US9380285B2 (en) | Stereo image processing method, stereo image processing device and display device | |
| CN102186004B (en) | Difference-based multi-viewpoint video image correcting method | |
| Ceulemans et al. | Globally optimized multiview video color correction using dense spatio-temporal matching | |
| CN106028018A (en) | Real-shot double-viewpoint 3D video optimization method and system oriented to naked 3D display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20121031 Termination date: 20150512 |
|
| EXPY | Termination of patent right or utility model |