CN101873509B - Method for eliminating background and edge shake of depth map sequence - Google Patents
Method for eliminating background and edge shake of depth map sequence Download PDFInfo
- Publication number
- CN101873509B CN101873509B CN 201010222349 CN201010222349A CN101873509B CN 101873509 B CN101873509 B CN 101873509B CN 201010222349 CN201010222349 CN 201010222349 CN 201010222349 A CN201010222349 A CN 201010222349A CN 101873509 B CN101873509 B CN 101873509B
- Authority
- CN
- China
- Prior art keywords
- depth map
- source images
- background
- sequence
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a method and a device for eliminating the background and edge shake of a depth map sequence. The method comprises the following steps: reading the background image of a source image sequence and calculating to obtain the depth map of the background image; calculating the frame difference between the source image sequence and the background image to obtain a frame difference array; calculating to obtain the depth map of each frame of the source image sequence; judging the foreground and the background of the depth map of a source image according to a preset threshold value and eliminating background shake; assigning a value to the foreground of the depth map of the source image to eliminate foreground profile shake; and optimizing through median filtering and bilateral filtering to obtain the final depth map of the source image. By adopting the method for eliminating the background and edge shake of the depth map sequence, the invention has the advantages that the defects of the prior art are overcome, the image shake is effectively eliminated, the interference noise is eliminated and the edge is smoothened.
Description
Technical field
The present invention relates to technical field of computer vision, particularly a kind of method of eliminating depth map sequence background and edge shake.
Background technology
3-D view and show it is a kind of important behaviour form of Future Information system, 3-D view and show that level is clearly bright in luster on the one hand has very strong visual impact, makes people's time in the scape of watching longer, stays deep impression; 3-D view and demonstration give truly, life-like in addition, and the personage is ready to appear, and sensation on the spot in person is arranged, and very high value of art appreciation is arranged.Just because of 3-D view and video have these plane pictures and the not available characteristics of video, thus such as computer demonstration, TV, video, robot, measure, pour examine, the field such as medical treatment, advertisement media, electronic game has broad application prospects.
Why the people soon world has third dimension, and being has a little different and have a parallax (disparity) because left eye and right eye are seen the visual angle in the world.Parallax refers to that left view and right view are corresponding to the horizontal displacement of two picture elements of same world point.The theorem of computer vision field points out, the degree of depth (depth, i.e. distance perspective) of the parallax of certain point and its corresponding world point is inversely proportional to; That is to say, the parallax value of the point away from watch a little more is less, and the parallax of infinite point is 0.The depth value that image is had a few has formed depth map (depth map).
Development based on the depth map generation technique of binocular stereo vision becomes possibility so that generate fast high-quality depth map, but because prior art is not considered temporal correlation, when image sequence is operated, each width of cloth depth map all is independent the generation, this depth map sequence that just causes generating serious background and the shake at edge can occur when playing continuously, affected whole demonstration effect.
Summary of the invention
The present invention is intended to one of solve the problems of the technologies described above at least.
For this reason, one object of the present invention has been to propose a kind of method of eliminating depth map sequence background and edge shake, and the method has overcome the deficiency of prior art, and effectively removal of images shake is eliminated interference noise, smooth edges.
One aspect of the present invention has proposed a kind of method of eliminating depth map sequence background and edge shake, may further comprise the steps: the background image that reads sequence of source images to and calculate the depth map of described background image; The frame that calculates described sequence of source images and described background image is poor to obtain frame difference group; Calculating is to obtain the depth map of described each frame of sequence of source images; Judge prospect and the background of described source images depth map according to predetermined threshold value, and eliminate background shake; Prospect assignment to described source images depth map is shaken to eliminate prospect profile; Obtain final described source images depth map by medium filtering and bilateral filtering with optimization.
In one embodiment of the invention, utilize the greyscale transformation formula that the data of each pixel of described background image and described source images are become gray value after, to calculate the depth map of described background image and described source images.
In one embodiment of the invention, the frame of the described sequence of source images of described calculating and described background image is poor further comprises to obtain frame difference group: read each two field picture of described sequence of source images pair; Each two field picture of described sequence of source images is done greyscale transformation; It is poor that described source images reference-view after the greyscale transformation and described Background reference-view are done, to obtain described frame difference group.
In one embodiment of the invention, described prospect and the background of judging described source images depth map according to predetermined threshold value, and eliminating background shake further comprises: by predetermined threshold value described frame difference group is carried out threshold decision, with prospect and the background of distinguishing described source images depth map; Be the background assignment of described source images depth map with value corresponding to described background depth map.
In one embodiment of the invention, described prospect assignment to described source images depth map further comprises to eliminate the prospect profile shake: add up all pixels in the described prospect scope; By calculate unified value as depth value to give described depth value to described prospect.
In one embodiment of the invention, describedly obtain final described source images depth map by medium filtering and bilateral filtering with optimization and further comprise: the depth map to the described sequence of source images of eliminating the shake of background shake and prospect profile carries out medium filtering, to remove the interference noise in the described background; Depth map to interference noise in the described removal background carries out bilateral filtering with the edge of level and smooth described depth map, obtains the depth map of the sequence of source images after the final optimization pass.
Another aspect of the present invention has proposed a kind of device of eliminating depth map sequence background and edge shake, comprising: image collection module, the background image that is used for reading sequence of source images to and calculate the depth map of described background image; Frame difference batch total is calculated module, and is poor to obtain frame difference group for the frame that calculates described sequence of source images and described background image; Computing module is used for calculating to obtain the depth map of described each frame of sequence of source images; The prospect background judge module is used for judging according to predetermined threshold value prospect and the background of described source images depth map, and eliminates background shake; The foreground image processing module is used for the prospect assignment of described source images depth map is shaken to eliminate prospect profile; The image optimization module is used for obtaining final described source images depth map by medium filtering and bilateral filtering with optimization.
According to the elimination depth map sequence background of the embodiment of the invention and the method for edge shake, overcome the deficiency of prior art, effectively removal of images shake is eliminated interference noise, smooth edges.The invention process is simple in addition, convenient operation.
The aspect that the present invention adds and advantage in the following description part provide, and part will become obviously from the following description, or recognize by practice of the present invention.
Description of drawings
Above-mentioned and/or the additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein:
Fig. 1 is the overall flow block diagram of the method for the elimination depth map sequence background of the embodiment of the invention and edge shake;
Fig. 2 processes depth map before for the method that adopts the embodiment of the invention;
Fig. 3 is that Fig. 2 adopts the depth map after the method for the embodiment of the invention is optimized;
Fig. 4 is for processing the three-dimensional rendering image of other images that obtain with the method for the embodiment of the invention; With
Fig. 5 is the elimination depth map sequence background of the embodiment of the invention and the device overall structure schematic diagram of edge shake.
Embodiment
The below describes embodiments of the invention in detail, and the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.
Describe the elimination depth map sequence background of the embodiment of the invention and the method for edge shake below with reference to accompanying drawing, the method has overcome the deficiency of prior art, and effectively removal of images shake is eliminated interference noise, smooth edges.The invention process is simple in addition, convenient operation.
As shown in Figure 1, be the overall flow block diagram of the method for the elimination depth map sequence background of the embodiment of the invention and edge shake.May further comprise the steps:
Step S101, the background image that reads sequence of source images to and calculate the depth map of described background image.
1.1 the definition image interface comprises: the view data pointer, picture traverse, highly, the position is dark, picture format.
1.2 according to described image interface definition image reading function, the background image of opening sequence of source images from hard disk with the image reading function is to file, so that the background extraction view data, and described background image data is kept in the Installed System Memory.
1.3, utilize the greyscale transformation formula, described background image data is changed into the gray level image data of respective background image.
Particularly, in one embodiment of the invention, utilize RGB (Red Green Blue, RGB) image conversion formula to source images to carrying out greyscale transformation, to obtain corresponding gray level image pair.Wherein, RGB image conversion formula is as follows in the embodiment of the invention:
Y＝0.212671×R+0.715160×G+0.072169×B (1)
Wherein, Y is the gray value of each pixel of gray level image, and R, G, B are respectively R, G, the B components of each pixel in the planar video two field picture.
1.4, utilize the Stereo Matching Algorithm of belief propagation to calculate the depth map of the background image of sequence of source images.In one embodiment of the invention, the depth map of background image is designated as: reDepth.
Step S102, the frame that calculates described sequence of source images and described background image is poor to obtain frame difference group.
2.1, according to image interface definition image reading function, open each two field picture of sequence of source images to file with the image reading function from hard disk, in order to obtain the view data of source images, and the view data of source images is kept in the internal memory.
2.2, utilize the greyscale transformation formula, each frame image data of sequence of source images is become the gray level image data.
2.3 it is poor that the Background reference-view after the greyscale transformation in the source images reference-view and 1.4 after the greyscale transformation in 2.2 is done, and take absolute value to obtain the frame difference group between source images reference-view and the Background reference-view.
Particularly, in one embodiment of the invention, the source images reference-view after the greyscale transformation is designated as: depth1, the Background reference-view is designated as: depth2, frame difference group is designated as: cha.Use following formula:
cha＝|depth1-depth2| (2)
Obtain corresponding frame difference group cha.
Step S103 calculates to obtain the depth map of described each frame of sequence of source images.
3.1, obtain the sequence of source images data after the greyscale transformation in 2.2.
3.2 the data in the Stereo Matching Algorithm calculating 3.1 of the belief propagation that proposes in the utilization 1.4 are to obtain the depth map of each frame of sequence of source images.In one embodiment of the invention, the depth map of each two field picture of sequence of source images is designated as: depth.As shown in Figure 2, be the depth map of each two field picture of sequence of source images of the embodiment of the invention.
Step S104 judges prospect and the background of described source images depth map according to predetermined threshold value, and eliminates background shake.
4.1 the definition threshold value is carried out threshold decision to the frame difference group that obtains in 2.3, because camera is motionless, thinks that then background is motionless, thus can be wherein less than the point of threshold value as a setting, and other point is as prospect.In one embodiment of the invention, threshold value is defined as 20.
4.2, to being divided into the zone of background, come to be source images depth map assignment with value corresponding to background depth map, use following formula:
In one embodiment of the invention, from formula (3), can find out, if frame difference group cha less than threshold value 20, then is the color assignment of the depth map depth of each two field picture of sequence of source images the color of the depth map reDepth of background image.Otherwise be prospect, keep original value.
Step S105 shakes to eliminate prospect profile the prospect assignment of described source images depth map.
5.1, pixels all in the prospect scope is added up.
5.3 the unified value that statistical computation obtains is given unified value in order to eliminate the shake of prospect profile as the depth value of prospect to the depth map of described prospect.In one embodiment of the invention, employing calculates unified value to the have a few average method of suing for peace again.
Step S106 obtains final described source images depth map by medium filtering and bilateral filtering with optimization.
6.1, the depth map of the sequence of source images processed in 5.3 is carried out medium filtering, can remove interference noise in the background by medium filtering.
6.2,6.1 depth maps of processing sequence of source images are carried out bilateral filtering, can smooth edges by bilateral filtering.
6.3, obtain the depth map of each frame of source images.In one embodiment of the invention, as shown in Figure 3, be the good depth map shown in Figure 2 of final optimization pass.
In another one embodiment of the present invention.As shown in Figure 4, for the elimination depth map sequence background that proposes by the present invention with other image sequences and the method test effect of edge shake, can find out this three-dimensional rendering image by Fig. 4, still not have obvious background and edge shake.
Another aspect of the present invention has proposed a kind of device of eliminating depth map sequence background and edge shake, the device 100 of this elimination depth map sequence background and edge shake comprises image collection module 110, frame difference batch total is calculated module 120, prospect background judge module 130, foreground image processing module 140, and image optimization module 150.Wherein, the background image that image collection module 110 is used for reading sequence of source images to and calculate the depth map of described background image, it is poor to obtain frame difference group for the frame that calculates described sequence of source images and described background image that frame difference batch total is calculated module 120, prospect background judge module 130 is used for judging according to predetermined threshold value prospect and the background of described source images depth map, and elimination background shake, foreground image processing module 140 is used for the prospect assignment of described source images depth map is shaken to eliminate prospect profile, and image optimization module 150 is used for obtaining final described source images depth map by medium filtering and bilateral filtering with optimization.
Wherein, image collection module 110 comprises background image acquisition module 1101, greyscale transformation module 1102, computing module 1103.Background image acquisition module 1101 is used for background extraction figure, greyscale transformation module 1102 adopts the greyscale transformation formula that the data of each pixel of described background image and described source images are become gray value, computing module 1103 is used for the described background image after the greyscale transformation and described source images, to calculate the depth map of described background image and described source images.
But those of ordinary skill in the art knows; the image optimization module 150 of the elimination depth map sequence background that the present invention proposes and the device 100 of edge shake can also be subdivided into medium filtering module and bilateral filtering module; certainly other forms of combination can also be arranged, these also should be thought within protection scope of the present invention.
By the elimination depth map sequence background of the present invention's proposition and the method for edge shake, overcome the deficiency of prior art, effectively removal of images shake is eliminated interference noise, smooth edges.The invention process is simple in addition, convenient operation.
Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification to these embodiment, scope of the present invention is by claims and be equal to and limit.
Claims (3)
1. a method of eliminating depth map sequence background and edge shake is characterized in that, may further comprise the steps:
The background image that reads sequence of source images to and calculate the depth map of described background image and calculate to obtain the depth map of described each frame of sequence of source images, wherein, after utilizing the greyscale transformation formula that the data of each pixel of described background image and source images are become gray value, to calculate the depth map of described background image and described source images;
The frame that calculates described sequence of source images and described background image is poor to obtain frame difference group, comprise and read each two field picture of described sequence of source images pair, each two field picture of described sequence of source images is done greyscale transformation, it is poor that source images reference-view after the greyscale transformation and Background reference-view are done, and take absolute value to obtain described frame difference group, wherein, described frame difference group cha=|depth1-depth2|, depth1 is the reference-view of the sequence of source images after the greyscale transformation, and depth2 is the reference-view of the background image after the greyscale transformation;
Judge prospect and the background of described source images depth map according to predetermined threshold value, and eliminate background shake, wherein, judge prospect and the background of described source images depth map by formula, described formula is,
Wherein, reDepth is the depth map of background image, and depth is the depth map of source images;
The prospect assignment of described source images depth map to eliminate the prospect profile shake, being comprised: add up all pixels in the described prospect scope, by calculate unified value as depth value to give described depth value to described prospect;
Obtain final described source images depth map by medium filtering and bilateral filtering with optimization.
2. the method for elimination depth map sequence background as claimed in claim 1 and edge shake is characterized in that, describedly obtains final described source images depth map by medium filtering and bilateral filtering with optimization and further comprises:
Depth map to the described sequence of source images of eliminating the shake of background shake and prospect profile carries out medium filtering, to remove the interference noise in the described background;
Depth map to interference noise in the described removal background carries out bilateral filtering with the edge of level and smooth described depth map, obtains the depth map of the sequence of source images after the final optimization pass.
3. a device of eliminating depth map sequence background and edge shake is characterized in that, comprising:
Image collection module, the background image that is used for reading sequence of source images to and calculate the depth map of described background image and calculate to obtain the depth map of described each frame of sequence of source images, wherein, after utilizing the greyscale transformation formula that the data of each pixel of described background image and source images are become gray value, to calculate the depth map of described background image and described source images;
Frame difference batch total is calculated module, poor to obtain frame difference group for the frame that calculates described sequence of source images and described background image, comprise and read each two field picture of described sequence of source images pair, each two field picture of described sequence of source images is done greyscale transformation, it is poor that source images reference-view after the greyscale transformation and Background reference-view are done, and take absolute value to obtain described frame difference group, wherein, described frame difference group cha=|depth1-depth2|, depth1 is the reference-view of the sequence of source images after the greyscale transformation, and depth2 is the reference-view of the background image after the greyscale transformation;
The prospect background judge module is used for judging according to predetermined threshold value prospect and the background of described source images depth map, and eliminates background shake, wherein, judges prospect and the background of described source images depth map by formula, and described formula is,
Wherein, reDepth is the depth map of background image, and depth is the depth map of source images;
The foreground image processing module, be used for the prospect assignment of described source images depth map is shaken to eliminate prospect profile, comprise: add up all pixels in the described prospect scope, by calculate unified value as depth value to give described depth value to described prospect; With
The image optimization module is used for obtaining final described source images depth map by medium filtering and bilateral filtering with optimization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010222349 CN101873509B (en) | 2010-06-30 | 2010-06-30 | Method for eliminating background and edge shake of depth map sequence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010222349 CN101873509B (en) | 2010-06-30 | 2010-06-30 | Method for eliminating background and edge shake of depth map sequence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101873509A CN101873509A (en) | 2010-10-27 |
| CN101873509B true CN101873509B (en) | 2013-03-27 |
Family
ID=42998125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010222349 Active CN101873509B (en) | 2010-06-30 | 2010-06-30 | Method for eliminating background and edge shake of depth map sequence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101873509B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102298708B (en) * | 2011-08-19 | 2013-08-07 | 四川长虹电器股份有限公司 | 3D mode identification method based on color and shape matching |
| CN102622768B (en) * | 2012-03-14 | 2014-04-09 | 清华大学 | Depth-map gaining method of plane videos |
| TW201432622A (en) * | 2012-11-07 | 2014-08-16 | Koninkl Philips Nv | Generation of a depth map for an image |
| CN103024419A (en) * | 2012-12-31 | 2013-04-03 | 青岛海信信芯科技有限公司 | Video image processing method and system |
| CN104093014A (en) * | 2014-07-21 | 2014-10-08 | 宇龙计算机通信科技(深圳)有限公司 | Image processing method and image processing device |
| CN104463775A (en) * | 2014-10-31 | 2015-03-25 | 小米科技有限责任公司 | Device and method for achieving depth-of-field effect of image |
| JP6565252B2 (en) * | 2015-03-23 | 2019-08-28 | 日本電気株式会社 | Information processing apparatus, information processing method, and program |
| CN105120129B (en) * | 2015-08-05 | 2018-01-30 | 西安电子科技大学 | The Infrared DIM-small Target Image sequence emulation mode of infrared area array cameras shake |
| CN105744171B (en) * | 2016-03-30 | 2019-03-08 | 联想(北京)有限公司 | A kind of image processing method and electronic equipment |
| CN111626086A (en) * | 2019-02-28 | 2020-09-04 | 北京市商汤科技开发有限公司 | Living body detection method, living body detection device, living body detection system, electronic device, and storage medium |
| CN110378945B (en) * | 2019-07-11 | 2021-06-18 | Oppo广东移动通信有限公司 | Depth map processing method and device and electronic equipment |
| CN114007058A (en) * | 2020-07-28 | 2022-02-01 | 阿里巴巴集团控股有限公司 | Depth map correction method, video processing method, video reconstruction method and related devices |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101237522A (en) * | 2007-02-02 | 2008-08-06 | 华为技术有限公司 | Motion detection method and device |
| CN101483788A (en) * | 2009-01-20 | 2009-07-15 | 清华大学 | Method and apparatus for converting plane video into tridimensional video |
| CN101542529A (en) * | 2006-11-21 | 2009-09-23 | 皇家飞利浦电子股份有限公司 | Generation of depth map for an image |
| CN101630408A (en) * | 2009-08-14 | 2010-01-20 | 清华大学 | Depth map treatment method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2704479C (en) * | 2007-11-09 | 2016-01-05 | Thomson Licensing | System and method for depth map extraction using region-based filtering |
-
2010
- 2010-06-30 CN CN 201010222349 patent/CN101873509B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101542529A (en) * | 2006-11-21 | 2009-09-23 | 皇家飞利浦电子股份有限公司 | Generation of depth map for an image |
| CN101237522A (en) * | 2007-02-02 | 2008-08-06 | 华为技术有限公司 | Motion detection method and device |
| CN101483788A (en) * | 2009-01-20 | 2009-07-15 | 清华大学 | Method and apparatus for converting plane video into tridimensional video |
| CN101630408A (en) * | 2009-08-14 | 2010-01-20 | 清华大学 | Depth map treatment method and device |
Non-Patent Citations (1)
| Title |
|---|
| 朱波等.面向虚拟视点图像绘制的深度图编码算法.《光电子·激光》.2010,第21卷(第5期),718-724. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101873509A (en) | 2010-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101873509B (en) | Method for eliminating background and edge shake of depth map sequence | |
| US7982733B2 (en) | Rendering 3D video images on a stereo-enabled display | |
| CN102741879B (en) | Method for generating depth maps from monocular images and systems using the same | |
| US9300947B2 (en) | Producing 3D images from captured 2D video | |
| US9123115B2 (en) | Depth estimation based on global motion and optical flow | |
| EP1839267B1 (en) | Depth perception | |
| WO2014083949A1 (en) | Stereoscopic image processing device, stereoscopic image processing method, and program | |
| US20130250053A1 (en) | System and method for real time 2d to 3d conversion of video in a digital camera | |
| CN107750370B (en) | Method and apparatus for determining a depth map for an image | |
| US9154762B2 (en) | Stereoscopic image system utilizing pixel shifting and interpolation | |
| JP2011223566A (en) | Image converting device and three-dimensional image display device including the same | |
| JP2015156607A (en) | Image processing method, image processing apparatus, and electronic device | |
| US8665262B2 (en) | Depth map enhancing method | |
| Schmeing et al. | Depth image based rendering: A faithful approach for the disocclusion problem | |
| KR20130040771A (en) | Three-dimensional video processing apparatus, method therefor, and program | |
| CN102098528A (en) | Method and device for converting planar image into stereoscopic image | |
| US20100246938A1 (en) | Image Processing Method for Providing Depth Information and Image Processing System Using the Same | |
| KR20100109069A (en) | Device for generating visual attention map and method thereof | |
| CN103369331B (en) | The complementing method of image cavity and device and the treating method and apparatus of video image | |
| CN104506872A (en) | Method and device for converting planar video into stereoscopic video | |
| US20120229600A1 (en) | Image display method and apparatus thereof | |
| CN102026012B (en) | Generation method and device of depth map through three-dimensional conversion to planar video | |
| US20130010059A1 (en) | Image processing apparatus, image processing method, and program | |
| CN105530505A (en) | Three-dimensional image conversion method and device | |
| CN110149508A (en) | A kind of array of figure generation and complementing method based on one-dimensional integrated imaging system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |