CN101715139A - Multi-mode error code covering method based on complementary covering mode in dimensional images - Google Patents
Multi-mode error code covering method based on complementary covering mode in dimensional images Download PDFInfo
- Publication number
- CN101715139A CN101715139A CN 200910234278 CN200910234278A CN101715139A CN 101715139 A CN101715139 A CN 101715139A CN 200910234278 CN200910234278 CN 200910234278 CN 200910234278 A CN200910234278 A CN 200910234278A CN 101715139 A CN101715139 A CN 101715139A
- Authority
- CN
- China
- Prior art keywords
- block
- covering
- cover
- mode
- errored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a multi-mode error code covering method based on a complementary covering mode in stereo images, belonging to the technical field of image communication. The multi-mode error code covering method comprises the following steps of: firstly, simultaneously scanning according to a grating scanning sequence to detect out a first block requiring covering in a left eye image and a right eye image; examining whether a reference frame searching area contains an invalid pixel or not; if containing, then carrying out boundary matching and covering by using a matched block; or else, carrying out block matching on four neighborhood blocks of an error code block to obtain the optical parallax vectors of the four neighborhood blocks, getting rid of the vector with larger deviation among the four optical parallax vectors, averaging the rest vectors as the optical parallax vector of an error code block to be processed, and finding out corresponding blocks in the reference frame for covering; if the corresponding blocks contains the invalid pixel, then processing the invalid pixel part with a bilinear interpolation method by using a valid pixel in a current frame; and if the covering results are in poor boundary continuation, then covering by using a directional interpolation method. The rest error code blocks of the two images are processed by adopting a complementary method till finishing covering completely.
Description
Technical field
The present invention is that a kind of two width of cloth images of stereo-picture that are applicable to all have under the situation of losing piece, and stereoscopic image is carried out code error shielding, belongs to the technical field of Image Communication.
Background technology
Great achievement that Internet is obtained and the following great development potentiality that contained are beyond any doubt.The video business has characteristics directly perceived, vivid, understandable and that contain much information as one of mainstream applications on the Internet.Simultaneously, people are also progressively developed to 3D from 2D the application of video, and three-dimensional video-frequency/image more and more receives publicity.Video had often been passed through compression and has been removed time, space and statistical redundancy degree, but also made the fault-tolerance of output code flow reduce greatly like this before transmission.A grouping even a bit lose or mistake all might make decoder lose synchronously, and then the picture quality of influence reconstruction.And the network that traditional IP-based Internet network is a kind of " doing one's best ", it can not provide the reliable transmission of data flow.Therefore, how to guarantee on IP network that the quality of data flow just becomes a major issue that can not be ignored.Have real-time because video is used, therefore, when the video data are made mistakes, use the retransmission technique error correction to receive.This just requires video encoding and decoding technique itself to handle misdata.Anti-error code (Error Resilient) technology is a kind of method of effective solution misdata, and still, it is a cost to increase extra bit rate often.If consider bandwidth considerations, code error shielding (ErrorConcealment) then can be regarded a kind of solution of misdata preferably as.It is a cost with the complexity that increases decoder only.And image is the video of single frames, and the concealing technique of research image can be regarded the basis of video concealing technique as.Therefore, a kind of method for shielding error code that is used for two width of cloth and all has the stereo-picture of losing piece of research seems particularly necessary and meaningful.
Summary of the invention
Technical problem: the multi-mode method for shielding error code that the purpose of this invention is to provide the mode of covering based on complementation in a kind of stereo-picture, this method is utilized the correlation between the stereo-picture, use the zone that does not have error code of piece image to cover the zone that another width of cloth image has error code as much as possible, and can be according to different error conditions, adopt the different patterns of covering adaptively, can both obtain to cover preferably effect thereby finally reach two width of cloth images.
Technical scheme: code error shielding technology extensive use owing to the subjective quality that can improve video preferably, but code error shielding technology often concentrates on the interframe code error shielding technology that the code error shielding of piece image is an interior code error shielding technology of frame or a video sequence, and they have mainly utilized the spatial coherence and the timeliness of image respectively.How to address this problem research less and occur error code during for transmitting stereoscopic image.
Be without loss of generality, suppose that two width of cloth images all have error code, and errored block isolates, promptly the piece (as shown in Figure 1) of its four neighborhoods all is the piece that can be correctly decoded.Because two width of cloth images of the right and left eyes of stereo-picture have stronger correlation, therefore, when carrying out code error shielding, utilize correlation as much as possible.
When covering beginning, according to raster scan order scanning, detect which width of cloth image of left-and right-eye images and meet with errored block earlier at first simultaneously, the errored block that meets with is just as first piece that need cover earlier, the frame that comprises this pending errored block is called present frame, and the frame that does not comprise then is called reference frame.Here suppose the errored block B that has first to cover in the right image
Lost, the pixel that makes a circle in this errored block week is regarded as an integral body, because error code isolates, therefore, each pixel in this integral body all is effective.Accordingly, the pixel in the errored block is called inactive pixels.This integral body is put into goes coupling among the region of search SR of left image, promptly seek a integral body the most similar.We claim that this process is the border coupling.Utilize this content of the piece that the most similar integral body comprised that finds to remove to cover the errored block of right image then.That is:
This is all to be to carry out under the situation of valid pixel in the region of search.At this moment concealing method is called the pattern of covering 1.
In case errored block is also arranged in the region of search, then abandon the border matching process, use the method for piece coupling instead.Can extract the piece of errored block four neighborhoods on every side that need cover and (use NB respectively
Up, NB
Down, NB
Left, NB
RightRepresent) carry out piece coupling.Each neighborhood piece is put into goes coupling in the region of search of reference frame.If do not have other errored block in the region of search this moment, when searching the errored block that mates most, record difference vector at this moment.If have errored block in the region of search this moment, then do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most.Four neighborhood pieces are all done same above-mentioned processing.Difference vector according to adjacent block should equate or close principle then, rejects the bigger vector of deviation in these four difference vectors, and its () may be the difference vector of mistake coupling.Remaining difference vector is asked on average, and as the difference vector of current pending errored block.In reference frame, find corresponding piece with this difference vector, cover pending errored block with the content of this piece.At this moment concealing method is called the pattern of covering 2.If this corresponding blocks that finds comprises inactive pixels, the zone that can't cover because of inactive pixels then adopts the method for bilinear interpolation to cover in present frame, and during bilinear interpolation, only utilizes the valid pixel value of present frame.Here become the pixel of valid pixel after the valid pixel of present frame has comprised the original pixel that just exists and before covered processing.At this moment concealing method is called covers mode 3.Its computing formula is:
Wherein, p
u, p
b, p
l, p
rBe respectively and wait to recover the most contiguous value of four valid pixels up and down of pixel, d
u, d
b, d
l, d
rBe respectively these four effective pixel points and wait to recover distance between the pixel.As shown in Figure 2.
This is covered the continuous level determinations that the result carries out the border, with the absolute difference of covering one circle pixel of inner periphery as a result and exterior circumferential one circle pixel (as shown in Figure 3) and (SAD) as the weighing criteria of continuous degree.The size of supposing to wait to cover piece is N * N, and the pixel in its inner upper left corner is p (x
1, y
1), level is to the right the X-axis positive direction, is the Y-axis positive direction vertically downward, then is formulated as:
SAD is more little, and then degree is good more continuously.If continuously degree is lower and continuous degree its four neighborhood pieces are higher relatively, show that then to cover effect not good enough, this is often because errored block has comprised the edge that crosses wherein.To this situation, then use the method for directional interpolation instead and carry out code error shielding.At this moment the pattern of covering is called the pattern of covering 4.So far, first errored block disposes, and it covers the pixel that comprised of back will be as valid pixel in follow-up covering.
Then, once more simultaneously according to raster scan order scanning, investigate which width of cloth experience errored block in left and right two width of cloth images, and it is carried out above-mentioned same covering.Covering and so forth all covered until last two width of cloth images and to be finished.
As can be seen, cover the live part in the time of well to cover and to make it to become another order image and cover the errored block in the order image in the region of search according to above-mentioned order.Clearly, the mode of covering of this complementation helps improving the effect of covering.
Cover the multi-mode method for shielding error code of mode in the stereo-picture of the present invention based on complementation, the steps include:
1) earlier simultaneously according to raster scan order scanning, detects first pending errored block in left and right two width of cloth images;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, promptly cover pattern 1, jump to step 5) then and continue to carry out;
4), then adopt the method for piece coupling to find match block if comprise inactive pixels in the region of search; If the piece that finds does not comprise inactive pixels, then directly cover with match block, promptly cover pattern 2; If comprise inactive pixels, then adopt and cover mode 3 and handle;
5) to covering the continuous level determinations that the result carries out the border, if it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then use the method for directional interpolation instead to carry out code error shielding, promptly cover pattern 4;
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process, so complementary covers, and all covers until last two width of cloth images to finish.
Adopt the method for piece coupling to find the step of match block to be: to adopt four neighborhood pieces up and down of pending errored block to carry out the piece coupling, piece for each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment; If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most, and four neighborhood pieces are all done same above-mentioned processing; Difference vector according to adjacent block should equate or close principle then, reject the bigger vector of deviation in these four difference vectors, they may be the difference vectors of mistake coupling, the difference vector of remainder are asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding match block with this difference vector.
The step of covering mode 3 is: at first cover errored block with effective pixel in the match block that finds, and for the zone of inactive pixels correspondence, then cover with the method for bilinear interpolation, and during bilinear interpolation, only utilize the valid pixel value of present frame at present frame.
Beneficial effect:
1) owing to utilized the correlation of images of left and right eyes two width of cloth images to cover, therefore, subjective quality and the objective quality (is weighing criteria with Y-PSNR PSNR) of covering the result all are better than concealing method in the simple frame;
2) owing to can select the different patterns of covering for use according to different situations adaptively, therefore, subjective quality and the objective quality (is weighing criteria with PSNR) of covering the result all are better than single concealing method;
3) owing to cover and be based on that complimentary fashion carries out, therefore, subjective quality and the objective quality (is weighing criteria with PSNR) of covering the result all are better than having covered earlier the concealing method that a frame is covered another frame again;
4) when the average conduct of the difference vector of getting four neighborhood pieces is waited to cover the difference vector of piece, rejected the big difference vector of deviation, thereby improved the estimated accuracy of the difference vector of waiting to cover piece, helped improving subjective quality and the objective quality (is weighing criteria with PSNR) covered;
5) the inventive method can be expanded and be used for three-dimensional video-frequency the code error shielding problem of losing piece is all arranged.
Description of drawings
Fig. 1 is four neighborhood piece schematic diagrames of errored block;
Fig. 2 adopts bilinear interpolation to cover schematic diagram for the inactive pixels zone;
Fig. 3 is a schematic diagram of covering the inside and outside circle pixel of piece;
Fig. 4 is that the present invention realizes the code error shielding flow chart.
Embodiment
Cover the multi-mode method for shielding error code of mode in the stereo-picture of the present invention based on complementation, its specific embodiments is as follows:
1), detects which width of cloth of left and right two width of cloth images and meet with errored block earlier, and choose the errored block that suffers from earlier as first pending errored block simultaneously according to raster scan order scanning;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, jump to step 5) then and continue to carry out.
4), then adopt four neighborhood pieces to carry out the piece coupling if comprise inactive pixels in the region of search.For the piece of each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment.If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most.Four neighborhood pieces are all done same above-mentioned processing.Difference vector according to adjacent block should equate or close principle then, rejects the bigger vector of deviation in these four difference vectors, and its () may be the difference vector of mistake coupling.Remaining difference vector is asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding piece with this difference vector.If the piece that finds does not comprise inactive pixels, then directly cover pending errored block with the content of this piece; If the corresponding blocks that finds comprises inactive pixels, the zone that can't cover because of inactive pixels then adopts the method for bilinear interpolation to cover in present frame, and during bilinear interpolation, only utilizes the value of present frame valid pixel.
5) to covering the continuous level determinations that the result carries out the border.With the absolute difference of covering one circle pixel of inner periphery as a result and exterior circumferential one circle pixel and (SAD) as the weighing criteria of degree continuously, SAD is more little, then continuous degree is good more.If it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then uses the method for directional interpolation instead and carry out code error shielding.
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process.So complementary covers, and all covers until last two width of cloth images to finish.
Claims (3)
1. cover the multi-mode method for shielding error code of mode in the stereo-picture based on complementation, it is characterized in that the steps include:
1) earlier simultaneously according to raster scan order scanning, detects first pending errored block in left and right two width of cloth images;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, promptly cover pattern 1, jump to step 5) then and continue to carry out;
4), then adopt the method for piece coupling to find match block if comprise inactive pixels in the region of search; If the piece that finds does not comprise inactive pixels, then directly cover with match block, promptly cover pattern 2; If comprise inactive pixels, then adopt and cover mode 3 and handle;
5) to covering the continuous level determinations that the result carries out the border, if it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then use the method for directional interpolation instead to carry out code error shielding, promptly cover pattern 4;
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process, so complementary covers, and all covers until last two width of cloth images to finish.
2. cover the multi-mode method for shielding error code of mode in the stereo-picture according to claim 1 based on complementation, it is characterized in that adopting the method for piece coupling to find the step of match block to be: adopt four neighborhood pieces up and down of pending errored block to carry out the piece coupling, piece for each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment; If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most, and four neighborhood pieces are all done same above-mentioned processing; Difference vector according to adjacent block should equate or close principle then, reject the bigger vector of deviation in these four difference vectors, they may be the difference vectors of mistake coupling, the difference vector of remainder are asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding match block with this difference vector.
3. cover the multi-mode method for shielding error code of mode in the stereo-picture according to claim 1 based on complementation, it is characterized in that the step of covering mode 3 is: at first cover errored block with effective pixel in the match block that finds, and for the zone of inactive pixels correspondence, then cover with the method for bilinear interpolation at present frame, and during bilinear interpolation, only utilize the valid pixel value of present frame.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200910234278 CN101715139B (en) | 2009-11-16 | 2009-11-16 | Multi-mode error code covering method based on complementary covering mode in dimensional images |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200910234278 CN101715139B (en) | 2009-11-16 | 2009-11-16 | Multi-mode error code covering method based on complementary covering mode in dimensional images |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101715139A true CN101715139A (en) | 2010-05-26 |
CN101715139B CN101715139B (en) | 2011-04-20 |
Family
ID=42418323
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 200910234278 Expired - Fee Related CN101715139B (en) | 2009-11-16 | 2009-11-16 | Multi-mode error code covering method based on complementary covering mode in dimensional images |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101715139B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103796028A (en) * | 2014-02-26 | 2014-05-14 | 北京大学 | Motion searching method and device based on image information in video coding |
CN108377359A (en) * | 2018-03-14 | 2018-08-07 | 苏州科达科技股份有限公司 | Video anti-error code method, device, electronic equipment, storage medium |
CN110335389A (en) * | 2019-07-01 | 2019-10-15 | 上海商汤临港智能科技有限公司 | Car door unlocking method and device, system, vehicle, electronic equipment and storage medium |
CN110766625A (en) * | 2019-10-15 | 2020-02-07 | 中国科学院光电技术研究所 | Method for rapidly filling digital image mask |
US10719991B2 (en) * | 2016-06-08 | 2020-07-21 | Sony Interactive Entertainment Inc. | Apparatus and method for creating stereoscopic images using a displacement vector map |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1269359C (en) * | 2003-03-03 | 2006-08-09 | 西南交通大学 | Video error blanketing method based on motion vector extrapolation and motion vector search |
US20100061452A1 (en) * | 2007-01-04 | 2010-03-11 | Thomson Licensing Corporation | Method and apparatus for video error concealment using high level syntax reference views in multi-view coded video |
CN100531400C (en) * | 2007-07-26 | 2009-08-19 | 上海交通大学 | Video error coverage method based on macro block level and pixel motion estimation |
CN101370145B (en) * | 2007-08-13 | 2010-06-09 | 中兴通讯股份有限公司 | Shielding method and apparatus for image frame |
-
2009
- 2009-11-16 CN CN 200910234278 patent/CN101715139B/en not_active Expired - Fee Related
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103796028A (en) * | 2014-02-26 | 2014-05-14 | 北京大学 | Motion searching method and device based on image information in video coding |
CN103796028B (en) * | 2014-02-26 | 2017-09-29 | 北京大学 | Method for searching motion based on image information in a kind of Video coding |
US10719991B2 (en) * | 2016-06-08 | 2020-07-21 | Sony Interactive Entertainment Inc. | Apparatus and method for creating stereoscopic images using a displacement vector map |
CN108377359A (en) * | 2018-03-14 | 2018-08-07 | 苏州科达科技股份有限公司 | Video anti-error code method, device, electronic equipment, storage medium |
CN108377359B (en) * | 2018-03-14 | 2020-08-04 | 苏州科达科技股份有限公司 | Video error code resisting method and device, electronic equipment and storage medium |
CN110335389A (en) * | 2019-07-01 | 2019-10-15 | 上海商汤临港智能科技有限公司 | Car door unlocking method and device, system, vehicle, electronic equipment and storage medium |
CN110766625A (en) * | 2019-10-15 | 2020-02-07 | 中国科学院光电技术研究所 | Method for rapidly filling digital image mask |
CN110766625B (en) * | 2019-10-15 | 2022-04-19 | 中国科学院光电技术研究所 | Method for rapidly filling digital image mask |
Also Published As
Publication number | Publication date |
---|---|
CN101715139B (en) | 2011-04-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7210381B2 (en) | Efficient Multiview Coding with Depth Map Estimation and Update | |
US10417748B2 (en) | Filtering and edge encoding and decoding for depth maps | |
EP2491722B1 (en) | Depth map generation techniques for conversion of 2d video data to 3d video data | |
WO2017080420A1 (en) | Auxiliary data for artifacts –aware view synthesis | |
CN101937578B (en) | Method for drawing virtual view color image | |
JP5970609B2 (en) | Method and apparatus for unified disparity vector derivation in 3D video coding | |
US8488870B2 (en) | Multi-resolution, multi-window disparity estimation in 3D video processing | |
US9736498B2 (en) | Method and apparatus of disparity vector derivation and inter-view motion vector prediction for 3D video coding | |
CN101715139B (en) | Multi-mode error code covering method based on complementary covering mode in dimensional images | |
US20140146139A1 (en) | Depth or disparity map upscaling | |
CN109218736A (en) | device and method for coding and decoding | |
CN109257598A (en) | Adaptive partition coding | |
WO2013068548A2 (en) | Efficient multi-view coding using depth-map estimate for a dependent view | |
Chen et al. | LSVC: A learning-based stereo video compression framework | |
CN104918032B (en) | Simplify the method that the block based on depth is split | |
EP2391135B1 (en) | Method and device for processing depth image sequence | |
US20150016517A1 (en) | Encoding device and encoding method, and decoding device and decoding method | |
CN108668135B (en) | Stereoscopic video B frame error concealment method based on human eye perception | |
US20180084276A1 (en) | Method of Lookup Table Size Reduction for Depth Modelling Mode in Depth Coding | |
Tai et al. | An effiicient full frame algorithm for object-based error concealment in 3D depth-based video | |
Kamolrat et al. | Adaptive motion-estimation-mode selection for depth video coding | |
CN103220533A (en) | Method for hiding loss errors of three-dimensional video macro blocks | |
Chen et al. | Depth map compression via edge-based inpainting | |
CN104994365B (en) | A kind of method and 2D video three-dimensional methods for obtaining non-key frame depth image | |
CN104853216B (en) | Block dividing method based on depth and electronic device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110420 Termination date: 20151116 |
|
CF01 | Termination of patent right due to non-payment of annual fee |