CN101715139A - Multi-mode error code covering method based on complementary covering mode in dimensional images - Google Patents

Multi-mode error code covering method based on complementary covering mode in dimensional images Download PDF

Info

Publication number
CN101715139A
CN101715139A CN 200910234278 CN200910234278A CN101715139A CN 101715139 A CN101715139 A CN 101715139A CN 200910234278 CN200910234278 CN 200910234278 CN 200910234278 A CN200910234278 A CN 200910234278A CN 101715139 A CN101715139 A CN 101715139A
Authority
CN
China
Prior art keywords
block
covering
cover
mode
errored
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200910234278
Other languages
Chinese (zh)
Other versions
CN101715139B (en
Inventor
唐贵进
朱秀昌
干宗良
詹学峰
崔子冠
宋丽娟
李欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Post and Telecommunication University
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing Post and Telecommunication University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Post and Telecommunication University filed Critical Nanjing Post and Telecommunication University
Priority to CN 200910234278 priority Critical patent/CN101715139B/en
Publication of CN101715139A publication Critical patent/CN101715139A/en
Application granted granted Critical
Publication of CN101715139B publication Critical patent/CN101715139B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses a multi-mode error code covering method based on a complementary covering mode in stereo images, belonging to the technical field of image communication. The multi-mode error code covering method comprises the following steps of: firstly, simultaneously scanning according to a grating scanning sequence to detect out a first block requiring covering in a left eye image and a right eye image; examining whether a reference frame searching area contains an invalid pixel or not; if containing, then carrying out boundary matching and covering by using a matched block; or else, carrying out block matching on four neighborhood blocks of an error code block to obtain the optical parallax vectors of the four neighborhood blocks, getting rid of the vector with larger deviation among the four optical parallax vectors, averaging the rest vectors as the optical parallax vector of an error code block to be processed, and finding out corresponding blocks in the reference frame for covering; if the corresponding blocks contains the invalid pixel, then processing the invalid pixel part with a bilinear interpolation method by using a valid pixel in a current frame; and if the covering results are in poor boundary continuation, then covering by using a directional interpolation method. The rest error code blocks of the two images are processed by adopting a complementary method till finishing covering completely.

Description

Cover the multi-mode method for shielding error code of mode in the stereo-picture based on complementation
Technical field
The present invention is that a kind of two width of cloth images of stereo-picture that are applicable to all have under the situation of losing piece, and stereoscopic image is carried out code error shielding, belongs to the technical field of Image Communication.
Background technology
Great achievement that Internet is obtained and the following great development potentiality that contained are beyond any doubt.The video business has characteristics directly perceived, vivid, understandable and that contain much information as one of mainstream applications on the Internet.Simultaneously, people are also progressively developed to 3D from 2D the application of video, and three-dimensional video-frequency/image more and more receives publicity.Video had often been passed through compression and has been removed time, space and statistical redundancy degree, but also made the fault-tolerance of output code flow reduce greatly like this before transmission.A grouping even a bit lose or mistake all might make decoder lose synchronously, and then the picture quality of influence reconstruction.And the network that traditional IP-based Internet network is a kind of " doing one's best ", it can not provide the reliable transmission of data flow.Therefore, how to guarantee on IP network that the quality of data flow just becomes a major issue that can not be ignored.Have real-time because video is used, therefore, when the video data are made mistakes, use the retransmission technique error correction to receive.This just requires video encoding and decoding technique itself to handle misdata.Anti-error code (Error Resilient) technology is a kind of method of effective solution misdata, and still, it is a cost to increase extra bit rate often.If consider bandwidth considerations, code error shielding (ErrorConcealment) then can be regarded a kind of solution of misdata preferably as.It is a cost with the complexity that increases decoder only.And image is the video of single frames, and the concealing technique of research image can be regarded the basis of video concealing technique as.Therefore, a kind of method for shielding error code that is used for two width of cloth and all has the stereo-picture of losing piece of research seems particularly necessary and meaningful.
Summary of the invention
Technical problem: the multi-mode method for shielding error code that the purpose of this invention is to provide the mode of covering based on complementation in a kind of stereo-picture, this method is utilized the correlation between the stereo-picture, use the zone that does not have error code of piece image to cover the zone that another width of cloth image has error code as much as possible, and can be according to different error conditions, adopt the different patterns of covering adaptively, can both obtain to cover preferably effect thereby finally reach two width of cloth images.
Technical scheme: code error shielding technology extensive use owing to the subjective quality that can improve video preferably, but code error shielding technology often concentrates on the interframe code error shielding technology that the code error shielding of piece image is an interior code error shielding technology of frame or a video sequence, and they have mainly utilized the spatial coherence and the timeliness of image respectively.How to address this problem research less and occur error code during for transmitting stereoscopic image.
Be without loss of generality, suppose that two width of cloth images all have error code, and errored block isolates, promptly the piece (as shown in Figure 1) of its four neighborhoods all is the piece that can be correctly decoded.Because two width of cloth images of the right and left eyes of stereo-picture have stronger correlation, therefore, when carrying out code error shielding, utilize correlation as much as possible.
When covering beginning, according to raster scan order scanning, detect which width of cloth image of left-and right-eye images and meet with errored block earlier at first simultaneously, the errored block that meets with is just as first piece that need cover earlier, the frame that comprises this pending errored block is called present frame, and the frame that does not comprise then is called reference frame.Here suppose the errored block B that has first to cover in the right image Lost, the pixel that makes a circle in this errored block week is regarded as an integral body, because error code isolates, therefore, each pixel in this integral body all is effective.Accordingly, the pixel in the errored block is called inactive pixels.This integral body is put into goes coupling among the region of search SR of left image, promptly seek a integral body the most similar.We claim that this process is the border coupling.Utilize this content of the piece that the most similar integral body comprised that finds to remove to cover the errored block of right image then.That is:
f ( p ) ^ = f ( p - d ) , Wherein p ∈ loses piece B Lost, d is the optimum Match vector
This is all to be to carry out under the situation of valid pixel in the region of search.At this moment concealing method is called the pattern of covering 1.
In case errored block is also arranged in the region of search, then abandon the border matching process, use the method for piece coupling instead.Can extract the piece of errored block four neighborhoods on every side that need cover and (use NB respectively Up, NB Down, NB Left, NB RightRepresent) carry out piece coupling.Each neighborhood piece is put into goes coupling in the region of search of reference frame.If do not have other errored block in the region of search this moment, when searching the errored block that mates most, record difference vector at this moment.If have errored block in the region of search this moment, then do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most.Four neighborhood pieces are all done same above-mentioned processing.Difference vector according to adjacent block should equate or close principle then, rejects the bigger vector of deviation in these four difference vectors, and its () may be the difference vector of mistake coupling.Remaining difference vector is asked on average, and as the difference vector of current pending errored block.In reference frame, find corresponding piece with this difference vector, cover pending errored block with the content of this piece.At this moment concealing method is called the pattern of covering 2.If this corresponding blocks that finds comprises inactive pixels, the zone that can't cover because of inactive pixels then adopts the method for bilinear interpolation to cover in present frame, and during bilinear interpolation, only utilizes the valid pixel value of present frame.Here become the pixel of valid pixel after the valid pixel of present frame has comprised the original pixel that just exists and before covered processing.At this moment concealing method is called covers mode 3.Its computing formula is:
p ( i , j ) = p u * d b + p b * d u + p l * d r + p r * d l d b + d u + d r + d l
Wherein, p u, p b, p l, p rBe respectively and wait to recover the most contiguous value of four valid pixels up and down of pixel, d u, d b, d l, d rBe respectively these four effective pixel points and wait to recover distance between the pixel.As shown in Figure 2.
This is covered the continuous level determinations that the result carries out the border, with the absolute difference of covering one circle pixel of inner periphery as a result and exterior circumferential one circle pixel (as shown in Figure 3) and (SAD) as the weighing criteria of continuous degree.The size of supposing to wait to cover piece is N * N, and the pixel in its inner upper left corner is p (x 1, y 1), level is to the right the X-axis positive direction, is the Y-axis positive direction vertically downward, then is formulated as:
SAD boundary = Σ x = x 1 x 1 + N - 1 | p ( x , y 1 - 1 ) - p ( x , y 1 ) | + Σ x = x 1 x 1 + N - 1 | p ( x , y 1 + N - 1 ) - p ( x , y 1 + N ) |
+ Σ y = y 1 y 1 + N - 1 | p ( x 1 - 1 , y ) - p ( x 1 , y ) | + Σ y = y 1 y 1 + N - 1 | p ( x 1 + N - 1 , y ) - p ( x 1 + N , y ) |
SAD is more little, and then degree is good more continuously.If continuously degree is lower and continuous degree its four neighborhood pieces are higher relatively, show that then to cover effect not good enough, this is often because errored block has comprised the edge that crosses wherein.To this situation, then use the method for directional interpolation instead and carry out code error shielding.At this moment the pattern of covering is called the pattern of covering 4.So far, first errored block disposes, and it covers the pixel that comprised of back will be as valid pixel in follow-up covering.
Then, once more simultaneously according to raster scan order scanning, investigate which width of cloth experience errored block in left and right two width of cloth images, and it is carried out above-mentioned same covering.Covering and so forth all covered until last two width of cloth images and to be finished.
As can be seen, cover the live part in the time of well to cover and to make it to become another order image and cover the errored block in the order image in the region of search according to above-mentioned order.Clearly, the mode of covering of this complementation helps improving the effect of covering.
Cover the multi-mode method for shielding error code of mode in the stereo-picture of the present invention based on complementation, the steps include:
1) earlier simultaneously according to raster scan order scanning, detects first pending errored block in left and right two width of cloth images;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, promptly cover pattern 1, jump to step 5) then and continue to carry out;
4), then adopt the method for piece coupling to find match block if comprise inactive pixels in the region of search; If the piece that finds does not comprise inactive pixels, then directly cover with match block, promptly cover pattern 2; If comprise inactive pixels, then adopt and cover mode 3 and handle;
5) to covering the continuous level determinations that the result carries out the border, if it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then use the method for directional interpolation instead to carry out code error shielding, promptly cover pattern 4;
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process, so complementary covers, and all covers until last two width of cloth images to finish.
Adopt the method for piece coupling to find the step of match block to be: to adopt four neighborhood pieces up and down of pending errored block to carry out the piece coupling, piece for each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment; If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most, and four neighborhood pieces are all done same above-mentioned processing; Difference vector according to adjacent block should equate or close principle then, reject the bigger vector of deviation in these four difference vectors, they may be the difference vectors of mistake coupling, the difference vector of remainder are asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding match block with this difference vector.
The step of covering mode 3 is: at first cover errored block with effective pixel in the match block that finds, and for the zone of inactive pixels correspondence, then cover with the method for bilinear interpolation, and during bilinear interpolation, only utilize the valid pixel value of present frame at present frame.
Beneficial effect:
1) owing to utilized the correlation of images of left and right eyes two width of cloth images to cover, therefore, subjective quality and the objective quality (is weighing criteria with Y-PSNR PSNR) of covering the result all are better than concealing method in the simple frame;
2) owing to can select the different patterns of covering for use according to different situations adaptively, therefore, subjective quality and the objective quality (is weighing criteria with PSNR) of covering the result all are better than single concealing method;
3) owing to cover and be based on that complimentary fashion carries out, therefore, subjective quality and the objective quality (is weighing criteria with PSNR) of covering the result all are better than having covered earlier the concealing method that a frame is covered another frame again;
4) when the average conduct of the difference vector of getting four neighborhood pieces is waited to cover the difference vector of piece, rejected the big difference vector of deviation, thereby improved the estimated accuracy of the difference vector of waiting to cover piece, helped improving subjective quality and the objective quality (is weighing criteria with PSNR) covered;
5) the inventive method can be expanded and be used for three-dimensional video-frequency the code error shielding problem of losing piece is all arranged.
Description of drawings
Fig. 1 is four neighborhood piece schematic diagrames of errored block;
Fig. 2 adopts bilinear interpolation to cover schematic diagram for the inactive pixels zone;
Fig. 3 is a schematic diagram of covering the inside and outside circle pixel of piece;
Fig. 4 is that the present invention realizes the code error shielding flow chart.
Embodiment
Cover the multi-mode method for shielding error code of mode in the stereo-picture of the present invention based on complementation, its specific embodiments is as follows:
1), detects which width of cloth of left and right two width of cloth images and meet with errored block earlier, and choose the errored block that suffers from earlier as first pending errored block simultaneously according to raster scan order scanning;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, jump to step 5) then and continue to carry out.
4), then adopt four neighborhood pieces to carry out the piece coupling if comprise inactive pixels in the region of search.For the piece of each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment.If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most.Four neighborhood pieces are all done same above-mentioned processing.Difference vector according to adjacent block should equate or close principle then, rejects the bigger vector of deviation in these four difference vectors, and its () may be the difference vector of mistake coupling.Remaining difference vector is asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding piece with this difference vector.If the piece that finds does not comprise inactive pixels, then directly cover pending errored block with the content of this piece; If the corresponding blocks that finds comprises inactive pixels, the zone that can't cover because of inactive pixels then adopts the method for bilinear interpolation to cover in present frame, and during bilinear interpolation, only utilizes the value of present frame valid pixel.
5) to covering the continuous level determinations that the result carries out the border.With the absolute difference of covering one circle pixel of inner periphery as a result and exterior circumferential one circle pixel and (SAD) as the weighing criteria of degree continuously, SAD is more little, then continuous degree is good more.If it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then uses the method for directional interpolation instead and carry out code error shielding.
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process.So complementary covers, and all covers until last two width of cloth images to finish.

Claims (3)

1. cover the multi-mode method for shielding error code of mode in the stereo-picture based on complementation, it is characterized in that the steps include:
1) earlier simultaneously according to raster scan order scanning, detects first pending errored block in left and right two width of cloth images;
2) check whether all pixels all are effective in the region of search;
3) if all pixels all are effective in the region of search, then the pixel that pending errored block week is made a circle is done as a whole, make it to be put in the region of search of reference frame and carry out the border coupling, and cover pending errored block with the content that the most similar integral body comprised that matches, promptly cover pattern 1, jump to step 5) then and continue to carry out;
4), then adopt the method for piece coupling to find match block if comprise inactive pixels in the region of search; If the piece that finds does not comprise inactive pixels, then directly cover with match block, promptly cover pattern 2; If comprise inactive pixels, then adopt and cover mode 3 and handle;
5) to covering the continuous level determinations that the result carries out the border, if it is lower and continuous degree its four neighborhood pieces are higher relatively to cover result's continuous degree, then use the method for directional interpolation instead to carry out code error shielding, promptly cover pattern 4;
6), investigate which width of cloth meets with errored block in left and right two width of cloth images, and it carried out above-mentioned steps 2 once more simultaneously according to raster scan order scanning)-5) process, so complementary covers, and all covers until last two width of cloth images to finish.
2. cover the multi-mode method for shielding error code of mode in the stereo-picture according to claim 1 based on complementation, it is characterized in that adopting the method for piece coupling to find the step of match block to be: adopt four neighborhood pieces up and down of pending errored block to carry out the piece coupling, piece for each neighborhood, if there are not other errored blocks in the region of search of reference frame, when searching the errored block that mates most, record difference vector at this moment; If errored block is arranged in the region of search, then to skip and do not match for the zone that errored block is arranged, all the other zones are coupling as usual, and notes the difference vector when mating most, and four neighborhood pieces are all done same above-mentioned processing; Difference vector according to adjacent block should equate or close principle then, reject the bigger vector of deviation in these four difference vectors, they may be the difference vectors of mistake coupling, the difference vector of remainder are asked on average, and as the difference vector of current pending errored block; In reference frame, find corresponding match block with this difference vector.
3. cover the multi-mode method for shielding error code of mode in the stereo-picture according to claim 1 based on complementation, it is characterized in that the step of covering mode 3 is: at first cover errored block with effective pixel in the match block that finds, and for the zone of inactive pixels correspondence, then cover with the method for bilinear interpolation at present frame, and during bilinear interpolation, only utilize the valid pixel value of present frame.
CN 200910234278 2009-11-16 2009-11-16 Multi-mode error code covering method based on complementary covering mode in dimensional images Expired - Fee Related CN101715139B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200910234278 CN101715139B (en) 2009-11-16 2009-11-16 Multi-mode error code covering method based on complementary covering mode in dimensional images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200910234278 CN101715139B (en) 2009-11-16 2009-11-16 Multi-mode error code covering method based on complementary covering mode in dimensional images

Publications (2)

Publication Number Publication Date
CN101715139A true CN101715139A (en) 2010-05-26
CN101715139B CN101715139B (en) 2011-04-20

Family

ID=42418323

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200910234278 Expired - Fee Related CN101715139B (en) 2009-11-16 2009-11-16 Multi-mode error code covering method based on complementary covering mode in dimensional images

Country Status (1)

Country Link
CN (1) CN101715139B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103796028A (en) * 2014-02-26 2014-05-14 北京大学 Motion searching method and device based on image information in video coding
CN108377359A (en) * 2018-03-14 2018-08-07 苏州科达科技股份有限公司 Video anti-error code method, device, electronic equipment, storage medium
CN110335389A (en) * 2019-07-01 2019-10-15 上海商汤临港智能科技有限公司 Car door unlocking method and device, system, vehicle, electronic equipment and storage medium
CN110766625A (en) * 2019-10-15 2020-02-07 中国科学院光电技术研究所 Method for rapidly filling digital image mask
US10719991B2 (en) * 2016-06-08 2020-07-21 Sony Interactive Entertainment Inc. Apparatus and method for creating stereoscopic images using a displacement vector map

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1269359C (en) * 2003-03-03 2006-08-09 西南交通大学 Video error blanketing method based on motion vector extrapolation and motion vector search
US20100061452A1 (en) * 2007-01-04 2010-03-11 Thomson Licensing Corporation Method and apparatus for video error concealment using high level syntax reference views in multi-view coded video
CN100531400C (en) * 2007-07-26 2009-08-19 上海交通大学 Video error coverage method based on macro block level and pixel motion estimation
CN101370145B (en) * 2007-08-13 2010-06-09 中兴通讯股份有限公司 Shielding method and apparatus for image frame

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103796028A (en) * 2014-02-26 2014-05-14 北京大学 Motion searching method and device based on image information in video coding
CN103796028B (en) * 2014-02-26 2017-09-29 北京大学 Method for searching motion based on image information in a kind of Video coding
US10719991B2 (en) * 2016-06-08 2020-07-21 Sony Interactive Entertainment Inc. Apparatus and method for creating stereoscopic images using a displacement vector map
CN108377359A (en) * 2018-03-14 2018-08-07 苏州科达科技股份有限公司 Video anti-error code method, device, electronic equipment, storage medium
CN108377359B (en) * 2018-03-14 2020-08-04 苏州科达科技股份有限公司 Video error code resisting method and device, electronic equipment and storage medium
CN110335389A (en) * 2019-07-01 2019-10-15 上海商汤临港智能科技有限公司 Car door unlocking method and device, system, vehicle, electronic equipment and storage medium
CN110766625A (en) * 2019-10-15 2020-02-07 中国科学院光电技术研究所 Method for rapidly filling digital image mask
CN110766625B (en) * 2019-10-15 2022-04-19 中国科学院光电技术研究所 Method for rapidly filling digital image mask

Also Published As

Publication number Publication date
CN101715139B (en) 2011-04-20

Similar Documents

Publication Publication Date Title
JP7210381B2 (en) Efficient Multiview Coding with Depth Map Estimation and Update
US10417748B2 (en) Filtering and edge encoding and decoding for depth maps
EP2491722B1 (en) Depth map generation techniques for conversion of 2d video data to 3d video data
WO2017080420A1 (en) Auxiliary data for artifacts –aware view synthesis
CN101937578B (en) Method for drawing virtual view color image
JP5970609B2 (en) Method and apparatus for unified disparity vector derivation in 3D video coding
US8488870B2 (en) Multi-resolution, multi-window disparity estimation in 3D video processing
US9736498B2 (en) Method and apparatus of disparity vector derivation and inter-view motion vector prediction for 3D video coding
CN101715139B (en) Multi-mode error code covering method based on complementary covering mode in dimensional images
US20140146139A1 (en) Depth or disparity map upscaling
CN109218736A (en) device and method for coding and decoding
CN109257598A (en) Adaptive partition coding
WO2013068548A2 (en) Efficient multi-view coding using depth-map estimate for a dependent view
Chen et al. LSVC: A learning-based stereo video compression framework
CN104918032B (en) Simplify the method that the block based on depth is split
EP2391135B1 (en) Method and device for processing depth image sequence
US20150016517A1 (en) Encoding device and encoding method, and decoding device and decoding method
CN108668135B (en) Stereoscopic video B frame error concealment method based on human eye perception
US20180084276A1 (en) Method of Lookup Table Size Reduction for Depth Modelling Mode in Depth Coding
Tai et al. An effiicient full frame algorithm for object-based error concealment in 3D depth-based video
Kamolrat et al. Adaptive motion-estimation-mode selection for depth video coding
CN103220533A (en) Method for hiding loss errors of three-dimensional video macro blocks
Chen et al. Depth map compression via edge-based inpainting
CN104994365B (en) A kind of method and 2D video three-dimensional methods for obtaining non-key frame depth image
CN104853216B (en) Block dividing method based on depth and electronic device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20110420

Termination date: 20151116

CF01 Termination of patent right due to non-payment of annual fee