WO2012026734A2 - Appareil de codage/décodage et procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur - Google Patents

Appareil de codage/décodage et procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur Download PDF

Info

Publication number
WO2012026734A2
WO2012026734A2 PCT/KR2011/006205 KR2011006205W WO2012026734A2 WO 2012026734 A2 WO2012026734 A2 WO 2012026734A2 KR 2011006205 W KR2011006205 W KR 2011006205W WO 2012026734 A2 WO2012026734 A2 WO 2012026734A2
Authority
WO
WIPO (PCT)
Prior art keywords
motion vector
depth image
image
encoding
apply
Prior art date
Application number
PCT/KR2011/006205
Other languages
English (en)
Korean (ko)
Other versions
WO2012026734A3 (fr
Inventor
오병태
박두식
위호천
Original Assignee
삼성전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 삼성전자 주식회사 filed Critical 삼성전자 주식회사
Publication of WO2012026734A2 publication Critical patent/WO2012026734A2/fr
Publication of WO2012026734A3 publication Critical patent/WO2012026734A3/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • One embodiment of the present invention relates to an apparatus and method for encoding / decoding for a 3D image, and more particularly, to an apparatus and method for sharing a motion vector of a color image when encoding and decoding a depth image.
  • the depth image is treated as an image independent from the color image, and then encoded / decoded using the conventional H.264 / MPEG-4 AVE compression method. In this case, a bit is generated for each of the depth image and the color image, causing a problem in the limited bandwidth.
  • depth images exhibit different properties than color images.
  • the depth image includes more low-frequency components than the color image, and the flat regions form distinct outlines and thus include frequency components in the intermediate band. Because of this property, it is difficult to expect high compression efficiency when applying block DCT (Discrete Cosine Transform) or quantization-based H.264 / MPEG-4 AVC.
  • block DCT Discrete Cosine Transform
  • quantization-based H.264 / MPEG-4 AVC quantization-based H.264 / MPEG-4 AVC.
  • An encoding apparatus includes a data extractor for extracting a motion vector and mode information of a color image; And a depth image encoder configured to encode the depth image by setting the extracted data as a motion vector and mode information of the depth image.
  • the encoding apparatus may further include an MVS determination unit that determines whether to apply motion vector sharing (MVS).
  • MVS motion vector sharing
  • Decoding apparatus includes an MVS determination unit for determining whether to apply the motion vector sharing; A data setting unit configured to set motion vector and mode information of a color image as motion vector and mode information of a depth image when the motion vector sharing is applied; And a depth image decoder which decodes the depth image using the motion vector and the mode information of the depth image.
  • An encoding method includes the steps of extracting a motion vector and mode information of a color image; And encoding the depth image by setting the extracted data as a motion vector and mode information of the depth image.
  • the encoding method according to an embodiment of the present invention may further include determining whether to apply Motion Vector Sharing (MVS).
  • MVS Motion Vector Sharing
  • Decoding method comprises the steps of determining whether to apply motion vector sharing; When applying the motion vector sharing, setting the motion vector and the mode information of the color image as the motion vector and the mode information of the depth image; And decoding the depth image using the motion vector and the mode information of the depth image.
  • the amount of bits generated when compressing and transmitting the depth image may be reduced.
  • whether the motion vector is shared or not is expressed as a flag or a threshold and transmitted to the decoding device, so that the decoding device may easily determine whether to apply the motion vector sharing.
  • the compression efficiency of the depth image may be improved by allocating a higher quantization parameter to a block to which motion vector sharing is applied than to a block to which motion vector sharing is not applied.
  • FIG. 1 is a diagram illustrating an encoding device and a decoding device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram illustrating a detailed configuration of an encoding apparatus according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing a detailed configuration of a decoding apparatus according to an embodiment of the present invention.
  • FIG. 4 is a diagram illustrating mode information on a depth image according to an embodiment of the present invention.
  • FIG. 5 illustrates an example of applying a quantization parameter differently for each block according to an embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating an encoding method according to an embodiment of the present invention.
  • FIG. 7 is a flowchart illustrating a decoding method according to an embodiment of the present invention.
  • FIG. 1 is a diagram illustrating an encoding device and a decoding device according to an embodiment of the present invention.
  • the encoding apparatus 101 may encode a depth image constituting a 3D image.
  • the encoded depth image may be transmitted to the decoding apparatus 102 in the form of a bitstream.
  • the encoding apparatus 101 may reduce the amount of bits generated for the depth image by sharing mode information with the motion vector of the color image. In detail, the encoding apparatus 101 may determine whether to share the mode information with the motion vector of the color image. In addition, when encoding the depth image, the encoding apparatus 101 may determine a method of processing the residual information of the depth image differently for each block.
  • the decoding apparatus 102 may decode the depth image using information indicating whether to share the mode information with the motion vector transmitted from the encoding apparatus 101.
  • FIG. 2 is a block diagram illustrating a detailed configuration of an encoding apparatus according to an embodiment of the present invention.
  • the encoding apparatus 201 may include an MVS determiner 202, a data extractor 203, and a depth image encoder 204.
  • the MVS determination unit 202 may determine whether to apply motion vector sharing (MVS) to the depth image. That is, when encoding the depth image, the MVS determination unit 202 may determine whether to apply motion vector sharing, which means sharing mode information and a motion vector of the color image with the depth image.
  • VMS motion vector sharing
  • the MVS determination unit 202 may use a motion vector sharing using a rate-distortion cost (RD cost) based on a rate of a depth image and a distortion of a composite image of an intermediate view. MVS) can be determined whether or not to apply.
  • the MVS determination unit 202 may determine whether to apply motion vector sharing for each block by determining a ratio distortion cost per block.
  • the MVS determination unit 202 may calculate a ratio distortion cost when the motion vector sharing is applied to each block and when it is not applied, and select a case where the ratio distortion cost is low.
  • the MVS determination unit 203 may calculate the distortion of the synthesized image of the intermediate view using at least one of a global disparity or a warping parameter. In addition, the MVS determination unit 203 may predict the distortion of the composite image of the intermediate view using the offset of the color image.
  • the MVS determination unit 203 may express information indicating that motion vector sharing is applied as a flag. For example, if the flag is 0, motion vector sharing is applied, and if the flag is 1, motion vector sharing is not applied. Such a flag may be set for each block constituting the depth image.
  • the decoding apparatus may determine whether motion vector sharing is applied to the block to be decoded through the flag.
  • the MVS determiner 203 may determine the ratio distortion cost based on the distortion of the composite image at the intermediate view and the ratio of the depth image to at least one of the remaining information of the color image, the flatness of the color image, and the flatness of the depth image.
  • Rate-Distortion Cost can be calculated to encode a threshold that represents the optimal rate of distortion.
  • the remaining information of the color image, the flatness of the color image, and the flatness of the depth image refer to information that can be used in the decoding apparatus.
  • the MVS determination unit 203 may calculate the ratio distortion cost of each of the residual information of the color image, the flatness of the color image, and the flatness of the depth image, and transmit only threshold values representing the optimal ratio distortion cost to the decoding apparatus.
  • the depth image encoder 204 may encode the depth image by setting the motion vector and the mode information of the color image as the motion vector and the mode information of the depth image. For example, the depth image encoder 204 may encode the residual information of the depth image with a quantization parameter (QP) higher than a preset quantization parameter or skip (SKIP) the encoding of the residual information. Residual information of the depth image means a value remaining according to motion compensation based on a motion vector.
  • QP quantization parameter
  • SKIP skip
  • the depth image encoder 204 may apply the same quantization parameter to the entire image, or may apply a different quantization parameter for any one unit of a frame unit, a group of picture (GOP), or a block unit.
  • the depth image encoder 204 may transmit the encoded depth image to the decoding apparatus through a bitstream.
  • FIG. 3 is a block diagram showing a detailed configuration of a decoding apparatus according to an embodiment of the present invention.
  • the decoding apparatus 301 may include an MVS determination unit 302, a data setting unit 303, and a depth image decoding unit 304.
  • the MVS determination unit 302 may determine whether to apply motion vector sharing. For example, the MVS determination unit 302 may determine whether to apply motion vector sharing based on the flag. As another example, the MVS determination unit 302 may apply motion vector sharing based on one of the remaining information of the color image, the flatness of the color image, and the flatness of the depth image, and a threshold value indicating an optimal ratio distortion cost. Can be determined.
  • the encoding apparatus may determine a motion vector sharing in which the depth image shares the motion vector and mode information of the color image, and transmit notification information of motion vector sharing such as a flag and a threshold to the decoding apparatus. Then, the decoding apparatus may determine whether to apply the motion vector sharing based on the notification information of the motion vector sharing.
  • the data setting unit 303 may set the motion vector and the mode information of the color image as the motion vector and the mode information of the depth image.
  • the depth image decoder 304 may decode the depth image using the motion vector and the mode information of the depth image. If the motion vector sharing is not applied, the depth image decoder 304 may decode the depth image by determining an optimal motion vector and mode information in the depth image itself.
  • FIG. 4 is a diagram illustrating mode information on a depth image according to an embodiment of the present invention.
  • any one of intra prediction, motion estimation / motion compensation, or motion vector sharing (MVS) may be determined as mode information on the depth image when encoding the depth image. .
  • Intra prediction and motion estimation / motion compensation mean that encoding is performed by finding an optimal motion vector and mode information in the depth image itself.
  • motion vector sharing means setting the motion vector and the mode information found in the color image as the motion vector and the mode information of the depth image. Due to the sharing of motion vectors, the amount of bits generated in the depth image can be reduced.
  • the encoding apparatus may determine whether to apply motion vector sharing for each block.
  • the rate distortion cost may be applied to determine whether to apply motion vector sharing.
  • the 3D image unlike the conventional 2D image compression, when defining the distortion of the depth image, it is necessary to determine how accurately the composite image representing the intermediate view is generated rather than the depth image itself. This is because the depth image is a kind of additional information for generating a composite image of an intermediate view.
  • the encoding apparatus may determine whether to apply the motion vector sharing or the motion vector sharing for each block constituting the depth image through the rate distortion cost.
  • the ratio distortion cost mentioned in the present invention may be calculated based on the ratio of the depth image and the distortion of the composite image at the intermediate view. In fact, the distortion of the depth image may be expressed as the distortion of the composite image of the intermediate view.
  • the distortion of the composite image of the intermediate view may be actually calculated using at least one of global disparity or warping parameters.
  • the distortion of the composite image of the intermediate view may be predicted using the offset of the color image. For example, assuming that a depth image and a color image are acquired in a parallel camera, when the error of the depth image generated by compression or sharing of motion vectors is ⁇ D, the prediction error of the intermediate image of the intermediate view ( Dc) is shown in Equation 1 below.
  • the encoding apparatus may calculate the rate distortion cost when the motion vector sharing is applied to each block and when the motion vector sharing is not applied, thereby selecting a case in which the rate distortion cost is small.
  • the encoding apparatus may determine whether the motion vector sharing is applied for each block in the decoding apparatus through a flag indicating whether to apply the motion vector sharing.
  • the encoding apparatus may predict a bit value of the flag and transmit it to the decoding apparatus. Specifically, the encoding apparatus may reduce overhead caused by the bit value of the flag by selectively determining whether motion vector sharing is applied based on some information that can be obtained by the decoding apparatus. Examples of information that can be used in the decoding apparatus may include residual information of a color image, flatness of a color image, or flatness of a depth image.
  • the encoding apparatus calculates a ratio distortion cost for each information that can be used by the decoding apparatus, calculates a threshold value having an optimal ratio distortion cost, and sends only a partial threshold value to the bit value of the flag by not sending a flag for each block. Overhead headers can be reduced.
  • the threshold value may be changed in units of blocks, frames, or groups of pictures (GOP).
  • FIG. 5 illustrates an example of applying a quantization parameter differently for each block according to an embodiment of the present invention.
  • Blocks to which motion vector sharing is applied have a lower importance than residual blocks to which motion vector sharing is not applied. Accordingly, as shown in FIG. 5, the encoding apparatus may assign a higher quantization parameter to a block to which motion vector sharing is applied to lower the quality of the block. Alternatively, the encoding apparatus may skip without performing encoding of the residual information.
  • QP denotes a quantization parameter
  • a block denoted by QP denotes a block to which motion vector sharing is not applied
  • a block denoted by QP or a block denoted by SKIP denotes a block to which motion vector sharing is applied. do.
  • different quantization parameters may be applied to each block.
  • the quantization result is derived at a lower value, so that the depth image can be compressed / transmitted with fewer bits to benefit from substantially limited bandwidth.
  • a quantization parameter determined higher than a block to which motion vector sharing is not applied may be assigned the same value to the entire depth image.
  • different values may be assigned according to frame units, GOP units, or block units.
  • FIG. 6 is a flowchart illustrating an encoding method according to an embodiment of the present invention.
  • the encoding apparatus may determine whether to apply motion vector sharing (S601).
  • the encoding apparatus may determine whether to apply the motion vector sharing using the ratio distortion cost based on the ratio of the depth image and the distortion of the composite image of the intermediate view. In this case, the encoding apparatus may calculate the distortion of the synthesized image of the intermediate view using at least one of a global disparity or a warping parameter. Alternatively, the encoding apparatus may predict the distortion of the synthesized image of the intermediate view using the offset of the color image.
  • the encoding apparatus may encode whether to apply motion vector sharing by using a flag.
  • the encoding apparatus may include a rate distortion cost based on a ratio of a depth image and a distortion of a composite image at an intermediate point in time to at least one of residual information of a color image, flatness of a color image, and flatness of a depth image.
  • the encoding apparatus may extract the motion vector and the mode information of the color image (S602).
  • the encoding apparatus may encode the depth image by setting the motion vector and the mode information of the color image as the motion vector and the mode information of the depth image (S603).
  • the encoding apparatus may encode the depth image by finding an optimal motion vector and mode information in the depth image itself for a block to which motion vector sharing is not applied.
  • FIG. 7 is a flowchart illustrating a decoding method according to an embodiment of the present invention.
  • the decoding apparatus may determine whether to apply motion vector sharing (S701). For example, the decoding apparatus may determine whether to apply motion vector sharing based on a flag transmitted by the encoding apparatus. Alternatively, the decoding apparatus may determine whether to apply the motion vector sharing based on one of the remaining information of the color image, the flatness of the color image, and the flatness of the depth image, and a threshold value indicating an optimal ratio distortion cost. have.
  • the decoding apparatus may set the motion vector and the mode information of the color image as the motion vector and the mode information of the depth image (S702).
  • the decoding apparatus may decode the depth image using the motion vector and the mode information of the depth image (S703).
  • Methods according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium.
  • the computer readable medium may include program instructions, data files, data structures, etc. alone or in combination.
  • Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un appareil de codage/décodage et un procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur. Quand une image de profondeur est compressée, un vecteur de mouvement d'une image couleur et des informations de mode sont partagés afin de réduire la quantité de bits générés pour une image de profondeur et d'améliorer la résolution d'une image combinée.
PCT/KR2011/006205 2010-08-24 2011-08-23 Appareil de codage/décodage et procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur WO2012026734A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020100081873A KR20120018906A (ko) 2010-08-24 2010-08-24 컬러 영상과 깊이 영상의 모션 벡터 공유를 이용한 부호화/복호화 장치 및 방법
KR10-2010-0081873 2010-08-24

Publications (2)

Publication Number Publication Date
WO2012026734A2 true WO2012026734A2 (fr) 2012-03-01
WO2012026734A3 WO2012026734A3 (fr) 2012-05-03

Family

ID=45723917

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2011/006205 WO2012026734A2 (fr) 2010-08-24 2011-08-23 Appareil de codage/décodage et procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur

Country Status (2)

Country Link
KR (1) KR20120018906A (fr)
WO (1) WO2012026734A2 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104065972A (zh) * 2013-03-21 2014-09-24 乐金电子(中国)研究开发中心有限公司 一种深度图像编码方法、装置及编码器

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101957873B1 (ko) * 2012-06-04 2019-03-13 삼성전자주식회사 3차원 영상을 위한 영상 처리 장치 및 방법
WO2014051404A1 (fr) * 2012-09-28 2014-04-03 삼성전자 주식회사 Appareil de codage/décodage et procédé de codage/décodage utilisant le partage d'informations de codage/décodage d'image en couleurs et d'image de profondeur
JP2016528807A (ja) * 2013-07-18 2016-09-15 エルジー エレクトロニクス インコーポレイティド ビデオ信号処理方法及び装置
MX365706B (es) * 2013-12-20 2019-06-11 Samsung Electronics Co Ltd Metodo de codificacion de video usando prediccion inter-vista y dispositivo del mismo, y metodo de decodificacion de video y dispositivo del mismo.
US10630992B2 (en) 2016-01-08 2020-04-21 Samsung Electronics Co., Ltd. Method, application processor, and mobile terminal for processing reference image
CN116708782A (zh) 2016-07-12 2023-09-05 韩国电子通信研究院 图像编码/解码方法以及用于该方法的记录介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100014553A (ko) * 2007-04-25 2010-02-10 엘지전자 주식회사 비디오 신호의 인코딩/디코딩 방법 및 장치
KR20100064196A (ko) * 2008-12-04 2010-06-14 삼성전자주식회사 깊이 추정 장치 및 방법, 및 3ｄ 영상 변환 장치 및 방법
KR20100087685A (ko) * 2009-01-28 2010-08-05 한국전자통신연구원 깊이 영상의 품질 개선 방법 및 장치
KR20100091864A (ko) * 2009-02-10 2010-08-19 한국전자통신연구원 비디오 동영상의 움직이는 다중 객체 자동 분할 장치 및 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100014553A (ko) * 2007-04-25 2010-02-10 엘지전자 주식회사 비디오 신호의 인코딩/디코딩 방법 및 장치
KR20100064196A (ko) * 2008-12-04 2010-06-14 삼성전자주식회사 깊이 추정 장치 및 방법, 및 3ｄ 영상 변환 장치 및 방법
KR20100087685A (ko) * 2009-01-28 2010-08-05 한국전자통신연구원 깊이 영상의 품질 개선 방법 및 장치
KR20100091864A (ko) * 2009-02-10 2010-08-19 한국전자통신연구원 비디오 동영상의 움직이는 다중 객체 자동 분할 장치 및 방법

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104065972A (zh) * 2013-03-21 2014-09-24 乐金电子(中国)研究开发中心有限公司 一种深度图像编码方法、装置及编码器
CN104065972B (zh) * 2013-03-21 2018-09-28 乐金电子(中国)研究开发中心有限公司 一种深度图像编码方法、装置及编码器

Also Published As

Publication number Publication date
KR20120018906A (ko) 2012-03-06
WO2012026734A3 (fr) 2012-05-03

Similar Documents

Publication Publication Date Title
WO2012026734A2 (fr) Appareil de codage/décodage et procédé utilisant le partage de vecteurs de mouvement d'une image couleur et d'une image de profondeur
US11902575B2 (en) Image encoding method using a skip mode, and a device using the method
WO2010087589A2 (fr) Procédé et appareil pour traiter des signaux vidéo au moyen d'un codage en mode intra des contours
AU2020286361B2 (en) Method and apparatus for improved implicit transform selection
WO2012057528A2 (fr) Procédé de codage et de décodage à prédiction intra adaptative
WO2010068020A9 (fr) Appareil et procédé de décodage/codage de vidéo multivue
WO2012134085A2 (fr) Procédé pour décoder une image dans un mode de prévision interne
WO2012144830A2 (fr) Procédés et appareils de codage et de décodage d'image par filtrage adaptatif
EP1534018A2 (fr) Dispositif et méthode de génération de motifs de blocs codés pour des images de canaux alpha et dispositif et méthode pour le codage et le décodage d'images de canaux alpha.
US20220132113A1 (en) Video or image coding applying adaptive loop filter
WO2014107083A1 (fr) Procédé et dispositif de traitement de signal vidéo
WO2010058895A2 (fr) Appareil et procédé de codage/décodage d'un signal vidéo
WO2012081877A2 (fr) Appareil et procédé d'encodage/de décodage vidéo à vues multiples
WO2013039031A1 (fr) Encodeur d'image, module de décodage d'image, et procédé et programme associés
KR20110073263A (ko) 인트라 예측 부호화 방법 및 부호화 방법, 그리고 상기 방법을 수행하는 인트라 예측 부호화 장치 및 인트라 예측 복호화 장치
WO2011090352A2 (fr) Procédé et dispositif de codage/décodage de zone
WO2014073877A1 (fr) Procédé et appareil pour traiter un signal vidéo multi-vues
WO2012128241A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et programme
WO2018070568A1 (fr) Procédé et appareil de codage vidéo utilisant un schéma d'extraction de mode de prédiction intra basé sur un décodeur
ES2964763T3 (es) Decisión de intra-codificación temprana
WO2011078562A2 (fr) Procédés de codage et de décodage intra-prédictifs, et dispositifs de codage et de décodage intra-prédictifs pour exécution de procédés
WO2013141075A1 (fr) Dispositif de codage d'image, dispositif de décodage d'image, procédé de codage d'image, procédé de décodage d'image, et programme
WO2014077573A2 (fr) Procédé et appareil de traitement de signaux vidéo
WO2014073873A1 (fr) Procédé et appareil de traitement de signaux vidéo
WO2012099352A2 (fr) Dispositif et procédé de codage/décodage d'images à points de vue multiples

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11820161

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11820161

Country of ref document: EP

Kind code of ref document: A2