WO2017124305A1 - 基于多方式边界填充的全景视频编码、解码方法和装置 - Google Patents

基于多方式边界填充的全景视频编码、解码方法和装置 Download PDF

Info

Publication number
WO2017124305A1
WO2017124305A1 PCT/CN2016/071383 CN2016071383W WO2017124305A1 WO 2017124305 A1 WO2017124305 A1 WO 2017124305A1 CN 2016071383 W CN2016071383 W CN 2016071383W WO 2017124305 A1 WO2017124305 A1 WO 2017124305A1
Authority
WO
WIPO (PCT)
Prior art keywords
boundary
image
filling
reference sample
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/071383
Other languages
English (en)
French (fr)
Inventor
王振宇
王荣刚
姜秀宝
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to PCT/CN2016/071383 priority Critical patent/WO2017124305A1/zh
Priority to US15/747,864 priority patent/US10341682B2/en
Publication of WO2017124305A1 publication Critical patent/WO2017124305A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/55—Motion estimation with spatial constraints, e.g. at image or region borders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/563—Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present application relates to the field of digital video codec technology, and in particular, to a panoramic video coding and decoding method and apparatus based on multi-mode boundary padding.
  • panoramic images and panoramic video are an important part. Since panoramic video records the entire picture of a 360-degree view with a very high amount of data, compression of panoramic video is a key technology in virtual reality applications. As an emerging media, panoramic video has the characteristics of large field of view, high resolution and large data volume compared with traditional video. With the panoramic video, the observer's viewpoint is unchanged, and the observation direction can be changed to observe all the surrounding scenes, while the ordinary two-dimensional video only reflects a certain part of the panoramic video.
  • Cylindrical panoramic video is a common panoramic video that acts as a virtual camera that projects a three-dimensional object in space onto a cylinder.
  • the generation of cylindrical panoramic video can be acquired using a multi-camera or single camera acquisition system.
  • the field of view of the panoramic video is 5-6 times that of the ordinary video
  • the amount of data of the panoramic video is 5-6 times that of the ordinary video when the same visual quality is provided to the user.
  • the traditional video transmission scheme the use of panoramic video in a network environment becomes difficult.
  • block coding and transmission become a common scheme for panoramic video network transmission.
  • the transmission method of the cylindrical panoramic video mainly includes the following steps:
  • the panoramic image is segmented and the sequence of each image block is independently encoded.
  • the transmission medium can be the Internet, a wireless network, a local area network, an optical network, other suitable transmission medium, or a suitable combination of these transmission media.
  • the block sequences are independently decoded and projected to obtain the desired image.
  • the size of the block has an important influence on the coding efficiency of the panoramic video and the transmission area, and these two factors directly determine the amount of data to be transmitted. If the coding block size is small, the transmission area is small, but the coding efficiency is low; if the coding block size is large, the coding efficiency is high, but the transmission area is also large. Therefore, under the same visual quality, the amount of data to be transmitted is different for different coding block sizes.
  • panoramic video has certain speciality with respect to ordinary video, for example, the panoramic video has cyclicity, the picture has large distortion, etc.
  • a special coding technique is needed to improve the compression efficiency of the panoramic video, and how to improve the compression efficiency of the panoramic video is Year-round research in this field A problem.
  • the present application provides a panoramic video encoding and decoding method and apparatus based on multi-mode boundary padding, which can improve the compression efficiency of panoramic video.
  • the present application provides a panoramic video coding method based on multi-mode boundary padding, including:
  • the inter prediction includes a boundary filling step of: when the reference sample of the pixel in the current image block is outside the boundary of the corresponding reference image, according to The boundary of the reference sample is adaptively selected by the boundary filling method to obtain the sample value of the reference sample;
  • the residual block is transformed, quantized, and entropy encoded to obtain an encoded code stream; the sequence header or image header of the encoded code stream is written with a boundary padding mode selected in the boundary filling step.
  • the boundary filling mode step is adaptively selected according to the coordinates of the reference sample, including: when the ordinate of the reference sample is located on the upper boundary and the lower side of the reference image Within the boundary, and when the abscissa of the reference sample is outside the left and right boundaries of the reference image, the lateral image boundary filling method is selected to obtain the sample value of the reference sample; when the ordinate of the reference sample is located at the upper and lower boundaries of the reference image When outside, select the vertical image boundary fill method to obtain the sample value of the reference sample.
  • the horizontal image boundary filling manner adopts a cyclic filling manner
  • the vertical image boundary filling method adopts a cyclic filling manner
  • the horizontal image boundary filling manner adopts a cyclic filling manner.
  • the vertical image boundary filling method adopts a repeated filling manner.
  • the present application further provides a panoramic video coding apparatus based on multi-mode boundary padding, including:
  • An image dividing module configured to divide the current image into a plurality of image blocks
  • An inter prediction module configured to obtain a predicted image block of a current image block by inter prediction;
  • the inter prediction module includes a boundary filling unit, where a reference sample of the pixel in the current image block is at a boundary of the corresponding reference image
  • the boundary filling method is adaptively selected according to the coordinates of the reference sample to obtain the sample value of the reference sample;
  • a calculating module configured to subtract the current image block from the predicted image block to obtain a residual block
  • a code stream generating module configured to transform, quantize, and entropy encode the residual block to obtain an encoded code stream; and a boundary padding manner selected by the boundary filling unit is written in a sequence header or an image header of the encoded code stream.
  • the boundary filling unit is configured to adaptively select a boundary filling manner according to coordinates of the reference sample when the reference sample of the pixel in the current image block is outside the boundary of the corresponding reference image, to obtain a sample value of the reference sample, specifically
  • the boundary filling unit is configured to select a lateral image boundary filling method when the ordinate of the reference sample is located within the upper boundary and the lower boundary of the reference image, and the abscissa of the reference sample is outside the left and right boundaries of the reference image.
  • the sample value of the reference sample when the ordinate of the reference sample is outside the upper and lower boundaries of the reference image, the longitudinal image boundary fill mode is selected to obtain the sample value of the reference sample.
  • the boundary filling unit is configured to fill the horizontal image boundary when the ordinate of the reference sample is within the upper and lower boundaries of the reference image and the abscissa of the reference sample is outside the left and right boundaries of the reference image
  • the method adopts a cyclic filling method; when the ordinate of the reference sample is outside the upper boundary and the lower boundary of the reference image, the vertical image boundary filling manner adopts a cyclic filling manner.
  • the boundary filling unit is configured to: when the ordinate of the reference sample is within the upper and lower boundaries of the reference image, and the abscissa of the reference sample is outside the left and right boundaries of the reference image, the horizontal image boundary
  • the filling method adopts a cyclic filling method; when the ordinate of the reference sample is outside the upper boundary and the lower boundary of the reference image, the vertical image boundary filling method adopts a repeated filling manner.
  • the present application further provides a panoramic video decoding method based on multi-mode boundary padding, including:
  • the inter prediction includes a boundary filling step of: using a sequence header of the encoded code stream or a boundary filling manner recorded in the image header to obtain The sample value of the reference sample;
  • the predicted image block and the reconstructed residual block are added to obtain a reconstructed image block.
  • the boundary fill mode when the boundary fill mode is recorded in the sequence header of the coded code stream, the image of all sequences of the coded code stream is filled with image boundary using the recorded boundary fill manner; when the boundary fill mode is recorded When encoding the image header of the code stream, the image corresponding to the image header is filled with image boundaries using the recorded boundary fill mode.
  • the present application further provides a panoramic video decoding apparatus based on multi-mode boundary padding, including:
  • a residual block reconstruction module configured to perform entropy decoding, inverse quantization, and inverse transform on the encoded code stream to obtain a reconstructed residual block
  • An inter prediction module configured to obtain a predicted image block of a current image block by inter prediction;
  • the inter prediction module includes a boundary filling unit, configured to use a sequence header of the encoded code stream or a boundary filling manner recorded in the image header, To obtain the sample value of the reference sample;
  • An image block reconstruction module is configured to add the predicted image block and the reconstructed residual block to obtain a reconstructed image block.
  • the boundary filling unit is configured to: when the boundary filling manner is recorded in the sequence header of the encoded code stream, the image of all sequences of the encoded code stream is filled with the image boundary by using the recorded boundary filling manner; When the boundary filling method is recorded in the image header of the encoded code stream, the image corresponding to the image header is filled with the image boundary using the recorded boundary filling method.
  • the panoramic video encoding and decoding method and device based on multi-mode boundary filling provided by the present application fully utilizes the feature that the horizontal image content is cyclically connected in the panoramic video, and optimizes the image boundary filling method, so that the encoding end can be based on The reference sample's coordinates adaptively select a more reasonable boundary filling method to achieve the purpose of improving compression efficiency.
  • 1 is a schematic diagram of a transmission method of cylindrical panoramic video
  • FIG. 2 is a schematic flowchart of a method for encoding a panoramic video based on multi-mode boundary padding in an embodiment of the present application
  • 3 is a schematic diagram of boundary filling in an inter prediction process
  • FIG. 4 is a schematic diagram of a principle of adopting a cyclic filling method in an embodiment of the present application
  • FIG. 5 is a schematic diagram of a principle of using a cyclic filling method and a repeated filling method in an embodiment of the present application
  • FIG. 6 is a schematic block diagram of a panoramic video coding apparatus based on multi-mode boundary padding in an embodiment of the present application
  • FIG. 7 is a schematic flowchart of a method for decoding a panoramic video based on multi-mode boundary padding according to an embodiment of the present application
  • FIG. 8 is a schematic block diagram of a panoramic video decoding apparatus based on multi-mode boundary padding in an embodiment of the present application.
  • the inventive concept of the present application is to uniformly use a method of repeatedly filling an image boundary in a conventional video codec standard. For sample points outside the image boundary, the value of the point closest to the sample point on the image boundary is used as the sample value of the sample point.
  • the horizontal image content is cyclically connected, so this feature can be used to optimize the image boundary filling method to achieve the purpose of improving compression efficiency. Therefore, in the panoramic video coding and decoding method and apparatus based on multi-mode boundary padding provided by the present application, a more reasonable boundary filling mode is adaptively selected according to the coordinates of the reference sample at the encoding end, so as to achieve the purpose of improving compression efficiency.
  • the embodiment provides a panoramic video coding method based on multi-mode boundary padding, including the following steps:
  • Step 1.1 Divide the current image into several image blocks. Specifically, the size of the sliced image block can be selected according to actual needs.
  • Step 1.2 Obtain a predicted image block by inter prediction.
  • step 1.2 includes a boundary filling step (step 1.21), and step 1.21 is: when the reference sample of the pixel in the current image block is outside the boundary of the corresponding reference image, adaptively selecting the boundary filling manner according to the coordinate of the reference sample. To obtain the sample value of the reference sample.
  • the coordinates of the current image block are added to the motion vector MV, and the new coordinate obtained is the coordinates of the predicted image block on the reference image.
  • Inter prediction is based on the coordinates of the predicted image block on the reference image as the predicted value of the current image block.
  • the pixel corresponding to the corresponding position in the image block is referred to as the reference sample of the pixel in the current image block.
  • the predicted image block on the reference image may be partially or completely outside the boundary of the reference image.
  • the current image block is image block A
  • the motion vector is MV
  • the predicted image block on the reference image is an image block.
  • a portion of image block B is outside the boundaries of the reference image, so the pixel values for this portion need to be calculated by boundary padding.
  • the horizontal image boundary filling manner is selected to obtain the reference sample.
  • Sample value when the ordinate of the reference sample is outside the upper and lower boundaries of the reference image, the longitudinal image boundary fill mode is selected to obtain the sample value of the reference sample.
  • the horizontal image boundary filling manner adopts cyclic filling.
  • the vertical image boundary filling method adopts a cyclic filling manner.
  • FIG. 4 is a schematic diagram of the principle of adopting the cyclic filling mode in the embodiment.
  • the loop filling manner may take the following form:
  • a sample point coordinate is (a x , a y ), as sample point A, when 0 ⁇ a y ⁇ h, and a x ⁇ 0 or a x >w–1 (since the pixel coordinates are from 0, the image width is w, the abscissa of the internal pixels of the image)
  • the sample value of sample point A is equal to the sample value of the sample point (w-(a x %w), a y ).
  • sample point coordinate is (b x , b y )
  • sample point B is recorded as sample point B.
  • the sample value of sample point B is equal to the sample point (b x , h-( Sample value of b y %h)).
  • the "%" operator is a modulo operation.
  • the horizontal image boundary filling manner adopts a cyclic filling manner.
  • the vertical image boundary filling method adopts a repeated filling manner.
  • FIG. 5 is a schematic diagram of a principle of a cyclic filling method and a repeated filling method.
  • the loop filling method and the repeated filling method may respectively take the following forms:
  • the upper left corner of the image is the coordinate far point; the horizontal direction is the x coordinate, the right direction is the positive direction; the vertical direction is the y coordinate, and the downward direction is the positive direction; the image width is w, and the image height is h.
  • the coordinates of a sample point are (a x , a y ), it is recorded as sample point A.
  • the sample value of sample point A is equal to the sample.
  • the sample value of the point (w-(a x %w), a y ).
  • the "%" operator is a modulo operation.
  • sample point coordinate is (b x , b y )
  • it is recorded as sample point B.
  • b y the sample value of sample point B is equal to the sample value of sample point (b x , 0); when b y > At h–1, the sample value of sample point B is equal to the sample value of the sample point (b x , h-1).
  • the calculation of the sample value may also be changed accordingly.
  • Step 1.3 Subpixels of the same position of the predicted image block are subtracted from each pixel of the current image block to obtain a residual block.
  • Step 1.4 Transform and quantize the residual block to obtain a quantized block.
  • each coefficient of the quantized block and the motion vector of the current image block are written into the encoded code stream by entropy coding.
  • the selected corresponding boundary filling manner needs to be written in the encoded code stream for decoding.
  • the boundary padding mode selected in the boundary filling step is written in the sequence header or the image header of the encoded code stream.
  • the information of the boundary filling mode is selected to be written into the sequence header or the image header, and can be selected according to actual needs.
  • Writing to the sequence header means that all images in the sequence are filled in the same way, and the image header no longer needs to write additional identifiers, which saves the overhead of the flag itself; writing the image header can only identify the current image filling method.
  • Each frame of image can be filled in different ways, which is more flexible.
  • the embodiment provides a panoramic video coding device based on multi-mode boundary padding, including an image division module 101 and an interframe. Prediction module 102, calculation module 103, and code stream generation module 104.
  • the image dividing module 101 is configured to divide the current image into a plurality of image blocks. Specifically, the size of the sliced image block can be selected according to actual needs.
  • the inter prediction module 102 is configured to obtain a predicted image block of the current image block by inter prediction.
  • the inter prediction module 102 includes a boundary filling unit 1021, configured to adaptively select a boundary filling manner according to coordinates of the reference sample when the reference samples of the pixels in the current image block are outside the boundary of the corresponding reference image. To obtain the sample value of the reference sample.
  • the coordinates of the current image block are added to the motion vector MV, and the new coordinate obtained is the coordinates of the predicted image block on the reference image.
  • Inter prediction is based on the coordinates of the predicted image block on the reference image as the predicted value of the current image block.
  • the pixel corresponding to the corresponding position in the image block is referred to as the reference sample of the pixel in the current image block.
  • the predicted image block on the reference image may be partially or completely outside the boundary of the reference image.
  • the current image block is image block A
  • the motion vector is MV
  • the predicted image block on the reference image is an image block.
  • a portion of image block B is outside the boundaries of the reference image, so the pixel values for this portion need to be calculated by boundary padding.
  • the boundary filling unit 1021 is configured to select a horizontal image boundary when the ordinate of the reference sample is located within the upper boundary and the lower boundary of the reference image, and the abscissa of the reference sample is located outside the left and right boundaries of the reference image.
  • the filling method determines the sample value of the reference sample; when the ordinate of the reference sample is outside the upper and lower boundaries of the reference image, the longitudinal image boundary filling method is selected to obtain the sample value of the reference sample.
  • the boundary filling unit 1021 is configured to be when the ordinate of the reference sample is located within the upper boundary and the lower boundary of the reference image, and the abscissa of the reference sample is located outside the left and right boundaries of the reference image.
  • the image boundary filling method adopts a cyclic filling method; when the ordinate of the reference sample is outside the upper boundary and the lower boundary of the reference image, the vertical image boundary filling manner adopts a cyclic filling manner.
  • FIG. 4 is a schematic diagram of the principle of adopting the cyclic filling mode in the embodiment.
  • the loop filling manner may take the following form:
  • a sample point coordinate is (a x , a y ), as sample point A, when 0 ⁇ a y ⁇ h, and a x ⁇ 0 or a x >w–1 (since the pixel coordinates are from 0, the image width is w, the abscissa of the internal pixels of the image)
  • the sample value of sample point A is equal to the sample value of the sample point (w-(a x %w), a y ).
  • sample point coordinate is (b x , b y )
  • sample point B is recorded as sample point B.
  • the sample value of sample point B is equal to the sample point (b x , h-( Sample value of b y %h)).
  • the "%" operator is a modulo operation.
  • the boundary filling unit 1021 is configured to be within the upper and lower boundaries of the reference image when the ordinate of the reference sample is located, and the abscissa of the reference sample is located at the left of the reference image
  • the horizontal image boundary filling method adopts a cyclic filling manner; when the ordinate of the reference sample is outside the upper boundary and the lower boundary of the reference image, the vertical image boundary filling manner adopts a repeated filling manner.
  • FIG. 5 is a schematic diagram of a principle of a cyclic filling method and a repeated filling method.
  • the loop filling method and the repeated filling method may respectively take the following forms:
  • the upper left corner of the image is the coordinate far point; the horizontal direction is the x coordinate, the right direction is the positive direction; the vertical direction is the y coordinate, and the downward direction is the positive direction; the image width is w, and the image height is h.
  • the coordinates of a sample point are (a x , a y ), it is recorded as sample point A.
  • the sample value of sample point A is equal to the sample.
  • the sample value of the point (w-(a x %w), a y ).
  • the "%" operator is a modulo operation.
  • sample point coordinate is (b x , b y )
  • it is recorded as sample point B.
  • b y the sample value of sample point B is equal to the sample value of sample point (b x , 0); when b y > At h–1, the sample value of sample point B is equal to the sample value of the sample point (b x , h-1).
  • the calculation of the sample value may also be changed accordingly.
  • the calculation module 103 is configured to subtract the current image block from the predicted image block to obtain a residual block.
  • the code stream generation module 104 is configured to transform, quantize, and entropy encode the residual block to obtain an encoded code stream.
  • the boundary padding mode selected by the boundary padding unit 1021 is written in the sequence header or image header of the encoded code stream for decoding.
  • the information of the boundary filling mode is selected to be written into the sequence header or the image header, and can be selected according to actual needs.
  • Writing to the sequence header means that all images in the sequence are filled in the same way, and the image header no longer needs to write additional identifiers, which saves the overhead of the flag itself; writing the image header can only identify the current image filling method.
  • Each frame of image can be filled in different ways, which is more flexible.
  • the present embodiment provides a multi-mode boundary padding-based panoramic video decoding method, which includes the following steps:
  • Step 2.1 Entropy decoding, inverse quantization and inverse transform on the encoded code stream to obtain a reconstructed residual block.
  • Step 2.2 Obtain a predicted image block by inter prediction.
  • step 2.2 includes a boundary filling step (step 2.21), and step 2.21 is: using a sequence header of the encoded code stream or a boundary filling manner recorded in the image header to obtain a sample value of the reference sample.
  • step 2.21 is the same as step 1.21 in the first embodiment, and details are not described herein again.
  • the boundary filling mode when the boundary filling mode is recorded in the sequence header of the encoded code stream, the image of all sequences of the encoded code stream is filled with the image boundary by the recorded boundary filling manner; when the boundary filling mode is recorded in the encoded code stream The image corresponding to the image header when in the image header Image boundary fill is performed using the recorded boundary fill method.
  • Step 2.3 Add the predicted image block and the reconstructed residual block to obtain the reconstructed image block.
  • the present embodiment provides a multi-mode boundary padding-based panoramic video decoding device, including a residual block reconstruction module 201 , Inter prediction module 202 and image block reconstruction module 203.
  • the residual block reconstruction module 201 is configured to perform entropy decoding, inverse quantization, and inverse transform on the encoded code stream to obtain a reconstructed residual block.
  • the inter prediction module 202 is configured to obtain a predicted image block of the current image block by inter prediction.
  • the inter prediction module 202 includes a boundary filling unit 2021, configured to use a sequence header of the encoded code stream or a boundary filling manner recorded in the image header to obtain a sample value of the reference sample.
  • the boundary filling unit 2021 is the same as the boundary filling unit 1021 in the second embodiment, and details are not described herein again.
  • the boundary filling unit 2021 is configured to: when the boundary filling manner is recorded in the sequence header of the encoded code stream, the image of all sequences of the encoded code stream is filled with the image boundary by using the recorded boundary filling manner; When recorded in the image header of the encoded code stream, the image corresponding to the image header is filled with image boundaries using the recorded boundary fill mode.
  • the image block reconstruction module 203 is configured to add the predicted image block and the reconstructed residual block to obtain a reconstructed image block.
  • a video processing device may include an encoding device and/or a decoding device, the encoding device including an encoding process and a decoding process, and the decoding device includes a decoding process.
  • the decoding process of the decoding device is the same as the decoding process of the encoding device.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: a read only memory. Random access memory, disk or optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种基于多方式边界填充的全景视频编码、解码方法和装置，在通过帧间预测得到当前图像块的预测图像块时，帧间预测包括边界填充步骤，边界填充步骤为：当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值。本申请提供的基于多方式边界填充的全景视频编码、解码方法和装置中，充分利用了全景视频中，水平方向图像内容是循环相连的这一特性，优化图像边界填充方法，使得在编码端能够根据参考样本的坐标自适应选择更合理的边界填充方式，以达到提升压缩效率的目的。

Description

基于多方式边界填充的全景视频编码、解码方法和装置 技术领域
本申请涉及数字视频编解码技术领域，具体涉及一种基于多方式边界填充的全景视频编码、解码方法和装置。
背景技术
目前，虚拟现实技术和相关应用正在快速发展。在虚拟现实技术中，全景图像和全景视频是一个重要的组成部分。由于全景视频记录了360度视角的全部画面，具有极高的数据量，因此全景视频的压缩是虚拟现实应用中的一个关键技术。全景视频作为一种新兴的媒体，和传统的视频相比，具有视野大，分辨率高，数据量大等特点。利用全景视频，观察者视点不变，改变观察方向能够观察到周围的全部场景，而普通的二维视频只反应了全景视频的某个局部。
柱面全景视频是一种常见的全景视频，它相当于一个虚拟的摄像机，把空间中的三维物体投影到柱面上。柱面全景视频的生成可以利用多摄像头或者单摄像头采集系统采集而成。
由于全景视频的视野范围是普通视频的5～6倍，在给用户提供相同的视觉质量的情况下，全景视频的数据量是普通视频的5～6倍。如果按照传统的视频传输方案，全景视频在网络环境下的使用变得困难重重。但是，由于在同一时刻，用户所需要看到的内容只是全景视频的某一部分，所以分块编码与传输成为了全景视频网络传输的常见方案。
请参考图1，柱面全景视频的传输方法主要包括下面步骤：
对全景图像进行分块，并对每个图像块的序列独立进行编码。
之后选择所需要的编码后的数据进行传输。在此可以根据用户当前的视角选择数据。传输媒介可以是因特网、无线网络、局域网、光学网络、其它合适的传输媒介、或者这些传输媒介的适当组合。
最后解码端接收到数据之后，对这些块序列进行独立的解码和投影变换，得到所需图像。
在全景视频的分块编码中，分块的尺寸对于全景视频的编码效率以及传输区域有着重要的影响，而这两项因素直接决定着需要传输的数据量。如果编码块尺寸小，则传输区域较小，但是编码效率会较低；如果编码块尺寸大，则编码效率较高，但是传输区域也较大。所以在相同的视觉质量下，不同的编码块尺寸，需要传输的数据量是不一样的。
由于全景视频相对普通视频具有一定的特殊性，例如全景视频具有循环性，画面存在较大畸变等，需要使用一个特殊的编码技术以提高全景视频的压缩效率，而如何提高全景视频的压缩效率是本领域常年研究 的一个问题。
发明内容
本申请提供一种基于多方式边界填充的全景视频编码、解码方法和装置，可以提高全景视频的压缩效率。
根据本申请的第一方面，本申请提供了一种基于多方式边界填充的全景视频编码方法包括：
将当前图像划分为若干图像块；
通过帧间预测得到当前图像块的预测图像块；所述帧间预测包括边界填充步骤，所述边界填充步骤为：当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值；
将当前图像块与预测图像块相减，得到残差块；
对残差块进行变换、量化和熵编码，以得到编码码流；所述编码码流的序列头或图像头中写入有边界填充步骤中所选择的边界填充方式。
优选的，当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式步骤，包括：当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
在一实施例中，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
在另一实施例中，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
根据本申请的第二方面，本申请还提供了一种基于多方式边界填充的全景视频编码装置，包括：
图像划分模块，用于将当前图像划分为若干图像块；
帧间预测模块，用于通过帧间预测得到当前图像块的预测图像块；所述帧间预测模块包括边界填充单元，用于在当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值；
计算模块，用于将当前图像块与预测图像块相减，得到残差块；
码流生成模块，用于对残差块进行变换、量化和熵编码，以得到编码码流；所述编码码流的序列头或图像头中写入有边界填充单元所选择的边界填充方式。
优选的，边界填充单元用于在当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值，具体为：边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
在一实施例中，边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
在另一实施例中，边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
根据本申请的第三方面，本申请还提供了一种基于多方式边界填充的全景视频解码方法，包括：
对编码码流进行熵解码、反量化和反变换，以得到重建的残差块；
通过帧间预测得到当前图像块的预测图像块；所述帧间预测包括边界填充步骤，所述边界填充步骤为：采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值；
将预测图像块和重建的残差块相加，得到重建的图像块。
在一实施例中，当所述边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当所述边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像采用所记录的边界填充方式进行图像边界填充。
.
根据本申请的第四方面，本申请还提供了一种基于多方式边界填充的全景视频解码装置，包括：
残差块重建模块，用于对编码码流进行熵解码、反量化和反变换，以得到重建的残差块；
帧间预测模块，用于通过帧间预测得到当前图像块的预测图像块；所述帧间预测模块包括边界填充单元，用于采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值；
图像块重建模块，用于将预测图像块和重建的残差块相加，得到重建的图像块。
在一实施例中，边界填充单元用于当所述边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当所述边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像采用所记录的边界填充方式进行图像边界填充。
本申请提供的基于多方式边界填充的全景视频编码、解码方法和装置，充分利用了全景视频中，水平方向图像内容是循环相连的这一特性，优化图像边界填充方法，使得在编码端能够根据参考样本的坐标自适应选择更合理的边界填充方式，以达到提升压缩效率的目的。
附图说明
图1为柱面全景视频的传输方法示意图；
图2为本申请一种实施例中基于多方式边界填充的全景视频编码方法的流程示意图；
图3为帧间预测过程中，边界填充的示意图；
图4为本申请一种实施例中采用循环填充方式的原理示意图；
图5为本申请一种实施例中分别采用循环填充方式和重复填充方式的原理示意图；
图6为本申请一种实施例中基于多方式边界填充的全景视频编码装置的模块示意图；
图7为本申请一种实施例中基于多方式边界填充的全景视频解码方法的流程示意图；
图8为本申请一种实施例中基于多方式边界填充的全景视频解码装置的模块示意图。
具体实施方式
本申请的发明构思在于：在传统的视频编解码标准中，对图像边界统一使用重复填充的方法。对于图像边界外的样本点，使用图像边界上距离样本点最近的点的值作为样本点的样本值。但在全景视频中，水平方向图像内容是循环相连的，因此可以利用这一特性优化图像边界填充方法，以达到提升压缩效率的目的。所以，本申请提供的基于多方式边界填充的全景视频编码、解码方法和装置中，在编码端根据参考样本的坐标自适应选择更合理的边界填充方式，以达到提升压缩效率的目的。
下面通过具体实施方式结合附图对本申请作进一步详细说明。
实施例一
请参考图2，本实施例提供了一种基于多方式边界填充的全景视频编码方法，包括下面步骤：
步骤1.1：将当前图像划分为若干图像块。具体的，切分的图像块的大小可以根据实际需求选择。
步骤1.2：通过帧间预测得到预测图像块。本实施例中，步骤1.2包括边界填充步骤(步骤1.21)，步骤1.21为：当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值。
首先，需要说明的是，编解码过程中进行帧间预测时，当前图像块的坐标加上运动矢量MV，得到的新坐标就是参考图像上预测图像块的坐标。帧间预测就是根据该坐标将参考图像上的预测图像块的值取出，作为当前图像块的预测值。具体到图像块内的某一个像素时，预测图像块内对应位置的像素称作当前图像块内该像素的参考样本。但是参考图像上的预测图像块可能会有一部分或全部位于参考图像边界外，如图3所示，当前图像块为图像块A，运动矢量为MV，因此参考图像上的预测图像块为图像块B。图像块B的一部分位于参考图像边界之外，因此这一部分的像素值就需要通过边界填充计算出来。
本实施例中，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
优选的，本实施例中，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
请参考图4，为本实施例中采用循环填充方式的原理示意图。
具体的，本实施例中，循环填充方式可以采用下面形式：
对于一个宽为w，高为h的图像，以图像左上角为坐标原点，横向向右为x坐标正方向，纵向向下为y坐标正方向；如果一个样本点坐标为(ax,ay)，记为样本点A，当0<ay<h，且ax<0或ax>w–1(由于像素坐标是从0开始的，图像宽度为w时，图像内部像素的横坐标的取值范围是0到w-1)时，样本点A的样本值等于样本点(w-(ax％w),ay)的样 本值。如果一个样本点坐标为(bx,by)，记为样本点B，当by<0或by>h–1时，样本点B的样本值等于样本点(bx,h-(by％h))的样本值。其中，“％”运算符为取模运算。
在另一实施例中，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
请参考图5，为其分别采用循环填充方式和重复填充方式的原理示意图。具体的，循环填充方式和重复填充方式分别可以采用下面形式：
以图像左上角为坐标远点；横向为x坐标，向右为正方向；纵向为y坐标，向下为正方向；图像宽度为w，图像高度为h。如果一个样本点坐标为(ax,ay)，记为样本点A，当0<ay<h，且ax<0或ax>w–1时，样本点A的样本值等于样本点(w-(ax％w),ay)的样本值。其中，“％”运算符为取模运算。如果一个样本点坐标为(bx,by)，记为样本点B，当by<0时，样本点B的样本值等于样本点(bx,0)的样本值；当by>h–1时，样本点B的样本值等于样本点(bx,h-1)的样本值。
当然，在其他实施例中，如果坐标系发生了相应的改变，例如坐标原点不再取图像左上角，那么样本值的计算方式也可以相应进行改变。
步骤1.3：当前图像块每个像素减去预测图像块相同位置的像素，得到残差块。
步骤1.4：对残差块进行变换、量化得到量化块；最后通过熵编码将量化块的每个系数以及当前图像块的运动矢量写入编码码流。本实施例中，由于在帧间预测过程中选择了不同的边界填充方式，因此，在编码码流中需要写入所选择的相应的边界填充方式，以用于解码。具体的，在编码码流的序列头或图像头中写入边界填充步骤中所选择的边界填充方式。
需要说明是，具体是选择将边界填充方式的信息写入序列头还是图像头，可以根据实际需求选择。写入序列头就表示序列内所有图像都采用相同的方式填充，而图像头不再需要写入额外的标识，可以节省标识位本身的开销；写入图像头就只能标识当前图像的填充方式，每一帧图像都可以采用不同的方式填充，方式较为灵活。
实施例二
请参考图6，对应于实施例一提供的基于多方式边界填充的全景视频编码方法，本实施例相应提供了一种基于多方式边界填充的全景视频编码装置，包括图像划分模块101、帧间预测模块102、计算模块103和码流生成模块104。
图像划分模块101用于将当前图像划分为若干图像块。具体的，切分的图像块的大小可以根据实际需求选择。
帧间预测模块102用于通过帧间预测得到当前图像块的预测图像块。本实施例中，帧间预测模块102包括边界填充单元1021，用于在当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值。
首先，需要说明的是，编解码过程中进行帧间预测时，当前图像块的坐标加上运动矢量MV，得到的新坐标就是参考图像上预测图像块的坐标。帧间预测就是根据该坐标将参考图像上的预测图像块的值取出，作为当前图像块的预测值。具体到图像块内的某一个像素时，预测图像块内对应位置的像素称作当前图像块内该像素的参考样本。但是参考图像上的预测图像块可能会有一部分或全部位于参考图像边界外，如图3所示，当前图像块为图像块A，运动矢量为MV，因此参考图像上的预测图像块为图像块B。图像块B的一部分位于参考图像边界之外，因此这一部分的像素值就需要通过边界填充计算出来。
本实施例中，边界填充单元1021用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
优选的，本实施例中，边界填充单元1021用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
请参考图4，为本实施例中采用循环填充方式的原理示意图。
具体的，本实施例中，循环填充方式可以采用下面形式：
对于一个宽为w，高为h的图像，以图像左上角为坐标原点，横向向右为x坐标正方向，纵向向下为y坐标正方向；如果一个样本点坐标为(ax,ay)，记为样本点A，当0<ay<h，且ax<0或ax>w–1(由于像素坐标是从0开始的，图像宽度为w时，图像内部像素的横坐标的取值范围是0到w-1)时，样本点A的样本值等于样本点(w-(ax％w),ay)的样本值。如果一个样本点坐标为(bx,by)，记为样本点B，当by<0或by>h–1时，样本点B的样本值等于样本点(bx,h-(by％h))的样本值。其中，“％”运算符为取模运算。
在另一实施例中，边界填充单元1021用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左 边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
请参考图5，为其分别采用循环填充方式和重复填充方式的原理示意图。具体的，循环填充方式和重复填充方式分别可以采用下面形式：
以图像左上角为坐标远点；横向为x坐标，向右为正方向；纵向为y坐标，向下为正方向；图像宽度为w，图像高度为h。如果一个样本点坐标为(ax,ay)，记为样本点A，当0<ay<h，且ax<0或ax>w–1时，样本点A的样本值等于样本点(w-(ax％w),ay)的样本值。其中，“％”运算符为取模运算。如果一个样本点坐标为(bx,by)，记为样本点B，当by<0时，样本点B的样本值等于样本点(bx,0)的样本值；当by>h–1时，样本点B的样本值等于样本点(bx,h-1)的样本值。
当然，在其他实施例中，如果坐标系发生了相应的改变，例如坐标原点不再取图像左上角，那么样本值的计算方式也可以相应进行改变。
计算模块103用于将当前图像块与预测图像块相减，得到残差块。
码流生成模块104用于对残差块进行变换、量化和熵编码，以得到编码码流。编码码流的序列头或图像头中写入有边界填充单元1021所选择的边界填充方式，以用于解码。
需要说明是，具体是选择将边界填充方式的信息写入序列头还是图像头，可以根据实际需求选择。写入序列头就表示序列内所有图像都采用相同的方式填充，而图像头不再需要写入额外的标识，可以节省标识位本身的开销；写入图像头就只能标识当前图像的填充方式，每一帧图像都可以采用不同的方式填充，方式较为灵活。
实施例三
请参考图7，基于实施例一提供的基于多方式边界填充的全景视频编码方法，本实施例相应提供了一种基于多方式边界填充的全景视频解码方法，包括下面步骤：
步骤2.1：对编码码流进行熵解码、反量化和反变换，以得到重建的残差块。
步骤2.2：通过帧间预测得到预测图像块。本实施例中，步骤2.2包括边界填充步骤(步骤2.21)，步骤2.21为：采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值。步骤2.21与实施例一中的步骤1.21相同，此处不再赘述。
本实施例中，当边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像 采用所记录的边界填充方式进行图像边界填充。
步骤2.3：将预测图像块和重建的残差块相加，得到重建的图像块
实施例四
请参考图8，对应于实施例三提供的基于多方式边界填充的全景视频解码方法，本实施例相应提供了一种基于多方式边界填充的全景视频解码装置，包括残差块重建模块201、帧间预测模块202和图像块重建模块203。
残差块重建模块201用于对编码码流进行熵解码、反量化和反变换，以得到重建的残差块。
帧间预测模块202用于通过帧间预测得到当前图像块的预测图像块。本实施例中，帧间预测模块202包括边界填充单元2021，用于采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值。边界填充单元2021与实施例二中的边界填充单元1021相同，此处不再赘述。
本实施例中，边界填充单元2021用于当边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像采用所记录的边界填充方式进行图像边界填充。
图像块重建模块203用于将预测图像块和重建的残差块相加，得到重建的图像块。
需要说明的是，本申请实施例中，仅对全景视频编解码帧间预测过程中的边界填充步骤进行了详细说明，对于全景视频编解码过程中的其他步骤，皆可以采用现有技术中的任意一种可行方法。另外，通常，视频处理装置可包括编码装置和/或解码装置，编码装置包括编码过程和解码过程，解码装置包括解码过程。解码装置的解码过程与编码装置的解码过程相同。
本领域技术人员可以理解，上述实施方式中各种方法的全部或部分步骤可以通过程序来控制相关硬件完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器、随机存取存储器、磁盘或光盘等。
以上内容是结合具体的实施方式对本申请所作的进一步详细说明，不能认定本申请的具体实施只局限于这些说明。对于本申请所属技术领域的普通技术人员来说，在不脱离本申请发明构思的前提下，还可以做出若干简单推演或替换。

Claims (12)

  1. 一种基于多方式边界填充的全景视频编码方法，其特征在于，包括：
    将当前图像划分为若干图像块；
    通过帧间预测得到当前图像块的预测图像块；所述帧间预测包括边界填充步骤，所述边界填充步骤为：当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值；
    将当前图像块与预测图像块相减，得到残差块；
    对残差块进行变换、量化和熵编码，以得到编码码流；所述编码码流的序列头或图像头中写入有边界填充步骤中所选择的边界填充方式。
  2. 如权利要求1所述的方法，其特征在于，当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式步骤，包括：当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
  3. 如权利要求2所述的方法，其特征在于，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
  4. 如权利要求2所述的方法，其特征在于，当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
  5. 一种基于多方式边界填充的全景视频编码装置，其特征在于，包括：
    图像划分模块，用于将当前图像划分为若干图像块；
    帧间预测模块，用于通过帧间预测得到当前图像块的预测图像块；所述帧间预测模块包括边界填充单元，用于在当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值；
    计算模块，用于将当前图像块与预测图像块相减，得到残差块；
    码流生成模块，用于对残差块进行变换、量化和熵编码，以得到编 码码流；所述编码码流的序列头或图像头中写入有边界填充单元所选择的边界填充方式。
  6. 如权利要求5所述的装置，其特征在于，边界填充单元用于在当前图像块中像素的参考样本在相应的参考图像的边界之外时，根据参考样本的坐标自适应选择边界填充方式，以求得参考样本的样本值，具体为：边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，选择横向图像边界填充方式求得参考样本的样本值；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，选择纵向图像边界填充方式求得参考样本的样本值。
  7. 如权利要求6所述的装置，其特征在于，边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用循环填充方式。
  8. 如权利要求6所述的装置，其特征在于，边界填充单元用于在当参考样本的纵坐标位于参考图像上边界和下边界以内，且参考样本的横坐标位于参考图像左边界和右边界之外时，横向图像边界填充方式采用循环填充方式；在当参考样本的纵坐标位于参考图像上边界和下边界之外时，纵向图像边界填充方式采用重复填充方式。
  9. 一种基于多方式边界填充的全景视频解码方法，其特征在于，包括：
    对编码码流进行熵解码、反量化和反变换，以得到重建的残差块；
    通过帧间预测得到当前图像块的预测图像块；所述帧间预测包括边界填充步骤，所述边界填充步骤为：采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值；
    将预测图像块和重建的残差块相加，得到重建的图像块。
  10. 如权利要求9所述的方法，其特征在于，当所述边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当所述边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像采用所记录的边界填充方式进行图像边界填充。
  11. 一种基于多方式边界填充的全景视频解码装置，其特征在于，包括：
    残差块重建模块，用于对编码码流进行熵解码、反量化和反变换，以得到重建的残差块；
    帧间预测模块，用于通过帧间预测得到当前图像块的预测图像块； 所述帧间预测模块包括边界填充单元，用于采用编码码流的序列头或图像头中记录的边界填充方式，以求得参考样本的样本值；
    图像块重建模块，用于将预测图像块和重建的残差块相加，得到重建的图像块。
  12. 如权利要求11所述的装置，其特征在于，边界填充单元用于当所述边界填充方式记录在编码码流的序列头中时，编码码流的所有序列的图像采用所记录的边界填充方式进行图像边界填充；当所述边界填充方式记录在编码码流的图像头中时，与该图像头对应的图像采用所记录的边界填充方式进行图像边界填充。
PCT/CN2016/071383 2016-01-19 2016-01-19 基于多方式边界填充的全景视频编码、解码方法和装置 Ceased WO2017124305A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2016/071383 WO2017124305A1 (zh) 2016-01-19 2016-01-19 基于多方式边界填充的全景视频编码、解码方法和装置
US15/747,864 US10341682B2 (en) 2016-01-19 2016-01-19 Methods and devices for panoramic video coding and decoding based on multi-mode boundary fill

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/071383 WO2017124305A1 (zh) 2016-01-19 2016-01-19 基于多方式边界填充的全景视频编码、解码方法和装置

Publications (1)

Publication Number Publication Date
WO2017124305A1 true WO2017124305A1 (zh) 2017-07-27

Family

ID=59361112

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/071383 Ceased WO2017124305A1 (zh) 2016-01-19 2016-01-19 基于多方式边界填充的全景视频编码、解码方法和装置

Country Status (2)

Country Link
US (1) US10341682B2 (zh)
WO (1) WO2017124305A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114598880A (zh) * 2022-05-07 2022-06-07 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110581995A (zh) * 2019-09-12 2019-12-17 北京奇艺世纪科技有限公司 一种视频处理的方法及装置
CN113034348B (zh) * 2021-03-24 2025-01-10 北京字节跳动网络技术有限公司 图像处理方法、装置、存储介质及设备
CN114005052B (zh) * 2021-10-22 2025-05-06 影石创新科技股份有限公司 全景图像的目标检测方法、装置、计算机设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101002479A (zh) * 2004-08-13 2007-07-18 庆熙大学校产学协力团 用于全景图像的运动估计和补偿的方法和设备
CN101563927A (zh) * 2006-12-21 2009-10-21 汤姆森许可贸易公司 用于对视频图像的块进行解码的方法
EP2346254A1 (en) * 2009-11-26 2011-07-20 Research In Motion Limited Video decoder and method for motion compensation for out-of-boundary pixels
CN105554506A (zh) * 2016-01-19 2016-05-04 北京大学深圳研究生院 基于多方式边界填充的全景视频编码、解码方法和装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7627194B2 (en) * 2004-08-13 2009-12-01 Samsung Electronics Co., Ltd. Method and device for making virtual image region for motion estimation and compensation of panorama image
US7623682B2 (en) * 2004-08-13 2009-11-24 Samsung Electronics Co., Ltd. Method and device for motion estimation and compensation for panorama image
KR101396365B1 (ko) * 2007-08-28 2014-05-30 삼성전자주식회사 영상의 시공간적 움직임 추정/보상 방법 및 장치
US20110122950A1 (en) * 2009-11-26 2011-05-26 Ji Tianying Video decoder and method for motion compensation for out-of-boundary pixels
EP2606646A1 (en) * 2010-10-01 2013-06-26 General Instrument Corporation Coding and decoding utilizing picture boundary padding in flexible partitioning
EP2533537A1 (en) * 2011-06-10 2012-12-12 Panasonic Corporation Transmission of picture size for image or video coding
US9083983B2 (en) * 2011-10-04 2015-07-14 Qualcomm Incorporated Motion vector predictor candidate clipping removal for video coding
US20150071357A1 (en) * 2013-09-12 2015-03-12 Qualcomm Incorporated Partial intra block copying for video coding
CN108293136B (zh) * 2015-09-23 2022-12-30 诺基亚技术有限公司 编码360度全景视频的方法、装置和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101002479A (zh) * 2004-08-13 2007-07-18 庆熙大学校产学协力团 用于全景图像的运动估计和补偿的方法和设备
CN101563927A (zh) * 2006-12-21 2009-10-21 汤姆森许可贸易公司 用于对视频图像的块进行解码的方法
EP2346254A1 (en) * 2009-11-26 2011-07-20 Research In Motion Limited Video decoder and method for motion compensation for out-of-boundary pixels
CN105554506A (zh) * 2016-01-19 2016-05-04 北京大学深圳研究生院 基于多方式边界填充的全景视频编码、解码方法和装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114598880A (zh) * 2022-05-07 2022-06-07 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质

Also Published As

Publication number Publication date
US20190007702A1 (en) 2019-01-03
US10341682B2 (en) 2019-07-02

Similar Documents

Publication Publication Date Title
CN105554506B (zh) 基于多方式边界填充的全景视频编码、解码方法和装置
AU2020397870B2 (en) Method and apparatus for point cloud coding
JP7574247B2 (ja) マルチラインフレーム内予測のための方法、装置、及びコンピュータプログラム
JP7062788B2 (ja) ビデオを復号する方法、装置およびコンピュータプログラム
JP6866299B2 (ja) 全方向視差ライトフィールド表示システム用の方法および装置
US11184641B2 (en) Coding spherical video data
KR20220128388A (ko) V-pcc용 스케일링 파라미터
TW201946448A (zh) 用於三百六十度視訊寫碼之環路濾波器填補
JP2025504039A (ja) メッシュ頂点変位コーディング
TW201911863A (zh) 用於360度視訊寫碼之參考圖推導及動作補償
TW201838407A (zh) 適應性擾動立方體之地圖投影
JP7504298B2 (ja) ３次元（３ｄ）メッシュのｕｖ座標を処理するための方法、装置及びコンピュータプログラム
JP6232076B2 (ja) 映像符号化方法、映像復号方法、映像符号化装置、映像復号装置、映像符号化プログラム及び映像復号プログラム
US10425656B2 (en) Method of inter-frame prediction for video encoding and decoding
JP2025530346A (ja) メッシュ圧縮のための変位コーディング
JP7758424B2 (ja) ビデオコーディングのための方法および装置
CN105681805A (zh) 视频编码、解码方法及其帧间预测方法和装置
CA3138068A1 (en) Method and apparatus for point cloud coding
JP7797675B2 (ja) 動的メッシュ圧縮における動きフィールドのコーディング
JP2023552114A (ja) メッシュ展開のための方法、装置及びプログラム
BR102019000922A2 (pt) Método para comprimir dados de campo de luz usando transformadas de quatro dimensões de tamanho de bloco variável e decomposição por planos de bits
JP2024515984A (ja) メッシュ圧縮のための境界ジオメトリ情報の予測コーディング
US10341682B2 (en) Methods and devices for panoramic video coding and decoding based on multi-mode boundary fill
CN118661420B (zh) 基于分辨率的解码方法、编码方法、解码器以及编码器
CN118541733A (zh) 位置压缩中的网格顶点分组

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16885578

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16885578

Country of ref document: EP

Kind code of ref document: A1