WO2019039920A1 - Procédé et dispositif de codage/décodage d'image de réalité virtuelle - Google Patents

Procédé et dispositif de codage/décodage d'image de réalité virtuelle Download PDF

Info

Publication number
WO2019039920A1
WO2019039920A1 PCT/KR2018/009835 KR2018009835W WO2019039920A1 WO 2019039920 A1 WO2019039920 A1 WO 2019039920A1 KR 2018009835 W KR2018009835 W KR 2018009835W WO 2019039920 A1 WO2019039920 A1 WO 2019039920A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
padding
information
prediction
Prior art date
Application number
PCT/KR2018/009835
Other languages
English (en)
Korean (ko)
Inventor
임화섭
임정윤
전대석
김재곤
박도현
윤용욱
Original Assignee
가온미디어 주식회사
한국항공대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020180062382A external-priority patent/KR20190022300A/ko
Application filed by 가온미디어 주식회사, 한국항공대학교산학협력단 filed Critical 가온미디어 주식회사
Publication of WO2019039920A1 publication Critical patent/WO2019039920A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/563—Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • the present invention relates to a method and apparatus for encoding / decoding an image. More particularly, the present invention relates to a method and apparatus for encoding / decoding a virtual reality image.
  • VR virtual reality
  • a recent VR system such as an HMD (Head Mounted Display) not only can provide a three-dimensional stereoscopic image in both sides of a user, but also can track the viewpoint in all directions. Therefore, (VR) image contents.
  • HMD Head Mounted Display
  • the 360 VR contents are composed of the multi-view image information of the simultaneous omnidirectionally synchronized spatio-temporal and binocular images
  • two large images synchronized with respect to the binocular space at all viewpoints in the production and transmission of the image And compresses and transmits them.
  • This increases the complexity and the bandwidth burden.
  • an area that is not actually viewed beyond a user's point of view is decoded, thus wasting an unnecessary process.
  • the present invention provides a method and apparatus for efficiently encoding / decoding a virtual reality image such as a 360-degree camera or a VR image using spatial structure information of a virtual reality image It has its purpose.
  • an image encoding method comprising: acquiring image information of a virtual reality image to be processed; Identifying an active area and a non-active area corresponding to a conversion format of the image information; Performing padding processing on the inactive area using pixel information of the active area; And performing coding or decoding processing corresponding to the padded image information.
  • an image processing apparatus including: an acquisition unit configured to acquire image information of a virtual reality image to be processed, An inactive area format determination unit for identifying the inactive area format; An inactive area padding unit for performing padding processing on the inactive area using pixel information of the active area; And an image processing apparatus that performs encoding or decoding processing corresponding to the padded image information.
  • the method may be embodied as a computer-readable recording medium having recorded thereon a program for execution on a computer.
  • padding processing and blending processing optimized for encoding and transmission are provided from a virtual reality image, and the amount, bandwidth, and complexity of transmission data of the image can be efficiently reduced.
  • FIGS. 1 and 2 are block diagrams illustrating an overall system in accordance with an embodiment of the present invention.
  • 3 to 4 are diagrams for explaining a signaling method of spatial structure information according to various embodiments of the present invention.
  • 5 to 11 are views illustrating a process of encoding a 360-degree virtual reality image according to an embodiment of the present invention.
  • FIGS. 12 and 13 are views for explaining an encoding apparatus and a decoding apparatus according to an embodiment of the present invention.
  • a combination of these in the expression of a mark-up form means a combination or combination of at least one selected from the group consisting of the constituents described in the expression of the mark-up form, Quot; is meant to include one or more selected from the group consisting of
  • Encoding may be performed using an HEVC (High Efficiency Video Coding) or a coding technique in progress of the current standardization, but the present invention is not limited thereto.
  • HEVC High Efficiency Video Coding
  • the encoding apparatus includes an encoding process and a decoding process, and the decoding apparatus has a decoding process.
  • the decoding process of the decoding apparatus is the same as the decoding process of the encoding apparatus. Therefore, the encoding apparatus will be mainly described below.
  • FIG 1 shows an overall system architecture according to an embodiment of the present invention.
  • an overall system includes a preprocessing apparatus 10, an encoding apparatus 100, a decrypting apparatus 200, and a post-processing apparatus 20.
  • the system according to an embodiment of the present invention can process virtual reality image information according to an embodiment of the present invention.
  • a virtual reality image is an image that provides an experience that a user is actually in, and can be an image that can be expressed in all directions synchronized with the user's time, and can also be called 360 video or virtual reality video.
  • the system includes a preprocessing apparatus 10 for preprocessing a plurality of viewpoint images through an operation such as merging or stitching to obtain a synchronized video frame, And outputs the bit stream; a decoding unit (200) for decoding the synchronized video frame by receiving the bit stream; and a decoding unit And a post-processing device 20 for outputting the output of the post-processing device 20 to the display device.
  • a preprocessing apparatus 10 for preprocessing a plurality of viewpoint images through an operation such as merging or stitching to obtain a synchronized video frame, And outputs the bit stream
  • a decoding unit (200) for decoding the synchronized video frame by receiving the bit stream
  • a decoding unit And a post-processing device 20 for outputting the output of the post-processing device 20 to the display device.
  • the input image may include a separate individual image, for example, sub-image information at various time points, in which one or more cameras are photographed in time and space synchronized state. Accordingly, the preprocessing apparatus 10 can acquire synchronized virtual reality image information by spatially merging or stitching the acquired multi-view sub image information with time.
  • the encoding apparatus 100 scans and predictively encodes the synchronized virtual reality image information to generate a bitstream, and the generated bitstream can be transmitted to the decoding apparatus 200.
  • the encoding apparatus 100 according to the embodiment of the present invention can extract spatial structure information from the synchronized image information, and can signal the signal to the decoding apparatus 200.
  • the spatial layout information may include basic information about the attributes and the layout of each sub-image, as one or more sub-images are merged into one video frame from the preprocessing apparatus 10 . Further, it may further include additional information on the relationship between each of the sub images and the sub images, which will be described later.
  • the spatial structure information according to the embodiment of the present invention can be transmitted to the decoding apparatus 200.
  • the decoding apparatus 200 can determine the decoding target and decoding order of the virtual reality image bitstream by referring to the spatial structure information and the user viewpoint information, which can induce efficient decoding.
  • the decoded video frame is separated into sub-images for each display through the post-processing device 20 and provided to a plurality of synchronized display systems such as an HMD, Images can be provided.
  • FIG. 2 is a block diagram illustrating a configuration of a virtual reality image encoding apparatus according to an embodiment of the present invention.
  • an encoding apparatus 100 includes a virtual reality image acquisition unit 110, a spatial structure information generation unit 120, a spatial structure information signaling unit 130, And a transmission processing unit 150.
  • the virtual reality image acquisition unit 110 acquires a virtual reality image using a virtual reality image acquisition unit such as a 360 degree camera.
  • the virtual reality image may include a plurality of time-and-space-synchronized subimages and may be received from the preprocessing apparatus 10 or may be received from a separate external input apparatus.
  • the spatial structure information generation unit 120 divides the virtual reality image into video frames in units of time, and extracts spatial structure information on the video frames.
  • the spatial structure information may be determined according to the attributes and arrangement state of the respective sub images, and may be determined according to information obtained from the preprocessing apparatus 10.
  • the spatial structure information signaling unit 130 performs information processing for signaling the spatial structure information to the decoding apparatus 200.
  • the spatial structure information signaling unit 130 may perform one or more processes to be included in the image data encoded by the image encoding unit, to construct a separate data format, or to include the metadata in the metadata of the encoded image .
  • the image encoding unit encodes the virtual reality image according to the time flow.
  • the image encoding unit can use the spatial structure information generated by the spatial structure information generation unit 120 as reference information to determine an image scanning order, a reference image, and the like.
  • the image encoding unit can perform encoding using HEVC (High Efficiency Video Coding) as described above, but can be improved in a more efficient manner with respect to the virtual reality image according to the spatial structure information.
  • HEVC High Efficiency Video Coding
  • 3 to 4 are diagrams for explaining a signaling method of spatial structure information according to various embodiments of the present invention.
  • the ERP projects and transforms a 360-degree image onto one face.
  • the ERP converts the position of the u, v coordinate system corresponding to the sampling position of the two-dimensional image and the hardness on the sphere corresponding to the u, v coordinate system position And latitude coordinate conversion processing.
  • the spatial structure information may include an ERP index and single plane information (for example, the face index is set to 0).
  • the CMP is a projection of a 360 degree image onto six regularly shaped faces and is a face index corresponding to PX, PY, PZ, NX, NY, and NZ (where P is positive and N is negative) f projected sub-images may be arranged.
  • an ERP image may be converted into a 3 x 2 cubemap image.
  • the spatial structure information may include a CMP index and each face index information corresponding to the subimage.
  • the post-processing apparatus 20 processes the two-dimensional position information on the sub-image according to the plane index, calculates the position information corresponding to the three-dimensional coordinate system, and outputs the three-dimensional 360-degree image inversely.
  • ACP like CMP, applies a function adjusted to three-dimensional bending deformation corresponding to two-dimensional projection transformation and three-dimensional inversion in projecting a 360-degree image onto six regular faces ,
  • the processing function is different, but the spatial structure information used may include ACP index and plane index information per sub-image. Accordingly, the post-processing apparatus 20 performs inverse transform processing of the two-dimensional position information on the sub image according to the adjusted index according to the surface index, calculates position information corresponding to the three-dimensional coordinate system, Can be output.
  • the OHP is used to project a 360 degree image onto six regular octagonal faces using six vertices, and the surfaces ⁇ F0, F1, F2, F3, F4, F5, F6, V0, V1, V2, V3, V3, V4, V5) may be arranged in the converted image.
  • the spatial structure information may include an OHP index, face index information corresponding to the subimage, and one or more vertex index information matching the face index information.
  • sub-image arrangement of the transformed image can be divided into a compact case and a non-compact case.
  • the spatial highlight information may further include the compact or nonidentical identification information.
  • the case of not compact, and the case of compactness, the face index and the vertex index matching information and the inverse transformation process can be determined differently.
  • face indexes 4 may be matched with vertex indices V0, V5, and V1 if they are not compact, and other matches may be processed with V1, V0, and V5 when they are compact.
  • the post-processing apparatus 20 performs inverse transform processing on the two-dimensional position information on the sub-image according to the plane index and the vertex index, calculates vector information corresponding to the three-dimensional coordinate system, and outputs the inverse- have.
  • the ISP projects a 360 degree image using 20 faces and 12 vertices, and the sub images according to each transformation can be arranged in the transformed image.
  • the spatial structure information may include at least one of an ISP index, a face index, a vertex index, and compact identification information, similar to the OHP.
  • the SSP divides the sphere of a 360 degree image into three segments: the North Pole, the Equator, and the Antarctic.
  • the North and South poles are mapped to two circles identified by indexes, And the equatorial projection method identical to ERP can be used.
  • the spatial structure information may include an SSP index and a face index corresponding to each equator, north pole and south pole segment.
  • the RSP may include a method of dividing a sphere of a 360-degree image into two equal-sized segments, and arranging the segmented images in two rows by unfolding the segmented images. And, the RSP can implement this arrangement using 6 sides as a 3X2 aspect ratio similar to CMP. Accordingly, the transformed image may include a first segment image of the upper segment and a second segment image of the lower segment.
  • the spatial structure information may include at least one of an RSP index, a segment video index, and a face index.
  • the TSP may include a method of deforming and projecting a frame of a 360-degree image projected on six cube surfaces in correspondence with a surface of a truncated rectangular pyramid. Accordingly, the sizes and shapes of the sub images corresponding to the respective surfaces may be different from each other.
  • the spatial structure information may include at least one of the TSP identification information and the face index.
  • the spatial structure information includes a viewport generation using rectilinear projection index information and a visual port
  • the spatial structure information may further include interpolation filter information to be applied in the image transformation.
  • the interpolation filter information may differ depending on each projection transformation method, and may include at least one of a nearest neighbor, a bilinear filter, a bicubic filter, and a Lanczos filter.
  • the conversion method and the index for evaluating the processing performance of the preprocessing conversion and post-processing inverse conversion can be separately defined.
  • the performance evaluation can be used to determine the preprocessing method in the preprocessor 10.
  • CPPR Crasters Parallel Projection
  • the table shown in FIG. 4 is arbitrarily arranged according to the input image, and can be changed according to the coding efficiency, the content distribution of the market, and the like.
  • the decoding apparatus 200 can parse the separately signaled table index and use it for decoding processing.
  • each of the layout information can be usefully used for partial decoding of an image. That is, sub-image layout information such as CUBIC LAYOUT can be used to distinguish between independent sub-images and dependent sub-images, thereby determining an efficient encoding and decoding scanning sequence or performing partial decoding for a specific time.
  • sub-image layout information such as CUBIC LAYOUT can be used to distinguish between independent sub-images and dependent sub-images, thereby determining an efficient encoding and decoding scanning sequence or performing partial decoding for a specific time.
  • 5 to 7 are views illustrating a process of encoding a 360-degree virtual reality image according to an embodiment of the present invention.
  • the preprocessing apparatus 10 or the encoding apparatus 100 can perform at least one of 360 image format conversion processing and image prediction processing.
  • Some formats of the 360 image are divided into an active area and an inactive area.
  • the inactive area is filled with the intermediate value of the expressible bit irrespective of the original image. This leads to a reduction in the coding efficiency due to prediction with intra-picture prediction or inter-picture prediction with respect to the active area, which is less correlated with the surrounding image. Therefore, a technique for constructing inactive regions efficiently is required.
  • the encoding apparatus 100 since the encoding apparatus 100 performs encoding with pixels in an inactive region having a low correlation with the active region, inefficient encoding is performed.
  • Method 2 Padding using the interpolation pixel of the active area
  • Method 3 Padding using the average pixel of the active area
  • Method 4 Padding for a reference picture stored in a DPB (Decoded Picture Buffer) in an image prediction step using any one of the methods 1, 2,
  • the coding efficiency can be improved through more accurate prediction in the prediction step by filling the inactive area with the correlation with the surrounding image. For example, when the pixels corresponding to the boundaries of the image projected in the rectangle are averaged to fill in the inactive region, the bit representing the additional image information is minimized, but the correlation between the inactive region and the active region becomes large and efficient prediction becomes possible.
  • the present invention can be padded with an appropriate pixel having a correlation with a surrounding image by applying one or more other padding methods in addition to the methods 1 to 4, and the present invention is not limited to the methods 1 to 4 described above.
  • FIG. 6 is a block diagram illustrating a configuration of an image format conversion unit for processing a process of encoding and decoding a virtual reality image according to a first embodiment of the present invention.
  • the padding process according to the embodiment of the present invention may be performed by the image format conversion process of the image format conversion unit 11 as shown in FIG.
  • the image format conversion unit 11 includes a conversion format selection unit 11a, an inactive area format determination unit 11b, and an inactive area padding unit 11c.
  • the conversion format selection unit 11a selects a conversion format corresponding to the video information of the virtual reality image.
  • the inactive area format determination unit 11b identifies the inactive area presence format for the selected format.
  • the inactive area padding unit 11c performs a padding process on the inactive area with at least one value selected from the active area boundary pixel, the active area interpolation pixel, and the active area average pixel.
  • the image format converting unit 11 may be included in the preprocessing unit 10 and the padding processed image may be encoded or decoded in the encoding apparatus 100 or the decoding apparatus 200.
  • the conversion format selection unit 11a of the image format conversion unit 11 converts the 360 image format and the 360-degree image into the image data of the 360-image format, such as ERP, CMP, ACP, EAP, OHP, ISP, TSP, SSP, RSP, and the like can be selected as the conversion format.
  • the selected conversion format can be classified into a format in which the inactive area exists (SSP, RSP, etc.) and a format in which the inactive area does not exist (ERP, CMP, etc.).
  • the image format conversion unit 11 converts the non-Using the correlation with the image, the inactive area can be padded with one or more various padding methods (active area boundary pixels, active area interpolation pixels, active area average pixels, other known padding processes, etc.).
  • the inactive area format determination unit 11b can use the pixels of the active area in padding the inactive area.
  • FIG. 5 illustrates a format (SSP, RSP) in which an inactive area exists, and a green line segment and a red line segment may denote a pixel of an active area in a 360 degree image.
  • the green line segment and the red line segment in FIG. 5 may be in contact with each other, and based on this, the inactive area padding unit 11c can perform the following padding process.
  • the inactive area padding unit 11c of the image format conversion unit 11 includes a first pixel 101 corresponding to the upper and lower active region boundaries in FIG. 5, a second pixel 102 ),
  • the non-active region 103 can be padded.
  • the padding may be performed by copying corresponding boundary pixels in multiple directions such as horizontal, vertical, and diagonal lines.
  • the non-active region 103 is divided into the inactive region 103 by the first pixel 101 corresponding to the upper and lower active region boundaries and the second pixel 102 corresponding to the circular active boundary by the inactive region padding portion 11c. So that the pixel can be padded with the generated pixel.
  • Various methods such as linear interpolation can be used as the interpolation method.
  • the non-active region padding portion 11c can obtain not only the pixels of one line of the active region boundary, but also partial pixels sufficient to fill a part of the boundary region or the non-active region, and padding can be performed using the partial pixels.
  • the inactive area 103 may be entirely padded using a portion of the active area (e.g., the area of the dotted line area including the first pixel 101 of the boundary area).
  • the inactive area 103 can be partially padded using a part of the active area (a dotted line area including the first pixel 101 of the boundary area).
  • the padded image information can be encoded or decoded in the encoding apparatus 100 or the decoding apparatus 200.
  • the inactive area presence format for the selected format can be identified and the information on the inactive area existence format can be signaled.
  • FIG. 7 illustrates a process of encoding and decoding a virtual reality image according to a second embodiment of the present invention.
  • the padding-based image predicting unit 21 may perform padding-based inter picture prediction decoding processing according to the image format have.
  • the padding-based image predicting unit 21 determines a prediction performing method according to the converted format information signaled from the image format converting unit 11 and forms a reference picture corresponding thereto, By performing the processing, padded reference picture-based prediction can be performed.
  • the padding-based image predicting unit 21 includes a prediction performing determiner 21a, a reference picture constructing unit 21b, an inactive area padding unit 21c, and a padding-based prediction performing unit 21b.
  • the predictive performance determination unit 21a determines whether the intra-picture prediction or the inter-picture prediction is performed when the signaled inactive area presence format signaled from the video format conversion unit 11 is identified, thereby determining a prediction scheme to be performed.
  • the reference picture composing unit 21b forms a reference picture of DPB (decoded picture buffer) for inter-picture prediction, , The active area interpolation pixel, and the active area average pixel to perform the padding process on the inactive area, thereby configuring the DPB using the padded reference picture.
  • DPB decoded picture buffer
  • the padding-based prediction performing unit 21d can perform intra-picture or intra-picture prediction using padded reference pictures.
  • the padding-based inter-picture prediction performing unit 21d performs the padding-based inter picture prediction on the 360-degree picture converted into the format in which the inactive area exists in the picture coding / decoding of the coding apparatus 100 or the decoding apparatus 200
  • the inter picture prediction encoding and decoding process can be performed.
  • At least one of intra-picture prediction and inter-picture prediction may be used.
  • the reference picture constructing unit 21b in constructing a reference picture in a DPB for inter-picture prediction, in order to increase the prediction efficiency, the reference picture constructing unit 21b uses various methods Pixels, an active area interpolation pixel, and an active area average pixel), the reference picture in the DPB can be constructed by padding the inactive area.
  • pixels of the active region may be used for padding the inactive region, and the padding scheme may be exemplified as follows.
  • the inactive area padding unit 11c of the image format conversion unit 11 includes a first pixel 101 corresponding to the upper and lower active region boundaries in FIG. 5, a second pixel 102 ),
  • the non-active region 103 can be padded.
  • the padding may be performed by copying corresponding boundary pixels in multiple directions such as horizontal, vertical, and diagonal lines.
  • the non-active region 103 is divided into the inactive region 103 by the first pixel 101 corresponding to the upper and lower active region boundaries and the second pixel 102 corresponding to the circular active boundary by the inactive region padding portion 11c. So that the pixel can be padded with the generated pixel.
  • Various methods such as linear interpolation can be used as the interpolation method.
  • the non-active region padding portion 11c can obtain not only the pixels of one line of the active region boundary, but also partial pixels sufficient to fill a part of the boundary region or the non-active region, and padding can be performed using the partial pixels.
  • the inactive area 103 may be entirely padded using a portion of the active area (e.g., the area of the dotted line area including the first pixel 101 of the boundary area).
  • the inactive area 103 can be partially padded using a part of the active area (a dotted line area including the first pixel 101 of the boundary area).
  • the padded image information is composed of reference pictures and can be used in the encoding apparatus 100 or the decoding apparatus 200 for encoding or decoding processing.
  • FIG. 8 is a diagram for explaining a decoding method according to another embodiment of the present invention.
  • a decoding apparatus 200 decodes image information (S103), analyzes the type of a decoded image 105 (105) , It is determined whether the decoded image includes an inactive image (S107).
  • the decoding apparatus 200 applies the padding for the inactive area through the padding-based image predicting unit 21 to perform the following padding-based predictive decoding (S109).
  • the decoding apparatus 200 can perform normal inter-picture prediction decoding (S111).
  • the spatial structure information corresponding to the conversion format may be transmitted to the decoding apparatus 200 through Header syntax such as Slice / Picture / Video or SEI MSG have.
  • the decoding apparatus 200 can determine whether there is an inactive area according to the conversion format of the spatial structure information.
  • the decoding apparatus 200 can selectively perform padding with respect to the inactive area when performing the image decoding in the inter-picture prediction (MC step) when the inactive area exists.
  • 9 to 10 are diagrams for explaining encoding and decoding processing according to another embodiment of the present invention.
  • the image format conversion unit 11 for processing the padding process according to the embodiment of the present invention is connected to the inactive area padding unit 11c corresponding to the inactive area described above, And a blending processing unit 11d for performing a blending process between the area and the existing active area.
  • the inactive area format determination unit 11b selects The non-active area padding unit 11c identifies the inactive area presence format for the selected format, and if the inactive area format is identified, the inactive area padding unit 11c selects at least one of the active area boundary pixels, the active area interpolation pixels, And performs padding processing for the inactive area.
  • the padding process according to the embodiment of the present invention further includes not only the active area boundary pixel, the active area interpolation pixel, and the active area average pixel but also processes according to various other padding processes, It is possible to paddle an appropriate pixel having a correlation with the pixel.
  • the blending processing unit 11d can alleviate the discontinuous boundary generated by performing the blending process between the previously padded inactive area and the existing area.
  • the image format converting unit 11 may be included in the preprocessing unit 10 and the padding processed image may be encoded or decoded in the encoding apparatus 100 or the decoding apparatus 200.
  • the image format conversion unit 11 converts various image formats such as ERP, CMP, ACP, EAP, OHP, ISP, TSP, SSP, RSP Can be selected and used.
  • the selected conversion format can be classified into a format in which the inactive area exists (SSP, RSP, etc.) and a format in which the inactive area does not exist (ERP, CMP, etc.).
  • SSP, RSP in which an inactive area exists
  • the image format conversion unit 11 converts one or more various methods (active region boundary pixels, active region interpolation pixels , Active area average pixel, other padding process, etc.).
  • the image format conversion part 11 can use the pixels of the active area.
  • the padded inactive area and the existing boundary area are blended by the blending processing part 11d, .
  • the padding process is the same as that described with reference to FIG.
  • the blending processing unit 11d can alleviate the discontinuous boundary by blending the padded inactive area with the existing area.
  • the padded area by the inactive area padding can alleviate the discontinuous boundary through blending with the existing area, and the blending processing part can perform the blending processing for it.
  • the blending processing unit 11d may perform a variety of blending operations including at least one of x-axis blending, y-axis blending, and xy-axis blending, for example.
  • the blending processing unit 11d may perform a blending process using at least one of distance proportional, convolution, and low-pass filters as a variable.
  • the padding-based image predicting unit 21 may further include a blending processing unit 21e.
  • the padding-based image predicting unit 21 can construct a padded reference picture using at least one padding method selected according to the prediction, and further performs a blending process corresponding to the padded reference picture can do.
  • the difference between the padded inactive area and the existing boundary boundary area can be alleviated through the blending processing of the blending processing part 21e.
  • the padding process and the inter-picture prediction are performed in the same manner as described with reference to FIG.
  • the blending processing of the blending processing unit 21e can alleviate the discontinuous boundary by blending the padded inactive region with existing regions.
  • the area padded by the inactive area padding of the reference picture can alleviate the discontinuous boundary through blending with the existing area, and the blending processing unit can perform blending processing for the blind processing have.
  • the blending processing unit may perform a variety of blending operations including at least one of x-axis blending, y-axis blending, and xy-axis blending, for example.
  • the blending processing unit may perform a blending process using at least one of distance proportional, convolution, and low-pass filters as a variable.
  • the inactive area padding units 11c and 21c of FIGS. 9 and 10 may apply inactive area padding to the brightness / color difference signals, respectively, in the above-described padding process. Since the characteristics of the brightness signal and the color difference signal are different, the inactive regions can be padded by different methods.
  • the encoding apparatus 100 may perform padding of various methods in advance, and select the most suitable padding method for each brightness and color difference signal, and signal the signal to the decoding apparatus 200.
  • the non-active area padding units 11c and 21c perform padding only on a part of frames, rather than performing sub-decoding by performing padding on all frames of an image, and output the corresponding signaling information to the decoding apparatus 200. [ Or to the post-processing device 20.
  • the decoding device 200 or the post-processing device 20 uses the information of the padded inactive area so that the inactive area padding parts 11c, You can pad inactive areas of a picture.
  • the decoding apparatus 200 or the post-processing apparatus 20 pads the inactive area of the I-picture using the information transmitted from the B-Picture or P-picture through the inactive area padding units 11c and 21c It is possible.
  • the non-active area padding units 11c and 21c also use padded inactive area information for not only the above-described I-picture but also B and P- Can be performed.
  • the decoding apparatus 200 or the post-processing apparatus 20 may use the padding information of the first I-picture through the inactive area padding units 11c and 21c to display the picture of all pictures or some pictures The padding process can be performed.
  • the decoding apparatus 200 or the post-processing apparatus 20 performs the padding process in the I-picture through the inactive area padding units 11c and 21c,
  • a padding process may be performed on pictures or some pictures other than all I-pictures in units of GOP by using the padding information in the P-picture.
  • the coding apparatus 100 or the preprocessing apparatus 10 signals pixel information padded in the I-picture through the inactive area padding units 11c and 21c, and outputs it to the decoding apparatus 200 or the post-processing apparatus 20, it is possible to paddle the inactive area through the inactive area padding parts 11c and 21c without using any additional calculation process using this information, thereby being efficient.
  • 11 is a diagram showing padding and blending results in an RSP format in which an inactive area exists.
  • FIG. 12 is a block diagram illustrating a configuration of a moving picture encoding apparatus according to an exemplary embodiment of the present invention. Referring to FIG. 12, each sub-image or entire frame of a virtual reality image according to an embodiment of the present invention is input as an input video signal .
  • a moving picture encoding apparatus 100 includes a picture dividing unit 160, a transform unit, a quantization unit, a scanning unit, an entropy coding unit, an intra prediction unit 169, an inter prediction unit 170, An inverse quantization unit, an inverse transform unit, a post-processing unit 171, a picture storage unit 172, a subtraction unit, and an adder 168.
  • the picture division unit 160 analyzes a video signal to determine a prediction mode by dividing a picture into a coding unit of a predetermined size for each largest coding unit (LCU: Largest Coding Unit), and calculates a prediction unit size .
  • LCU Largest Coding Unit
  • the picture division unit 160 sends the prediction unit to be encoded to the intra prediction unit 169 or the inter prediction unit 170 according to a prediction mode (or a prediction method). Further, the picture division unit 160 sends the prediction unit to be encoded to the subtraction unit.
  • the picture may be composed of a plurality of slices, and the slice may be composed of a plurality of maximum coding units (LCU).
  • LCU maximum coding units
  • the LCU can be divided into a plurality of coding units (CUs), and the encoder can add information indicating whether or not to be divided to a bit stream.
  • the decoder can recognize the position of the LCU by using the address (LcuAddr).
  • the coding unit CU in the case where division is not allowed is regarded as a prediction unit (PU), and the decoder can recognize the position of the PU using the PU index.
  • PU prediction unit
  • the prediction unit PU may be divided into a plurality of partitions. Also, the prediction unit PU may be composed of a plurality of conversion units (TUs).
  • TUs conversion units
  • the picture dividing unit 160 may send the image data to the subtracting unit in a block unit (for example, PU unit or TU unit) of a predetermined size according to the determined coding mode.
  • a block unit for example, PU unit or TU unit
  • CTB Coding Tree Block
  • CU coding unit
  • the coding unit may have a form of a quad tree according to the division.
  • the coding unit may have a binary tree shape divided into binary segments in the quad tree or terminal node, and the maximum size according to the encoder standard is 256X256 to 64X64 Lt; / RTI >
  • the coding apparatus 10 easily divides the edge region of a coding unit divided into a specific direction length by a quadtree and a binary tree division
  • the coding unit may be divided into a ternary tree or a triple tree structure which can be divided.
  • a structure of a multi-type tree supporting multiple types of tree structure division can be considered.
  • the multi-type tree or the triple tree structure can be used more effectively in the padding and blending processing corresponding to the virtual reality image according to the embodiment of the present invention, and the division of the multi- Lt; / RTI > coding unit.
  • the multi-type tree or the triple tree structure may require a triple division in various ways for the coding tree unit, but it is preferable that only a predetermined optimized form is allowed in consideration of the coding bandwidth and the decoding complexity and the transmission bandwidth by signaling .
  • the picture dividing unit 160 can determine whether to divide the current coding unit into a specific type of triad tree structure only when the current coding unit is against a preset condition. Also, according to the permission of such a triad tree, the division ratio of the binary tree can be expanded and varied not only at 1: 1 but also at 3: 1, 1: 3, and so on. Therefore, the division structure of the coding unit according to the embodiment of the present invention may include a composite tree structure that is subdivided into a quadtree, a binary tree, or a triad tree according to a ratio.
  • the picture partitioning unit 160 processes the quadtree partitioning, corresponding to the maximum size of the block (e.g., pixel-based 128 x 128, 256 x 256, etc.)
  • a hybrid tree processing for processing at least one of a double tree structure and a triple tree structure division corresponding to a terminal node can be performed.
  • the picture division unit 110 divides a first binary division (BINARY 1), a second binary division (BINARY 2) corresponding to the property and size of the current block, ) And a division structure of any one of the first ternary splitting (TRI 1) or the second ternary splitting (TRI 2) which is the triple tree splitting.
  • the first ternary division may correspond to a vertical or horizontal division having a ratio of N: 2N: N
  • the second ternary division may correspond to a vertical or horizontal division having a ratio of N: 6N: N
  • the ternary root CU can be divided into CU0, CU1 and CU2 of each size specified in the partition table.
  • the picture dividing unit 160 sets the depth to 0 when the depth is 3 when the maximum coding unit is a LCU (Largest Coding Unit) And performs encoding by searching for an optimal prediction unit recursively up to the unit (CU).
  • a coding unit of a terminal node divided into QTBT for example, a PU (Prediction Unit) and a TU (Transform Unit) may have the same form as the divided coding unit or may have a further divided form.
  • a prediction unit for performing prediction is defined as a PU (Prediction Unit).
  • Each coding unit (CU) is predicted by a unit divided into a plurality of blocks, and is divided into a square and a rectangle to perform prediction.
  • the transform unit transforms the original block of the input prediction unit and the residual block, which is the residual signal of the intra prediction unit 169 or the prediction block generated by the inter prediction unit 170.
  • the residual block is composed of a coding unit or a prediction unit.
  • a residual block composed of a coding unit or a prediction unit is divided into optimal transform units and transformed. Different transformation matrices may be determined depending on the prediction mode (intra or inter). Also, since the residual signal of the intra prediction has directionality according to the intra prediction mode, the transformation matrix can be adaptively determined according to the intra prediction mode.
  • the transformation unit can be transformed by two (horizontal, vertical) one-dimensional transformation matrices. For example, in the case of inter prediction, a predetermined conversion matrix is determined.
  • the intra prediction mode when the intra prediction mode is horizontal, the probability that the residual block has the direction in the vertical direction becomes high. Therefore, the DCT-based integer matrix is applied in the vertical direction, Or a KLT-based integer matrix.
  • the intra prediction mode is vertical, a DST-based or KLT-based integer matrix is applied in the vertical direction and a DCT-based integer matrix is applied in the horizontal direction.
  • DCT-based integer matrix is applied in both directions. Further, in the case of intra prediction, the transformation matrix may be adaptively determined depending on the size of the conversion unit.
  • the predetermined size may be 8x8 or 16x16. And quantizes the coefficients of the transform block using a quantization matrix determined according to the determined quantization step size and the prediction mode.
  • the quantization unit uses the quantization step size of the quantization unit adjacent to the current quantization unit as a quantization step size predictor of the current quantization unit.
  • the quantization unit may search the left quantization unit, the upper quantization unit, and the upper left quantization unit of the quantization unit in order, and generate a quantization step size predictor of the current quantization unit using one or two effective quantization step sizes.
  • the effective first quantization step size searched in the above order can be determined as a quantization step size predictor.
  • the average value of the two effective quantization step sizes searched in the above order may be determined as a quantization step size predictor, or when only one is effective, it may be determined as a quantization step size predictor.
  • a difference value between the quantization step size of the current encoding unit and the quantization step size predictor is transmitted to the entropy encoding unit.
  • the quantization step sizes of the quantization units adjacent to the current coding unit and the quantization unit immediately before the coding order in the maximum coding unit can be candidates.
  • the order may be changed, and the upper left side quantization unit may be omitted.
  • the quantized transform block is provided as an inverse quantization unit and a scanning unit.
  • the scanning unit scans the coefficients of the quantized transform block and converts them into one-dimensional quantization coefficients. Since the coefficient distribution of the transform block after quantization may be dependent on the intra prediction mode, the scanning scheme is determined according to the intra prediction mode.
  • the coefficient scanning method may be determined depending on the size of the conversion unit.
  • the scan pattern may vary according to the directional intra prediction mode.
  • the scan order of the quantization coefficients is scanned in the reverse direction.
  • the same scan pattern is applied to the quantization coefficients in each subset.
  • the scan pattern between subset applies zigzag scan or diagonal scan.
  • the scan pattern is preferably scanned to the remaining subsets in the forward direction from the main subset containing the DC, but vice versa.
  • a scan pattern between subsets can be set in the same manner as a scan pattern of quantized coefficients in a subset.
  • the scan pattern between the sub-sets is determined according to the intra-prediction mode.
  • the encoder transmits to the decoder information indicating the position of the last non-zero quantization coefficient in the transform unit.
  • the inverse quantization unit 135 dequantizes the quantized quantized coefficients.
  • the inverse transform unit restores the inversely quantized transform coefficients into residual blocks in the spatial domain.
  • the adder combines the residual block reconstructed by the inverse transform unit with the intra prediction unit 169 or the received prediction block from the inter prediction unit 170 to generate a reconstruction block.
  • the post-processing unit 171 performs a deblocking filtering process for eliminating the blocking effect generated in the reconstructed picture, an adaptive offset application process for compensating the difference value from the original image on a pixel-by-pixel basis, and a coding unit And performs an adaptive loop filtering process to compensate the value.
  • the deblocking filtering process is preferably applied to the boundary of a prediction unit and a conversion unit having a size larger than a predetermined size.
  • the size may be 8x8.
  • the deblocking filtering process may include determining a boundary to be filtered, determining a bounary filtering strength to be applied to the boundary, determining whether to apply a deblocking filter, And selecting a filter to be applied to the boundary if it is determined to apply the boundary.
  • Whether or not the deblocking filter is applied is determined based on i) whether the boundary filtering strength is greater than 0 and ii) whether a pixel value at a boundary between two blocks adjacent to the boundary to be filtered (P block, Q block) Is smaller than a first reference value determined by the quantization parameter.
  • the filter is preferably at least two or more. If the absolute value of the difference between two pixels located at the block boundary is greater than or equal to the second reference value, a filter that performs relatively weak filtering is selected.
  • the second reference value is determined by the quantization parameter and the boundary filtering strength.
  • the adaptive offset application process is to reduce a distortion between a pixel in the image to which the deblocking filter is applied and the original pixel. It may be determined whether to perform the adaptive offset applying process in units of pictures or slices.
  • the edge type to which each pixel belongs is determined and the corresponding offset is applied.
  • the edge type is determined based on the distribution of two pixel values adjacent to the current pixel.
  • the adaptive loop filtering process can perform filtering based on a value obtained by comparing a reconstructed image and an original image through a deblocking filtering process or an adaptive offset applying process.
  • the adaptive loop filtering can be applied to the entire pixels included in the 4x4 block or the 8x8 block.
  • Whether or not the adaptive loop filter is applied can be determined for each coding unit.
  • the size and the coefficient of the loop filter to be applied may vary depending on each coding unit.
  • Information indicating whether or not the adaptive loop filter is applied to each coding unit may be included in each slice header.
  • the shape of the loop filter may have a rectangular shape unlike the luminance.
  • Adaptive loop filtering can be applied on a slice-by-slice basis. Therefore, information indicating whether or not adaptive loop filtering is applied to the current slice is included in the slice header or the picture header.
  • the slice header or picture header additionally includes information indicating the horizontal and / or vertical direction filter length of the luminance component used in the adaptive loop filtering process.
  • the slice header or picture header may include information indicating the number of filter sets. At this time, if the number of filter sets is two or more, the filter coefficients can be encoded using the prediction method. Accordingly, the slice header or the picture header may include information indicating whether or not the filter coefficients are encoded in the prediction method, and may include predicted filter coefficients when the prediction method is used.
  • the slice header or the picture header may include information indicating whether or not each of the color difference components is filtered.
  • information indicating whether or not to filter Cr and Cb can be joint-coded (i.e., multiplexed coding).
  • the picture storage unit 172 receives the post-processed image data from the post-processing unit 171, and restores and restores the pictures in units of pictures.
  • the picture may be a frame-based image or a field-based image.
  • the picture storage unit 172 has a buffer (not shown) capable of storing a plurality of pictures.
  • the inter-prediction unit 170 performs motion estimation using at least one reference picture stored in the picture storage unit 172, and determines a reference picture index and a motion vector indicating a reference picture.
  • a prediction block corresponding to the prediction unit to be coded is extracted from the reference pictures used for motion estimation among a plurality of reference pictures stored in the picture storage unit 172 and output .
  • the intra prediction unit 169 performs intraprediction encoding using the reconstructed pixel values in the picture including the current prediction unit.
  • the intra prediction unit 169 receives the current prediction unit to be predictively encoded and selects one of a predetermined number of intra prediction modes according to the size of the current block to perform intra prediction.
  • the intraprediction unit 169 adaptively filters the reference pixels to generate intra prediction blocks. If reference pixels are not available, reference pixels may be generated using available reference pixels.
  • the entropy encoding unit entropy-codes quantization coefficients quantized by the quantization unit, intra prediction information received from the intra prediction unit 169, motion information received from the inter prediction unit 170, and the like.
  • the inter prediction coding apparatus may include a motion information determination unit, a motion information coding mode determination unit, a motion information coding unit, a prediction block generation unit, a residual block generation unit, a residual block coding unit, and a multiplexer .
  • the motion information determination unit determines the motion information of the current block.
  • the motion information includes a reference picture index and a motion vector.
  • the reference picture index indicates any one of the previously coded and reconstructed pictures.
  • the current block when the current block is bi-directionally predictive-coded, it may include an index indicating one or two pictures among the reference pictures of the composite list LC generated by combining the list 0 and the list 1.
  • the motion vector indicates the position of the prediction block in the picture indicated by each reference picture index.
  • the motion vector may be a pixel unit (integer unit) or a sub-pixel unit.
  • the prediction block is generated from the pixels of the integer unit.
  • the motion information encoding mode determination unit determines whether motion information of the current block is coded in a skip mode, a merge mode, or an AMVP mode.
  • the skip mode is applied when there is a skip candidate having the same motion information as the current block motion information, and the residual signal is zero.
  • the skip mode is also applied when the current block is the same size as the coding unit.
  • the current block can be viewed as a prediction unit.
  • the merge mode is applied when there is a merge candidate having the same motion information as the current block motion information.
  • the merge mode is applied when there is a residual signal when the current block is different in size from the coding unit or the size is the same.
  • the merge candidate and the skip candidate can be the same.
  • AMVP mode is applied when skip mode and merge mode are not applied.
  • the AMVP candidate having the motion vector most similar to the motion vector of the current block is selected as the AMVP predictor.
  • the motion information encoding unit encodes motion information according to a method determined by the motion information encoding mode determining unit.
  • the motion information encoding mode is a skip mode or a merge mode
  • a merge motion vector encoding process is performed.
  • the motion information encoding mode is AMVP
  • the AMVP encoding process is performed.
  • the prediction block generation unit generates a prediction block using the motion information of the current block. If the motion vector is an integer unit, the block corresponding to the position indicated by the motion vector in the picture indicated by the reference picture index is copied to generate a prediction block of the current block.
  • the pixels of the prediction block are generated from the pixels in the integer unit in the picture indicated by the reference picture index.
  • a prediction pixel in the case of a luminance pixel, can be generated using an 8-tap interpolation filter.
  • a 4-tap interpolation filter can be used to generate a predictive pixel.
  • the residual block generator uses residual blocks of the current block and the current block to generate residual blocks. If the current block size is 2Nx2N, a residual block is generated using a 2Nx2N prediction block corresponding to the current block and the current block.
  • the current block size used for prediction is 2NxN or Nx2N
  • a prediction block for each of the 2NxN blocks constituting 2Nx2N is obtained, and the 2Nx2N final prediction block using the 2NxN prediction blocks is calculated Can be generated.
  • the 2Nx2N residual block may be generated using the 2Nx2N prediction block. It is possible to overlap-smoothing the pixels of the boundary portion to solve the discontinuity of the boundary portion of 2NxN-sized two prediction blocks.
  • the residual block coding unit divides the generated residual block into one or more conversion units. Then, each conversion unit is transcoded, quantized, and entropy encoded. At this time, the size of the conversion unit may be determined according to the size of the residual block in a quadtree manner.
  • the residual block coding unit transforms the residual block generated by the inter prediction method using an integer-based transform matrix.
  • the transform matrix is an integer-based DCT matrix.
  • the residual block coding unit uses a quantization matrix to quantize the coefficients of the residual block transformed by the transform matrix.
  • the quantization matrix is determined by a quantization parameter.
  • the quantization parameter is determined for each coding unit equal to or larger than a predetermined size.
  • the predetermined size may be 8x8 or 16x16. Therefore, when the current coding unit is smaller than the predetermined size, only the quantization parameters of the first coding unit are encoded in the coding order among the plurality of coding units within the predetermined size, and the quantization parameters of the remaining coding units are the same as the parameters. You do not have to.
  • the coefficients of the transform block are quantized using a quantization matrix determined according to the determined quantization parameter and the prediction mode.
  • the quantization parameter determined for each coding unit equal to or larger than the predetermined size is predictively encoded using a quantization parameter of a coding unit adjacent to the current coding unit.
  • a quantization parameter predictor of the current coding unit can be generated by searching the left coding unit of the current coding unit, the upper coding unit order, and using one or two valid quantization parameters available.
  • a valid first quantization parameter retrieved in the above order may be determined as a quantization parameter predictor.
  • the first coding unit may be searched in order of the coding unit immediately before in the coding order, and the first validation parameter may be determined as a quantization parameter predictor.
  • the coefficients of the quantized transform block are scanned and converted into one-dimensional quantization coefficients.
  • the scanning scheme can be set differently according to the entropy encoding mode. For example, in the case of CABAC encoding, the inter prediction encoded quantized coefficients can be scanned in a predetermined manner (zigzag or raster scan in the diagonal direction). On the other hand, when encoded by CAVLC, it can be scanned in a different manner from the above method.
  • the scanning method may be determined according to the intra-prediction mode in the case of interlacing, or the intra-prediction mode in the case of intra.
  • the coefficient scanning method may be determined depending on the size of the conversion unit.
  • the scan pattern may vary according to the directional intra prediction mode.
  • the scan order of the quantization coefficients is scanned in the reverse direction.
  • the multiplexer multiplexes the motion information encoded by the motion information encoding unit and the residual signals encoded by the residual block encoding unit.
  • the motion information may vary depending on the encoding mode.
  • the index indicating the predictor is included.
  • the reference picture index, the difference motion vector, and the AMVP index of the current block are included.
  • the picture dividing unit 160 receives the prediction mode information and the size of the prediction block, and the prediction mode information indicates the intra mode.
  • the size of the prediction block may be a square of 64x64, 32x32, 16x16, 8x8, 4x4, or the like, but is not limited thereto. That is, the size of the prediction block may be non-square instead of square.
  • the reference pixels are read from the picture storage unit 172 to determine the intra-prediction mode of the prediction block.
  • the reference pixels are used to determine the intra prediction mode of the current block.
  • pixels adjacent to the upper side of the current block are not defined.
  • pixels adjacent to the left side of the current block are not defined.
  • pixels are not usable pixels. In addition, it is determined that the pixels are not usable even if the current block is located at the slice boundary and pixels adjacent to the upper or left side of the slice are not encoded and reconstructed.
  • the intra prediction mode of the current block may be determined using only available pixels.
  • the available reference pixels of the current block may be used to generate reference pixels of unusable positions.
  • the pixels of the upper block may be created using some or all of the left pixels, or vice versa.
  • available reference pixels at positions closest to the predetermined direction from the reference pixels at unavailable positions can be copied and generated as reference pixels.
  • the usable reference pixel at the closest position in the opposite direction can be copied and generated as a reference pixel.
  • the reference pixel may be determined as an unavailable reference pixel according to the encoding mode of the block to which the pixels belong.
  • the pixels can be determined as unavailable pixels.
  • an intra prediction mode of the current block is determined using the reference pixels.
  • the number of intra prediction modes that can be allowed in the current block may vary depending on the size of the block. For example, if the current block size is 8x8, 16x16, or 32x32, there may be 34 intra prediction modes. If the current block size is 4x4, 17 intra prediction modes may exist.
  • the 34 or 17 intra prediction modes may include at least one non-directional mode and a plurality of directional modes.
  • the one or more non-directional modes may be a DC mode and / or a planar mode.
  • the DC mode and the planar mode are included in the non-directional mode, there may be 35 intra-prediction modes regardless of the size of the current block.
  • it may include two non-directional modes (DC mode and planar mode) and 33 directional modes.
  • the planner mode generates a prediction block of the current block using at least one pixel value (or a predicted value of the pixel value, hereinafter referred to as a first reference value) located at the bottom-right of the current block and the reference pixels .
  • the configuration of the moving picture decoding apparatus can be derived from the configuration of the moving picture coding apparatus described with reference to FIGS. 1, 2 and 9, and for example,
  • the image can be decoded by performing an inverse process of the encoding process as described with reference to FIG.
  • FIG. 13 is a block diagram illustrating a configuration of a moving picture decoding apparatus according to an embodiment of the present invention.
  • the moving picture decoding apparatus includes an entropy decoding unit 210, an inverse quantization / inverse transform unit 220, an adder 270, a deblocking filter 250, a picture storage unit 260, An intra prediction unit 230, a motion compensation prediction unit 240, and an intra / inter changeover switch 280.
  • the entropy decoding unit 210 decodes the encoded bit stream transmitted from the moving picture encoding apparatus into an intra prediction mode index, motion information, a quantized coefficient sequence, and the like.
  • the entropy decoding unit 210 supplies the decoded motion information to the motion compensation prediction unit 240.
  • the entropy decoding unit 210 supplies the intra prediction mode index to the intraprediction unit 230 and the inverse quantization / inverse transformation unit 220. In addition, the entropy decoding unit 210 supplies the inverse quantization coefficient sequence to the inverse quantization / inverse transformation unit 220.
  • the inverse quantization / inverse transform unit 220 transforms the quantized coefficient sequence into an inverse quantization coefficient of the two-dimensional array.
  • One of a plurality of scanning patterns is selected for the conversion.
  • One of a plurality of scanning patterns is selected based on at least one of a prediction mode of the current block (i.e., one of intra prediction and inter prediction) and the intra prediction mode.
  • the intraprediction mode is received from an intraprediction unit or an entropy decoding unit.
  • the inverse quantization / inverse transform unit 220 restores the quantization coefficients using the selected quantization matrix among the plurality of quantization matrices to the inverse quantization coefficients of the two-dimensional array.
  • a different quantization matrix is applied according to the size of the current block to be restored and a quantization matrix is selected based on at least one of a prediction mode and an intra prediction mode of the current block with respect to the same size block.
  • the reconstructed quantized coefficient is inversely transformed to reconstruct the residual block.
  • the adder 270 reconstructs the image block by adding the residual block reconstructed by the inverse quantization / inverse transforming unit 220 to the intra prediction unit 230 or the prediction block generated by the motion compensation prediction unit 240.
  • the deblocking filter 250 performs deblocking filter processing on the reconstructed image generated by the adder 270. Accordingly, the deblocking artifact due to the video loss due to the quantization process can be reduced.
  • the picture storage unit 260 is a frame memory for holding a local decoded picture in which the deblocking filter process is performed by the deblocking filter 250.
  • the intraprediction unit 230 restores the intra prediction mode of the current block based on the intra prediction mode index received from the entropy decoding unit 210.
  • a prediction block is generated according to the restored intra prediction mode.
  • the motion compensation prediction unit 240 generates a prediction block for the current block from the picture stored in the picture storage unit 260 based on the motion vector information.
  • a prediction block is generated by applying a selected interpolation filter.
  • the intra / inter selector switch 280 provides the adder 270 with a prediction block generated in either the intra prediction unit 230 or the motion compensation prediction unit 240 based on the coding mode.
  • the current block is reconstructed using the predicted block of the current block restored in this manner and the residual block of the decoded current block.
  • a PS (parameter set) is divided into a picture parameter set (hereinafter, simply referred to as PPS) and a sequence parameter set (hereinafter simply referred to as SPS) which are data corresponding to the heads of each picture.
  • PPS picture parameter set
  • SPS sequence parameter set
  • the PPS and the SPS may include initialization information required to initialize each encoding, and may include SPATALAYOUT INFORMATION according to an embodiment of the present invention.
  • the SPS is common reference information for decoding all pictures coded in the random access unit (RAU), and may include a profile, a maximum number of pictures usable for reference, a picture size, and the like.
  • the PPS may include a variable length coding method as reference information for decoding the picture, an initial value of the quantization step, and a plurality of reference pictures.
  • the slice header SH includes information on the corresponding slice when coding is performed on a slice basis.
  • the method according to the present invention may be implemented as a program for execution on a computer and stored in a computer-readable recording medium.
  • Examples of the computer-readable recording medium include a ROM, a RAM, a CD- , A floppy disk, an optical data storage device, and the like.
  • the computer readable recording medium may be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And, functional programs, codes and code segments for implementing the above method can be easily inferred by programmers of the technical field to which the present invention belongs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Selon un mode de réalisation, la présente invention concerne un procédé de traitement d'image qui comprend les étapes consistant à : acquérir des informations d'image d'une image de réalité virtuelle à traiter; identifier une zone active et une zone inactive qui correspondent à un format de conversion des informations d'image; traiter par remplissage la zone inactive à l'aide des informations de pixel de la zone active; et effectuer un codage ou un décodage correspondant aux informations d'image traitées par remplissage.
PCT/KR2018/009835 2017-08-25 2018-08-24 Procédé et dispositif de codage/décodage d'image de réalité virtuelle WO2019039920A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2017-0107718 2017-08-25
KR20170107718 2017-08-25
KR10-2018-0062382 2018-05-31
KR1020180062382A KR20190022300A (ko) 2017-08-25 2018-05-31 가상 현실 영상의 부호화/복호화 방법 및 그 장치

Publications (1)

Publication Number Publication Date
WO2019039920A1 true WO2019039920A1 (fr) 2019-02-28

Family

ID=65439106

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2018/009835 WO2019039920A1 (fr) 2017-08-25 2018-08-24 Procédé et dispositif de codage/décodage d'image de réalité virtuelle

Country Status (1)

Country Link
WO (1) WO2019039920A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100789753B1 (ko) * 2005-07-18 2008-01-02 한국전자통신연구원 시간 및 시점간 참조영상 버퍼를 활용한 예측부호화/복호화장치 및 그 방법
JP2015159519A (ja) * 2014-02-25 2015-09-03 株式会社Ｊｖｃケンウッド 符号化装置、符号化方法および符号化プログラム
KR20160001430A (ko) * 2014-06-27 2016-01-06 삼성전자주식회사 영상 패딩영역의 비디오 복호화 및 부호화 장치 및 방법
JP6100904B2 (ja) * 2013-07-22 2017-03-22 ルネサスエレクトロニクス株式会社 動画像符号化装置およびその動作方法
JP2017130938A (ja) * 2014-07-07 2017-07-27 寰發股▲ふん▼有限公司ＨＦＩ Ｉｎｎｏｖａｔｉｏｎ Ｉｎｃ． イントラブロックコピー検索と補償範囲の方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100789753B1 (ko) * 2005-07-18 2008-01-02 한국전자통신연구원 시간 및 시점간 참조영상 버퍼를 활용한 예측부호화/복호화장치 및 그 방법
JP6100904B2 (ja) * 2013-07-22 2017-03-22 ルネサスエレクトロニクス株式会社 動画像符号化装置およびその動作方法
JP2015159519A (ja) * 2014-02-25 2015-09-03 株式会社Ｊｖｃケンウッド 符号化装置、符号化方法および符号化プログラム
KR20160001430A (ko) * 2014-06-27 2016-01-06 삼성전자주식회사 영상 패딩영역의 비디오 복호화 및 부호화 장치 및 방법
JP2017130938A (ja) * 2014-07-07 2017-07-27 寰發股▲ふん▼有限公司ＨＦＩ Ｉｎｎｏｖａｔｉｏｎ Ｉｎｃ． イントラブロックコピー検索と補償範囲の方法

Similar Documents

Publication Publication Date Title
WO2018048078A1 (fr) Procédé de codage/décodage d'image multivue synchronisée à l'aide d'informations de structure spatiale et appareil associé
WO2019132577A1 (fr) Procédé et dispositif d'encodage et de décodage d'image, et support d'enregistrement avec un train de bits stocké dedans
WO2018066988A1 (fr) Procédé et appareil de codage et de décodage de données d'images
WO2020218793A1 (fr) Procédé de codage basé sur une bdpcm et dispositif associé
WO2018044089A1 (fr) Procédé et dispositif pour traiter un signal vidéo
WO2018105759A1 (fr) Procédé de codage/décodage d'image et appareil associé
WO2021086055A1 (fr) Procédé de codage d'images basée sur une transformée et appareil associé
WO2018124819A1 (fr) Procédé et appareil pour traiter des signaux vidéo
WO2018066990A1 (fr) Procédé et appareil de codage/décodage de données d'images
WO2019009667A1 (fr) Procédé de traitement d'image synchronisée et appareil associé
WO2018221946A1 (fr) Procédé et dispositif de traitement de signal vidéo
WO2016200235A1 (fr) Procédé de traitement d'image basé sur un mode de prédiction intra et appareil associé
WO2021086061A1 (fr) Procédé de codage d'images basé sur une transformée et dispositif associé
WO2020242183A1 (fr) Procédé et dispositif de codage d'image sur la base d'une intraprédiction à grand angle et d'une transformée
WO2018074813A1 (fr) Procédé et dispositif de codage ou de décodage d'image
WO2018174542A1 (fr) Procédé et dispositif de traitement de signal vidéo
WO2021133100A1 (fr) Procédé et appareil de codage/décodage de vidéo permettant d'effectuer une pdpc, et procédé de transmission de flux binaire
WO2021086056A1 (fr) Procédé de codage d'image basé sur une transformée et dispositif associé
WO2021086064A1 (fr) Procédé de codage d'image basé sur une transformée, et appareil associé
WO2021010687A1 (fr) Procédé de codage d'image basé sur une transformée et dispositif associé
WO2019182293A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2019190203A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2019190197A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2019066529A1 (fr) Procédé et appareil pour coder/décoder une image de réalité virtuelle, qui permet le conditionnement de trame
WO2019039920A1 (fr) Procédé et dispositif de codage/décodage d'image de réalité virtuelle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18848272

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18848272

Country of ref document: EP

Kind code of ref document: A1