KR20180059348A - Method and apparatus for parallel processing image - Google Patents

Abstract

The present disclosure relates to a method and apparatus for encoding/decoding an image that includes a plurality of perspective images. The method of encoding an image according to the present disclosure may include a step of identifying the processing order of a perspective image to be encoded among the plurality of perspective images, a step of determining whether the identified processing order is identical to a predetermined processing order, and a step of converting the perspective image to be encoded when the identified processing order is not the same as the predetermined processing order. It is possible to improve coding/decoding efficiency.

Description

METHOD AND APPARATUS FOR PARALLEL PROCESSING IMAGE < RTI ID = 0.0 >

The present disclosure relates to a method and apparatus for image subdivision / decoding.

Specifically, the present disclosure relates to a method and apparatus for attaching / decoding virtual reality images, and more particularly, to a parallel processing method and apparatus for attaching / decoding virtual reality images.

Recently, broadcasting services having HD (High Definition) and UHD (Ultra High Definition) resolutions (1920x1080 or 3840x2160) have been expanded not only in the domestic market but also in the world. Users are becoming accustomed to high-resolution, high-definition video, and many organizations are spurring development on next-generation imaging devices.

Currently, Moving Picture Experts Group (MPEG) and Video Coding Experts Group (VCEG) jointly completed the standardization of High Efficiency Video Coding (HEVC), the next generation video codec. According to this, the compression efficiency of the image including UHD image is twice as high as that of H.264 / AVC. As the resolution of the image increases and the technology is developed, there is a growing interest in stereoscopic image as well as 2D image, realistic image such as forward (360 degrees) video. As a result, many companies have introduced head-mount displays or devices capable of reproducing realistic images.

SUMMARY OF THE INVENTION The present invention is directed to a method and apparatus for enhancing image subdivision / decoding efficiency.

It is another object of the present invention to provide a method and apparatus for improving the efficiency of subtraction / decryption of a virtual reality image.

It is another object of the present invention to provide a method and apparatus for parallel processing of attaching / decrypting a virtual reality image.

It is another object of the present invention to provide a method and apparatus for improving the speed of attaching / decrypting an image having a plurality of perspectives, such as a polyhedron 360 degree VR image.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to the present disclosure, there is provided a method of encoding an image including a plurality of perspective images, comprising the steps of: identifying a processing order of a subject perspective image in the plurality of perspective images; And if the identified processing order is not the same as the predetermined processing order, converting the to-be-encoded perspective image.

In the coding method according to the present disclosure, the processing sequence is identified as a processing start position and a processing progress direction, and the processing start position of the predetermined processing sequence is the upper left position of the perspective image, The direction may be clockwise.

In the encoding method according to the present disclosure, the transform may include at least one of a symmetric transform and a motion transform on the to-be-coded perspective image.

In the encoding method according to the present disclosure, the symmetric transformation may be performed when the processing progress direction of the identified processing sequence is different from the processing progress direction of the predetermined processing sequence.

In the encoding method according to the present disclosure, the symmetric transform may be one of a left-right symmetry transform, a vertically symmetric transform, and a diagonal symmetric transform.

In the encoding method according to the present disclosure, the rotation conversion may be performed when the process start position of the identified process sequence is different from the process start position of the predetermined process sequence.

In the encoding method according to the present disclosure, the rotation transformation may be a transformation that rotates the to-be-viewed perspective image so that the processing start position of the to-be-encoded perspective image becomes equal to the processing start position of the predetermined processing order.

In the encoding method according to the present disclosure, the step of converting the subject perspective image includes: Determining whether or not the processing order of the first-subject perspective image is the same as the predetermined processing order; and a step of processing the first converted perspective video And performing a second conversion of the first converted perspective view image if the order is not the same as the predetermined processing order.

In the encoding method according to the present disclosure, the first transformation may be one of a symmetric transformation and a motion transformation, and the second transformation may be the other one.

According to the present disclosure, an apparatus for encoding an image including a plurality of perspective images includes a converter, wherein the converter identifies a processing sequence of a perspective image to be encoded out of the plurality of perspective images, Determining whether the order is the same as the predetermined processing order, and when the identified processing order is not the same as the predetermined processing order, converting the to-be-viewed perspective image.

According to the present disclosure, there is provided a method of decoding an image including a plurality of perspective images, comprising the steps of: identifying a processing order of a perspective perspective image to be decoded among the plurality of perspective images; And if the identified processing order is not the same as the predetermined processing order, transforming the decoding target perspective image.

In the decoding method according to the present disclosure, the processing sequence is identified as a processing start position and a processing progress direction, the processing start position of the predetermined processing sequence is the upper left position of the perspective image, The direction may be clockwise.

In the decoding method according to the present disclosure, the transform may include at least one of a symmetric transform and a motion transform on the to-be-pasted perspective image.

In the decoding method according to the present disclosure, the symmetric transformation may be performed when the processing progress direction of the identified processing sequence is different from the processing progress direction of the predetermined processing sequence.

In the decoding method according to the present disclosure, the symmetric transform may be one of a left-right symmetric transform, a vertically symmetric transform, and a diagonal symmetric transform.

In the decoding method according to the present disclosure, the rotation conversion may be performed when the processing start position of the identified processing order is different from the processing start position of the predetermined processing order.

In the decoding method according to the present disclosure, the rotation transformation may be a transformation that rotates the decoding subject perspective image so that the processing start position of the decoding subject perspective image becomes equal to the processing start position of the predetermined processing order.

In the decoding method according to the present disclosure, the step of converting the subject image to be decoded includes: Determining whether or not the processing order of the first converted subject perspective image is the same as the predetermined processing order; and a step of determining whether or not the first converted first- And a second conversion of the first converted perspective perspective image if the processing order is not the same as the predetermined processing order.

In the decoding method according to the present disclosure, the first conversion may be one of a symmetric conversion and a shift conversion, and the second conversion may be another one.

According to the present disclosure, an apparatus for decoding an image including a plurality of perspective images includes a converter, wherein the converter identifies a processing order of the perspective perspective image to be decoded among the plurality of perspective images, Determine whether the order is the same as the predetermined processing order, and convert the to-be-pasted perspective image if the identified processing order is not the same as the predetermined processing order.

According to the present disclosure, a computer-readable recording medium can store a bitstream generated by a method and / or apparatus for encoding an image according to the present disclosure.

According to the present disclosure, it is possible to provide a method and apparatus for improving the image subdivision / decoding efficiency.

Further, according to the present disclosure, a method and an apparatus for improving the sub-decoding efficiency of a virtual reality image can be provided.

Furthermore, according to the present disclosure, it is possible to provide a method and apparatus for parallel processing of attaching / decrypting a virtual reality image.

Further, according to the present disclosure, it is possible to provide a method and apparatus for improving the speed of attaching / decrypting an image having a plurality of perspectives, such as a polyhedron 360-degree VR image.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a diagram illustrating various types of cameras for photographing a 360-degree video.
Fig. 2 is a diagram exemplarily showing an image obtained by projecting 360-degree video onto a three-dimensional space.
3 is a diagram illustrating a structure of an image encoding apparatus according to the present disclosure.
4 is a diagram illustrating a structure of an image decoding apparatus according to the present disclosure.
5 is a diagram for explaining an example of a diversity camera image and a converged camera image.
6 is an exemplary diagram for explaining division of an image.
7 is a diagram for explaining the WPP processing procedure in an image.
8 is a diagram for explaining tile-based parallel processing.
FIG. 9 is a diagram illustrating a developed view for embedding / decoding 360 degrees forward video.
10 is a diagram for explaining a developed view when a 360-degree omnidirectional image is projected on a cube.
11 is a view for explaining various methods for developing an omnidirectional image projected on a cube in two dimensions.
12 is an exemplary diagram for explaining the dependency between CTUs belonging to adjacent perspectives.
FIG. 13 is a diagram for explaining a subdecryption ending point according to a developing method.
14 is a diagram for explaining a difference in image processing time according to a developed view.
15 is a conceptual diagram for explaining image parallel processing according to the present disclosure.
16 is a flowchart for explaining image parallel processing according to the present disclosure.
17 is a diagram illustrating a processing procedure of an omni-directional image on a cube.
FIG. 18 is a view illustrating an exploded view of the forward direction image of FIG. 17; FIG.
19 is an example of a 360 degree VR image in the form of a cube-like developed view.
FIG. 20 is a diagram for explaining a step (step 1) of parallelly embedding / decoding the video of FIG. 19;
FIG. 21 is a diagram for explaining a step (step 2) of parallelly embedding / decoding the video of FIG. 19; FIG.
FIG. 22 is a diagram for explaining a step (step 3) of parallelly embedding / decoding the video of FIG. 19; FIG.
Fig. 23 is an exemplary diagram for explaining the minus / decryption direction of the perspective. Fig.
FIG. 24 is a diagram for explaining a subdecrypting order for two non-neighboring perspectives in a cubic omnidirectional image.
FIG. 25 is a diagram for explaining a subdecoding sequence of two opposite perspectives in a cubic omnidirectional image.
Fig. 26 is a diagram for explaining a difference according to (a) and (b) of Fig. 25;
FIG. 27 is a diagram for explaining the subdecryption according to FIG. 25 (b). FIG.
Fig. 28 is a diagram for explaining the subdecryption of a perspective capable of referring to two or more perspectives. Fig.
FIG. 29 is a diagram for explaining a method of dividing one perspective into two areas, and to embed / decode each area. FIG.
FIG. 30 is a diagram for explaining a negative / decryption processing procedure in the case of referring to two or more perspectives.
31 is an exemplary diagram for explaining a subdecoding sequence of a cubic omni-directional image.
32 is a diagram illustrating a configuration of an encoding apparatus according to the present disclosure;
33 is a diagram illustrating a configuration of a decoding apparatus according to the present disclosure.
34 is an exemplary diagram for explaining a method of converting the processing order of the perspective.
35 is a diagram illustrating 33 spatial prediction modes.
36 is an exemplary diagram for explaining a spatial prediction of a current perspective referring to a prediction mode of a neighboring perspective.
FIG. 37 is a diagram for explaining a difference between prediction modes of a current block and a reference block. FIG.
38 is a diagram for explaining a method of analyzing a directional prediction mode of a reference block.
FIG. 39 is a diagram illustrating eight cases of the directional prediction mode according to the expansion method of the perspective.
40 is a view for explaining the position of a reference sample used for spatial prediction of a cubic omnidirectional image.
41 is a diagram for explaining a method of parallel processing using continuity to a developed image of a cubic omnidirectional image.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

This disclosure relates to parallel processing of images. Specifically, the present disclosure relates to an apparatus for increasing the processing speed by parallelly attaching / decrypting images having various viewpoints. An image having various viewpoints can be, for example, a 360-degree moving image, an omnidirectional video, a polyhedron 360-degree VR image, and the like. In the image subdivision / decoding according to the present disclosure, it is possible to use a reference in various directions in an image or between images. In the present disclosure, perspective, or perspective, may refer to the position of a camera or the direction of a camera that captures video.

Conventional 2D video has a fixed viewpoint. However, omni-directional video or 360-degree video means video that can be viewed at a desired point in time. Omni-directional video or 360-degree video can be obtained by taking all directions of 360 degrees from one point using a plurality of cameras, a fish-eye lens, or a reflector, and projecting them onto polyhedrons or spheres in three-dimensional space. The omnidirectional video or the 360-degree video can be reproduced only in the entire area corresponding to the viewpoint of the user.

1 is a diagram illustrating various types of cameras for photographing a 360-degree video.

Fig. 1 (a) illustrates several cameras, and Fig. 1 (b) illustrates a fish-eye lens. Fig. 1 (c) illustrates a reflector.

Fig. 2 is a diagram exemplarily showing an image obtained by projecting 360-degree video onto a three-dimensional space.

2 (a) shows an example in which a 360-degree image is projected onto a polyhedron in a three-dimensional space. 2 (b) shows an example in which a 360-degree image is projected onto a sphere in a three-dimensional space. In the example of FIG. 2, the shaded area indicates the position where the 360-degree image is reproduced. That is, the shaded portion in FIG. 2 may be an area corresponding to a point of view of the user.

To compression-encode omnidirectional video (360-degree video), a method and apparatus for compression-encoding 2D video may be used.

3 and 4 are views illustrating a structure of an image encoding apparatus and a decoding apparatus according to the present disclosure.

The image encoding apparatus of FIG. 3 can generate a bitstream by receiving an input image and encoding the input image. The image encoding apparatus includes a reconstruction picture buffer 301, an inter picture prediction unit 302, an intra prediction unit 303, a switch 304, a subtracter 305, a transform unit 306, a quantization unit 307, An inverse transform unit 308, an inverse quantization unit 309, an inverse transform unit 310, and / or an adder 311. [

The input image can be encoded in block units. The current block can be predicted intra-picture or inter-picture based on the already coded / decoded image.

When the current block is intra-picture predicted, the switch 304 can be switched to the intra-picture prediction unit 303. [ The intra prediction unit 303 can predict the current block by referring to the already coded / decoded area in the current picture. When the current block is inter-picture predicted, the switch 304 can be switched to the inter-picture predicting unit 302. [ The inter-picture prediction unit 302 can predict a current block by referring to at least one picture already stored in the reconstruction picture buffer 301 after being added / decoded.

The prediction block of the current block generated by intra-picture prediction or inter-picture prediction can be transmitted to the subtractor 305. [ The subtractor 305 may generate a difference block between the current block and the prediction block of the current block. The conversion in the conversion unit 306 and / or the quantization in the quantization unit 307 can be performed on the generated difference block. The residual coefficients that have undergone transformation and / or quantization can be encoded in the encoding unit 308. [ The encoding unit 308 can perform entropy encoding based on, for example, the probability of occurrence of a symbol.

The inverse quantization in the inverse quantization unit 309 and / or the inverse transformation in the inverse transformation unit 310 may be performed on the transformed and / or quantized difference block. The inverse quantized and / or inverse transformed difference block may be sent to the adder 311. The adder 311 may add the transmitted difference block to the prediction block of the current block to restore the current block. The restored picture including the restored block may be stored in the restored picture buffer 301. [

The image decoding apparatus of FIG. 4 can generate a reconstructed image by receiving and decoding a bitstream. The video decoding apparatus includes a decoding unit 401, an inverse quantization unit 402, an inverse transformation unit 403, an adder 404, an inter picture prediction unit 405, an intra prediction unit 406, a switch 407, and / Or a reconstructed picture buffer 408.

The decoding unit 401 may generate a difference block of a current block by decoding the bitstream. The decoding unit 401 can perform entropy decoding based on, for example, the probability of occurrence of a symbol. The difference block may comprise transformed and / or quantized difference coefficients.

The inverse quantization in the inverse quantization unit 402 and / or the inverse transformation in the inverse transformation unit 403 may be performed on the difference block including the transformed and / or quantized difference coefficient. The inverse quantized and / or inverse transformed differential block may be sent to the adder 404. The adder 404 may add the transmitted difference block to the prediction block of the current block to restore the current block. The reconstructed picture including the reconstructed block may be stored in the reconstructed picture buffer 408.

The prediction block of the current block input to the adder 404 may be generated based on the already decoded image. When the current block is predicted on the screen, the switch 407 can be switched to the intra prediction unit 406. [ The intra prediction unit 406 can predict the current block by referring to the already-coded / decoded area in the current picture. When the current block is inter-picture predicted, the switch 407 can be switched to the inter-picture predicting unit 405. The inter-picture prediction unit 405 can predict a current block by referring to at least one picture that has already been coded / decoded and stored in the reconstruction picture buffer 408.

The sub-decoding method according to the present disclosure can be applied to realistic media images in addition to forward (360) video. The realistic media image may include a divergent camera image, a convergent camera image, and the like.

5 is a diagram for explaining an example of a diversity camera image and a converged camera image.

The divergent camera image may include images W1, W2, and W3 in various directions obtained using a plurality of cameras C1, C2, and C3. Convergent camera images may include images W, W2, W3 and W4 in a specific direction obtained using a plurality of cameras C1 to C4.

The moving picture encoding apparatus can compress a still image or a moving image. The moving picture encoding apparatus can perform frame based compression for compressing an entire image in one processing unit or block based compression for processing an image in units of specific blocks. The image coding / decoding apparatus for performing frame-based compression has a problem that it is difficult to apply a prediction technique and requires a large amount of memory and a calculation amount. Therefore, most image decoding / decoding devices are performing block-based compression.

The block-based compression can be performed, for example, by dividing an image into a large-block unit Coding Tree Unit (CTU). CTUs can be coded / decoded in progressive scan order. The progressive scan sequence may be a raster scan sequence or a Z scan sequence. For example, the CTU may be divided into a coding unit (CU), which is a smaller block unit, and then added / decoded. This method is advantageous in terms of compression efficiency because the block size can be adjusted more precisely according to the characteristics of the image.

6 is an exemplary diagram for explaining division of an image.

6 (a) shows a state in which one image is divided into CTU units. Referring to Fig. 6 (a), each CTU can be decoded in the order of CTU A to CTU K according to the progressive scanning order.

The block-based moving picture compression apparatus eliminates redundant information by using the degree of correlation between neighboring blocks in an image, thereby reducing the amount of information to be transmitted and improving compression efficiency. Referring to FIG. 6A, the neighboring CTUs B, C, D, and E have been subdivided and decoded at the time when CTU F starts to be subdivided / decoded. Therefore, the compression efficiency of the CTU F can be increased by using information of neighboring CTUs. This may mean that the decoding of CTU B, C, D, E must be completed in order to decode CTU F.

6 (b) shows an example of recursively dividing the CTU. As described above, the CTU can be divided into a plurality of CUs, where recursive partitioning can be applied. For recursive partitioning, for example, quadtree partitioning may be used. As in the example shown in FIG. 6 (b), the CTU F can be divided into four CUs. At this time, the division flag indicating whether CTU F is divided may be 1. Each divided CU can be added / decoded in Z scan order. In addition, each CU can be further divided into four CUs. Alternatively, each CU can be interrupted at its depth. The division flag of each CU may indicate whether the corresponding CU is divided or not. Such recursive partitioning can be stopped by the partition flag value or the maximum allowable division depth specified in the image. As shown in FIG. 6 (b), the CTU F may be divided into four CUs, and the first, second and third CUs may not be divided in Z scan order. In addition, the fourth CU F1 can be further divided into four CUs. The first, second and third CUs in the Z scan order among the four CUs generated by dividing the CU F1 may not be divided. In addition, the fourth CU F2 can be further divided into four CUs. The size of the prediction block can be adjusted in the image through the divided structure illustrated in FIG. 6 (b), and the efficiency of prediction can be improved.

Wavefront parallel processing (WPP) and / or tile-based subtraction / decryption may be used to improve the speed of image subtraction / decoding. Wavefront technology is a technology that starts parallel processing when dependence on pixel information is resolved. Referring to FIG. 6 (a), CTU D can solve the dependency on pixel information when the B / C decoding, which is the CTU of the upper row, is completed. Therefore, a prediction algorithm using information of CTU B can be applied. Likewise, CTU H can resolve the dependence on the pixel information once the top row CTU E has been subdivided / decoded. Therefore, a prediction algorithm using information of CTU E can be applied. However, entropy coding can not be performed in parallel even if images are parallelly added / decoded in consideration of pixel information dependency, and they are executed in a sequential scanning order. Therefore, entropy coding is becoming a bottleneck in parallel processing during subtraction / decryption. This problem can be solved by applying WPP. WPP can additionally support parallel concatenation / decryption for entropy coding. For this purpose, a sub-bit stream is generated in units of rows of each CTU and the positions of the sub-bit streams are transmitted to the slice header, thereby enabling parallel entropy coding for each sub-bit stream. That is, entropy coding can be used independently for each row through WPP.

In WPP, to minimize the degradation of coding efficiency of entropy coding and to support parallel partitioning / decoding, the bottom row acquires an entropy context model at the time when the second block of the upper row is completed and starts subdecryption. The current block and neighboring blocks are highly correlated. Thus, rather than using a fully initialized context model in each row, it is more advantageous to obtain updated context information from the neighboring block and initialize the context model of the current row to increase the compression efficiency of the sub-decoder.

7 is a diagram for explaining the WPP processing procedure in an image.

The block in the image of Fig. 7 means CTU. The number in the block indicates the processing order of each CTU. The arrow in Fig. 7 indicates that the context information of the upper row is acquired and the context model of the current row is initialized.

In Fig. 7, the shaded CTU represents a CTU that has been encoded and / or decoded. Also, a non-shadowed CTU represents a CTU that has not yet been encoded and / or decoded.

8 is a diagram for explaining tile-based parallel processing.

As shown in FIG. 8, one picture may be divided into several tiles (e.g., six tiles) having a rectangular shape. Each tile can be subdivided into regions independent of each other. The CTUs included in each tile can be subdivided / decoded in order of sequential scanning. That is, in the case of tile-based parallel processing, blocks located at the boundary of a tile can be encoded or decoded without using information such as a pixel value or a motion vector of a block belonging to a neighboring tile. Thus, when using tiles, similar to the case of slices, each tile area can be encoded or decoded simultaneously.

The tile segmentation information may be signaled, for example, via a Picture Parameter Set (PPS). The tile partition information may include information about a starting point of each tile. Or tile segmentation information may include information about one or more vertical boundaries and / or one or more horizontal boundaries that divide the picture. Or tile segmentation information may include information about whether to partition the picture into tiles of equal size.

FIG. 9 is a diagram illustrating a developed view for embedding / decoding 360 degrees forward video.

An image of the 360-degree omnidirectional video can be projected and processed in an expanded view of a polyhedron or an equirectangular form in which a sphere is expanded, as shown in FIG. When an image is reproduced, the image is projected on a polyhedron or a sphere, and the user can reproduce the image in the form of viewing the image from the center of the polyhedron or sphere. For example, as shown in FIG. 2, it can be assumed that the viewpoint of the user is at the center of a polyhedron or a sphere.

10 is a diagram for explaining a developed view when a 360-degree omnidirectional image is projected on a cube.

As in the example shown in FIG. 10, when a 360-degree omnidirectional image is projected on the cube, the image information can be stored for each of the six planes P1, P2, P3, P4, P5 and P6. When an image is reproduced, as in the example shown in Fig. 10A, each of the six images can be reproduced by being projected onto a cube of a three-dimensional space. Here, the position of the viewer is at the center of the cube, and each of the faces can be the position and / or direction (perspective) of the viewpoint of the eye. For example, in the example shown in Fig. 10, if the perspective P4 is the front face of the cube, the perspective P1 is the upper side of the cube, the perspective P2 is the back of the cube, the perspective P3 is the left side of the cube, the perspective P5 is the cube, May correspond to the lower side of the cube.

An omnidirectional image projected on a polyhedron can be expressed in various developed views depending on how the polyhedron is developed. 11 is a view for explaining various methods for developing an omnidirectional image projected on a cube in two dimensions.

For example, as in the example shown in Fig. 11, there may be eleven modes of deployment. Alternatively, the developed view obtained by symmetrically projecting the developed view shown in Fig. 11 up / down / left / right may be treated as another developed view.

In the case of independently embedding / decoding each perspective, the developed view shown in FIG. 11 may have no meaning other than information on where the image is located. However, if sub-decoding is performed in the order of sequential scanning using the continuity of image information between perspectives, there may be a difference in the sub-decoding speed depending on the type of the developed scene.

Hereinafter, the difference in the subdecryption rate according to the developed view will be described in detail.

12 is an exemplary diagram for explaining the dependency between CTUs belonging to adjacent perspectives. In FIG. 12, each of A, B, C, D, E, and F represents the position of the deployed perspectives on the developed view, and the thick line represents the boundary between perspectives. A small rectangle inside each perspective represents a block (e.g., CTU) that is a unit of image processing.

When the image is added / decoded through WPP parallel processing, CTU e1 of perspective E has dependency on CTU b1 of perspective B and CTU d2 of perspective D. Therefore, the subdecryption of the CTU b1 of the perspective B and the CTU d2 of the perspective D are completed, so that the CTU e1 of the perspective E can be subdivided / decrypted. For the same reason, the subdecryption of CTU c1 of perspective C and CTU e2 of perspective E must be completed before subtraction / decryption of CTU f1 of perspective F is possible.

In the case of omnidirectional images, information may not exist at a specific position of the image depending on the developed view. For example, the image of the perspective D shown in Fig. 12 may not exist depending on the developed view. However, in the conventional WPP parallel processing method, the partial decoding of the perspective E can be started after the subtraction / decoding of the perspective B is performed, even if the perspective D image does not exist. Therefore, the sub / decryption processing time of the entire image is not different from when the perspective image D exists. However, if there is no perspective F image on the developed view, the sub-decoding of the entire image can be completed when the image of the perspective E is decoded / decoded.

FIG. 13 is a diagram for explaining a subdecryption ending point according to a developing method.

FIG. 13A is a diagram showing a case where the last perspective is the D position in FIG. 12 in the sub-decoding order. FIG. 13B is a diagram showing a case where the last perspective is at the position E in FIG. 12. FIG. In Figs. 13A and 13B, a thick line indicates a boundary between perspectives. Also, the small rectangle in each perspective represents the CTU. In addition, the numbers inside each CTU indicate the order in which the corresponding CTU is added / decoded.

Assuming that the sub-decode processing time of each CTU is the same, in the case of FIG. 13 (a), it is understood that the sub-decode of the entire image is completed when the 23rd CTU is processed. 13 (b), it can be confirmed that the entire image is added / decoded when the 28th CTU is processed. As described above, when the WPP method for performing the subtraction / decimation in the order from the upper left to the lower right is used, it can be seen that the time at which the entire image is added / decoded is different even if the number of perspectives included in the image is the same . That is, the position (or deployment method) of the perspective may affect the subdecryption processing speed.

The order of addition / decryption processing using WPP can be determined by image information dependency. That is, the processing of the CTU can be started only when two CTUs in the upper row have been processed. Assuming that the subtotal processing time of each CTU is the same, the delay per row becomes the time to process two CTUs due to the above dependency. Also, since the processing of the CTU can be started when one CTU adjacent to the left side is processed, the delay per column becomes the time to process one CTU.

14 is a diagram for explaining a difference in image processing time according to a developed view.

Referring to Fig. 14 (a), each viewpoint (perspective) has 5 x 5 CTUs. The last point is located in the third row of the fourth row, and the last CTU is located in the 20th row and the 15th column. As described above, there are two CTUs per row, one CTU delay per row. Therefore, the last CTU is processed in (2 x 20 (row) - 1) + (1 x 15 (column) - 1) th. As a result, in FIG. 14 (a), it can be confirmed that the last CTU is completed at the 53rd time. Therefore, if the processing time of one CTU is t (CTU), the time required for the processing of the entire image according to the developed view of Fig. 14 (a) is 53 * t (CTU).

FIG. 14 (b) is a diagram illustrating another developed view of an omnidirectional image having 5 × 5 CTUs per viewpoint. Referring to FIG. 14B, the CTU located at the lower right of the developed view exists in 15th row, 5th column, 10th row and 20th column. If the processing time is calculated for each case, the CTUs in the 15th row and the 5th column are processed in the (33) th row (2 x 15 (row) -1) + (1 x 5 (column) -1). The CTUs of the 10th row and the 20th column are processed in the (38th) (2 占 10 (rows) -1) + (1 占 20 (columns) -1). Therefore, the time required for processing the entire image is 38 * t (CTU). As described above, it can be seen that the processing speed of the WPP differs depending on how the cube of the same size is developed but how it is developed.

The delay in processing time caused by the dependency between adjacent CTUs can be mitigated by applying a tile based technique. That is, each surface (i.e., a region having different perspective) is treated as an independent tile and is concatenated / decoded. In the case of parallel processing of each perspective into independent tiles, the processing time of the entire image can be shortened to a time for processing one perspective (e.g., 25 * t (CTU) in the example of Fig. 14). The tile-based parallel division / decryption can be usefully applied to an image that a user views at a certain distance. However, in the case of a realistic image such as a 360-degree VR image, a user can view a part of the image at a very close position, and therefore, the resolution must be much larger than that of existing images. Particularly, in the case of a 360-degree VR image, the degree of immersion is very important, so that a high image quality is required such that a pixel can not be seen during image reproduction.

Using the tile-based technique, each tile is treated independently, reducing the reference area that can be used for prediction and reducing the compression efficiency. Therefore, it may not be appropriate to apply the tile-based technology to the 360-degree omnidirectional image, which requires a large compression efficiency.

Therefore, for parallel processing of 360-degree VR images, it is important to use the dependency between the perspectives to decode. 13 and 14, it has been described that various development methods can be applied for the processing of a 360-degree VR image, and parallel processing can be performed using the dependencies between adjacent perspectives on each developed view. However, if only continuity between perspectives on a developed view is taken into consideration, there is a limit to shorten the entire image processing time. For example, a developed view of a 360-degree VR image is created by spreading three-dimensionally connected perspectives, as shown in FIG. Therefore, on the polyhedron, there are perspectives adjacent to the number of sides of the perspective. That is, the number of adjacent perspectives available on the polyhedron is greater than the number of adjacent perspectives available on the developed view.

Hereinafter, a description will be given of a method and an apparatus for parallel division / decoding of images considering the characteristics of a 360 degree VR image. Specifically, according to the present disclosure, it is possible to more efficiently handle the parallel processing of subtitling / deciphering using one or more continuity between perspectives in an image having a plurality of perspectives such as 360-degree video (forward video) .

As in the example shown in FIG. 10A, the 360-degree omnidirectional image is stored in a developed view of a polyhedron having various perspectives such as a cube. On the developed view, as in the example shown in FIG. 10 (c), the perspective P1 is adjacent to the perspective of the bottom. However, in reality, as in the example shown in Fig. 10A, each perspective is adjacent to four perspectives in the left-right and up-down directions. This means that the image information can be encoded / decoded in a direction different from the progressive scanning direction of the upper left to lower right of the conventional art. In addition, there is no dependency between non-neighboring perspectives in three-dimensional space because there is no continuity between images. That is, since non-neighboring perspectives do not refer to each other, the prediction efficiency does not deteriorate even if the decoding is performed in parallel. Decryption parallel processing is performed by utilizing the characteristic that there are continuous perspectives in various directions and that the direction of subdecryption is possible in various directions and that one or more perspectives having no correlation between images exist, The processing speed can be increased while minimizing the loss.

15 is a conceptual diagram for explaining image parallel processing according to the present disclosure.

16 is a flowchart for explaining image parallel processing according to the present disclosure.

17 is a diagram illustrating a processing procedure of an omni-directional image on a cube.

FIG. 18 is a view illustrating an exploded view of the forward direction image of FIG. 17; FIG.

 In FIGS. 15, 17, and 18, the faces having the same alphabets refer to the points that are not referred to each other. For example, A and A 'represent points of time that do not refer to each other. B and B 'represent points of time other than A and A' that are not referred to each other. C, and C 'represent points in time that do not refer to each other, except for the above points. Points that do not refer to each other can be concatenated and decoded.

In Fig. 15, an arrow connecting the viewpoint and the viewpoint represents a reference relationship between the viewpoints. For example, the arrow connecting the viewpoint A and the viewpoint B refers to the viewpoint A during the subdecryption of the viewpoint B. The number of threads can be determined by the number of time points that can be added / decoded in parallel. For example, if there are N number of viewpoints that can be concatenated and decoded, N or more threads can be used.

With reference to FIG. 15, the subdecryption method according to the present disclosure will be described. Referring to FIG. 15 (a), one or more viewpoints which can be simultaneously decoded at the same time as the viewpoint A and the viewpoint A can be added / decoded in parallel. For example, since the point A and the point A 'have no continuity, they can be processed in parallel. Referring to (b) of FIG. 15, after some or all of the time points A and A 'processed in (a) of FIG. 15 are added / decoded, Decode one or more viewpoints which can be concurrently concatenated with the viewpoint B and the viewpoint B in parallel. For example, since the point of time B and the point of time B 'have no continuity, they can be processed in parallel. Referring to (c) of FIG. 15, one or more viewpoints that can be simultaneously decoded at the time point C and at the same time as the point-in-time C can be added / decoded in parallel. In this way, the remaining viewpoints can be concatenated and decoded at the time points that can be added / decoded in parallel.

The method described with reference to Fig. 15 can be represented by the flowchart of Fig.

In step S1610 of FIG. 16, the subdecryption for one perspective or two or more non-neighboring perspectives may begin.

In step S1620, the subdecryption for the perspective in which the subdecryption is started can be proceeded. The addition / decryption may proceed with the encoding units included in each perspective (e.g., in CTU units).

In step S1630, it can be determined whether subdecryption for all perspectives has been completed. If the addition / decoding of all the perspectives has been completed (Yes in step S1630), the video / decoding can be terminated.

If the subdecryption for all the perspectives has not been completed (No at step S1630), at step S1640, it can be determined whether there is a neighborhood perspective whose dependency has been solved by the partial / decrypted or completed perspective at step S1640. If there are no neighboring perspectives for which the dependency has been resolved (No at step S1640), at step S1620, the subdecryption for the perspective undergoing subdecryption can continue.

If there is a neighboring perspective for which the dependency has been resolved (Yes at step S1640), then at step S1650, subdecryption for the neighboring perspective may begin.

Thereafter, in step S1620, the subdecryption for the plurality of perspectives in which the subdecryption is started can be continued. The steps S1620 to S1650 may be repeated until the subdecryption for all the perspectives is completed.

19 is an example of a 360 degree VR image in the form of a cube-like developed view.

Figs. 20 to 22 are diagrams for explaining steps of parallelly embedding / decoding the video of Fig. 19; Fig. In the following description with reference to Figs. 20 to 22, the symbols representing the respective perspectives are the same as those of the corresponding perspective shown in Fig. The arrows shown in Figs. 21 and 22 show reference relationships between perspectives.

First, as shown in Fig. 20, non-neighboring perspectives A and / or A 'can be added / decoded in parallel.

Then, as shown in Fig. 21, perspective B and / or B 'can be concatenated / decoded with reference to perspective A and / or A'. The subdecryption of Perspective B and / or B 'can be started when the dependence on Perspective A and / or A' is solved. That is, the partial decoding of the perspective B and / or B 'can be started when the partial decoding of the perspective A and / or A' is partially or completely performed. As shown in FIG. 21, perspective B and / or B 'can be added / decoded with reference to perspective A and / or A'. Perspective B and / or B 'can be added / decoded at the boundary portion adjacent to the perspective A and / or A', respectively, from the portion where the dependency has been resolved according to the negative / decryption of the perspective A and / or A '.

For example, the upper boundary of perspective B is adjacent to the left boundary of perspective A. Thus, when the block in the second row, first column of perspective A is negative / decoded, the decoding / decoding of the block in the first row, first column of perspective B can be started. In this case, subtraction / decoding of perspective B referring to perspective A can be performed in accordance with the raster scan order.

Also, for example, the lower boundary of perspective B is adjacent to the left boundary of perspective A '. Thus, when the block of the second row, the first column of the perspective A 'is negative / decoded, the decoding / decoding of the block of the last row and the last column of perspective B can be started. In this case, subtraction / decoding of perspective B referring to perspective A 'may be performed in accordance with an inverse raster scan order (e.g., from bottom right to top left).

When the perspective B and / or B 'are mined / decoded referring to the perspective A and / or A', the information of the referenced block may be used as it is, or may be transformed May be used.

Then, referring to the perspective A, A ', B and / or B' which have been partially / decrypted for some or all, the perspective C and / or C 'can be subtracted / decoded . In Fig. 22, the case where the perspective C and / or C 'is subtracted by referring to the perspective A and / or A' is shown. However, the present invention is not limited thereto. For example, perspective C and / or C 'may be subdivided by referring to perspective B and / or B'. Or perspective C and / or C 'can be added / decoded with reference to one or more perspectives of the perspectives A, A', B and B '. In this case, the referenced block may be referenced after being appropriately modified according to the point in time at which the current subdecryption is in progress.

In the case of the cube, the addition / decryption of the 360-degree VR image may be terminated in the three steps shown in FIGS. 20 to 22. FIG. However, in the case of a regular octahedron or a regular dichroic body, there may be a perspective in which the addition / decryption has not been completed yet. Thus, the above process can be repeated until all the perspectives have been subdivided / decoded.

As described above, each perspective of an omnidirectional image has continuity with a plurality of perspectives. Also, each perspective may not have continuity with at least one perspective. The present disclosure can use this to parallelize the decoding of the perspectives of the omnidirectional image.

Further, in addition to the above-described method, applying WPP-based technology to each perspective enables processing in parallel and faster while maintaining a high compression efficiency. However, as described with reference to Fig. 7, when each row is processed in parallel from the upper left block to the lower right block of each perspective, the dependency of the upper left block of each perspective is solved, Decryption can not be started.

Fig. 23 is an exemplary diagram for explaining the minus / decryption direction of the perspective. Fig.

23, even if the perspective P 2 and P 4 are parallelly subdivided using the WPP-based technique in the cube facial omnidirectional image, the minus / decryption of the perspective P 4 is almost in order to start the WPP-based partial decoding of the perspective P 6 It must be completed. That is, two CTUs of the last row of the perspective P4 must be added / decoded to start the decoding / decoding of the perspective P6.

However, in an omnidirectional image, all sides of each perspective are connected to different perspectives. Therefore, by adjusting the subdecryption sequence of perspective P6, subdecryption of perspective P6 can be started sooner. For example, as in the example shown in FIG. 23, when the subdecryption of the perspective P2 is performed from the lower right block to the upper left block, the dependency of the lower left CTU of the perspective P6 can be solved by only two CTUs of the perspective P2 have. Once the dependency of the bottom left CTU of perspective P6 is solved, the WPP for perspective P6 can proceed from the bottom left block to the top right block.

In performing the WPP-based sub-decryption, the pixel information of the neighboring perspective may be utilized, and the entropy context model may be initialized by retrieving the updated entropy context from the neighboring perspective. In the example shown in FIG. 23, Indicates the direction in which the entropy context is taken and initialized. As shown in FIG. 23, reference of information between adjacent perpectives is possible.

Hereinafter, with reference to FIG. 23, the sub-decoding method according to the present disclosure will be described in more detail.

First, we can start subdecryption on one perspective or on two or more non-neighboring perspectives. In each perspective, subdecryption can be performed in the order described with reference to FIG. For example, since the perspectives P2 and P4 are not adjacent to each other, subdecryption can be started in parallel. At this time, for more efficient subdecryption, the perspectives P2 and P4 can be added / decoded in different order.

FIG. 24 is a diagram for explaining a subdecrypting order for two non-neighboring perspectives in a cubic omnidirectional image. In FIGS. 24B and 24C, small blocks represent image blocks (for example, CTUs) constituting the perspectives P4 and P2, and numbers in the blocks indicate the processing order of the corresponding blocks. In addition, the dotted arrow shows the processing direction of one row in each perspective, and each block (CTU) can be used by updating the entropy context of the previous column of the same row.

As shown in (a) of FIG. 24, the perspectives P2 and P4 are not adjacent to each other, so they are not referred to each other. Thus, the perspectives P2 and P4 can be independently decoded / decoded.

As shown in (b) of FIG. 24, for example, the perspective decoding can be performed in the order from upper left to lower right in accordance with the normal raster scanning order for the perspective P4. At this time, a normal raster scan order can also be applied to the perspective P2. As shown in FIG. 24 (c), for the perspective P2, the subdecryption can proceed from the lower right end to the upper left end.

Thereafter, when the partial decoding of the perspectives P2 and / or P4 is partially or completely performed, the subdecryption of the neighboring perspectives in which the dependency is solved can be started. At this time, the WPP processing direction of the neighboring perspective can be determined based on the WPP processing direction applied to the reference perspective. For example, in Fig. 23, when the subdecryption of the block in the first row and the second column of the perspective P4 is completed, the subdecryption of the block in the last row and the first column of the perspective P1 can be started. At this time, the sub-decoding of the perspective P1 can proceed from the bottom left to the top right.

Further, when the subdecoding of the block in the last row and the last column in the perspective P2 is completed, the decoding / decoding of the block in the last row and the first column of the perspective P6 can be started. At this time, the subdivision / decoding of the perspective P6 can proceed from the bottom left to the top right.

Subsequently, subdecryption for the neighboring perspective in which the dependency with the perspective (s) to which the addition / decryption has been performed is solved can be started. At this time, the direction of the WPP can be determined in accordance with a perspective in which dependencies among one or more reference perspectives are resolved more quickly. If the dependencies of two or more reference perspectives are solved at the same time, subtraction / decoding of the current perspective can proceed simultaneously on several sides. In this case, the current perspective can be divided and processed.

Referring to FIG. 23, when the subdecoding / decoding of the block in the last second column and the last row in the perspective P2 is completed, the decoding / decoding of the block in the last row and the first column of the perspective P3 can be started. Further, when the addition / decryption of the block of the second row and the first column of the perspective P4 is completed, the subdecryption of the block of the first row and the last column of the perspective P3 can be started. That is, the sub-decoding of the perspective P3 can be carried out simultaneously on several sides with reference to a plurality of perspectives. At this time, the minus / decode order of the perspective P3 can be determined in consideration of at least one of the neighboring perspectives P1, P2, P4, and P6. At this time, the subdecrypting order of the perspective P3 can be determined in the direction in which the embedding / decoding can be completed as soon as possible.

The method described with reference to FIGS. 23 and 24 can be repeated until the subdecryption of all the time points is completed.

In the example shown in Fig. 24, the subdecryption of the perspective P4 proceeds from the upper left to the lower right on the developed view, and the subdecryption of the perspective P2 proceeds from the lower right to the upper left. This is to better distribute the amount of data being processed in parallel for more efficient parallelism.

FIG. 25 is a diagram for explaining a subdecoding sequence of two opposite perspectives in a cubic omnidirectional image.

The sub-decoding order of the two perspectives facing each other in the three-dimensional space can be set to the opposite direction to each other as shown in Fig. 25 (a). Or may be set in the same direction to the left and right as shown in Fig. 25 (b).

Fig. 26 is a diagram for explaining a difference according to (a) and (b) of Fig. 25;

As shown in Fig. 25A, when the affixing / decoding of the perspectives P4 and P2 is started at symmetrical positions in the up, down, left, and right directions, as shown in Fig. 26 (b) Are opposite to each other at the sides where the perspectives P2 and P1 meet. This is true of the sides where the perspectives P6 and P4 meet. When two perspectives are opposite in the processing direction of each perspective on the adjacent side, the information of neighboring perspectives can not be utilized efficiently. That is, the processing order with respect to the neighborhood perspective is reversed, so that a delay occurs until the decoding of the reference row of the neighboring perspective is completed.

As shown in Fig. 26 (b), when the side / decryption of the perspectives P4 and P2 proceed in the same direction from left to right, as shown in Fig. 26 (c), the perspectives P2 And the CTU heat treatment direction of P1 are the same. This is true of the sides where perspectives P6 and P4 meet. Therefore, the delay does not occur since the time at which the sub-decoding of the last row of the perspectives P1 and P6 is started and the time at which the last row CTU of the perspectives P4 and P2, which will be used as the reference area, are completed. FIG. 27 is a diagram for explaining the subdecryption according to FIG. 25 (b). FIG.

As described above, if two CTUs are processed in each of the perspectives P2 and P4, it is possible to perform decoding via the WPP of the neighboring perspective. Since the perspective P1 has continuity with the perspective P4, the dependency of the first CTU of the perspective P1 is solved when the subtraction / decoding of the first row and second CTU of the perspective P4 is completed, as shown in Fig. 27 (b). Therefore, an entropy context can be obtained from the perspective P4 and initialized and used.

Likewise, the perspective P6 can also start subdecryption at the time when the second CTU of the perspective P2 has been added / decoded. As shown in FIG. 27 (c), the WPP can also be performed for the perspective P6 in accordance with the direction in which the perspective P2 is processed. 27 (b) and (c), the solid line arrow indicates the direction in which the entropy context is taken and initialized. The dashed arrows indicate the processing direction of the CTU in each row.

As shown in Figures 27 (b) and (c), an entropy context may be taken from another perspective. The reference of the entropy context may be performed through the context extractor described later with reference to FIGS. 32 and 33. FIG.

Fig. 28 is a diagram for explaining the subdecryption of a perspective capable of referring to two or more perspectives. Fig.

As shown in Fig. 27, in the case of perspectives P3 and P5, neighboring perspectives are all being subdivided / decoded. Therefore, the sub-decoding efficiency can be improved by performing parallel processing using all of the neighboring perspectives.

As shown in FIG. 28, the perspectives P3 and P5 can utilize the information of the neighboring perspectives P1, P2, P4 and / or P6. The solid-line arrows and the dotted-line arrows used in Fig. 28 have the same meanings as the solid-line arrows and the dotted-line arrows used in Fig. That is, the solid line arrow indicates a direction in which entropy information (for example, entropy context, entropy context table, etc.) is obtained. In addition, the dotted arrows indicate the row processing direction through WPP in each perspective. The number in the CTU block in each perspective indicates the order of the sub-decryption processing.

As shown in Fig. 28, dependencies can be solved simultaneously in two or more directions, depending on the sub-decoding of the neighboring perspective in the case of the perspective P3 or the perspective P5. According to the minus / decryption sequence, the perspectives P2 and P4 start minus / decryption first than the perspectives P1 and P6. Therefore, the dependencies required for the subdecryption of the perspectives P3 and P5 can be solved first by the perspectives P2 and P4. Thus, the perspectives P3 and / or P5 can select one of the perspectives P2 and P4 to determine the WPP processing direction. Alternatively, the perspective P3 and / or P5 may be divided into two regions to utilize both perspectives, and the WPP processing direction according to perspectives P2 and P4 may be determined for each divided region.

For example, the subdecryption of the perspective P3 can be performed after the dependence on the perspective P2 has been resolved. In this case, the CTU 8 of the perspective P3 can be added / decoded based on the two CTUs 1, 3 of the perspective P2. Or at the point when two CTUs of the perspectives P2 and P4 are respectively added / decoded, the decoding / decoding of the perspective P3 starts in both directions. In this case, the subdecryption of the two CTUs of the perspective P3 whose dependence is solved by the sub-decoding of the perspectives P2 and P4 can be started in parallel. Parallel concatenated / decoded means that the perspective P3 is added / decoded independently of each other in both directions. Thus, parallel concatenation / decoding may or may not be performed at the same time.

FIG. 29 is a diagram for explaining a method of dividing one perspective into two areas, and to embed / decode each area. FIG.

Figure 29 (a) shows one perspective divided by regions A and B, and Figure 29 (b) shows WPP processing after dividing the perspective into areas of equal size. FIG. 29 (c) shows WPP processing after dividing the perspective into regions of different sizes. The meanings of the solid line arrows and the dotted line arrows in Fig. 29 are the same as those described with reference to Figs. 27 and 28.

The A region and the B region may be represented by a first region and a second region. The first area and the second area may be subdivided / decoded based on the degree of subdecryption of the adjacent perspective, respectively. For example, when the first area and the first perspective are adjacent to each other, a block of the first area whose dependency is solved by sub-decoding of the first perspective can be added / decoded. Decoding directions of the blocks included in the first area may be determined based on the subdecryption direction for the first perspective. For example, the subregions / decryption directions for the first area and the first perspective may be directions orthogonal to each other.

The method of dividing one perspective into two regions is not limited to the embodiment described with reference to Fig. For example, it may be divided into shapes or sizes not shown in Fig. A method of dividing a perspective into two regions may be signaled through a bitstream or predefined in an encoder and decoder.

Neighbor CTUs having continuity with perspectives P3 and / or P5 may be included in one row rather than columns in perspectives P2 and / or P6. As described above, when the WPP process is performed, the lower row subdivision / decoding can be started after the subtitling / decoding of the CTU 2 column of the upper row is completed, so that a delay occurs per row by 2 * t (CTU). 28 (a), the perspective P3 has 2 * t (CTU) instead of 1 * t (CTU) for each column because the sub / CTU) occurs.

FIG. 30 is a diagram for explaining a negative / decryption processing procedure in the case of referring to two or more perspectives.

If more than two perspectives can be referenced, subdecryption on the current perspective can be performed based on the degree of subdecryption of the two or more perspectives being referenced. For example, even if the dependency is solved by sub-decoding of the first perspective, it is possible to wait until the dependency is solved by the sub-decoding of the second perspective, and then start sub-decoding of the current perspective. Referring to two or more perspectives, as compared to when referring to a perspective, a delay may occur. However, since two or more blocks can be referred to, improvement in coding efficiency can be expected.

As described above, a delay per column is generated by 2 * t (CTU) in the subdecryption processing of the first row of the perspective P3. Therefore, as shown in Fig. 30 (a), even if the subtraction / decoding of the leftmost lowermost CTU 4 of the perspective P3 is completed, the CTU 6 of the next column can not be immediately decoded / decoded. Since the delay of 2 * t (CTU) per column occurs, when the subdecryption is started each time the dependency of each CTU of the perspective P3 is solved, the last CTU of the first row in the processing sequence of perspective P3 becomes the 11th negative / Lt; / RTI >

In this case, considering the fact that the last CTU of the first row of the perspective P3 is subtracted and / or decoded for the 11th time as shown in Fig. 30 (b), the sub-decoding start of the first CTU of the first row of the perspective P3 The point of time can be determined as the seventh. As shown in FIG. 30 (b), even if the start / end point of the first CTU of the perspective P3 is delayed, the start / end point of the last CTU of the first row is not changed. .

Further, when the perspective P3 is processed according to the procedure shown in FIG. 30 (b), there is an advantage that the number of neighboring CTUs that can be referred to is increased. That is, in the subdecryption of the first CTU of the perspective P3, the entropy context as well as the pixel information of the CTU (5, hatched portion) of the second row of the perspective P6 are available.

30 (c), if the addition / decryption of the perspective P3 is further delayed by 1 * t (CTU), the CTU 11 (hatched area) of the neighboring perspective P1 can also be referred to. This is because the CTU processing times of the perspectives P1 and P2 differ by only 2 * t (CTU). Therefore, in the case of perspective P3, as in the example shown in FIG. 30C, the subdecryption start time of perspective P3 is determined based on the processing time of the right CTU (belonging to the neighborhood perspective) of the last CTU of the first row .

30 (c) may cause slightly more delay than the case of FIG. 30 (b). However, since more areas can be referred to, the subdecryption efficiency can be improved.

In this disclosure, information that can be referred to from adjacent blocks, CTUs, and / or perspectives is described as pixel information, entropy context, entropy context table, and the like. However, the present invention is not limited to this, and it is possible to refer to all the sub-decode information of the adjacent area. For example, in addition to the pixel information existing in the image, the information that can be referred to may be simply a prediction mode used for spatial prediction, a motion vector used for temporal prediction, merge index, quantization parameter information, in-loop filter information, And includes all information generated or used in the decoding process.

As described above, in the case of a regular octahedron or a regular trilinear image other than a cube, the above-described steps described in consideration of a cube can be repeatedly performed.

31 is an exemplary diagram for explaining a subdecoding sequence of a cubic omni-directional image.

When carrying out the subdecryption of the cube omni-directional image according to the procedure described with reference to Figs. 23 to 30, the subdecryption can proceed according to the procedure shown in Fig.

FIG. 31 (a) is a diagram illustrating an example of a subdecryption sequence on a developed view. The solid and dashed arrows used in FIG. 31 (a) have the same meaning as the solid and dotted arrows used in FIG. 27 and the like.

31 (b) and 31 (c) illustrate the order in which the subdecryption is progressed on the cube. 31 (b) and (c) refer to an encoding unit block (for example, CTU), and numerals represent the order in which the blocks are processed. The shaded block means the block in which the addition / decryption has been completed, and the non-shaded block means the block in which the addition / decryption has not yet started.

As described above, when performing the min / decode of the omnidirectional image according to the present disclosure, the WPP processing order may vary from perspective to perspective. Or decoding / decoding may be started in many directions simultaneously for one perspective. If the processing directions of a plurality of perspectives included in one omnidirectional image are different, it is not possible to embed / decode all the perspectives into devices having the same configuration, and in consideration of the subdecoding direction of each perspective, It is necessary to provide a separate service.

32 is a diagram illustrating a configuration of an encoding apparatus according to the present disclosure;

33 is a diagram illustrating a configuration of a decoding apparatus according to the present disclosure.

32 may include a transform and distributor 3201, an image information converter 3202, a context information extractor 3203, and a plurality of encoders. The encoding apparatus of FIG. 32 can generate a bitstream by encoding an input image. The input image may be an omnidirectional image, and each of the encoders may encode a perspective of one of a plurality of perspectives constituting an omnidirectional image. Each of the encoders includes an encoding processor, a reconstructed picture buffer, and / or an entropy encoder, and may correspond to the encoding apparatus of FIG.

The decoding apparatus of FIG. 33 may include an image information converter 3301, a context information extractor 3302, a transformer and integrator 3303, and a plurality of decoders. The decoding apparatus of FIG. 33 can output an image by decoding the bit stream. The output image may be an omni-directional image, and each decoder may decode a perspective of one of a plurality of perspectives constituting an omnidirectional image. Each of the decoders includes a decoding processor, an entropy decoder, and / or a restored picture buffer, and may correspond to the decoding apparatus of FIG.

If a different subdecoder is required depending on the subdecryption direction of each perspective, the plurality of encoders and the plurality of decoders of FIGS. 32 and 33 have different configurations, . Therefore, even if the processing direction of each perspective is different, an additional configuration is required so that it can be processed by an encoder or a decoder having the same configuration. To this end, the encoder according to the present disclosure may further include a transform and distributor 3201 and / or an image information transformer 3202. In addition, the decoder according to the present disclosure may further include an image information converter 3301 and / or a transformer and integrator 3302.

In order to parallelly process the respective perspectives constituting the omnidirectional image, the transform and distributor 3201 of Fig. 32 can transmit image information of each perspective to one or more encoding processors. The image information can be modified so that the processing direction of each perspective matches the predetermined processing direction before transmission of the image information. The predetermined processing direction may be, for example, a raster scanning order, but is not limited thereto, and may be predetermined or signaled as the processing direction commonly applied to all the perspectives.

The transform and distributor 3201 may operate as follows.

First, if the input image includes a plurality of perspectives in an integrated form, the input image is divided for each perspective (step 1). The omni-directional images can be stored separately for each perspective or can be integrated and stored in the form of a single developed view. At this time, if it is stored in the form of one developed view, it is divided per perspective for parallel processing. Each of the perspectives can be sent to each encoding processor.

Next, if the CTU processing order in the perspective is counterclockwise, a symmetric image is generated so as to be clockwise (step 2). 34 is an exemplary diagram for explaining a method of converting the processing order of the perspective. Each of (a), (b) and (c) in FIG. 34 shows an embodiment for converting the processing order of the perspective P6, which is the counterclockwise processing order of the CTU, to be clockwise.

In Figure 34 (a), the perspective P6 of the stage 1 can be symmetrically left and right to create a symmetrical perspective of the stage 2. [ In Fig. 34 (b), the symmetric perspective of the stage 2 can be generated by symmetry of the perspective P6 of the stage 1 up and down. 34C, a symmetric perspective of the stage 2 can be generated by symmetrically mirroring the perspective P6 of the stage 1 in a diagonal direction. In each of FIGS. 34 (a), (b) and (c), it can be seen that the processing order of the CTU of the symmetry perspective of stage 2 is converted clockwise.

Finally, the image is rotated on the basis of the center of the perspective so that the sub / decoding start position of the symmetry perspective of the stage 2 is on the upper left side (step 3). In the case of (a) in FIG. 34, since the sub-decoding start position is located at the lower right end, the symmetry perspective of the stage 2 can be rotated 180 degrees. In the case of FIG. 34 (b), since the sub-decode start position is located at the upper left end, there is no need to rotate the stage 2 symmetry perspective. In the case of (c) in FIG. 34, since the sub / demodulation start position is located at the lower left end, the symmetry perspective of the stage 2 can be rotated clockwise by 90 degrees. A perspective of stage 3 can be generated by rotation of the symmetry perspective of stage 2. [

If the processing order of CTUs in the perspective of stage 1 is clockwise, the symmetry may not be performed and only rotation, for example, may be performed. As described above, by performing symmetry and / or rotation of the perspective, it is possible to match the affine / decryption processing order of the perspective in a predetermined order. The above example described with reference to FIG. 34 is an example of how the subdecryption of each perspective can be started from the upper left corner. In addition, the symmetry and / or rotation may be represented by a matrix. Also, the symmetry and rotation may be expressed in the form of a matrix multiplication and may be performed by a single operation.

Each of the embodiments described with reference to FIG. 34 is a part of an embodiment for matching the CTU processing order in the perspective to a predetermined order. That is, the CTU processing sequence can be matched by applying a method other than the embodiments described with reference to FIG. For example, unlike the embodiment described with reference to FIG. 34, the CTU processing sequence may be matched by rotating the perspective first and then symmetrically. That is, in the case of FIG. 34 (a), perspective of stage 3 can be obtained by rotating the perspective of stage 1 180 degrees and then symmetrically rotating the rotated perspective.

In addition, a method of simultaneously applying rotation and symmetry can be considered. In addition to the rotation and the symmetry, the perspective may be converted using the mapping information between the processing order of the perspective of stage 1 and the processing order of the perspective of stage 3. For example, the mapping information may include information for identifying a processing order of the stage 1's perspective and conversion information for the perspective of the stage 1. For example, the conversion information may be information for converting the perspective of the stage 1 into the perspective of the stage 3. For example, the conversion information may be information regarding the correspondence between the CTUs of the stage 1 and the CTUs of the stage 3. The correspondence may be the position of the CTU of stage 1 and the position of the CTU of stage 3 corresponding thereto. The positions of the CTUs may be indicated in an order according to a predetermined scan order. The predetermined scan order may be, for example, a raster scan order. Or the position of the CTU may be a two-dimensional coordinate with respect to a predetermined corner of the perspective. The predetermined corner may be, for example, the upper left corner of the perspective.

In the example described with reference to FIG. 34, when the CTU processing order of the input perspective is different from the predetermined processing order, symmetry and / or rotation are performed to convert the perspective. However, the processing order of the subdecoder may be modified without converting the input perspective. For example, when the input perspective is the same as the perspective of FIG. 34 (a), and the predetermined processing order is the raster scan order, the sub-decoder performs subtraction / decryption in the counterclockwise direction starting from the lower left CTU of the input perspective . At this time, the above-described mapping information between the processing order of the perspective of stage 1 and the processing order of the perspective of stage 3 can be used.

As described above, when processing a block included in the perspective, the image information of the block included in the neighboring perspective can be referred to. As described above, in addition to the pixel information existing in the image, the image information may include a prediction mode used for spatial prediction, a motion vector used for temporal prediction, merge index, quantization parameter information, in-loop filter information, And may include any information that is generated or used during the subtraction / decryption process.

However, in the process of expressing an image on a three-dimensional space, the position and direction of the perspective are different. Therefore, in order to share image information that can be used for different perspective predictions, it is necessary to convert the image information according to the direction of the developed view of each perspective. As described with reference to Fig. 34, it is necessary to convert the image information by performing symmetric and / or rotational transformation for matching the affine / decode processing order of the perspective from the upper left corner to the lower right corner.

The decoder can determine a conversion method of the video information to be referred to based on the information on the developed view used by the encoder and / or the information on the transform applied to each perspective. The subscriber / decoder can pre-promise information about the developed view, information about the location on the developed view of each perspective, and / or information about the transformations applied to each perspective. Or all or part of the information may be signaled explicitly or implicitly through the bitstream. Or the decoder may derive some or all of the information based on other syntax or variables relating to the subdecryption.

The above-described conversion of the image information can be performed by the image information converter 3202 of FIG. 32 and the image information converter 3301 of FIG. 33.

If the subscriber / decoder can use one promised method, the flag information is included in an image header such as a Video Parameter Set (VPS) or a Sequence Parameter Set (SPS) You can signal whether it is used or not.

If the encoder uses a certain developed view among the plurality of developed views or if the processing order of the perspective processed by the transformer and distributor 3201 is variable, information on the type of developed view used in encoding or the processing order of perspectives is explicitly Lt; / RTI > Or the index of the look-up table when the sub-decoder shares a look-up table for the information or the like.

The transforming and combining unit 3303 of FIG. 33 can reverse the process of the transforming and distributing unit 3201 of FIG. More specifically, the following process can be performed.

First, when a rotation is applied to the perspective image in the encoding apparatus, the encoding apparatus can rotate the image in the opposite direction by the rotated angle (step 1).

Whether the rotation has been applied to the perspective image and / or the rotation angle can be signaled through the bitstream. The rotation application and / or rotation angle can be directly added / decoded. For example, a flag indicating whether or not the rotation is applied and a syntax element related to the rotation angle can be added / decoded.

There may be a limited number of rotation angles depending on the shape of the perspective image. For example, if the perspective image is a rectangle, the rotation angle may be one of 90 degrees, 180 degrees, and 270 degrees. Thus, the syntax element for the angle of rotation may be information indicating one of three cases.

Bit string " 00 " is " not rotated ", bit string " 01 " is " rotated 90 degrees ", bit string " 10 & And the bit string " 11 " may indicate " rotated 270 degrees ". The bit string and its corresponding meaning can be pre-set in the subdecoder and stored, for example, in the form of a table.

If the perspective image is n-angular, there may be one case where rotation is not applied and n-1 cases where rotation is applied. In the case of (n-1) kinds, it may mean that the rotation angles are different from each other. That is, when the perspective image is n-angular, the information on the rotation transformation may be information indicating one of n cases. Therefore, using the log ₂ n bit syntax element, information on whether or not the rotation is applied and information about the rotation angle can be added / decoded.

Next, when the encoding apparatus applies the symmetry to the perspective image, the symmetric image can be generated based on the center line used in the encoding apparatus (step 2).

Information about whether the symmetry is applied to the perspective image and / or the center line used for symmetry can be signaled through the bitstream. Information on whether or not the symmetry is applied and / or the center line can be directly added / decoded. For example, a flag indicating whether symmetry is applied or not and a syntax element relating to a center line may be added / decoded.

The number of available centerlines may be limited. For example, the subdecoder may preliminarily promise the number of available centerlines. Or the number of available centerlines may be signaled.

For example, when the center line is one of three kinds of vertical lines, horizontal lines, and diagonal lines, the information about the center line may be information indicating one of three cases.

The bit string " 00 " is " not symmetrical ", the bit string " 01 " is " symmetrical using a vertical line ", the bit string " 10 " Symmetric ", and the bit string " 11 " may indicate " symmetric using diagonal lines ". The bit string and its corresponding meaning can be pre-set in the subdecoder and stored, for example, in the form of a table.

Next, when one developed view in which a plurality of perspectives are combined is generated, the perspective images passed through Step 1 and / or Step 2 can be integrated into one according to the developed view.

Steps 1 and 2 can be performed by applying the operation used by the transformer and distributor 3201 of the encoding apparatus inversely. Step 3 may be performed if the user of the decoding apparatus requests one developed view of a plurality of perspectives combined.

As described above, in order to share the pixel information that can be utilized in inter-perspective prediction, the position difference between the perspectives generated by expressing the image of the three-dimensional space on the developed image and / or the sub- The pixel information can be corrected in consideration of the conversion for matching.

Further, in order to smoothly refer to the image information of the adjacent perspective used for the prediction, it is necessary to modify the image information of the adjacent perspective.

For example, in the case of a motion vector used for temporal prediction, symmetric and / or rotational transformation is performed in the same manner as the correction considering the positional difference between perspectives on the developed view and / or the transformation for matching the sub-decoding processing order in a predetermined order .

However, the modification of the spatial prediction mode may not be solved by the above method. The reason will be described later. In order to perform intra prediction, for example, adjacent pixel information at the lower left, upper left, upper and upper right of the current block can be used. Further, as the directional prediction mode, as shown in Fig. 35, 33 prediction modes can be used. Alternatively, more directional prediction modes may be used to support more precise directionality.

If image subdivision / decoding is performed from the upper left to the lower right, pixel information exists only in a specific direction. Therefore, the directional prediction mode that can be referred to may be the prediction mode of the lower left, left, upper, and / or upper right. In addition, a mode number according to the direction of the spatial prediction is stored unlike the motion vector. Therefore, the mode number may not match the direction of the actual spatial prediction. Therefore, when the perspective is converted into symmetry or rotation, the prediction information of the neighboring block can not be utilized as it is.

36 is an exemplary diagram for explaining a spatial prediction of a current perspective referring to a prediction mode of a neighboring perspective.

36 (a) shows the position of the block A of the perspective P2 and the position of the block B of the perspective P6 on the developed view of the cubic omnidirectional image. Figure 36 (b) shows the positions of the blocks A and B after correcting the positions of the perspectives P2 and the perspective P6 in consideration of the positional difference on the developed view. FIG. 36C shows the positions of the blocks A and B after the correction in which the order of decoding / decoding of the perspective P6 is made to coincide with a predetermined direction, for example, from the upper left corner to the lower right corner.

The perspective P6 in FIG. 36C can be encoded in the order from the top left to the bottom right through the conversion and distributor 3201 in FIG. Therefore, the block B can be predicted using the intra prediction mode illustrated in FIG. However, the encoded pixel information of the block A of FIG. 36 (c) is converted so that the block B can refer to it. However, since the image information other than the pixel information is before conversion, the block B can not be used as it is. Therefore, it is also necessary to perform the conversion process corresponding to the conversion of the pixel information with respect to other image information. At this time, the conversion of the pixel information is as described with reference to Fig. 34, for example.

However, in the case of the intra prediction, since the mode number is used for the directionality, after the conversion process, the mode number for the directionality may not be the same or may not exist.

If the directionality of the spatial prediction available for the current block (e.g., block B in FIG. 36) and the reference block (e.g., block A in FIG. 36) is the same, only the modification of the mode number changed by the transformation may be required. However, the mode corresponding to the prediction mode of the current block may not exist for the reference block.

FIG. 37 is a diagram for explaining a difference between prediction modes of a current block and a reference block. FIG.

FIG. 37A exemplarily shows a directional prediction mode available for the block A and the block B shown in FIG. 36C. As shown in FIG. 37 (a), block A is transformed according to the transformation of block B, and it can be seen that a mode that block B can utilize for prediction other than block A is other than mode number 26. However, the prediction direction itself of the block A may be useful for the sub-decoding of the prediction direction of the block B. Therefore, by interpreting the mode number of the block A so that the block B can use it, the efficiency of the prediction technique such as the Most Probable Mode (MPM) can be improved.

The interpretation of the directional prediction mode can be performed as follows. When the prediction direction of the reference block is present in a directional mode (for example, the directional mode described with reference to FIG. 35) according to a predetermined processing order (for example, a progressive scanning order), the directional prediction mode of the reference block is a prediction direction As shown in FIG. For example, in Fig. 37A, when the prediction direction of the reference block A is the upper right direction, the upper right direction direction is included in the directional mode shown in Fig. Therefore, in this case, the directional prediction mode of the reference block A can be interpreted as '34'.

If the prediction direction of the reference block does not exist in the directional mode according to a predetermined processing order (for example, a progressive scanning order), that is, in the downward direction or the right direction, Direction can be interpreted as a mode number assigned to the direction. For example, in Fig. 37 (a), when the prediction direction of the reference block A is the right direction, the right direction is not included in the directional mode shown in Fig. Therefore, in this case, the directional prediction mode of the reference block A can be interpreted as a mode number " 10 " assigned to the left direction that is 180 degrees symmetrical in the right direction.

37B shows an example in which the bottom directionality (solid line arrow) in FIG. 37A is interpreted as the upward direction (dotted line arrow). The mode 2 of the directional mode of FIG. 37 (b) is included in the existing directional mode. Therefore, 180 degrees may not be symmetrically interpreted.

37C shows an example in which the right directionality (solid line arrow) in FIG. 37A is interpreted as left directionality (broken line arrow). The 34th mode of the directional mode of FIG. 37 (c) is included in the existing directional mode. Therefore, 180 degrees may not be symmetrically interpreted.

38 is a diagram for explaining a method of analyzing a directional prediction mode of a reference block.

In step S3810, the image information of the reference block can be converted. Conversion of the image information of the reference block can be performed in accordance with the current block. For example, considering the conversion of the current block, or considering the difference between the current block and the perspective including the reference block.

In step S3820, it may be determined whether the directional prediction mode of the uninterpreted reference block is available for the current block. For example, if the directional prediction mode of the uninterpreted reference block is included in the available directional prediction mode for the current block, the corresponding directional prediction mode of the reference block can be determined to be available for the current block.

If the directional prediction mode of the reference block is not available for the current block (No at step S3820), at step S3830, a mode that is 180 degrees symmetric to the corresponding directional prediction mode of the reference block may be generated. Thereafter, in step S3840, the directional prediction mode of the reference block can be interpreted. For example, the directional prediction mode of the reference block may be interpreted as a mode of the current block having the same directionality as the directional prediction mode of the reference block generated in step S3830.

If the directional prediction mode of the reference block is available for the current block (Yes at step S3820), at step S3840, the directional prediction mode of the reference block is interpreted as a mode of the current block having the same directionality as the directional prediction mode of the reference block .

In step S3850, it may be determined whether all the directional prediction modes of the reference block have been interpreted. If there is an uninterpreted mode among the directional prediction modes of the reference block, the process may be repeated at step S3820.

Assuming the prediction mode shown in FIG. 35, the directional prediction mode of the reference block may be, for example, one of the eight cases shown in FIG. 39, depending on which development view is used to develop each perspective of the cubic all- .

Among the eight cases shown in Fig. 39, it is necessary to interpret the directional mode of the reference block in accordance with the current block, except for the case of the current block (for example, in the case of (a) of Fig. 39). The interpretation of the directional mode of the reference block can be performed as described with reference to FIG. Alternatively, the directional mode of the reference block can be interpreted by referring to a lookup table (analysis table) shared by the sub / decoding apparatus. When referring to a table, it is necessary to determine which of the eight cases of Fig. 39 the reference block corresponds to. Which can be signaled via a bitstream. Or information on the developed view used, and other partial / decoded information.

Table 1 below is an example of the directional mode interpretation table. Table 1 is a table of 33 directions. However, the number of directions is not limited thereto.

Analysis mode (a) (b) (c) (d) (e) (f) (g) (h) 2 18 18 34 * * 34 2 3 19 17 33 19 17 3 33 4 20 16 32 20 16 4 32 5 21 15 31 21 15 5 31 6 22 14 30 22 14 6 30 7 23 13 29 23 13 7 29 8 24 12 28 24 12 8 28 9 25 11 27 25 11 9 27 10 26 10 26 26 10 10 26 11 27 9 25 27 9 11 25 12 28 8 24 28 8 12 24 13 29 7 23 29 7 13 23 14 30 6 22 30 6 14 22 15 31 5 21 31 5 15 21 16 32 4 20 32 4 16 20 17 33 3 19 33 3 17 19 18 2, 34 2, 34 18 2, 34 2, 34 18 18 19 3 33 17 3 33 19 17 20 4 32 16 4 32 20 16 21 5 31 15 5 31 21 15 22 6 30 14 6 30 22 14 23 7 29 13 7 29 23 13 24 8 28 12 8 28 24 12 25 9 27 11 9 27 25 11 26 10 26 10 10 26 26 10 27 11 25 9 11 25 27 9 28 12 24 8 12 24 28 8 29 13 23 7 13 23 29 7 30 14 22 6 14 22 30 6 31 15 21 5 15 21 31 5 32 16 20 4 16 20 32 4 33 17 19 3 17 19 33 3 34 * * 2 * 18 2 34

In addition, it is necessary to correct the image information in consideration of the perspective difference in the spatial prediction. 40 is a view for explaining the position of a reference sample used for spatial prediction of a cubic omnidirectional image. In Figure 40, the dashed box represents the location of the reference samples used for spatial prediction of each CTU (blocks A, B, and C). As shown in FIG. 40 (a), when performing spatial prediction of a cubic omnidirectional image, there may be no reference sample on the developed view. For example, in the case of the upper left CTU (block A) shown in FIG. 40A, the upper left reference sample X does not exist. Also, in the case of the rightmost CTU (block B) and the bottom CTU (block C), a correct reference sample may not be selected due to the difference between perspectives. As described above, when performing spatial prediction, when there is no reference sample and / or correction of pixel information is required due to the difference between perspectives, the sample value of the neighboring perspective is used to calculate the non- Or by transforming the neighborhood perspective to the current perspective to determine a reference sample.

For example, the reference sample X of the block A in FIG. 40 (a) can be generated by interpolating the image information of the perspectives P1 and P2. Further, in the case of perspectives P4 and / or P6 different from the perspective P3, it is possible to generate a correct reference sample through the viewpoint correction.

41 is a diagram for explaining a method of parallel processing using continuity to a developed image of a cubic omnidirectional image.

The two cube-shaped developed views used in Fig. 41 are shown in Figs. 11 (a) and (e). In the example shown in FIG. 41, each perspective includes 5x5 CTUs, and the numbers in each CTU represent the WPP processing order. The addition / decryption processing can be started from the hatched blocks a1 and a2 in Fig. 41 (a). Perspectives A and B may not refer to each other on a simple flat rather than a three-dimensional view. When the processing of the row to which the block a2 belongs reaches the column to which the block a1 belongs, even if the sub-decoding starts simultaneously at the positions of the blocks a1 and a2 in Fig. 41A, the dependency is solved, I never do that.

Similarly, even in the case of FIG. 41B, since the dependence is solved when the processing of the row to which the block b2 belongs reaches the column to which the block b1 belongs, even if the subdecryption starts simultaneously at the positions of the blocks b1 and b2, There is no delay caused by < / RTI >

In the example shown in FIG. 41, the encoder starts subtraction / decoding at two or more positions on the developed view. Thus, for example, the encoder may use two entropy information (e.g., an entropy context table) for blocks a1 and a2. The decoder needs to know the method used in the encoder to perform decoding. Therefore, the entropy information used in the encoder, the information on the start time and / or the development start time of the encoding, and the like can be signaled through the bit stream. Or by referring to a look-up table. Or other coding parameters and / or internal variables.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (21)

A method of encoding an image including a plurality of perspective images,
Identifying a processing order of a perspective image to be coded among the plurality of perspective images;
Determining whether the identified processing order is the same as a predetermined processing order; And
If the identified processing order is not the same as the predetermined processing order,
And converting the perspective image to be coded. The method according to claim 1,
The processing sequence is identified as a processing start position and a processing progress direction,
Wherein the processing start position of the predetermined processing order is the upper left position of the perspective image and the processing progress direction of the predetermined processing order is a clockwise direction. 3. The method of claim 2,
Wherein the transformation includes at least one of a symmetric transformation and a motion transformation for the perspective image to be encoded. The method of claim 3,
Wherein the symmetric transformation is performed when the process progress direction of the identified process sequence differs from the process progress direction of the predetermined process sequence. The method of claim 3,
Wherein the symmetric transform is one of a left-right symmetry transform, a vertically symmetric transform, and a diagonal symmetric transform. The method of claim 3,
Wherein the rotation conversion is performed when the processing start position of the identified processing sequence is different from the processing start position of the predetermined processing sequence. The method according to claim 6,
Wherein the rotation transformation is a transformation that rotates the subject perspective image so that the processing start position of the subject perspective image becomes equal to the processing start position of the predetermined processing order. The method according to claim 1,
Wherein the step of transforming the subject perspective image comprises:
A first transforming the perspective image to be encoded;
Determining whether the processing order of the first converted perspective image is the same as the predetermined processing order; And
If the processing order of the first converted subject perspective image is not the same as the predetermined processing order,
And a second transforming the first converted perspective image to be encoded. 9. The method of claim 8,
Wherein the first transform is one of a symmetric transform and a motion transform, and the second transform is the other one. An apparatus for encoding an image including a plurality of perspective images,
Wherein the encoding apparatus includes a converter,
The converter comprising:
And a determination unit configured to identify a processing sequence of the perspective image of the plurality of perspective images and determine whether or not the identified processing sequence is the same as the predetermined processing sequence and determine whether the identified processing sequence is the same as the predetermined processing sequence And convert the to-be-encoded perspective image. A method of decoding an image including a plurality of perspective images,
Identifying a processing order of the perspective perspective image among the plurality of perspective images;
Determining whether the identified processing order is the same as a predetermined processing order; And
If the identified processing order is not the same as the predetermined processing order,
And converting the perspective perspective image to be decoded. 12. The method of claim 11,
The processing sequence is identified as a processing start position and a processing progress direction,
Wherein the processing start position of the predetermined processing order is the upper left position of the perspective image and the processing progress direction of the predetermined processing order is clockwise. 13. The method of claim 12,
Wherein the transformation includes at least one of a symmetric transformation and a motion transformation on the perspective perspective image to be decoded. 14. The method of claim 13,
Wherein the symmetric transformation is performed when the process progress direction of the identified process sequence is different from the process progress direction of the predetermined process sequence. 14. The method of claim 13,
Wherein the symmetric transform is one of a left-right symmetry transform, a vertically symmetric transform, and a diagonal symmetric transform. 14. The method of claim 13,
Wherein the rotation conversion is performed when a processing start position of the identified processing order is different from a processing start position of the predetermined processing order. 17. The method of claim 16,
Wherein the rotation transformation is a transformation to rotate the decoding subject perspective image so that a processing start position of the decoding subject perspective image becomes equal to a processing start position of the predetermined processing order. 12. The method of claim 11,
Wherein the converting the perspective perspective image comprises:
A first transforming the perspective perspective image to be decoded;
Determining whether the processing order of the first converted subject perspective image is the same as the predetermined processing order; And
If the processing order of the first converted perspective perspective image is not the same as the predetermined processing order,
And a second transforming the first transformed perspective perspective image. 19. The method of claim 18,
Wherein the first transform is one of a symmetric transform and a motion transform, and the second transform is the other one. An apparatus for decoding an image including a plurality of perspective images,
Wherein the decoding apparatus includes a converter,
The converter comprising:
And a determination step of determining whether or not the identified processing order is the same as the predetermined processing order, and if the identified processing order is not the same as the predetermined processing order And to convert the perspective perspective image to be decoded. A computer-readable recording medium storing a bitstream generated by a method of encoding an image including a plurality of perspective images,
A method of encoding an image,
Identifying a processing order of a perspective image to be coded among the plurality of perspective images;
Determining whether the identified processing order is the same as a predetermined processing order; And
If the identified processing order is not the same as the predetermined processing order,
And converting the perspective image to be coded.

Images