KR20190050714A - A METHOD AND APPARATUS FOR ENCODING/DECODING 360 Virtual Reality VIDEO - Google Patents

Abstract

A method for encoding/decoding a 360-degree virtual reality (VR) image and an apparatus thereof are provided. The method for encoding an image comprises the steps of: dividing a 360-degree VR image into a plurality of regions based on a division structure of the 360-degree VR image; generating a region sequence by using the plurality of divided regions; generating a bit stream for the generated region sequence; and transmitting the generated bit stream, wherein the region sequence can include regions at the same position in at least one frame included in the 360-degree VR image.

Description

TECHNICAL FIELD [0001] The present invention relates to a 360 VR image encoding /

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to encoding and decoding of an interactive image, and more particularly, to encoding and decoding of an interactive image such as a 360 VR (Virtual Reality) image in which a reproduction region is changed according to a user's motion.

When an interactive image such as a 360 degree VR (Virtual Reality) is provided, the entire image is simply encoded and transmitted to the terminal. The terminal decodes the entire image, and thereafter a portion corresponding to a viewport Lt; / RTI > However, encoding and transmitting the whole image in this way transmits to the area not seen by the user at high image quality, which causes waste of network bandwidth.

Accordingly, methods are available that can reduce the transmission bit rate by transmitting only a portion of the 360 VR image that the user can view at a specific time.

In the 360 VR image, the reproduction area of the image must be changed according to the movement of the user. If there is no previous frame or surrounding area necessary for decoding the current viewport according to the characteristics of image encoding / decoding referring to the previous frame and the surrounding area, A problem that the area can not be decoded occurs.

Therefore, in order to avoid such a problem, there are conventional techniques of dividing an input image into a plurality of tiles and encoding each tile with an independent encoder.

Conventionally, in order to independently encode and decode a tile, a video encoder and a video decoder are required as many as the number of tiles. This causes an increase in the cost for the configuration of the encoder. In the case of the decoder, most terminals do not support the decoder as many as the number of tiles, so that there is a difficulty in the general-purpose service.

It is an object of the present invention to provide a tile-based 360 VR image coding method and a tile-based 360 VR image decoding method for coding a high-resolution 360 VR image based on a tile using a conventional video encoder and a video decoder without any modification .

According to another aspect of the present invention, there is provided a method for processing a 360 VR image, the method comprising: dividing the 360 VR image into a plurality of regions based on a division structure of a 360 VR (Virtual Reality) Generating an area sequence using the plurality of divided areas; Generating a bitstream for the generated region sequence; And transmitting the generated bitstream, wherein the region sequence includes regions of the same position in at least one frame included in the 360 VR image.

In the 360 VR image coding method according to the present invention, the area may be at least one of a tile and a subpicture.

In the 360 VR image coding method according to the present invention, the division structure of the 360 VR image is determined on a GOP (Group of Picture) basis, and the step of generating the bitstream comprises: And generating a bitstream for the sequence.

In the 360 VR image coding method according to the present invention, the step of generating the bitstream may include repeatedly generating a bitstream for all the region sequences included in the GOP.

In the 360 VR image coding method according to the present invention, the bitstream may include a first bitstream and a second bitstream generated from at least one region sequence included in the GOP, The second bitstream may have different image qualities.

In the 360 VR image coding method according to the present invention, the first bitstream may have higher image quality than the second bitstream.

In the 360 VR image coding method according to the present invention, the first bitstream is generated using a first video encoder, and the second bitstream is generated using a second video encoder different from the first video encoder .

In the 360 VR image coding method according to the present invention, the division structure of the 360 VR image may include at least one of the number, position, size information and frame rate of the area.

In the 360 VR image coding method according to the present invention, the frame rate may be determined such that a time for generating a bitstream for all area sequences included in a GOP is a time for generating a bitstream for all frames included in the GOP, Can be set to be the same.

According to another aspect of the present invention, there is provided a method of encoding a bitstream, the method comprising: receiving a bitstream encoded in an area sequence; Decoding the received bit stream to obtain a plurality of regions; And rendering the image to be reproduced based on the plurality of regions, wherein the region sequence may include regions of the same position in at least one frame included in a 360 VR (Virtual Reality) image.

In the 360 VR image decoding method according to the present invention, the area may be at least one of a tile and a sub-picture.

In the 360 VR image decoding method according to the present invention, the bitstream includes at least two bitstreams having different image qualities, and the viewport region includes a bitstream having a higher image quality than the region excluding the viewport region Lt; / RTI >

In the 360 VR image decoding method according to the present invention, the viewport area may be determined based on a first frame among frames included in a group of pictures (GOP).

In the 360 VR image decoding method according to the present invention, the viewport area may be updated in units of GOPs.

In the 360 VR image decoding method according to the present invention, at least two or more bit streams having different image qualities may be decoded by one video decoder.

In the 360 VR image decoding method according to the present invention, the step of rendering the image to be reproduced may include arranging the plurality of regions at the same position as that of inputting to the video encoder, have.

In the 360 VR image decoding method according to the present invention, the arrangement may be repeatedly performed until all the area sequences included in the GOP are arranged.

In the 360 VR image decoding method according to the present invention, when the viewport area is changed, a bit having a higher image quality than the remaining area excluding the viewport area, based on at least one of the changed position and the GOP information, A stream may be received.

The 360 VR image decoding method according to the present invention is characterized in that the plurality of regions are divided from the 360 VR image based on the division structure of the 360 VR image, Location information, size information, and frame rate.

In the 360 VR image decoding method according to the present invention, the frame rate may be determined such that a time for generating a bitstream for all area sequences included in a GOP is a time for generating a bitstream for all frames included in the GOP, Can be set to be the same.

The 360 VR image coding method and the 360 VR image decoding method according to the embodiments of the present invention can encode and decode a 360 VR image based on a tile or a sub picture without using a plurality of video encoders or video decoders .

Also, the 360 VR image coding method and the 360 VR image decoding method according to the embodiments of the present invention can be applied regardless of a conventional image coding method such as H.264 and HEVC (High Efficiency Video Coding).

Further, in the tile-based 360 VR image coding method and the tile-based 360 VR image decoding method according to the embodiments of the present invention, since each tile is coded without spatial correlation, only a part of tiles are transmitted And it is possible to provide smooth rendering with only two video encoders using a low-quality bitstream and a high-quality bitstream. In particular, the number of video encoders does not change even if a large number of clients are connected, and it can be applied irrespective of the type of codec.

In addition, since the 360 VR image coding method and the 360 VR image decoding method of the present invention can embody a coding function in a graphic card and can perform high-speed image coding and decoding even in a personal computer (PC) Area of the image.

1 is a conceptual diagram for explaining a tile-based 360 VR image encoding and decoding process in a 360 VR system according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram for explaining a tile-based 360 VR image coding process according to an embodiment of the present invention in more detail.
FIG. 3 is a conceptual diagram for explaining a tile-based 360 VR image decoding process according to an embodiment of the present invention in more detail.
4 is a conceptual diagram for explaining a tile-based 360 VR image encoding process using two video encoders according to another embodiment of the present invention.
5 is a conceptual diagram for explaining a tile-based 360 VR image decoding process using two encoders according to another embodiment of the present invention.
FIG. 6 is a view showing a viewport area in the frames F0, F1, ..., F29, F30, F31, ..., F59 according to the movement of the user's head or eyes during the tile-based 360 VR image decoding process of FIG. (512).
7 is a flowchart illustrating a tile-based 360 VR image encoding process in a 360 VR system according to an embodiment of the present invention.
FIG. 8 is a flowchart for explaining a tile-based 360 VR image decoding process in a 360 VR system according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms including the first, second, etc. may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The term " and / or " includes any combination of a plurality of related entry items or any of a plurality of related entry items.

When an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may be present in between. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, Should not be construed to preclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The description will be omitted.

A viewport can be defined as a portion of a spherical image currently displayed that is viewed by a user as an area viewed by a user throughout the image.

A method of dividing a 360 VR image into a plurality of regions and generating and parsing a bitstream for each unit region will be described in the following embodiments. Here, the division unit of the 360 VR image projected on the 2D plane may be a subpicture, a tile, or the like. The divided regions may have an equal size, or may have different sizes. As an example, the size of any one of the divided regions may have a different size from the other regions. Here, the size may be at least one of a width, a length, a diagonal length of the area, and a length at a predetermined position of the area.

In the present invention, a set of spatial regions at the same position in each of the frames may be defined as a " set " or " sequence ". In one example, the region set or region sequence may refer to a set of spatial regions in the same location within a plurality of frames.

In the following embodiment, it is assumed that the 360 VR image is divided into tiles. In addition, it is assumed that each tile has the same size. However, it is obvious that the embodiment described below can be applied to the case where the division unit of the 360-degree VR image is a sub-picture or when the size of each tile is not uniform.

1 is a conceptual diagram for explaining a coding and decoding process of a tile-based 360 VR image in a 360 VR (Virtual Reality) system according to an embodiment of the present invention.

1, the 360 VR system according to an exemplary embodiment of the present invention includes a 360 VR server 100a and a 360 VR terminal 100b. The 360 VR server 100a includes an Input Manager 10 and a video encoder 20. The 360 VR terminal 100b includes a video decoder 30 and an output manager 40. [

When the 360 VR image 11a is input to the 360 VR server 100a, the input manager 10 of the 360 VR server 100a can divide the input 360 VR image 11a spatially into a plurality of regions. For example, the input manager 10 may divide a tile of a 360 degree VR image 11a into a plurality of tiles and transmit at least one tile 13a to the video encoder 20 at a high speed / sequentially. The video encoder 20 receives the generated at least one tile 13a and generates a bit stream 21 in units of a tile set.

The video decoder 30 receives a bitstream of sets of tiles necessary for rendering the viewport from the server, decodes the bitstreams of the received sets of tiles, and outputs the decoded tiles to the output manager 40 (13b). The output manager 40 allows the decoded tiles 13b to be arranged and rendered so as to constitute a viewport of a 360 VR image.

FIG. 2 is a conceptual diagram for explaining a tile-based 360 VR image coding process according to an embodiment of the present invention in more detail.

Assuming that a GOP (Group of Pictures) to be set when coding an image is 30 and one frame is divided into 32 spatial regions, that is, 32 tiles, A tile-based 360 VR image encoding and decoding process according to embodiments will be described. According to the encoding implementation, the GOP may have a value other than 30, and one frame may have a number of tiles other than 32. [

Referring to FIG. 2, the first GOP includes 30 frames from frame 0 to frame 29, and the second GOP includes 30 frames from frame 30 to frame 59. Frames included in each GOP may have the same division structure. Here, the divided structure may include at least one of the number of divided areas, the position of the divided area, or the size of the divided area. The division structure of the frames may be set differently for each GOP. In one example, if the second GOP has a different partition structure than the first GOP, information on the updated partition structure for the second GOP may be coded.

In the example shown in FIG. 2, frames F0, F1, F2, ..., F29 of the first GOP and frames F30, F31, ... F59 of the second GOP are T0, T1, ..., T31 Lt; RTI ID = 0.0 > 32 < / RTI > tiles. Unlike the illustrated example, the number of tiles included in each of the frames included in the first GOP and the frames included in the second GOP may be set differently. Alternatively, the number of tiles is the same, but the position or size of the tiles may be set differently.

Referring again to FIG. 2, the 360 VR image 11a is sequentially input to the input manager 10 in the order of frames F0, F1, ..., F29, F30, ..., F60, .

When coding an image, a GOP is set, and the input manager 10 can buffer as many images as GOP. The input manager 10 divides the frames (F0, F1, ..., F29, F30, ..., F60, ...) included in each GOP into tiles as shown in 220 of FIG. 2, And transmits the tile 13a to the video encoder 20 in sequence. Each of the tiles can be independently encoded. As an example, motion constraints between tiles may be applied so that encoding parameters may not have dependencies between tiles.

The input manager 10 sequentially transmits tiles (hereinafter referred to as a tile set or a tile sequence) at the same position from the first frame of the GOP to the last frame of the GOP to the video encoder 20 Can be input. In addition, the video encoder 20 can generate a bit stream in units of tiles, instead of generating a bit stream in units of frames. This process is repeated until all the tile sets in the GOP are input, and a bit stream can be generated by the number of tile sets. The tile set input process in the GOP is repeated in units of GOP.

In the above process, the video encoder 20 can be set such that the processing time (n sec) for encoding in the frame unit and the processing time (n sec) for encoding in the tile set unit are the same for the frames in the GOP. That is, the input manager 10 sequentially sets the tile set to the video encoder (step 230) so that the time (n sec) at which all the tiles in the GOP are processed (see 230 in FIG. 2) 20).

The size of the input image 13a of the video encoder 20 is set to the size of the tile and the number of frames of the input image 13a of the video encoder 20 The rate can be set to the total number of tiles in the GOP (i.e., (frame rate of 360 VR image) x (number of tiles)). For example, in the example of FIG. 2, when the 360 VR image has a size of 3840 x 2160 and a frame rate of 30 fps, the image size-related encoding parameter of the video encoder 20 is 480 x 540 (i.e., (The frame rate of the input image 13a of the video encoder 20) has 960 fps, which is the total number of tiles in the GOP, . In general, as the size of an image becomes smaller, a coding speed becomes faster in proportion thereto, and thus a high-speed processing of a tile is possible. The frame rate of the input image may be included in the above-described divided structure.

FIG. 3 is a conceptual diagram for explaining a tile-based 360 VR image decoding process according to an embodiment of the present invention in more detail.

Referring to FIG. 3, the video decoder 30 receives a bit stream of tile sets necessary for constructing the viewport from the 360 VR server, and decodes the bit streams of the received tile sets in the reverse order of the above-described encoding . Specifically, the video decoder 30 can receive and decode only the bitstream of the tile sets necessary for constructing the viewport among the bitstreams of the tile sets.

In the 0th frame F0, the viewport of the rectangle 312 is shown to exist over the tiles T1, T2, T3, T4, T9, T10, T11 and T12.

Accordingly, the video decoder 30 sequentially receives and decodes a set of tiles for the tiles T1, T2, T3, T4, T9, T10, T11, and T12 corresponding to the viewport. The set of tiles may include the same location tiles in all frames in the GOP (i.e., from the first frame F0 to the last frame F29). That is, the set of tiles used to decode the viewport of the first frame may include tiles T1, T2, T3, T4, T9, T10, T11 and T12 of all frames in the GOP. The rectangular dotted lines indicated by 320 in FIG. 3 are decoded by the video decoder 30 and set as a tile corresponding to one GOP-tiles T1, T2, T3, T4, T9, T10, and T11 included in the viewport area of the first frame And T12 - are output for n sec at the decoder.

That is, as described above, the video decoder 30 receives and decodes the set of tiles corresponding to the viewport, and sequentially delivers the decoded tiles 13b to the output manager 40. [

The output manager 40 reconstructs the decoded tiles 13b into a 360 VR image so as to render the decoded tiles 13b. In order to reconstruct the 360 VR image, it is necessary to know the position of each tile in the 360 VR image. For example, the position of each tile in the 360 VR image is determined by using the spatialdata description (SRD) of the MPEG DASH standard .

In the tile-based 360 VR image coding and decoding method according to an embodiment of the present invention, a 360 VR image service can be provided using only one video encoder and one video decoder.

If the viewport is not changed while the GOP is being played back, it is possible to improve system efficiency by selectively receiving a bit stream of tile sets based on the viewport of the first frame of the GOP, as in the example shown in Fig. However, according to the method shown in FIG. 3, when the viewport is changed by the user's head or eye movement during GOP playback, the area to be newly included to constitute the viewport as the viewport is changed corresponds to the changed viewport There may be a problem that the changed viewport can not be rendered completely because the tiles corresponding to the region - tiles to be rendered are not being received. That is, since the area corresponding to the changed viewport can be decoded at the start of the next GOP, if the viewport is changed while the GOP is being reproduced, the 360 VR image may be interrupted during playback, which may cause inconvenience to 360 VR video viewing.

That is, when the viewport is out of the area of tiles T1, T2, T3, T4, T9, T10, T11, and T12 in the first GOP as indicated by 312 in FIG. 3, tiles or tiles are required but new tiles or tiles can not be decoded until the next GOP.

Accordingly, in another embodiment of the present invention to provide a 360 VR video service, 360 VR video playback can be implemented to provide a smooth 360 VR video service by extending to use a plurality of video encoders.

For convenience of explanation, it is assumed that the number of encoders is two in the embodiment described later. It is also within the scope of the present invention to use more than two encoders.

4 is a conceptual diagram for explaining a tile-based 360 VR image encoding process using two video encoders according to another embodiment of the present invention.

Referring to FIG. 4, the 360 VR system according to another embodiment of the present invention includes a 360 VR server 400a and a 360 VR terminal 400b. The 360 VR server 400a includes an input manager 410, A first video encoder 420a and a second video encoder 420b and the 360 VR terminal 400b includes a video decoder 430 and an output manager 440. [

The input manager 410 and the video encoders 420a and 420b operate as a tile-based fast encoding method as described above.

More specifically, when the 360 VR image 401a is input to the 360 VR server 400a, the input manager 410 of the 360 VR server 400a divides the inputted 360 VR image 401a into a plurality of tiles, And transfers one tile 413a to the first video encoder 420a and the second video encoder 420b at high speed / sequentially. The video encoders 420a and 420b receive the generated at least one tile 413a to generate bit streams 421a and 421b of tiles.

The first video encoder 420a and the second video encoder 420b can encode the same video source with different quality. Specifically, the first video encoder 420a encodes the tiles in a high image quality to generate a high-quality bit stream, and the second video encoder 420b encodes the tiles in a low image quality to generate a low- .

Video decoder 430 may request the server of a bitstream of sets of tiles needed to render the viewport. At this time, the video decoder 430 receives and decodes the bit stream 421a of the high-quality tile sets, and the area not corresponding to the viewport receives and decodes the bit stream 421b of the low-quality tile sets .

The video decoder 430 decodes the received tile stream and delivers the decoded tiles 413b to the output manager 440. The output manager 440 decodes the decoded tiles 413b to 360 VR So that the image can be arranged and rendered to construct a viewport of the image.

Therefore, if the viewport is out of the high-quality encoded tile area by the user's head or line of sight movement, the 360 VR image can be rendered based on the decoded result of the low-quality tile set bitstream corresponding to the changed viewport. In addition, when the next GOP starts, a high-quality tile set bit stream to be received is re-selected based on the changed viewport, thereby providing a smooth 360 VR video service.

The high-quality tile set bitstream and the low-quality tile set bitstream may be processed by one video decoder or may be implemented by two video decoders.

5 is a conceptual diagram for explaining a tile-based 360 VR image decoding process using two encoders according to another embodiment of the present invention.

Referring to FIG. 5, the first GOP includes 30 frames from frame 0 to frame 29, and the second GOP includes 30 frames from frame 30 to frame 59. Frames included in each GOP may have the same division structure. In the example shown in FIG. 5, frames F0, F1, F2, ..., F29 of the first GOP and frames F30, F31, ... F59 of the second GOP are T0, T1, ..., T31 Lt; RTI ID = 0.0 > 32 < / RTI > tiles. Unlike the illustrated example, the number of tiles included in each of the frames included in the first GOP and the frames included in the second GOP may be set differently. Alternatively, the number of tiles is the same, but the position or size of the tiles may be set differently.

In decoding the first GOP, the decoder 430 can receive and decode the high-quality tile set bit stream for the areas 512-0 to 512-29 corresponding to the viewport of the first frame F0. Specifically, the decoder can decode high-quality tile sets of T1, T2, T3, T4, T9, T10, T11 and T12. The decoder 430 can receive and decode the low-quality tile set bit stream for the remaining area excluding the viewport corresponding area of F0.

Meanwhile, in decoding the second GOP, the decoder 430 may receive and decode the high-quality tile set bit stream for the regions (512-30 to 512-59) corresponding to the viewport of the first frame F30. Specifically, the decoder 430 can decode high-quality tile sets of T4, T5, T6, T7, T12, T13, T14 and T15. The decoder 430 can receive and decode the low quality tile set bit stream for the remaining area except for the viewport corresponding area of F30.

The high-quality tile portion in the tiles 413b decoded by the video decoder 430 is represented by 512, corresponding to the high-quality tile region for each frame of the GOP described above.

The video decoder 430 must process both low-quality tiles and high-quality tiles in the GOP in n seconds as shown in FIG.

FIG. 6 is a view illustrating a viewport area (in FIG. 5) in the frames F0, F1, ..., F29, F30, F31, ..., F59 according to the movement of the user's head or eyes during the tile- 512). For example, in FIG. 6, the viewport area 512-0 in the frame F0, the viewport area 512-1 in the frame F1, ..., the viewport area 512-29 in the frame F29, The viewport area 512-30 of the frame F31, ..., and the viewport area 512-59 of the frame F59.

7 is a flowchart illustrating a tile-based 360 VR image encoding process in a 360 VR system according to an embodiment of the present invention.

Referring to FIG. 7, a tile-based 360 VR image coding method according to an embodiment of the present invention divides the 360 VR image into a plurality of regions based on a division structure of a 360 VR (Virtual Reality) A step S710 of generating a region sequence using the divided regions, a step S720 of generating a region sequence using the divided regions, a step S730 of generating a bitstream for the generated region sequence, and / (S740).

The region sequence may include regions of the same position in at least one frame included in the 360 VR image.

The area may be at least one of a tile and a subpicture.

The division structure of the 360 VR image may be determined on a GOP (Group of Picture) basis.

In operation S730, the bitstream for the generated region sequence may be generated by generating a bitstream for at least one region sequence included in the GOP. In addition, the step of generating a bitstream for the generated region sequence (S730) may include repeatedly generating a bitstream for all the region sequences included in the GOP.

The bitstream may include a first bitstream and a second bitstream generated from at least one region sequence included in the GOP, and the first bitstream and the second bitstream may have different image qualities. For example, the first bitstream may have a higher image quality than the second bitstream. That is, the first bitstream may be a high-quality bitstream and the second bitstream may be a low-quality bitstream.

The first bitstream may be generated using a first video encoder and the second bitstream may be generated using a second video encoder different from the first video encoder.

The division structure of the 360 VR image may include at least one of the number, position, size information and frame rate of the area.

The frame rate may be set so that a time for generating a bitstream for all the area sequences included in the GOP is equal to a time for generating a bitstream for all frames included in the GOP.

FIG. 8 is a flowchart for explaining a tile-based 360 VR image decoding process in a 360 VR system according to an embodiment of the present invention.

Referring to FIG. 8, a tile-based 360 VR image decoding method according to an embodiment of the present invention includes a step of receiving a bitstream encoded in an area sequence unit (S810), decoding the received bitstream (S820) of obtaining a plurality of regions and / or rendering (S830) an image to be reproduced based on the plurality of regions.

The region sequence may include regions of the same position in at least one frame included in 360 Virtual Reality (VR) images.

The area may be at least one of a tile and a subpicture.

The bitstream may include at least two bitstreams having different image qualities and the viewport region may receive a bitstream having a higher image quality than the region other than the viewport region. That is, the viewport area may be a high-quality bitstream, and the remaining area may be a low-quality bitstream.

The viewport area may be determined based on the first frame among the frames included in the GOP (Group of Pictures).

The viewport area may be updated in units of GOPs.

At least two bit streams having different image qualities may be decoded by one video decoder. Alternatively, the high-quality bit stream and the low-quality bit stream can be processed by a plurality of video decoders.

The step of rendering the image to be reproduced based on the plurality of regions may include arranging the plurality of regions in the same position as that of inputting to the video encoder in the region sequence unit.

The above arrangement can be repeatedly performed until all the region sequences included in the GOP are arranged.

When the viewport area is changed, a bitstream having a higher image quality than the rest of the area other than the viewport area may be received for the modified viewport area based on at least one of the changed position and the GOP information.

Wherein the plurality of regions are divided from the 360 VR image based on a division structure of the 360 VR image, and the division structure of the 360 VR image includes at least one of the number, position, size information, and frame rate of the region can do.

The frame rate may be set so that a time for generating a bitstream for all the area sequences included in the GOP is equal to a time for generating a bitstream for all frames included in the GOP.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to be illustrative of the typical aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (20)

Dividing the 360 VR image into a plurality of regions based on a division structure of 360 Virtual Reality (VR) images;
Generating an area sequence using the plurality of divided areas;
Generating a bitstream for the generated region sequence; And
And transmitting the generated bitstream,
Wherein the region sequence includes regions of the same position in at least one frame included in the 360 VR image. The method according to claim 1,
Wherein the region is at least one of a tile and a subpicture. The method according to claim 1,
The division structure of the 360 VR image is determined on a GOP (Group of Picture) basis,
Wherein generating the bitstream comprises:
And generating a bitstream for at least one region sequence included in the GOP. The method of claim 3,
Wherein generating the bitstream comprises:
And repeatedly generating a bitstream for all the region sequences included in the GOP. 5. The method of claim 4,
The bitstream may include:
A first bitstream and a second bitstream generated from at least one region sequence included in the GOP,
Wherein the first bitstream and the second bitstream have different image qualities. 6. The method of claim 5,
Wherein the first bitstream is higher in image quality than the second bitstream. 6. The method of claim 5,
Wherein the first bitstream is generated using a first video encoder,
Wherein the second bitstream is generated using a second video encoder different from the first video encoder. The method according to claim 1,
Wherein the division structure of the 360 VR image includes at least one of the number, position, size information, and frame rate of the area. 9. The method of claim 8,
The frame rate
Wherein a time for generating a bitstream for all the region sequences included in the GOP is set to be equal to a time for generating a bitstream for all frames included in the GOP. Receiving a bitstream encoded in an area sequence unit;
Decoding the received bit stream to obtain a plurality of regions; And
And rendering the image to be reproduced based on the plurality of regions,
Wherein the region sequence includes regions of the same position in at least one frame included in a 360 VR (Virtual Reality) image. 11. The method of claim 10,
Wherein the area is at least one of a tile and a subpicture. 11. The method of claim 10,
The bitstream may include:
And at least two bit streams having different image qualities,
And a viewport area receives a bitstream having a higher image quality than the other areas excluding the viewport area. 13. The method of claim 12,
Wherein the viewport area is determined based on a first one of frames included in a group of pictures (GOP). 13. The method of claim 12,
Wherein the viewport area is updated in units of GOPs. 13. The method of claim 12,
Wherein at least two bitstreams having different image qualities are decoded by one video decoder. 11. The method of claim 10,
Wherein the rendering of the image to be reproduced comprises:
And arranging the plurality of regions at the same position as that of inputting to the video encoder by arranging the plurality of regions in the region sequence unit. 17. The method of claim 16,
Preferably,
And repeats the process until all the region sequences included in the GOP are arranged. 13. The method of claim 12,
Receiving a bitstream having a higher image quality than the remaining area excluding the viewport area for the modified viewport area based on at least one of the changed position and the GOP information when the viewport area is changed, Decoding method. 11. The method of claim 10,
Wherein the plurality of regions are divided from the 360 VR image based on the division structure of the 360 VR image,
Wherein the division structure of the 360 VR image includes at least one of the number, position, size information, and frame rate of the area. 20. The method of claim 19,
The frame rate
Wherein a time for generating a bitstream for all region sequences included in a GOP is set to be equal to a time for generating a bitstream for all frames included in the GOP.

Images