KR20200007733A - Encoding/decoding method and apparatus for efficient compression of point cloud coding(pcc) data - Google Patents

Abstract

Disclosed are an encoding/decoding method of point cloud data and an apparatus thereof. The encoding/decoding method of point cloud data comprises: a step of grouping patches for one between depth information and texture information of point cloud data by using a projection plane and position information of the patches to generate prescribed groups; a step of performing packing on a patch included in at least one group among the generated prescribed groups; and a step of encoding information for the patch. The information for the patch includes at least one among information of a pixel map corresponding to the patch, information for a bounding box, and the packing information. The step of performing the packing includes a step of performing packing to divide the patch included in the at least one group among the generated prescribed groups into one between the same tile and the same slice in the same picture.

Description

ENCODING / DECODING METHOD AND APPARATUS FOR EFFICIENT COMPRESSION OF POINT CLOUD CODING (PCC) DATA}

The present invention relates to a method and apparatus for encoding / decoding point cluster data. More particularly, the present invention relates to an encoding / decoding method and apparatus for partially reconstructing point cluster data.

As the demand for immersive services for multimedia data increases, various applications for 3D display have been developed, and there is a need for a method of compressing and transmitting multimedia data suitable for such applications. MPEG has established compression standards for data acquired in stereo view and / or multiview for three-dimensional display.

An object of the present invention is to provide a method and apparatus for encoding / decoding point cluster data.

Moreover, an object of this invention is to provide the recording medium which stored the bitstream produced | generated by the encoding / decoding method or apparatus of this invention.

According to the present invention, generating a predetermined group by grouping a patch for at least one of the texture information and depth information of the point cloud data using at least one of the projection plane and the position information of the patch; Performing packing on a patch included in at least one of the generated predetermined groups; Encoding information on the patch, wherein the information on the patch includes at least one of information on a pixel map corresponding to the patch, information on a bounding box, and the packing information, and performs the packing. The method may include providing a method of encoding point cluster data including performing a packing such that a patch included in at least one of the generated predetermined groups is divided into one of the same slice and the same tile in the same picture. .

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present invention, a method and apparatus for encoding / decoding of point cluster data may be provided.

Further, according to the present invention, a recording medium storing a bitstream generated by the encoding / decoding method or apparatus of the present invention can be provided.

In addition, according to the present invention, it is possible to improve the encoding / decoding efficiency of the point cluster data.

Effects obtained in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a diagram illustrating an operation of a point cluster data encoding apparatus according to an embodiment of the present invention.
2 is a diagram illustrating an operation of a point cluster data decoding apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a method of projecting one frame data in various directions according to an embodiment of the present invention.
4 is a diagram illustrating a texture picture packed from the same frame, a depth image picture, and a pixel map of a picture according to an embodiment of the present invention.
5 is a flowchart illustrating a process of decoding a pixel map and patch reprojection information when the bounding boxes are overlapped and overlapped according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of decoding a pixel map and patch reprojection information when a bounding box is reprojected without overlapping according to an embodiment of the present invention.
7 is a flowchart illustrating a process of decoding only corresponding information when a decoder needs only some information of a 3D object according to an embodiment of the present invention.
8 is a flowchart illustrating a process of grouping and encoding patches by an encoder in order to effectively decode only some information of a 3D object in a decoder according to an embodiment of the present invention.
FIG. 9 is a flowchart illustrating an operation of a decoder when a patch is grouped and encoded by an encoder.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects. The shape and size of elements in the drawings may be exaggerated for clarity. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Described below, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

In the present invention, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When any component of the invention is said to be “connected” or “connected” to another component, it may be directly connected or connected to that other component, but other components may be present in between. It should be understood that it may. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The components shown in the embodiments of the present invention are shown independently to represent different characteristic functions, and do not mean that each component is made of separate hardware or one software component unit. In other words, each component is included in each component unit for convenience of description, and at least two components of each component may be combined into one component, or one component may be divided into a plurality of components to perform a function. Integrated and separate embodiments of the components are also included within the scope of the present invention without departing from the spirit of the invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present invention, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof. In other words, the description "include" a specific configuration in the present invention does not exclude a configuration other than the configuration, it means that additional configuration may be included in the scope of the technical idea of the present invention or the present invention.

Some of the components of the present invention are not essential components for performing essential functions in the present invention, but may be optional components for improving performance. The present invention can be implemented including only the components necessary to implement the essentials of the present invention, except for the components used for improving performance, and a structure including only essential components except for the optional components used for improving performance. Also included in the scope of the present invention.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, when it is determined that detailed descriptions of related well-known structures or functions may obscure the gist of the present specification, the detailed description is omitted, and like reference numerals denote the same elements in the drawings. Duplicate descriptions of the same components are omitted.

Hereinafter, terms used in the present invention will be described below.

Video Decoding Apparatus is a Personal Computer (PC), Notebook Computer, Portable Multimedia Player (PMP), Wireless Communication Terminal, Smart Phone, TV Application Server. It may be a device included in the server terminal, such as a service server. Also, the decoding apparatus may include a user terminal such as various devices, a communication device such as a communication modem for performing communication with a wired / wireless communication network, a memory for storing various programs and data for decoding an image or performing a rate control for decoding. It may refer to a variety of devices including a microprocessor for executing and operating a program.

In addition, the image encoded in the bitstream by the encoder is real-time or non-real-time through the wired or wireless communication network, such as the Internet, local area wireless communication network, wireless LAN network, WiBro network, mobile communication network, cable, universal serial bus (Universal Serial Bus) It can be transmitted to a video decoding apparatus through various communication interfaces such as Bus, USB, etc., decoded, reconstructed, and played back.

The input data of the encoder is information about a three-dimensional space, and the data may include three-dimensional coordinates, color information of corresponding coordinates, characteristic information, time information, and the like. In the present invention, point cloud data may be quantized based on time information, and a set of points for data mapped to the same time value in the quantization process may be defined as one frame. In addition, a set of a plurality of frames may be defined as a group of frames and a set of frame groups may be defined as a sequence. The frame, frame group and / or sequence may include syntax information required in the encoding / decoding step of the corresponding data unit.

Point cluster data defined in one frame may represent data in a three-dimensional space. According to the present invention, the encoder can project the point cluster data to a two-dimensional plane, encode the information on the projected two-dimensional data and the projection plane to be transmitted to the decoder. Also, the decoder can decode the 3D data by decoding and rendering the 2D data and the projection information.

1 is a diagram illustrating an operation of a point cluster data encoding apparatus according to an embodiment of the present invention.

The input data in the form of point clusters may be projected into two-dimensional image planes for encoding. Depending on the application, the image planes may be represented by N (N is a positive integer greater than 0), and information about the N projection planes may be fixed by an appointment defined by an encoder / decoding period. In this case, the encoder may not transmit information about the N projection planes to the decoder.

Meanwhile, the number and / or direction of the projection planes may be the same in one sequence, and when the number and / or direction of the projection planes is changed, the encoder may separately transmit the corresponding information to the decoder. Also, the number and / or directions of the projection planes may be transmitted in a sequence or frame group unit. In this case, each projection plane may have the same scale value in the 3D space and the 2D image mapping. In addition, the scale value may change according to the change in the number and / or the direction of the projection plane, and the change in the scale value is possible in the unit in which the number and / or the direction of the projection plane changes, that is, in the sequence or frame group unit. It can be sent via syntax in the unit.

The data projected onto the N planes may be represented by color information or texture information indicating a characteristic of a point, and may have depth information based on a projection plane according to an application. Texture information in each projection plane may be classified into M sets 110 according to spatial characteristics of data, and the classified information may be defined as a patch. On the other hand, if there is depth information as well as texture information, for example, a patch may have a patch for texture and a patch for depth information, and in another example, one texture patch may have one depth information patch and have the same irregular shape. May appear.

Data in which patches are defined in the N projection planes may be packed into a rectangular-shaped texture picture 140 and / or a depth information picture 130 for encoding (120). The depth information picture and / or texture picture generated through the packing process may be encoded using a picture encoding method (180). In this case, as the encoding method, a conventional image encoding method may be used. For example, depth information and texture information may be encoded using the same encoder, or may be encoded using a separate encoder. Information about the encoder may be defined by a predefined appointment in the encoder / decoding period, and information of the encoder may be transferred to the decoder if necessary. Meanwhile, in the case of using a separate encoder, the separate encoding period encoding method may not be the same, which is not illustrated in detail in FIG. 1. However, in the example of FIG. If you grow up, you will understand.

The encoder may encode 150 information about which position of each projection plane is packed in the packing process and transmit the encoded information to the decoder 150. In addition, since the atypical data is converted into a rectangle, information on whether data exists in each pixel is required for decoding, and the encoder may encode the pixel map information and transmit it to the decoder 150.

Patches projected into a two-dimensional space for encoding / decoding efficiency can be reprojected to other spaces in the same projection plane during packing. In this case, reprojection information of each patch needs to be transmitted from the encoder to the decoder. Encoding of the patch reprojection information may be added to the sequence header in the form of syntax and transmitted to the decoder. For example, when it is specified that the patch reprojection is encoded in the syntax, the encoder may encode the patch reprojection information together with each other and encode the patch reprojection information for each frame and transmit the same to the decoder.

2 is a diagram illustrating an operation of a point cluster data decoding apparatus according to an embodiment of the present invention.

The decoder may decode information encoding the texture picture and / or the depth information picture from the bitstream received from the encoder. If the information on the encoder is defined by the encoder / decoding period predefined appointment, the information can be extracted without decoding.

The decoder may separate 200 texture information and / or depth information data composed of pictures, pixel map, and patch reprojection information from the bitstream.

The decoder may decode 210 the text information and / or the depth information data composed of the pictures. In addition, the decoder may decode 220 packing information and pixel map information of the patch. In addition, the decoder may configure 240 the 3D structure information of each patch and restore 250 the 3D cluster data by mapping the decoded texture information 210 to the structure information.

On the other hand, if the patch is not reprojected, the corresponding information may not be transmitted in the bitstream. In this case, decoding of the patch reprojection information may be omitted (220). The presence or absence of patch reprojection information may be included in the form of syntax in the sequence header.

3 is a diagram illustrating a method of projecting one frame data in various directions according to an embodiment of the present invention.

3 (a) and 3 (b) show that the frame data is projected onto each plane of the cuboid. Although the number of planes is the same according to the position and / or size of the projection plane in the 3D data, the projection may be performed in various ways.

3 (c) shows that a projection is projected onto a total of 12 planes including two cuboids. The direction of each surface of the rectangular parallelepiped may be configured in parallel as in the example of FIG. 3C, but is not limited thereto and may be configured in various directions.

3 (d) shows the projection relationship between the position of the projection plane and the frame data. Referring to (d) of FIG. 3, the spatial relationship between the projection plane and the frame data may appear in various ways, the shape of the projection plane may not appear in a specific three-dimensional shape, and the data is projected in different directions with respect to the same plane. Can vary. In this case, the data projected in the same frame may be generated and packed in a separate patch according to the projection direction.

Meanwhile, information about the projection direction may be transmitted from the encoder to the decoder. As shown in (a) and (b) of FIG. 3, in the case of forming a three-dimensional plane of N planes, information on each plane may be separately encoded, or only information on a specific plane of a figure may be transmitted and the remaining planes may be encoded. Information about can be extracted from the decoder. As in the example of FIG. 3 (c), when two or more identical three-dimensional figures are parallel to each other and a projection plane is generated, information about one three-dimensional figure, size information of another three-dimensional figure, and a relationship between two three-dimensional figures By transmitting the information or the like, the projection plane information can be transmitted. As in the example of FIG. 3 (d), when the position and / or orientation relationship of each plane does not constitute a particular solid, information on the direction and / or spatial position of the plane may be transmitted for all planes. have.

4 illustrates a texture picture, a depth image picture, and a pixel map of a picture packed from the same frame according to an embodiment of the present invention.

Specifically, FIG. 4A illustrates a texture picture packed in the same frame, and FIG. 4B illustrates a depth image picture packed in the same frame. 4C and 4D are diagrams illustrating pixel maps of pictures packed in the same frame. FIG. 4C illustrates an example of overlapping re-projection of a bounding box having a minimum size of rectangle including each patch, and FIG. 4D illustrates an example of FIG. 4C during a packing process of the same frame. A pixel map newly obtained by using another reprojection method, which is an example of reprojecting the bounding box without overlapping.

As in the examples of FIGS. 4C and 4D, even when information of the same frame is projected onto the same plane, the shape and / or shape of the picture may appear different depending on how the patch is reprojected. .

5 is a flowchart illustrating a process of decoding a pixel map and patch reprojection information when the bounding boxes are overlapped and overlapped according to an embodiment of the present invention.

The decoder may parse the packing information and the patch reprojection information (500).

The decoder may decode the information to extract information about the projection plane and / or at which position in which projection plane each patch was reprojected (510).

The decoder may define a bounding box to decode the pixel map corresponding to the patch (520). In addition, the decoder can perform decoding only within the bounding box defined above. In this case, when the bounding boxes are overlapped and overlapped, pixel map data of N patches may exist in one bounding box. Since the decoder cannot determine which patch of the pixelmap data is the information on which patch, the pixelmap data may decode the information on which patch the pixelmap data is (530).

The decoder may decode the pixel map using the decoded information (540). In detail, the decoder may decode the pixel map by reflecting which patch reprojection information corresponds to the corresponding pixel map data.

The decoder may restore 3D position and structure information by using the decoded information (550). The restoration step 550 may correspond to the dimensional position and structure restoration step 240 of FIG. 2.

6 is a flowchart illustrating a process of decoding a pixel map and patch reprojection information when the bounding box is reprojected without overlapping according to an embodiment of the present invention.

The decoder may parse 600 packing information and patch reprojection information.

The decoder may decode the information to extract information about the projection plane and / or at which position in each projection plane each patch is 610.

The decoder may define a bounding box to decode the pixel map corresponding to the patch (620). In addition, the decoder can perform decoding only within the bounding box defined above.

The decoder may decode the pixel map (630).

The decoder may restore 3D position and structure information using the decoded pixelmap data and reprojection information of a patch (640). The restoration step 640 may correspond to the dimension location and structure restoration step 240 of FIG. 2.

7 is a flowchart illustrating a process of decoding only corresponding information when a decoder needs only some information of a 3D object according to an embodiment of the present invention.

The decoder may parse the packing information and the patch reprojection information (700). The decoder may decode the information to extract information about the projection plane and / or at which position in each projection plane each patch is 710.

The decoder may define a projection plane corresponding to the direction of the 3D object to be extracted using the information on the extracted projection plane (720).

The decoder may decode packing information, patch reprojection information, and / or pixelmap information of the defined projection plane (730).

The decoder may reconstruct the 3D structure information by decoding the depth information picture corresponding to the pixel map information (740). In addition, the decoder may decode the specific information of the 3D object by decoding the texture information corresponding to the structure information (740).

8 is a flowchart illustrating a process of grouping and encoding patches in an encoder in order to effectively decode only some information of a 3D object in a decoder according to an embodiment of the present invention.

The encoder may group the patches by reflecting the projection plane or the position information of the patch before packing the texture and depth information into the picture (800).

The encoder may perform packing so that patches in one group may be divided into the same slice, the same slice, or the same tile (810).

The encoder may encode texture information and / or depth information (820). The step may correspond to the encoding process of FIG. 1.

The encoder may additionally encode information of slices and / or tiles corresponding to the projection plane or the patch when encoding the patch map and / or the packing information (830). In addition, the encoder may transmit the encoded information to the decoder.

As an example, when the reference grouped by the encoder is a projection plane, the encoder may encode corresponding information of tiles or slices for each projection plane.

As another example, when the criterion grouped by the encoder is location information of a patch, the encoder may encode information of a tile or slice corresponding to each patch.

FIG. 9 is a flowchart illustrating an operation of a decoder when a patch is grouped and encoded by an encoder.

FIG. 9 may illustrate an operation of a decoder corresponding to the operation of the encoder in FIG. 8.

The decoder may parse the packing information and / or patch reprojection information (900).

The decoding may extract slice or tile information of the projection plane of each patch and the texture picture and / or the depth information picture corresponding to the projection plane (910). The decoder may extract the projection plane and / or patch reprojection information corresponding to the direction of the 3D object to be extracted using the extracted projection information (920).

The decoder may decode packing information, patch reprojection information, and / or pixelmap information of the projection plane (930).

The decoder may reconstruct the three-dimensional structure information by decoding the depth information picture corresponding to the projection plane (940). Also, the decoder may decode specific information of the 3D object by decoding the texture information corresponding to the structure information.

In the above-described embodiments, the methods are described based on a flowchart as a series of steps or units, but the present invention is not limited to the order of the steps, and any steps may occur in a different order or at the same time than the other steps described above. Can be. Also, one of ordinary skill in the art appreciates that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps in the flowcharts may be deleted without affecting the scope of the present invention. I can understand.

The above-described embodiments include examples of various aspects. Although not all possible combinations may be described to represent the various aspects, one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the present invention cover all other replacements, modifications and variations that fall within the scope of the following claims.

Embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and limited embodiments and drawings, it is provided to help a more general understanding of the present invention, but the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from such a description.

Therefore, the spirit of the present invention should not be limited to the embodiments described above, and all of the equivalents or equivalents of the claims, as well as the claims below, are included in the scope of the spirit of the present invention. I will say.

Claims (1)

Generating a predetermined group by grouping a patch for at least one of texture information and depth information of point cloud data using at least one of a projection plane and position information of the patch;
Packing a patch included in at least one of the generated predetermined groups;
Encoding information about the patch;
The information on the patch includes at least one of information on a pixel map corresponding to the patch, information on a bounding box, and the packing information.
The packing may include the step of performing packing such that a patch included in at least one of the generated predetermined groups is divided into one of the same slice and the same tile in the same picture. .

Images