JP2017126890A - Encoder and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To encode a point group of an N dimensional space with high compression ratio.SOLUTION: An encoder has a setting section for setting an initial block including a point group of encoding object in an N dimensional space, a tree structure generating section for generating the data of a tree structure where the initial block is the uppermost layer, by dividing the initial block into multiple blocks, and repeating recursive division for each division block obtained by dividing, a tree structure encoding section for encoding the data of a tree structure, an intra-block encoding section for encoding the position of a point in a block of interest of tree structure, in units of the space in the block of interest, a tree structure encoding section, and an output section for outputting the encoded data, generated in the intra-block encoding section, as the encoded data of the coordinate information of a point group.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for encoding a point group in an N-dimensional space.

  Conventionally, a 3D point cloud coding technique using an octree is known. First, a method for encoding 3D point cloud position information (xyz coordinates of each point) disclosed in Non-Patent Document 1 will be described. In Non-Patent Document 1, a 3D point group is encoded with octree data and block point group data. In the generation of octree data, first, a rectangular parallelepiped (hereinafter referred to as a block) containing a 3D point cloud is defined, and the three sides of the x-, y-, and z-axes of the block are each divided into two and divided into eight blocks. . When attention is paid to one block after division, if there is a point in the target block, block division of the target block is recursively repeated. This recursive block division stops at a predetermined hierarchy. The octree data indicating whether or not each block has a point is output as a binary flag. Based on the octree data, a region with a point is divided into blocks of the lowest (fine) hierarchy, and information on whether or not a point exists in each block of the lowest hierarchy can be determined. Then, as the point cloud data in the block, the number of points in each block and the coordinate data in the block indicating where each point in the block is in the block are output. Thereby, the coordinates of the 3D point group are encoded.

  In the method of Non-Patent Document 1, all block division is terminated at a fixed hierarchy. On the other hand, Patent Document 1 is cited as a document that discloses a technique for adaptively canceling block division in encoding of three-dimensional volume data. In this patent document 1, the improvement of the compression rate of three-dimensional volume data is achieved by making the block division abortion an adaptive hierarchy.

Japanese Patent No. 4759291

Real-time Compression of Point Cloud Streams, 2012 IEEE International Conference on Robotics and Automation RiverCentre, Saint Paul, Minnesota, USA, May 14-18, 2012

  The fixed layer block division described in Non-Patent Document 1 has a problem that the compression rate is lower than that of the adaptive layer block division described in Patent Document 1. The reason is as follows. When encoding a 3D point cloud using octree data and point group data in a block, when the number of points in the block is large, if block division is performed, the coordinate data in the block per point becomes small, so the data amount can be reduced. . However, when the number of points in the block is small, the data amount of the octree data indicating the presence or absence of block division becomes larger. Therefore, from the viewpoint of the compression rate, since the optimal truncation layer differs for each block, the compression rate is low in the fixed layer division.

  On the other hand, in the case of a 3D point group output by adaptive hierarchical block division, there is a problem that the density of points at the time of progressive decoding for decoding the data of the upper hierarchy of the octree data differs for each region. In the octree data of the fixed hierarchy block division, the data of the same hierarchy can be decoded for all the blocks having points in the hierarchy above the fixed hierarchy. Therefore, at the time of progressive decoding, if rendering is performed such that one representative point (such as the center of the block) is displayed for a block including points, the points can be rendered at equal intervals. On the other hand, in the progressive decoding of adaptive hierarchical block division, since the block sizes are different, one representative point is displayed for a wide area in a large block, and the density of the points varies depending on the area.

  The object of the present invention is to solve the above problems of the prior art. The present invention provides a technique for encoding a point group in an N-dimensional space at a high compression rate using encoding using a tree structure.

In order to solve this problem, for example, the encoding apparatus of the present invention has the following configuration. That is,
An encoding device for encoding coordinate information of a point group in an N-dimensional coordinate space,
A setting means for setting the depth of the reference hierarchy;
Tree structure generating means for generating a tree structure block for dividing the N-dimensional space;
Tree structure encoding means for encoding the data of the tree structure;
Intra-block encoding means for encoding the position of the point in the target block of the tree structure in units of the space in the target block;
An output means for outputting the information including the encoded data generated by the tree structure encoding means and the intra-block encoding means as encoded data of the coordinate information of the point group;
The tree structure generated by the tree structure generation means is (1) blocks above the reference hierarchy are always divided as long as 3D points are included in the block.
(2) The block below the reference hierarchy is adaptively switched as to whether or not to divide.

  According to the present invention, it is possible to encode a point group in an N-dimensional space with a high compression rate.

The block diagram of the encoding apparatus which outputs the code data of 1st Embodiment. The flowchart of the encoding apparatus which outputs the code | symbol data of 1st Embodiment. The figure explaining the process of 1st Embodiment. The figure which shows the structure of the code | cord | chord data output in 1st Embodiment. The figure explaining the example of the tree structure produced | generated in 1st Embodiment. The block block diagram of the decoding apparatus which decodes the code data of 1st Embodiment. The flowchart of the decoding apparatus which decodes the code data of 1st Embodiment. The figure which shows the structure of the code | cord | chord data output in 2nd Embodiment. The flowchart of the encoding apparatus which outputs the code data of 2nd Embodiment. The block block diagram of the decoding apparatus which decodes the code data of 2nd Embodiment. The flowchart of the decoding apparatus which decodes the code data of 2nd Embodiment. The flowchart of the encoding apparatus which outputs the code data of 3rd Embodiment. The figure explaining the overdivision tree structure of 3rd Embodiment. The block block diagram of a computer.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In addition, each embodiment described below shows an example in which the present invention is specifically implemented, and is one of the specific embodiments having the configurations described in the claims.

[First Embodiment]
In the first embodiment, a 3D point group encoding apparatus that generates tree structure data satisfying the following conditions 1 and 2 and uses the tree structure data will be described.
Condition 1: A block above the reference layer is always divided as long as 3D points are included in the block.
Condition 2: Whether or not to divide a block below the reference layer is adaptively switched.

  In order to generate tree structure data that satisfies the above conditions, in this embodiment, an initial block including all 3D point groups to be encoded is generated as an initial block. Next, as long as points are included in the target block, block division is always performed up to the reference layer. And in the hierarchy below a reference | standard hierarchy, whether to perform block division is switched adaptively. Through the above processing, a tree structure (an octree structure in the case of 3D space) that satisfies condition 1 and condition 2 is generated.

  FIG. 1 is a block diagram of a coding apparatus according to the embodiment. In the following, the processing content of each unit constituting the apparatus according to the present embodiment will be described with reference to FIG. 3D point cloud data is input to the encoding device. The 3D point cloud data referred to here is an enumeration of points each having three coordinates of x, y, and z. Each coordinate is expressed as a floating-point or fixed-point real number. Each point may have color information (RGB) and normal vector information as additional information in addition to the x, y, and z coordinate values. In this embodiment, in order to simplify the explanation, each point is a coordinate value of x, y, z, and is expressed by a fixed point, and has no additional information other than coordinates. Will be described.

  The setting unit 101 sets a reference hierarchy as a parameter. The reference hierarchy may be set by the user, or once set, it may be held in a non-volatile memory (such as an HDD) for use as the next default value unless otherwise changed. The initial block generation unit 102 generates a cube that contains all the points of the input 3D point group as an initial block (block of the highest layer). In the embodiment, an example in which a cube is an initial block will be described, but the initial block may be a rectangular parallelepiped.

  The switching unit 103 determines whether or not to divide the target block to be processed. In the present embodiment, the switching unit 103 always divides the block of interest when at least one 3D point is included in the block of interest and the block of interest is a block above the reference hierarchy. Then, when at least one 3D point is included in the block of interest and the block of interest is located in a layer below the reference hierarchy, the switching unit 103 adaptively determines whether or not to divide. Note that when the 3D point is not included in the block of interest, the switching unit 103 does not divide the block regardless of the layer in which the block of interest is located. When the switching unit 103 determines that the block of interest is to be divided, the dividing unit 104 divides the block of interest into eight. If the switching unit 103 determines not to perform block division, the intra-block coordinate encoding unit 105 calculates difference coordinates from the representative coordinates in the block for all points in the block of interest, Turn into.

Tree structure encoding unit 106 encodes tree structure data indicating the attribute of each block. Each block has one of the following three attributes.
(1) The block of interest does not include any 3D points, and no division is performed (no points).
(2) The block of interest contains at least one 3D point and is not divided (no division with points)
(3) The block of interest includes at least one 3D point, and division is performed (with division with points).
When the attribute of the block of interest is “no division with points”, intra-block encoding by the intra-block coordinate encoding unit 105 is performed.

  The code data output unit 107 outputs data obtained by encoding the depth of the reference layer, the tree structure, and the intra-block difference coordinates. Here, the initial block generation unit 102, the switching unit 103, and the division unit 104 can be combined as a tree structure generation unit 108. The role of the tree structure generation unit 108 is based on the reference hierarchy set by the setting unit 101. When one 3D point is included in a block above the reference hierarchy, the block is divided, and blocks below the reference hierarchy are Is to adaptively switch between dividing and not dividing. Therefore, the detailed block configuration of the tree structure generation unit 108 may take a configuration other than the processing blocks 102, 103, and 104 described above. This is the end of the description of each unit of the encoding device in FIG.

  Next, according to the flowchart of FIG. 2, details of processing of each unit included in the apparatus of FIG. 1 will be described. In the following, “S” means a step in the flowchart. First, the encoding apparatus inputs point group data to be encoded having x, y, and z coordinates as shown in FIG. In addition, the kind of input source is not ask | required. In S201, setting unit 101 sets a reference hierarchy. The reference hierarchy is set according to the input by the user as a parameter. The reference hierarchy may be set by any other method, a fixed parameter may be included in the program in advance, or the reference hierarchy may be determined based on the total number of input point groups. Next, in S202, the initial block generation unit 102 generates an initial block. As shown in the example of FIG. 3B, the initial block generates a cubic block containing all the input 3D point groups. From the viewpoint of encoding efficiency, it is desirable that the cube generated here has a minimum volume including all 3D points.

  In S203, the switching unit 103 determines whether the block of interest is located in a layer above the reference hierarchy. If the block is located in a hierarchy above the reference hierarchy, the switching unit 103 determines to perform block division processing. When it is determined that the block is to be divided, the dividing unit 104 performs the process of dividing the block of interest. If the block of interest does not include any 3D points, the dividing unit 104 skips the dividing process. That is, the dividing unit 104 performs division processing when the block of interest includes at least one 3D point. FIG. 3C shows an example in which the block of FIG. As shown here, block division is performed so that the length of each side of the x, y, and z axes of the block of interest is divided into two. By repeating S203 and S204, the block containing 3D points is divided up to the reference layer. Thereafter, the process proceeds to S205. Note that the division of the following description is for the hierarchy below the reference hierarchy.

  In S205, the switching unit 103 determines whether to divide the block of interest. It is desirable that the switching unit 103 determines whether or not to perform block division according to the viewpoint of making the point cloud highly compressed. As already described, when the number of points in the block is large, the code amount is smaller when the division is performed, and when the number of points is small, the code amount is smaller when the division is not performed. Therefore, in this embodiment, a predetermined threshold value n is set in advance, and is determined to be divided when the number of 3D points included in the block of interest is n or more, and is not divided when the number is less than n. I decided to judge.

  Note that the criteria for determining whether or not to divide is not limited to the above. For example, the code amount when the target block is divided or not divided may be estimated, and the smaller code amount may be selected. In this case, it can be expected that the compression will be higher than that determined by the number of 3D points, but on the other hand, the amount of calculation for code amount estimation will increase, and the time required to complete the coding will increase. Will end up. Therefore, depending on the case, the user may select whether to give priority to the code amount or to give priority to the speed, and may determine whether or not to divide according to one of the determination criteria. Here, as described above, the description will be continued assuming that the division determination is performed by comparing the number of 3D points with a threshold value.

  If the switching unit 103 determines in S205 to divide the block of interest, the process proceeds to S206, passes the block of interest to the dividing unit 104, and divides the block of interest into eight. The division process performed here is the same as in S204. That is, the target block is divided on the condition that the target block includes at least one 3D point.

  In S205, when the switching unit 103 determines not to divide the block of interest, the process proceeds to S207 to cause the intra-block coordinate encoding unit 105 to execute intra-block coordinate encoding of the block of interest. Here, the intra-block coordinate encoding will be described. FIG. 3D shows a block of interest that is a target for intra-block coordinate encoding, and 3D points included therein. In the embodiment, the difference between the coordinates of each 3D point is encoded from the representative coordinates of the block of interest. In other words, 3D points in the block of interest are encoded as relative coordinates based on the representative coordinates of the block of interest. In the present embodiment, the representative coordinates of the block of interest are the coordinates of the corner at which the x, y, and z coordinates of the block of interest are minimized. Therefore, for the illustrated point P1, which is one of the 3D points in the block of interest, the vector indicated by the illustrated arrow is entropy encoded. Other 3D points are similarly encoded. Note that one of all 3D points in the block of interest may be encoded as a vector from the representative coordinates, and the remaining 3D points may be encoded as a difference vector from the already encoded 3D point coordinates. The processing from S205 to S207 is repeated until S208 determines that all blocks have been divided. When the intra-block coordinate encoding of all the blocks is completed, the process proceeds to S209, and the tree structure encoding unit 106 encodes the octree data generated by the previous division processing. In step S210, the encoded data output unit 107 arranges the encoded data of the octree data and the intra-block coordinate encoded data in a preset data structure and outputs the data structure.

FIG. 4 shows an example of the data structure (file structure) of the encoded data output in S210. This will be described in detail below. First, the following information is included in the first header.
(1) Information for defining the cube of the initial block (top block) (2) Number of elements of octree data (3) Depth of reference hierarchy Here, the information of (1) above is typically May be the minimum value of each component of the x, y, and z axes that define the cube and the maximum value of each. The header is followed by octree data above the reference hierarchy, octree data below the reference hierarchy, and point block data within the block.

  Here, the octree data will be supplemented with reference to FIG. FIG. 5A shows the attribute of each block obtained by the processing flow of FIG. 2 in an octree structure. As shown in the legend in the figure, a white circle is a “no dot” block, a black circle is a “with dot division” block, and a circle written as D is a “no dot division” block. The highest block (node) corresponds to the initial block. The eight blocks immediately below the highest block correspond to divided blocks obtained by dividing the highest block into eight. The arrangement order of the eight divided blocks only needs to be common between the encoding device and the decoding device.

  From FIG. 5A, it can be seen that all blocks with dots are divided until the reference hierarchy is reached. The octree data is a list of the attributes of each node according to a certain order. From the viewpoint of progressive encoding, it is desirable to list attributes in the order of width priority. In addition, an identifiable code may be assigned to each attribute and listed as numerical data. In this embodiment, the octree data is reversibly entropy encoded and output.

  An example of encoding the octree data will be described with reference to FIG. The figure shows a binary code word “0 (1 bit)” in the “no dot” block of FIG. 5A, a binary code word “10 (2 bits)” in the “with dot division” block, This shows a state in which the binary code word “11 (2 bits)” is assigned to the “no division with dots” block. Data in which the code words are arranged in the order of width priority indicated by the broken lines in the figure is encoded data of the octree data. It should be noted that binary run-length encoding or arithmetic encoding may be further performed on the illustrated codeword.

Finally, the encoded data of the point cloud data in the block is added to the block in the octree data that is not divided with points. The point group data in one block is the number of points in the block and the in-block coordinates encoded in S207. Here, it supplements about the threshold value n which determines whether it divides | segments as described in S205. In the present embodiment, n = 2 ^N / N is defined in order to achieve high compression. This is because the amount of data that increases when an N-dimensional space is divided is 1 bit (or 2 bits) for every 2 ^N child blocks, and the amount of data that is reduced by division is N coordinates per block. This is to reduce the bit. In this embodiment, since N = 3, n = 3, that is, when there are three or more points in the block, the division is performed. Since an appropriate value of n depends on the performance of entropy coding for compressing each data, a value other than the above may be used for n. This is the end of the description of the code data output in S210 and the flowchart of FIG.

  Next, the configuration of a decoding apparatus corresponding to the encoding apparatus of FIG. 1 will be described using the block diagram of FIG. Here, a decoding apparatus capable of progressively decoding the code data shown in FIG. 4 will be described. The header analysis unit 601 analyzes the header of the encoded data file and reads header information including the depth of the reference layer. The tree structure decoding unit 602 reads the octree data above the reference hierarchy and the encoded data of the octree data below the reference hierarchy. Then, a cubic block having a tree structure is reconstructed based on the octree data. The intra-block point cloud data decoding unit 603 decodes the point cloud data by decoding the number of points in each block and the coordinates of the points existing in the block and adding them to the representative coordinates of the reconstructed block. The point group conversion unit 604 is a block used for progressive decoding, and converts it into a point group having representative coordinates of each block reconstructed by the tree structure decoding unit 602. The point group obtained here has a lower resolution than the point group decoded by the intra-block point group data decoding unit 603, and can be used to grasp the rough distribution of the point group. The decoding device 605 including the processing units 601 to 604 outputs low-resolution point cloud data or point cloud data and displays it on the display unit 606. This is the end of the description of each part in FIG.

  Next, the progressive decoding process in the decoding apparatus shown in FIG. 6 will be described with reference to FIG. In S701, the header analysis unit 601 analyzes the header in the encoded data. In S <b> 702, the tree structure decoding unit 602 reads and decodes the encoded data of the octree data above the reference hierarchy and reconstructs blocks above the reference hierarchy. In step S703, the point group conversion unit 604 generates a point group including the reference coordinates obtained in step S702 and the representative coordinates of the block that satisfies the condition that there is a point. The representative coordinates are the coordinates of the corners at which all the xyz components are minimum in the block, as in the case of obtaining the in-block coordinates by the encoding device. Other points such as the center coordinates of the block may be used. In S704, the point cloud conversion unit 604 sends the low resolution point cloud data generated in S703 to the display unit 606 for display.

  In S705, tree structure decoding section 602 decodes the encoded data of the octree data below the reference hierarchy and constructs a block below the reference hierarchy. In step S706, the intra-block point group data decoding unit 603 decodes the number of points in the block and the intra-block coordinates of each point. Then, the intra-block point cloud data decoding unit 603 decodes the point cloud data by adding the intra-block coordinates of each point obtained by decoding to the representative coordinates of the block decoded in S705. In step S707, the intra-block point cloud data decoding unit 603 displays the point cloud on the display unit 606. Progressive decoding can be realized by the above flow.

The feature of the above progressive decoding is that by outputting the code data divided into blocks up to the reference layer at the time of encoding, the density of the low-resolution point cloud data displayed by progressive decoding becomes constant regardless of the area, and the appearance is This is a natural display. In this embodiment, an example in which a 3D point group is encoded using an octree structure has been described. However, the configuration is the same as that of the encoding device and the decoding device illustrated in FIGS. It can also be applied to the encoding / decoding of a 2 ^N tree of point clouds. For example, a 2D point group can be encoded using a quadtree structure with an apparatus having a similar block configuration.

This is the end of the description of the present embodiment. The encoding device described in the present embodiment enables a natural-looking display by making the density of points at the time of progressive decoding constant, and achieves high compression by dividing the octree in the adaptive hierarchy can do. As described above, according to the first embodiment, the coordinate information of the 3D point group is encoded at a high compression rate while maintaining a natural appearance by making the density of the points at the time of progressive decoding constant. Is possible. More generally, in the coding by the tree structure that divides the block containing the point group in the N-dimensional coordinate space into 2 ^N blocks, the natural appearance is obtained by making the density of the points at the time of progressive decoding constant. It is possible to perform compression at a high compression rate while maintaining.

[Modification of First Embodiment]
Each unit shown in FIG. 1 may be configured by hardware, but may be implemented as software (computer program). In this case, the software is installed in a memory of a general computer such as a PC (personal computer). Then, when the CPU of the computer executes the installed software, the computer realizes the function of the above-described encoding device. That is, this computer can be applied to the above-described encoding device. A hardware configuration example of a computer applicable to the encoding device according to the first embodiment will be described with reference to the block diagram of FIG.

  The CPU 1501 uses the computer programs and data stored in the RAM 1502 and the ROM 1503 to control the entire computer and executes the above-described processes described as being performed by the image processing apparatus. The RAM 1502 is an example of a computer-readable storage medium. The RAM 1502 has an area for temporarily storing computer programs and data loaded from the external storage device 1507, the storage medium drive 1505, and the network interface 1510. Further, the RAM 1502 has a work area used when the CPU 1501 executes various processes. That is, the RAM 1502 can provide various areas as appropriate. The ROM 1503 is an example of a computer-readable storage medium, and stores computer setting data, a boot program, and the like.

  A keyboard 1504 and a mouse 1505 can be operated by a computer operator to input various instructions to the CPU 1501. The display device 1506 is configured by a CRT, a liquid crystal screen, or the like, and can display a processing result by the CPU 1501 using an image, text, or the like.

  The external storage device 1507 is an example of a storage medium that performs computer reading / writing, and is a large-capacity information storage device represented by a hard disk drive device. The external storage device 1507 stores an OS (Operating System) and computer programs and data for causing the CPU 1501 to implement the processes shown in FIG. Computer programs and data stored in the external storage device 1507 are appropriately loaded into the RAM 1502 under the control of the CPU 1501 and are processed by the CPU 1501.

  The storage medium drive 1508 reads a computer program and data recorded on a storage medium such as a CD-ROM or DVD-ROM, and outputs the read computer program or data to the external storage device 1507 or the RAM 1502. Note that part or all of the information described as being stored in the external storage device 1507 may be recorded on this storage medium and read by this storage medium drive 1508. An I / F 1509 is an interface for inputting parameters such as a tree-structured reference hierarchy, and is a USB (Universal Serial Bus) as an example. A bus 1510 connects the above-described units.

  In the above configuration, when the computer is turned on, the CPU 1501 loads the OS from the external storage device 1507 to the RAM 1502 according to the boot program stored in the ROM 1503. As a result, an information input operation can be performed via the keyboard 1504 and the mouse 1505, and a GUI can be displayed on the display device 1506. When the user operates the keyboard 1504 or the mouse 1505 and inputs an activation instruction for an application program for gray image quantization, for example, stored in the external storage device 1507, the CPU 1501 loads the program into the RAM 1502 and executes it. As a result, the computer functions as an encoding device or a decoding device.

  Note that the encoding program executed by the CPU 1501 basically includes functions corresponding to the respective units shown in FIG. Here, the image processing result is stored in the external storage device 1507. Further, the decoding program includes functions corresponding to the respective units shown in FIG. Further, it is apparent that this computer can be similarly applied to an encoding device and a decoding device according to the embodiments described below.

[Second Embodiment]
In the second embodiment, an encoding device that changes the configuration of code data output by the code data output unit and outputs code data that can be partially decoded in the first embodiment will be described. A decoding device that partially decodes the code data is also described.

  The encoding apparatus in the second embodiment has the same configuration as the encoding apparatus described in FIG. However, the configuration of data output from the code data output unit 107 is different from that of the first embodiment. The configuration of the code data output in the second embodiment will be described with reference to FIG. FIG. 8 shows the data structure (file structure) of the code data of the 3D point group divided into the tree structure shown in FIG. 5 as in the first embodiment. Of these, the octree data above the header and the reference hierarchy is the same as in the first embodiment. The data structure following the octree data above the reference hierarchy is a feature of the second embodiment. As shown in FIG. 8, each block in the reference hierarchy is a partial block, and each partial block has a partial structure. A decodable data structure is adopted. For this purpose, the data necessary for decoding the data in each block is collected in one place, and the address in the code data is described. As a result, only data necessary for decoding each partial block can be read and partially decoded at the time of decoding. The data necessary for decoding each partial block includes the octree data in the area below the partial block and the point cloud data (number of points and (In-block coordinates). The details have already been described in the first embodiment.

  A processing procedure performed by the code data output unit 107 in the present embodiment in order to output the code data shown in FIG. 8 will be described according to the flowchart shown in FIG. In S901, the code data output unit 107 generates and outputs a header. In step S902, the code data output unit 107 outputs octree data of a hierarchy higher than the reference hierarchy. These processes are the same as those described in S210. Next, the processing of S903 to S906 is performed for each partial block. First, in S903, the code data output unit 107 calculates the head address of the partial block. As can be seen from FIG. 8, the start address of the partial block to be encoded first follows the header, the octree data above the reference layer, and the list of encoded data start addresses of each partial block. Can be obtained by adding. The amount of octree data above the header and reference layers can be easily calculated because it has already been output in S901 and S902. Since the code data start address of each partial block may be output as a natural number of 32 bits, for example, it may be multiplied by the number of partial blocks. In S904, the code data output unit 107 outputs the head address that is the location of the partial block data. In next S905 and S906, the code data output unit 107 outputs data necessary for decoding the partial block to be processed. This data has already been described in the description of FIG. Similarly, the code data may be output so that the second and subsequent partial blocks have the data structure shown in FIG. This is the end of the description of FIG. Up to this point, the description of the encoding device that outputs partially decodable code data illustrated in FIG. 8 has been completed.

  FIG. 10 is a block configuration diagram of the decoding device 1005 according to the second embodiment. The decoding device 1005 is different from the first embodiment in that the code data can be partially decoded. The basic processing of the header analysis unit 1001, the tree structure decoding unit 1002, the intra-block point group data decoding unit 1003, and the point group conversion unit 1004 in FIG. 10 includes the header analysis unit 601, the tree structure decoding unit 602, and the block in FIG. This is almost the same as the inner point group data decoding unit 603 and the point group conversion unit 604. However, in the second embodiment, in order to enable partial decoding, a decoding block setting unit 1007 and a head address reading unit 1008 are added, and accordingly, the tree structure decoding unit 1002 and block point group data are added. The difference is that the decoding unit 1003 performs processing suitable for partial decoding.

  Hereinafter, the processing procedure of the encoding apparatus according to the second embodiment will be described with reference to the flowchart of FIG. Steps S1101 to S1104 are the same as the processing of the same name in the flowchart of FIG. At the stage where S1104 is completed, among the encoded data in FIG. 8, octree data above the header and the reference hierarchy is read, and a low-resolution point cloud is displayed on the display unit 1006. In step S1105, the decoding block setting unit 1007 sets which partial block in the reference layer is to be decoded. The block to be decrypted is specified by the user operating an operation unit (not shown). That is, a gaze area that the user who observes the point cloud wants to see is received as coordinates from the display unit 1006 (hereinafter, gaze coordinates), and a partial block including the gaze coordinates is set. For example, when the display unit is a two-dimensional display, the gaze coordinates may be low-resolution point cloud data displayed at a position closest to the center of the display. In addition, the gaze coordinate can be set by the user clicking with the mouse while looking at the low resolution point group displayed on the display. In step S1106, the start address of the code data of the partial block to be decoded is received. In step S1107, the octree data of the partial block is decoded. In step S1108, intra-block point cloud data of the partial block is decoded. In step S1109, the high-resolution point cloud data of the partial block is decoded, and the point cloud data combined with the low-resolution point cloud obtained in step S1104 is sent to the display unit. This is the end of the description of FIGS.

  In the second embodiment, as shown in FIG. 8, the data necessary for decoding one partial block is collectively output in one continuous area in the code data. However, the data necessary for decoding the partial blocks may be output in a plurality of locations. In this case, however, it is necessary to output a plurality of addresses that can specify the position of data necessary for decoding the partial block.

  This is the end of the description of the encoding device and the decoding device according to the second embodiment. As with the first embodiment, the encoding device described in the second embodiment can display a point cloud with a uniform density at the time of progressive decoding, and can divide the octree in the adaptive hierarchy. Code data that achieves high compression can be output. Further, as a feature not in the first embodiment, by including the address of data necessary for decoding the partial block in the code data, output and decoding of the code data that can be partially decoded in units of the partial block in the reference layer Is possible.

[Third Embodiment]
In the third embodiment, an example in which processing related to generation of a tree structure is different from that of the first embodiment will be described. The apparatus configuration of the third embodiment is the same as that of FIG. The difference is the processing contents of the tree structure generation unit 108.

The tree structure generation unit 108 is similar to the first embodiment in that tree structure data that satisfies the conditions 1 and 2 described in the first embodiment is generated. The tree structure generation in the first embodiment The unit 108 generates a tree structure that satisfies the above conditions 1 and 2 by recursive division. In contrast, the tree structure generation unit 108 of the third embodiment generates a tree structure that satisfies the above conditions 1 and 2 by recursive integration. Specifically, tree structure data is generated according to the following steps A and B.
A: Tree structure data is generated up to a preset lower limit layer positioned lower than the reference layer set by the setting unit 101.
B: When eight blocks having the same block in the upper layer are located in a lower layer than the reference layer, and the total number of points included in these eight blocks is less than a preset threshold value, These eight blocks are integrated as one block in the upper layer.

  A processing procedure performed by the tree structure generation unit 108 in the third embodiment will be described in detail with reference to the flowchart of FIG. The processing performed in S1201, S1206, S1208, and S1209 is the same as the processing of the same name in FIG. In S1202, the tree structure generation unit 108 generates an overdivided tree structure as an initial state of the tree structure. As shown in FIG. 13, the over-divided tree structure is a tree structure in which all blocks including a point are divided up to the lowest hierarchy (lower limit layer). The lowest hierarchy may be any hierarchy as long as it is deeper than the reference hierarchy. The greater the difference between the lowest hierarchy and the reference hierarchy, the higher the degree of freedom of the finally generated tree structure, and thus the compression rate is considered to improve. However, the processing time increases accordingly. Here, a position that is one layer lower than the reference layer is set as the lowest layer. As another method for setting the lowest hierarchy, there may be a method of determining that, on the basis of the total number of point groups of the input 3D point cloud, one point is averaged in each block of the lowest hierarchy. Since this method can set the lowest hierarchy suitable for the 3D point cloud to be compressed as compared with the lowest hierarchy set in advance, there is a high possibility that the compression will be high for a small processing time.

  S1203 to S1205 are adaptive block integration processing for eight blocks having the same parent block. In step S1203, it is first determined whether or not the processing target reference hierarchy has been integrated. If the reference hierarchy has been reached, the process advances to step S1206. If not, it is determined in step S1204 whether to integrate the target blocks. The criterion for determination is based on a criterion that, when a block is integrated with a parent block, if the number of points in the block is equal to or less than a threshold value, integration is performed. In the first embodiment, the division is performed when the number of 3D points included in the block of interest is equal to or larger than the threshold value, but in the third embodiment, the opposite is true. Note that other standards may be used as the standard for integration. For example, a method of comparing the code amount with and without the integration and selecting the one with the smaller code amount can be considered.

  In S1205, eight child blocks are integrated into one parent block. If it is determined in S1204 that the integration is not performed, the process proceeds to S1206. In S1206, as in the process of the same name in FIG. 2, the in-block coordinates of each point are encoded for a block that is determined not to be integrated. In step S1207, it is determined whether all blocks have been integrated. If all blocks have been integrated, the process advances to step S1208. The subsequent processing is the same as the processing flow of FIG. When the above process is completed, a tree structure is generated in which blocks above the reference hierarchy are always divided, and whether blocks below the reference hierarchy are adaptively switched, as in the first embodiment. be able to. This is the end of the description of the encoding apparatus according to the third embodiment.

  The code data output by the encoding device can be decoded by the decoding device shown in FIG. 6 as in the first embodiment. In the first embodiment, the tree structure data is generated by dividing a large initial block. In the third embodiment, tree structure data of a small preset excessively divided block is generated and integration processing is performed. The tree structure data was generated. However, it is also possible to easily combine both division and integration. This is the end of the description of the third embodiment. As with the first embodiment, the encoding apparatus described in the third embodiment can display a point cloud with a constant density, as in the first embodiment, and the tree structure layer can be switched adaptively. Highly compressed code data can be obtained.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 101 ... Setting part, 102 ... Initial block generation part, 103 ... Switching part, 104 ... Dividing part, 105 ... Intra-block coordinate encoding part, 106 ... Tree structure encoding part, 107 ... Code data output part, 108 ... Tree structure Generator

Claims (12)

An encoding device for encoding coordinate information of a point group in an N-dimensional coordinate space,
A setting means for setting the depth of the reference hierarchy;
Tree structure generating means for generating a tree structure block for dividing the N-dimensional space;
Tree structure encoding means for encoding the data of the tree structure;
Intra-block encoding means for encoding the position of the point in the target block of the tree structure in units of the space in the target block;
An output means for outputting the information including the encoded data generated by the tree structure encoding means and the intra-block encoding means as encoded data of the coordinate information of the point group;
The tree structure generated by the tree structure generation means is (1) blocks above the reference hierarchy are always divided as long as 3D points are included in the block.
(2) Whether or not to divide the blocks below the reference hierarchy is adaptively switched,
An encoding apparatus characterized by that. The tree structure generating means
An initial block generating means for generating an initial block including a point cloud to be encoded in an N-dimensional coordinate space;
A dividing unit that divides the initial block into a plurality of blocks and recursively repeats division for each of the divided blocks obtained by the division, thereby generating tree-structured data having the initial block as the highest layer. And
The dividing means includes
(1) When the target block is located in a hierarchy higher than a preset reference hierarchy, the target block is divided as long as at least one point exists in the target block;
(2) If the block of interest is located in a layer below a preset layer, if there is no point in the block of interest, or if a point exists but the predetermined condition is not met, Stop dividing the block of interest,
(3) The block of interest is divided when the block of interest is located in a hierarchy below a preset hierarchy and the block of interest satisfies the predetermined condition. The encoding device described. The tree structure generating means
As an attribute of the block of interest,
(1) Do not include points,
(2) contains points and is not divided,
(3) includes points and is divided;
The encoding device according to claim 2, wherein any one of the attributes is assigned. 4. The encoding apparatus according to claim 2, wherein the dividing unit divides one block into 2 ^N blocks by dividing the block into 2 along each N-dimensional axis. The predetermined condition is:
The encoding apparatus according to any one of claims 2 to 4, wherein the number of points existing in the block of interest is equal to or greater than a preset threshold value. The tree structure generation means
An over-divided tree structure generating means for generating a tree structure up to a preset hierarchy that is lower than the reference hierarchy;
A plurality of blocks having the same block as an upper layer located in a lower layer than the reference layer and satisfying a predetermined condition; an integration means for integrating the plurality of blocks as an upper layer block; The encoding device according to claim 1, comprising: The predetermined condition is:
The encoding apparatus according to claim 6, wherein a total number of points included in the plurality of blocks is equal to or greater than a preset threshold value. The intra-block encoding means includes
The data indicating the number of points existing in the block of interest and the coordinates of each point from the preset corner of the block of interest to a vector directed to each point is generated as encoded data. The encoding device according to any one of 1 to 6. 9. The output unit according to claim 1, wherein the output unit further outputs, for each block in the reference layer, information indicating a location of encoded data necessary for a point group of the block. Encoding device.   A program for causing a computer to function as each unit included in the apparatus according to any one of claims 1 to 9 when the computer reads and executes the computer.   A computer-readable storage medium storing the program according to claim 10. A control method of an encoding device for encoding coordinate information of a point group in an N-dimensional coordinate space,
A setting step in which the setting means sets the depth of the reference hierarchy;
A tree structure generation step in which the tree structure generation means generates a tree structure block that divides the N-dimensional space;
A tree structure encoding step, wherein the tree structure encoding means encodes the tree structure data;
An intra-block encoding means for encoding a position of a point in the target block of the tree structure in units of a space in the target block; and
The output means includes an output step of outputting information including the encoded data generated in the tree structure encoding step and the intra-block encoding step as encoded data of the coordinate information of the point group. ,
The tree structure generated in the tree structure generation step is (1) Blocks above the reference hierarchy are always divided as long as 3D points are included in the block.
(2) Whether or not to divide the blocks below the reference layer is adaptively switched.
A control method for an encoding device.

Images