KR20150089364A - Real-time transmitting system and method for point cloud data, and apparatus applied to the same - Google Patents

Abstract

The present invention relates to a system and a method for transceiving point cloud data in real time, and a device applied thereto. More specifically, depth information for transceiving three-dimensional point cloud data in real time is generated and transmitted as an image, and a bandwidth needed in transmitting point cloud data can be reduced. According to the present invention, a depth value of point cloud data generated through three-dimensional scanning is quantized and assigned to a channel value for each channel included in each pixel constituting a two-dimensional image frame, so that a depth image is generated. The depth image can be compressed through an image codec and can be transmitted through a network. When three-dimensional point cloud data are transmitted, the transmission is possible only through a bandwidth needed in transmitting a two-dimensional image frame, so real-time transmission of point cloud data can be conveniently supported.

Description

Technical Field [0001] The present invention relates to a real-time transmission / reception system and method for point cloud data,

More particularly, the present invention relates to a real-time transmission / reception system and method for point cloud data, and more particularly, to a method and apparatus for real-time transmission / reception of point cloud data, And more particularly, to a system and method for real-time transmission and reception of point-of-view data, and an apparatus applied thereto.

With the development of video technology and various networks, it has become possible to acquire various kinds of information through the network by transmitting a two-dimensional image captured through a camera.

However, in order to acquire the latest 3D image according to the limit of the information obtainable from such a two-dimensional image, studies have been progressing actively. As a result, a three-dimensional image Development and introduction of a measuring instrument including a laser scanner for generating point cloud data, a three-dimensional sensing camera, and the like have been spreading.

The 3D point cloud data obtained through the above-mentioned measuring apparatus provides additional information about the depth value in addition to the existing two-dimensional coordinate information, thereby enabling the environment and the object to be recognized in three dimensions at various points of time. It can provide accurate information on identification and environment.

However, since the bandwidth required for transmitting the point cloud data is not large enough to accommodate the bandwidth of the currently provided broadband network compared to the bandwidth required for the existing two-dimensional image, it is impossible to transmit the point cloud data in real time, There is a difficulty in supporting to utilize in real time.

In addition, when the point cloud data is compressed and transmitted by the currently provided compression method, the loss of the point cloud data generated in the compression process is very large, so that the distortion of the three-dimensional image obtained from the point cloud data occurs. And the utility can not be guaranteed.

Therefore, there is a demand for a system capable of transmitting such point cloud data in real time through currently supported network bandwidth while minimizing data loss.

Korea Patent No. 10-0930626

In order to solve the above-described problems, the present invention provides a depth image generated by quantizing a depth value included in the point cloud data and mapping a quantized depth value to an image channel value, so that the depth image can be transmitted through a network. It is an object of the present invention to support real-time transmission of point cloud data by reducing the bandwidth required for transmission to the bandwidth required for image transmission.

In addition, the present invention can improve the quantization accuracy of the depth value by mapping the depth value of the point cloud data to the channel value of the image in consideration of the error range of the depth value generated in the process of quantizing the depth value, The point cloud data can be restored while minimizing the error due to quantization at the receiving side of the point cloud data, thereby providing accurate information on the point cloud data.

A real-time transmission apparatus for point cloud data according to an embodiment of the present invention includes a point cloud data generation unit for obtaining point cloud data from a measurement unit that scans a measurement object, a quantization unit for quantizing a depth value obtained from the point cloud data, A depth mapping unit for mapping depth values of the input depth values to three channels of a two-dimensional depth image as channel values; and a video coding unit for compressing the depth images, And a communication unit for transmitting the compressed depth image.

As an example related to the present invention, the channel mapping unit maps the channel value of each channel to a quantized depth value in units of a plurality of bit areas in consideration of compression and restoration errors of an image codec at the time of three channel mapping .

According to an embodiment of the present invention, the channel mapping unit allocates the channel to one of the channels 1 to 3 according to the magnitude of the quantized depth value.

The channel mapping unit may map the quantized depth values to channel values of two channels among the three channels according to a remainder obtained by dividing the quantized depth value by a preset unit value and a quotient, And maps the difference between the maximum value and the minimum value according to the error tolerance preset for each channel to the channel value of each channel multiplied by the size of the remainder or quotient corresponding to each channel.

In one embodiment of the present invention, the channel mapping unit divides the quantized depth value into a difference between a maximum value and a minimum value according to a predetermined error tolerance range for the remaining channels excluding the two channels, And maps the channel value to a multiple of the channel value according to the size.

As an example related to the present invention, the three channels may be R, G, and B channels, respectively.

As an example related to the present invention, the measurement unit may include a laser scanner or a three-dimensional camera for generating point cloud data.

A real-time receiving apparatus for point cloud data according to an embodiment of the present invention includes a receiver for receiving a compressed image for point cloud data, an image decoding unit for restoring the compressed image, A point cloud data generator for generating point cloud data from the restored pixel depth values; and a point cloud data generator for generating point cloud data from the point cloud data based on the restored pixel depth values, A correction unit for correcting the blank area and a display unit for expressing the corrected point cloud data.

According to an embodiment of the present invention, the channel restoration unit includes a lookup table in which a quantization level obtained by quantizing the depth value in a plurality of steps and a pixel value of each pixel are mapped to each other, And restores the depth value to a depth value preset to a quantization level corresponding to the pixel-by-pixel channel value.

In an exemplary embodiment of the present invention, the error tolerance of a channel value corresponding to each quantization level is set in the lookup table, and the channel decompression unit restores different channel values belonging to the error tolerance to the same depth value .

In an exemplary embodiment of the present invention, the lookup table may store a maximum value and a minimum value reflecting an error in the channel value in an error tolerance range of the channel value.

In an exemplary embodiment of the present invention, the correction unit corrects the free space by removing the ghost point through a depth test or generating a polygon in the free space.

A method for real-time transmission and reception of point cloud data according to an embodiment of the present invention includes the steps of: acquiring point cloud data from a measurement unit of a transmission device communicating with a measurement unit for scanning a measurement object three-dimensionally; Generating a depth image by quantizing the obtained depth value and mapping the quantized depth value to each of the three channels constituting each pixel of the two-dimensional image as a channel value; and transmitting, by the transmitting apparatus, A step of receiving the compressed depth image from the transmitting device and restoring the compressed depth image; and a step of receiving the compressed depth image from the transmitting device, Generating point cloud data from the restored pixel-by-pixel depth values by restoring the star depth values; The receiving apparatus may include a step of correcting a ghost point or a blank area generated according to the use of the image codec in the point cloud data, and a step of expressing the point cloud data corrected by the receiving apparatus.

A real-time transmission / reception system for point cloud data according to an embodiment of the present invention acquires point cloud data from a measurement unit that scans a measurement object, and quantizes depth values obtained from the point cloud data to generate three A transmitting device for compressing the depth image into a predetermined image codec and transmitting the depth image, and a receiver for receiving and restoring the compressed depth image from the transmitting device, A point group data generating unit for generating point cloud data from the restored pixel-by-pixel depth values; a point cloud data generating unit for generating point cloud data based on the restored pixel- And a receiving device for representing the data.

The present invention quantizes the depth value of the point cloud data generated through three-dimensional scanning to generate a depth image by assigning the depth value of the point cloud data to the channel value of each channel included in each pixel constituting the two-dimensional image frame, It is possible to transmit the 3D point cloud data only through the bandwidth required for transmission of the 2D image frame, thereby facilitating the real time transmission of the point cloud data.

In addition, the present invention can restores the point cloud data by restoring the depth value easily through the depth image in the receiving apparatus that receives the depth image with respect to the depth value of the point cloud data. Also, according to the use of the image codec The loss of the initial point group data can be minimized by correcting the generated ghost point or the free area, thereby improving the accuracy of information on the point cloud data.

In addition, the present invention can consider the error range according to the use of the image codec when quantizing the depth value in the transmitting apparatus, and can precisely map the depth value to the channel value by subdividing the quantized depth value into a plurality of quantization levels, Accordingly, even when an error occurs in a channel value due to use of an image codec in a receiving apparatus, a depth value is easily restored from a channel value in which an error is generated through a lookup table in which an error tolerance range is set, It is possible to restore point-group data at a high speed through the network, and at the same time, improve the restoration accuracy of the point-group data.

1 is a configuration diagram of a real-time transmission / reception system for point cloud data according to an embodiment of the present invention;
2 is a diagram illustrating a configuration for mapping a depth value to a channel value in a transmitting apparatus according to an embodiment of the present invention;
3 is a diagram illustrating a configuration of a depth image compressing process in a transmitting apparatus according to an embodiment of the present invention.
FIG. 4 is an exemplary view illustrating a depth value difference between a first depth image and a restored depth image according to a channel value change according to an embodiment of the present invention; FIG.
5 is a diagram illustrating an example of a channel value change occurring during a compression and decompression process of a transmitting apparatus and a receiving apparatus according to an embodiment of the present invention;
6 is a diagram illustrating an example of a ghost point generated upon restoration of point cloud data in a real time transmission / reception system for point cloud data according to an embodiment of the present invention.
FIG. 7 is an exemplary view showing initial point group data and restored point group data using R, G, and B channels of a real time transmission / reception system for point cloud data according to an embodiment of the present invention;
FIG. 8 is an exemplary diagram illustrating a quantization process of a transmission apparatus according to an embodiment of the present invention; FIG.
9 is a diagram illustrating an example of a correction process for a free area in a real time transmission / reception system for point cloud data according to an embodiment of the present invention.
10 illustrates an example of channel mapping of depth values using two channels of a transmitting apparatus according to an embodiment of the present invention.
11 is an exemplary view of a lookup table of a receiving apparatus according to an embodiment of the present invention;
FIG. 12 is an exemplary diagram for comparing point cloud data reconstructed using R, G, and B channels of a real time transmission / reception system for point cloud data according to an embodiment of the present invention and point cloud data reconstructed using two channels; FIG.
13 is a diagram illustrating channel mapping of depth values using three channels of a transmitting apparatus according to an embodiment of the present invention;
FIG. 14 illustrates an example of point cloud data reconstructed using three channels in a real time transmission / reception system for point cloud data according to an embodiment of the present invention; FIG.
15 is an exemplary diagram illustrating a real-time transmission / reception configuration of a frame image and a depth image in a real-time transmission / reception system for point cloud data according to an embodiment of the present invention;
16 is a flowchart of a real time transmission / reception method for point cloud data according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1, the real-time transmission / reception system for the point cloud data includes a transmission apparatus 100 and a reception apparatus 200, as shown in FIG. 1, according to an embodiment of the present invention. , The transmitting apparatus 100 can transmit data to the receiving apparatus 200 through the network.

At this time, the network may include 3G, 4G, LTE, WiFi, and the like.

In the above-described configuration, the transmission apparatus 100 quantizes the depth value of the point cloud data, maps the depth value of the point cloud data to a plurality of channels included in the image, generates a two-dimensional depth image, compresses To the receiving apparatus 200.

Accordingly, the receiving apparatus 200 receives the compressed image from the transmitting apparatus 100, and obtains the restored depth image by decoding (decoding) the compressed image. Thereafter, the receiving apparatus 200 may recover the depth value from the restored depth image, and obtain the point cloud data including the restored depth value.

As described above, according to the present invention, the point cloud data is mapped to the channels constituting the image to generate a depth image, and the depth image is compressed by the image codec and transmitted, thereby obtaining point cloud data at the bandwidth level required for transmission of the frame image constituting the image It is possible to easily support the real-time transmission of the point cloud data.

The detailed configuration of the transmitting apparatus 100 and the receiving apparatus 200 will be described in detail with reference to FIG.

The transmitting apparatus 100 may include a point cloud data generating unit 110, a quantizing unit 120, a channel mapping unit 130, an image coding unit 140, and a communication unit 150 .

The point cloud data generation unit 110 generates point cloud data based on the scan information transmitted from the measurement unit 10 that scans the measurement object three-dimensionally and provides the point cloud data to the quantization unit 120 .

At this time, the measurement unit 10 may include a three-dimensional camera or a laser scanner for scanning the measurement object to generate point cloud data.

The point cloud data generation unit 110 generates coordinate point values (x, y) and pixel depth values z (z, y) for each pixel constituting an image of a predetermined size from the scan information transmitted from the measurement unit 10, ) Of the point cloud data.

Meanwhile, the quantization unit 120 may quantize the depth value of the point cloud data. For example, the depth value may be quantized into a quantization level calculated by substituting the depth value into Equation (1) .

2, the channel mapping unit 130 may map the depth value to a channel value corresponding to the quantization level. For this purpose, the channel mapping unit 130 may map A range corresponding to the quantization level may be preset for each channel.

Accordingly, the channel mapping unit 130 can quantize the depth values of the pixels included in the point cloud data and generate depth images mapped to the pixel-by-pixel channel values.

For example, the depth image may be composed of three channels of R, G, and B for each pixel, and each channel may have a value of 0 to 255, which is composed of 8 bits. Therefore, the three channels may have a value of 768 in total.

Accordingly, the channel mapping unit 130 may select a channel to which the quantization level belongs by comparing the quantization level with a range of each channel, and may convert the quantization level into a channel value corresponding to the selected channel.

Accordingly, the quantization unit 120 and the channel mapping unit 130 can map the depth value of each pixel constituting the point cloud data to the pixel value of each pixel constituting the depth image.

2, the channel mapping unit 130 converts a depth value of a specific pixel to a depth value (depth) of the specific pixel through the quantization unit 120, 1, the quantization level is allocated to the G channel because the quantization level is included in the range of the G channel. When the quantization level is 300, the value of the G channel corresponding to the quantization level 300 is 44, To a channel value corresponding to 44 of the G channel.

In this case, as described above, the depth value is mapped to one of the three channels because if the three channels of R, G, and B are configured as a single channel, the data units are increased to 24 bits in total, If an error occurs in the bit, the unit difference from the existing depth value becomes too large and it is difficult to correct it. Therefore, as described above, the channel can be divided into 8-bit channels and mapped to each channel to compensate for each channel, thereby minimizing the influence of errors in the upper bits.

As described above, the quantization unit 120 and the channel mapping unit 130 quantize the depth value of each point-in-time data of the point cloud data into a quantization level, and quantize the quantization level using the R, G, and B channels of the image The depth image 20 can be generated by mapping the depth value to the channel value for each pixel as shown in FIG. 3, by mapping the channel value to the corresponding channel value .

In other words, through the above-described process, the channel mapping unit 130 can generate a two-dimensional image (depth image) including a channel-specific channel value mapped to a pixel-by-pixel depth value included in the point cloud data.

3, the image coding unit 140 receives the depth image 20 from the channel mapping unit 130 and compresses the depth image 20 according to a predetermined image codec And can transmit the compressed image to the receiving apparatus 200 through the communication unit 150. [

At this time, the image coding unit 140 may compress the depth image 20 into H.264 / AVC, or may compress the depth image 20 using various image codecs.

In addition, the image coding unit 140 may compress the frame image corresponding to the depth image to generate a compressed image, and may transmit the compressed image to the receiving apparatus 200.

As described above, the transmission apparatus 100 according to the embodiment of the present invention quantizes depth values per pixel included in the point cloud data and maps depth values of the quantized pixels to R, G, and B channels, Thereby generating a depth image composed of the pixel-by-pixel channel values.

Accordingly, the transmitting apparatus 100 can transmit the depth information on the depth value of each point-in-point data of the point cloud data to the receiving apparatus 200 via the network in the form of a two-dimensional image, Thereby enabling real-time transmission of point cloud data.

1, the receiving apparatus 200 includes a communication unit 210, an image decoding unit 220, a channel restoring unit 230, a point cloud data generating unit 240, a correcting unit 250, (260).

The communication unit 210 may receive a compressed image transmitted from the transmission apparatus 100 via a network and the image decoding unit 220 may compress the compressed image received through the communication unit 210 The restored depth image can be generated by restoring according to the codec.

At this time, the image decoding unit 220 can restore the image using the H.264 / AVC codec, and the image codec corresponding to the image codec applied to the image coding unit 140 can be restored to the image decoding unit 220 Can be applied.

On the other hand, the channel restoring unit 230 stores a lookup table for a quantization level corresponding to a channel value for each channel included in the depth image in advance and stores the lookup table corresponding to the pixel value of each pixel of the restored depth image on the basis of the lookup table And recover the depth value from the quantization level.

At this time, the channel decompression unit 230 may store the depth values corresponding to the plurality of quantization levels in advance, and may transmit the depth values of the restored pixels to the point cloud data generation unit 240 .

Accordingly, the point cloud data generator 240 can recover the point cloud data by collecting the coordinate values of each pixel and the depth values corresponding to the pixels.

At this time, in the compression process using the image codec of the transmission apparatus 100 and the restoration process of the reception apparatus 200, a meaningless partial area of the depth image is removed, and as shown in FIG. 4, .

Accordingly, the channel restoring unit 230 of the receiver restores the depth value of each pixel based on the restored depth image having a value different from the channel value of each pixel included in the original depth image before compression, The difference between the depth value of the point group data restored by the point cloud data generator 240 and the depth value of the first point cloud data generated by the point cloud data generator 110 of the transmission apparatus 100 Lt; / RTI >

In addition, various algorithms such as anti-aliasing are applied to the depth image to compensate the boundary line of the depth image as shown in FIG. 5 (a) Likewise, pixel-by-pixel channel values may be varied to produce a reconstructed depth image having different channel values at the same pixel as the original depth image.

For example, in order to compensate for the boundary line through anti-aliasing in the case of pixels 2, 3 and 5 shown in FIG. 5 (a), the image coding unit 140 may calculate the depth image Lt; RTI ID = 0.0 > pixel. ≪ / RTI >

The channel reconstruction unit 230 reconstructs the depth value according to the changed channel value from the reconstructed depth image according to the change of the channel value, and thus the point cloud data generation unit 240 generates one or more pixels The restored point-group data having different depth values are generated in the point-group data and the restored point-group data may cause a difference in the depth value at the same pixel.

In addition, the change of the channel value is different from the original point group data generated by the transmitting apparatus 100 shown in FIG. 6 (a), as shown in FIG. 6 (b) A ghost point that does not exist in the initial point group data can be generated in the area A of the first point group.

According to the change of the channel value, there is a difference between the depth values of the initial point group data and the restored point group data, thereby providing inaccurate information.

The receiving apparatus 200 may include the corrector 250 in order to prevent a difference in the depth value at one or more pixels between the original point cloud data and the restored point cloud data to provide incorrect information. have.

First, the corrector 250 receives the restored point cloud data from the point cloud data generator 240, and removes the ghost point by applying a depth test to the restored point cloud data.

7 (b), the correction unit 250 may transmit the restored point group data to the expression unit 260, and the expression unit 260 may express the restored point group data as shown in FIG. 7 (b) have.

7 (b), the reconstructed point group data represented by the expression unit 260 is different from the original point group data shown in FIG. 7 (a) Value, which indicates that a layer layer is formed.

In the quantization of the layer layer, as shown in FIG. 8, the quantization unit 120 of the transmission apparatus 100 determines that a plurality of different depth values among point group data corresponding to different pixels have the same quantization level An error caused by quantization and an error due to a change in a channel value occurring during compression and decompression through the image coding unit 140 and the decoding unit 220, and the like. Thus, upon restoration of the point cloud data, this layer layer forms an empty spot between the layers.

Therefore, as shown in FIG. 9A, the corrector 250 generates a polygon in a blank area generated from the restored point cloud data, corrects the blank area, 260).

9 (b), the expression unit 260 can express the point cloud data in which the ghost point and the free area are corrected through the corrector 250. [

However, since the corrected point cloud data through the corrector 250 is formed by the quantization of the depth value, the layer layer is formed to provide the point cloud data having a depth value different from the original point cloud data, thereby providing inaccurate information .

Therefore, an embodiment for solving this will be described below with reference to the above-described configuration.

First, the quantization unit 120 of the transmission apparatus 100 can subdivide the quantization level into a plurality of levels, thereby increasing the number of layer layers included in the unit area in the receiving apparatus 200, And can be expressed in a form close to a plane.

The quantization unit 120 of the transmission apparatus 100 may quantize the depth value of the point cloud data provided from the point cloud data generation unit 110 according to the range of the quantization level and convert the depth value into any one of a plurality of quantization levels, The quantization unit 120 may previously store a depth value in a predetermined range corresponding to each quantization level.

For example, the depth value can be quantized to have a quantization level of 1 when the depth value is 0.5011 to 0.502m, and quantized to have a quantization level of 2 when the depth value is 0.5021 to 0.503m.

In this manner, the quantization unit 120 can quantize the depth value to generate a pixel-specific quantization level.

Instead of allocating the quantization level to the three channels of R, G, and B according to the size, the channel mapping unit 130 allocates the remainder and quotient of the quantization level divided by a predetermined unit value to different channels A channel value of each channel corresponding to the quantization level can be generated by adding a value obtained by multiplying each channel by a predetermined multiple unit according to the size of the remainder and quotient to the default value set for each channel.

At this time, the channel restoring unit 230 of the receiving apparatus 200 generates a channel restoring unit 230 when the image coding unit 140 of the transmitting apparatus 100 and the image decoding unit 220 of the receiving apparatus 200 compress and restore The quantization level can be easily restored when the changed channel value is within the error tolerance range in consideration of the error caused by the change of the channel value of each channel. A description of such an error tolerance range will be described in more detail below.

Accordingly, the channel mapping unit 130 sets the channel value of the same channel corresponding to the quantization levels adjacent to each other to the difference between the maximum value and the minimum value according to the error tolerance in units of the multiples, The range can be set.

For example, as shown in FIG. 10, the channel mapping unit 130 may set three channels: a primary channel, a reference channel, and a null channel. A remainder obtained by dividing the quantization level by a unit value may be allocated to a main channel, and a quotient may be allocated to a subchannel. At this time, the null channel is not used and the quantization level can be mapped to two channels.

If the quantization level is 1, the channel mapping unit 130 divides the unit value by 40, and if the quantization level is 1, the channel mapping unit 130 divides the unit value by 40, 5, which is a value multiplied by 5, and adds the calculated 5 to the default value 10 preset in the main channel, thereby generating 15 as the channel value of the main channel corresponding to the quantization level 1. [

If the quantization level is 1, the channel mapping unit 130 calculates 0 by multiplying the subchannel by 10, which is a predetermined multiple of 10, and adds it to the default value to add the subchannel corresponding to the quantization level 1 It is possible to calculate 10 as the channel value.

In this case, the multiple units previously set in the main channel and the sub-channel may be different from each other and may be the same as described above.

If the quantization level is 39, the channel mapping unit 130 applies a factor of 5, which is a preset multiple of 39, to the remainder 39, and sets the default value of 10 And 205 as the channel value of the main channel corresponding to the quantization level 39. Since the quotient is 0, 10 can be calculated as the channel value of the subchannel corresponding to the quantization level 39 as described above.

Meanwhile, when the quantization level is 40, the channel mapping unit 130 calculates the channel value of the main channel corresponding to the quantization level 40 since the remainder obtained by dividing the unit value 40 by 0 is 0 and the quotient is 1, 10 is multiplied by 10, which is a predetermined multiple unit, and 10 is calculated and added to 10, which is the default value of the subchannel, to calculate 20 as the channel value of the subchannel corresponding to the quantization level 40.

In this case, the channel mapping unit 130 sets the default value for each channel. However, it is for setting an error range with respect to the lowest quantization level. It is preferable to set a default value as in the above- It is of course possible to calculate the channel value of each channel without a default.

In addition, the channel mapping unit 130 calculates the channel value by draining the remainder or quotient in units of the multiplier. However, the same channel value is also calculated in the multiples of the remainder or the quotient according to the size of the quotient , And may ultimately be calculated by multiplying the quotient or remainder by the multiple unit, thereby calculating the channel value.

As described above, it can be seen that the main channel is refreshed with a cycle of unit values, and the subchannel increases by a predetermined value each time the main channel is refreshed.

Accordingly, according to this configuration, the channel mapping unit 130 generates a depth value of any one of 0 to 255 when the range of the quantization level is 0 to 255 as shown in the figure, and generates a depth value corresponding to the generated quantization level A channel value can be assigned to each channel of the main channel and the sub-channel. Accordingly, the channel mapping unit 130 may map the depth value to the channel value of each channel allocated according to the quantization level generated by quantizing the depth value for each pixel, You can create an assigned depth image.

In other words, the channel mapping unit 130 may map the depth value of the pixel to the channel value of each channel constituting the depth image according to the quantization level generated by quantizing the depth value.

In this case, the channel mapping unit 130 may set a channel value of each channel corresponding to each quantization level, and may easily map the channel value to a channel value corresponding to a quantization level generated by quantizing the depth value for each pixel Thereby generating a depth image including a channel value for each channel corresponding to the depth value for each pixel.

Accordingly, the channel mapping unit 130 divides the depth value of each pixel into 256 levels of quantization level according to the size, and generates a depth image mapped to a pixel-by-pixel channel value.

However, since the channel values that can be allocated through the R, G, and B channels are 0 to 767, the depth value can be quantized into the quantization level range of 768 steps, and 256 The accuracy of the quantization level range of the step is higher than that of the quantization level range of the step.

The channel mapping unit 130 may store values of 0 to 255 in units of 8 bits for each channel. Therefore, when two channels including a main channel and a subchannel are used, the subchannel has a value of 250 The main channel can be divided into 40 levels of quantization levels and the subchannels can be divided into 24 levels of quantization only by using two channels of the main channel and the subchannel. If the number of steps of the main channel and the subchannel is multiplied, the quantization level range can be subdivided into 960 steps (0 to 959).

Accordingly, when three channels of R, G, and B are used, the quantization level range is limited up to 768 steps. On the other hand, only the quantization level range is set to 960 by using only two channels of the main channel and the sub channel The channel mapping unit 130 may map the depth values more precisely using a smaller number of channels than the R, G, and B channels.

In this case, the quantization unit 120 can generate a quantization level by quantizing depth values corresponding to a quantization level range of the channel mapping unit 130. [

In addition, the channel mapping unit 130 can adjust the multiple units set in the main channel and the sub-channel, and can set the quantization level range to 0 to 999 by setting the unit of the multiple to be small.

According to the above-described configuration, the channel mapping unit 130 may generate a depth image by mapping the depth value of each pixel to the main channel and the sub-channel included in each pixel as a channel value, And transmits the compressed image generated by compressing the depth image through a predetermined codec as described above to the receiving apparatus 200 through the communication unit 150. [

The image decoding unit 220 included in the receiving apparatus 200 receives the compressed image through the communication unit 210 and reconstructs the compressed image to generate a reconstructed depth image and provides the reconstructed depth image to the channel reconstructing unit 230 can do.

At this time, the channel restoring unit 230 may store a lookup table for the main channel and the subchannel as shown in FIG.

The channel reconstructing unit 230 changes the channel values due to the compression and decompression processes of the image coding unit 140 and the image decoding unit 220 in the depth image transmitted by mapping the depth values to the R, G, and B channels A difference in the depth value between the initial depth image and the restored depth image is generated as described above.

That is, when the channel reconstruction unit 230 uses R, G, and B channels, if the quantization level is 1, it is mapped to 1 of the R channel, which is the channel value of the channel. If the value of the quantization level is 2, The difference of channel values corresponding to adjacent quantization levels is 1, and in order to set the error tolerance range, it is necessary to set the error tolerance of each quantization level in units of decimals, but the channel value of each channel is 8 bits Error tolerance range can not be set.

Therefore, when a change occurs in the channel value corresponding to the first quantization level generated by quantization for a specific pixel in the transmitting apparatus 100, the receiving apparatus 200 can obtain a quantization level different from the first quantization level So that there is a difference in the depth value.

Therefore, since the main channel and the subchannel also change in the image compression and decompression processes of the image coding unit 140 and the image decoding unit 220, the channel restoring unit 230 restores the main channel and the subchannel by the channel corresponding to each quantization level. A lookup table in which an error tolerance range of channel values (minimum and maximum values of channel values for each channel corresponding to the respective quantization levels) is set, compares the lookup table with a channel value for each pixel, Even if the value changes, if the channel value of each channel is changed within the error tolerance range, it can be easily restored to the same quantization level as that of the originally generated quantization level for each pixel.

In this case, as described above, the channel mapping unit 130 of the transmission apparatus 100 sets the multiples of the main channel and the subchannel according to the tolerance range of the lookup table stored in the channel restoring unit 230, Since the channel values of the same channel corresponding to the adjacent quantization level differ by a multiple unit, it is possible to prevent duplication between adjacent channel values between adjacent quantization levels by using a multiple unit interval between adjacent quantization levels, The range can be set.

In summary, the channel reconstructing unit 230 may determine whether there is a pixel having a different channel value among the same pixels of the reconstructed depth image and the original depth image, if the changed channel value falls within the tolerance range set in the lookup table A quantization level equal to the quantization level of the first depth image may be generated for each pixel included in the restored depth image based on the lookup table.

Meanwhile, the channel decompression unit 230 can restore the depth value of each pixel and transmit the depth value to the point cloud data generation unit 240 according to the depth value corresponding to each quantization level.

Accordingly, the point cloud data generator 240 may generate (restore) the point cloud data by collecting the depth values of the pixels reconstructed from the channel reconstructor 230, and may transmit the data to the corrector 250.

At this time, the channel restoring unit 230 may transmit the quantization level for each pixel obtained from the restored depth image to the point cloud data generating unit 240, and the point cloud data generating unit 240 may detect the quantization level The depth value may be set, and the depth value may be restored from the per-pixel quantization level to the correcting unit 250.

Thereafter, the corrector 250 may remove the ghost point through the depth test as described above, and may generate a polygon in the free area on the point cloud data to correct the free area.

12B, when the main channel and the subchannel are used, the depth of the layer layer is set to be greater than the depth of the layer layer, The depth of the layer layer represented by the restored point cloud data by applying the R, G, and B channels shown in FIG. 12 (a) is reduced, and the space between the layer layers becomes more dense, .

This is because the quantization unit 120 of the transmission apparatus 100 further subdivides the quantization levels of the R, G, and B channels through the main channel and the sub-channel in the quantization process, and the channel mapping unit 130 Maps the depth value of each pixel to the channel value of the main channel and the subchannel and restores the depth value considering the error range according to the change of the channel value by the channel restoring unit 230 of the receiving apparatus 200, The occurrence of a change can be minimized.

Meanwhile, in the above-described configuration, when the quantization unit 120 of the transmission apparatus 100 uses only two channels of the main channel and the sub-channel and uses the remaining one channel as a null channel Respectively.

Therefore, the quantization unit 120 may configure the remaining one channel as a step channel to subdivide the quantization level range into 0 to 9999 as shown in FIG.

In addition, the additional channel can be set to a multiple unit as in the case of the main channel and the sub-channel. The channel mapping unit 130 assigns a predetermined default value 10 to the additional channel, and the quantization level is set to 1000 digits The main channel and the subchannel are allocated with a channel value corresponding to a remainder obtained by dividing the quantization level excluding 1000 units by a predetermined unit value and a quotient as described above can do.

Accordingly, the quantization unit 120 quantizes the depth value of each pixel to one of the quantization level ranges from 0 to 9999, and the channel mapping unit 130 quantizes the depth value of each pixel, A depth image including a channel value of each channel corresponding to a depth value for each pixel can be generated by mapping the depth value of the pixel to a channel value per channel preset for the level.

In other words, the quantization unit 120 can quantize the quantization level to any one of a quantization level range of 0 to 9999 according to the depth value calculated for each pixel, and accordingly, the channel mapping unit 130 can quantize Subchannel, and supplementary channel with respect to the quantization level corresponding to the depth value, thereby generating a depth image in which the per-pixel depth value is mapped to the per-pixel channel value.

In this case, the channel mapping unit 130 may set the channel values of the main channel, the sub-channel, and the additional channel corresponding to the respective quantization levels in advance, And map the depth value to the channel value of the additional channel.

Accordingly, the quantization unit 120 can further improve the precision by further subdividing the depth value divided by the quantization level range of 0 to 959 into the quantization level range of 0 to 9999. [ For example, when 0.5011 to 0.502 m is quantized to the quantization level 1 in the quantization level range of 0 to 959, and 0.50111 to 0.5012 m is quantized to the quantization level 1 in the range of 0 to 9999, The expression can be further refined.

The transmitting apparatus 100 may compress the depth image through the image coding unit 140 to generate a compressed image and transmit the compressed image to the receiving apparatus 200 through the communication unit 150.

Accordingly, the channel decompression unit 230 of the reception apparatus 200 receives the subdivided depth image in the quantization level range of 0 to 9999 through the image decoding unit 220, Restoring the error tolerance of the channel value to a quantization level corresponding to a channel-specific channel value set for each pixel of the restored depth image based on a preset look-up table, and obtaining a preset depth value corresponding to the quantization level on a pixel- .

Accordingly, the point cloud data generator 240 may integrate the depth values of the pixels to restore the point cloud data, and may transmit the restored point cloud data to the corrector 250.

The corrector 250 may remove the ghost point by applying the depth test and apply a polygon to the vacant area of the restored point cloud data.

At this time, the restored point cloud data obtained from the restored depth image segmented into the quantization level range of 0 to 9999 is obtained from the depth image subdivided into the quantization level range of 0 to 959 in FIG. 12 (b) It can be seen that the number of layer layers included in a predetermined area is increased more than the point cloud data and the depth of the layer is further reduced by increasing the precision of the depth value for each quantization level.

Accordingly, the expression unit 260 can express point cloud data in which the surface distortion is further relaxed when the corrected point cloud data is expressed through the correction unit 250. [

According to the above description, the transmission apparatus 100 generates a depth image by assigning a quantization level corresponding to a depth value to a channel value for each channel included in each pixel constituting the depth image, It can be compressed through the good video codec and transmitted to the receiving device through the network only with the bandwidth required for the video.

Accordingly, the transmission apparatus 100 can convert the point cloud data into a frame image form and transmit the frame cloud image data, so that not only real time transmission is easily supported, but only the performance required for transmission is required for the compressed video, It is possible to reduce the construction cost of the mobile terminal 200.

Meanwhile, the present invention can greatly improve the precision of the quantization level with respect to the depth value of the point cloud data. By dividing and allocating the channel values corresponding to the respective quantization levels into three channels, the quantization process and the compression It is possible to minimize the image error due to the error of the channel value generated in the process, thereby providing accurate information on the point cloud data.

15 is a diagram illustrating an example of a depth image transmitting process of a transmitting apparatus according to an embodiment of the present invention. As shown in FIG. 15, a transmitting apparatus 100 transmits a depth image generated according to the above- In the process of transmitting the compressed image to the device 200, a frame image corresponding to the depth image may be compressed and transmitted together.

In addition, the transmission apparatus 100 may divide one point cloud data into a plurality of depth images and transmit the point cloud data to the reception apparatus 200 in an image form.

FIG. 16 is a flowchart of a real time transmission / reception method for point cloud data according to an embodiment of the present invention. The transmission device 100 communicating with a measurement unit for three-dimensionally scanning a measurement object receives point cloud data from the measurement unit 10 (S1).

Thereafter, the transmission apparatus 100 quantizes the depth value obtained from the point cloud data (S2), maps the quantized depth value to each of the three channels constituting each pixel of the two-dimensional image (S3) , A depth image can be generated (S4).

Next, the transmitting apparatus 100 may compress the depth image into a preset image codec and transmit the compressed image to the receiving apparatus 200 (S5).

Thereafter, the receiving apparatus 200 can receive and restore the compressed depth image from the transmitting apparatus 100 (S6).

In step S8, the receiving apparatus 200 may generate point cloud data from the restored pixel-by-pixel depth values by restoring the pixel-by-pixel depth values from the pixel-by-pixel channel values of the restored depth image (S7) .

The receiving apparatus 200 may correct the ghost point or the blank area generated according to the use of the image codec from the point cloud data (S9), and express the corrected point cloud data (S10).

The foregoing and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings. However, the present invention is not limited to the above-described embodiments, and various changes and modifications may be made by those skilled in the art without departing from the scope of the present invention. .

10: Measuring section 20: Depth image
100: transmitting apparatus 110: point cloud data generating unit
120: quantization unit 130: channel mapping unit
140: Image coding unit 150:
200: Receiving device 210:
220: Image decoding unit 230:
240: point cloud data generation unit 250:
260: Expression section

Claims (14)

A point cloud data generation unit for acquiring point cloud data from a measurement unit that scans an object to be measured;
A quantization unit for quantizing a depth value obtained from the point cloud data;
A channel mapping unit for mapping the quantized depth values to channel values of three channels of a two-dimensional depth image;
An image coding unit for compressing the depth image mapped by the quantized depth value to the channel value into a predetermined image codec; And
A communication unit for transmitting the compressed depth image
The point-of-arrival data for the point-of-view data.
The method according to claim 1,
Wherein the channel mapping unit maps the channel value of each channel to a quantized depth value in units of a plurality of bit areas in consideration of a compression and decompression error of an image codec at the time of three channel mapping, Device.
The method according to claim 1,
Wherein the channel mapping unit allocates the channel to one of the channels 1 to 3 according to a magnitude of the quantized depth value.
The method according to claim 1,
Wherein the channel mapping unit maps the quantized depth values to channel values of two channels among the three channels according to a remainder obtained by dividing the quantized depth value by a preset unit value and a quotient, Is mapped to a channel value of each channel multiplied by a difference between a maximum value and a minimum value according to a size of the remainder or a quotient corresponding to each channel.
The method of claim 4,
The channel mapping unit may multiply the quantized depth value by a difference between a maximum value and a minimum value according to a predetermined tolerance range for the remaining channels excluding the two channels according to the size of the most significant digit of the quantized depth value Wherein the point-to-point data is mapped.
The method according to claim 1,
Wherein the three channels are R, G, and B channels, respectively.
The method according to claim 1,
Wherein the measuring unit includes a laser scanner or a three-dimensional camera for generating point cloud data.
A receiving unit for receiving a compressed image for point cloud data;
An image decoding unit for reconstructing the compressed image;
A channel reconstruction unit for reconstructing a depth value for each pixel from a channel value for each pixel constituting the reconstructed image frame;
A point cloud data generating unit for generating point cloud data from the restored pixel-by-pixel depth values;
A correction unit for correcting a ghost point or a blank area generated according to the use of the image codec for the compressed image in the point cloud data; And
And a display unit for displaying the corrected point-
Wherein the point-of-arrival data comprises point-to-point data.
The method of claim 8,
Wherein the channel restoration unit includes a lookup table in which a quantization level obtained by quantizing the depth value and a channel value of each pixel are mapped to each other,
Wherein the channel restoration unit restores the depth value to a depth value preset to a quantization level corresponding to the pixel-by-pixel channel value based on the look-up table.
The method of claim 9,
The look-up table sets an error tolerance range of a channel value corresponding to each quantization level,
Wherein the channel restoration unit restores different channel values belonging to the error tolerance to the same depth value.
The method of claim 10,
Wherein the lookup table stores a maximum value and a minimum value reflecting an error in the channel value in an error tolerance range of the channel value.
The method of claim 8,
Wherein the correcting unit removes the ghost point through a depth test and corrects the free space or generates a polygon in the free space to correct the free space.
A transmitting apparatus communicating with a measuring unit for three-dimensionally scanning an object to be measured obtains point cloud data from the measuring unit;
The transmitting apparatus quantizing the depth value obtained from the point cloud data and mapping the quantized depth value to each of the three channels constituting each pixel of the two dimensional image to generate a depth image;
Compressing the depth image into a predetermined image codec and transmitting the compressed image to a receiving apparatus;
Receiving and reconstructing a compressed depth image from the transmitting device;
Generating a point cloud data from the restored depth value of each pixel by restoring a depth value of each pixel from a channel value of each pixel constituting the restored depth image;
Correcting a ghost point or a blank area generated according to use of the image codec from the point cloud data by the receiving apparatus; And
The receiving device expressing the corrected point cloud data
Wherein the point-to-point data is transmitted to a point-to-point communication terminal.
Acquiring point cloud data from a measurement unit that scans an object to be measured, quantizing depth values obtained from the point cloud data, mapping the depth values obtained by the point cloud data to channel values of each of three channels constituting each pixel of the two- A transmitting apparatus for compressing and transmitting a depth image to a predetermined image codec; And
Receiving the compressed depth image from the transmission device and restoring the restored depth image, restoring the depth value per pixel from the per-pixel channel value constituting the restored depth image, generating point cloud data from the restored depth value per pixel, A pointing device for expressing point cloud data obtained by correcting ghost points or free areas generated according to the use of the image codec in the point cloud data,
Wherein the point-to-point data is transmitted to the point-to-point communication system.

Images