KR101817137B1 - Coding Method and Coding Device for Depth Video through Depth Average in Block - Google Patents

Abstract

The present invention provides a method and a device for encoding a depth image through a block average, which can increase a compression ratio while maintaining quality of the depth image in consideration of a property of the depth image. According to an embodiment of the present invention, the method for encoding a depth image through a block average comprises the following steps of: capturing a three-dimensional space, and generating the depth image; separating the depth image into a plurality of blocks; determining an average value of a depth value of pixels in a target block among the plurality of blocks; calculating a difference between the average value and the depth value of the pixels in the target block, and compressing the target block; and encoding the target block.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for encoding a depth image through a block average,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a depth image encoding method and an encoding apparatus using a block average.

Since the advent of the digital age, various multimedia technologies have been rapidly developing, and the digital content market is growing rapidly every year. In addition, the development of various communication means and transmission technology such as the Internet is leading the activation of various multimedia content industries. In recent years, the demand of users has changed from using simple consumption type to using interactive contents and realistic contents. In the field of digital imaging, related technologies from past black-and-white analog TV to recent UHD digital TV are constantly evolving. Recently, interest in 3D images has increased and various depth cameras have been developed for 3D stereoscopic image production. In addition, the demand for high quality 3D stereoscopic images is increasing in various applications.

As such, the demand for high resolution, high quality images is increasing in various applications. However, as the image has high resolution and high quality, the amount of information about the image also increases. Therefore, when image information is transmitted using a medium such as a conventional wired / wireless broadband line, or image information is stored using an existing storage medium, information transmission cost and storage cost are increased. There is a demand for a highly efficient depth image compression technique for effectively transmitting, storing and reproducing information of high resolution and high quality images.

Korean Patent Publication No. 10-2015-0037203

It is an object of the present invention to provide a method and apparatus for encoding a depth image with a block average that can increase the compression rate while maintaining the quality of the depth image considering the characteristics of the depth image.

According to another aspect of the present invention, there is provided a method of encoding a depth image through block averaging, comprising: generating a depth image by capturing a three-dimensional space; Dividing the depth image into a plurality of blocks; Determining an average value of depth values of pixels in a target block among the plurality of blocks; Compressing the target block by subtracting the average value from the depth value of the pixels in the target block; And a step of encoding the target block. The present invention also provides a method of encoding a depth image through a block average.

According to another aspect of the present invention, there is provided a method of coding a depth image through a block average, the method comprising: averaging the average value of the target block with an average value of the immediately preceding block of the target block to compress an average value of the blocks; A method of encoding the depth image through the block averaging may be provided.

According to another aspect of the present invention, the step of determining the average value of the depth values of the pixels in the target block among the plurality of blocks in the depth image coding method using the block average may include: 0 < / RTI > pixels of the depth image, the average value of the depth values of the remaining pixels is determined.

According to another aspect of the present invention, there is provided an encoding apparatus including: a depth detector for capturing a three-dimensional space to generate a depth image; A block dividing unit dividing the depth image into a plurality of blocks; A block analyzer for determining an average value of depth values of pixels in a target block among the plurality of blocks; A pixel compression unit for subtracting the average value from a depth value of pixels in the target block to compress the target block; And a coding unit for coding the target block.

The coding apparatus according to another embodiment of the present invention further comprises an average value compressing unit compressing an average value of the blocks by subtracting an average value of the target blocks from an average value of the immediately preceding blocks of the target block, . ≪ / RTI >

The embodiment of the present invention can represent the depth image with a digital code amount as small as possible without losing the quality of the original depth image by removing other information while preserving important information included in the original depth image.

FIG. 1 is a block diagram of an encoding apparatus according to an embodiment of the present invention, and a configuration thereof.
2 is a conceptual diagram for explaining the concept of depth applied in the embodiment of the present invention.
3 is a block diagram of an averaging unit.
4 shows depth values of pixels of a target block.
FIG. 5 shows lossless compression of depth values of pixels through block averaging.
6 shows the values before and after the average value compression.
FIG. 7 shows a method of storing an average value immediately before each block pixel value.
8 is a flowchart illustrating a method of encoding a depth image through a block average.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below, but may be implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added. Also, in the drawings, for convenience of explanation, the components may be exaggerated or reduced in size. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

<Encoder>

FIG. 1 is a block diagram of an encoding apparatus and structures of the encoding apparatus according to an embodiment of the present invention. FIG. 2 is a conceptual diagram illustrating the concept of depth applied in an embodiment of the present invention.

Referring to FIG. 1, an encoding apparatus 10 according to an exemplary embodiment of the present invention includes a depth measurement unit 100 for capturing a three-dimensional space to generate a depth image, A coding unit 400 for coding the depth image in accordance with the coding mode, a memory unit 400 for dividing the depth image into a plurality of blocks according to the size determined by the coding control unit 300, A dividing unit 600 and an averaging unit 700. [

The depth measuring unit 100 constituting the embodiment of the present invention may include a camera capable of photographing a three-dimensional space.

In addition, the pixels of the image taken by the depth measuring unit 100 may be depth information of an integer of mm, for example, not the RGB color information.

Also, the depth measuring unit 100 can capture a three-dimensional space and generate a depth image. Here, the depth image may be represented by a gray level, for example, and each pixel belonging to the depth image may have a depth value, and the depth value may be a value corresponding to a pixel corresponding to the corresponding pixel of the scene shot by the camera It can mean the distance to the camera.

<Depth measurement method>

In addition, the depth measuring unit 100 can irradiate light onto the three-dimensional space and receive reflected light reflected from the object in the three-dimensional space. And depth information of the object can be detected using the time information from the output of light to the point of time of receiving the reflected light. More specifically, the depth measuring unit 100 may include a TOF (Time Of Flight) camera. The TOF camera may include an image sensor including an in-phase receptor and an out-phase receptor, and a light emitting diode that emits light. In addition, the image sensor is activated in synchronization with the emitted light by being modulated from the light emitting diode. When the light emitting diode is turned on, the in-phase receptor is activated and when the light emitting diode is turned off, have. Since the in phase receptors and the out phase receptors are activated at different points of time with a time difference, there is a difference in the amount of light received (accumulated) depending on the distance to the object, and the distance to the object is measured can do.

Unlike the depth measurement described above, the depth measuring unit 100 is composed of a camera and a projector that outputs structured light, and it is possible to display a narrow band light toward an object to obtain a depth, Dimensional space in which an object exists, by photographing the lines of light, i.e., a light pattern, by using a stereo camera, and by finding a matching point between the left and right images by using a stereo camera, Dimensional depth information can be extracted using the difference between the three-dimensional depth information.

However, the depth detection method of the depth measuring unit 100 is not limited to the above-described method, and any apparatus can be used as long as it can extract three-dimensional depth information.

<The concept of depth>

Referring to FIG. 2, the concept of the depth in the depth image will be described in more detail. A three-dimensional space in which the camera and the image of the depth measuring unit 100 are elevated is referred to as a world coordinate system, and the camera also has a camera coordinate system, which is a three-dimensional coordinate system. The camera may be located at a camera center, which is a point in the clock coordinate system. The X, Y, and Z axes of the camera are referred to as a vertical axis, a horizontal axis, and an optical axis, respectively, and the optical axis is referred to as a principal axis in the direction of a straight line of a camera ray. The point at which the optical axis meets the image plane is called a principal point as a point in the image plane and the distance from the center of the camera to the principal point is the focal length of the camera. Then, each point of the three-dimensional scene to be photographed by projection of the camera is projected onto a two-dimensional image plane. That is, the position of the depth measuring unit 100 represents the origin of the three-dimensional camera coordinate system, and the Z axis (optical axis, principal axis) is in line with the direction of the eye. And an arbitrary point P = (X, Y, Z) in the world coordinate system can be projected to an arbitrary point p = (x, y) of a two-dimensional image plane perpendicular to the Z axis.

In this case, each point p = (x, y) of the 3D scene can be expressed as the Z value of P = (X, Y, Z) of the 3D coordinate system, have.

In describing the present invention, it can be assumed that the world coordinate system and the camera coordinate system coincide with each other.

<Concept of depth information>

If the depth image is redefined based on the above description, it can be said to be a set of information obtained by digitizing the distance between the position of the depth measuring unit 100 and the real object relative to the depth measuring unit 100, , A slice unit, or the like, and the depth information in the depth image can be expressed in pixel units.

In addition, since a pixel value within one frame has a correlation with its neighboring pixel values, encoding can be performed using this correlation. Accordingly, a pixel block may be formed by collecting a plurality of pixels in a certain range within a frame to be encoded.

More specifically, the block dividing unit 600 divides the received depth image into a plurality of blocks according to a size determined by the encoding control unit 300, and outputs the divided blocks.

Each of the plurality of blocks may be defined as an area consisting of M * N pixels. For example, M * N can be defined as an area consisting of 16 * 16 pixels. When the resolution is increased, the basic unit can be defined as an area consisting of 32 * 32 pixels or 64 * 64 pixels.

The encoding unit 400 may encode each of the divided blocks output from the block dividing unit 600 according to the encoding mode determined by the encoding control unit 300. [

The encoding unit 400 can encode the depth information of the pixel in accordance with the encoding mode determined by the encoding control unit 300. [

<Depth image coding through block averaging>

3 is a block diagram of an averaging unit. FIG. 4 shows the depth values of the pixels of the target block, and FIG. 5 shows lossless compression of the depth values of the pixels through the block average. 6 shows the values before and after the average value compression. And FIG. 7 shows a method of storing an average value immediately before each block pixel value.

3, the averaging unit 700 may include a block analysis unit 710, a pixel compression unit 730, an average value compression unit 740, and a block storage unit 750.

The block division unit 600 divides the depth image into a plurality of blocks according to the size determined by the encoding control unit 300, and one of the divided blocks is a block of 8 * 8, for example, as shown in FIG. 10 .

The block analyzer 710 of the averaging unit 700 may determine the average value of the depth values by detecting depth values of all the pixels in the target block. Alternatively, the block analyzing unit 710 may receive information on the depth value detected by the pixel search unit 200 and extract the depth value in the target block from the received depth values.

In addition, the pixel compression unit 730 can perform lossless compression of the amount of information by subtracting the number of bits as shown in FIG. 11 by calculating a difference between the detected average value 1197 and the depth value of the pixel of the target block.

On the other hand, if the depth value of a pixel is detected as 0 in the target block, the corresponding pixel is excluded from the average value calculation, and a special code not designated as a depth value is marked for a pixel having a depth value of 0, .

In addition, the averaging unit 700 may compress the amount of information through the averaging for each of the divided blocks, and may store the average value information for each block.

Referring to FIG. 6, the averaging unit 730 of the averaging unit 700 predicts the average value of each block using the value of the immediately preceding block, and performs compression using the error with the predicted value . That is, if there is an immediately preceding block, the average value of the previous block can be calculated by subtracting the average value of the target block to compress the average value.

The block storage unit 750 may store the number of bits and the average value of each pixel in the block in which the number of bits is reduced according to pixel compression and average value compression of each target block.

Each of the target blocks compressed by the encoding unit 400 may be encoded according to a coding mode such as a discrete cosine transform (DCT) or an intra prediction.

Also, the encoding unit 400 can store the blocks stored in the block storage unit 750 in order and encode them. In this case, the average value can be located at the top of each block as shown in FIG. 7, and a separate table can be created and stored therein.

<Coding method of depth image through block averaging>

8 is a flowchart illustrating a method of encoding a depth image through a block average.

Referring to FIG. 8, a method of coding a depth image through block averaging according to an embodiment of the present invention includes generating a depth image by capturing a three-dimensional space (S100), dividing the depth image into a plurality of blocks (S200), determining an average value of depth values of pixels in a target block among the plurality of blocks (S600), compressing the target block by subtracting the average value from a depth value of pixels in the target block And encoding the target block (S900).

The method of coding a depth image through a block average according to an embodiment of the present invention further includes a step S800 of subtracting an average value of the target blocks from an average value of the immediately preceding blocks of the target block to compress an average value of the blocks .

)을 생성하는 단계, 상기 j개의 일차 방정식에 최소자승법을 적용하여 일차 방정식의 매개 변수(

)를 결정하는 단계 및 상기 매개 변수를 가진 일차 방정식을 결정하고 각 j 값에 대응하는 화소의 보정된 깊이 값을 결정하는 단계를 포함할 수 있다. 본 발명에 따른 실시예는 깊이 영상 내의 깊이 값의 오차를 제거할 수 있는 블록 평균을 통한 깊이 영상의 부호화 방법 및 부호화 장치를 제공할 수 있고, 또한 깊이 영상을 부호화할 때 깊이 영상에 포함된 오차를 제거함으로써 효율적인 부호화가 이루어지도록 할 수 있다.According to another aspect of the present invention, there is provided a method of encoding a depth image through a block average, the method comprising: generating a depth image by capturing a three-dimensional space; dividing the depth image into a plurality of blocks; Detecting a depth value of pixels in a block in one direction, detecting a difference between a depth value of one of the pixels in the one direction and a depth value of the next pixel, Determining a mean value of corrected depth values of pixels in a target block among the plurality of blocks, calculating a difference value between the corrected depth value of the pixels in the target block and the average value, Compressing an average value of the target block and a mean value of the target block; And the method comprising may comprise the step of coding the current block. Here, one direction may be a horizontal direction or a vertical direction of the pixels, and the search area may be any one of the depth image and the depth image divided blocks. The step of correcting the depth value of the searched pixels based on the difference in depth value may further include comparing the difference value of the detected depth value with a preset value, And correcting a depth value of the pixels to be subjected to linearization correction. The pixels to be subjected to linearization correction may include first to k-th (k is a natural number equal to or greater than 2) rectilinearally-correcting pixels, kk (f is a natural number equal to or larger than 1 and k-1) The depth value of the last searched pixel and the depth value of the first searched pixel in the (k-f + 1) th straightening correction target pixels differ by more than the predetermined value. In addition, the step of correcting the depth value of the pixels subject to linearization correction (S530) may include a step of calculating a pixel number (j) for distinguishing a plurality of pixels in any one of the straightening- Using each of the depth values, j linear equations (

), Applying a least squares method to the j number of linear equations to calculate a parameter of the linear equation

) And determining a linear equation with the parameter and determining a corrected depth value of the pixel corresponding to each j value. The embodiment of the present invention can provide a method and apparatus for encoding a depth image through a block average that can eliminate an error of a depth value in a depth image, So that efficient coding can be performed.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as &quot; essential &quot;, &quot; importantly &quot;, etc., it may not be a necessary component for application of the present invention.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

10: Encoding device
100: depth detector
200:
300:
400:
600: Block division
700: Averageizer
710: block analysis unit
730:
740: average value compression section
750: Block storage unit

Claims (9)

Capturing a three-dimensional space to generate a depth image;
Dividing the depth image into a plurality of blocks;
Determining an average value of depth values of pixels in a target block among the plurality of blocks;
Compressing the target block by subtracting the average value from the depth value of the pixels in the target block;
Encoding the target block;
Searching depth values of pixels in a target block among the divided blocks in one direction;
Detecting a difference between a depth value of one of pixels in one direction and a depth value of a next pixel; And
And correcting a depth value of the searched pixels based on the difference of the depth values,
Wherein the average value is an average value of corrected depth values of pixels in the target block,
Wherein the step of correcting the depth value of the searched pixels based on the difference of the depth values comprises:
Comparing the difference value of the detected depth value with a preset value;
Dividing pixels having a difference within the predetermined value into linearization correction target pixels; And
And correcting the depth value of the pixels subject to the linearization correction
Method of coding depth image through block averaging. The method according to claim 1,
And averaging the average value of the target blocks to an average value of the immediately preceding blocks of the target block to compress the average value of the blocks
Method of coding depth image through block averaging. The method according to claim 1,
Wherein the step of determining an average value of depth values of pixels in a target block among the plurality of blocks comprises:
Wherein an average value of depth values of pixels other than a pixel having a depth value of 0 among the pixels in the target block is determined
Method of coding depth image through block averaging. delete delete A depth detector for capturing a three-dimensional space to generate a depth image;
A block dividing unit dividing the depth image into a plurality of blocks;
A block analyzer for determining an average value of depth values of pixels in a target block among the plurality of blocks;
A pixel compression unit for subtracting the average value from a depth value of pixels in the target block to compress the target block;
An encoding unit encoding the target block; And
And a pixel search unit for searching a depth value of pixels in a target block among the divided blocks in one direction to detect a depth value of the pixels,
Detecting a difference between a depth value of one of the pixels in the one direction and a depth value of the next pixel, correcting a depth value of the searched pixels based on the difference in depth value,
The difference value of the detected depth value is compared with the preset value,
The pixels having a difference within the preset value are classified into pixels to be subjected to linearization correction, and the depth values of the pixels subjected to linearization correction are corrected
Encoding apparatus. The method according to claim 6,
And an average value compressing unit compressing an average value of the blocks by subtracting an average value of the target blocks from an average value of the immediately preceding blocks of the target block
Encoding apparatus. delete delete

Images