KR20190087764A - Apparatus and method for encording kinect video data - Google Patents

Abstract

The present invention relates to an apparatus for encoding Kinect video data and a method thereof. According to one embodiment of the present invention, the apparatus for encoding Kinect video data comprises: an offset correction unit for correcting an offset for a range difference of depth video data input from a Kinect; a normalization processing unit for performing a normalization process to include a depth value of the corrected depth video data in a maximum bit range which can be expressed; and a depth video encoding unit for outputting a depth bit stream by performing an encoding process on the normalized depth video data. According to the present invention, it is possible to compress data without data loss.

Description

TECHNICAL FIELD [0001] The present invention relates to an apparatus and method for encoding a Kinect image data,

The present invention relates to an apparatus and method for encoding a Kinect image data, and more particularly, to a Kinect image data encoding apparatus and method for separating Kinect image data (i.e., RGB image data and depth image data) The present invention relates to a Kinect image data encoding apparatus and a method thereof, for reducing data loss by applying a method.

3D video is attracting attention as a next-generation multimedia content format and is expected to replace 2D video. This 3D video can obtain depth information directly from objects using active sensor based kinect.

"Kinect" refers to a peripheral device that is connected to the Xbox 360, which allows users to experience games and entertainment without using a controller.

Here, the kinetic object is displayed in three dimensions with the center point of the infrared camera as the origin. The Z axis is perpendicular to the image plane, the X axis is perpendicular to the Z axis, and the direction from the infrared camera to the laser projector. The Y axis is perpendicular to the Z axis and the X axis.

The Kinect consists of an RGB camera, an infrared sensor, an infrared projector and four microphones. The RGB camera acquires color information, and the infrared sensor and the infrared projector transmit the infrared light of the pixel unit to the front object, receive the reflected light, and obtain the distance information.

Sensors can obtain a color view, a depth view representing depth information of an image, and a skeleton view representing the skeleton of an object. At this time, the sensors detect 47 parts of human body 30 times per second.

The depth image data represents the relative distance between the pixel and the object by the pixel, and the depth map is referred to as the image type information. Pixels close to the camera have bright pixels, or high values, and the farther they are, the farther they are.

Referring to FIG. 1, the depth video data is represented by a total of 16 bits of data. The 3 bits are a player index for detecting a human shape, and 13 bits are depth bits . Here, 12 bits out of 13 bits of depth bits contain depth information of each pixel, and 1 bit is used to deny the depth measurement. 1 is a diagram showing a frame of depth image data.

The depth map plays an important role in 3D video synthesis. Its efficient compression can save additional bits and consequently improve quality during video transmission, storage and playback.

However, since an algorithm for reflecting depth image data is not designed in the 2D video codec, a method for encoding / decoding depth image data in a standardized manner has not yet been systematically established.

Accordingly, there is a need to provide a method for encoding / decoding three-dimensional video including depth image data.

Korean Registered Patent No. 10-1603467 (Registered on Mar. 23, 2016)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a Kinect image data encoding / decoding apparatus for separating Kinect image data (i.e., RGB image data and depth image data) acquired from a Kinect and applying appropriate encoding schemes according to data types, Decoding apparatus and method therefor.

An apparatus for encoding a Kinect image data according to an exemplary embodiment of the present invention includes an offset correcting unit for correcting an offset of a range difference of depth image data input from a Kinect; A normalization processor for performing a normalization process so as to be included in a maximum bit range that can be expressed with respect to a depth value of the corrected depth image data; And a depth image encoding unit for performing a depth encoding process on the normalized depth image data to output a depth bitstream.

According to an embodiment of the present invention, the apparatus may further include an RGB image encoding unit for performing encryption of RGB image data input from the keynote.

The depth image encoding unit may perform an encoding process using an H.265 / HEVC codec.

The RGB image encoding unit may perform an encoding process using an H.264 codec.

The offset correcting unit may adjust the minimum value in the range of the depth image data to a zero point.

The maximum representable bit may be 12 bits.

According to another aspect of the present invention, there is provided a method of encoding a Kinect image data, comprising: correcting an offset of a range difference of depth image data input from a Kinect; Performing a normalization process so as to be included in a maximum bit range that can be expressed with respect to a depth value of the corrected depth image data; And outputting a depth bit stream by performing a coding process on the normalized depth image data.

According to an embodiment of the present invention, the method may further include encrypting RGB image data input from the keynote.

The present invention can reduce data loss by separating Kinect image data (i.e., RGB image data and depth image data) acquired from the Kinect and applying an appropriate encoding method according to the data type.

In addition, the present invention can compress the depth image data without data loss by encoding using the H.265 / HEVC codec.

Further, the present invention can produce a three-dimensional image based on an active sensor, thereby enabling a high-performance three-dimensional image generation and storage and a three-dimensional image synthesis from data at low cost.

In addition, the present invention compresses a 2D image using the H.265 / HEVC codec, and further stores the depth image data in an extension profile, thereby making it widely applicable to data encoding in 3D image production.

Also, the present invention can be utilized for a mobile device which is expected to be limited by capacity overhead in a 3D image rendering operation.

In addition, the present invention adopts the monochrome 12, which is an expansion profile of the H.265 / HEVC codec, to store depth image data without wasted space.

In addition, the present invention can be efficiently utilized in a system for processing information based on data extracted by extracting features using machine learning.

In addition, since the present invention can represent three dimensions by reducing the capacity to be subjected to data processing when the motion is mainly processed, data storage and processing can be easily implemented.

In addition, since the H.265 / HEVC codec has an improved compression ratio than other codecs and can process high-resolution image data with a low capacity, the H.265 / HEVC codec is advantageous in processing three-dimensional video including depth image data. This can provide benefits.

1 is a diagram showing a frame of depth image data,
2 is a block diagram of a Kinect image data encoding apparatus according to an embodiment of the present invention.
3 is a block diagram of a Kinect image data decoding apparatus according to an embodiment of the present invention.
FIG. 4 illustrates a method of encoding / decoding Kinect image data according to an embodiment of the present invention. Referring to FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed in an ordinary or dictionary sense, and the inventor shall properly define the terms of his invention in the best way possible It should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

In the accompanying drawings, some of the elements are exaggerated, omitted or schematically shown, and the size of each element does not entirely reflect the actual size. The invention is not limited by the relative size or spacing depicted in the accompanying drawings.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the spirit or scope of the present invention. Also, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise. It will be understood that terms such as "comprise" or "comprise ", when used in this specification, specify the presence of stated features, integers, , But do not preclude the presence or addition of one or more other features, elements, components, components, or combinations thereof.

Also, as used herein, the term "part " refers to a hardware component such as software, FPGA or ASIC, and" part " However, "part" is not meant to be limited to software or hardware. "Part" may be configured to reside on an addressable storage medium and may be configured to play back one or more processors. Thus, by way of example, and not limitation, "part (s) " refers to components such as software components, object oriented software components, class components and task components, and processes, Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and "parts " may be combined into a smaller number of components and" parts " or further separated into additional components and "parts ".

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 2 is a block diagram of a Kinect image data encoding apparatus according to an embodiment of the present invention, and FIG. 3 is a block diagram of a Kinect image data decoding apparatus according to an embodiment of the present invention.

2, the Kinect image data encoding apparatus 100a according to an embodiment of the present invention includes a key data inputting unit 110 for inputting the key-knock image data (i.e., RGB image data and depth image data) And data loss can be reduced by applying an appropriate encoding method according to the data type.

As shown in FIG. 1, depth video data can be represented by a maximum value of 12 bits, and a method of encoding (compressing) the video data using an image codec as a method for compressing the depth video data can be considered.

The Kinect 10 can generate RGB image data and depth maps of 640 x 480 pixels. The depth map is represented by data and requires a coding process to be delivered to the stream or signal.

In general, video information uses a codec when it is encoded. A codec refers to software or hardware that encodes and decodes an arbitrary data stream.

Here, the case where the RGB image data is encoded according to the H.264 codec and the depth image data is encoded according to the H.265 / HEVC (High Efficiency Video Coding) codec will be described. For example, MP3G-1, MPEG-2, MPEG-4, H.264 / AVC, MVC, and SVC may be applied.

In particular, the H.265 / HEVC codec is a next-generation image compression technology developed by ISO / IEC MPEG, which developed the existing H.264 codec, and the image coding expert group of ITU-U. These H.265 / HEVC codecs have a number of main profiles. In version 1, Main, Main 10 and Main Still Picture are main, and in extension 2, 21 extension profiles are added. The extension profile will include various elements such as bit depth, 4: 2: 2/4: 4: 4 chroma sampling, multi-view video coding (MVC), and extended video coding (SCV).

As such, the profile of the H.265 / HEVC codec will represent 12 bit depth image data using the codec using the version 2 monochrome 12 or higher profile.

This is due to the fact that the input value range of the depth image data can be expressed within a range of 12 bits.

In other words, information can be lost if the depth image data is compressed by an H.264 codec that performs 8-bit encoding for each pixel. However, since the readability of the depth image data can be changed depending on whether the data can be decoded without data loss after coding, a method of compressing the data without loss of information is required. For example, a sign language recognition system requires a lossless compression scheme because it can increase the readability of the gesture depending on whether it can decode the sophisticated depth image data.

Accordingly, the depth image data is encoded by applying the H.265 / HEVC codec separately from the RGB image data, instead of encoding the H.264 codec by using the same method as the RGB image data.

Referring again to FIG. 2, the Kinect image data encoding apparatus 100a needs to process the depth image data into a desired format in the H.265 / HEVC codec in order to encode the depth image data using the H.265 / HEVC codec. To this end, the Kinect image data encoding apparatus 100a performs a preprocessing process of removing the offset according to the KNext version and converting it into 12 bits.

The Kinect image data encoding apparatus 100a includes an RGB image encoding unit 110a, a depth image preprocessing unit 210a, and a depth image encoding unit 220a.

The RGB image encoding unit 110a performs an encoding process on RGB image data input from the keynote 10 and outputs an RGB bit stream. At this time, the RGB image encoding unit 110a applies the H.264 codec to process the RGB image data.

The depth image preprocessing unit 210a preprocesses the depth image data inputted from the keynote 10 by applying the H.265 / HEVC codec to the depth image data. The depth image preprocessing unit 210a includes an offset correction unit 211 and a normalization processing unit 212. [

The offset correcting unit 211 corrects the offset of the range difference of the depth image data indicated by the KNext version. That is, the offset correcting unit 211 performs offset correction, which adjusts the offset to a zero point (0 point) when a range difference from 0 to 4096 or 500 to 4500 occurs in the depth image data according to the Kinect version.

When the depth image data has a depth value of 500 as a minimum value such as 500 to 4500, the offset correction unit 211 adjusts the depth value as a zero point reference by performing an offset correction as shown in Equation 1 below.

The normalization processing unit 212 performs an offset correction process through the offset correction unit 211 and performs a normalization process so that the corrected depth value falls within a range of 12 bits (i.e., 4096 values).

That is, the normalization processing unit 212 uses the value when the maximum value does not exceed 12 bits (that is, 4096 values), and performs the normalization processing when the maximum value exceeds 12 bits (that is, 4096 values) .

The normalization processing unit 212 performs a normalization process on the depth value as shown in Equation (2) below.

The depth image encoding unit 220a performs a process for normalizing the depth image data from the normalization processing unit 212 to output a depth bit stream. At this time, the depth image encoding unit 220a applies the H.265 / HEVC codec to process the depth image data.

As such, the RGB bit stream and depth bit stream can be stored in a file or transmitted over a network.

Referring to FIG. 3, the Kinect image data decoding apparatus 100b decodes the RGB bit stream and the depth bit stream. That is, the Kinect image data decoding apparatus 100b reverses the operation of the Kinect image data encoding apparatus 100a described above in order to reproduce or extract information inversely to the RGB bit stream and the depth bit stream.

The Kinect image data decoding apparatus 100b includes an RGB image decoding unit 110b, a depth image decoding unit 220b, and a depth image post-processing unit 210b.

The RGB bit stream can be reproduced on the screen through decoding or the depth bit stream can be expressed as a monochrome screen through decoding. At this time, the process of converting from 0 to 255 is performed for each pixel.

The sign language recognition system can extract the feature vector by decoding the depth bit stream and use it for machine learning.

FIG. 4 illustrates a method of encoding / decoding Kinect image data according to an embodiment of the present invention. Referring to FIG.

The Kinect image data encoding apparatus 100a outputs an RGB bit stream through encoding of RGB image data (S101). At this time, the Kinect image data encoding apparatus 100a uses the H.264 codec.

At the same time, the Kinect image data encoding apparatus 100a performs offset correction on the depth image data (S201). At this time, the Kinect image data encoding apparatus 100a adjusts the minimum range value of the depth image data on the basis of the zero point.

Thereafter, the Kinect image data encoding apparatus 100a performs a normalization process on the corrected depth image data (S202). At this time, the Kinect image data encoding apparatus 100a performs the normalization process so as to be included within the range of the maximum bit (i.e., 12 bits) that can be expressed with respect to the corrected depth value.

Then, the Kinect image data encoding apparatus 100a outputs the depth bit stream through the encoding of the depth image data (S203). At this time, the Kinect image data encoding apparatus 100a uses the H.265 / HEVC codec.

The method according to some embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CDROMs, DVDs, magneto-optical media such as floptical disks, Magneto-optical media, and hardware devices specifically configured to store and perform program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

Although the foregoing is directed to novel features of the present invention that are applicable to various embodiments, those skilled in the art will appreciate that the apparatus and method described above, without departing from the scope of the present invention, It will be understood that various deletions, substitutions, and alterations can be made in form and detail without departing from the spirit and scope of the invention. Accordingly, the scope of the present invention is defined by the appended claims rather than the foregoing description. All variations within the scope of the appended claims are embraced within the scope of the present invention.

10: Kinect 110a: RGB image coding unit
210a: depth image preprocessing unit 211: offset correction unit
212: normalization processing unit 220a: depth image coding unit
110b: RGB image decoding unit 210b: depth image post-
220b: depth image decoding unit

Claims (8)

An offset correcting unit for correcting an offset of a range difference of depth image data input from a key knot;
A normalization processor for performing a normalization process so as to be included in a maximum bit range that can be expressed with respect to a depth value of the corrected depth image data; And
A depth image encoding unit for performing a depth encoding process on the normalized depth image data to output a depth bitstream;
And a second key data generator for generating a second key data.
The method according to claim 1,
An RGB image encoding unit for encoding the RGB image data input from the keynote;
Further comprising a keycode generator for generating a keycode for the keycode.
The method according to claim 1,
Wherein the depth image encoding unit comprises:
Wherein the encoding process is performed using the H.265 / HEVC codec.
3. The method of claim 2,
Wherein the RGB image encoding unit comprises:
Wherein the encoding process is performed using the H.264 codec.
The method according to claim 1,
Wherein the offset correcting unit comprises:
And adjusts the minimum value in the range of the depth image data to a zero point.
The method according to claim 1,
Wherein the maximum expressible bit is 12 bits.
Correcting an offset for a range difference of depth image data input from a key knot;
Performing a normalization process so as to be included in a maximum bit range that can be expressed with respect to a depth value of the corrected depth image data; And
Outputting a depth bit stream by performing a coding process on the normalized depth image data;
And generating a keycode image.
8. The method of claim 7,
Performing encryption on RGB image data input from the keynote;
Further comprising the steps of:

Images