KR101904170B1 - Coding Device and Method for Depth Information Compensation by Sphere Surface Modeling - Google Patents

Abstract

A method of coding a depth image through spherical modeling according to an embodiment of the present invention includes: dividing a depth image into a plurality of blocks; Detecting object information of a spherical surface in a reference block among the plurality of blocks; Predicting a depth value of a pixel in a target block among the plurality of blocks based on object information of the spherical surface; And encoding the target block based on a predicted depth value of a pixel in the target block.

Description

Technical Field [0001] The present invention relates to a depth information correction method, a depth information correction method,

The present invention relates to a method and apparatus for correcting and encoding depth information in a depth image through spherical modeling. And more particularly to an encoding apparatus for detecting information of a spherical surface in a depth image and modeling information of the spherical surface to thereby correct depth information in the depth image and to perform coding based on the corrected depth information.

There has been actively studied a method of applying the depth image having the depth information indicating the distance information of the pixels in the image to the image processing. At this time, it is possible to acquire the position information and the distance information of the object, which can not be obtained in the existing color image, using the depth image, and acquire the information of the new object through the acquiring. Due to the characteristics of depth images, new applications using depth images are being studied.

  In this paper, we propose a new method to detect objects in a color image by using depth camera. Using the distance information of the depth image, we used the specific information in the depth image to remove image distortion and noise. In addition, we study the recognition of the touch in the background area using the distance information of the depth image and the research that provides various events using it. In addition to this, several studies have been carried out to recognize the face of a person by recognizing the shape of the person.

 Due to the increased application of depth images, the need for depth image coding has increased. First, a method of coding a depth image using a depth lookup table has been studied. Also, a method of using the boundary information of the object for depth image coding has been proposed. A method of coding the depth image by analyzing the depth image based on the histogram has also been proposed. Many studies have been carried out to improve the image coding efficiency by using the feature of depth image. However, the depth image coding study so far has been limited to the method of encoding the depth image in cooperation with the color image, or as a supplementary to the color image coding.

Also, when measuring the depth, a depth image including noise due to measurement error is taken. This noise degrades the accuracy in information processing using depth. In order to improve the accuracy of information processing using depth, it is essential to remove the noise of the depth image.

The existing correction method using the depth image has been studied to remove the noise using the noise model in the plane model. Also, a method to remove the depth error by matching with the color image was proposed. However, the method using the noise model is not suitable for general use because the noise model varies depending on the depth measurement method of the depth camera. In addition, the method of correcting the depth using the color image has a limitation that it is difficult to expect an accurate depth correction due to the characteristics of the color image affected by the environment.

In addition, since the image is viewed by a human being in the case of a general image, it is necessary to appropriately compress the image within a constraint requiring good color and good image quality. However, in the case of a depth image, appropriate compression is required for information processing such as object detection or motion recognition A new compression technique that is different from the compression technique of general image is needed.

(Paper No. 001) S. Liu, P. Lai, D. Tian, and C. W. Chen, " New Depth Coding Techniques with Utilization of Corresponding Video, " IEEE Transactions on Broadcasting, Vol. 57, No. 2, 2011, pp. 551-561. (Paper 002) B. T. Oh, J. Lee, and D. S. Park, "Depth Map Coding Based on Synthesized View Distortion Function," IEEE Journal of Selected Topics in Signal Processing, Vol. 5, No. 7, 2011, pp. 1344-1352. (Paper 003) S. H. Park and J. S. Yoo, "Depth Map Pre-processing using Gaussian Mixture Model and Mean Shift Filter," Journal of the Korean Institute of Information and Communication Engineering, Vol. 15, No. 5, pp. 1155-1163, 2011.

It is an object of the present invention to provide a depth information correction method using spherical modeling which can improve the accuracy of depth values by correcting depth information through spherical modeling.

Also, the present invention can provide a device capable of extracting information on a spherical region in a depth image to predict a depth value, and a method of correcting a depth image through spherical modeling.

The present invention also provides a new encoding method and an encoding apparatus capable of efficiently detecting and encoding an object having a spherical surface in a depth image.

In addition, the present invention can provide a coding apparatus capable of increasing the compression ratio while maintaining the quality of the depth image, and a method of coding the depth image through spherical modeling in consideration of the characteristics of the depth image.

A method of coding a depth image through spherical modeling according to an embodiment of the present invention includes: dividing a depth image into a plurality of blocks; Detecting object information of a spherical surface in a reference block among the plurality of blocks; Predicting a depth value of a pixel in a target block among the plurality of blocks based on object information of the spherical surface; And encoding the target block based on a predicted depth value of a pixel in the target block.

The object information of the spherical surface of the depth image coding method using spherical modeling according to another embodiment of the present invention has a minimum difference between the depth value determined from the object information of the spherical surface of the pixel in the reference block and the measured depth value The method of the present invention can be applied to a method of encoding a depth image through spherical modeling.

According to another aspect of the present invention, there is provided a method of coding a depth image through spherical modeling, the method comprising: encoding an error between a predicted depth value of a pixel in the target block and a depth value of a pixel in the target block; The depth image may be encoded by using the depth information.

According to another aspect of the present invention, there is provided a depth image coding method using spherical modeling, comprising the steps of: generating a depth image in which a depth coordinate value of a coordinate value of an object on a three-dimensional space is projected to a measured depth value of a pixel; Dividing the depth image into a plurality of blocks; Modeling an object in a reference block of the divided blocks into an equation of a first sphere represented by a first undetermined parameter; Determining a value of the first undetermined parameter based on a first undetermined depth parameter of the first pixel in the reference block and a measured depth value of the first pixel to generate a first determination parameter; Generating a prediction depth value by predicting a depth value of a second pixel in a target block among the divided blocks from the first determination parameter; And encoding the target block based on the predicted depth value. The present invention also provides a method of encoding a depth image using spherical modeling.

According to still another aspect of the present invention, there is provided a method of coding a depth image through spherical modeling, the method comprising: generating a first depth value by determining a value of the first undetermined depth parameter from the first determination parameter; And correcting the measured depth value of the first pixel to the first determined depth value. The present invention also provides a method of encoding a depth image using spherical modeling.

According to another aspect of the present invention, there is provided a method of coding a depth image through spherical modeling, the method comprising: modeling an object in the object block into an equation of a second sphere expressed by a second undetermined parameter; Determining a value of the second undetermined parameter based on a second undetermined depth parameter of the second pixel and a measured depth value of the second pixel to generate a second determination parameter; Determining a value of the second uncertainty depth variable from the second decision parameter to generate a second crystal depth value; And correcting the measured depth value of the second pixel to the second determined depth value. The present invention also provides a method of encoding a depth image using spherical modeling.

The step of encoding the target block based on the predicted depth value of the method of coding a depth image through spherical modeling according to another embodiment of the present invention may further include the step of calculating the predicted depth value and the corrected depth value of the second pixel And the object block is encoded based on the motion vector of the current block.

The encoding of the target block based on the predicted depth value of the method of encoding the depth image through spherical modeling according to another embodiment of the present invention may include encoding the predicted depth value and the measured depth value of the second pixel The depth image may be encoded using spherical modeling to encode the object block.

According to still another aspect of the present invention, there is provided a method of coding a depth image through spherical modeling, the method comprising the steps of: determining an error between a first microcrystalline depth parameter of a first pixel in a reference block and a depth value of a measured first pixel, And determining a value of a first undetermined parameter to generate a first determination parameter. The method may further include a step of providing a depth image encoding method using spherical modeling.

According to still another aspect of the present invention, there is provided a method of coding a depth image through spherical modeling, wherein a value of the first uncertain parameter is determined by applying a least squares method to minimize the error. Thereby providing a depth image encoding method.

The step of generating the predicted depth value of the depth image coding method using spherical modeling according to another embodiment of the present invention may further include a step of calculating a predicted depth value based on the first determination parameter, the coordinate value of the second pixel, And generating a depth image by using the spherical surface modeling method.

The embodiment according to the present invention can correct the depth image irrespective of the type of the camera.

Also, the embodiment of the present invention can accurately correct the depth information by finding the spherical information in the depth image using the depth value in the block and estimating the depth value of the depth image including the spherical surface.

In addition, the embodiment of the present invention can improve the accuracy of correction of the depth value not only for an image of a spherical surface but also for an image having a complex depth value.

In addition, the embodiment of the present invention can acquire gesture recognition or modeling information for a shooting region by finding accurate depth values and using more accurate depth information.

Also, in the embodiment of the present invention, depth information can be expressed with as little digital code amount as possible without losing the quality of the original depth image, while preserving important information included in the original depth image while removing other information.

In addition, the embodiment of the present invention can find the spherical information in the depth image using the depth value in the block, correct the depth value of the depth image including the spherical surface, correct the depth value of the target block predicted from the spherical reference block, The depth image can be encoded using the error of the depth value.

FIG. 1A is a block diagram of an encoding apparatus according to an embodiment of the present invention and constructions thereof.
1B is a block diagram of an encoding apparatus according to another embodiment of the present invention, and a configuration thereof.
2 is a conceptual diagram for explaining the concept of depth applied in the embodiment of the present invention.
3 is a conceptual diagram of a point of a sphere projected on an image plane when the sphere is displayed on the world coordinate system.
FIG. 4 is a rectangular coordinate system showing an arbitrary point along the X and Z axes and a point of the 2D image plane. FIG. 4 is a circle showing a sphere formed by cutting the sphere of FIG. 3 into a plane perpendicular to the Y axis.
5 is a block diagram of the constructions constituting the depth information correction section.
6 is a block diagram of the constructions constituting the encoding unit.
7 to 9 are flowcharts illustrating a method of correcting and encoding a depth image through spherical modeling according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below, but may be implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added. Also, in the drawings, for convenience of explanation, the components may be exaggerated or reduced in size. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

<Encoder>

FIG. 1A is a block diagram of an encoding apparatus according to an embodiment of the present invention, and FIG. 1B is a block diagram of an encoding apparatus and a configuration of the encoding apparatus according to another embodiment of the present invention. 2 is a conceptual diagram for explaining the concept of depth applied in the embodiment of the present invention.

1A and 1B, the same reference numerals are assigned to the same components, and redundant explanations of functions and effects are omitted.

Also, each block of each structure is classified for convenience of description and may be integrated into one configuration.

An encoding apparatus 10 according to an embodiment of the present invention may include a depth measurement unit 100, a depth information correction unit 200, a control unit 300, and an encoding unit 400 .

1B, an encoding apparatus 10 according to another embodiment of the present invention includes a depth measurement unit 100, a depth information correction unit 200, a control unit 300, an encoding unit 400, (600).

The depth measuring unit 100 constituting the embodiment of the present invention may include a camera capable of photographing a three-dimensional space.

In addition, the pixels of the image captured by the depth measuring unit 100 may not be the RGB color information, but may be, for example, depth information in units of mm, but not limited thereto.

Also, the depth measuring unit 100 can capture a three-dimensional space and generate a depth image. Here, the depth image can be represented by, for example, a gray level, but is not limited thereto. Also, each pixel belonging to the depth image may have a depth value, and the depth value may mean the distance from the point corresponding to the corresponding pixel of the scene shot by the camera to the camera.

<Depth measurement method>

In addition, the depth measuring unit 100 can irradiate light onto the three-dimensional space and receive reflected light reflected from the object in the three-dimensional space. And depth information of the object can be detected using the time information from the output of light to the point of time of receiving the reflected light. More specifically, the depth measuring unit 100 may include a TOF (Time Of Flight) camera. The TOF camera may include an image sensor including an in-phase receptor and an out-phase receptor, and a light emitting diode that emits light. In addition, the image sensor is activated in synchronization with the emitted light by being modulated from the light emitting diode. When the light emitting diode is turned on, the in-phase receptor is activated and when the light emitting diode is turned off, have. Since the in phase receptors and the out phase receptors are activated at different points of time with a time difference, there is a difference in the amount of light received (accumulated) depending on the distance to the object, and the distance to the object is measured can do.

Unlike the depth measurement described above, the depth measuring unit 100 is composed of a camera and a projector that outputs structured light, and it is possible to display a narrow band light toward an object to obtain a depth, Dimensional space in which an object exists, by photographing the lines of light, i.e., a light pattern, by using a stereo camera, and by finding a matching point between the left and right images by using a stereo camera, Dimensional depth information can be extracted using the difference between the three-dimensional depth information.

However, the depth detection method of the depth measuring unit 100 is not limited to the above-described method, and any apparatus can be used as long as it can extract three-dimensional depth information.

<The concept of depth>

Referring to FIG. 2, the concept of the depth in the depth image will be described in more detail. A three-dimensional space in which the camera and the image of the depth measuring unit 100 are elevated is referred to as a world coordinate system, and the camera also has a camera coordinate system, which is a three-dimensional coordinate system. The camera may be located at a camera center, which is a point in the clock coordinate system. The X, Y, and Z axes of the camera are referred to as a vertical axis, a horizontal axis, and an optical axis, respectively, and the optical axis is referred to as a principal axis in the direction of a straight line of a camera ray. The point at which the optical axis meets the image plane is called a principal point as a point in the image plane and the distance from the center of the camera to the principal point is the focal length of the camera. Then, each point of the three-dimensional scene to be photographed by projection of the camera is projected onto a two-dimensional image plane. That is, the position of the depth measuring unit 100 represents the origin of the three-dimensional camera coordinate system, and the Z axis (optical axis, principal axis) is in line with the direction of the eye. And an arbitrary point P = (X, Y, Z) in the world coordinate system can be projected to an arbitrary point p = (x, y) of a two-dimensional image plane perpendicular to the Z axis.

In this case, each point p = (x, y) of the 3D scene can be expressed as the Z value of P = (X, Y, Z) of the 3D coordinate system, have.

In describing the present invention, it can be assumed that the world coordinate system and the camera coordinate system coincide with each other.

<Concept of depth information>

If the depth image is redefined based on the above description, it can be said to be a set of information obtained by digitizing the distance between the position of the depth measuring unit 100 and the real object relative to the depth measuring unit 100, , A slice unit, or the like, and the depth information in the depth image can be expressed in pixel units.

Also, since the pixel values in one frame have a correlation with the neighboring pixel values, the depth information can be corrected using this correlation. Therefore, a pixel block can be constructed by collecting a plurality of pixels in a certain range within a frame in which depth information is to be corrected.

More specifically, the block dividing unit 600 divides the received depth image into a plurality of blocks according to a size determined by the controller 300, and outputs the divided blocks.

Each of the plurality of blocks may be defined as an area consisting of M * N (M, N is a natural number) pixels. For example, M * N can be defined as an area consisting of 16 * 16 pixels. When the resolution is increased, the basic unit can be defined as an area consisting of 32 * 32 pixels or 64 * 64 pixels.

Also, the depth information correction unit 200 can correct the depth information according to the spherical modeling correction. The spherical modeling correction is to estimate the spherical surface that forms the surface of the object with respect to the spherical object in the depth image, and to divide the specific area or one frame within one frame into M * N size And the depth information is corrected by modeling the spherical surface closest to the surface represented by the depth information of the corresponding block.

The encoding unit 400 may encode each of the divided blocks output from the block dividing unit 600 according to the encoding mode determined by the control unit 300. [

Also, The encoding mode may include a spherical modeling encoding mode.

The spherical modeling coding mode is a method of estimating the spherical surface forming the surface of the object with respect to the spherical object in the depth image. In the depth image segmented by M * N size, the surface represented by the depth information of the corresponding block, It is to model the adjacent spheres.

Also, the controller 300 may determine a block to be encoded through the spherical modeling mode among the divided blocks based on the depth values of the pixels of the divided blocks. For example, if the depth value gradually increases or decreases from a pixel in a divided block toward an adjacent pixel region moving away from the arbitrary pixel with respect to an arbitrary pixel, it can be predicted that a spherical region is included in the corresponding block , It is possible to store information of blocks classified by dividing blocks predicted to include the spherical area.

<Calibration through spherical modeling>

FIG. 3 is a conceptual view of a point of a sphere projected on an image plane when a sphere is displayed on a world coordinate system, and FIG. 4 is a diagram illustrating a point of an arbitrary point along the X, 3 as a coordinate system, the sphere being cut in a plane perpendicular to the Y-axis and being a circle.

Referring to FIGS. 3 and 4, the depth information correction unit 200 may generate a corrected depth value based on the coordinates and depth values of n pixels surrounding the target pixel, where the value of n Dimensional space can be adjusted based on the information of the object in the three-dimensional space. For example, if there is no object with a sphere on the 3D space, the value of n may be larger than n if there is an object with a sphere on the 3D space.

In addition, the surface of the spherical object in the depth image can be modeled as Equation 1 in the 3D camera coordinate system in which the Z axis is represented by the optical axis of the camera. Such a spherical surface may be projected through a depth detector 100 with a depth value d to a pixel positioned at a virtual image plane (h, w) coordinate located at a camera focal distance f. The coordinates on the three-dimensional space projected onto this pixel are represented by a triangle formed by the camera and the image plane, a triangle formed by a straight line parallel to the X-axis passing through a point on the camera and the captured spherical object, .

 [Equation 1]

In Equation 1, (x, y, z) is a coordinate of a point of the spherical object, a, b, and c are centers of the sphere, and r is the radius of the sphere. And a, b, c, and r, which are parameters of the sphere equation, can be defined as undetermined parameters whose values are not determined.

&Quot; (2) &quot;

In Equation (2), f is the focal length.

&Quot; (3) &quot;

는 수직 방향의 화각을 의미하고,

는 수평방향의 화각을 의미한다.In Equation (3), height denotes the vertical size of the effective sensor of the image sensor provided in the camera of the depth detector 100, and width denotes the size of the effective sensor in the horizontal direction. Also

Means an angle of view in the vertical direction,

Means an angle of view in the horizontal direction.

The equation represented by the equation (1) representing the spherical surface expressed by the coordinates (h, w) of the pixel coordinate system meets the equation (4) (equation (2) is substituted into equation (2)). .

&Quot; (4) &quot;

Further, if the root of Equation (4) expressed by the quadratic expression of d is found, d can satisfy Equation (5).

&Quot; (5) &quot;

In this case, d in Equation (5) can be regarded as an ideal depth value when the parameters a, b, c, and r of the spherical surface represented by Equation (1) are given.

Also, the ideal depth value d in Equation (5) can be defined as a pending depth variable whose value is not determined.

의 차이가 제일 작을 때의 a, b, c, r(값이 결정된 매개변수: 결정매개변수)를 구함으로써 대상 블록에서 최적의 구의 방정식을 구하여 이상적인 깊이 값을 결정할 수 있다. 이 때 모델링된 표면의 (h, w)에서의 근사된 구면의 미결정깊이변수와 실제로 측정된 깊이의 오차

가 최소가 되도록 인자를 결정하기 위해 최소자승법을 적용할 수 있다. 이 때 e는 매개변수에 대해 비선형식으로 나타나므로 가우스-뉴턴법을 적용할 수 있다.The ideal depth value d (undetermined depth variable) and the depth value actually measured (h, w)

The ideal depth value can be determined by obtaining the optimal sphere equation in the target block by obtaining a, b, c, and r (the parameter whose value is determined: the decision parameter) when the difference is smallest. At this time, the microcrystalline depth parameter of the approximated spherical surface at (h, w) of the modeled surface and the error of actually measured depth

The least squares method can be applied to determine the factor to be the minimum. In this case, e is nonlinear with respect to the parameter, so the Gauss-Newton method can be applied.

의 차로 이루어진 행렬

과

에서의 자코비안 행렬

, 미결정매개변수 값을 나타내는

은 수학식 6을 충족한다.In the Gauss-Newton method, d (uncertainty depth variable) and actual measured depth value

A matrix consisting of

and

Jacobian procession in

, Indicating the value of the indeterminate parameter

Satisfies Equation (6).

&Quot; (6) &quot;

,

,

,

,

을 구할 수 있다.Also, by applying the Gauss-Newton method, as shown in Equation 7,

Can be obtained.

&Quot; (7) &quot;

의 이상적인 깊이 값 d(값이 결정된 미결정깊이값= 결정깊이값)를 결정함으로써 보정 대상 화소의 깊이 값을 이상적인 깊이 값(결정깊이값)으로 보정할 수 있다.This is repeated P times to obtain the spherical surface closest to the surface with a given depth value and to determine the value of the uncertainty parameter of the equation of the sphere. Then, by substituting the determined parameter, the coordinate value of the correction target pixel, and the focal length f into the equation (5), the correction target pixel

The depth value of the correction target pixel can be corrected to the ideal depth value (crystal depth value) by determining the ideal depth value d (the undetermined depth value = the determined depth value).

를 정함에 있어

과

값은 블록 중심의 좌표인

와

를 수학식 2에 대입하여 구할 수 있고,

은 임의의 초기값

를 대입한다.

은 블록 중심의 깊이 값

에서

을 더한 수치를 초기 값으로 할 수 있다.At this time,

In determining

and

The value is the coordinate of the block center

Wow

Can be obtained by substituting in Equation (2)

Lt; RTI ID = 0.0 &gt;

.

Is the depth value of the block center

in

Can be used as an initial value.

에서의

와 구면 모델링 과정을 수행하여 찾은 매개변수에 따른 깊이 값

(결정깊이값)의 차이를 이용하여 각 화소를 부호화할 수 있고, 그 후 각 대상 블록에서의 블록 내 부호화된 화소와 함께 구의 매개변수 값(결정매개변수)도 부호화하여 전체 영상을 부호화할 수 있다. 다만 이에 한정하는 것은 아니고, 후술하는 참조블록의 구면으로부터 대상블록의 화소의 깊이 값을 예측하고, 예측된 예측깊이값과 대상블록의 화소의 측정된 또는 보정된 깊이 값의 오차를 부호화할 수도 있다. 이에 대한 상세한 설명은 후술한다.Also, the encoding unit 400 determines the number of pixels (pixels) in the target block based on the obtained parameters

In

And the depth value according to the parameters found by performing the spherical modeling process

(Decision depth value), and then the parameter values (decision parameters) of the sphere as well as the intra-block coded pixels in each target block are coded to encode the entire image have. The depth value of the pixel of the target block may be predicted from the spherical surface of the reference block to be described later and the error of the predicted depth value and the measured or corrected depth value of the pixel of the target block may be encoded . A detailed description thereof will be described later.

&Lt; Coding method of depth image through spherical modeling coding mode >

FIG. 5 is a block diagram of the constructions constituting the depth information correction unit, and FIG. 6 is a block diagram of constructions constituting the encoder.

5 and 6, the depth information correction unit 200 according to the embodiment of the present invention includes a depth image spherical modeling unit 210, a parameter operation unit 220, and a correction depth information operation unit 230 .

The depth image spherical modeling unit 210 may model the spherical object included in the depth image as an equation of a sphere such as Equation (1).

Further, the parameter operation unit 220 can determine the parameters (determination parameters) of the sphere equation according to the above-described Equations 2 to 6 so that the depth value and the coordinate value of the pixels in the depth image satisfy the equation of the sphere.

The correction depth information calculation unit 230 calculates an ideal depth value (a depth value) of the pixels in the depth image according to Equation (5) using the parameter (determination parameter) of the equation of the sphere determined from the parameter calculator 220, Can be determined.

Meanwhile, the depth image spherical modeling unit 210 may model a spherical object in a reference block (or a target block) from a plurality of divided blocks into spherical equations. In this case, the parameter operation unit 220 may obtain the determination parameters of the equation of the sphere such that the depth value and the coordinate value of the pixels in the reference block (or the target block) satisfy the equation of the sphere. The correction depth information operation unit 230 can obtain the depth value of the pixels in the reference block (or the target block).

The encoding unit 400 may include a target block spherical modeling unit 401 and a target block encoding unit 402.

The target block spherical modeling unit 401 receives the parameter of the sphere equation for the spherical object in the reference block determined from the parameter calculator 220 and calculates the coordinates of the pixels in the target block and the focal length information The predicted depth value can be obtained by predicting the depth value of the pixels in the target block. That is, the depth value of a pixel of a target block can be predicted by using a parameter of a sphere equation for a spherical object included in a reference block, rather than a parameter of a sphere equation for a spherical object included in a target block.

Here, the reference block may be a coded pixel, but is not limited thereto. Here, the reference block may be composed of pixels that are in contact with the upper and left sides of the target block, but the present invention is not limited thereto.

The object block coding unit 403 encodes the error between the predicted depth value of each pixel in the target block received from the target block spherical modeling unit 402 and the measured depth value of each pixel in the target block, can do.

The target block coding unit 403 may calculate the predicted depth value of each pixel in the target block received from the target block spherical modeling unit 402 and the corrected depth value of the target block received from the depth information correction unit 200 And the target block can be encoded.

In addition, the above process may be applied to all of the target blocks to sequentially encode the target blocks to encode the entire image.

7 to 9 are flowcharts illustrating a method of correcting and encoding a depth image through spherical modeling according to an embodiment of the present invention.

Referring to FIG. 7, a method of correcting and encoding a depth image through spherical modeling according to an embodiment of the present invention is a method of interpolating a depth image in which a depth coordinate value of an object on a three-dimensional space is projected to a measured depth value of a pixel (S200) of dividing the depth image into a plurality of blocks (S200). In the step S200, the object in the reference block among the divided blocks is transformed into a block represented by the first undetermined parameter (S310), determining a value of the first undetermined parameter based on a first undetermined depth parameter of the first pixel in the reference block and a measured depth value of the first pixel, (S500) of generating a prediction depth value by predicting a depth value of a second pixel in a target block among the divided blocks from the first determination parameter (S320) It can be the basis of this value to a step (S600) for encoding the current block.

The embodiment further includes a step of determining a value of the first microcrystalline depth variable from the first determination parameter to generate a first crystal depth value (S330), and a step of comparing the measured depth value of the first pixel with the first decision depth (S340). &Lt; / RTI &gt;

The embodiment may further include the steps of modeling an object in the object block into an equation of a second sphere expressed by a second undetermined parameter (S410), calculating a second undetermined depth variable of the second pixel and a depth value of the measured second pixel Determining a value of the second microcrystalline parameter based on the second determination parameter to generate a second determination parameter (S420); determining a value of the second microcrystalline depth variable from the second determination parameter, And correcting the measured depth value of the second pixel to the second determined depth value (S440).

In addition, in step S600 of encoding the target block based on the predicted depth value, the target block may be encoded based on the predicted depth value and the corrected depth value of the second pixel.

In addition, in step S600 of encoding the target block based on the predicted depth value, the target block may be encoded based on the predicted depth value and the measured depth value of the second pixel.

In addition, the step of generating the first decision parameter (S320) may be such that the error of the depth value of the first pixel measured and the first undetermined depth variable of the first pixel in the reference block is minimized Value to generate a first determination parameter. More specifically, the least squares method may be applied to determine the value of the first undetermined parameter to minimize the error.

The step of generating the predicted depth value (S500) may be based on the first determination parameter, the coordinate value of the second pixel, and the focal length.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as &quot; essential &quot;, &quot; importantly &quot;, etc., it may not be a necessary component for application of the present invention.

Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. It will be understood that the invention can be variously modified and changed without departing from the technical scope thereof. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

10: Encoding device
100: depth detector
200: Depth information detector
210: Depth image spherical modeling unit
220: Parameter operation unit
230: correction depth information calculating section
300:
400:
401: Object block spherical modeling unit
402: object block coding unit
600: Block division

Claims (11)

delete delete delete Generating a depth image in which a depth coordinate value of a coordinate value of an object on a three-dimensional space is projected to a measured depth value of the pixel;
Dividing the depth image into a plurality of blocks having a size determined as an area made up of M * N (M, N is a natural number) pixels;
Modeling an object in a reference block of the divided blocks into an equation of a first sphere represented by a first undetermined parameter;
Determining a value of the first undetermined parameter based on a first undetermined depth parameter of the first pixel in the reference block and a measured depth value of the first pixel to generate a first determination parameter;
Generating a prediction depth value by predicting a depth value of a second pixel in a target block different from the reference block among the divided blocks from the first determination parameter; And
And encoding the error between the predicted depth value and the measured depth value of the second pixel to encode the target block,
Wherein generating the predicted depth value comprises:
Based on the first determination parameter, the coordinate value of the second pixel, and the focal length,
Determining a value of the first undetermined parameter such that an error between the first undetermined depth parameter of the first pixel in the reference block and the measured depth value of the first pixel is minimized to generate the first determination parameter
A method of coding a depth image through spherical modeling. 5. The method of claim 4,
Determining a value of the first microcrystalline depth variable from the first determination parameter to generate a first crystal depth value; And
And correcting the measured depth value of the first pixel to the first determined depth value
A method of coding a depth image through spherical modeling. 5. The method of claim 4,
Modeling an object in the object block into an equation of a second sphere represented by a second undetermined parameter;
Determining a value of the second undetermined parameter based on a second undetermined depth parameter of the second pixel and a measured depth value of the second pixel to generate a second determination parameter;
Determining a value of the second uncertainty depth variable from the second decision parameter to generate a second crystal depth value; And
And correcting the measured depth value of the second pixel to the second determined depth value
A method of coding a depth image through spherical modeling. The method according to claim 6,
Wherein the encoding of the target block comprises:
And the target block is encoded based on the predicted depth value and the corrected depth value of the second pixel
A method of coding a depth image through spherical modeling. delete delete 5. The method of claim 4,
Determining a value of the first uncertain parameter that minimizes the error by applying a least squares method
A method of coding a depth image through spherical modeling. delete

Images