KR101904125B1 - VIDEO PROCESSING Device and Method For Depth Video by Concave Curved Surface Modeling - Google Patents

Abstract

According to an embodiment of the present invention, a depth image processing method through concave surface modeling can increase a compression ratio while maintaining the quality of a depth image. The depth image processing method comprises the steps of: modeling an equation of a first curved surface composed of a first coordinate and a parameter on a camera coordinate system; converting the equation of the first curved surface into an equation of a second curved surface composed of a second coordinate on a coordinate system of an image plane onto which the first coordinate is projected, the parameter, and a predicted depth variable; determining a value of the parameter based on the predicted depth variable and a measured depth value of a pixel of a depth image corresponding to the second coordinate, and generating a factor of the curved surface; and determining a value of the predicted depth variable based on the factor of the curved surface and the measured depth value and position information of the pixel of the depth image, and generating a predicted depth value.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for processing a depth image using a concave surface modeling method,

The present invention relates to a method and apparatus for processing depth images through concave surface modeling. And more particularly, to a method and apparatus capable of correcting and / or encoding a depth image by detecting information of the surface in the depth image and modeling information of the surface.

There has been actively studied a method of applying the depth image having the depth information indicating the distance information of the pixels in the image to the image processing using the depth image having the constituent pixel elements. At this time, it is possible to acquire the position information and the distance information of the object, which can not be obtained in the existing color image, using the depth image, and acquire the information of the new object through the acquiring. Due to the characteristics of depth images, new applications using depth images are being studied.

  In this paper, we propose a new method to detect objects in a color image by using depth camera. Using the information of the plane using the distance of the depth image, the image distortion and noise are removed through this. In addition, we study the recognition of the touch in the background area using the distance information of the depth image and to provide various events using it. In addition to this, several studies have been carried out to recognize the face of a person by recognizing the shape of the person.

 Due to the increased application of depth images, the need for depth image coding has increased. First, a method of coding a depth image using a depth lookup table has been studied. In addition, a method of using boundary information of an object for depth image coding has been proposed. A method of coding the depth image by analyzing the depth image based on the histogram has also been proposed. Many studies have been carried out to improve the image coding efficiency by using the feature of depth image. However, the depth image coding study so far has been limited to the method of encoding the depth image in cooperation with the color image, or the supplementary processing of the color image coding.

Korean Patent Publication No. 10-2011-0121003 A1

The present invention can provide a method and apparatus for effectively detecting an object having a concave surface in a depth image and correcting and / or encoding the depth information.

In addition, the present invention can provide a depth image processing method and apparatus capable of increasing the compression ratio while maintaining the quality of the depth image in consideration of the characteristics of the depth image.

In addition, the present invention can provide a depth image processing method and apparatus capable of extracting information on a concave surface area in a depth image and predicting a depth value.

The method of processing a depth image through concave surface modeling according to an embodiment of the present invention includes: modeling an equation of a first curve consisting of a first coordinate and a parameter on a camera coordinate system; Converting the equation of the first curved surface into an equation of a second curved surface consisting of a second coordinate on the coordinate system of the projected image plane and the parameter and the predicted depth variable; Determining a value of the parameter based on the predicted depth variable and a measured depth value of a pixel of the depth image corresponding to the second coordinate and generating a factor of the curved surface; And determining a value of the predicted depth variable based on the position information of the pixel of the depth image and the measured depth value, and generating a predicted depth value.

According to another aspect of the present invention, there is provided a method of processing a depth image through concave surface modeling, the method further comprising: correcting a depth value of pixels of the depth image to the predicted depth value.

According to another aspect of the present invention, there is provided a method of processing a depth image through concave surface modeling, the method further comprising: encoding the depth image based on a difference between the predicted depth value and the measured depth value.

According to another aspect of the present invention, there is provided a method of processing a depth image through concave surface modeling, the method further comprising: encoding the depth image based on a difference between the predicted depth value and the measured depth value; You may.

The step of modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system in a depth image processing method using concave surface modeling according to another embodiment of the present invention includes: The equation of the concave surface satisfying Equation (1) may be modeled based on the measured depth value.

[Equation 1]

X, Y, and Z are first coordinates, and a and b are parameters.

The method of processing a depth image through a concave surface modeling method according to another aspect of the present invention is characterized in that the equation of the first curved surface is transformed to a second coordinate on the coordinate system of the projected image plane, Transforming the first coordinate on the three-dimensional camera coordinate system into the second coordinate on the two-dimensional image plane, and converting the equation (1) into the equation (2).

&Quot; (2) "

h and w are the vertical and horizontal coordinates of the image plane, f is the focal length, and d is the predicted depth variable.

The value of the parameter is calculated on the basis of the predicted depth variable in the depth image processing method through the concave surface modeling according to the embodiment of the present invention and the depth value of the pixel of the depth image corresponding to the second coordinate And generating the parameter of the curved surface may determine the value of the parameter to minimize the difference between the predicted depth variable and the measured depth value.

According to another aspect of the present invention, there is provided a method of processing a depth image through concave surface modeling, the method comprising: determining whether a depth value of pixels is corrected based on an error between the depth value and the depth value, And correcting the depth value of the pixels to the predicted depth value based on the determination result.

In yet another aspect, a non-volatile computer readable medium is a non-volatile computer readable medium storing instructions that is executable by the processor to cause at least one processor to perform operations, : Modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system, and calculating an equation of the first curved surface based on a second coordinate on the coordinate system of the projected image plane, And determining a value of the parameter based on the depth value of the pixel of the depth image corresponding to the predicted depth variable and the second coordinate and generating a factor of the curved surface , The curvature of the curved surface, and the position information of the pixel of the depth image and the measured depth value, Side depth determines the value of the variable, and may provide a non-transient computer-readable medium, comprising: generating a predictive depth value.

In another aspect, the operations of the non-transitory computer-readable medium may include interpolating the depth value of pixels of the depth image to the predicted depth value, or interpolating the depth image based on the difference between the predicted depth value and the measured depth value And < / RTI >

In another aspect, an apparatus for processing depth images through concave surface modeling includes at least one memory; And at least one processor, wherein the at least one processor models an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system, and wherein the equation of the first curved surface is expressed as the projection of the first coordinate A second curve on the coordinate system of the image plane and a second curve comprising the parameter and the predicted depth variable; and based on the depth value of the pixel of the depth image corresponding to the predicted depth variable and the second coordinate, Determining a value of the parameter, generating a factor of the curved surface, determining a value of the predicted depth variable based on the factor of the curved surface, the position information of the pixel of the depth image and the measured depth value, .

The processor of the apparatus for processing a depth image through a concave surface modeling method according to another aspect of the present invention is characterized in that the processor of the apparatus for processing a depth image through a concave surface modeling method includes a step of correcting a depth value of pixels of the depth image by the predicted depth value And may be configured to encode the depth image.

In the embodiment of the present invention, depth information can be represented with a digital code amount as small as possible without losing the quality of the original depth image, while preserving important information included in the original depth image while removing other information.

In the embodiment of the present invention, the depth information of the depth image including the curved surface is predicted by finding the curved surface information in the depth image using the depth value in the block, and the depth information can be corrected using the predicted depth value The accuracy of the correction to the depth value is improved.

In addition, according to the embodiment of the present invention, when coding a curved surface object, a higher coding efficiency than the conventional coding method is obtained.

1 is a block diagram illustrating a depth image processing apparatus configured to process a depth image according to an exemplary embodiment of the present invention.
2 is an exemplary diagram illustrating a method of processing depth images through concave surface modeling according to an embodiment of the present invention.
3 is an exemplary view showing the relationship between the concave surface on the camera coordinate system and the image plane on which the camera coordinates are projected.
4 is an exemplary diagram illustrating a method of processing a depth image through concave surface modeling according to another embodiment of the present invention.
5 is an exemplary diagram illustrating a method of processing a depth image through concave surface modeling according to another embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments described below, but may be implemented in various forms. In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. Also, the singular expressions include plural expressions unless the context clearly dictates otherwise. Also, the terms include, including, etc. mean that there is a feature, or element, recited in the specification and does not preclude the possibility that one or more other features or components may be added. Also, in the drawings, for convenience of explanation, the components may be exaggerated or reduced in size. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

1 is a block diagram illustrating a depth image processing apparatus configured to process a depth image according to an exemplary embodiment of the present invention.

The depth image processing apparatus 100 may provide image data to the data receiving apparatus 200. [

The depth image processing apparatus 100 and the data receiving apparatus 200 may be used in various applications such as desktop computers, notebook (i.e., laptop) computers, tablet computers, set top boxes, telephone handsets such as so- Quot; may include any of a wide variety of devices including pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like.

In some implementations, the depth image processing device 100 and the data receiving device 200 may be provided with a configuration 10 for wireless communication.

Also, the data receiving apparatus 200 may receive the image data processed through the computer readable medium.

The computer readable medium may include any type of media or device capable of moving image data processed by the depth image processing apparatus 100 to the data receiving apparatus 200. In one example, the computer readable medium may include a communications medium, such as a transmit channel, that enables depth image processing apparatus 100 to transmit image data directly to data receiving apparatus 200 in real time.

The processed image data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the data receiving apparatus 200. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency spectrum or one or more physical transmission lines. The communication medium may form part of a packet based network, such as a global network such as a local area network, a wide area network, or the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the depth imaging device 100 to the data receiving device 200. In some instances, image processed image data may be output from the output interface 130 to a computer readable storage medium, such as a non-transitory computer readable storage medium, i.e., a data storage device. Similarly, the image data may be accessed from the storage device by the input interface 230 of the data receiving apparatus 200. [ The storage device may be a variety of distributed or removable media such as hard drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing image data. And non-volatile data storage media that are locally accessed. In a further example, the storage device may correspond to a file server or other intermediate storage device that may store image data generated by the depth image processing device 100. [

The data receiving apparatus 200 may access the stored image data from the storage device through streaming or downloading.

In the example of FIG. 1, the depth image processing apparatus 100 may include an image source 110 and an image processing unit 120. In addition, the depth image processing apparatus 100 may further include an output interface 130.

The data receiving apparatus 200 may include an input interface 230 and a data processing unit 220. In addition, the data receiving apparatus 200 may further include a display device 210.

In another example, the depth image processing apparatus 100 and the data processing unit 220 may include other components.

For example, the depth image processing apparatus 100 may receive an external video source, such as an image from an external camera, and the external camera may be a depth image capturing device that generates a depth image. Likewise, the data receiving device 200 may interface with the external display device 210 rather than with the integrated display device 210.

The image source 110 of the depth image processing apparatus 100 may include a depth image capture device, such as a camera, an archive that includes previously captured depth images, and / or a depth image from a depth image content provider Interface.

In some embodiments, the depth imaging device may provide a depth image that represents scene depth information in 256-level 8-bit images or the like. The number of bits for representing one pixel of the depth image can be changed instead of 8 bits. The depth image capturing device can provide a depth image having a value proportional to or inversely proportional to the distance by measuring the distance from the depth image capturing device to the object and the background using an infrared ray or the like.

The pixel value of the depth image may be, for example, depth information in the form of integers in mm, but not limited to, RGB color information, for example.

Each of the depth image processing apparatus 100 and the data receiving apparatus 200 may include one or more memories and one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof.

The memory includes instructions (e.g., executable instructions) such as computer readable instructions or processor readable instructions. The instructions may include one or more instructions executable by the computer, such as by each of the one or more processors.

For example, the one or more instructions may be executed by one or more processors for performing operations, including processing depth images to correct depth values of depth images and / or to encode depth images, You may.

In particular, the image processing unit 120 may include one or more memories 121 for storing instructions and at least one processor 122 for executing the instructions.

The processor 122 of the image processing unit 120 may be configured to apply the techniques for correcting the depth value of the depth image.

In some implementations, the processor 122 of the image processing unit 120 may be configured to apply techniques for encoding depth images.

In another embodiment, the processor 122 of the image processing unit 120 may be configured to apply a technique of correcting a depth value of a depth image and a technique of encoding a depth image.

The data processing unit 220 may be configured to transmit image data from the image processing unit 120 to an external device, perform display, analysis, and the like.

In some implementations, the data processing unit 220 may be configured to decode the encoded image data from the image processing unit 120.

Although not shown in FIG. 1, in some embodiments, the depth image processing apparatus 100 and the data processing apparatus may be an integrated apparatus. For example, the depth image processing apparatus 100 may be configured to encode the depth image and to decode the encoded image data.

2 is an exemplary diagram illustrating a method of processing depth images through concave surface modeling according to an embodiment of the present invention. 3 is an exemplary view showing the relationship between the concave surface on the camera coordinate system and the image plane on which the camera coordinate is projected.

Referring to FIGS. 2 and 3, a depth image processing method using concave surface modeling includes a step S110 of modeling an equation of a first curved surface composed of a first coordinate and a parameter on a camera coordinate system (S110), an equation of a first curved surface (S120) of transforming the first coordinate into a second curved surface equation consisting of a second coordinate on the coordinate system of the projected image plane and a parameter and a predictive depth variable, Determining a value of the parameter based on the measured depth value of the depth image and generating a factor of the curved surface (S130); determining a value of the predicted depth variable based on the surface information and the measured depth value And generating a predicted depth value (S140).

Also, in some embodiments, the method of processing the depth image through the concave surface modeling may further include an image processing step (S150), wherein the image processing step (S150) (S151) of correcting the depth value and / or encoding the depth image based on the difference between the predicted depth value and the measured depth value (S152).

4 is an exemplary diagram illustrating a method of processing a depth image through concave surface modeling according to another embodiment of the present invention.

According to another embodiment, a depth image processing method using concave surface modeling includes dividing a depth image into a predetermined block unit (S210), detecting shape information of an object based on depth values of pixels in the object block (S230) of modeling an equation of a first curved surface made up of a first coordinate and a parameter on the camera coordinate system (S230), calculating an equation of the first curved surface by using the second coordinate on the coordinate system of the projected image plane (S240) of transforming the predicted depth variable into an equation of a second curved surface composed of a variable and a predictive depth variable, determining the value of the parameter based on the predicted depth variable and the measured depth value of the pixel of the object block corresponding to the second coordinate, A step of generating an argument (S250) and a step of determining a value of the predicted depth variable based on the printing of the curved surface, the position information of the pixel of the target block and the measured depth value, It may include a system (S260).

Also, in some embodiments, the method of processing the depth image through the concave surface modeling may further include image processing (S270), and the image processing step (S270) (Step S271) of encoding the depth image and / or encoding the depth image based on the difference between the predicted depth value and the measured depth value (S272).

Each step of FIG. 4 will now be described in more detail.

The processor 122 may divide the depth image into a predetermined block unit (S210).

For example, the depth image can be divided into blocks of m * n. Each of the plurality of blocks can be defined as an area made up of m * n (m, n is a natural number) pixels.

For example, a block of m * n can be defined as an area consisting of 8 * 8 pixels or 16 * 16 pixels, and when the resolution is increased, the basic unit can be defined as an area consisting of 32 * 32 or 64 * 64 pixels. However, the present invention is not limited thereto, and the processor 122 may correct and / or code the depth value in the depth image without dividing the depth image as in the embodiment of FIG.

The processor 122 may detect the shape information of the object based on the pixel values in each block (S220). For example, the processor 122 may detect the shape information of the object based on the depth value of the pixel in the block and the adjacent pixels of the pixel. Each of the pixels has a depth value, and can detect the shape information of the object based on a relative depth difference between the pixels, and a distribution of depth values of pixels around the reference pixel and the reference pixel.

The processor 122 may determine that the shape of the object has a shape of either a plane, a spherical surface, or a curved surface based on pixel values in each block, and the curved surface may be a concave surface, but the present invention is not limited thereto .

For example, an arbitrary point of an object on a camera coordinate system has (X, Y, Z) coordinates, where (X, Y) is projected onto a point having (w, h) Is an ideal depth value in (w, h) coordinates, and the ideal depth value can be expressed in each pixel unit. If the depth value Z gradually decreases gradually from an arbitrary pixel to a neighboring pixel, it can be seen that at least a part of the object has a concave surface. Accordingly, the processor 122 can detect the shape information of the object based on the coordinates of each pixel and the depth value of each pixel.

In addition, the processor 122 may model the equation of the first curved surface composed of the first coordinates (X, Y, Z) on the camera coordinate system and the parameters (a, b) (S230). For example, the equation of the first curved surface may be a concave surface equation as shown in Equation (1).

[Equation 1]

X, Y and Z are first coordinates, which are arbitrary points on the camera coordinate system in the three-dimensional space, and a and b are parameters constituting the equation of the first surface of the parameter.

However, if the world coordinate system, which is a three-dimensional coordinate system, and the camera coordinate system, which is a three-dimensional coordinate system, are inconsistent with each other before the steps described in S230 are performed, the world coordinate system is converted into a three- Coordinate transformation may be performed.

In addition, the processor 122 calculates the equation of the first curved surface of Equation 1 by using the second coordinates (h, w) on the coordinate system of the image plane in which the first coordinates (X, Y, Z) ) And the predicted depth variable (d) (S240).

&Quot; (2) "

h and w are the vertical and horizontal coordinates of the image plane, f is the focal length, and d is the predicted depth variable.

However, before performing the steps described in S240, if the centers of the camera coordinate system and the image plane coordinate system do not coincide with each other, a coordinate transformation may be performed to match the center positions.

The processor 122 may also determine the value of the parameter a, b based on the predicted depth variable d and the measured depth value of the pixel in the object block corresponding to the second coordinate h, w S250). And the parameters (values a and b) are the parameters of the curved surface.

의 차이(수학식 4에 따라)가 최소가 되도록 하는 매개변수(a, b)의 값을 결정할 수 있다.For example, the processor 122 may calculate the predicted depth variable d and the measured depth value d in accordance with Equation (3)

(A, b) that minimizes the difference (in accordance with equation (4)) of the parameters a and b.

&Quot; (3) "

In Equation (3), d is the predicted depth variable, and when the values of the parameters a and b of the equation of the second curved surface of Equation 2 are determined, the ideal depth value of the pixels corresponding to the second coordinates of the image plane have. However, at this stage, the value of d is not determined and is therefore defined as the predicted depth variable.

&Quot; (4) "

의 차이(

은 오차)가 제일 작을 때의 매개변수 a, b를 구함으로써 대상 블록에서 최적의 곡면 방정식을 구하고 예측깊이변수 d의 값을 결정할 수 있다. 이 때 모델링된 표면의 (h, w)에서의 근사된 곡면의 예측깊이변수 d와 실제로 측정된 깊이

의 오차

가 최소가 되도록 인자를 결정하기 위해 최소자승법을 적용할 수 있다. 이 때

는 매개변수에 대해 비선형식으로 나타나므로 가우스-뉴턴법을 적용할 수 있다.As shown in equation (4), the depth value actually measured in the predicted depth variable d and (h, w)

Difference between

The optimal curved surface equation can be obtained in the target block and the value of the predicted depth variable d can be determined by obtaining the parameters a and b when the error is smallest. At this time, the predicted depth variable d of the approximated curved surface at (h, w) of the modeled surface and the actually measured depth

Error of

The least squares method can be applied to determine the factor to be the minimum. At this time

Gauss-Newton method can be applied since it appears in a non-linear form with respect to the parameter.

의 차로 이루어진 행렬

과

에서의 자코비안 행렬

, 미결정매개변수 값을 나타내는

은 수학식 5를 충족한다.In the Gauss-Newton method, at step n, the value of d and the actually measured depth value

A matrix consisting of

and

Jacobian procession in

, Indicating the value of the indeterminate parameter

Satisfies the expression (5).

&Quot; (5) "

을 구할 수 있다. In addition, by applying the Gauss-Newton method, as shown in Equation 6,

Can be obtained.

&Quot; (6) "

의 이상적인 깊이 값 d(예측깊이값)를 결정할 수 있다(S260).The processor 122 can repeat the above-described operation P times to obtain a curved surface having a given depth value and the closest curved surface, and determine the value of the parameter of the equation of the first curved surface. Then, the processor 122 substitutes the determined factor, the coordinate values of the pixels in the object block, and the focal length f into Equation (2)

An ideal depth value d (predicted depth value) of the target pixel can be determined (S260).

의 측정된 깊이 값을 이상적인 깊이 값(결정깊이값)으로 보정할 수 있다(S271).Also, in the image processing step S270, the processor 122 sets the correction target pixel

Can be corrected to an ideal depth value (crystal depth value) (S271).

에서의

와 모델링 과정을 수행하여 찾은 매개변수에서의 깊이 값

(결정깊이값)의 차이를 이용하여 각 화소를 부호화할 수 있다. 그 후 부호화된 블록과 곡면의 매개변수 값을 함께 부호화하여 전체 깊이 영상을 부호화할 수도 있다.Further, in the image processing step S270, the processor 122 determines whether or not each pixel

In

And the depth value of the parameter found by performing the modeling process

(The depth value of the crystal). Then, the parameter values of the encoded block and the curved surface may be encoded together to encode the entire depth image.

The embodiment has an effect that the entropy power is reduced and the coding efficiency is increased as compared with the case where coding is performed through the conventional DPCM (Differential Pulse Code Modulation).

5 is an exemplary diagram illustrating a method of processing a depth image through concave surface modeling according to another embodiment of the present invention.

Referring to FIG. 5, in still another embodiment, a depth image processing method using concave surface modeling includes dividing a depth image into units of predetermined blocks (S310), calculating a shape of the object based on depth values of pixels in the object block (S330) of modeling an equation of a first curved surface made up of a first coordinate and a parameter on the camera coordinate system (S330), calculating an equation of the first curved surface on the coordinate system of the projected image plane (S340) of transforming the second depth value into a second surface equation consisting of a second coordinate, a parameter, and a predictive depth variable, calculating a value of the parameter based on the depth value of the target block corresponding to the predicted depth variable and the second coordinate (S350), determining the value of the predicted depth variable based on the curved surface, the position information of the pixel of the target block and the measured depth value, and determining the predicted depth value (S360), and determines whether the depth value of the pixels in the target block is corrected based on the error between the predicted depth variable and the measured depth value and the size of the block, and corrects the depth value of the pixels based on the determination result Step S370.

)의 오차(

) 및 블록의 크기에 기초하여 대상 블록 내 화소들의 깊이 값의 보정 여부를 판단하고, 판단 결과에 기초하여 화소들의 깊이 값을 보정하는 단계(S370)는 프로세서(122)가 오차(

)에 대상 블록의 크기를 나누는 연산을 수행하여 평가치를 생성하는 단계(S371), 프로세서(122)가 평가치와 기 설정된 임계값(T)과 비교하는 단계(S372) 및 프로세서(122)는 평가치가 임계값(T)보다 작은 경우 대상 블록 내의 화소들의 깊이 값을 예측깊이값으로 보정하고, 평가치가 임계값(T) 이상 경우 대상 블록 내의 화소들의 깊이 값을 예측깊이값으로 보정하지 않는 단계(S373)를 포함할 수 있다.In addition, if the processor 122 determines that the predicted depth variable d and the measured depth value < RTI ID = 0.0 >

) Of the error

(Step S370) of determining whether the depth value of the pixels in the target block is corrected based on the size of the block and the depth value of the pixels based on the determination result,

(Step S371). The processor 122 compares the evaluation value with a predetermined threshold value T (step S372), and the processor 122 evaluates the evaluation value Correcting the depth value of the pixels in the target block to a predicted depth value when the value is smaller than the threshold value T and not correcting the depth value of the pixels in the target block to the predicted depth value when the evaluated value is equal to or greater than the threshold value T S373).

)에 깊이 영상 자체의 크기를 나누는 연산을 수행하여 평가치를 생성할 수도 있다.In some embodiments, if the depth image is not to be partitioned into blocks,

) By dividing the size of the depth image itself.

The embodiment can improve the accuracy of the depth value correction by evaluating the concave surface modeling and correcting the depth value of the pixels when the modeling performance is appropriate.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

While the present invention has been described in connection with what is presently considered to be practical and exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

100: Depth image processing device
110: image source
120:
121: Memory
122: Processor
130: Output interface
200: Data receiving device
210: Display Device
220:
230: Input Interface

Claims (12)

Modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system;
Converting the equation of the first curved surface into an equation of a second curved surface consisting of a second coordinate on the coordinate system of the projected image plane and the parameter and the predicted depth variable;
Determining a value of the parameter based on the predicted depth variable and a measured depth value of a pixel of the depth image corresponding to the second coordinate and generating a factor of the curved surface; And
Determining a value of the predicted depth variable based on the position of the pixel of the depth image and the measured depth value and generating a predicted depth value;
A method for processing depth images through concave surface modeling. The method according to claim 1,
And correcting the depth value of the pixels of the depth image to the predicted depth value
A method for processing depth images through concave surface modeling. The method according to claim 1,
And encoding the depth image based on a difference between the predicted depth value and the measured depth value
A method for processing depth images through concave surface modeling. 3. The method of claim 2,
And encoding the depth image based on a factor of the curved surface, a difference between the predicted depth value and the measured depth value
A method for processing depth images through concave surface modeling. The method according to claim 1,
Modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system,
Modeling the concave surface equation satisfying the equation (1) based on the position information of the depth image and the measured depth value
A method for processing depth images through concave surface modeling.
[Equation 1]

X, Y, and Z are first coordinates, and a and b are parameters. 6. The method of claim 5,
Converting the equation of the first curved surface into an equation of a second curved surface consisting of the second coordinate on the coordinate system of the projected image plane and the parameter and the predicted depth variable,
Transforming the first coordinate on the three-dimensional camera coordinate system into the second coordinate on the two-dimensional image plane, and converting the equation (1) into the equation (2)
A method for processing depth images through concave surface modeling.
&Quot; (2) "

h and w are the vertical and horizontal coordinates of the image plane, f is the focal length, and d is the predicted depth variable. The method according to claim 1,
Determining the value of the parameter based on the predicted depth variable and the measured depth value of the pixel of the depth image corresponding to the second coordinate and generating the parameter of the curved surface,
Determining a value of the parameter that minimizes the difference between the predicted depth variable and the measured depth value
A method for processing depth images through concave surface modeling. The method according to claim 1,
Determining whether the depth value of the pixels is corrected based on the error between the predicted depth variable and the measured depth value and the size of the depth image, and correcting the depth value of the pixels to the predicted depth value based on the determination result; Further comprising
A method for processing depth images through concave surface modeling. 17. A non-transitory computer readable medium storing instructions,
Wherein the instructions are executable by the processor to cause at least one processor to perform operations,
The operations include:
Modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system,
Transforming the equation of the first curved surface into an equation of a second curved surface composed of a second coordinate on the coordinate system of the projected image plane and the parameter and the predicted depth variable,
Determining a value of the parameter based on the predicted depth variable and a measured depth value of a pixel of the depth image corresponding to the second coordinate, generating a factor of the curved surface,
Determining a value of the predicted depth variable based on the parameter of the curved surface, the position information of the pixel of the depth image and the measured depth value, and generating a predicted depth value
Non-transitory computer readable medium. 10. The method of claim 9,
The operations include,
The depth value of the pixels of the depth image is corrected to the predicted depth value or
And encoding the depth image based on a difference between the predicted depth value and the measured depth value
Non-transitory computer readable medium. At least one memory; And
At least one processor,
The at least one processor
Modeling an equation of a first curved surface consisting of a first coordinate and a parameter on a camera coordinate system,
Transforming the equation of the first curved surface into an equation of a second curved surface composed of a second coordinate on the coordinate system of the projected image plane and the parameter and the predicted depth variable,
Determining a value of the parameter based on the predicted depth variable and a measured depth value of a pixel of the depth image corresponding to the second coordinate, generating a factor of the curved surface,
And determining a value of the predicted depth variable based on position information of the pixel of the depth image and a measured depth value and generating a predicted depth value
An apparatus for processing depth images through concave surface modeling. 12. The method of claim 11,
The processor comprising:
The depth value of the pixels of the depth image is corrected to the predicted depth value or
And to code the depth image based on a difference between the prediction depth value and the measurement depth value
An apparatus for processing depth images through concave surface modeling.

Images