KR102224321B1 - Coding Method and Device for Depth Video Plane Modeling - Google Patents

Abstract

According to an embodiment of the present invention, provided is a depth image encoding method through plane modeling. The depth image encoding method through plane modeling comprises the steps of: dividing a depth image into preset blocks; converting image coordinates for pixels in the depth image into three-dimensional coordinates; modeling a block with a surface model based on the three-dimensional coordinates; generating an estimated depth value using a plane model; and encoding the block based on the estimated depth value and a plane coefficient of the plane model by varying precision of the plane coefficient. Accordingly, according to an embodiment of the present invention, the compression rate can be increased while maintaining the quality of the depth image in consideration of characteristics of the depth image.

Description

Depth Video Plane Modeling Method and Device for Depth Video Plane Modeling

The present invention relates to a method and apparatus for encoding a depth image through plane modeling. More specifically, it relates to a method and apparatus capable of modeling a plane of an object in a depth image and encoding coefficients of a plane equation of the plane model.

Research is being actively conducted on a method of applying to image processing using a depth image having depth information representing distance information of a pixel in an image as a component pixel element. In this case, by using the depth image, position information and distance information of the object, which could not be obtained in the existing color image, may be acquired, and information on a new object may be obtained through the acquisition. Due to the characteristics of such depth images, research on new applications using depth images is expanding.

A study has been made to perform more accurate object detection by overcoming the disadvantage of being sensitive to the environment in color images and the existence of different color information in the object through a depth camera. A study was also conducted to remove distortion and noise of the image by using the information of the plane by using the distance of the depth image. In addition, a study on recognizing a touch in a background area using distance information of a depth image, and a study on providing various events using the same have been conducted. In addition, several studies have been conducted to recognize a person's face by recognizing the shape of a person.

Due to the expansion of the application fields of such depth images, the need for encoding depth images has increased. First, a method of encoding a depth image using a depth lookup table was studied. In addition, a method of using the boundary information of an object for depth image encoding has been proposed. A method of encoding the depth image by analyzing the depth image based on the histogram was also proposed. As such, many studies have been conducted to increase the efficiency of image encoding by using the features appearing in the depth image. However, studies on depth image encoding conducted so far have been limited to methods of encoding a depth image in conjunction with a color image or in connection with a color image.

Korean Registered Patent Publication 10-1817140 B1

The present invention can provide a method and apparatus capable of modeling a plane in a depth image, predicting depth using a plane model, and encoding a depth image using depth prediction and coefficients of the plane model.

In addition, the present invention can provide a depth image encoding method and apparatus through plane modeling capable of increasing a compression rate while maintaining the quality of the depth image in consideration of the characteristics of the depth image.

In addition, the present invention can provide a method and apparatus for encoding a depth image through plane modeling capable of increasing a compression rate while maintaining the quality of a depth image by varying the precision of coefficients of a plane model.

In addition, the present invention can provide a method and apparatus for encoding a depth image through plane modeling capable of selectively changing and encoding a coefficient of a plane model according to a depth image.

In addition, the present invention can provide a depth image encoding method and apparatus through plane modeling capable of encoding a depth image using a detailed and sophisticated plane model by modeling a plane in a depth image in a local unit.

A method for encoding a depth image through plane modeling according to an embodiment includes: dividing the depth image into preset block units; Converting image coordinates of pixels in the depth image into 3D coordinates; Modeling the block with a surface model based on the three-dimensional coordinates; Generating a predicted depth value using a planar model; And encoding the block by varying the precision of the plane coefficient based on the prediction depth value and the plane coefficient of the plane model.

In addition, in the depth image encoding method through plane modeling, the plane model may be modeled using a first plane equation of Equation 1 consisting of 3D coordinates and parameters.

[Equation 1]

ax+by-z+c=0

x, y, and z are three-dimensional coordinates, and a, b, and c are plane coefficients.

In addition, the depth image encoding method through planar modeling may include varying the precision of the plane coefficient to encode the block, comprising: varying the precision of the plane coefficient; And entropy encoding the variable plane coefficients. Including, the step of varying the precision of the plane coefficient, the plane coefficients a, b, and c may be converted into a fixed-point form.

In addition, in the depth image encoding method through plane modeling, the step of varying the precision of the plane coefficient may include converting the plane coefficients a and b into a fixed-point form with one decimal place and transforming the plane coefficient c into an integer form can do.

In addition, in the depth image encoding method through plane modeling, the entropy encoding may be an arithmetic encoding algorithm.

In addition, in the depth image encoding method through plane modeling, the predicted depth value may be obtained using a second plane equation of Equation 2 consisting of an image coordinate and a focal length.

[Equation 2]

은 제1 추정깊이 값이고, f는 초점거리이고, a, b, c는 매개변수이고, w, h는 영상좌표이다.

Is the first estimated depth value, f is the focal length, a, b, c are parameters, and w, h are image coordinates.

According to an embodiment, an apparatus for encoding a depth image through plane modeling includes: at least one memory for storing commands; And at least one processor; wherein the instructions are executable by the processor to cause the processor to perform operations, and the operations include: dividing a depth image into preset block units, and Convert the image coordinates of to 3D coordinates, model the block as a surface model based on the 3D coordinates, generate a predicted depth value using a plane model, and generate the predicted depth value and the plane of the planar model It may include encoding the block by varying the precision of the plane coefficient based on the coefficient.

The non-transitory computer-readable medium according to the embodiment is a non-transitory computer-readable medium that stores the instructions according to an apparatus for encoding a depth image through plane modeling.

According to an embodiment of the present invention, a plane within a depth image may be modeled, a depth may be predicted using a plane model, and a depth image may be encoded using a depth prediction and coefficients of the plane model.

In addition, according to an exemplary embodiment of the present invention, while maintaining the quality of the depth image in consideration of the characteristics of the depth image, it is possible to increase the compression rate.

In addition, according to an exemplary embodiment of the present invention, the compression ratio can be increased while maintaining the quality of the depth image by varying the precision of the coefficients of the planar model.

In addition, according to an exemplary embodiment of the present invention, the precision of coefficients of the planar model may be selectively changed and coded according to the depth image.

In addition, according to an embodiment of the present invention, a plane within a depth image may be modeled in a local unit, and a depth image may be encoded using a detailed and sophisticated plane model.

1 is a block diagram showing a depth image processing apparatus according to the present invention.
2 is a flowchart of a depth image encoding method through plane modeling according to an embodiment of the present invention.
3 shows that a point on the camera coordinate system is projected onto the image plane.
4 is an exemplary diagram for a first plane modeling method.
5 is a flowchart illustrating a method of encoding a target block of FIG. 2.
6 is a flowchart illustrating a method of varying the precision of the plane coefficient of FIG. 5.
7 is an experimental example for explaining intra-screen prediction accuracy according to a change in the precision of a plane coefficient.
8 is an exemplary diagram for a planar modeling method using a local plane.
9 is a flowchart of a method of generating the local plane of FIG. 8.
FIG. 10 shows a state in which pixels in a depth image are transformed into a 3D coordinate system in the method of generating a local plane of FIG. 9.
11 is a flowchart illustrating a method of detecting a feature of a local plane in the local plane of FIG. 9.
12 is a three-dimensional coordinate system for describing a normal vector that is a feature of the local plane of FIG. 8.
13 is a flowchart illustrating a method of calculating the similarity between local planes of FIG. 8.
14 is a flowchart illustrating a planar modeling method through grouping according to the similarity between local planes of FIG. 8.
15 is a flow chart for another method of generating the local plane of FIG. 8.
16 is a flowchart illustrating a method of detecting a feature of a local plane of FIG. 15.
FIG. 17 is an example of a plurality of local vectors in a local plane for obtaining an average local vector of the local plane of FIG. 15.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one constituent element from other constituent elements rather than a limiting meaning. In addition, expressions in the singular include plural expressions unless the context clearly indicates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility of adding one or more other features or components in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, and thus the present invention is not necessarily limited to what is shown.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding constituent elements are assigned the same reference numerals, and redundant descriptions thereof will be omitted. .

1 is a block diagram showing a depth image processing apparatus according to the present invention.

The depth image processing apparatus 100 may provide image data to the data receiving apparatus 200.

The depth image processing apparatus 100 and the data receiving apparatus 200 are desktop computers, notebook (ie, laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" phones. "May include any of a wide variety of devices including pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like.

In some implementations, the depth image processing apparatus 100 and the data receiving apparatus 200 may be equipped with a configuration 10 for wireless communication.

In addition, the data receiving apparatus 200 may receive image-processed image data through a computer-readable medium.

The computer-readable medium may include any type of medium or device capable of moving image data processed from the depth image processing apparatus 100 to the data receiving apparatus 200. For example, the computer-readable medium may include a communication medium, such as a transmission channel, which enables the depth image processing apparatus 100 to directly transmit image data to the data receiving apparatus 200 in real time.

The image-processed image data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the data receiving apparatus 200. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the depth image processing apparatus 100 to the data receiving apparatus 200. In some examples, the image-processed image data may be output from the output interface 130 to a computer-readable storage medium, such as a non-transitory computer-readable storage medium, that is, a data storage device. Similarly, image data may be accessed from the storage device by the input interface 220 of the data receiving apparatus 200. The storage device can be any of a variety of distributed or distributed digital storage media, such as a hard drive, Blu-ray disks, DVDs, CD-ROMs, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing image data. It may include any of the non-transitory data storage media that are accessed locally. In a further example, the storage device may correspond to a file server or other intermediate storage device that may store image data generated by the depth image processing apparatus 100.

The data receiving apparatus 200 may access the stored image data from the storage device through streaming or download.

In the example of FIG. 1, the depth image processing apparatus 100 may include an image source 110 and an image processor 120. In addition, the depth image processing apparatus 100 may further include an output interface 130.

The data receiving device 200 may include an input interface 220 and a data processing unit 210.

In another example, the depth image processing apparatus 100 and the data processing unit 210 may include different components.

For example, the depth image processing apparatus 100 may receive an image from an external video source, such as an external camera, and the external camera may be a depth image photographing device that generates a depth image.

The image source 110 of the depth image processing apparatus 100 receives a depth image from a depth image photographing device, such as a camera, an archive including a previously photographed depth image, and/or a depth image from a depth image content provider. It may also include an interface.

In some implementations, the depth image capturing device may provide a depth image in which depth information of a scene is expressed in an integer data type of 16 bits per pixel (pixel). The number of bits for representing one pixel of the depth image may be changed. The depth image capturing device may provide a depth image having a value proportional or inversely proportional to the distance by measuring a distance from the depth image capturing device to an object and a background using infrared rays or the like.

The pixel value of the depth image may be, for example, not color information of RGB, but depth information of an integer of mm unit (but not limited thereto).

Each of the depth image processing apparatus 100 and the data receiving apparatus 200 includes one or more memories, one or more microprocessors, digital signal processors (DSPs), custom integrated circuits (ASICs), field programmable gate arrays. (FPGAs), individual logic circuits, software, hardware, firmware, or any combination thereof.

The memory contains instructions (eg, executable instructions) such as computer readable instructions or processor readable instructions. Instructions may include one or more instructions executable by a computer, such as by each of one or more processors.

For example, the one or more instructions may be executable by one or more processors to cause the one or more processors to perform operations including processing the depth image to encode the depth image.

In detail, the image processing unit 120 may include one or more memories 121 for storing instructions and at least one processor 122 for executing the instructions.

The processor 122 of the image processing unit 120 may be configured to apply techniques for modeling a plane in the depth image.

In some implementations, the processor 122 of the image processing unit 120 may be configured to apply techniques for predicting a depth value of a depth image.

In some implementations, the processor 122 of the image processing unit 120 may be configured to apply techniques for encoding a depth image.

In another embodiment, the processor 122 of the image processing unit 120 may be configured to apply a technique of encoding and decoding a depth image.

The data processing unit 210 may be configured to transmit, display, and analyze the depth image data from the image processing unit 120 to an external device.

In some implementations, the data processing unit 220 may be configured to decode the encoded depth image data from the image processing unit 120.

Although not shown in FIG. 1, in some embodiments, the depth image processing apparatus 100 may be a device integrated with an encoding apparatus. In some embodiments, the depth image processing apparatus 100 and the data processing apparatus 200 may be integrated devices. For example, the depth image processing apparatus 100 may be configured to encode a depth screen and decode the encoded depth screen.

2 is a flowchart of a method for encoding a depth image through plane modeling according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating that a point on a camera coordinate system is projected onto an image plane, and FIG. 4 is a diagram for a first plane modeling method. FIG. 5 is an exemplary diagram, and FIG. 5 is a flow chart for the method of encoding the target block of FIG. 2, FIG. 6 is a flow chart for a method of varying the precision of the plane coefficients of FIG. 5, and FIG. 7 is This is an experimental example to explain my prediction accuracy.

Referring to FIG. 2, in the method of intra prediction encoding of a depth image according to an embodiment of the present invention, the processor 122 of the depth image processing apparatus 100 divides the depth image into preset block units (S100). , The processor 122 of the depth image processing apparatus 100 converts the image coordinates into 3D coordinates (S200), and the processor 122 of the depth image processing apparatus 100 converts the image coordinates into a plane model based on the 3D coordinates. Modeling (S300), the processor 122 of the depth image processing apparatus 100 generates a predicted depth value based on the planar model (S400), the processor 122 of the depth image processing apparatus 100 is a block Encoding (S500), the processor 122 of the depth image processing apparatus 100 may include combining all the encoded blocks to encode the depth image (S600). Each step of a method for encoding a depth image through plane modeling according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 to 17.

<Block division>

The processor 122 of the depth image processing apparatus 100 may divide the depth image into preset block units (S100).

For example, the depth image may be divided into blocks of e*f. Each of the plurality of blocks may be defined as an area composed of e*f (e and f are natural numbers) pixels.

For example, an e*f block can be defined as an area consisting of 8*8 pixels or 16*16 pixels, and when the resolution increases, the basic unit can be defined as an area consisting of 32*32 or 64*64 pixels.

<Coordinate conversion>

The processor 122 of the depth image processing apparatus 100 may convert the image coordinates into 3D coordinates (S200). More specifically, referring to FIG. 3, an arbitrary point of an object in the camera coordinate system has (x, y, z) coordinates and is viewed as a three-dimensional coordinate. Here, (x, y, z) is projected to the image coordinate, which is a point with the (w, h) coordinate of the image plane, and the z value becomes the depth value d at the (w, h) coordinate. In addition, the depth value may be expressed in units of each pixel. Specifically, a screen captured when capturing a depth image is projected onto an image plane in which points in the capturing area are virtual planes. And if a point (x, y, z) in the camera coordinate system is projected to a point (w, h) in the image plane with the center as the origin, the three-dimensional coordinates (x, y, z) and the image coordinates (w, h) The relationship between) satisfies Equation 1.

[Equation 1]

x=dw/f

y=dh/f

z=d

In Equation 1, f is the focal length of the camera and is defined as the distance between the origin and the image plane in the camera coordinate system.

<Surface modeling and predicted depth value>

The processor 122 of the depth image processing apparatus 100 may model a plane model based on 3D coordinates (S300). Also, the processor 122 of the depth image processing apparatus 100 may generate a predicted depth value based on the planar model (S400).

More specifically, as an example, referring to FIG. 4, the plane model may be modeled using the first plane equation according to Equation 2.

[Equation 2]

ax+by-z+c=0

In Equation 2, a, b, and c are plane coefficients. Any one point of the object in the camera coordinate system has (x, y, z) coordinates, where (x, y) is projected to a point with (w, h) coordinates of the image plane, and the z value is (w, h) It becomes an ideal depth value at the coordinates, and the corresponding ideal depth value can be expressed in units of each pixel. The image coordinates (w, h) may be pixels in which noise does not occur in a block containing noise. In addition, the modeled plane model can be represented by the determinant of Equation 3. That is, the modeled plane model may be represented by a matrix related to the three-dimensional coordinates of matrices A and B in Equation 4 and modeled plane coefficients of the matrix R.

[Equation 3]

AR=B

[Equation 4]

In addition, a matrix R composed of plane coefficients can be obtained using Equation 5.

[Equation 5]

)으로 이루어진 수학식 6의 평면의 제2 평면 방정식으로 변환할 수 있다.The processor 122 converts the first plane equation of the plane of Equation 2 into second coordinates (w, h) and plane coefficients (a, b, and) on the coordinate system of the image plane on which the first coordinates (x, y, z) are projected. c) and the predicted depth value (

It can be converted into a second plane equation of the plane of Equation 6 consisting of ).

[Equation 6]

는 평면의 예측깊이 값이다.h, w are the vertical and horizontal coordinates of the image plane, f is the focal length,

Is the predicted depth value of the plane.

)은 수학식 6에 의해 구할 수 있다.In other words, the predicted depth value of the plane (

) Can be obtained by Equation 6.

)은 평면의 임의의 (w, h) 좌표를 가진 점과 수학식 6을 이용하여 구할 수 있다.As another example, the planar model may be modeled using the local plane described in FIGS. 8 to 17. A method of generating a planar model using a local plane will be described later. Also, the predicted depth value (

) Can be obtained using a point with arbitrary (w, h) coordinates of the plane and Equation 6.

<block coding>

The processor 122 of the depth image processing apparatus 100 may encode the block (S500).

More specifically, the processor 122 may encode the target block based on the prediction depth value generated using the planar model and the plane coefficient of the planar model. For example, in the case of encoding the predicted depth value, the processor 122 may encode each pixel in the target block by using a difference between the predicted depth value and the measured depth value. The measurement depth value may be a depth value measured in a pixel having (w, h) coordinates of the image plane.

Referring to FIG. 5, for example, in the case of plane coefficients, the processor 122 may vary the precision of the plane coefficients (S510). Experimental examples of plane coefficients a, b, and c modeled for each block in the depth image are shown in Tables 1 to 3 below. In each table, a coefficient of a predetermined corresponding position means a coefficient of a plane modeled for a predetermined and the same block.

Plane factor a modeled for a block 0.28958 0.26918 0.26298 0.23043 0.24107 0.24443 0.22918 0.24899 0.26498 0.28927 0.29014 0.23852 0.2518 0.22987 0.224 0.22947

Plane factor b modeled for one block 0.23946 0.23618 0.29009 0.28802 0.22646 0.27191 0.28112 0.29213 0.28811 0.23305 0.27969 0.27497 0.28643 0.24994 0.28183 0.30154

Plane coefficient c modeled for a block 838.7978 847.5082 856.1072 866.4822 847.4191 856.3975 866.017 875.8726 855.7426 864.5186 873.5252 881.2644 846.6515 856.1254 865.3145 874.8263

Referring to Tables 1 to 3, the values of the plane coefficients a, b, and c are in the form of a converted decimal point in which the number of digits after the decimal point is 2 or more. When the processor 122 converts the precision of the plane coefficients a, b, and c, it may mean that the decimal places of the plane coefficients a, b, and c are varied. That is, the processor 122 may convert the plane coefficient into a fixed-point form in which the first decimal place is used, or convert the plane coefficient into an integer to convert the precision. In particular, the processor 122 may convert the plane coefficient a and the plane coefficient b into a fixed point having one decimal place and convert the plane coefficient c into an integer. In this case, the processor 122 may increase the compression rate while maintaining the quality of the image. For an experiment to prove this, for example, referring to FIG. 7, a mean squared error (MSE) according to a change in precision of plane coefficients a, b, and c was calculated. Figure 7 (a) is the MSE according to the change in the decimal point precision of the plane coefficient a, Figure 7 (b) is the MSE according to the change in the decimal point precision of the plane coefficient b, and Figure 7 (c) is the decimal point of the plane coefficient c It is the MSE according to the precision change. Referring to (a) and (b) of FIG. 7, even if the precision is converted to the first decimal place for the plane coefficients a and b, the MSE does not change significantly, but when converted to an integer, the MSE is rapidly increased and the accuracy of the prediction in the screen. It can be seen that the fall significantly. On the other hand, looking at (c) of FIG. 7, it can be seen that even though the plane coefficient c is expressed as an integer, there is little change in the MSE, so that the accuracy of the prediction in the screen hardly changes. Therefore, even if the plane coefficient is converted to precision, the quality of the original image can be maintained. In addition, as an example of another experiment, the compression ratio according to the precision transformation of the plane coefficients a and b generated when a screen is predicted by plane coding in units of 16 × 16 blocks for a depth image of 640 × 480 resolution is shown in Tables 4 and 5 below. same.

Compression rate according to the precision of plane coefficient a Decimal precision of coefficients Number of bits Compression ratio 32 floating point 4800 One 5 2421 1.982652 4 2415 1.987578 3 2331 2.059202 2 2197 2.184797 One 1170 4.102564 0 361 13.2964

Compression ratio according to the precision of the plane coefficient b Decimal precision of coefficients Number of bits Compression ratio 32 floating point 4800 One 5 2427 1.97775 4 2421 1.982652 3 2337 2.053915 2 2064 2.325581 One 1127 4.259095 0 389 12.33933

The compression ratio is calculated by dividing the size before compression by the size after compression. Referring to Tables 4 and 5, when the plane coefficients a and b are encoded in a fixed-point form with an integer or one decimal place, the compression rate is high. I can see that. Therefore, the plane coefficients a and b are expressed in a fixed-point format in which the precision is converted to the first decimal place, and the plane coefficient c is expressed as an integer, which is the most efficient because the compression ratio is good while maintaining the original depth image.

A method of varying the precision of the plane coefficient by the processor 122 of S510 may be a method of varying the precision of the plane coefficient according to FIG. 6. Referring to FIG. 6, the processor 122 may determine the precision according to the user's selection or the type of the depth image (S511). The precision may include a first precision or a second precision. The first precision is the precision converted by rounding when converting the plane coefficients a, b, and c to a fixed point or an integer. The second precision is the precision converted by rounding off the plane coefficients a, b, and c when converting them to fixed decimal points or integers. The present invention is not limited thereto, and either the first precision or the second precision may be a precision converted by rounding up when converting the plane coefficients a, b, and c to a fixed point or an integer. If it is determined as the first precision, the processor 122 may convert the precision of the plane coefficients a, b, and c according to the first precision (S512 and 513). If the determined precision is not the first precision, the processor 122 may convert the precision of the plane coefficients a, b, and c according to the second precision (S512 and S514).

Referring to FIG. 5, as an example, in the case of plane coefficients, the processor 122 may entropy-encode the plane coefficients having a variable precision (S520). Entropy encoding includes an encoding method using a Huffman code, an arithmetic encoding method, an adaptive arithmetic encoding (CABAC) method, and the like, but is not limited thereto and may follow other entropy encoding algorithms. As an example, the arithmetic coding method is one of the entropy coding algorithms used for lossless compression. It determines a fixed statistical model of symbols appearing in a message and replaces the entire message with one real number n, which is greater than 0 and less than 1.

<Depth video encoding>

The processor 122 of the depth image processing apparatus 100 may include a step S600 of adding all the encoded blocks to encode a depth image.

Accordingly, according to an embodiment of the present invention, a plane in a depth image may be modeled, a depth may be predicted using a plane model, and a depth image may be encoded using a depth prediction and coefficients of the plane model. In addition, according to an exemplary embodiment of the present invention, while maintaining the quality of the depth image in consideration of the characteristics of the depth image, it is possible to increase the compression rate. In addition, according to an exemplary embodiment of the present invention, the compression ratio can be increased while maintaining the quality of the depth image by varying the precision of the coefficients of the planar model. In addition, according to an exemplary embodiment of the present invention, the precision of coefficients of the planar model may be selectively changed and coded according to the depth image. In addition, according to an embodiment of the present invention, a plane within a depth image may be modeled in a local unit, and a depth image may be encoded using a detailed and sophisticated plane model.

8 is an exemplary diagram of a method for modeling a second plane using a local plane.

Referring to FIG. 8, in the second plane modeling method, the processor 122 of the depth image processing apparatus 100 generates a local plane for each pixel of the depth image (S1000), and the depth image processing apparatus 100 A step of detecting a feature of a local plane by the processor 122 of (S2000), a step of calculating a similarity between adjacent local planes by using the feature of the detected local plane by the processor 122 of the depth image processing apparatus 100 ( S3000) and if the adjacent local planes of the processor 122 of the depth image processing apparatus 100 are equal to or greater than a predetermined degree of similarity, grouping and detecting them as the same plane (S4000). Each step of the second planar model method will be described in more detail with reference to FIGS. 9 to 17.

FIG. 9 is a flowchart illustrating a method of generating a local plane of FIG. 8, and FIG. 10 is a diagram illustrating a state in which pixels in a depth image are transformed into a 3D coordinate system in the method of generating a local plane of FIG. 9.

Referring to FIG. 9, in a method of generating a local plane according to one method, the processor 122 of the depth image processing apparatus 100 uses a center transform coordinate of a 3D coordinate system with a camera as an origin for a pixel in the depth image. It may include the step of generating (S1100).

The method of generating a local plane according to one method is that the processor 122 of the depth image processing apparatus 100 transforms the first to fourth peripherals of a 3D coordinate system with a camera as an origin for a neighboring pixel of one pixel in the depth image. It may include generating coordinates (S1200). More specifically, the surrounding pixels of one pixel may mean a pixel located adjacent to the one pixel. The processor 122 may generate four neighboring pixels of one pixel P(x,y) as first to fourth peripheral transform coordinates. As an example, the peripheral pixel is a first vertical peripheral pixel that is a k-th pixel vertically downward from one pixel (k is a natural number of 1 or more), a second vertical peripheral pixel that is a k-th pixel vertically upward from one pixel, and one pixel. A first horizontal peripheral pixel that is a k-th pixel in a horizontal right direction, and a second horizontal peripheral pixel that is a k-th pixel in a horizontal left direction from one pixel may be included. The processor 122 uses a first vertical peripheral pixel as a first peripheral transformed coordinate, a second vertical peripheral pixel as a second peripheral transformed coordinate, a first horizontal peripheral pixel as a third peripheral transformed coordinate, and a second horizontal peripheral pixel as a second peripheral transformed coordinate. It can be converted into a fourth peripheral transform coordinate.

In the method of generating a local plane according to an embodiment, the processor 122 of the depth image processing apparatus 100 generates a plane including the center transform coordinates and the first to fourth peripheral transform coordinates as a local plane of one pixel. It may include step S1300.

FIG. 11 is a flowchart illustrating a method of detecting a feature of a local plane in the local plane of FIG. 9, and FIG. 12 is a three-dimensional coordinate system for describing a normal vector that is a feature of the local plane of FIG. 8.

)를 생성하는 단계(S2100)를 포함할 수 있다. 보다 구체적으로, 국소 평면의 제1 국소 벡터(

)는 중심 변환 좌표(Pc(w,h))를 지나며 제2 주변 변환 좌표(Pc(w,h+k))에서 제1 주변 변환 좌표(Pc(w,h-k))를 지나는 제1 방향의 벡터 일 수 있다. 일 예로, 제1 방향은 수직 하측 방향 또는 수직 하측 방향에 가까운 방향일 수 있다. 제1 국소 벡터(

)는 Pc(w,h-k)-Pc(w, h+k)로 나타낼 수 있다. Referring to FIG. 11, in a method of detecting a feature of a local plane according to one method, the processor 122 of the depth image processing apparatus 100 uses a first local vector of the local plane (

) May include the step of generating (S2100). More specifically, the first local vector of the local plane (

) Passes through the center transform coordinate (Pc(w,h)) and passes through the first peripheral transform coordinate (Pc(w,hk)) from the second peripheral transform coordinate (Pc(w,h+k)). It can be a vector. For example, the first direction may be a vertical downward direction or a direction close to a vertical downward direction. The first local vector (

) Can be represented as Pc(w,hk)-Pc(w, h+k).

)를 생성하는 단계(S2200)를 포함할 수 있다. 보다 구체적으로, 국소 평면의 제2 국소 벡터(

)는 중심 변환 좌표(Pc(w,h))를 지나며 제4 주변 변환 좌표(Pc(w-k,h))에서 제3 주변 변환 좌표(Pc(w+k,h))를 지나는 제2 방향의 벡터 일 수 있다. 일 예로, 제2 방향은 수평 우측 방향 또는 수평 우측 방향에 가까운 방향일 수 있다. 제2 국소 벡터(

)는 Pc(w+k,h)-Pc(w-k, h)로 나타낼 수 있다.In a method of detecting a feature of a local plane according to one method, the processor 122 of the depth image processing apparatus 100 performs a second local vector of the local plane (

) May include generating (S2200). More specifically, the second local vector of the local plane (

) Passes through the center transform coordinate (Pc(w,h)) and passes from the fourth peripheral transform coordinate (Pc(wk,h)) to the third peripheral transform coordinate (Pc(w+k,h)) in the second direction. It can be a vector. For example, the second direction may be a horizontal right direction or a direction close to a horizontal right direction. The second local vector (

) Can be expressed as Pc(w+k,h)-Pc(wk, h).

) 및 제2 국소 벡터(

)를 이용하여 일 화소(P)에 대한 국소 평면의 법선 벡터(n(w,h))를 생성하는 단계(S2300)를 생성하는 단계를 포함할 수 있다. 보다 구체적으로, 법선 벡터(n(w,h))는 수학식 7에 따라 제1 국소 벡터(

)와 제2 국소 벡터(

)의 외적으로 구할 수 있다. In a method of detecting a feature of a local plane according to one method, the processor 122 of the depth image processing apparatus 100 uses a first local vector (

) And the second local vector (

) To generate a local plane normal vector (n(w,h)) for one pixel P (S2300). More specifically, the normal vector (n(w,h)) is the first local vector (

) And the second local vector (

) Can be obtained externally.

[Equation 7]

)와 제1 국소 벡터(

)의 곱으로 순서를 달리하여 구할 수 있다.It is not limited thereto, and the normal vector (n(w,h)) is the second local vector (

) And the first local vector (

It can be obtained in a different order by multiplying by ).

)를 생성하는 단계(S2400)를 포함할 수 있다. 보다 구체적으로, 중심 변환 좌표(Pc(w,h))의 국소 평면의 법선 벡터(n(w,h))의 성분을 (

)이라고 했을 때, 중심 변환 좌표(Pc(w,h))를 지나고 법선 벡터(n(w,h))를 가지는 평면은 수학식 8과 같이 표현 할 수 있다.In a method of detecting a feature of a local plane according to an embodiment, the processor 122 of the depth image processing apparatus 100 calculates a center transform coordinate (Pc(w,h)) and a normal vector (n(x,y)). Using the feature distance of the local plane, which is the distance between the local plane and the camera (

) May include generating (S2400). More specifically, the component of the local plane normal vector (n(w,h)) of the center transform coordinate (Pc(w,h)) is (

), a plane passing through the center transform coordinate (Pc(w,h)) and having a normal vector (n(w,h)) can be expressed as in Equation 8.

[Equation 8]

)는 수학식 9를 통해 계산될 수 있다. At this time, the feature distance, which is the distance from the camera on the local plane (

) Can be calculated through Equation 9.

[Equation 9]

13 is a flowchart illustrating a method of calculating the similarity between local planes of FIG. 8.

Referring to FIG. 13, in a method of calculating the similarity between local planes according to one method, the processor 122 of the depth image processing apparatus 100 uses a local plane of one pixel and a normal vector of a local plane of another pixel. 1 It may include the step of calculating the similarity (U) (S3100). More specifically, the first degree of similarity (U) can be determined by the angle between the normal vector (n(w,h)) of the local plane of one pixel and the normal vector (m(w,h)) of the local plane of another pixel. have. For example, the first similarity U may be determined according to Equation 10.

[Equation 10]

In addition, in a method of calculating the similarity between local planes according to one method, the processor 122 of the depth image processing apparatus 100 uses a second similarity (D) using a feature distance between a local plane of one pixel and a local plane of another pixel. ) It may include a calculating step (S3200). More specifically, the second similarity D may be determined by a difference between a feature distance Dn of a local plane of one pixel and a feature distance Dm of a local plane of another pixel. For example, the second similarity D may be determined according to Equation 11.

[Equation 11]

14 is a flowchart illustrating a method of modeling a second plane through grouping according to the similarity between local planes of FIG. 8.

Referring to FIG. 14, in the second plane modeling method through grouping according to the similarity between local planes according to a method example, the processor 122 of the depth image processing apparatus 100 determines that a first similarity U is a predetermined value. It may include a step (S4100) of determining whether it is abnormal. More specifically, the first similarity U may be set in advance according to the level, and may be adjusted through programming of the memory 121.

In addition, in the second plane modeling method through grouping according to the similarity between local planes according to one method, when the processor 122 of the depth image processing apparatus 100 satisfies the first similarity (U), the second similarity (D) It may include determining whether is less than or equal to a predetermined value (S4200 and S4300). More specifically, the first similarity U may be satisfied when the first similarity U is greater than or equal to a predetermined value. If it is determined that the first similarity U is not satisfied, the processor 122 may perform steps S420 and S470 of comparing one pixel with other surrounding pixels that have not been compared. That is, the processor 122 may determine whether the first similarity U of one pixel and another pixel is equal to or greater than a predetermined value (S4700 and S4100).

In addition, in the second planar modeling method through grouping according to the similarity between local planes according to one method, when the processor 122 of the depth image processing apparatus 100 satisfies the second similarity D, one pixel and the other pixel are A grouping step (S4400, S4500) may be included. More specifically, satisfying the second degree of similarity (D) may be a case where the second degree of similarity (D) is less than or equal to a predetermined value. The second similarity D may be set in advance according to the level, and may be adjusted through programming of the memory 121. If it is determined that the second similarity D is not satisfied, the processor 122 may perform steps (S440, S470) of comparing one pixel with other surrounding pixels that have not been compared. That is, the processor 122 may determine whether the first similarity U of one pixel and another pixel is equal to or greater than a predetermined value (S4700 and S4100).

In addition, in the second planar modeling method through grouping according to the similarity between local planes according to one method, the processor 122 of the depth image processing apparatus 100 determines whether there are other surrounding pixels that are not compared with one pixel. It may include a step (S4600). The processor 122 may perform an operation (S4600, S4700) of comparing one pixel with other surrounding pixels that have not been compared with other pixels in the surrounding area, if there is another pixel in the vicinity that has not been compared with one pixel. That is, the processor 122 may determine whether the first similarity U of one pixel and another pixel is equal to or greater than a predetermined value (S4700 and S4100).

In addition, in the second planar modeling method through grouping according to the similarity between local planes according to one method, if the processor 122 of the depth image processing apparatus 100 does not have other surrounding pixels that are not compared with one pixel, A step of detecting the grouped pixels on the same plane (S4800) may be included. According to an embodiment of the present invention, a plane can be quickly and accurately estimated through depth information in a depth image.

Accordingly, according to an exemplary embodiment of the present invention, precision is improved by detecting a planar area in units of pixels. In addition, according to an embodiment of the present invention, the data processing speed is improved by calculating a planar factor in units of a local plane and grouping adjacent local planes having similar plane factors into the same plane to detect regions of each plane. In addition, according to the exemplary embodiment of the present invention, even when a depth image contains a lot of noise, reliable plane estimation is possible.

In some embodiments, the processor 122 may group pixels satisfying the first and second similarities of a local plane among a plurality of pixels in the depth image according to the above-described process to detect a plane, and A plurality of planes may be detected. The detection of a plurality of planes in the depth image may be a case in which the first and second similarities between the detected planes are not satisfied. That is, the detected plurality of planes are planes that are not grouped together into one plane. The detected planes may transfer information of each plane area to other applications for use in other applications. Here, the plane information may be, for example, the number of detected planes, the position of the upper left corner in the depth image of each plane, and the height and width of the plane in the depth image. The processor 122 may provide plane information in response to a request from such other software. When describing a plurality of planes in one depth image in more detail, the processor 122 uses the feature distance information of each of the local planes of a pixel included in one of the planes to determine all pixels in the corresponding plane. It is possible to generate an average value of the feature distances. For example, the processor 122 may generate a first average value for the feature distances of the first pixels in the first plane, and may generate a second average value for the feature distances of the second pixels in the second plane. . The processor 122 may store difference information between the first and second average values. Herein, the difference information between the first and second average values may be one of plane distribution information provided by the processor 122 to the user. The embodiment may provide distribution information between the objects when a plurality of objects having planes on a plurality of photographing planes exist in a depth image. Accordingly, when objects having a level difference from each other exist in the photographing area, the corresponding level difference information can be detected. For example, when photographing a desk arranged in a space, the height of the desk can be estimated by detecting a difference in height between the upper surface of the desk and the floor surface of the space. In addition, it can be used as basic data to obtain various information about an object by estimating the step information of an object having a plane.

Each step of the second plane modeling method through grouping according to similarity between local planes according to one method will be described in more detail with reference to FIGS. 3 and 15 to 17. Other methods, except for the same description as this method, focus on the method of generating a different local plane and the method of detecting the features of the local plane.

15 is a flow chart for another method of generating the local plane of FIG. 8.

Referring to FIG. 15, in a method of generating a local plane according to another method, the processor 122 of the depth image processing apparatus 100 uses a camera for one pixel in the depth image as an origin. It may include the step of generating (S11100). As an example, referring to FIG. 13, the processor 122 converts one pixel (P(w,h)) in the depth image into a center transformed coordinate (Pc(w,h)) which is a three-dimensional coordinate system using Equation 1. Can be converted.

In addition, in a method of generating a local plane according to another method, the processor 122 of the depth image processing apparatus 100 is N*N (N is 3, 5, 7, …) square around one pixel in the depth image. It may include generating (S11200) peripheral transformed coordinates of a 3D coordinate system with a camera as an origin of the surrounding pixels of the region. Accordingly, another embodiment makes it possible to obtain an accurate local plane from a noisy depth image. More specifically, the processor 122 defines pixels located outside the N*N rectangular area around one pixel (P(w,h)) as surrounding pixels, and generates the surrounding pixels as a plurality of peripheral transform coordinates. I can. As an example, the plurality of peripheral transform coordinates may be 4 (N-1).

In addition, in a method of generating a local plane according to another method, the processor 122 of the depth image processing apparatus 100 generates a plane including the center transform coordinates and the peripheral transform coordinates as a local plane of one pixel (S11300). ) Can be included.

FIG. 16 is a flowchart illustrating a method of detecting a feature of a local plane of FIG. 15, and FIG. 17 is an example of a plurality of local vectors in a local plane for obtaining an average local vector of the local plane of FIG. 15.

)를 생성하는 단계(S12100)을 포함할 수 있다. 보다 구체적으로, 제1 평균 국소 벡터(

)는 국소 평면의 복수의 제1 국소 벡터의 평균 일 수 있다. 복수의 제1 국소 벡터는 주변 변환 좌표들 중 상측에 위치한 주변 변환 좌표들과 하측에 위치한 주변 변환 좌표들이 각각 쌍을 이루어 상측 주변 변환 좌표에서 하측 주변 변환 좌표로 향한 수직 성분의 벡터들일 수 있다. 이에 제한되는 것은 아니고, 복수의 제1 국소 벡터는 주변 변환 좌표들 중 하측에 위치한 주변 변환 좌표들과 상측에 위치한 주변 변환 좌표들이 각각 쌍을 이루어 하측 주변 변환 좌표에서 상측 주변 변환 좌표로 향한 수직 성분의 벡터들일 수 있다. 또한, 복수의 제1 국소 벡터 중 임의의 제1 국소 벡터(

)는 수학식 12에 따라 결정될 수 있다.Referring to FIG. 16, in a method of detecting a feature of a local plane according to another method, the processor 122 of the depth image processing apparatus 100 uses a first average local vector of the local plane (

) May include generating (S12100). More specifically, the first average local vector (

) May be an average of a plurality of first local vectors in the local plane. The plurality of first local vectors may be vectors of vertical components from the upper peripheral transform coordinates to the lower peripheral transform coordinates by pairing the upper peripheral transform coordinates and the lower peripheral transform coordinates among the peripheral transform coordinates. It is not limited thereto, and the plurality of first local vectors is a vertical component from the lower peripheral transform coordinates to the upper peripheral transform coordinates by pairing the lower peripheral transform coordinates and the upper peripheral transform coordinates among the peripheral transform coordinates. May be vectors of. In addition, any first local vector from among the plurality of first local vectors (

) May be determined according to Equation 12.

[Equation 12]

)를 결정할 수 있다. 제1 평균 국소 벡터(

)는 수학식 13에 따라 복수의 제1 국소 벡터들을 합하여 각 벡터가 가지는 방향의 평균 방향을 가지도록 생성될 수 있다.As an example, referring to FIG. 17, if N=3, the processor 122 has three first local vectors according to Equation 12 (

) Can be determined. First mean local vector (

) May be generated to have an average direction of a direction of each vector by summing a plurality of first local vectors according to Equation 13.

[Equation 13]

)를 생성하는 단계(S12200)을 포함할 수 있다. 보다 구체적으로, 제2 평균 국소 벡터(

)는 국소 평면의 복수의 제2 국소 벡터의 평균 일 수 있다. 복수의 제2 국소 벡터는 주변 변환 좌표들 중 좌측에 위치한 주변 변환 좌표들과 우측에 위치한 주변 변환 좌표들이 각각 쌍을 이루어 좌측 주변 변환 좌표에서 우측 주변 변환 좌표로 향한 수평 성분의 벡터들일 수 있다. 이에 제한되는 것은 아니고, 복수의 제2 국소 벡터는 주변 변환 좌표들 중 우측에 위치한 주변 변환 좌표들과 좌측에 위치한 주변 변환 좌표들이 각각 쌍을 이루어 우측 주변 변환 좌표에서 좌측 주변 변환 좌표로 향한 수평 성분의 벡터들일 수 있다. 또한, 복수의 제2 국소 벡터 중 임의의 제2 국소 벡터(

)는 수학식 12에 따라 결정될 수 있다. 일 예로, 도 17을 참조하면, 프로세서(122)는 N=3이면 수학식 12에 따라 3개의 제2 국소 벡터(

)를 결정할 수 있다. 제2 평균 국소 벡터(

)는 수학식 13에 따라 복수의 제2 국소 벡터들을 합하여 각 벡터가 가지는 방향의 평균 방향을 가지도록 생성될 수 있다.In addition, in a method of detecting a feature of a local plane according to another method, the processor 122 of the depth image processing apparatus 100 uses the second average local vector of the local plane (

) May include generating (S12200). More specifically, the second average local vector (

) May be an average of a plurality of second local vectors in the local plane. The plurality of second local vectors may be vectors of a horizontal component from a left peripheral transform coordinate to a right peripheral transform coordinate by forming a pair of peripheral transform coordinates located at the left and the surrounding transform coordinates located at the right among the surrounding transform coordinates. It is not limited thereto, and the plurality of second local vectors is a horizontal component of the peripheral transformation coordinates located on the right and the peripheral transformation coordinates located on the left of the peripheral transformation coordinates, respectively, forming a pair, and going from the right peripheral transformation coordinate to the left peripheral transformation coordinate. May be vectors of. In addition, any second local vector from among the plurality of second local vectors (

) May be determined according to Equation 12. As an example, referring to FIG. 17, if N=3, the processor 122 uses three second local vectors according to Equation 12 (

) Can be determined. Second mean local vector (

) May be generated to have an average direction of a direction of each vector by summing a plurality of second local vectors according to Equation 13.

) 및 제2 평균 국소 벡터(

)를 이용하여 일 화소(P)에 대한 국소 평면의 법선 벡터(

)를 생성하는 단계(S12300)를 생성하는 단계를 포함할 수 있다. 보다 구체적으로, 법선 벡터(

)는 수학식 14에 따라 제1 평균 국소 벡터(

)와 제2 평균 국소 벡터(

)의 외적으로 구할 수 있다.In addition, in a method of detecting a feature of a local plane according to another method, the processor 122 of the depth image processing apparatus 100 uses the first average local vector (

) And the second mean local vector (

) Using the local plane normal vector (

) May include the step of generating the step of generating (S12300). More specifically, the normal vector (

) Is the first average local vector (

) And the second mean local vector (

) Can be obtained externally.

[Equation 14]

)는 제2 평균 국소 벡터(

)와 제1 평균 국소 벡터(

)의 곱으로 순서를 달리하여 구할 수 있다.It is not limited thereto, and the normal vector (

) Is the second mean local vector (

) And the first mean local vector (

It can be obtained in a different order by multiplying by ).

)를 이용하여 국소 평면과 카메라의 거리인 국소 평면의 특징 거리(

)를 생성하는 단계(S12400)를 포함할 수 있다. 보다 구체적으로, 중심 변환 좌표(Pc(w,h))의 국소 평면의 법선 벡터(

)의 성분을 (

)이라고 했을 때, 중심 변환 좌표(Pc(w,h))를 지나고 법선 벡터(n(w,h))를 가지는 제2 평면은 수학식 8과 같이 표현 할 수 있다. 이 때, 국소 평면의 카메라로부터의 거리인 특징 거리(

)는 수학식 9를 통해 계산될 수 있다. In addition, in a method of detecting a feature of a local plane according to another method, the processor 122 of the depth image processing apparatus 100 performs a center transform coordinate (Pc(w,h)) and a normal vector (

) Using the feature distance of the local plane, which is the distance between the local plane and the camera (

) May include the step of generating (S12400). More specifically, the normal vector of the local plane of the center transform coordinate (Pc(w,h)) (

) Of the component (

), the second plane passing through the center transform coordinate Pc(w,h) and having a normal vector (n(w,h)) can be expressed as Equation 8. At this time, the feature distance, which is the distance from the camera on the local plane (

) Can be calculated through Equation 9.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

Specific implementations described in the present invention are examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections that can be replaced or additionally It may be referred to as a connection, or circuit connections. In addition, if there is no specific mention such as “essential” or “importantly”, it may not be an essential component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, the spirit of the present invention described in the claims to be described later if one of ordinary skill in the relevant technical field or those of ordinary skill in the relevant technical field And it will be understood that various modifications and changes can be made to the present invention within a range not departing from the technical field. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

100: depth image processing device
110: video source
120: image processing unit
121: memory
122: processor
130: output interface
200: data receiving device
210: data processing unit
220: input interface

Claims (8)

Dividing the depth image into preset block units;
Converting image coordinates of pixels in the depth image into 3D coordinates;
Modeling the block with a surface model based on the three-dimensional coordinates;
Generating a predicted depth value using a planar model;
Encoding the block by varying the precision of the plane coefficient based on the predicted depth value and the plane coefficient of the plane model; and
The plane model is modeled using the first plane equation of Equation 1 consisting of three-dimensional coordinates and parameters,
The step of encoding the block by varying the precision of the plane coefficient,
Varying the precision of the plane coefficient; And
Entropy encoding the variable plane coefficients; Including,
The step of varying the precision of the plane coefficient,
Converting the plane coefficients a, b and c to a fixed point form
Depth image coding method through plane modeling.
[Equation 1]
ax+by-z+c=0
x, y, and z are three-dimensional coordinates, and a, b, and c are plane coefficients. delete delete The method of claim 1,
The step of varying the precision of the plane coefficient,
A depth image encoding method through plane modeling in which the plane coefficients a and b are transformed into a fixed-point form having one decimal place and the plane coefficient c is transformed into an integer form. The method of claim 1,
The entropy encoding is an arithmetic encoding algorithm, a depth image encoding method through plane modeling. The method of claim 1,
The depth image encoding method through plane modeling, wherein the predicted depth value is obtained using a second plane equation of Equation 2 consisting of an image coordinate and a focal length.
[Equation 2]

Is the first estimated depth value, f is the focal length, a, b, c are parameters, and w, h are image coordinates. At least one memory for storing instructions; And
At least one processor; Including,
The instructions are executable by the processor to cause the processor to perform operations,
The above actions are:
Divide the depth image into preset block units,
Convert the image coordinates of the pixels in the depth image into 3D coordinates,
Model the block as a surface model based on the three-dimensional coordinates,
Generate the predicted depth value using the planar model,
And encoding the block by varying the precision of the plane coefficient based on the predicted depth value and the plane coefficient of the plane model,
The plane model is modeled using the first plane equation of Equation 3 consisting of three-dimensional coordinates and parameters,
Encoding the block by varying the precision of the plane coefficient,
Varying the precision of the plane coefficient, and entropy encoding the changed plane coefficient,
To vary the precision of the plane coefficient,
Converting the plane coefficients a, b and c to a fixed point form
An apparatus for encoding a depth image through plane modeling.
[Equation 3]
ax+by-z+c=0
x, y, and z are three-dimensional coordinates, and a, b, and c are plane coefficients. A non-transitory computer-readable medium storing the instructions according to claim 7.

Images