TWI700670B - Method and electrical device for transforming 3d depth information - Google Patents

Abstract

A method for transforming 3D depth information includes: obtaining a depth map having depth values in a first range; obtaining multiple sub-ranges from the first range; for each sub-range, transforming the depth values in the sub-range into a second range according to a transfer function to obtain an image, in which the second range is narrower than the first range; and fusing the images to obtain an output image.

Description

Three-dimensional depth information conversion method and electronic device

The present invention relates to a method for converting three-dimensional depth information into two-dimensional images.

Three-dimensional depth information can be used to describe the uneven surface of an object. It is important information used to identify the object. In order to make the depth information more precise, high-bit (for example, 16-bit) digital data is used to represent the depth information. However, the general image processing method is used to process low-bit (such as 8-bit) images, so how to convert high-bit depth information into low-bit images while preserving the details of the surface of the object, this area Topics of concern to technicians.

An embodiment of the present invention provides a method for converting three-dimensional depth information, which is suitable for electronic devices. The method includes: obtaining a depth map. The depth map includes a plurality of depth values, and each depth value is in a first range; Get multiple sub-ranges in the range; for each sub-range, set the The depth value is converted to a second range through a conversion function to obtain an image, wherein the second range is smaller than the first range; and the image is fused to obtain an output image.

In some embodiments, the transfer function corresponding to each sub-range is a non-linear function.

In some embodiments, the aforementioned sub-ranges at least partially overlap each other.

In some embodiments, the step of converting the depth value in the sub-range to the second range through a conversion function to obtain an image includes: setting the depth value outside the sub-range to the maximum or minimum value of the second range.

In some embodiments, the step of fusing images to obtain an output image includes: for each image, obtaining a high frequency part and a low frequency part; taking the average of the low frequency part as the output low frequency part; taking the high frequency part The maximum value of the division is used as the output high frequency part; and the output low frequency part and the output high frequency part are combined to generate the output image.

From another perspective, an embodiment of the present invention provides an electronic device including a memory and a processor. The memory stores program codes, and the processor executes the program codes to implement the above conversion method.

In the above conversion method, the converted image has fewer bits while retaining the details in the region of interest.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

110‧‧‧Depth sensor

120‧‧‧Electronic device

121‧‧‧Processor

122‧‧‧Memory

210‧‧‧Depth map

220‧‧‧Histogram

221, 222‧‧‧ sub-range

301, 302‧‧‧ Conversion function

410、420‧‧‧Two-dimensional image

411, 421‧‧‧Low frequency part

412, 422‧‧‧High frequency part

431‧‧‧Output low frequency part

432‧‧‧High frequency output

440‧‧‧Output image

501~504‧‧‧Step

[Fig. 1] is a schematic diagram of a system for converting 3D depth information into 2D images according to an embodiment.

[Figure 2] is a schematic diagram illustrating a conversion method according to an embodiment.

[Fig. 3] is a schematic diagram showing a conversion function according to an embodiment.

[Fig. 4] is a schematic diagram showing a fusion image according to an embodiment.

[Fig. 5] is a flowchart of a method for converting 3D information according to an embodiment.

Regarding the "first", "second", etc. used in this text, it does not specifically refer to the order or sequence, but only to distinguish elements or operations described in the same technical terms.

FIG. 1 is a schematic diagram of a system for converting 3D depth information into 2D images according to an embodiment. Please refer to FIG. 1, this system includes a depth sensor 110 and an electronic device 120, and the electronic device 120 includes a processor 121 and a memory 122. The depth sensor 110 is, for example, an infrared sensor or other sensors capable of sensing depth. In some embodiments, the system may also be equipped with an infrared transmitter and other devices, but the invention is not limited thereto. The processor 121 can be a central processing unit, a microprocessor, an image processing chip, etc. The memory 122 can be a volatile memory or a non-volatile memory. The memory 122 stores program codes, and the processor 121 executes these programs. Code to implement the conversion method of 3D depth information. The conversion method will be described in detail below.

Fig. 2 is a schematic diagram illustrating a conversion method according to an embodiment. First, a depth map 210 is obtained from the depth sensor 110. The depth map 210 includes a plurality of depth values, and each depth value falls within the first range. For example, each depth value is encoded with 16 bits, so the first range is 0~2 ¹⁶ -1. The above-mentioned first range is only an example, and the present invention does not limit each depth value to be encoded with several bits. A histogram 220 can be drawn according to the depth values in the depth map 210, where the horizontal axis represents the depth value, and the vertical axis represents the number of depth values. Since the scene corresponding to the depth map 210 may have very close objects and a very far background, it is necessary to filter out regions of interest first and filter out regions that are not of interest. Specifically, a plurality of sub-ranges 221 and 222 may be set in the first range, and these sub-ranges may at least partially overlap, and the depth values in the sub-ranges will be converted to the second range through a conversion function to obtain an image. The above-mentioned second range is, for example, 0~255, which means that the second range will be smaller than the first range.

For example, please refer to FIG. 3, which is a schematic diagram illustrating a conversion function according to an embodiment. The sub-range 221 can also be expressed as x ₁ ~ y ₁ , which corresponds to the conversion function 301. The depth value less than the value x ₁ will be converted to 0, and the depth value greater than the value y ₁ will be converted to 255, and the rest of the depth values The conversion function 301 will be converted to between 0 and 255, so the depth map 210 will be converted into a two-dimensional image. Similarly, the sub-range 222 is represented as x ₂ ~ y ₂ , the depth value less than the value x ₂ will be converted to 0, and the depth value greater than the value y ₂ will be converted to 255, and the remaining depth values will be converted through the conversion function 302 It is converted to between 0 and 255, so the depth map 210 will also be converted into another two-dimensional image. From another perspective, the depth values outside the sub-ranges 221 and 222 will be set as the maximum or minimum value of the second range (0~255).

In the embodiments of FIG. 2 and FIG. 3, there are two sub-ranges 221 and 222, but in other embodiments, more sub-ranges can be designed to generate more two-dimensional images. In addition, the conversion functions 301 and 302 are non-linear functions, such as polynomial functions, exponential functions, trigonometric functions, or combinations thereof. The present invention does not limit the mathematical formulas of the conversion functions 301 and 302. In some embodiments, the conversion functions 301 and 302 are incremental functions, that is, f(x _i )>f(x _j )if x _i >x _j .

Next, the multiple 2D images generated according to the sub-ranges 221 and 222 are merged to calculate the output image. Specifically, referring to FIG. 4, the two-dimensional image 410 is generated according to the sub-range 221, and the two-dimensional image 420 is generated according to the sub-range 222. For each two-dimensional image 410, 420, a corresponding high frequency part and low frequency part can be generated. For example, after performing a low-pass filter (such as a Gaussian filter) on the two-dimensional image 410, the low-frequency part 411 can be generated, and the low-frequency part 411 is subtracted from the two-dimensional image 410 to obtain the high-frequency part 412. According to the same method, the two-dimensional image 420 can be divided into a low frequency part 421 and a high frequency part 422. Next, take the average of the low frequency part 411 and the low frequency part 421 as the output low frequency part 431. In other words, for a value in the low frequency part 411, this value will be averaged with the value at the same position in the low frequency part 421 To get the value of the same position in the output low frequency part 431. The maximum value of the high frequency part 412 and the high frequency part 422 can be used as the output high frequency part 432. In other words, for a value in the high frequency part 412, this value is the same as the value in the high frequency part 422. The maximum value of can be set to the value of the same position in the output high frequency part 432. Finally, the output image 440 can be obtained by combining (adding) the output low frequency part 431 and the output high frequency part 432.

In some embodiments, the aforementioned low frequency part and high frequency part It can also be generated through any space-to-frequency domain conversion, such as wavelet transform, Fourier transform, etc. The present invention is not limited thereto. Therefore, the steps of combining the output low-frequency part 431 with the output high-frequency part 432 may also be inverse wavelet transform, inverse Fourier transform, etc. The present invention is not limited to this. In addition, the symbol "♁" in FIG. 4 is only for illustration, and does not necessarily mean addition. It can also be the steps of taking the maximum value, averaging, and inverse wavelet transformation.

Fig. 5 is a flowchart illustrating a method for converting 3D information according to an embodiment. Referring to FIG. 5, in step 501, a depth map is obtained. The depth map includes multiple depth values, and each depth value is in the first range. In step 502, multiple sub-ranges are obtained in the first range. In step 503, for each sub-range, the depth value in the sub-range is converted to the second range through a conversion function to obtain an image. In step 504, the images are merged to obtain an output image. However, each step in FIG. 5 has been described in detail as above, and will not be repeated here. It should be noted that each step in FIG. 5 can be implemented as a plurality of program codes or circuits, and the present invention is not limited thereto. In addition, the method in FIG. 5 can be used in conjunction with the above embodiments, or can be used alone. In other words, other steps can also be added between the steps in FIG. 5.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make slight changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

501~504‧‧‧Step

Claims (8)

A conversion method of three-dimensional depth information is suitable for an electronic device. The conversion method includes: obtaining a depth map, the depth map includes a plurality of depth values, each of the depth values in a first range; A plurality of sub-ranges are obtained in the range; for each of the sub-ranges, the depth values in the sub-range are converted to a second range through a conversion function to obtain an image, wherein the second range is smaller than the first range , The transfer function is an increasing function; for each of these images, obtain a high-frequency part and a low-frequency part; take the average of the low-frequency parts as an output low-frequency part; take the high-frequency parts The maximum value of is used as an output high frequency part; and the output low frequency part and the output high frequency part are combined to generate an output image. In the conversion method described in item 1 of the scope of patent application, the conversion function corresponding to each of the sub-ranges is a non-linear function. As the conversion method described in item 1 of the scope of patent application, the sub-ranges at least partially overlap each other. Such as the conversion method described in item 1 of the scope of patent application, wherein the depth values in the sub-range are converted to the The step of obtaining the image in the second range includes: setting the depth values outside the sub-range to a maximum value or a minimum value of the second range. An electronic device comprising: a memory storing a program code; and a processor for executing the program code to implement multiple steps: obtaining a depth map, the depth map including a plurality of depth values, each of the depth values In a first range; obtain multiple sub-ranges in the first range; for each of the sub-ranges, transform the depth values in the sub-ranges to a second range through a conversion function to obtain an image , Wherein the second range is smaller than the first range, the transfer function is an increasing function; for each of the images, a high frequency part and a low frequency part are obtained; the average of the low frequency parts is taken as an output Low frequency part; take the maximum value of the high frequency parts as an output high frequency part; and combine the output low frequency part and the output high frequency part to generate an output image. For the electronic device described in item 5 of the scope of patent application, the transfer function corresponding to each of the sub-ranges is a non-linear function. For the electronic device described in item 5 of the scope of patent application, The sub-ranges at least partially overlap each other. For the electronic device described in item 5 of the scope of patent application, the step of converting the depth values in the sub-range to the second range through the conversion function to obtain the image includes: The depth values are set to a maximum value or a minimum value of the second range.

Images