WO2022016331A1

WO2022016331A1 - Method of compensating tof depth map and electronic device

Info

Publication number: WO2022016331A1
Application number: PCT/CN2020/103077
Authority: WO
Inventors: Teruchika MIURA
Original assignee: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
Priority date: 2020-07-20
Filing date: 2020-07-20
Publication date: 2022-01-27

Abstract

Disclosed is a method of compensating a ToF depth map. The method includes acquiring a color image captured by an image sensor module, acquiring a ToF depth map and an IR image, both of which are captured by a range sensor module and correspond to the color image, creating a stereo depth map based on the color image and the IR image, and compensating the ToF depth map based on the stereo depth map.

Description

METHOD OF COMPENSATING TOF DEPTH MAP AND ELECTRONIC DEVICE

FIELD

The present disclosure relates to a method of compensating a ToF depth map, and an electronic device implementing such method.

BACKGROUND

In recent years, smartphones have been equipped with not only image sensor modules that capture color images but also range sensor modules for distance measurement such as ToF sensors that capture ToF (Time of Flight) depth maps. Each pixel of the ToF depth map indicates the distance from a smartphone to a target/subject. A ToF depth map is used, for example, to estimate a size of the subject or to add blur to the color image captured by the image sensor module.

In the ToF method, distance is measured based on the time that it takes for light (e.g., infrared light) emitted from a smartphone to travel to a subject and then back to the smartphone. Specifically, there are two types of ToF methods, i.e., the indirect ToF (iToF) method and the direct ToF (dToF) method. In the case of the iToF method, the range sensor module emits light with a certain frequency and in phase, and receives the light reflected by the subject. The distance can be determined from a phase difference between the emitted light and the received light.

The iToF method is preferable for smartphones since it can be realized with a smaller circuit than the dToF and has cost advantages. However, the iToF method cannot measure distances when the phase difference is greater than a limit value (e.g., 2π) even if an intensity of the received light is higher than a threshold value of the range sensor module.

Further, there is a case where the iToF method cannot measure distances even if a subject is close to the smartphone. That is to say, the iToF method cannot measure distances when a subject absorbs the emitted light well and the intensity of the received light is lower than a threshold value of the range sensor module. In this case, the smartphone may determine that the subject is far from the smartphone and that a distance between the subject and the smartphone cannot be calculated.

SUMMARY

The present disclosure aims to solve at least one of the technical problems mentioned above. Accordingly, the present disclosure needs to provide a method of compensating a ToF depth map and an electronic device implementing such method.

In accordance with the present disclosure, a method of compensating a ToF depth map may include:

acquiring a color image captured by an image sensor module;

acquiring a ToF depth map and an IR image, both of which are captured by a range sensor module and correspond to the color image;

creating a stereo depth map based on the color image and the IR image; and

compensating the ToF depth map based on the stereo depth map.

In some embodiments, the compensating the ToF depth map may include filling an unmeasurable area in the ToF depth map with depth values of a corresponding area of the stereo depth map, the unmeasurable area being an area where a distance from an electronic device having the image sensor module and the range sensor module to a subject cannot be measured.

In some embodiments, the method may further include preprocessing the color image after the acquiring the color image captured by the image sensor module.

In some embodiments, the preprocessing the color image may include calibrating the color image to match the IR image by using calibration parameters.

In some embodiments, the preprocessing the color image may further include shrinking the calibrated color image so that it has substantially the same size as a size of the IR image.

In some embodiments, the creating the stereo depth map may include:

calculating, for each pixel, a parallax amount between the preprocessed color image and the IR image; and

calculating, for each parallax amount, a distance between a subject and the range sensor module.

In some embodiments, the distance may be calculated by means of an equation (1) ,

D = B×f/Z ... (1) ,

where D is the distance, B is a baseline length between the image sensor module and the range sensor module, f is a focal length of the range sensor module, and Z is the parallax amount.

In some embodiments, the method may further include postprocessing the compensated ToF depth map after the compensating the ToF depth map.

In some embodiments, the postprocessing the compensated ToF depth map may include calibrating the compensated ToF depth map to match the color image by using calibration parameters.

In some embodiments, the postprocessing the compensated ToF depth map may further include enlarging the calibrated ToF depth map.

In accordance with the present disclosure, an electronic device may include: a processor and a memory for storing instructions, wherein the instructions, when executed by the processor, cause the processor to perform the method according to the present disclosure.

In accordance with the present disclosure, a computer-readable storage medium, on which a computer program is stored, wherein the computer program is executed by a computer to implement the method according to the present disclosure, is provided.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of embodiments of the present disclosure will become apparent and more readily appreciated from the following descriptions made with reference to the drawings, in which:

FIG. 1 is a circuit diagram illustrating a schematic configuration of an electronic device 100 according to an embodiment of the present disclosure;

FIG. 2 is a functional block diagram of a main processor of the electronic device according to an embodiment of the present disclosure;

FIG. 3 is an example of a color image captured by an image sensor module;

FIG. 4 is an example of a ToF depth map captured by a range sensor module;

FIG. 5 is an example of an IR image captured by a range sensor module;

FIG. 6 is an image after adjusting the contrast of the IR image shown in FIG. 5;

FIG. 7 is a flowchart illustrating a method for compensating a ToF depth map according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detail and examples of the embodiments will be illustrated in the accompanying drawings. The same or similar elements and elements having same or similar functions are denoted by like reference numerals throughout the descriptions. The embodiments described herein with reference to the drawings are explanatory and aim to illustrate the present disclosure, but shall not be construed to limit the present disclosure.

Referring to FIG. 1, an electronic device 100 according to an embodiment of the present disclosure will be described in detail. FIG. 1 is a circuit diagram illustrating a schematic configuration of the electronic device 100.

The electronic device 100 is a mobile device such as a smartphone or a tablet terminal. But the electronic device 100 may be other types of electronic devices configured to perform image processing.

As shown in FIG. 1, the electronic device 100 includes an image sensor module 10, a range sensor module 20, an image signal processor 30, a main processor 40, and a memory 50. In the following description, the image sensor module 10 is also referred to as an RGB camera 10, and the range sensor module 20 is also referred to as a ToF camera 20.

The image sensor module 10 includes a lens 10a that is capable of focusing on a subject/target (e.g., a person, an object) , an image sensor 10b that detects an image inputted via the lens 10a, and an image sensor driver 10c that drives the image sensor 10b, as shown in FIG. 1.

Optionally, the image sensor module 10 may consist of a master camera and a slave camera for binocular stereo viewing. Each of the master camera and the slave camera may have a lens, an image sensor and an image sensor driver, like the image sensor module 10.

The image sensor module 10 captures color images. In the present disclosure, RGB images are captured as color images. FIG. 3 shows an example of a color image captured by the image sensor module 10. The color image includes a subject S1 and a subject S2. The subject S1 is a doll on a desk and the subject S2 is a black computer mouse on the desk.

In some embodiments, the color images may be another type of image such as YUV images.

The range sensor module 20 includes a lens 20a, a range sensor 20b, a range sensor driver 20c that drives the range sensor 20b, and a projector 20d, as shown in FIG. 1. The projector 20d emits infrared light with a certain frequency and in phase (i.e. coherent) toward a subject. The range sensor 20b detects the light reflected from the subject through the lens 20a.

In order to improve ranging accuracy, the projector 20d may emit a plurality of lights having frequency different from each other. For example, the projector may emit a first light having a first frequency and a second light having a second frequency simultaneously. The range sensor 20b detects the first and second lights reflected from the subject through the lens 20a. in this case, the distance can be determined based on a first phase difference and a second phase. The first phase difference is a phase difference between the emitted first light and the received first light. The second phase difference is a phase difference between the emitted second light and the received second light.

The range sensor module 20 simultaneously captures a ToF depth map and an IR image (infrared image) . The ToF depth map and the IR image correspond to the color image captured by the image sensor module 10. The resolution of an IR image is usually lower than the resolution of a color image captured by the image sensor module 10.

The ToF depth map is a depth map which is created by means of the iToF method. More specifically, the ToF depth map is an image in which each pixel indicates a depth value according to a phase difference between the emitted light and the reflected light. FIG. 4 shows an example of a ToF depth map captured by the range sensor module 20. The ToF depth map corresponds to the color image shown in FIG. 3.

The IR image is an image according to the intensity of the reflected infrared light. FIG. 5 shows an example of an IR image captured by the range sensor module 20. The IR image corresponds to the color image shown in FIG. 3 and the ToF depth map shown in FIG. 4.

The background of the IR image shown in FIG. 5 is extremely dark and contains no information. However, as can be seen from FIG. 6 after adjusting the contrast of the IR image of FIG. 5, the background contains information.

The image signal processor 30 controls the image sensor module 10 to acquire a color image, and stores the color image in the memory 50. The image signal processor 30 also controls the range sensor module 20 to acquire a ToF depth map and an IR image, and stores them in the memory 50.

The ToF depth map may contain one or more unmeasurable areas. An unmeasurable area is an area where a distance from the electronic device 100 (the range sensor module 20) to a subject cannot be measured. As mentioned, there are two cases where said distance cannot be measured by the iToF method.

One case is where the phase difference between the emitted light and the reflected light is greater than a limit value (e.g., 2π) (see in FIG. 4 an area A1 which is background) . In this case, distance cannot be measured although the intensity of the received light is higher than the threshold value of the range sensor 20b.

Another case is where, although a subject is relatively close to the electronic device 100, the intensity of the received light is lower than the threshold value because the subject easily absorbs the emitted light (see in FIG. 4 an area A2 which is a part of the black mouse S2) .

On the other hand, as can be seen from FIG. 5, the IR image includes information (intensity of the reflected light) in an area which corresponds to the unmeasurable area of the ToF depth map. As will be described in later, depth values in the unmeasurable area are calculated by viewing the color image and the IR image in stereo.

The main processor 40 reads the color image, the ToF depth map and the IR image from the memory 50. The main processor 40 is configured to compensate the ToF depth map by the method of the present disclosure.

The main processor 40 controls the GNSS module 61, the wireless communication module 62, the CODEC 63, the speaker 64, the microphone 65, the display module 66, the input module 67 and the IMU 68.

The memory 50 stores color images such as RGB images captured by the image sensor module 10, and IR images and ToF depth maps captured by the range sensor module 20. The memory 50 also stores a program and data required for the image signal processor 30 and the main processor 40.

Further, the memory 50 stores parameters for the RGB camera 10 and the ToF camera 20. Specifically, the memory 50 stores calibration parameters and conversion parameters for these cameras. The parameters in the memory 50 are used when a color image is calibrated to match an IR image, and when a compensated ToF depth map is calibrated to match the color image.

Still further, the memory 50 stores camera information which includes a baseline length between the image sensor module 10 and the range sensor module 20 (i.e., a length between the lens 10a and the lens 20a) .

The electronic device 100 further includes a global navigation satellite system (GNSS) module 61, a wireless communication module 62, a CODEC 63, a speaker 64, a microphone 65, a display module 66, an input module 67, and an inertial measurement unit (IMU) 68, as shown in FIG. 1.

The GNSS module 61 measures the current position of the electronic device 100. The wireless communication module 62 performs wireless communications with the Internet. The CODEC 63 bi-directionally performs encoding and decoding, using a predetermined encoding/decoding method. The speaker 64 outputs a sound in accordance with sound data decoded by the CODEC 63. The microphone 65 outputs sound data to the CODEC 63 based on inputted sound. The display module 66 displays various information such as captured images. The input module 67 inputs information being operated by a user. The IMU 68 detects an angular velocity and an acceleration of the electronic device 100.

Next, referring to FIG. 2, the main processor 40 according to the embodiment of the present disclosure will be described in detail.

The main processor 40 includes an acquiring unit 41, a preprocessing unit 42, a stereo depth map creating unit 43, a ToF depth map compensating unit 44 and a postprocessing unit 45, as shown in FIG. 2. The

units

41, 42, 43, 44 and 45 are implemented as software (aprogram) performed by the main processor 40. Optionally, at least one of the

units

41, 42, 43, 44 and 45 may be implemented as hardware such as a Digital Signal Processor (DSP) or a Graphics Processing Unit (GPU) .

Each of the

units

41, 42, 43, 44 and 45 will be described in detail below.

The acquiring unit 41 is configured to acquire a color image captured by the RGB camera 10 and to acquire a ToF depth map and an IR image captured by the ToF camera 20. The ToF depth map and the IR image correspond to the color image. The acquiring unit 41 reads the color image, the ToF depth map and the IR image from the memory 50.

The preprocessing unit 42 is configured to preprocess the color image acquired by the acquiring unit 41. Specifically, the preprocessing unit 42 calibrates the color image to match the IR image by using calibration parameters stored in the memory 50. The calibration is performed so that a shooting range and/or direction of the color image is matched to those of the IR image. For example, the color image is cut out according to a shooting range of the IR image.

The preprocessing unit 42 shrinks the calibrated color image so that it has substantially the same size as a size of the IR image. In other words, a size of the calibrated color image is reduced so that it has the same number of pixels as the IR image.

The stereo depth map creating unit 43 is configured to create a stereo depth map based on the color image and the IR image. Specifically, the stereo depth map creating unit 43 creates the stereo depth map as described below.

First, the stereo depth map creating unit 43 calculates, for each pixel, a parallax amount between the calibrated and shrunk color image (i.e., the preprocessed color image) and the IR image. For example, the similarity between the preprocessed color image and the IR image is calculated by a method called template matching or block matching, in which a template image (e.g., the IR image) is scanned with an input image (e.g., the preprocessed color image) in a search range, and a block with the highest degree of similarity is detected.

The calculation of similarity gives the corresponding pixels between the preprocessed color image and the IR image. The difference in the horizontal pixel coordinates between the corresponding pixels indicates a parallax amount. Please note that there are many methods for calculating the degree of similarity such as SAD, NCC, SNCC and ZNCC, but any method may be used in the present disclosure.

Next, the stereo depth map creating unit 43 calculates, for each parallax amount, a distance (or a depth value) between a subject and the range sensor module 20 to obtain a stereo depth map. For example, the stereo depth map creating unit 43 calculates the distance by means of the equation (1) ,

D = B×f/Z ... (1) ,

where D is the distance or depth value, B is a baseline length between the image sensor module 10 and the range sensor module 20, f is a focal length of the range sensor module 20 (i.e., the lens 20a) , and Z is the parallax amount.

The ToF depth map compensating unit 44 is configured to compensate the ToF depth map based on the stereo depth map. Specifically, the ToF depth map compensating unit 44 fills an unmeasurable area in the ToF depth map with depth values of a corresponding area of the stereo depth map.

The postprocessing unit 45 is configured to postprocess the compensated ToF depth map. For example, the postprocessing is performed according to an instruction from an application software executed on the electronic device 100. Postprocessing makes the compensated ToF depth map easier for the application software to handle.

Specifically, the postprocessing unit 45 calibrates the compensated ToF depth map to match the color image by using calibration parameters stored in the memory 50. The calibration is performed so that a shooting range and/or direction of the compensated ToF depth map is/are matched to those of the color image acquired by the acquiring unit 41.

The postprocessing unit 45 enlarges the calibrated ToF depth map. The calibrated ToF depth map is enlarged by an interpolation method such as Nearest Neighbor, Bilinear or Bicubic method. The enlargement ratio may be a value required by an application to measure the distance or to add blur to a captured color image.

Next, a method for compensating a ToF depth map according to an example of the present disclosure for the electronic device 100 will be described with reference to the flowchart shown in FIG. 7. The method includes the following steps.

In the step S1, the acquiring unit 41 acquires, from the memory 50, a color image captured by the image sensor module (RGB camera) 10.

In the step S2, the acquiring unit 41 acquires, from the memory 50, a ToF depth map and an IR image captured by the range sensor module (ToF camera) 20.

In the step S3, the preprocessing unit 42 preprocesses the color image acquired by the acquiring unit 41 in the step S1. As mentioned, the preprocessing unit 42 calibrates and shrinks the color image to match the IR image acquired in the step S2.

In the step S4, the stereo depth map creating unit 43 creates a stereo depth map based on the color image preprocessed in the step S3 and the IR image acquired in the step S2. As described above, the stereo depth map is created by calculating, for each pixel, a parallax amount between the preprocessed color image and the IR image, and by calculating, for each parallax amount, a distance (or a depth value) between a subject and the range sensor module 20 by means of the equation (1) .

In the step S5, the ToF depth map compensating unit 44 compensates the ToF depth map acquired in the step S2 based on the stereo depth map created in the step S4. As mentioned, an unmeasurable area in the ToF depth map is filled with depth values of a corresponding area of the stereo depth map.

In the step S6, the postprocessing unit 45 postprocesses the ToF depth map compensated in the step S5. As mentioned, the compensated ToF depth map is calibrated and enlarged according to an instruction from an application software executed on the electronic device 100.

As described above, according to the present disclosure, a ToF depth map created by the iToF method can be compensated even if it has an unmeasurable area where a distance from an electronic device 100 to a subject cannot be measured because a phase difference between the emitted light and the reflected light is greater than a limit value or because the subject easily absorbs the emitted light.

As a result, according to the present disclosure, the scope of application of the iToF method can be expanded.

The present disclosure can be applied to an application which estimates a size of the subject or adds blur to a color image. Further, the present disclosure can be applied to various applications such as face recognition, Virtual Reality (VR) and Augmented Reality (AR) .

In the description of embodiments of the present disclosure, it is to be understood that terms such as "central" , "longitudinal" , "transverse" , "length" , "width" , "thickness" , "upper" , "lower" , "front" , "rear" , "back" , "left" , "right" , "vertical" , "horizontal" , "top" , "bottom" , "inner" , "outer" , "clockwise" and "counterclockwise" should be construed to refer to the orientation or the position as described or as shown in the drawings in discussion. These relative terms are only used to simplify the description of the present disclosure, and do not indicate or imply that the device or element referred to must have a particular orientation, or must be constructed or operated in a particular orientation. Thus, these terms cannot be constructed to limit the present disclosure.

In addition, terms such as "first" and "second" are used herein for purposes of description and are not intended to indicate or imply relative importance or significance or to imply the number of indicated technical features. Thus, a feature defined as "first" and "second" may comprise one or more of this feature. In the description of the present disclosure, "a plurality of" means “two or more than two” , unless otherwise specified.

In the description of embodiments of the present disclosure, unless specified or limited otherwise, the terms "mounted" , "connected" , "coupled" and the like are used broadly, and may be, for example, fixed connections, detachable connections, or integral connections; may also be mechanical or electrical connections; may also be direct connections or indirect connections via intervening structures; may also be inner communications of two elements which can be understood by those skilled in the art according to specific situations.

In the embodiments of the present disclosure, unless specified or limited otherwise, a structure in which a first feature is "on" or "below" a second feature may include an embodiment in which the first feature is in direct contact with the second feature, and may also include an embodiment in which the first feature and the second feature are not in direct contact with each other, but are in contact via an additional feature formed therebetween. Furthermore, a first feature "on" , "above" or "on top of" a second feature may include an embodiment in which the first feature is orthogonally or obliquely "on" , "above" or "on top of" the second feature, or just means that the first feature is at a height higher than that of the second feature; while a first feature "below" , "under" or "on bottom of" a second feature may include an embodiment in which the first feature is orthogonally or obliquely "below" , "under" or "on bottom of" the second feature, or just means that the first feature is at a height lower than that of the second feature.

Various embodiments and examples are provided in the above description to implement different structures of the present disclosure. In order to simplify the present disclosure, certain elements and settings are described in the above. However, these elements and settings are only by way of example and are not intended to limit the present disclosure. In addition, reference numbers and/or reference letters may be repeated in different examples in the present disclosure. This repetition is for the purpose of simplification and clarity and does not refer to relations between different embodiments and/or settings. Furthermore, examples of different processes and materials are provided in the present disclosure. However, it would be appreciated by those skilled in the art that other processes and/or materials may also be applied.

Reference throughout this specification to "an embodiment" , "some embodiments" , "an exemplary embodiment" , "an example" , "a specific example" or "some examples" means that a particular feature, structure, material, or characteristics described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. Thus, the appearances of the above phrases throughout this specification are not necessarily referring to the same embodiment or example of the present disclosure. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

Any process or method described in a flow chart or described herein in other ways may be understood to include one or more modules, segments or portions of codes of executable instructions for achieving specific logical functions or steps in the process, and the scope of a preferred embodiment of the present disclosure includes other implementations, in which it should be understood by those skilled in the art that functions may be implemented in a sequence other than the sequences shown or discussed, including in a substantially identical sequence or in an opposite sequence.

The logic and/or step described in other manners herein or shown in the flow chart, for example, a particular sequence table of executable instructions for realizing the logical function, may be specifically achieved in any computer readable medium to be used by the instructions execution system, device or equipment (such as a system based on computers, a system comprising processors or other systems capable of obtaining instructions from the instructions execution system, device and equipment executing the instructions) , or to be used in combination with the instructions execution system, device and equipment. As to the specification, "the computer readable medium" may be any device adaptive for including, storing, communicating, propagating or transferring programs to be used by or in combination with the instruction execution system, device or equipment. More specific examples of the computer readable medium comprise but are not limited to: an electronic connection (an electronic device) with one or more wires, a portable computer enclosure (a magnetic device) , a random access memory (RAM) , a read only memory (ROM) , an erasable programmable read-only memory (EPROM or a flash memory) , an optical fiber device and a portable compact disk read-only memory (CDROM) . In addition, the computer readable medium may even be a paper or other appropriate medium capable of printing programs thereon, this is because, for example, the paper or other appropriate medium may be optically scanned and then edited, decrypted or processed with other appropriate methods when necessary to obtain the programs in an electric manner, and then the programs may be stored in the computer memories.

It should be understood that each part of the present disclosure may be realized by the hardware, software, firmware or their combination. In the above embodiments, a plurality of steps or methods may be realized by the software or firmware stored in the memory and executed by the appropriate instructions execution system. For example, if it is realized by the hardware, likewise in another embodiment, the steps or methods may be realized by one or a combination of the following techniques known in the art: a discrete logic circuit having a logic gate circuit for realizing a logic function of a data signal, an application-specific integrated circuit having an appropriate combination logic gate circuit, a programmable gate array (PGA) , a field programmable gate array (FPGA) , etc.

Those skilled in the art shall understand that all or parts of the steps in the above exemplifying method of the present disclosure may be achieved by commanding the related hardware with programs. The programs may be stored in a computer readable storage medium, and the programs comprise one or a combination of the steps in the method embodiments of the present disclosure when run on a computer.

In addition, each function cell of the embodiments of the present disclosure may be integrated in a processing module, or these cells may be separate physical existence, or two or more cells are integrated in a processing module. The integrated module may be realized in a form of hardware or in a form of software function module and is sold or used as a standalone product, the integrated module may be stored in a computer readable storage medium.

The storage medium mentioned above may be read-only memories, magnetic disks, CD, etc.

Although embodiments of the present disclosure have been shown and described, it would be appreciated by those skilled in the art that the embodiments are explanatory and cannot be construed to limit the present disclosure, and changes, modifications, alternatives and variations can be made in the embodiments without departing from the scope of the present disclosure.

Claims

A method of compensating a ToF depth map, comprising:

acquiring a color image captured by an image sensor module;

acquiring a ToF depth map and an IR image, both of which are captured by a range sensor module and correspond to the color image;

creating a stereo depth map based on the color image and the IR image; and

compensating the ToF depth map based on the stereo depth map.
The method according to claim 1, wherein the compensating the ToF depth map comprises filling an unmeasurable area in the ToF depth map with depth values of a corresponding area of the stereo depth map, the unmeasurable area being an area where a distance from an electronic device having the image sensor module and the range sensor module to a subject cannot be measured.
The method according to claim 1 or 2, further comprising preprocessing the color image after the acquiring the color image captured by the image sensor module.
The method according to claim 3, wherein the preprocessing the color image comprises calibrating the color image to match the IR image by using calibration parameters.
The method according to claim 4, wherein the preprocessing the color image further comprises shrinking the calibrated color image so that it has substantially the same size as a size of the IR image.
The method according to claim 5, wherein the creating the stereo depth map comprises:

calculating, for each pixel, a parallax amount between the preprocessed color image and the IR image; and

calculating, for each parallax amount, a distance between a subject and the range sensor module.
The method according to claim 6, wherein the distance is calculated by means of an equation (1) ,

D = B×f/Z ... (1) ,

where D is the distance, B is a baseline length between the image sensor module and the range sensor module, f is a focal length of the range sensor module, and Z is the parallax amount.
The method according to any one of claims 1 to 7, further comprising postprocessing the compensated ToF depth map after compensating the ToF depth map.
The method according to claim 8, wherein the postprocessing the compensated ToF depth map comprises calibrating the compensated ToF depth map to match the color image by using calibration parameters.
The method according to claim 9, wherein the postprocessing the compensated ToF depth map further comprises enlarging the calibrated ToF depth map.
An electronic device, comprising a processor and a memory for storing instructions, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any of claims 1 to 10.
A computer-readable storage medium, on which a computer program is stored, wherein the computer program is executed by a computer to implement the method according to any of claims 1 to 10.