TWI707152B - Computation apparatus, sensing apparatus, and processing method based on time of flight - Google Patents

Abstract

A computation apparatus, a sensing apparatus, and a processing method based on time-of-flight (ToF) are provided. In the method, intensity information of at least one pixel is obtained. The intensity information is obtained by sensing a modulating light through time difference or phase difference. Current depth information of a pixel to be evaluated in the at least on pixel is calculated according to the intensity information at a current time point. Whether to use the current depth information to be an output of the pixel to be evaluated at the current time point is determined according to a difference between the current depth information and a previous depth information of the pixel to be evaluated corresponding to at least one previous time point. Accordingly, the influence caused by noise would be improved for depth information estimation, and non-motion object condition is considered.

Description

Computing device, sensing device and processing method based on flight time ranging

The present invention relates to an optical measurement technology, and particularly relates to a computing device, sensing device and processing method based on Time of Flight (ToF) distance measurement.

With the development of technology, optical three-dimensional measurement technology has gradually matured, and time-of-flight ranging is currently a common active depth sensing technology. The basic principle of ToF ranging technology is that modulated light (for example, infrared light, or laser light, etc.) will be reflected when it encounters an object after being emitted, and it is converted by the reflection time difference or phase difference of the reflected modulated light The distance of the object to be photographed can generate depth information relative to the object (ie, relative distance).

It is worth noting that due to physical limitations such as the change of the light source (used to emit modulated light) to the waveform, or the thermal noise of the sensor (used to sense modulated light), the light source is different from The sensor may cause noise. For static objects, such The noise also causes differences in the distance measured at different time points. On the other hand, when using ToF ranging technology to calculate depth information, if it encounters motion blur, it will result in inaccurate depth distance or blurry picture. Therefore, how to provide a simple and effective method to reduce the influence of noise and motion blur has become one of the goals to be worked on in the related fields.

In view of this, the embodiments of the present invention provide an arithmetic device, a sensing device and a processing method based on time-of-flight ranging, which can effectively reduce the influence of noise on the depth calculation result, taking into account the dynamic blur.

The arithmetic device based on time-of-flight ranging in the embodiment of the present invention includes a memory and a processor. The memory records intensity information corresponding to at least one pixel and a program code corresponding to the processing method used in the computing device, and the intensity information is related to the signal intensity obtained by sensing the modulated light through time difference or phase difference. The processor is coupled to the memory and is configured to execute code, and the processing method includes the following steps: calculating the current depth information of one of the pixels to be evaluated according to the intensity information at the current time point. Based on the difference between the current depth information of the pixel to be evaluated and the previous depth information corresponding to at least one previous time point, it is determined whether to use the current depth information as the output of the pixel to be evaluated at the current time point, and this previous depth information is related to At least one intensity information obtained at a previous point in time.

On the other hand, the processing method based on the time-of-flight distance measurement in the embodiment of the present invention includes the following steps: obtaining intensity information corresponding to at least one pixel, and the intensity data The signal is related to the signal strength sensed by time-of-flight ranging technology. Calculate the current depth information of each pixel based on the intensity information at the current time point. Based on the difference between the current depth information of a pixel to be evaluated and its previous depth information in those pixels, decide whether to use the current depth information as the output of the pixel to be evaluated at the current time point, and this previous depth information is related to at least one previous time point Strength information obtained.

In addition, the sensing device based on the time-of-flight distance measurement in the embodiment of the present invention includes the aforementioned arithmetic device based on the time-of-flight distance measurement, a modulated light emitting circuit and a modulated light receiving circuit. The modulated light emitting circuit is used for emitting modulated light. The modulated light receiving circuit is coupled to the arithmetic device and used for receiving the modulated light to generate a sensing signal.

Based on the foregoing, the embodiment of the present invention is based on the computing device, sensing device, and processing method for time-of-flight ranging. For each pixel, the difference in depth information between before and after time points is judged to evaluate whether the photographed object is static or not, and based on it Noise is eliminated for pixels corresponding to static objects, and the depth information of pixels corresponding to non-static objects (ie, dynamic objects) is preserved. In this way, the influence of noise on the estimation of depth information can be effectively reduced, and the situation of non-static objects can be considered at the same time.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

10: Ranging system

100: sensing device

110: Modulated light emitting circuit

120: Modulated light receiving circuit

122: photoelectric sensor

130: processor

140: signal processing circuit

150: memory

160: computing device

410, 430: measured value

CA, CB: capacitance

QA, QB: the amount of charge changed

CS: Control signal

CSB: Inverted control signal

DS: Sensing signal

EM: Modulated light

MS: Modulation signal

NA, NB: Node

REM: reflected modulated light

SW1, SW2: switch

V _A , V _B : voltage signal

TA: target object

S310~S330, S510~S570, S610~S673: steps

Fig. 1 is a schematic diagram of a ranging system according to an embodiment of the present invention.

2A is a circuit diagram of a modulated light receiving circuit according to an embodiment of the present invention Figure.

FIG. 2B is a schematic diagram of signal waveforms according to the embodiment of FIG. 2A.

FIG. 3 is a flowchart of a processing method based on time-of-flight ranging according to an embodiment of the present invention.

Fig. 4A is an example illustrating the image obtained by sensing.

Fig. 4B is an example illustrating the effect of noise and the result of noise elimination.

Fig. 5 is a flowchart of a processing method based on time-of-flight ranging according to the first embodiment of the present invention.

Fig. 6 is a flowchart of a processing method based on time-of-flight ranging according to a second embodiment of the present invention.

FIG. 1 is a schematic diagram of a ranging system 10 according to an embodiment of the invention. Please refer to FIG. 1, the ranging system 10 includes a ToF-based sensing device 100 and a target object TA.

The sensing device 100 includes, but is not limited to, a modulated light emitting circuit 110, a modulated light receiving circuit 120, a processor 130, a signal processing circuit 140, and a memory 150. The sensing device 100 can be applied to fields such as three-dimensional model modeling, object recognition, vehicle auxiliary systems, positioning, production line testing, or error correction. The sensing device 100 may be a stand-alone device, or may be modularized and mounted on other devices, which is not intended to limit the scope of the present invention.

The modulated light emitting circuit 110 is, for example, a vertical cavity surface emitting laser array (VCSEL), light emitting diode (LED), laser diode or collimated light generating device, and the modulated light receiving circuit 120 is, for example, a camera device or a light source sensing device (including at least a light sensor, a reading circuit Wait). The signal processing circuit 140 is coupled to the modulated light emitting circuit 110 and the modulated light receiving circuit 120. The signal processing circuit 140 is used to provide the modulated signal MS to the modulated light transmitting circuit 110 and provide the control signal CS to the modulated light receiving circuit 120. The modulated light emitting circuit 110 is used for emitting the modulated light EM according to the modulated signal MS, and the modulated light EM is, for example, infrared light, laser light or collimated light of other wavebands. For example, the modulation signal MS is a pulse signal, and the rising edge of the modulation signal MS corresponds to the trigger time of the modulation light EM. The modulated light EM will be reflected after encountering the target object TA, and the modulated light receiving circuit 120 can receive the reflected modulated light REM. The modulated light receiving circuit 120 demodulates the reflected modulated light REM according to the control signal CS to generate a sensing signal DS.

More specifically, FIG. 2A is a schematic circuit diagram of the modulated light receiving circuit 120 according to an embodiment of the invention. Please refer to FIG. 2A. For the convenience of description, this drawing uses a unit/single pixel circuit as an example. The circuit corresponding to the unit/single pixel in the modulated light receiving circuit 120 includes a photo sensor element 122, a capacitor CA, a capacitor CB, a switch SW1 and a switch SW2. The photo sensor 122 is, for example, a photodiode or other light sensing element having a function similar to sensing the reflected modulated light REM. One end of the photoelectric sensor 122 receives a common reference voltage (for example, ground GND), and the other end is coupled to one end of the switch SW1 and the switch SW2. The other end of the switch SW1 is coupled to the capacitor CA through the node NA and is controlled by the inverted signal CSB of the control signal CS. The other end of the switch SW2 is coupled to the capacitor CB through the node NB and is controlled by the control signal CS. Voltage on the output node NA 120 modulation light receiving circuit (or current) signal V _{A and the} voltage at the node NB (or current) signal V _B as a sensing signal DS. In another embodiment, the modulated light receiving circuit 120 can also select the difference between the output voltage signal V _A and the voltage signal V _B as the sensing signal DS (which can be used as intensity information).

The embodiment of FIG. 2A is only for illustration, and the circuit structure of the modulated light receiving circuit 120 is not limited thereto. The modulated light receiving circuit 120 may have a plurality of photoelectric sensors 122, or more capacitors or switches. Those with general knowledge in this field can make appropriate adjustments based on general knowledge and actual needs.

FIG. 2B is a schematic diagram of signal waveforms according to the embodiment of FIG. 2A. 2A and 2B, when the inverted control signal CSB is at a low level (for example, logic 0), the switch SW1 is turned on, and the control signal CS will be at a high level (for example, logic 1), and the switch SW2 No conduction. Conversely, when the control signal CS is at a low level (for example, logic 0), the switch SW2 is turned on, and the inverted control signal CSB is at a high level (for example, logic 1), and the switch SW1 is not turned on. In addition, the photoelectric sensor 122 is turned on to enable the photoelectric sensor 122 to receive the reflected modulated light REM. When the photo sensor 122 and the switch SW1 are both turned on, the capacitor CA is discharged (or charged). QA in FIG. 2B represents the amount of charge changed by the capacitor CA, and the voltage signal V _A on the node NA will change accordingly. When the photo sensor 122 and the switch SW2 are both turned on, the capacitor CB is discharged (or charged). QB in FIG. 2B represents the amount of charge changed by the capacitor CB, and the voltage signal V _B on the node NB will change accordingly.

The processor 130 is coupled to the modulated light receiving circuit 120. The processor 130 may be a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessor), digital signal processors (Digital Signal Processor, DSP), programmable Integrated circuit (Application-Specific Integrated Circuit, ASIC) or other similar components or a combination of the above components. In the embodiment of the present invention, the processor 130 may calculate the phase difference between the control signal CS and the reflected modulated light REM according to the sensing signal DS, and perform distance measurement according to the phase difference. For example, referring to FIG. 2B, the processor 130 can calculate the phase difference between the control signal CS and the reflected modulated light REM according to the difference between the voltage signal V _A and the voltage signal V _B. It should be noted that, in some embodiments, the processor 130 may be built-in or electrically connected with an analog-to-digital converter (Analogy-to-Digital, ADC), and convert the sensing signal DS through the analog-to-digital converter Signals in digital form.

The memory 150 is coupled to the processor 140, and the memory 150 can be any type of fixed or removable random access memory (Random Access Memory, RAM), flash memory (Flash Memory), or traditional hard disk (Hard Disk). Drive, HDD), solid-state disk (Solid-State Disk, SSD), non-volatile (non-volatile) memory or similar components or a combination of these components. In this embodiment, the memory 150 is used to store buffered or permanent data (for example, intensity information, threshold value, depth information, etc. corresponding to the sensing signal DS), code, software module, operating system, and application program , Driver and other data or files, and their detailed content will be detailed in subsequent embodiments. It is worth noting that the program code recorded in the memory 150 is used for the processing method of the sensing device 100, and subsequent embodiments will describe this processing method in detail.

It should be noted that, in some embodiments, the processor 130 and the memory 150 may be separated to become the computing device 160. The computing device 160 can be a desktop computer, a notebook computer, a server, a smart phone, or a tablet computer. The computing device 160 and the sensing device 100 further have a communication transceiver capable of communicating with each other (for example, a transceiver supporting Wi-Fi, Bluetooth, Ethernet and other communication technologies), so that the computing device 160 can obtain data from The sensing signal DS of the sensing device 100 or the corresponding intensity information (can be stored in the memory 140 for the processor 130 to access).

In order to facilitate the understanding of the operation process of the embodiment of the present invention, a number of embodiments will be given below to describe in detail the operation process of the sensing device 100 and/or the computing device 160 in the embodiment of the present invention. Hereinafter, various components and modules in the sensing device 100 and the computing device 160 will be used to describe the method according to the embodiment of the present invention. Each process of the method can be adjusted accordingly according to the implementation situation, and is not limited to this.

FIG. 3 is a flowchart of a processing method based on time-of-flight ranging according to an embodiment of the present invention. 3, the processor 130 calculates the current depth information of a pixel to be evaluated in at least one pixel of the modulated light receiving circuit 120 (ie, a certain pixel in the image at the current time point) according to the intensity information at the current time point (Step S310). Specifically, in the embodiment of FIG. 2B, the modulation signal MS is synchronized with the control signal CS, but the signal processing circuit 140 can also make the modulation signal MS and the control signal CS asynchronous. In other words, there can be a reference phase between the control signal CS and the modulation signal MS. The signal processing circuit 140 delays or advances the phase of the modulation signal MS or the control signal CS according to different reference phases, so that the modulation signal MS and the control signal CS have a phase difference/phase delay.

In the continuous wave (CW) measurement mechanism, the phase difference is, for example, 0 degrees, 90 degrees, 180 degrees, and 270 degrees, that is, the four-phase method. Different phases correspond to the charge accumulation time intervals at different starting and ending time points. In other words, the modulated light receiving circuit 120 uses four phases to receive the reflected modulated light REM with a time delay. The reflected modulated light REM is sensed by those phases delayed in time to obtain sensing signals DS corresponding to different phases, and these sensing signals DS can be further used as intensity information. The intensity information of each pixel may record the amount of charge accumulated by its corresponding modulated light receiving circuit 120 (as shown in FIG. 2A) or be further converted into an intensity value. That is, the intensity information of each pixel uses those phases that are delayed in time (ie, phase difference or time difference) to sense the signal intensity of the reflected modulated light REM (ie, ToF technology).

The processor 130 obtains the intensity information of the pixel to be evaluated at fixed or variable intervals (for example, sampling time). In this article, each sampling time point at which the intensity information is obtained is referred to as a time point. The current time point represents the current sampling time point; the previous time point represents the sampling time point before the current sampling time point or a sampling time point earlier than the current time point.

Then, the processor 130 determines whether to use the current depth information as the output of the pixel to be evaluated at the current time point according to the difference between the current depth information of the pixel to be evaluated and its previous depth information (step S330). Specifically, it can be known through experiments that noise phenomena such as the modulated light EM and the modulated light receiving circuit 120 will cause differences in intensity information between different time points. For example, FIG. 4A is an example illustrating the image obtained by sensing. Please refer to Figure 4A, which shows the target object in the shooting scene Both the TA and the sensing device 100 are images generated according to the sensing signal DS in a static state (for example, no shaking, no movement, etc.). Fig. 4B is an example illustrating the effect of noise and the result of noise elimination. Please refer to FIG. 4B. The measurement value 410 shown in the figure is a measurement value (ie, intensity information) obtained by measuring the sensing signal DS at each different time point. It can be seen that, for a static object TA, the measured value 410 will vary greatly over time. If the measurement value 410 at each sampling time point is averaged with the previous measurement value of at least one previous time point (take 10 sampling time points as an example), a relatively small (more stable) measurement can be obtained The value is 430. Although the averaging of the measurement values can eliminate noise, applying the numerical averaging method to the object TA or the sensing device 100 in a non-static state may cause incorrect measurement results.

In order to solve this problem, the embodiment of the present invention will first determine whether the target object TA is a static object according to the sensing result of each pixel obtained by shooting the target object TA, and exclude the case where the numerical average method is applied to the target object TA as a non-static object. Avoid interpreting changes in the intensity or depth information obtained by shooting a dynamic target object TA as noise. In other words, the processor 130 will determine, based on the dynamic evaluation result, whether each pixel in an image at the current time point is to use a numerical average method to obtain its depth information. For non-static objects, the current depth information obtained at the current time point can represent the relative distance of the object TA. That is to say, the difference between the current depth information and the previous depth information will be one of the key factors in determining whether to use the current depth information or the numerical average result as a representative of the output of the pixel at the current point in time. It should be noted that the previous depth information here is related to intensity information obtained at at least one previous time point (or within a certain time interval).

The following will describe in detail how to obtain the dynamic evaluation result and the corresponding processing method: FIG. 5 is a flowchart of a processing method based on the time-of-flight ranging according to the first embodiment of the present invention. Referring to FIG. 5, the processor 130 inputs the pixel to be evaluated at time t+1 from the memory 150 or the modulated light receiving circuit 120 (step S510). Specifically, the processor 130 obtains the depth information corresponding to the pixel to be evaluated at the current time point. Then, the processor 130 determines which depth information to use based on the difference between the information obtained by the same pixel to be evaluated at different time points.

For a pixel to be evaluated, the processor 130 determines whether the difference between the current depth information at time t+1 and the previous depth information at time t (ie, the previous time point) is greater than the noise threshold (step S530). Specifically, the experimental observation shows that noise is one of the main reasons that affect the sensing results of static objects. In other words, the difference between the measurement results of static objects at different time points should be affected by noise. If the difference between the measurement results of static objects at different time points is too large, the difference should not only be affected by noise, but more likely to be caused by non-static objects. In the embodiment of the present invention, the noise threshold is used to determine whether the pixel to be evaluated is a static object at the current time point. This noise threshold represents the maximum depth variation caused by noise. In addition, since the intensity of the modulated light EM and the relative distance of the object TA will affect the intensity of the sensing signal DS, and accordingly cause different degrees of noise, the noise threshold value is related to the current time point of the pixel to be evaluated Strength information. That is, for different time points, the processor 130 will change the noise threshold value according to the depth information corresponding to the pixel to be evaluated.

It should be noted that the embodiment of the present invention uses the average value of the depth information of at least one previous time point as the previous depth information. That is, the depth information of these previous time points is accumulated/summed and divided by the number. The sensed objects at these previous time points are static objects, and their depth information does not change much. Therefore, the embodiment of the present invention uses the average value of these depth information as the basis for comparison.

It should be noted that the embodiment of the present invention does not limit the number of previous time points. For example, the processor 130 may use the average value of all sampling time points from the next sampling time point of the time point when the non-static object was detected last to the previous sampling time point of the current time point, or use a specific The average of the number of previous time points.

Then, if the difference between the current depth information and the previous depth information is greater than the noise threshold, the processor 130 determines that the sensing result of the pixel to be evaluated is a non-static object, and the processor 130 updates the pixel to be evaluated to Depth information at time t+1 (step S535). The processor 130 may use the current depth information at the current time point as the output of the pixel to be evaluated at the current time point. In other words, for the pixels where the non-static object is sensed, the processor 130 will not/disable using the numerical average result as the output at the current time point. This output can represent the distance (ie, depth information) of the object TA sensed by the pixel to be evaluated at the current point in time, and can be used by other components or applications (for example, camera programs, ranging programs, etc.).

On the other hand, if the difference between the current depth information and the previous depth information is not greater than the noise threshold, the processor 130 determines that the sensing result of the pixel to be evaluated is a static object, and the processor 130 will determine the pixel to be evaluated. Update to average depth Degree information (step S550). Regarding the average depth information, the processor 130 can determine the output of the pixel to be evaluated at the current time point according to the current depth information and the previous depth information.

In the embodiment of the present invention, if the difference between the current depth information and the previous depth information is not greater than the noise threshold, the processor 130 accumulates the current depth information of the pixel to be evaluated into cumulative information. The accumulated information is the sum of the previous depth information corresponding to the pixel to be evaluated at at least one previous time point. For example, if it is judged that it belongs to a static object from t-5 to t, the above-mentioned accumulated information is the sum of the previous depth information from t-5 to t. At the current time point t+1, the current depth information accumulated to the cumulative information is the sum of the depth information values from the time point t-5 to t+1. Then, the processor 130 uses the average result of the accumulated information as the output of the pixel to be evaluated at the current time point. The average result (that is, the average depth information) is to average the value of the depth information of the pixel to be evaluated in the time domain as a basic algorithm for noise elimination: Dt ' = ( D 1+ D 2+ D 3+ … Dt )/ t ... (1) D1 is the depth information obtained at the first time point. By analogy, Dt is the value of the depth information obtained at time t, and Dt' is the output at time t. That is, the sum of the values of the depth information divided by the number.

It should be noted that the embodiment of the present invention does not limit the number of previous time points in the accumulated information. For example, the processor 130 may use all the sampling time points between the next sampling time point of the time point when the non-static object is detected last time and the previous sampling time point of the current time point, or use a specific number of The previous point in time. In addition, in other embodiments, the processor 130 may also directly use the current depth information Or one of the previous depth information is used as the output of the pixel to be evaluated at the current time point.

Then, the processor 130 may display the updated depth information after step S535 or S550 (step S570). For example, the processor 130 presents an output image including all pixels through the display. The updated in-depth information is the output of the aforementioned current time point. By analogy, the processor 130 can update the depth information for all pixels in the image. It should be noted that for each time point, the processor 130 repeats the process of the first embodiment to obtain the output at each time.

Fig. 6 is a flowchart of a processing method based on time-of-flight ranging according to a second embodiment of the present invention. In this embodiment, it will be further evaluated whether the captured image has global motion blur (global motion blur), that is, the sensing device 100 is moving rather than stationary during shooting, and the depth information to be used will be determined based on that. This can effectively reduce the influence of global motion blur on depth information estimation. Referring to FIG. 6, the processor 130 inputs the image at time t+1 (ie, the image at the current time point) from the memory 150 or the modulated light receiving circuit 120 (step S610).

For a single pixel to be evaluated in the image, the processor 130 determines whether the difference between the current depth information at time t+1 and the previous depth information at time t (ie, the previous time point) is less than the noise threshold (step S630). It should be noted that the judgment process of the dynamic evaluation result in step S630 can refer to the content of step S530, which will not be repeated here. If the difference is not less than the noise threshold (for example, the difference is greater than the noise threshold), it means that the sensing result belongs to a non-static object, and the processor 130 will update the depth information at time t+1 according to step S535, and The global dynamic count value is further accumulated (step S631). The global dynamic count value will be explained later. another On the other hand, if the difference is less than the noise threshold (for example, the difference is not greater than the noise threshold), it means that the sensing result belongs to a static object, and the processor 130 will update the average depth information according to step S550 (step S633).

The processor 130 determines whether the evaluation of all pixels in the image is completed (step S650). For example, whether all pixels have obtained dynamic evaluation results and update the depth information accordingly. If there are still pixels in the image that have not been evaluated or updated, the process returns to step S630 for this pixel, that is, the processes of steps S630 to S650 are performed separately for these pixels that have not yet been evaluated.

If all pixels have updated the depth information, the processor 130 determines whether the image at the current time point (ie, time point t+1) has global motion blur. Specifically, if the sensing device 100 moves, the sensing result will have global motion blur. All the pixels acquired at the current time point will also be affected by global motion blur. The embodiment of the present invention judges whether global motion blur occurs based on the global dynamic count value. The processor 130 can add up the blur number of all pixels in the image whose difference is greater than the noise threshold value (ie, the global dynamic count value), and determine the global area according to the ratio of the blur number to the number of all pixels in an image The occurrence of motion blur. In other words, the amount of blur represents the number of pixels in an image that are judged to have non-static objects, and the amount of blur will be the basis for judging global dynamic blur.

In this embodiment, whenever a pixel is determined to have a non-static object in step S631, the processor 130 will increase the global dynamic count value by one. Then, the processor 130 determines that the global dynamic count value obtained after updating the depth information of all pixels is Whether the proportion of all pixels is greater than the proportion threshold (step S670). This global dynamic count value is the aforementioned fuzzy number. In other words, if the difference between the depth information corresponding to a certain pixel at different time points is greater than the noise threshold, the global dynamic count value is accumulated, and after all pixels are evaluated, the final blur amount (ie , The accumulated global dynamic count value).

It is worth noting that the ratio threshold is a reference benchmark for judging global dynamic blur. For example, the threshold of the ratio is 20%, 30%, or 40%. Taking a 240×180 resolution image as an example, if the ratio threshold value for comparison is set to 20%, the number of blurred pixels in the image frame is 8640. However, the ratio threshold and the blur amount may be adjusted according to different resolutions or other conditions, which are not limited in the embodiment of the present invention.

If the proportion of the global dynamic count value is not greater than the proportion threshold, it means that the global dynamic blur has not occurred, and the processor 130 will maintain the output corresponding to all pixels in the image (step S671). That is, the updated depth information in steps S631 and S633 is used as output.

On the other hand, if the proportion of the global dynamic count value is greater than the proportion threshold, it represents the occurrence of global dynamic blur. The processor 130 changes the output corresponding to all pixels in the image to the corresponding current depth information, and then displays t +1 time point of video (step S673). Therefore, in step S673, the pixels previously updated as average depth information in step S633 will be changed to use the current depth information (time point t+1) as output. On the other hand, the pixels updated to time t+1 in step S631 will continue to use the current depth information as output.

It should be noted that the second embodiment uses the number of blurs to determine the occurrence of global motion blur. In other embodiments, the processor 130 may also be additionally mounted on the sensing device 100 via a posture detector (for example, a gravity sensor (G-sensor)/accelerometer), inertial (Inertial) sensing A sensor, a gyroscope, a magnetometer, or a combination thereof) evaluate the posture information of the sensing device 100 (for example, three-axis gravitational acceleration, angular velocity, or magnetic force), and then obtain the sensing Whether the device 100 is moving (ie, the occurrence of global motion blur).

In summary, the arithmetic device, sensing device, and processing method based on the time-of-flight distance measurement in the embodiments of the present invention can determine whether there is a non-static object based on the difference in depth information between the current time point and the previous time point, and make a comparison accordingly. Static objects use numerical averaging to eliminate noise, and non-static objects use current depth information as output. In addition, if a global motion blur phenomenon is evaluated, the embodiment of the present invention can update the output of all pixels in the image to the current depth information. In this way, the influence of noise and motion blur on the depth information estimation can be reduced in a simple manner.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

S310~S330: steps

Claims (13)

An arithmetic device based on flight time ranging, including: A memory for recording intensity information corresponding to at least one pixel and a code corresponding to a processing method used in the computing device, wherein the intensity information is related to the signal intensity obtained by sensing a modulated light through a time difference or a phase difference; as well as A processor coupled to the memory and configured to execute the program code, the processing method includes: Calculating a current depth information of a pixel to be evaluated in the at least one pixel according to the intensity information at a current time point; and According to the difference between the current depth information of the pixel to be evaluated and a previous depth information corresponding to at least a previous time point, it is determined whether to use the current depth information as an output of the pixel to be evaluated at the current time point. For example, the arithmetic device based on time-of-flight distance measurement described in item 1 of the scope of patent application, wherein the processing method further includes: Determine whether the difference is greater than a noise threshold; In response to the difference being greater than the noise threshold, use the current depth information as the output of the pixel to be evaluated at the current point in time; and In response to the difference being not greater than the noise threshold value, the output of the pixel to be evaluated at the current time point is determined based on the current depth information and the previous depth information. As described in item 2 of the scope of patent application, the computing device based on time-of-flight distance measurement, wherein the processing method further includes: In response to the difference being not greater than the noise threshold, the current depth information is accumulated to a cumulative information, where the cumulative information is related to the sum of the previous depth information corresponding to the pixel to be evaluated at the at least one previous time point ;as well as The average result of the accumulated information is used as the output of the pixel to be evaluated at the current time point. As described in item 2 of the scope of patent application, the computing device based on time-of-flight distance measurement, wherein the processing method further includes: Determining whether global motion blur occurs in an image at the current time point, where the image includes all the at least one pixel; Responding to the occurrence of the global motion blur, changing the output corresponding to all the at least one pixel in the image to the corresponding current depth information; and In response to the global motion blur not occurring, the output corresponding to all the at least one pixel in the image is maintained. For example, the computing device based on time-of-flight ranging according to item 4 of the scope of patent application, wherein the processing method further includes: Sum up all the blur quantities of the at least one pixel in the image whose difference is determined to be greater than the noise threshold; and The occurrence of the global motion blur is determined according to the proportion of the blur amount. For example, the computing device based on the time-of-flight ranging according to the scope of the patent application, wherein the noise threshold value is related to the intensity information at the current time point. A processing method based on flight time ranging, including: Obtaining intensity information corresponding to at least one pixel, where the intensity information is related to the signal intensity obtained by sensing a modulated light through a time difference or a phase difference; Calculating a current depth information of a pixel to be evaluated in the at least one pixel according to the intensity information at a current time point; and According to the difference between the current depth information of the pixel to be evaluated and a previous depth information corresponding to at least a previous time point, it is determined whether to use the current depth information as an output of the pixel to be evaluated at the current time point. For example, the processing method based on the time-of-flight ranging according to item 7 of the scope of patent application, wherein the step of determining whether to use the current depth information as the output of the pixel to be evaluated at the current time point includes: Determine whether the difference is greater than a noise threshold; In response to the difference being greater than the noise threshold, use the current depth information as the output of the pixel to be evaluated at the current point in time; and In response to the difference being not greater than the noise threshold value, the output of the pixel to be evaluated at the current time point is determined based on the current depth information and the previous depth information. For example, the processing method based on the time-of-flight ranging according to item 8 of the scope of patent application, wherein the step of determining the output of the pixel to be evaluated at the current time point according to the current depth information and the previous depth information includes: Accumulating the current depth information into cumulative information, where the cumulative information is related to the sum of the previous depth information corresponding to the pixel to be evaluated at the at least one previous point in time; and The average result of the accumulated information is used as the output of the pixel to be evaluated at the current time point. For example, the processing method based on time-of-flight ranging according to item 8 of the scope of patent application, after the step of judging whether the difference is greater than the noise threshold, further includes: Determining whether a global motion blur occurs in an image at the current time point, where the image includes all the at least one pixel; Responding to the occurrence of the global motion blur, changing the output corresponding to all the at least one pixel in the image to the corresponding current depth information; and In response to the global motion blur not occurring, the output corresponding to all the at least one pixel in the image is maintained. For example, the processing method based on time-of-flight ranging according to item 10 of the scope of patent application, wherein the step of determining whether the image at the current time point has global motion blur occurs includes: Sum up all the blur quantities of the at least one pixel in the image whose difference is determined to be greater than the noise threshold; and The occurrence of the global motion blur is determined according to the proportion of the blur amount. For example, the processing method based on time-of-flight ranging according to item 8 of the scope of patent application, wherein the noise threshold value is related to the intensity information at the current time point. A sensing device based on time-of-flight ranging, including: The computing device based on time-of-flight distance measurement as described in any one of items 1 to 6 of the scope of patent application; A modulated light emitting circuit which emits a modulated light; and A modulated light receiving circuit is coupled to the computing device and receives the modulated light to generate a sensing signal.

Images