CN115407365A - Depth measuring device and system - Google Patents

Abstract

The application relates to a degree of depth measuring device and system, the device include central processing unit, TOF unit, microcontroller and RGB sensor, wherein: the central processor is used for respectively generating a TOF control signal and an RGB control signal to the TOF unit and the RGB sensor and simultaneously sending a correction signal to the microcontroller; the TOF unit is used for emitting different modulation light beams according to the TOF control signal and collecting the reflected modulation light beams to generate at least two frames of catch images, transmitting the images to the central processing unit according to the TOF control signal and simultaneously generating a feedback signal to the microcontroller; the microcontroller is used for correcting the feedback signal according to the correction signal to generate an RGB trigger signal; the RGB sensor is used for acquiring RGB images according to the RGB trigger signals and sending the images to the central processing unit according to the RGB control signals; and the central processing unit is also used for calculating at least two frames of the wideband images to obtain a TOF depth image so as to synchronously output the RGB image and the TOF depth image. The embodiment of the application can improve the synchronization effect of the images.

Description

Depth measuring device and system

Technical Field

The application relates to the technical field of image acquisition and processing, in particular to a depth measuring device and system.

Background

Depth cameras can be classified into Time of flight (TOF), structured light, and the like according to the operating principle. Depth cameras have increasingly been used in various areas of human production and life, for example: robotics, interactive games, augmented Reality (AR), virtual Reality (vtoffual Reality, VR), unmanned and 3D modeling, and the like.

The TOF depth measurement technology has the advantages of good stability, strong practicability and the like, and is a technology which is more prominent in a plurality of optical three-dimensional measurement technologies. TOF depth measurement techniques include two, one of which is often referred to as pulse ranging, whose basic principle is to calculate the distance between an object to be measured (or an object detection area) and a TOF depth measurement device by the time interval from emission to reception of a laser pulse emitted by the TOF depth measurement device; another method, often referred to as phase difference ranging, is based on the principle of calculating the distance between an object to be measured (or an object detection area) and a TOF depth measuring device by means of a phase change generated by a laser beam emitted by the TOF depth measuring device traversing the object once.

On one hand, in order to improve detection accuracy and detection range, TOF depth measurement equipment generally adopts a multi-frequency fusion mode to perform depth detection, but when a multi-frequency fusion algorithm is used, multi-frame images need to be acquired to perform phase unwrapping. When the image is used for processing, the acquired multi-frame image needs to be spatially aligned and time aligned, the spatial alignment can be realized through image calibration, but the time alignment is difficult to realize.

On the other hand, to improve the texture of the TOF depth measuring device, an RGB camera is generally used to acquire an RGB image for texture mapping. However, in the prior art, the RGB camera and the TOF depth measuring device adopt respective timestamps to achieve synchronization, and when the implementation manner is adopted, on one hand, due to the difference of the exposure time of the RGB camera and the TOF depth measuring device, the printed timestamps have deviation, and finally, synchronization deviation can be caused; on the other hand, since there is task priority scheduling in the SOC chip and the synchronization task of the RGB camera and the TOF depth measuring device is behind the task priority, it is difficult to achieve synchronized output of the RGB camera and the TOF depth measuring device.

Disclosure of Invention

In view of this, embodiments of the present application provide a depth measuring device and a depth measuring system, which can solve at least one technical problem in the related art.

In a first aspect, an embodiment of the present application provides a depth measuring device, including: RGB sensor, TOF sensor, central processing unit and microcontroller, color sensor, including TOF unit, central processing unit and the microcontroller that throws module and TOF sensor, wherein: the central processing unit is used for respectively transmitting TOF control signals and RGB control signals to the TOF unit and the RGB sensor and simultaneously transmitting correction signals to the microcontroller; the TOF unit is used for receiving the TOF control signal and simultaneously generating a gating signal or a vertical synchronization signal to the microcontroller; the TOF sensor is used for collecting different modulation light beams reflected back by the target scene so as to generate at least two frames of wideband images and transmitting the wideband images to the central processing unit according to the TOF control signal; the microcontroller is used for correcting the gating signal or the vertical synchronizing signal according to the correction signal to generate an RGB trigger signal so as to trigger the RGB sensor to start working; the RGB sensor is used for acquiring RGB images according to the RGB trigger signals and sending the RGB images to the central processing unit according to the RGB control signals; and the central processing unit is also used for performing phase unwrapping and depth fusion calculation on at least two frames of the wideband images to obtain a TOF depth image so as to synchronously output the TOF depth image and the RGB image.

In a second aspect, an embodiment of the present application provides a depth measurement system, including: a plurality of depth measuring devices as described in embodiments of the first aspect.

The depth measuring device has the advantages that the depth measuring device achieves measurement of the extended distance, exposure synchronization is achieved through software and hardware synchronously, and the image synchronization effect is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic structural diagram of a depth measuring device according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of an image shape obtained by two exposure modes according to an embodiment of the present disclosure;

FIG. 3 is a timing diagram illustrating operations of modules of a depth measuring device according to an embodiment of the present disclosure;

fig. 4 is a timing diagram illustrating operation of modules of a master device according to an embodiment of the present disclosure;

fig. 5 is a timing diagram illustrating operation of modules when a slave device is synchronously triggered according to an embodiment of the present application;

fig. 6 is a timing diagram illustrating operation of modules when another slave device is triggered synchronously according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a connection between multiple depth measurement devices according to another embodiment of the present disclosure;

FIG. 8 is a schematic view of another embodiment of the present disclosure showing the connection between multiple depth measurement devices;

FIG. 9 is a schematic diagram of a depth measurement system according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another depth measurement system according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically stated.

Further, in the description of the present application, "a plurality" means two or more. The terms "first" and "second," etc. are used merely to distinguish one description from another, and are not to be construed as indicating or implying relative importance.

In order to explain the technical solution described in the present application, the following description will be given by way of specific examples.

Fig. 1 is a schematic structural diagram of a depth measuring device according to an embodiment of the present disclosure. As shown in fig. 1, the depth measuring device includes an RGB sensor 11, a TOF Unit 12, a central processor 13, and a Microcontroller (MCU) 14, wherein:

the central processing unit 13 is used for sending a TOF control signal to the TOF unit 12 to control the TOF unit 12 to start working, synchronously sending an RGB control signal to the RGB sensor 11, and simultaneously sending a correction signal to the microcontroller 14;

the TOF unit 12, including a projection module and a TOF sensor, is configured to receive a TOF control signal and simultaneously generate a feedback signal to the microcontroller 14; under the trigger of the TOF control signal, the projection module is used for emitting at least two different modulated light beams to the target scene, the TOF sensor is exposed twice at least to collect the different modulated light beams reflected back by the target scene so as to generate at least two frames of catch images, namely, one modulated light beam at least corresponds to one frame of catch image, and the catch image is transmitted to the central processor 13 according to the TOF control signal; the feedback signal comprises a gating signal or a vertical synchronization signal, and the raw data of the optical signal acquired by the TOF sensor is converted into a digital signal by the rapphase image;

the microcontroller 14 is used for receiving the correction signal and the feedback signal, and is further used for correcting the feedback signal according to the correction signal to generate an RGB trigger signal and transmitting the RGB trigger signal to the RGB sensor 12;

the RGB sensor 11 is used for receiving the RGB trigger signal, carrying out exposure according to the RGB trigger signal so as to acquire an RGB image of a target scene, and sending the RGB image to the central processing unit 13 according to the RGB control signal; wherein the exposure time of the RGB sensor 11 is aligned with the total time of at least two exposures of the TOF sensor;

and the central processing unit 13 is configured to synchronously receive at least two frames of the rake hash image and the RGB image according to the TOF control signal and the RGB control signal, and perform phase unwrapping and depth fusion calculation on the received at least two frames of the rake hash image to obtain a TOF depth image so as to synchronously output the TOF depth image and the RGB image.

It should be noted that the time interval of at least two exposures of the TOF sensor may be the same or different, but the time of one exposure of the RGB sensor 11 is aligned with the total time of at least two exposures of the TOF sensor.

In one embodiment, the microcontroller 14 may be configured with a real-time system for receiving the signals in real time to generate the trigger signals to the sensors in real time. That is, in the microcontroller 14, the microcontroller 14 is a single task processor, and is only used for processing the synchronous trigger task in the application, so that the microcontroller 14 receives signals of various paths and feeds back the trigger signal in real time. In one embodiment, the microcontroller 14 receives the correction signal and the feedback signal in real time, and corrects the feedback signal according to the correction signal to generate an RGB trigger signal, and the RGB trigger signal is used for the RGB sensor 12 to start working.

Under the trigger of TOF control signal, the synchronization mode of the projection module and the TOF sensor in the TOF unit 12 includes: a TOF sensor in the TOF unit 12 receives a TOF control signal to generate a TOF exposure signal to control the body exposure, and synchronously outputs a projection trigger signal to the projection module to drive the projection module to emit modulated light beams with different frequencies to the target scene. Preferably, the modulated beam comprises continuous waves of different frequencies or pulsed beams of different pulse periods.

In one embodiment, to ensure that the TOF sensor is exposed at least twice and aligned with the exposure time of the RGB sensor 11, a corresponding time stamp may be assigned to the image obtained at each exposure of the TOF sensor or the RGB sensor 11, so as to synchronously deliver the raw image and the RGB image to the central processor 13 according to the time stamp; if one frame of two frames of the flash picture and the RGB picture are synchronously conveyed according to the time stamp, because the total exposure time of the two frames of the flash picture and the exposure time of one frame of the RGB picture are aligned, the corresponding time stamps are respectively given to the two frames of the flash picture and the one frame of the RGB picture when the exposure is finished, and the synchronous conveying of one frame of the RGB picture can be realized while the two frames of the flash picture are conveyed.

In one embodiment, the TOF sensor or RGB sensor 11 exposure mode is rolling shutter exposure or global exposure. When the exposure mode is roller shutter exposure, it is also necessary to ensure that the exposure center of the TOF sensor which is exposed at least twice is aligned with the exposure center of the RGB sensor 11, so as to avoid poor alignment effect of the rewflash image and the RGB image. It should be noted that the exposure center is understood as the center of an image obtained after the sensor is exposed.

Specifically, when the RGB sensor 11 is exposed to a roller shutter, the resulting image is parallelogram-shaped; while the TOF sensor is globally exposed, the resulting image is rectangular, as shown in fig. 2. Therefore, the images obtained based on the two exposure modes are not aligned, so that the central processing unit 13 needs to adjust the exposure centers of the images to align the exposure centers, thereby obtaining aligned images. Specifically, the exposure centers of the exposure images obtained by different exposure modes are compared in advance, if the exposure centers are not aligned, the offset is calculated, and then the images are adjusted according to the offset for the alignment of the images and are synchronously transmitted to the central processing unit 13. It should be noted that the calculated offset may be obtained in advance and stored for use in subsequent image synchronization.

In one embodiment, the TOF depth image is obtained by performing phase unwrapping and depth fusion calculation on at least two frames of the wideband images, and the exposure time corresponding to the TOF depth image is a certain time between the exposure time periods of the two frames of the wideband images, so that in order to ensure the synchronous output effect of the RGB image and the TOF depth image, the central processor 13 also needs to correct the exposure centers of the RGB image and the TOF depth image to align the exposure centers.

In one embodiment, the TOF sensor comprises at least one pixel, each pixel comprising at least 3 and more than 3 taps (charge accumulating elements for collecting electrical signals generated by the light beam reflected back through the target scene), each tap being fixed and invariant at the moment of exposure within successive frame periods. In order to perform phase unwrapping on different modulated light beams reflected back by a target scene to obtain wrapping cycle numbers, the TOF sensor needs to respectively obtain at least one frame of rap image corresponding to the different modulated light beams in continuous different frame periods and transmit the frame of rap image to the central processing unit 13, and the central processing unit 13 performs unwrapping calculation according to the rap images corresponding to the different modulated light beams to obtain the wrapping cycle numbers of the different modulated light beams; and calculating by utilizing each winding period number to obtain depth values obtained based on different modulation light beams, and finally giving corresponding weights to each depth value corresponding to different modulation light beams for fusion calculation to obtain a TOF depth image.

In one embodiment, to improve depth value accuracy, the taps included in each pixel preferably collect the amount of charge generated by the reflected beam or background light in a round robin fashion. Taking three taps a, b and c as examples to realize a rotation mode, and in a first frame period, sequentially turning on an accumulated charge signal by a first tap, a second tap and a third tap; and in a second frame period, the second tap, the third tap and the first tap are sequentially switched on to accumulate the charge signals. The TOF sensor acquires at least two frames of wideband images under different modulated light beams according to the mode and transmits the images to the central processing unit 13; taking two different modulated light beams as an example, at this time, the TOF sensor needs to acquire 4 frames of rake images, that is, the TOF sensor needs to be exposed four times. The central processing unit 13 correspondingly superimposes the rapphase images obtained based on different frame periods according to different modulated light beams to eliminate noise to obtain a high-precision rapphase image, so that the depth precision is improved; for example, the same modulated light beam is superposed based on the rewhaze images obtained from different frame periods to eliminate noise. That is, when the projection module projects n different modulated light beams, the TOF sensor can acquire at least n frames of wideband images; to improve depth accuracy, the TOF sensor can each acquire at least 2n frames of a rake-hash image under n different modulated beams for noise cancellation.

It should be noted that the rotation mode of the taps is not limited to the above mode, and the third tap, the first tap, and the second tap may be turned on in sequence; and so on when each pixel includes more than 3 taps, and so on will not be described herein.

For ease of understanding, the embodiments of fig. 3 to 6 are illustrated with the TOF sensor acquiring at least two frames of a raw image each under at least two different modulated light beams and transmitting the acquired images to the central processor 13 to generate a frame of a TOF depth map.

Fig. 3 is a working timing diagram of each module of the depth measuring device according to the present application, and as shown in fig. 1 and fig. 3, when the TOF unit 12 receives a TOF control signal, a TOF sensor in the TOF unit generates a TOF exposure signal (denoted as TOF exposure) and performs at least four exposures according to the TOF exposure signal to acquire at least four frames of a rwhase image, and synchronously transmits a projection trigger signal to the projection module to emit at least two different modulated light beams to a target scene, and also simultaneously generates a high-level TOF gating signal (denoted as TOF strobe out); the TOF gating signal will be monitored by a real-time system carried by the microcontroller 14. When the real-time system carried by the microcontroller 14 monitors the TOF gating signal, the TOF gating signal is corrected according to the correction signal received from the central processor 13 synchronously, and a high-level RGB trigger signal (denoted as RGBtriggle in) is generated and sent to the RGB sensor 11 to control the RGB sensor 11 to perform one exposure (denoted as RGB exposure) to acquire an RGB image of the target scene. It should be noted that, the present embodiment is only described by taking the TOF gating signal as an example, in some other embodiments, the TOF unit 12 may also generate a Vertical Synchronization (VSYNC) signal or generate other synchronization signals to achieve synchronous exposure when receiving the TOF control signal, which is not limited herein.

An embodiment of the present application further provides a depth measurement system, where the depth measurement system includes a plurality of depth measurement apparatuses (measurement apparatuses for short), any one of the measurement apparatuses is selected as a Master device (Master device), the measurement apparatus is in a Master mode (Master mode), and the remaining measurement apparatuses are Slave devices (Slave devices), that is, the remaining measurement apparatuses are in a Slave mode (Slave mode). Wherein, the master device can generate a synchronization trigger signal (denoted as Ext trigger out) to the slave device to trigger the slave device to operate synchronously, and the synchronization trigger signal can be generated by any one of the RGB sensor 11, the TOF sensor or the microcontroller 14 in the master device. Preferably, in a situation where the microcontroller 14 is mounted on a real-time system, a synchronization trigger signal (denoted as Ext trigger out) may be generated by the microcontroller 14 in the master device to trigger the slave devices to synchronize, so as to ensure accurate and stable synchronization among multiple devices.

Specifically, fig. 4 shows an operation timing diagram of each module of the master device, and with reference to fig. 1 and 4, when a TOF gating signal (denoted as TOF strobe out) of the TOF unit 12 of the master device is fed back to the microcontroller 14, the microcontroller 14 may send an RGB trigger signal (RGB trigger in) to the RGB sensor 11 and simultaneously send a synchronization trigger signal (Ext trigger out) to the slave device to implement synchronous operation of the master device and the slave device. In one embodiment, to avoid interference between the master device and the slave device, a rising edge of the synchronization trigger signal sent to the slave device differs from a rising edge of the TOF control signal received by the TOF unit of the master device or a TOF exposure signal corresponding to the first exposure of the TOF sensor by a delay time Δ T, where the configuration of Δ T requires an exposure time greater than a single frame phase of the TOF sensor and less than a two-frame time interval, where the two-frame time interval depends on the number of the master device and the slave device.

Further, because this application needs to gather at least two frames of the rake chase image and carries out TOF depth image synthesis, consequently, based on the clearance of two frames of the rake chase image inserts multichannel measuring device and carries out the common work, can utilize camera self characteristic to realize that the multimachine is synchronous and avoided the multimachine to disturb by the maximize. Specifically, assuming that the maximum exposure time of the TOF sensor is 600us, and the idle time between each frame of IR image is 8ms, theoretically, about 13 measurement devices can be inserted to work together in the idle time, and the idle time may be more or less, and may be designed according to the exposure time of the actual TOF sensor, which is not limited herein.

Fig. 5 is a timing chart of operations of the modules when the slave device is synchronously triggered. Specifically, as shown in fig. 1 and fig. 5, the microcontroller 14 of the slave device receives a synchronous trigger signal (Ext trigger in) sent by the master device to generate a TOF trigger signal (denoted as TOF trigger in, which may be the TOF gating signal or the vertical distribution signal described above) to make the TOF unit 12 perform at least four exposures to obtain at least four frames of swathhase images, and simultaneously generates an RGB trigger signal to the RGB sensor 11 to make the RGB sensor 11 perform one exposure to obtain an RGB image.

On the basis of the timing chart shown in fig. 5, fig. 6 is a complete timing chart for receiving an external trigger signal from a measurement device to control the operation of the measurement device. As shown in fig. 1 and fig. 6, the microcontroller 14 receives an external trigger signal (denoted as Ext trigger in) sent by the synchronization device to control the exposure of the local RGB sensor 11 and the TOF unit 12, and outputs an output signal (Ext trigger out) to trigger the synchronous exposure of the next measurement apparatus, so that other devices do not interfere with the local exposure.

It should be noted that the master-slave device may also generate corresponding trigger signals to the TOF unit 12 and the RGB sensor 11 through the respective microcontroller 14 at regular time to achieve data synchronization of the master-slave device.

In addition, when there are multiple measuring devices, the microcontroller 14 of the slave device can send the TOF trigger signal and the RGB trigger signal to the TOF unit 12 and the RGB sensor 11, respectively, and in contrast to the single-machine working mode, send the TOF trigger signal to the TOF unit 12 first, and then generate the RGB trigger signal according to the TOF trigger signal to achieve synchronous output. It should be understood that in either manner, the exposure center should be corrected to ensure a synchronized output.

In one embodiment, the connection between the multiple measurement devices may include a star mode or a chain mode. Specifically, any one of the plurality of measuring devices is selected as a master, and the rest of the measuring devices are slave measuring devices. As shown in fig. 7, in the chain mode, the signal output end of the master device is connected to the signal input end of one slave device, the signal output end of one slave device is connected to the signal input end of the next slave device, and so on until the signal output end of the last slave device is connected to the signal input end of the master device, thereby forming a chain structure. As shown in fig. 8, in the star mode, the signal input terminal of each slave device is connected to the signal output terminal of the master device.

In some embodiments, the depth measurement system may include a server in addition to the plurality of measurement devices described above, each measurement device being connected to the server. The server is used as a host, and each measuring device is used as a slave, so that the images acquired by the plurality of measuring devices are transmitted to the server for synchronous processing. The connection between the measuring device and the server may be a wired or wireless connection. As a possible implementation, as shown in fig. 9, the connection between the plurality of measuring devices is in a chain mode, and each measuring device is connected to the server. As another possible implementation manner, as shown in fig. 10, the connection between the plurality of measuring devices is in a star mode, and each measuring device is connected to the server.

In one embodiment, each frame of image is time stamped as it is acquired by the measurement device. The depth measurement system performs the following method to synchronize multiple measurement device image timestamps or system times to achieve image synchronized output. When the following method is executed, the server in the depth measurement system is a Master (Master device), and each measurement device is a Slave (Slave device) of the Master.

S1: the server sends its own time and a request signal to the plurality of measuring devices to request recording of the time of each measuring device.

S2: and returning the self time by each measuring device, and setting the self time as the time T sent by the server.

S3: the server records RTT time (namely round trip time) of a plurality of measuring devices; for the measuring device with the abnormal RTT time deviation, repeating the steps S1, S2 and S3; the RRT time is the time used by the server for sending the time to the measuring device, and the measuring device sets the time to be the time sent by the server and sends a setting success signal to the server.

S4: the server sends the RTT to the measuring device, which then sets its own time to T + RTT/2, where only the timestamp of the written image is modified or the system time of the measuring device is directly changed.

In some embodiments, the system time of each measurement device may be time calibrated at regular intervals.

In some embodiments, further, the system time of each measuring device may be modified according to the trigger signal received by each measuring device to achieve synchronization. More specifically, the method comprises the following steps:

s10: a microcontroller in each measuring device receives a rising edge of a synchronous trigger signal (Ext trigger in), normally triggers a corresponding frame synchronization function, and records the system time at the moment as Ts;

s20: polling and reading the level of the current synchronous trigger signal of each measuring device; calculating the duration time t ms (unit: millisecond) of the high level; if T is larger than a preset threshold value, the time synchronization signal is judged, and if | T0+ (n-1) × NT-Ts | > NT, n = floor [ (Ts-T0 + NT/2)/NT ]; then uniformly configuring the system time of each measuring device into T0+ (n-1) multiplied by NT + T; wherein, floor [ ] indicates rounding down, the preset threshold is preferably 10, T0 is the initial starting time of the system (which can be defined as 0 min 0 s 0 ms 0 ps 1 s1 in 1970), and n indicates the number of received time synchronization signals.

It should be noted that, in this embodiment, signals of Ext trigger in and Ext trigger out are multiplexed, and when the system receives a time synchronization signal of Ext trigger in, the device configures the system time to be T0+ NT; n is the cycle length of the time synchronization of multiple measuring devices, the unit can be set to be 1/30 second or frame time, N is not too large or too small, and the suggested interval can be [1000, 10000].

According to the embodiment of the application, the exposure synchronization is achieved through software and hardware synchronization, and the synchronization effect of the images is improved. In addition, in some embodiments, the microcontroller is loaded with a real-time system, so that real-time feedback can be performed on each channel of data and signals in real time, the control precision can reach microsecond level, and the stability and precision of the synchronization effect are ensured.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present application, and they should be construed as being included in the present application.

Claims (13)

1. The depth measuring device is characterized by comprising a color sensor, a TOF unit comprising a projection module and a TOF sensor, a central processing unit and a microcontroller, wherein:

the central processing unit is used for respectively sending TOF control signals and RGB control signals to the TOF unit and the RGB sensor and sending correction signals to the microcontroller;

the TOF unit is used for receiving the TOF control signal and simultaneously generating a gating signal or a vertical synchronization signal to the microcontroller; under the trigger of the TOF control signal, the projection module is used for emitting at least two different modulated light beams to a target scene, and the TOF sensor is used for collecting the different modulated light beams reflected back by the target scene to generate at least two frames of catch images and transmitting the catch images to the central processor according to the TOF control signal;

the microcontroller is used for correcting the gating signal or the vertical synchronous signal according to the correction signal to generate an RGB trigger signal so as to trigger the RGB sensor to start working;

the RGB sensor is used for collecting RGB images according to the RGB trigger signals and sending the RGB images to the central processing unit according to the RGB control signals;

the central processing unit is further configured to perform phase unwrapping and depth fusion calculation on the at least two frames of rapphase images to obtain a TOF depth image, so as to synchronously output the TOF depth image and the RGB image.

2. The depth measurement device of claim 1, wherein the exposure time for the RGB sensor to acquire an RGB image is aligned with the total exposure time for the TOF sensor to acquire the at least two frames of the wideband image.

3. The depth measurement device of claim 1 or 2, wherein the microcontroller carries a real-time system for receiving signals in real-time to generate trigger signals to the TOF sensor or the RGB sensor in real-time.

4. The depth measurement device of claim 1 or 2, wherein the central processor is further configured to align the exposure centers of the at least two swathhase images and the RGB image; the central processing unit is also used for aligning the exposure centers of the depth image and the RGB image and then synchronously outputting the aligned depth image and the RGB image.

5. A depth measurement system, comprising: a plurality of depth measuring devices as claimed in any one of claims 1 to 4.

6. The depth measurement system of claim 5, wherein any of the depth measurement devices is a master device, and the remaining depth measurement devices are slave devices, the master device being configured to generate a synchronization trigger signal to a microcontroller of the slave device to trigger the slave device to operate synchronously with the microcontroller of the slave device; wherein the synchronization trigger signal is generated by any one of the RGB sensor, TOF sensor, or the microcontroller in the master device.

7. The depth measurement system of claim 7, wherein any one of the depth measurement devices is a master device, and the remaining depth measurement devices are slave devices, and the master device and the slave devices are timed by the respective microcontrollers to generate synchronization trigger signals to be sent to the respective RGB sensors and TOF sensors to achieve data synchronization.

8. The depth measurement system of any one of claims 5-7, wherein the plurality of depth measurement devices are coupled in a manner comprising a star mode or a chain mode.

9. The depth measurement system of any of claims 6 to 7, wherein a rising edge of the synchronized trigger signal sent to the slave device differs by a delay time Δ T from a rising edge of the TOF trigger signal received by the TOF unit of the master device or from a TOF exposure signal of a first exposure of the TOF sensor.

10. The depth measurement system of claim 9, further comprising a server, wherein a plurality of the depth measurement devices are each connected to the server, wherein the server is a master, wherein a plurality of the depth measurement devices are slaves to the master, and wherein the master receives the RGB images and the depth images output by each of the depth measurement devices.

11. The depth measurement system of claim 10, wherein the system time or image time stamps of each of the depth measurement devices are synchronized.

12. The depth measurement system of claim 11, wherein calibrating the system time for each of the depth measurement devices comprises:

the server is used for sending self time T and a request signal to the depth measuring devices to request to record the time of each depth measuring device;

each depth measuring device is used for returning self time according to the request signal and setting the self time as self time T issued by the server;

the server is further used for recording round trip times RTT of a plurality of depth measuring devices and sending the round trip times RTT to the depth measuring devices so that the depth measuring devices set the self time to be T + RTT/2, wherein the round trip time is the time when the server sends the self time to the depth measuring devices;

the depth measuring device sets the self time as the self time issued by the server and sends a setting success signal to the time used by the server.

13. The depth measurement system of claim 11, wherein calibrating the system time for each of the depth measurement devices comprises:

recording the system time Ts when the microcontroller in each measuring device normally triggers the corresponding frame synchronization function when receiving the rising edge of the synchronization trigger signal;

polling and reading the current synchronous trigger signal level of each measuring device, and calculating the duration time t of high level; if T is larger than a preset threshold, judging the signal to be a time synchronization signal, and if | T0+ (N-1) × NT-Ts | > NT, setting N = floor [ (Ts-T0 + NT/2)/NT ], wherein T0 is the initial starting time of the system, and N is the period length of time synchronization of each measuring device;

configuring the system time of each measuring device to be T0+ (n-1) multiplied by NT + T; where n represents the number of received time synchronization signals.

Images