CN109963136B - Working method and device of light depth camera with smart phone structure - Google Patents
Working method and device of light depth camera with smart phone structure Download PDFInfo
- Publication number
- CN109963136B CN109963136B CN201711415009.9A CN201711415009A CN109963136B CN 109963136 B CN109963136 B CN 109963136B CN 201711415009 A CN201711415009 A CN 201711415009A CN 109963136 B CN109963136 B CN 109963136B
- Authority
- CN
- China
- Prior art keywords
- infrared
- camera
- image
- trigger signal
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/25—Image signal generators using stereoscopic image cameras using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/254—Image signal generators using stereoscopic image cameras in combination with electromagnetic radiation sources for illuminating objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/296—Synchronisation thereof; Control thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Studio Devices (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a working method and a device of an optical depth camera with a smart phone structure, wherein, an RGB camera is adopted to synchronously trigger or actively trigger, synchronously controlling the exposure time, the on/off sequence and the like of the RGB camera, the infrared laser coding pattern projector, the infrared flood light source and the infrared IR camera, RGB images are acquired by the RGB camera, infrared images or infrared coding images are acquired by the infrared IR camera, wherein the infrared code image can obtain a depth image through depth perception decoding, the infrared image can carry out infrared detail enhancement, the method and the device can synchronously control the sequence and the exposure time of the projector, the pan light source, the RGB and the IR receiving cameras, and are beneficial to greatly reducing the working power consumption of the structured light depth camera in the processes of 3D self-shooting, face unlocking and 3D scanning reconstruction application of the smart phone.
Description
Technical Field
The invention belongs to the technical field of mobile communication, image processing, computer vision and integrated circuits, and particularly relates to a working method and a working device of an optical depth camera with a smart phone structure.
Background
Three-dimensional reconstruction, which is a major goal of human vision research, is an important direction in the field of computer vision. The method mainly adopts a certain technical means to realize the shape modeling of the three-dimensional object. With the push of vision calculation theory, the three-dimensional reconstruction has achieved breakthrough progress. The depth information is acquired by utilizing an image processing technology to perform real-time identification and motion capture of the three-dimensional image, so that the interaction between a person and a terminal in natural modes such as expressions, gestures, somatosensory motions and the like becomes possible.
The rapid development of mobile equipment of mobile phones has a qualitative leap in both hardware and software, which makes the application of three-dimensional reconstruction to mobile phones possible. In the prior art, a mobile phone generally has one camera or a front camera and a rear camera, but when taking a picture, only one camera can be started to take a picture, under the condition, the taken picture can only be planar, an output image is also planar, a stereo picture cannot be well and directly taken, the method for generating the stereo picture is to take pictures at different visual angles of an object in multiple angles, and then multiple groups of pictures are integrated and processed manually through imaging software, so that not only is time wasted, but also the imaging cost is increased. An optical depth camera with an integrated structure of a smart phone is a trend, and can be used as a front depth camera or a rear depth camera to perform face unlocking, 3D self-shooting, small object 3D scanning and the like.
Disclosure of Invention
In view of the above, the present invention provides a working method and device for an optical depth camera with a smart phone structure, which support two working modes, namely RGB camera synchronous triggering and active triggering, to realize front-back synchronous control of the RGB camera, an infrared laser coding pattern projector, an infrared flood light source, and an infrared IR camera, wherein an RGB image is acquired by the RGB camera, an infrared image or an infrared coding image is acquired by the infrared IR camera, and the depth image can be acquired by the infrared coding image through depth decoding. The method and the device can realize the synchronous control and the exposure time of the infrared laser coding pattern projector, the infrared flood light source, the RGB camera and the infrared IR camera, and are beneficial to reducing the working power consumption of the depth camera in the processes of 3D self-shooting, face unlocking and 3D scanning reconstruction.
Specifically, the invention provides a working method of an optical depth camera with a smart phone structure, which comprises an RGB camera synchronous triggering mode and an active triggering mode. The synchronous trigger mode of the RGB camera comprises the following steps:
step 1, a processor starts an RGB camera and initializes an infrared IR camera;
step 2, the RGB camera generates a field synchronization trigger signal vsync _ RGB to a synchronization control module;
step 3, the synchronous control module generates a new field synchronous trigger signal vsync _ con according to the vsync _ rgb signal and sends the new field synchronous trigger signal vsync _ con to the infrared IR camera, and the signal contains 1 or 2 or more field synchronous signals;
step 4, the infrared IR camera feeds back a strobe _ IR trigger signal to the synchronous control module according to the vsync _ con signal;
step 6, an infrared floodlight source or an infrared laser coding pattern projector emits light to irradiate a target object or a projection space according to an effective trigger signal, wherein the infrared floodlight source emits uniform infrared light, and the infrared laser coding pattern projector projects a laser random speckle coding pattern or a regular symbol M array coding pattern or a mixed coding pattern of the two patterns;
step 7, the IR camera collects an infrared image or an infrared coding image, if the trigger object is an infrared pan light source, the infrared image is collected, and if the trigger object is an infrared laser coding pattern, the infrared coding image is collected;
and 8, performing infrared enhancement processing on the infrared image by the depth perception module, and performing depth decoding on the infrared coding image in combination with the reference coding image.
The active trigger mode is substantially the same as the synchronous trigger mode except that step 1 and step 2 in the synchronous trigger mode are not needed, and step 3 is modified to be that the synchronous control module actively generates a field synchronous trigger signal vsync _ con to be sent to the IR camera, the signal may contain 1 or 2 or more continuous field synchronous signals, and the remaining steps 4, 5, 6, 7 and 8 are the same.
The active trigger mode can also control the synchronous control module to generate a strobe _ led trigger signal to control the exposure time of the infrared floodlight source according to an instruction of the smart phone AP, and an infrared receiving camera is adopted to collect an infrared image according to a fixed frequency of 15Hz/30Hz/45Hz/60Hz/120 Hz; once the face or the face change or the target object movement is detected, the mobile phone AP sends an instruction to the synchronous control module, a strobe _ pro trigger signal is generated to control the exposure time of the infrared laser coding pattern projector, and the strobe _ led trigger signal is closed at the same time, so that the infrared receiving camera can collect one frame or multiple frames of infrared coding patterns according to the original frequency, and the collected infrared coding patterns are sent to the depth perception module to generate the depth map.
In addition, the invention also provides a working device of the light depth camera with the intelligent mobile phone structure, which specifically comprises the following modules:
and the synchronous control module can work in an RGB camera synchronous triggering mode or an active triggering mode. In the RGB camera synchronous triggering mode, a new field synchronous triggering signal is generated according to a field synchronous triggering signal vsync _ RGB input by the RGB camera and is sent to the IR camera; the module receives a feedback signal of the IR camera and then generates a trigger signal to synchronously control the infrared floodlight source or the infrared laser coding pattern projector, wherein the active trigger mode does not need to actively generate a field synchronization trigger signal to the IR camera by external triggering, and the module receives the feedback signal of the IR camera and then generates the trigger signal to synchronously control the infrared floodlight source or the infrared laser coding pattern projector, including controlling the exposure time, the on-off state or the off state of the infrared floodlight source or the infrared laser coding pattern projector;
an infrared floodlight source for generating uniform infrared light to irradiate a target object or a projection space, and a VCSEL or an LED is generally used as a light source;
the infrared laser coding pattern projector is used for projecting a structured light coding pattern to a target object or a projection space surface, wherein the structured light coding pattern comprises a laser random speckle coding pattern or a regular symbol M array coding pattern or a fusion coding pattern of the laser random speckle coding pattern and the regular symbol M array coding pattern;
the RGB camera can be used as a front camera or a rear camera of a mobile phone, a high-pixel high-image-quality camera is generally adopted, the camera is influenced by a shooting environment, automatic exposure and the like, the frame frequency of shooting is variable, the RGB camera outputs a field synchronization signal, and then the infrared IR camera synchronously shoots one frame, two frames or multiple frames with the infrared IR camera;
the infrared IR camera, together with the infrared pan light source and the infrared laser coding pattern projector, can be used as a front infrared depth camera device or a rear infrared depth camera device of a mobile phone and is used for shooting an infrared image or an infrared coding pattern;
and the depth perception module is used for enhancing the details of the infrared image, performing depth decoding on the infrared coding image in combination with the reference coding image to generate a depth image, and selectively outputting the depth image and the reference coding image.
Under the synchronous trigger mode of the RGB camera, 1 frame of RGB image can be output, and 1 frame of infrared image, 1 frame of depth image, 2 frames of depth image and 2 frames of infrared image can be output. Under the active trigger mode, 1 frame of RGB image can be output to follow a multi-frame depth image, a multi-frame infrared image and 1 frame of infrared image 1 frame depth image.
The method and the device can realize the synchronous control and the exposure time of the infrared laser coding pattern projector, the pan light source, the RGB and IR receiving cameras, and are beneficial to reducing the working power consumption of the depth camera in the processes of 3D self-shooting, face unlocking and 3D scanning reconstruction.
Drawings
FIG. 1 is a flow chart of RGB camera synchronous trigger mode/active trigger mode;
FIG. 2 is a schematic diagram of a smartphone structured light depth camera apparatus;
FIG. 3 is a flowchart of depth decoding of an IR encoded map;
FIG. 4 is a diagram of RGB burst mode full depth mode;
FIG. 5 is a diagram of RGB triggered mode full infrared mode;
FIG. 6 is an RGB triggered mode 1 frame speckle 1 frame infrared image;
FIG. 7 is a diagram of an active triggered mode full depth mode;
FIG. 8 is a diagram of an active trigger mode full IR mode;
fig. 9 is an infrared alternating graph of 1 frame speckle 1 frame of active trigger mode.
Detailed Description
The method and apparatus provided by the present invention will be described in further detail below with reference to fig. 1-9.
The invention provides a working method of a light depth camera with a smart phone structure, which comprises an RGB camera synchronous triggering mode and an active triggering mode as shown in figure 1.
In a first embodiment, the working method in the RGB camera synchronous trigger mode includes the following steps:
step 1, the processor starts the RGB camera and initializes the infrared IR camera. Specifically, the RGB camera is started up by the mobile phone processor AP, and the infrared IR camera is initialized.
And step 2, the RGB camera generates a field synchronization trigger signal vsync RGB to the synchronization control module.
And 3, generating a new field synchronization trigger signal vsync _ con by the synchronization control module according to the vsync _ rgb signal and sending the new field synchronization trigger signal vsync _ con to the infrared IR camera. The synchronous control module generates a field synchronous signal, the field frequency is generally 30Hz or 60Hz, and the field synchronous signal can also be set by self. Specifically, the vsync _ con signal may contain 1 or 2 or more field sync signals.
And step 4, feeding back a strobe _ IR trigger signal to the synchronous control module by the infrared IR camera according to the synchronous trigger signal vsync _ con signal.
And 5, generating a strobe _ led trigger signal by the synchronous control module to the infrared floodlight source or a strobe _ pro trigger signal to the infrared laser coding pattern projector, wherein the two trigger signals have controllable polarity and variable pulse width, cannot be triggered simultaneously and are respectively used for controlling the exposure time of the infrared floodlight source or the infrared laser coding pattern projector.
The strobe _ led trigger signal triggers the infrared floodlight source and controls the exposure time, on or off of the infrared floodlight source.
Wherein the strobe _ pro trigger signal triggers the infrared laser code pattern projector for controlling the exposure time, on or off of the code pattern projector.
And 6, an infrared floodlight source or an infrared laser coding pattern projector emits light to irradiate the target object or the projection space according to the effective trigger signal, wherein the infrared floodlight source emits uniform infrared light, and the infrared laser coding pattern projector projects laser random speckle coding patterns or regular symbol M array coding patterns or the combination of the laser random speckle coding patterns and the regular symbol M array coding patterns.
And 7, acquiring an infrared image or an infrared coding image by the IR camera, acquiring the infrared image if the trigger object is an infrared pan-light source, and acquiring the infrared coding image if the trigger object is an infrared laser coding pattern projector.
And 8, performing infrared enhancement processing on the infrared image by the depth perception module, and performing depth decoding on the infrared coding image in combination with the reference coding image to obtain a depth image.
The depth perception module performs depth decoding on the ir coded picture in combination with the reference coded picture, which is specifically shown in fig. 3.
Specifically, as shown in fig. 3, the operation of the depth decoding includes the following steps:
preprocessing, including correcting the distorted infrared image or infrared coding image by using a fish eye correction method, performing detail enhancement on the infrared image, and performing consistency enhancement and binarization processing on the infrared coding image;
image rotation, which can be selected to rotate 90 degrees or not to rotate the image, so as to select block matching and depth calculation in the X-axis direction or the Y-axis direction;
block matching, namely performing block matching motion estimation on the input infrared coding image by combining a solidified reference coding image, and obtaining an offset between each pixel point in the input infrared coding image and the central point of the optimal matching block in the reference coding image, namely a motion estimation vector, according to an image block mode through comparison and calculation of block matching acquaintance;
and (3) depth calculation, namely obtaining depth value information corresponding to each pixel point of the infrared coding image according to a structured light monocular depth calculation formula, wherein the calculation formula is as follows:
where f is the focal length of the camera, s is the baseline, d is the known reference distance, Δ x is the offset, μ is the pixel size, dis is the depth value, and finally the infrared code is encoded by the above formulaThe map is converted into a depth map.
The reference code pattern is used as a reference comparison standard, is projected in advance to a plane vertical to the optical axis of the infrared speckle code pattern projector, is collected and is obtained through the same preprocessing, and the vertical reference distance d between the reference code pattern and the projector is known. The RGBD data acquisition and processing format under the RGB camera synchronous triggering mode comprises the following steps: 1) synchronously acquiring 2 frames of depth maps by 1 frame of RGB map; 2)1 frame of RGB image synchronously collects 2 frames of infrared images; 3)1 frame RGB image synchronously collects 1 frame infrared image 1 frame depth image.
In another embodiment, the only difference between the active triggering mode of the RGB camera and the synchronous triggering mode of the RGB camera is that step 3 is directly started without steps 1 and 2, and step 3 is changed to that the synchronous control module actively generates a field synchronous triggering signal vsync _ con to be sent to the IR camera, where the field synchronous triggering signal vsync _ con may include 1 or 2 or more continuous field synchronous signals. The remaining steps 4, 5, 6, 7, 8 are the same.
The active triggering mode of the RGB camera can also control the synchronous control module to generate a strobe _ led triggering signal to control the exposure time of the infrared floodlight source according to an instruction of the smart phone AP, and an infrared receiving camera is adopted to acquire an infrared image according to a fixed frequency of 15Hz/30Hz/45Hz/60Hz/120 Hz; once the face or the face change or the target object movement is detected, the AP sends an instruction to the synchronous control module, a strobe _ pro trigger signal is generated to control the exposure time of the infrared laser coding pattern projector, and the strobe _ led trigger signal is closed at the same time, so that the infrared receiving camera can collect one frame or multiple frames of the infrared coding pattern according to the original frequency. And the collected infrared coding image is sent to a depth perception module to generate a depth image.
The RGBD data acquisition and processing format in the active triggering mode comprises the following steps: 1)1, synchronously acquiring a multi-frame depth map by using the RGB map; 2)1, synchronously collecting a plurality of frames of infrared images by using RGB images; 3)1 frame of RGB image synchronously collects 1 frame of infrared image 1 frame depth image, and the images are alternated; 4) and (3) processing AP control by combining the mobile phone, collecting 1 or more frames of continuous infrared images, and collecting 1 or more frames of depth images at intervals of AP instructions.
In addition, the present invention further provides a working device of an optical depth camera with a smart phone structure, as shown in fig. 2, including an IR camera 300, an RGB camera 100, an infrared laser coding pattern projector 400, an infrared pan-light source 500, a synchronization control module 200, and a depth perception module 800. The device specifically comprises the following modules:
the synchronous control module 200 may work in two modes, namely, a synchronous trigger mode and an active trigger mode of the RGB camera 100, wherein the RGB camera 100 generates a new field synchronous trigger signal to the IR camera 300 according to the field synchronous trigger signal vsync _ RGB input by the RGB camera 100 in the synchronous trigger mode; the active trigger mode does not need external trigger to actively generate a field synchronization trigger signal to the IR camera 300, and the synchronization control module 200 receives a feedback signal from the IR camera 300 and then generates a trigger signal to synchronously control the infrared floodlight source 500 or the infrared laser coding pattern projector 400, including controlling the exposure time, on or off of the floodlight source or projector.
The infrared flood light source 500 is used to generate uniform infrared light to illuminate a target object or a projection space, and generally employs a VCSEL or an LED as a light source.
An infrared laser coded pattern projector 400 for projecting a structured light coded pattern onto a target object or projection space surface, the structured light coded pattern comprising a laser random speckle coded pattern or a regular symbol M-array coded pattern, or a fused coded pattern of both. The projector generally adopts a vertical cavity surface emitting laser VCSEL or a laser diode LD as a light source, and structurally adopts a VCSEL + collimating mirror + diffraction optical device DOE mode or an LD + collimating mirror + diffraction optical device DOE mode.
The RGB camera 100, which may be a front camera or a rear camera of a mobile phone, generally employs a high-pixel high-quality camera, which is affected by a shooting environment and automatic exposure, and the frame frequency of the shot is variable, and the RGB camera 100 outputs a field synchronization signal, and then the infrared IR camera 300 is synchronized with the field synchronization signal to shoot one, two or more frames.
The IR camera 300, together with the infrared pan light source 500 and the infrared laser code pattern projector 400, can be used as a front infrared depth camera device or a rear infrared depth camera device of a mobile phone for photographing an infrared image or an infrared code pattern.
The depth perception module 800 performs detail enhancement processing on the ir map, performs depth decoding on the ir code map in combination with the reference code map to generate a depth map, and both of them can be selectively output.
Synchronous trigger mode of RGB camera, in panorama mode: the non-rotation mode can output 1 frame of RGB image and follow 2 frames of depth image, and the rotation mode can output 1 frame of RGB image and follow 2 frames of depth image after 1 frame of time delay, as shown in FIG. 4. In the full infrared mode: after 1 frame delay, the 1 frame RGB graph is output to follow the 2 frame non-rotated infrared graph, as shown in fig. 5. Speckle 1 frame at 1 frame infrared: the non-rotation mode can output 1 frame of RGB image following 1 frame of depth image 1 frame of infrared image, the rotation mode can output 1 frame of RGB image following 1 frame of infrared image after 1 frame of time delay, and then output 1 frame of depth image after 1 frame of time delay, as shown in FIG. 6.
Under the initiative trigger mode of RGB camera, in panorama mode: the non-rotation mode can output a depth map with 1 frame of RGB map following multiple frames continuously, and the rotation mode can output a depth map with 1 frame of RGB map following multiple frames continuously and rotatably after 1 frame of time delay, as shown in FIG. 7. In the full infrared mode: after 1 frame delay, outputting a 1 frame RGB map following the multi-frame non-rotated infrared map, as shown in FIG. 8. Alternating in 1 frame speckle and 1 frame infrared: the non-rotation mode can output 1 frame of RGB image following 1 frame of depth image 1 frame of infrared image, and the rotation mode can output 1 frame of RGB image following 1 frame of infrared image and then output 1 frame of depth image after 1 frame of time delay, which can be alternated, as shown in FIG. 9.
The working process method and the device of the structured light depth camera of the smart phone are not only suitable for controlling the structured light depth camera module of the smart phone, but also suitable for controlling structured light depth cameras in other intelligent terminal fields, including monocular and binocular structured light depth camera modules; meanwhile, the content and the device of the invention can be realized by adopting an ASIC chip, an IP core embedded SoC application or various SoC processors (CPU, GPU, DSP, AP) and software, so that the modification and perfection without departing from the spirit and the scope of the invention are all included in the scope of the above claims.
Claims (10)
1. A synchronous trigger working method of an optical depth camera with a smart phone structure is characterized by comprising the following steps:
the method comprises the following steps:
step 1: the processor starts the RGB camera and initializes the infrared IR camera;
step 2, the RGB camera generates a field synchronization trigger signal vsync _ RGB to a synchronization control module;
step 3, the synchronous control module generates a new field synchronous trigger signal vsync _ con according to the vsync _ rgb signal and sends the new field synchronous trigger signal vsync _ con to the IR camera;
step 4, the IR camera feeds back a strobe _ IR trigger signal to the synchronous control module according to the synchronous trigger signal vsync _ con;
step 5, the synchronous control module generates a strobe _ led trigger signal to the infrared floodlight source or a strobe _ pro trigger signal to the infrared laser coding pattern projector;
step 6, the infrared floodlight source or the infrared laser coding pattern projector emits light to irradiate a target object or a projection space according to the effective trigger signal;
step 7, the IR camera collects an infrared image or an infrared coding image, if the trigger object is an infrared pan light source, the infrared image is collected, and if the trigger object is an infrared laser coding pattern, the infrared coding image is collected;
and 8, performing infrared enhancement processing on the infrared image by the depth perception module, and performing depth decoding on the infrared code image by combining the reference code image to obtain a depth image.
2. An active trigger working method of an optical depth camera with a smart phone structure is characterized in that:
the method comprises the following steps:
step 1, a synchronous control module actively generates a field synchronous trigger signal vsync _ con to an infrared IR camera;
step 2, the IR camera feeds back a strobe _ IR trigger signal to the synchronous control module according to the synchronous trigger signal vsync _ con;
step 3, the synchronous control module generates a strobe _ led trigger signal to the infrared floodlight source or a strobe _ pro trigger signal to the infrared laser coding pattern projector;
step 4, the infrared floodlight source or the infrared laser coding pattern projector emits light to irradiate a target object or a projection space according to the effective trigger signal;
step 5, the IR camera collects an infrared image or an infrared coding image, if the trigger object is an infrared pan light source, the infrared image is collected, and if the trigger object is an infrared laser coding pattern, the infrared coding image is collected;
and 6, carrying out infrared enhancement processing on the infrared image by the depth perception module, and carrying out depth decoding on the infrared code image by combining the reference code image to obtain a depth image.
3. The operating method according to claim 1 or 2, characterized in that:
and the synchronous control module can be controlled to generate a strobe _ led trigger signal to control the exposure time of the infrared floodlight source according to an instruction of the smart phone AP, and an infrared receiving camera is adopted to collect an infrared image according to a fixed frequency.
4. The method of operation of claim 3, wherein:
if the face or the face change or the target object movement is detected, the smart phone AP sends an instruction to the synchronous control module again, generates a strobe _ pro trigger signal to control the exposure time of the infrared laser coding pattern projector, and closes the strobe _ led trigger signal at the same time, so that the infrared receiving camera collects the infrared coding pattern according to the original frequency.
5. A working device for a smartphone structured light depth camera in a synchronous trigger mode is characterized in that:
the working device comprises an IR camera, an RGB camera, an infrared laser coding pattern projector, an infrared flood light source, a synchronous control module and a depth perception module;
the synchronous control module is used for generating a new field synchronous trigger signal vsync _ con to the IR camera according to a field synchronous trigger signal vsync _ RGB input by the RGB camera, receiving a feedback signal strobe _ IR of the IR camera and then generating a trigger signal strobe _ led to synchronously control the infrared pan light source or the trigger signal strobe _ pro to the infrared laser coding pattern projector;
the infrared floodlight source is used for generating uniform infrared light to irradiate a target object or a projection space;
an infrared laser coded pattern projector for projecting a structured light coded pattern onto a target object or projection space surface;
the RGB camera is used for generating a field synchronization trigger signal vsync _ RGB;
the IR camera is used for shooting an infrared image or an infrared coding image;
and the depth perception module is used for performing detail enhancement processing on the infrared image and performing depth decoding on the infrared coding image and the reference coding image to generate a depth image.
6. The work apparatus of claim 5, wherein:
the infrared coding pattern comprises a laser random speckle coding pattern or a regular symbol M array coding pattern or a fusion coding pattern of the laser random speckle coding pattern and the regular symbol M array coding pattern.
7. The work apparatus of claim 5, wherein:
the infrared laser coding pattern projector adopts a vertical cavity surface emitting laser VCSEL or a laser diode LD as a light source, and structurally adopts a mode that the VCSEL is combined with a collimating mirror and then combined with a diffraction optical device DOE, or the LD is combined with the collimating mirror and then combined with the diffraction optical device DOE.
8. A working device of a smart phone structured light depth camera for an active trigger mode is characterized in that:
the working device comprises an IR camera, an RGB camera, an infrared laser coding pattern projector, an infrared flood light source, a synchronous control module and a depth perception module;
the synchronous control module actively generates a field synchronous trigger signal vsync _ con to the infrared IR camera, receives a feedback signal strobe _ IR of the IR camera and then generates a trigger signal strobe _ led to synchronously control the infrared floodlight source or the infrared laser coding pattern projector through the trigger signal strobe _ pro;
the infrared floodlight source is used for generating uniform infrared light to irradiate a target object or a projection space;
an infrared laser coded pattern projector for projecting a structured light coded pattern onto a target object or projection space surface;
the IR camera is used for shooting an infrared image or an infrared coding image;
and the depth perception module is used for performing detail enhancement processing on the infrared image and performing depth decoding on the infrared coding image and the reference coding image to generate a depth image.
9. The work apparatus of claim 8, wherein:
the infrared coding pattern comprises a laser random speckle coding pattern or a regular symbol M array coding pattern or a fusion coding pattern of the laser random speckle coding pattern and the regular symbol M array coding pattern.
10. The work apparatus of claim 8, wherein:
the infrared laser coding pattern projector adopts a vertical cavity surface emitting laser VCSEL or a laser diode LD as a light source, and structurally adopts a mode that the VCSEL is combined with a collimating mirror and then combined with a diffraction optical device DOE, or the LD is combined with the collimating mirror and then combined with the diffraction optical device DOE.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711415009.9A CN109963136B (en) | 2017-12-22 | 2017-12-22 | Working method and device of light depth camera with smart phone structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711415009.9A CN109963136B (en) | 2017-12-22 | 2017-12-22 | Working method and device of light depth camera with smart phone structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109963136A CN109963136A (en) | 2019-07-02 |
| CN109963136B true CN109963136B (en) | 2021-06-18 |
Family
ID=67020405
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711415009.9A Active CN109963136B (en) | 2017-12-22 | 2017-12-22 | Working method and device of light depth camera with smart phone structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109963136B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110798675A (en) * | 2019-12-16 | 2020-02-14 | 宁波为森智能传感技术有限公司 | Camera module |
| CN111123277A (en) * | 2019-12-31 | 2020-05-08 | 深圳奥比中光科技有限公司 | Synchronizer of depth measuring system |
| CN113327348A (en) * | 2021-05-08 | 2021-08-31 | 宁波盈芯信息科技有限公司 | Networking type 3D people face intelligence lock |
| CN115190288B (en) * | 2022-06-23 | 2023-04-25 | 海信电子科技（深圳）有限公司 | Method and equipment for synchronously acquiring images by multiple cameras |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661555A (en) * | 2008-08-25 | 2010-03-03 | 索尼株式会社 | Vein imaging apparatus, vein imaging method and vein authentication apparatus |
| CN102589476A (en) * | 2012-02-13 | 2012-07-18 | 天津大学 | High-speed scanning and overall imaging three-dimensional (3D) measurement method |
| CN102706880A (en) * | 2012-06-26 | 2012-10-03 | 哈尔滨工业大学 | Road information extraction device based on two-dimensional image and depth information and road crack information detection method based on same |
| CN102706290A (en) * | 2012-06-20 | 2012-10-03 | 北京航空航天大学 | Structured light projector meeting variation ranges of high speed and high brightness |
| CN105306924A (en) * | 2015-10-22 | 2016-02-03 | 凌云光技术集团有限责任公司 | Active texture method for linear array binocular 3D imaging |
| CN105931240A (en) * | 2016-04-21 | 2016-09-07 | 西安交通大学 | Three-dimensional depth sensing device and method |
| CN205657802U (en) * | 2016-04-21 | 2016-10-19 | 宁波盈芯信息科技有限公司 | Three -dimensional degree of depth perception equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8723923B2 (en) * | 2010-01-14 | 2014-05-13 | Alces Technology | Structured light system |
| KR101697200B1 (en) * | 2015-06-12 | 2017-01-17 | 성균관대학교산학협력단 | Embedded system, fast structured light based 3d camera system and method for obtaining 3d images using the same |
-
2017
- 2017-12-22 CN CN201711415009.9A patent/CN109963136B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661555A (en) * | 2008-08-25 | 2010-03-03 | 索尼株式会社 | Vein imaging apparatus, vein imaging method and vein authentication apparatus |
| CN102589476A (en) * | 2012-02-13 | 2012-07-18 | 天津大学 | High-speed scanning and overall imaging three-dimensional (3D) measurement method |
| CN102706290A (en) * | 2012-06-20 | 2012-10-03 | 北京航空航天大学 | Structured light projector meeting variation ranges of high speed and high brightness |
| CN102706880A (en) * | 2012-06-26 | 2012-10-03 | 哈尔滨工业大学 | Road information extraction device based on two-dimensional image and depth information and road crack information detection method based on same |
| CN105306924A (en) * | 2015-10-22 | 2016-02-03 | 凌云光技术集团有限责任公司 | Active texture method for linear array binocular 3D imaging |
| CN105931240A (en) * | 2016-04-21 | 2016-09-07 | 西安交通大学 | Three-dimensional depth sensing device and method |
| CN205657802U (en) * | 2016-04-21 | 2016-10-19 | 宁波盈芯信息科技有限公司 | Three -dimensional degree of depth perception equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109963136A (en) | 2019-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11199706B2 (en) | Head-mounted display for virtual and mixed reality with inside-out positional, user body and environment tracking | |
| CN109963136B (en) | Working method and device of light depth camera with smart phone structure | |
| EP3107065A1 (en) | Methods and systems for providing virtual lighting | |
| Fuchs et al. | Immersive 3D telepresence | |
| JP6821646B2 (en) | Virtual / augmented reality system with dynamic domain resolution | |
| TW201227159A (en) | Method of taking pictures for generating three-dimensional image data | |
| TW201921028A (en) | Video display system, video display method, and video display program | |
| CN106840034A (en) | 3 D scanning system and its application with the speckle projector | |
| JP2002354505A (en) | Stereoscopic system | |
| KR20190090980A (en) | Apparatus for generating 3d model using filter-equipped lighting and drone | |
| CN109961395B (en) | Method, device and system for generating and displaying depth image and readable medium | |
| JP2024013947A (en) | Imaging device, imaging method of imaging device and program | |
| WO2020044809A1 (en) | Information processing device, information processing method and program | |
| WO2011010893A2 (en) | Method for photographing stereoscopic images, and horizontally movable stereoscopic image photographing apparatus and stereoscopic image photographing camera using the method | |
| JP2016059004A (en) | Camera and illumination system | |
| US20240169475A1 (en) | Generating a real-time video stream of a user face based on oblique real-time 3d sensing | |
| JP2019219929A (en) | Constant calibration system and method thereof | |
| JP5979110B2 (en) | Head mounted display and control program | |
| KR102460762B1 (en) | Camera apparatus with integrated heterogeneous video | |
| CN102480632A (en) | Three-dimensional image processing system and photographing device as well as three-dimensional image generating device of three-dimensional image processing system | |
| JP2013015679A (en) | Stereoscopic photographing apparatus and stereoscopic photographing method | |
| JP2012248946A (en) | Video display device and viewing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |