CN109889690A - A kind of method and depth camera group improving depth image frame per second - Google Patents
A kind of method and depth camera group improving depth image frame per second Download PDFInfo
- Publication number
- CN109889690A CN109889690A CN201910160619.1A CN201910160619A CN109889690A CN 109889690 A CN109889690 A CN 109889690A CN 201910160619 A CN201910160619 A CN 201910160619A CN 109889690 A CN109889690 A CN 109889690A
- Authority
- CN
- China
- Prior art keywords
- depth
- synchronization signal
- depth camera
- image
- depth image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a kind of methods and depth camera group for improving depth image frame per second, this method includes controlling the fluorescent lifetime section of light source and generating synchronization signal according to the fluorescent lifetime section, the synchronization signal is sent to each depth camera in depth camera group, the depth camera group includes at least two depth cameras；The depth camera is exposed according to the synchronization signal, the sampling depth image in exposure period；It is merged according to the depth image of the synchronization signal to transmission, the depth image after being merged.The embodiment of the present invention improves depth image frame per second, reduces picture delay and meets practical application request, optimizes user experience.
Description
Technical field
The present invention relates to technical field of image processing, and in particular to a kind of method and depth phase for improving depth image frame per second
Unit.
Background technique
Currently, depth camera is all played an increasingly important role in all trades and professions, scape is got by depth camera
Deep image, may be implemented body-sensing interaction, gesture interaction, and object scanning etc. improves the production efficiency and user's body of traditional industries
It tests.Depth camera can be divided into three kinds: TOF (Time Of Flight) depth camera, RGB-D binocular phase according to its working principle
Machine, structure light depth camera, no matter which kind of depth camera, frame per second is all an important finger for directly affecting depth image delay
Mark, the frame per second of different classes of depth camera is different, but the frame per second of existing depth camera is lower, is not able to satisfy certain answer
The application demand of the occasion of quick response is needed with scene such as body-sensing interaction etc., it would be highly desirable to improve.
Summary of the invention
The present invention provides a kind of methods and depth camera group for improving depth image frame per second, are replaced by more depth cameras
The latter incorporated mode of sampling depth image is exposed to improve the depth image frame per second of output, meets application demand, optimizes user's body
It tests.
According to the one aspect of the application, a kind of raising depth image frame per second method is provided, comprising:
It controls the fluorescent lifetime section of light source and synchronization signal is generated according to the fluorescent lifetime section, the synchronization signal is sent out
Each depth camera in depth camera group is given, the depth camera group includes at least two depth cameras；
The depth camera is exposed according to the synchronization signal, the sampling depth image in exposure period；
The depth image is merged according to the synchronization signal, the depth image after being merged.According to this Shen
Other side please provides a kind of depth camera group, comprising: light source, image processor, the first depth camera and second is deeply
Camera is spent,
The light source controls signal transmitting according to the fluorescent lifetime section and swashs for receiving fluorescent lifetime section control signal
Light；
First depth camera, for receiving the first synchronization signal and being exposed according to first synchronization signal,
The sampling depth image in exposure period；
Second depth camera, for receiving the second synchronization signal and being exposed according to second synchronization signal,
The sampling depth image in exposure period, wherein first synchronization signal and second synchronization signal are respectively according to
The fluorescent lifetime section of light source generates；
Described image processor, for receiving first synchronization signal, second synchronization signal, and according to described
One synchronization signal, second synchronization signal merge the depth image, the depth image after being merged.
Using the raising depth image frame per second method and depth camera group of the embodiment of the present invention, pass through shining according to light source
Period generates synchronization signal, alternately synchronization signal is sent to each depth camera, so that depth camera is according to synchronous signal acquisition
Depth image is obtained, is then merged according to depth image of the synchronization signal to acquisition, the depth image after being merged.By
This, controls each depth camera by synchronization signal and works alternatively sampling depth image in the different fluorescent lifetime sections of light source, will
Depth image merges the frame per second for improving depth image, reduces picture delay and meets practical application request, optimizes use
Family experience.
Detailed description of the invention
Fig. 1 is the flow chart of the raising depth image frame per second method of one embodiment of the invention；
Fig. 2 is the block diagram of the depth camera group of the raising depth image frame per second of one embodiment of the invention；
Fig. 3 is a time-consuming schematic diagram of frame depth image of one embodiment of the invention；
Fig. 4 is the working stage schematic diagram of the depth camera of one embodiment of the invention；
Fig. 5 is the schematic diagram for replacing the work of controlling depth camera according to synchronization signal of one embodiment of the invention；
Fig. 6 is the schematic diagram that image merging is carried out according to synchronization signal of one embodiment of the invention；
Fig. 7 is the block diagram of the depth camera group of one embodiment of the invention.
Specific embodiment
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real
Applying mode, the present invention is described in further detail.Obviously, described embodiments are some of the embodiments of the present invention, without
It is whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative work
Every other embodiment obtained is put, shall fall within the protection scope of the present invention.
Design concept of the invention is: it is lower for the frame per second of depth camera in the prior art, it is not able to satisfy certain answer
The technical issues of with applicable requirements under scene, proposes that a kind of technical solution using the work of polyphaser time-interleaved is defeated to improve
The frame per second of depth image out reduces picture delay.
Fig. 1 is the flow chart of the depth image frame per second improvement method of one embodiment of the invention, referring to Fig. 1, the present embodiment
Depth image frame per second improvement method, including the following steps:
Step S101 controls the fluorescent lifetime section of light source and generates synchronization signal according to the fluorescent lifetime section, will be described
Synchronization signal is sent to each depth camera in depth camera group, and the depth camera group includes at least two depth cameras；
Step S102, the depth camera are exposed according to the synchronization signal, the sampling depth in exposure period
Image；
Step S103 merges the depth image according to the synchronization signal, the depth image after being merged.
As shown in Figure 1 it is found that the present embodiment raising depth image frame per second method, by control light source it is luminous when
Between, synchronization signal is generated in different fluorescent lifetime sections, each depth camera at least two depth cameras sends same
Signal is walked, so that depth camera collects depth image according to synchronization signal concurrent working, then according to synchronization signal to adopting
The depth image of collection merges the depth image after being merged, and exports with each depth camera work in series in the prior art
The mode of depth image is compared, and the output frame rate of the depth image of the present embodiment can be improved twice or more, and meeting to apply needs
It asks, reduces picture delay so as to improve user experience.
Below with the quantity of depth camera for 2, for the classification of depth camera is TOF camera, to method shown in Fig. 1
Realize that step is illustrated.
Specifically, including the first TOF depth camera and the 2nd TOF depth camera in depth camera group, controlled in abovementioned steps
The fluorescent lifetime section of light source processed simultaneously generates synchronization signal according to the fluorescent lifetime section, and the synchronization signal is sent to depth phase
Each depth camera in unit includes: to generate the first synchronization signal and the second synchronization signal according to the fluorescent lifetime section, by institute
It states the first synchronization signal and is sent to the first TOF depth camera, second synchronization signal is sent to the 2nd TOF depth camera,
The depth camera is exposed according to the synchronization signal, and sampling depth image includes: described first in exposure period
TOF depth camera is exposed according to first synchronization signal, the first depth map is collected in the first exposure period
Picture, the 2nd TOF depth camera are exposed according to second synchronization signal, acquire in the second exposure period
To the second depth image.
It should be noted that the quantity of the depth camera of the present embodiment is not limited to 2, it is also possible to 3,4 or more
More, the classification of depth camera is not limited to TOF camera, is also possible to structure light depth camera etc..The frame per second of two depth cameras can
It can also be different with identical, it is without limitation.When depth camera TOF depth camera identical for two frame per second, due to TOF
Camera itself has the function of output depth image, and according to the solutions of the embodiments of the present invention, each depth camera does not export directly
Depth image but the depth image of acquisition is exported to processor by exporting again after being handled, being merged, on outside
For the machine of position, the frame per second that the frame per second of the depth image of the present embodiment output is exported than single depth camera is doubled,
To significantly reduce the delay of image.
It has been observed that frame per second is to directly affect an important indicator of depth image delay, the maximum value of delay is 1/f, this
In f represent frame per second, for example, the delay of depth image may be 33ms, currently, many depth when frame per second f is equal to 30fps
The frame per second of camera is substantially 30fps, can be increased to the output frame rate of depth image using after the technical solution of the application
60fps。
In order to make it easy to understand, being briefly described here to the course of work of depth camera and working stage.TOF depth phase
Machine generally comprises imaging sensor and picture processing chip, and imaging sensor obtains depth data for acquiring object reflected light,
Picture processing chip is used to carry out depth data processing and is converted to digital signal and controls digital data transmission to outside
Receiving end.
Based on this, when the first TOF depth camera of the present embodiment is by the exposure of the imaging sensor of the first TOF depth camera
Between section be arranged to the fluorescent lifetime segment sync indicated with the first synchronization signal, received by imaging sensor and reflected by acquisition object
The digital signal that is converted into of light energy, the first depth image is obtained according to digital signal, and will in the first transmission period
First depth image is transmitted to processor；When 2nd TOF depth camera is by the exposure of the imaging sensor of the 2nd TOF depth camera
Between section be set as with the second synchronization signal indicate fluorescent lifetime segment sync after, by imaging sensor (be here the 2nd TOF depth
Spend the imaging sensor of camera) digital signal being converted by the light energy that acquisition object reflects is received, it is obtained according to digital signal
The second depth image is obtained, and the second depth image is transmitted to processor in the second transmission period.
That is, the TOF depth camera of the present embodiment receives the synchronization signal of microcontroller, and control imaging sensor
Exposure period and synchronization signal instruction light source fluorescent lifetime segment sync, acquisition obtain depth image, obtain depth
After image, depth image is exported and is handled to processor with further by TOF depth camera, being further processed for example here
It is first depth map for receiving the first TOF depth camera by processor and being transmitted in first transmission period
Picture receives second depth image that the 2nd TOF depth camera transmits in second transmission period, according to micro-
The first synchronization signal and the second synchronization signal that controller is sent close first depth image and second depth image
And the depth image as continuous two frames depth image, after being merged, wherein when the first transmission period and the second exposure
Between section it is equal, the first exposure period is equal with the second transmission period.After processor supplies the depth image output after merging
It is continuous to use.
As a result, according to the synchronization signal of microcontroller, processor determines the initial position of each frame in depth image, will be former
Carry out single depth camera directly export after depth image is revised as the depth image merging based on two depth camera interleaved acquisitions
Depth image is exported, the frame per second of depth image is significantly improved, meets application demand.
A technical concept is belonged to aforementioned raising depth image frame per second method, the embodiment of the invention also provides one kind
The system for improving depth image frame per second, Fig. 2 are the block diagram of the system of the raising depth image frame per second of one embodiment of the invention, ginseng
See Fig. 2, the system of the raising depth image frame per second of the present embodiment includes:
Microcontroller (MCU illustrated in Fig. 2), the depth camera group being connect with the microcontroller, depth camera group packet
Include at least two depth cameras (the first TOF depth camera, the 2nd TOF depth camera illustrated in Fig. 2), with depth camera group with
And the processor (FPGA illustrated in Fig. 2) that microcontroller is all connected with；The microcontroller, for controlling the fluorescent lifetime of light source
Section, and synchronization signal is generated according to the fluorescent lifetime section, the synchronization signal is sent to the depth camera；Depth camera,
For being exposed according to the synchronization signal, the sampling depth image in exposure period, and by the depth map of acquisition
As being transmitted to the processor；Processor, for according to the synchronization signal from the microcontroller to described in transmission
Depth image merges, the depth image after being merged.
Referring to fig. 2, depth camera includes the first TOF depth camera and the 2nd TOF depth camera, depth phase in the present embodiment
Machine be it is existing, two depth cameras are all each provided with complete depth image output function.Depth camera is generally passed by image
Sensor, imaging sensor special lens and picture processing chip composition.Referring to fig. 2, the microcontroller of the present embodiment is used for
The signal source of synchronization signal is provided to the first TOF depth camera and the 2nd TOF depth camera, so that depth camera is according to controllable
Working frequency and the time carry out work.FPGA (Field Programmable Gate Array, i.e. field-programmable in Fig. 2
Gate array) synchronization signal that microcontroller is sent is received, while receiving the first TOF depth camera and the 2nd TOF depth camera
Then the depth image signal of transmission merges the depth image that two depth cameras acquire, final output one merging
The depth image of high frame per second later is to host computer.Here FPGA is as one of field specific integrated circuit (ASIC)
Semi-custom circuit and occur, not only solved the deficiency of custom circuit, but also to overcome original programming device gate circuit number limited
The shortcomings that.
The microcontroller illustrated in Fig. 2 include timer and GPIO (General Purpose Input Output,
Universal input/output) interface, it is connect by GPIO interface with the TOF light source for emitting laser in Fig. 2, TOF light here
The main function in source is that light source necessary to work is provided for depth camera, that is, provides benefit to the imaging sensor of depth camera
Light effect shoots external object, and the light source luminescent why controlled independently of depth camera is because TOF is that one kind is based on
It actively shines and carrys out the technology of ranging.The implementation of TOF light source is, for example, to be based on vertical cavity surface emitting laser VCSEL
The laser of (Vertical Cavity Surface Emitting Laser) or Light-emitting diode LED.
The first TOF depth camera of the present embodiment and the structure of the 2nd TOF depth camera are identical, include imaging sensor
With picture processing chip.It should be noted that the first TOF depth camera of the present embodiment and the 2nd TOF depth camera can be adopted
With existing depth camera, the difference is that the triggering mode of depth camera, that is, the exposure triggering mode of camera is different.It is existing
There is each depth camera individually to control itself light source luminescent, and the time for exposure of camera is synchronous with the fluorescent lifetime of light source, this
In embodiment, be by camera outside microcontroller control the fluorescent lifetime of TOF light source, and according to fluorescent lifetime Duan Sheng
Synchronization signal is exported simultaneously at corresponding synchronization signal to the first TOF depth camera or the 2nd TOF depth camera, makes two depths
Degree camera is worked alternatively according to synchronization signal, and then improves the frame per second of depth image.
It has been observed that including picture processing chip in depth camera, in conjunction with Fig. 3 it is found that the course of work of TOF depth camera point
For two Main Stages, the first stage is: picture processing chip control imaging sensor time for exposure, allow it can and light source
Fluorescent lifetime it is synchronous, imaging sensor collects the light energy that object reflects in environment, is then converted into digital signal, issues
Picture processing chip.Second stage is: picture processing chip receives the light energy that imaging sensor transmits and calculates depth
Degree evidence is then transferred to the external FPGA received in integrated circuit such as Fig. 3.
It should be understood that the working frequency of depth camera is determined by the time-consuming in above-mentioned two stage of picture processing chip
It is fixed.As shown in figure 4, the time spent by the generation of each frame image of depth camera is all by the time for exposure T1 of imaging sensor
And depth data transmission time T2 is codetermined.Here depth data transmission time refers to that depth image is defeated from depth camera
Time span spent by the FPGA in external chip, such as the present embodiment is arrived out.The sum of T1 and T2 determine depth camera
Frame per second, that is, frame per second f=1/ (T1+T2).For example, when for time of exposure, (time for exposure here not only includes depth camera
Expose the time expended and including carrying out calculating the spent time to control exposure) it is 15ms (millisecond), for deep
Degree is 18ms according to the time for being transferred to external equipment, then depth camera amounts to used time 33ms in a sampling period, thus
It can obtain, the frame per second of depth camera is 1s/33ms=30 (unit, fps).
In order to improve frame per second, the present embodiment utilizes two depth cameras, and using independently of the microcontroller except camera
MCU generates synchronization signal to control the work of two depth cameras.
Specifically, microcontroller generates the first synchronization signal and the second synchronization signal according to fluorescent lifetime section respectively, it will
First synchronization signal is sent to the first TOF depth camera, and second synchronization signal is sent to the 2nd TOF depth phase
Machine, the first TOF depth camera are exposed according to first synchronization signal, collect first in the first exposure period
First depth image is transmitted to processor in the first transmission period by depth image；2nd TOF depth camera according to
Second synchronization signal is exposed, and the second depth image is collected in the second exposure period, in the second transmission
Between second depth image is transmitted to processor in section.Processor, such as the FPGA in Fig. 2, to the first depth image and
Two depth images are merged according to synchronization signal, and the depth image after being merged is for exporting.
Here the first TOF depth camera, especially by by the exposure period of the imaging sensor of the first TOF depth camera
After being arranged to the fluorescent lifetime segment sync indicated with first synchronization signal, the light of object reflection is received by imaging sensor
Light energy is converted into digital signal by energy, obtains the first depth image according to digital signal, and in the first transmission period
First depth image is transmitted to processor.
Likewise, the 2nd TOF depth camera was specifically used for the time for exposure of the imaging sensor of the 2nd TOF depth camera
After section is arranged to the fluorescent lifetime segment sync indicated with second synchronization signal, it is anti-that object is received by described image sensor
Light energy is converted into digital signal by the light energy penetrated, and obtains the second depth image according to digital signal, and in the second transmission
Between second depth image is transmitted to processor in section.
In one embodiment, microcontroller is by timer generation synchronization signal and through GPIO (universal input/output
Interface) control light source fluorescent lifetime section.That is, MCU both provide work for depth camera A and depth camera B must
The synchronization signal of palpus, the synchronization signal are generated by the timer inside MCU, and since the work of depth camera is according to extraneous defeated
The frequency of the synchronization signal entered carries out.
Referring to fig. 4, in this operating mode, the initial position of each frame of MCU controlling depth camera illustrates in Fig. 4
The generation process of two field pictures, the square-wave signal that Fig. 4 illustrates top are that MCU is generated and exported to the synchronization signal of depth camera,
It illustrates that depth camera is started to work according to each synchronization signal below Fig. 4, that is, starts exposure, transmission data to complete one
The acquisition and transmission of frame image.
Since the depth camera of this active light formula of the present embodiment must can just be worked using special light source, so working as
Two depth cameras are easy to appear the case where light source interferes with each other when working together.
In this regard, the present embodiment avoids light source from interfering using microcontroller generation synchronization signal, it is specifically described below.
It has been observed that the time spent by two working stages of depth camera is respectively T1 and T2, pass through control two here with MCU
The working time of camera makes its alternation, to avoid interfering with each other between two cameras.As shown in figure 5, being adopted at one
Sample moment, MCU control TOF light source luminescent simultaneously generate synchronization signal 1, synchronization signal 1 are sent to depth camera A, depth camera
According to synchronization signal 1 start control image sensor exposure, acquisition exterior object reflection light energy obtain depth data to get
To a frame, depth data is transferred to FPGA.In next sampling instant, MCU control TOF light source luminescent simultaneously generates synchronization signal
2, synchronization signal 2 is sent to depth camera B, depth camera starts to control image sensor exposure, acquisition according to synchronization signal 2
The light energy of exterior object reflection obtains depth data to get to a frame, and depth data is transferred to FPGA.
Based on this present embodiment, only one depth camera works in a fluorescent lifetime section of TOF light source, adjacent
In two fluorescent lifetime sections, two depth cameras work respectively, avoid light source interference.
In the present embodiment, FPGA receives the first depth map that the first TOF depth camera transmits in the first transmission period
Picture receives second depth image that the 2nd TOF depth camera transmits in the second transmission period, is sent out according to microcontroller
First depth image and the second depth image are incorporated as continuous two frame by the first synchronization signal, the second synchronization signal sent
Depth image, the depth image after being merged, wherein the first transmission period is equal with the second exposure period, and first exposes
The light period is equal with the second transmission period.For example, after two depth cameras collect depth image, by the depth of acquisition
Image is exported receives the synchronization signal of MCU to FPGA, FPGA simultaneously.FPGA is transmitted across two depth cameras according to synchronization signal
The depth image come merges.
As shown in fig. 6, the depth image after synchronizing by MCU signal crosss into FPGA in time
(i.e. when a frame depth data of depth camera A is transferred to FPGA in Fig. 6, depth camera B is in the time for exposure of imaging sensor
Section, when a frame depth data of depth camera B is transferred to FPGA, depth camera A is in the exposure period of imaging sensor),
So FPGA can merge the picture signal of intersection (that is, the depth that depth camera A is transmitted in depth data transmission time
The depth image for the depth image that image and depth camera B are transmitted in depth data transmission time merges), to export
One frame per second is single twice of depth camera frame per second of depth image signal to host computer.
The embodiment of the invention also provides a kind of depth camera group, Fig. 7 is the depth camera group of one embodiment of the invention
Block diagram, referring to Fig. 7, the depth camera group 700 of the present embodiment includes: light source 701, image processor 702, the first depth camera
703 and second depth camera 704,
The light source 701 controls signal transmitting according to the fluorescent lifetime section for receiving fluorescent lifetime section control signal
Laser；
First depth camera 703, for receiving the first synchronization signal and being exposed according to first synchronization signal
Light, the sampling depth image in exposure period；
Second depth camera 704, for receiving the second synchronization signal and being exposed according to second synchronization signal
Light, the sampling depth image in exposure period, wherein first synchronization signal and second synchronization signal basis respectively
The fluorescent lifetime section of the light source 701 generates；
Described image processor 702, for receiving first synchronization signal, second synchronization signal, and according to institute
State the first synchronization signal, second synchronization signal merges the depth image, the depth image after being merged.
In one embodiment of the invention, image processor 702 is specifically used for according to first synchronization signal, described
Second transmitted in the first depth image and the second transmission period transmitted in second synchronization signal, the first transmission period
First depth image and second depth image are incorporated as continuous two frames depth image, obtained by depth image
Depth image after merging, wherein first transmission period is equal with second exposure period, first exposure
Period is equal with second transmission period, and first depth image is the first depth camera transmission, described
Second depth image is the second depth camera transmission.
In one embodiment of the invention, the first depth camera 703 is the first TOF depth camera, the second depth camera
704 be the 2nd TOF depth camera, the first TOF depth camera, by the imaging sensor of the first TOF depth camera
After exposure period is set as the fluorescent lifetime segment sync indicated with first synchronization signal, connect by described image sensor
It receives the light energy of object reflection and the light energy is converted into digital signal, obtain the first depth map according to the digital signal
Picture；The 2nd TOF depth camera, by the exposure period of the imaging sensor of the 2nd TOF depth camera be set as with
After the fluorescent lifetime segment sync of the second synchronization signal instruction, the light energy of object reflection is received by described image sensor
And the light energy is converted into digital signal, the second depth image is obtained according to the digital signal.
In one embodiment of the invention, light source 701 is vertical cavity surface emitting laser VCSEL or LED laser device.
In conclusion the raising depth image frame per second method and depth camera group of the embodiment of the present invention, utilize multiple depth
Camera interleaved acquisition depth image, and merge the frame per second that the mode exported after the depth image of acquisition improves depth data, it is full
The demand of the fast responding scenes such as foot body feeling interaction, gesture interaction, improves efficiency, reduces system delay, optimize use
Family experience.
The above description is merely a specific embodiment, under above-mentioned introduction of the invention, those skilled in the art
Other improvement or deformation can be carried out on the basis of the above embodiments.It will be understood by those skilled in the art that above-mentioned tool
Body description only preferably explains that the purpose of the present invention, protection scope of the present invention are subject to the protection scope in claims.
Claims (9)
1. a kind of method for improving depth image frame per second characterized by comprising
It controls the fluorescent lifetime section of light source and synchronization signal is generated according to the fluorescent lifetime section, the synchronization signal is sent to
Each depth camera in depth camera group, the depth camera group include at least two depth cameras；
The depth camera is exposed according to the synchronization signal, the sampling depth image in exposure period；
The depth image is merged according to the synchronization signal, the depth image after being merged.
2. the method according to claim 1, wherein the depth camera includes the first TOF depth camera and the
Two TOF depth cameras,
The fluorescent lifetime section of the control light source simultaneously generates synchronization signal according to the fluorescent lifetime section, and the synchronization signal is sent out
Each depth camera given in depth camera group includes:
The first synchronization signal and the second synchronization signal are generated according to the fluorescent lifetime section, first synchronization signal is sent to
Second synchronization signal is sent to the 2nd TOF depth camera by the first TOF depth camera,
The depth camera is exposed according to the synchronization signal, and sampling depth image includes: in exposure period
The first TOF depth camera is exposed according to first synchronization signal, acquires in the first exposure period
To the first depth image, the 2nd TOF depth camera is exposed according to second synchronization signal, in the second exposure
Between collect the second depth image in section.
3. according to the method described in claim 2, it is characterized in that, it is described according to the synchronization signal to the depth of transmission
Image merges, and the depth image after being merged includes:
According to first depth transmitted in first synchronization signal, second synchronization signal, the first transmission period
Second depth image transmitted in image and the second transmission period, deeply by first depth image and described second
Degree image is incorporated as continuous two frames depth image, the depth image after being merged,
Wherein, first transmission period is equal with second exposure period, first exposure period with it is described
Second transmission period is equal.
4. according to the method described in claim 2, it is characterized in that, the first TOF depth camera, synchronous according to described first
Signal is exposed, and the first depth image is collected in the first exposure period and includes:
The first TOF depth camera, by the exposure period of the imaging sensor of the first TOF depth camera be set as with
After the fluorescent lifetime segment sync of the first synchronization signal instruction, the light energy of object reflection is received by described image sensor
And the light energy is converted into digital signal, the first depth image is obtained according to the digital signal；
The 2nd TOF depth camera is exposed according to second synchronization signal, acquires in the second exposure period
Include: to the second depth image
The 2nd TOF depth camera, by the exposure period of the imaging sensor of the 2nd TOF depth camera be set as with
After the fluorescent lifetime segment sync of the second synchronization signal instruction, the light energy of object reflection is received by described image sensor
And the light energy is converted into digital signal, the second depth image is obtained according to the digital signal.
5. the method according to claim 1, wherein the fluorescent lifetime section of the control light source includes:
The fluorescent lifetime section for emitting the vertical cavity surface emitting laser VCSEL of laser is controlled, or, control is for emitting laser
LED laser device fluorescent lifetime section.
6. a kind of depth camera group, which is characterized in that depth camera group includes: light source, image processor, the first depth camera and
Second depth camera,
The light source controls signal according to the fluorescent lifetime section and emits laser for receiving fluorescent lifetime section control signal；
First depth camera is being exposed for receiving the first synchronization signal and being exposed according to first synchronization signal
Sampling depth image in the light period；
Second depth camera is being exposed for receiving the second synchronization signal and being exposed according to second synchronization signal
Sampling depth image in the light period, wherein first synchronization signal and second synchronization signal are respectively according to the light source
Fluorescent lifetime section generate；
Described image processor, for receiving first synchronization signal, second synchronization signal, and it is same according to described first
Step signal, second synchronization signal merge the depth image, the depth image after being merged.
7. depth camera group according to claim 6, which is characterized in that described image processor is specifically used for according to
The first depth image transmitted in first synchronization signal, second synchronization signal, the first transmission period and the second transmission
First depth image and second depth image are incorporated as continuously by the second depth image transmitted in the period
Two frame depth images, the depth image after being merged,
Wherein, first transmission period is equal with second exposure period, first exposure period with it is described
Second transmission period is equal, and first depth image is the first depth camera transmission, second depth image
It is that second depth camera transmits.
8. depth camera group according to claim 6, which is characterized in that first depth camera is the first TOF depth
Camera, second depth camera are the 2nd TOF depth camera,
The first TOF depth camera, by the exposure period of the imaging sensor of the first TOF depth camera be set as with
After the fluorescent lifetime segment sync of the first synchronization signal instruction, the light energy of object reflection is received by described image sensor
And the light energy is converted into digital signal, the first depth image is obtained according to the digital signal；
The 2nd TOF depth camera, by the exposure period of the imaging sensor of the 2nd TOF depth camera be set as with
After the fluorescent lifetime segment sync of the second synchronization signal instruction, the light energy of object reflection is received by described image sensor
And the light energy is converted into digital signal, the second depth image is obtained according to the digital signal.
9. depth camera group according to claim 6, which is characterized in that the light source is vertical cavity surface emitting laser
VCSEL or LED laser device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910160619.1A CN109889690B (en) | 2019-03-04 | 2019-03-04 | Method for improving frame rate of depth image and depth camera group |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910160619.1A CN109889690B (en) | 2019-03-04 | 2019-03-04 | Method for improving frame rate of depth image and depth camera group |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109889690A true CN109889690A (en) | 2019-06-14 |
| CN109889690B CN109889690B (en) | 2022-08-16 |
Family
ID=66930448
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910160619.1A Active CN109889690B (en) | 2019-03-04 | 2019-03-04 | Method for improving frame rate of depth image and depth camera group |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109889690B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112203076A (en) * | 2020-09-16 | 2021-01-08 | 青岛小鸟看看科技有限公司 | Alignment method and system for exposure center points of multiple cameras in VR system |
| WO2022222497A1 (en) * | 2021-04-20 | 2022-10-27 | 上海图漾信息科技有限公司 | Depth data measurement head, depth data computing device, and corresponding method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120176476A1 (en) * | 2011-01-12 | 2012-07-12 | Sony Corporation | 3d time-of-flight camera and method |
| CN105812651A (en) * | 2015-07-27 | 2016-07-27 | 维沃移动通信有限公司 | Video data processing method and terminal device |
| CN106210584A (en) * | 2016-08-02 | 2016-12-07 | 乐视控股（北京）有限公司 | A kind of video recording method and device |
| CN108683852A (en) * | 2018-05-23 | 2018-10-19 | 努比亚技术有限公司 | A kind of video recording method, terminal and computer readable storage medium |
| CN109104547A (en) * | 2018-08-15 | 2018-12-28 | 中国空气动力研究与发展中心超高速空气动力研究所 | A kind of ultrahigh speed imaging sequences device and method |
-
2019
- 2019-03-04 CN CN201910160619.1A patent/CN109889690B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120176476A1 (en) * | 2011-01-12 | 2012-07-12 | Sony Corporation | 3d time-of-flight camera and method |
| CN105812651A (en) * | 2015-07-27 | 2016-07-27 | 维沃移动通信有限公司 | Video data processing method and terminal device |
| CN106210584A (en) * | 2016-08-02 | 2016-12-07 | 乐视控股（北京）有限公司 | A kind of video recording method and device |
| CN108683852A (en) * | 2018-05-23 | 2018-10-19 | 努比亚技术有限公司 | A kind of video recording method, terminal and computer readable storage medium |
| CN109104547A (en) * | 2018-08-15 | 2018-12-28 | 中国空气动力研究与发展中心超高速空气动力研究所 | A kind of ultrahigh speed imaging sequences device and method |
Non-Patent Citations (2)
| Title |
|---|
| S. KIM ET AL.: "A CMOS Image Sensor Based on Unified Pixel Architecture With Time-Division Multiplexing Scheme for Color and Depth Image Acquisition", 《IEEE JOURNAL OF SOLID-STATE CIRCUITS》 * |
| 盖赟等: "多尺度方法结合卷积神经网络的行为识别", 《计算机工程与应用》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112203076A (en) * | 2020-09-16 | 2021-01-08 | 青岛小鸟看看科技有限公司 | Alignment method and system for exposure center points of multiple cameras in VR system |
| CN112203076B (en) * | 2020-09-16 | 2022-07-29 | 青岛小鸟看看科技有限公司 | Alignment method and system for exposure center points of multiple cameras in VR system |
| US11962749B2 (en) | 2020-09-16 | 2024-04-16 | Qingdao Pico Technology Co., Ltd. | Virtual reality interaction method, device and system |
| WO2022222497A1 (en) * | 2021-04-20 | 2022-10-27 | 上海图漾信息科技有限公司 | Depth data measurement head, depth data computing device, and corresponding method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109889690B (en) | 2022-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3819872A1 (en) | Image acquisition and processing methods and apparatuses for three-dimensional scanning, and three-dimensional scanning device | |
| JP6663718B2 (en) | Intraoral scanning device with illumination frame incorporated into image frame | |
| CN105812673B (en) | Flash lamp control system and method | |
| KR20130045018A (en) | 3d image acquisition apparatus and method of acqiring depth information in the 3d image acquisition apparatus | |
| CN107948515A (en) | A kind of camera synchronous method and device, binocular camera | |
| CN106131419B (en) | It is a kind of for synchronizing the method and system and virtual reality system of multiple wireless camera equipment | |
| CN104603574A (en) | Motion sensor device having plurality of light sources | |
| CN109889690A (en) | A kind of method and depth camera group improving depth image frame per second | |
| JP2008252401A (en) | Imaging system, imaging method, and imaging program | |
| US11831993B2 (en) | Information processing apparatus, information processing system, device for position and posture acquisition, and device information acquisition method | |
| CN110431841A (en) | Virtually, the depth sense technology of enhancing and mixed reality system | |
| CN106576159A (en) | Photographing device and method for acquiring depth information | |
| US20150154035A1 (en) | Multiple sensors processing system for natural user interface applications | |
| CN106817794A (en) | TOF circuit modules and its application | |
| KR101288030B1 (en) | Hybrid 3D scanner | |
| CN115428017A (en) | Event sensor based depth estimation | |
| CN105005994B (en) | A kind of 3D scan components, scanning system and 3D printing system | |
| WO2022068193A1 (en) | Wearable device, intelligent guidance method and apparatus, guidance system and storage medium | |
| KR20160111828A (en) | Digital Micromirror Device Controller for processing light exposure data at high speed with type based by location and system thereof | |
| JP2013160624A (en) | Three-dimensional measurement system and method | |
| CN110609555A (en) | Method, apparatus, electronic device, and computer-readable storage medium for signal control | |
| JP4954143B2 (en) | Position detection device | |
| CN103791832A (en) | Binocular stereo vision multi-thread tracking and positioning method | |
| CN108694383A (en) | A kind of gesture identifying device and its control method, display device | |
| US20220417412A1 (en) | Imaging system, control method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |