CN109274864A - Depth camera, depth calculation System and method for - Google Patents
Depth camera, depth calculation System and method for Download PDFInfo
- Publication number
- CN109274864A CN109274864A CN201811030786.6A CN201811030786A CN109274864A CN 109274864 A CN109274864 A CN 109274864A CN 201811030786 A CN201811030786 A CN 201811030786A CN 109274864 A CN109274864 A CN 109274864A
- Authority
- CN
- China
- Prior art keywords
- depth
- depth calculation
- camera
- image
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of depth camera, depth calculation System and method for, which includes: emitting mould train, for objective emission light beam；Receiving module, for receiving the light beam being reflected back by target；Depth calculation chip carries out depth calculation for beam information based on the received to obtain depth image；The depth calculation chip further includes configuration module, for carrying out the configuration of configuration parameter when executing depth calculation to the depth calculation chip, to export the depth image of different-effect.By the way that configuration module is arranged in depth calculation chip, the configuration of configuration parameter is carried out when executing depth calculation to depth calculation chip, so that depth algorithm can export the depth image of different-effect under the premise of algorithm frame is immovable, to meet the needs of different application.
Description
Technical field
The invention belongs to electronic technology fields, are to be related to a kind of depth camera, depth calculation system and side more specifically
Method.
Background technique
Depth camera can be used to obtain the depth information of target, and three-dimensional reconstruction, face may be implemented based on depth information
The functions such as identification, gesture interaction.Insert depth camera will become future terminal and be regarded from traditional 2D vision to 3D in terminal device
Feel the key of transformation, but also faces some challenges simultaneously.
Depth camera generally comprises transmitter, receiver and processor, and wherein processor is for calculating depth image
And other terminal devices are further transferred to do further calculation processing.Such as existing depth camera such as KINECT,
ASTRA etc., internal to contain depth calculation processor, depth calculation algorithm is cured to be promoted on the depth calculation processor and calculate
Efficiency.After depth camera is integrated into terminal device, depth image can be called with reality when executing specific application program
Now corresponding function, however, many kinds of due to application program, the requirement to depth image is also inconsistent, such as Three-dimensional Gravity
It builds and requires the frame per second that depth image precision is high, gesture interaction then requires depth image high.
Therefore, existing scheme is difficult to meet the diversified demand of depth image.
Summary of the invention
To solve the above problems, the present invention proposes a kind of depth camera, depth calculation system and method, the camera institute is utilized
The depth image of acquisition has the characteristics that diversified, can satisfy the demand of different application.
The present invention provides a kind of depth camera, comprising: emitting mould train is used for objective emission light beam；Receiving module is used for
Receive the light beam being reflected back by target；Depth calculation chip, for beam information based on the received carry out depth calculation with
Obtain depth image；The depth calculation chip further includes configuration module, for executing depth to the depth calculation chip
The configuration of configuration parameter is carried out when calculating, to export the depth image of different-effect.
In some embodiments, the configuration parameter include: the zoning parameter of image, depth calculation efficiency parameters,
One of depth calculation precision parameter, depth calculation resolution parameter, depth calculation efficacy parameter are a variety of.
Wherein, the zoning parameter of described image includes starting pixels and termination pixel value；The depth calculation efficiency
Parameter or depth calculation precision parameter include search window value, field of search thresholding, the number of iterations, similarity threshold.The depth
Calculating resolution parameter includes search paces value；The depth calculation efficacy parameter includes the illumination invariant factor.
The present invention also provides a kind of depth calculation systems, comprising: depth camera as described above, for calculating depth map
Picture；Processor, for carrying out whole control to depth calculation system；Memory includes that the depth camera is being held for storing
Required supplemental characteristic when row depth calculation.
In some embodiments, depth calculation system further include display, it is interface, magazine one or more；It is described
Display is for showing image；The interface for being attached with outside, the camera include color camera, panorama camera,
One of light-field camera, wide angle camera are a variety of.
In some embodiments, the depth camera is structure light depth camera；In further embodiments, the depth
Camera is TOF depth camera.
The present invention also provides a kind of depth computing methods, comprising: processor controls emitting mould train transmitting light beam and control
Receiving module receives the light beam；Depth calculation chip receives the light beam and calls the supplemental characteristic stored in memory, leads to
It crosses the configuration of configuration parameter and then calculates depth image.
Beneficial effects of the present invention: by the way that configuration module is arranged in depth calculation chip, depth calculation chip is being held
When row depth calculation carry out configuration parameter configuration so that depth algorithm can under the premise of algorithm frame is immovable,
The depth image of different-effect is exported, to meet the needs of different application.
Detailed description of the invention
Fig. 1 is depth camera side schematic view in one embodiment of the invention.
Fig. 2 is depth calculation system structure diagram in one embodiment of the invention.
Fig. 3 is depth calculation system structure diagram in another embodiment of the invention.
Fig. 4 is the schematic diagram of depth algorithm routine module in another embodiment of the invention.
Fig. 5 is depth calculation system structure diagram in further embodiment of the present invention.
Specific embodiment
With reference to embodiment and compares attached drawing invention is further described in detail, it should be emphasised that,
Following the description is only exemplary, the range and its application being not intended to be limiting of the invention.
Shown in FIG. 1 is depth camera side schematic view according to an embodiment of the invention.10 main groups of depth camera
There are emitting mould train 101, receiving module 102, control circuit board 105, depth calculation chip 103 and memory (in figure at component
It is not shown).Other cameras such as color camera 104 is further provided in some depth cameras.Emitting mould train 101, receiving module 102
And color camera 104 is generally mounted in the same depth camera plane, and is in same baseline, each mould group or phase
Machine all corresponds to an aperture window.Depth calculation chip 103 is generally integrated in control circuit board 105, other mould groups and control
Circuit board 105 processed connects, and depth calculation chip 103 is realized with certain interface to each mould group via control circuit board 105
Control.Depth calculation chip 103 also includes output interface 106, such as USB interface, MIPI interface, is used for depth image, coloured silk
Chromatic graph picture is output to other equipment.Emitting mould train 101 is used to go out to objective emission light beam, and receiving module 102 is for receiving by mesh
The light beam being reflected back is marked, depth calculation chip 103 then carries out depth calculation for beam information based on the received to obtain
Depth image.
Depth camera includes structure light depth camera, TOF depth camera etc..In one embodiment, depth camera is knot
Structure optical depth camera, wherein emitting mould train receives mould for projecting encoded structured light patterns light beam into object space
Structured light patterns are carried out depth calculation to obtain by output to depth calculation chip, chip after group collects the structured light patterns
The depth image of object space.Emitting mould train may include light source, lens and diffraction optical element, the structure light figure emitted
Case is infrared speckle pattern, and pattern is relatively uniform with distribution of particles but has very high irrelevance, and receiving module is at this time
Infrared camera.Structured light patterns are also possible to the other forms such as striped, two-dimensional pattern.In one embodiment, depth camera is
TOF depth camera, wherein emitting mould train is for emitting modulated light beam, and receiving module is for receiving the light being reflected back by object
Beam, depth calculation chip is for demodulating the light beam being reflected back, to calculate transmitting and received time difference, and when utilizing
Between flight method principle calculate the depth image of target.
It is understood that depth camera also may include other types, such as binocular structure light depth camera etc..With
Under explanation in will be illustrated by taking structure light depth camera as an example, principle can be applied to other kinds of depth camera
In.
Terminal device has brought the terminals such as unlimited convenience, such as mobile phone, plate, computer, TV and may be implemented to lead to
The functions such as words, amusement, office, while the function of mobile terminal will also constantly extend, such as recognition of face unlock, mobile payment
Deng.Thus also higher and higher to the hardware requirement of mobile terminal, the trend in terminal device future will be become by integrating depth camera.
Fig. 2 is the terminal device schematic diagram according to an embodiment of the invention for being integrated with depth camera.Terminal device 20
It is integrated with depth camera therefore has depth image computing function, unify this kind of equipment being known as depth calculation system below
System.Depth calculation system 20 includes the depth camera, processor 205, memory 206 etc. for depth calculation, can also include
Such as display 210, camera 212 and interface 211.
Wherein depth camera includes receiving module 201, emitting mould train 202 and depth calculation chip 203, concrete function
Referring to the description to Fig. 1.
Processor 205 is for carrying out whole control to whole system, and processor 205 can be single processor can also
To include multiple processor units, including but not limited to central processing unit (CPU), graphics processor (GPU), Digital Signal Processing
Device (DSP), neural network processor (NPU), image-signal processor (ISP) etc..In some embodiments, processor 20 can be with
It is integrated system-on-chip (SoC) or specific integrated circuit (ASIC), comprising processors such as CPU, on-chip memory, controller, leads to
Believe interface etc..In some embodiments, processor 20 is application processor AP, such as mobile application processor, is mainly responsible for shifting
Except the realization, such as text-processing, image procossing etc. of the outer other function of communication in dynamic terminal.
Display 210 is presented to the user for showing image under the control of the processor 205 to apply etc., is in addition shown
Device also may include touch function, and display is also used as man-machine interactive interface at this time, for receiving the input of user.
Interface 211 is used for so that system is attached with outside, to further realize the function such as data transmission, power transmission
Can, interface 211 is controlled via the communication interface in processor 205.Interface includes USB interface, WIFI interface etc..
Camera 212 is generally used for acquisition image, such as color camera, panorama camera, light-field camera, wide angle camera etc..
It is understood that camera 212 is also possible to a part of depth camera, as shown in Figure 1.
Memory 206 for saving data, such as system data 207, application data 208, supplemental characteristic 209 and
Temporary designator that processor 205 saves in the process of implementation and data etc..The operations systems such as system data such as Android, Linux
System, the common chat program of the such as user of application data 208, shopping program, unlocking program, payment program etc., parameter
Data 209 include the data that depth calculation chip 203 is needed when executing depth calculation, such as reference configuration light image, depth
Camera inside and outside parameter (such as outer ginseng between the internal reference and receiving module and emitting mould train of receiving module) etc..Such as list
It include reference configuration light image in supplemental characteristic 209 for mesh structure light depth camera, depth calculation chip 203 will utilize ginseng
The structure light image of table structure light image and current goal carries out depth calculation to obtain the depth image of current goal.Depth gauge
It calculates chip 203 and the triggering of each mould group, such as synchronous touching is connect and controlled with receiving module 201, emitting mould train 202 by interface
Hair is to acquire structure light image.In some embodiments, it is also possible to directly be connect with each mould group by processor 205 same to realize
The controls such as step triggering, it is such to be advantageous in that, it needs to be executed using the application program of depth image by processor 205 some
When, application processor 20 can directly control each mould group, to simplify operation, raising efficiency.Here interface can be with
Including various interfaces, such as integrated bus interface I2C。
In one embodiment, cancel the memory in depth camera to be beneficial to reduce the volume of depth camera, reduce
Power consumption, easily facilitates integrated, and the supplemental characteristic that is related to can be saved in that system 20 is public to deposit in the calculating such as depth image
In reservoir 206.When carrying out depth calculation, processor 205 can call the supplemental characteristic 209 in memory 206 and be transferred to
Depth calculation chip 203 is further to execute depth image calculating.Depth calculation chip 203 can also directly with memory 206
Connection, with the supplemental characteristic 209 for calling it to store.
Memory 206 can be made of single or multiple memories, can be that RAM, FLASH etc. are any to be can be used for saving
The form of memory of data.It is understood that memory both can be used as a part of system, it can also be independently of mobile whole
End exists, such as cloud memory, and the data saved can be waited by interface 211 and be communicated with depth calculation system 20.Using
Program such as recognition of face, gesture interaction application is generally held in non-volatile readable storage medium, when the execution application
When, processor will call corresponding program to execute from the storage medium.Memory 206 can also be used as the one of depth camera
Part, as shown in Figure 1.
Fig. 3 is the schematic diagram of the depth calculation system of another embodiment according to the present invention.In the present embodiment, it eliminates
Dedicated for the depth calculation chip of depth calculation to reduce cost, the calculating task of depth image will be by depth calculation system
Processor 303 execute.Depth calculation algorithm (depth algorithm) 306 will in form of software programs (i.e. depth algorithm routine)
It is stored in memory 305.Processor 303 directly connect with receiving module 301, emitting mould train 302, for example passes through I2C interface
With realize to the control of mould group, by the interfaces such as MIPI can transmit by receiving module 301 transmit come infrared (IR) structure light
Image (is illustrated) by taking infrared camera as an example.When carrying out depth calculation, processor 303 will call the depth in memory 305
The supplemental characteristic 307 for spending algorithm 306 and auxiliary operation, executes corresponding depth algorithm meter after receiving IR structure light image
Instruction is calculated to calculate depth image, in one embodiment, depth image can further be shown by display 304.Depth is calculated
Method 306 can be single algorithm routine also and may include many algorithms program, for example can simultaneously include based on monocular, binocular etc.
The algorithm routine of a variety of depth calculations.In one embodiment, corresponding for monocular initiating structure light Depth Imaging
Depth algorithm routine generally comprise image preprocessing, images match, depth value calculate and etc..Wherein, image preprocessing be
The noise of structure light image, contrast, brightness etc. are handled after receiving structure light image；Images match refers to utilization
With algorithm, such as MAD algorithm (MAD), absolute error and algorithm (SAD), error sum of squares algorithm (SSD), mean error
Quadratic sum algorithm (MSD), normalization product correlation al gorithm (NCC), sequential similarity detection algorithm (SSDA), hadamard transformation are calculated
Method (SATD) etc. carries out matching primitives to structure light image and reference configuration light image to export the deviation value of respective pixel；It is deep
Angle value calculates the depth value that pixel is calculated using deviation value and structure light trigonometry principle.
Can be called after depth camera is integrated into terminal device, when executing specific application program depth image with
Realize corresponding function, however, many kinds of due to application program, the requirement to depth image is also inconsistent, such as face
Identification requires the frame per second that depth image precision is high, gesture interaction then requires depth image high.In order to solve to depth image multiplicity
Property require the problem of, the present invention provides several solutions.
As shown in Fig. 2, being provided with configuration module 204 in depth calculation chip 203, configuration module 204 can be to depth gauge
The configuration that the depth algorithm run in chip 203 carries out parameter is calculated, so that depth algorithm can not change in algorithm frame
Under the premise of, export the depth image of different-effect.Configuration parameter includes but is not limited to following several (with structure optical depth algorithm
For be illustrated):
The zoning parameter of 1 image.When structure light image is carried out depth calculation, usually entire image is carried out
Matching primitives, final output whole picture depth image.However in some applications, it is only necessary to which a few pixels therein are calculated
, for example for face recognition application, often human face region only accounts for a part of entire image, is accounted for human face region area
The half and human face region of all images area are illustrated for being placed exactly in picture centre, it is assumed that the length and width of image be (L,
H), unit is pixel, and the human face region upper left corner and lower right corner pixel coordinate are respectively (L/4, H/4), (3L/4,3H/4).Then exist
In the present embodiment, recognition of face is carried out first with color image or infrared floodlight image, that is, identifies that the pixel of human face region is sat
Secondly cursor position only carries out depth calculation to face region, specifically, held by configuring the parameter of reflection zoning
Row depth calculation, for example configuration parameter is starting pixel value (such as top left corner pixel coordinate value) and to terminate pixel value (such as right
Inferior horn pixel coordinate value), starting pixel value and termination pixel value can accurately reflect zoning, and depth calculation chip is being held
It is then calculated according to starting pixels with pixel is terminated when row depth calculation, final output is partial depth image.With it is existing
Technology is compared, and since the pixel number of calculating reduces half, calculation amount halves, and computational efficiency is significantly improved.
2 depth calculation efficiency parameters.When carrying out depth calculation, computational efficiency is often depending on the efficiency of matching algorithm,
And matching algorithm efficiency is then frequently depend upon the window size of matching algorithm, region of search size, matched the number of iterations, similar
Spend threshold value etc..Therefore in the more demanding application of some pairs of depth image computational efficiencies, for example gesture interaction requires frame per second high,
The depth image of high frame per second can be then exported by configuring these efficiency parameters, for example search window is reduced, region of search
Reduce, the number of iterations reduces, similarity threshold reduces (similarity value is bigger to indicate more related) etc..Efficiency parameters include but unlimited
In parameters such as above mentioned window size, region of search size, matched the number of iterations, similarity thresholds.
3 depth calculation precision parameters.The precision of depth calculation is often depending on the search precision of matching algorithm, further again
It is frequently depend upon window size, region of search size, matched the number of iterations, the similarity threshold etc. of matching algorithm.Some
In the application more demanding to depth image computational accuracy, for example 3-D scanning requires depth image precision high, then can pass through
Configure these efficiency parameters to export high accuracy depth image, for example search window increase, region of search are increased or extended
(for example being scanned for along horizontal, longitudinal), the number of iterations increase, similarity threshold increase etc..Efficiency parameters including but not limited to
The parameters such as above mentioned window size, region of search size, matched the number of iterations, similarity threshold.
It cannot concurrently reach between depth calculation precision and depth calculation efficiency, the promotion of depth calculation precision frequently results in
The decline of depth calculation efficiency.Also give in the above description can by the window size of matching algorithm, region of search size,
The parameters such as matched the number of iterations, similarity threshold come realize precision and efficiency output control, below with a specific example
To be illustrated.In one embodiment, the actual pixels resolution ratio of receiving module is 1280x960, and the gesture run now is handed over
The output frame rate of mutual application requirement depth image is 60fps.When executing depth calculation algorithm, set search window to
21x21, region of search are that entire image, the number of iterations be unlimited, similarity threshold is that 0.95 (i.e. search window is in region of search
It scans for, and calculates the similarity of search window and target window in real time, for example utilize SSD matching algorithm, when similarity reaches
When to 0.95, search terminates, and the pixel of the window center searched is to want matched object pixel, when similarity is lower than 0.95
When, continue searching, i.e., continuous iteration), the calculation amount of algorithm is very big at this time, it is difficult to realize the depth image output of 60fps.This
When, if reducing algorithm window size to 11x11, similarity calculation can be accelerated, to promote computational efficiency；Or by region of search
It is limited in along certain innings of region of base direction, search speed can be accelerated to promote computational efficiency；Or the number of iterations is limited
At 5 times, i.e., no matter the pixel that similarity reaches threshold value can be searched in 5 times, terminate to search for, it thus also can greatly
Fast computational efficiency；Or by similarity threshold be set as 0.8 even lower value (in the present embodiment, the higher expression of similarity value
It is more similar), it thus can also accelerating algorithm computational efficiency；Or two or more parameters are changed simultaneously, efficiency of algorithm
Promotion can be more obvious.On the contrary, above-mentioned parameter is adjusted in the opposite direction, institute if wanting to promote depth calculation precision
Bring is the result is that calculation amount increases, and computational efficiency reduces, but more preferably due to matching result, the depth image precision obtained from
It is higher.
4 depth calculation resolution parameters.Depth calculation resolution ratio determines the true resolution ratio of the depth image of generation, one
If depth calculation is carried out to each pixel in structure light image as and exports depth identical with structure light image resolution ratio
Image.In practical application, in the application low for some pairs of resolution requirements, such as the application of short distance Depth Imaging, low point
Resolution depth image also may include enough depth informations.Resolution ratio can be adjusted by search paces, search strategy etc.,
For example be set as carrying out matching primitives every a pixel by search paces, then resolution ratio can be dropped to original 1/4, or
Person uses sub-pix search strategy, then the calculating of super-resolution depth image may be implemented.
5 depth calculation efficacy parameters.It is different to the calculating effect requirements of depth image in different application, such as some
Higher to the precision of object edge in depth image using requiring nothing more than, the hole of some application requirement depth images is few, some are answered
It can be at this time to reach these to calculate effect by configuring some efficacy parameters to illumination-insensitive etc. with depth calculation is required,
For example depth calculation is reduced to the susceptibility etc. of illumination by adjusting the illumination invariant factor parameter in matching algorithm.
Above configuration parameter is merely exemplary to be classified, and the configuration parameter in actually distinct classification can also phase
Together, each classification is also related between each other, for example reduction resolution ratio can then promote computational efficiency, therefore in practical applications,
Needs can be applied according to different, be not limited to more than zoning, computational accuracy, computational efficiency, calculating resolution, calculating
The demands such as effect, configuration parameter can also carry out different settings, and configuration parameter is also not limited to the parameter that example illustrated above goes out.
It is understood that can equally be arranged in its depth calculation chip for depth camera shown in FIG. 1
Configuration module, configuration module can carry out the configuration of parameter to the depth algorithm run in depth calculation chip, so that deep
Degree algorithm can export the depth image of different-effect under the premise of algorithm frame is immovable.It is possible thereby to make depth phase
Machine exports the depth image of different-effect, to meet the needs of different application.
In one embodiment, configuration module includes register, and the unit for saving various parameters is arranged in inside, such as
Depth calculation region parameter unit, depth calculation efficiency parameters unit, depth calculation precision parameter unit, depth calculation resolution ratio
Parameters unit and depth calculation efficacy parameter unit etc., the parameter saved in each unit, can be with when executing depth calculation
It is called in due course by other modules in chip.The parameter saved in these units can also be configured according to different needs, than
The processor for being such as configured module-external is accessed to configure corresponding parameter.
Similarly, in embodiment shown in Fig. 3, in order to meet different application demands, depth algorithm routine needs
Special setting is carried out.Fig. 4 is the exemplary diagram of depth algorithm routine module according to an embodiment of the invention.Depth algorithm
Module 401 contains at least two depth calculation algorithm, for example includes algorithm 1, algorithm 2 and algorithm 3, different calculations
Method can export the depth image of different-effect.For example the core matching algorithm of algorithm 1 is gray-scale Matching algorithm (such as digitized map
As related algorithm), high-precision matching primitives can be carried out to export high accuracy depth image；And feature is used in algorithm 2
Matching algorithm can carry out quickly calculating to export the depth image of high frame per second.By the way that a variety of different algorithms are bundled to together
It can realize that the depth image of different demands calculates in one software with lower cost.With shown in Fig. 2 with hard-wired calculation
Method is compared, and cost is lower, it is clear that multiple and different algorithms is cured to difficulty on chip and cost is much greater.
In some embodiments, it is also possible to which some configuration parameters are arranged to depth algorithm routine, for example implement to shown in Fig. 3
In depth algorithm 306 in example, configuration parameter can make depth algorithm routine when executing depth calculation to export different effects
The depth image of fruit.It for another example can be to the single or multiple depth algorithms in embodiment illustrated in fig. 4 in depth algorithm mould group 401
Some configuration parameters are set, such as the configuration parameter 405,406 and 407 in Fig. 4.These configuration parameters can be when algorithm executes
According to the demand of current application program, further each algorithm is adjusted, in above-mentioned Fig. 2 of function reference pair of configuration parameter
Configuration module 2 in the explanation of configuration parameter.
In some embodiments, for example the application high to depth image security requirement, such as payment are applied, depth algorithm
Security requirement is higher, and depth algorithm may operate on the processor with security context at this time, such as arm processor, number
Character information processor (DSP) etc..
Fig. 5 is the schematic diagram of the depth calculation system of further embodiment according to the present invention.Place in depth calculation system
Reason device 503 contain a special module --- depth calculation engine 504, the engine then dedicated for execute depth calculation, than
Such as when processor 503 is using arm processor as system level chip (SoC) of core, in the SoC be arranged one piece dedicated for
The engine of depth calculation, the engine include but is not limited to one of DSP, ISP, GPU, ASIC, FPGA, NPU or the shapes such as combination
Formula.Likewise it is possible to configuration module 505 be arranged in depth calculation engine 504 to realize the depth calculation of different demands.One
In a embodiment, with the configuration module of depth calculation chip, the configuration module in depth calculation engine is also to preserve each seed ginseng
Several registers can be called in due course by other modules in depth calculation engine.
Fig. 2, Fig. 3 and three kinds of schemes shown in fig. 5 are used only as illustrating, not to the limitation of the present invention program.Three kinds of sides
Case respectively has an advantage and disadvantage, and for scheme shown in Fig. 2 due to using independent depth calculation chip, computational efficiency is high, although can be with
The depth image of different demands is exported by configuration parameter, but since algorithm frame has been cured in the chips, it is immutable,
Therefore the depth image of output does not have change largely, simultaneously because independent chip, therefore cost is also higher；
Algorithm in scheme shown in Fig. 3 is saved in memory in form of software programs, is called when needing by processor, advantage
Be that the entirely different depth algorithm of frame can be set, and be highly convenient for the algorithm update in later period, replacement etc., but disadvantage exists
It is slow in computational efficiency, power consumption is relatively high, safety is lower；For scheme shown in fig. 5, for another two scheme,
Higher computational efficiency and safety are not only remained, but also is reduced costs.
The step of executing respective depth algorithm according to concrete application program according to an embodiment of the invention, is as follows.
Step 601, current application program, such as the people for mobile terminal are executed by the processor in depth calculation system
Face solution locked application, when mobile terminal be lifted or by button trigger after, face solution locked application is triggered, i.e., by
Processor calls corresponding application program in memory to be performed.
Step 602, requirement of the processor according to current application program to depth image, execute corresponding depth algorithm or
Depth algorithm is executed according to corresponding configuration parameter.In one embodiment, application program is set in advance to depth image
Demand parameter setting and depth algorithm configuration parameter setting, for example the parameters such as resolution ratio, precision, frame per second, processor exist
When executing the application program, while extracting corresponding demand parameter, and according to the demand parameter come configure corresponding parameter or
Corresponding algorithm then executes corresponding depth algorithm or executes depth algorithm according to corresponding configuration parameter.Here it says
The corresponding depth algorithm of execution it is corresponding be embodiment shown in Fig. 3, depth algorithm is executed according to different configuration parameters
Then it is suitable for Fig. 2, Fig. 3 and embodiment shown in fig. 5.
Step 603, calculated depth image is further processed processor according to application program, for example is showing
It has been shown that, image segmentation, Face datection, recognition of face, 3D modeling, gesture interaction etc. are handled on device.
In some embodiments, when executing depth calculation in step 602, the same application program in the different stages may be used
To carry out the adjustment of algorithm or the adjustment of configuration parameter.For example 3D face is unlocked or is paid and is applied, can substantially it be divided into
Face datection, recognition of face, unlock or payment several stages do not need high-precision or high score in the Face datection stage of beginning
The depth image of resolution can calculate the depth map of low precision or low resolution by corresponding algorithm or configuration parameter at this time
Picture calculates high-precision or high-resolution depth by algorithm adjustment or configuration parameter adjustment then in the recognition of face stage
Image is spent to meet recognition of face demand.
The above content is combine it is specific/further detailed description of the invention for preferred embodiment, cannot recognize
Fixed specific implementation of the invention is only limited to these instructions.For those of ordinary skill in the art to which the present invention belongs,
Without departing from the inventive concept of the premise, some replacements or modifications can also be made to the embodiment that these have been described,
And these substitutions or variant all shall be regarded as belonging to protection scope of the present invention.
Claims (10)
1. a kind of depth camera characterized by comprising
Emitting mould train is used for objective emission light beam；
Receiving module, for receiving the light beam being reflected back by target；
Depth calculation chip carries out depth calculation for beam information based on the received to obtain depth image；
The depth calculation chip further includes configuration module, for carrying out to the depth calculation chip when executing depth calculation
The configuration of configuration parameter, to export the depth image of different-effect.
2. depth camera as described in claim 1, which is characterized in that the configuration parameter includes: the zoning ginseng of image
In number, depth calculation efficiency parameters, depth calculation precision parameter, depth calculation resolution parameter, depth calculation efficacy parameter
It is one or more.
3. depth camera as claimed in claim 2, which is characterized in that the zoning parameter of described image includes starting pixels
Value and termination pixel value.
4. depth camera as claimed in claim 2, which is characterized in that the depth calculation efficiency parameters or the computational accuracy
Parameter includes search window value, field of search thresholding, the number of iterations, similarity threshold.
5. depth camera as claimed in claim 2, which is characterized in that the depth calculation resolution parameter includes search paces
Value.
6. depth camera as claimed in claim 2, which is characterized in that the depth calculation efficacy parameter include illumination invariant because
Son.
7. a kind of depth calculation system characterized by comprising
Depth camera as described in any one of claims 1 to 6, for calculating depth image；
Processor, for carrying out whole control to depth calculation system；
Memory includes the depth camera supplemental characteristic required when executing depth calculation for storing.
8. depth calculation system as claimed in claim 7, which is characterized in that further include display, interface, magazine one kind
Or it is a variety of；The display is for showing image；For the interface for being attached with outside, the camera includes colored phase
One of machine, panorama camera, light-field camera, wide angle camera are a variety of.
9. depth calculation system as claimed in claim 7, which is characterized in that the depth camera includes structure light depth camera
Or TOF depth camera.
10. a kind of depth computing method of such as described in any item depth calculation systems of claim 7-9, which is characterized in that packet
It includes:
Processor control emitting mould train, which emits light beam and controls and receives mould group, receives the light beam；
Depth calculation chip receives the light beam and calls the supplemental characteristic stored in memory, by the configuration of configuration parameter into
And calculate depth image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811030786.6A CN109274864A (en) | 2018-09-05 | 2018-09-05 | Depth camera, depth calculation System and method for |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811030786.6A CN109274864A (en) | 2018-09-05 | 2018-09-05 | Depth camera, depth calculation System and method for |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109274864A true CN109274864A (en) | 2019-01-25 |
Family
ID=65187855
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811030786.6A Pending CN109274864A (en) | 2018-09-05 | 2018-09-05 | Depth camera, depth calculation System and method for |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109274864A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112700482A (en) * | 2019-10-23 | 2021-04-23 | 深圳市优必选科技股份有限公司 | Camera depth resolution determination method and device, storage medium and intelligent device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103198473A (en) * | 2013-03-05 | 2013-07-10 | 腾讯科技（深圳）有限公司 | Depth image generating method and device |
| CN105931240A (en) * | 2016-04-21 | 2016-09-07 | 西安交通大学 | Three-dimensional depth sensing device and method |
| CN106323190A (en) * | 2016-09-26 | 2017-01-11 | 深圳奥比中光科技有限公司 | Depth measurement range-customizable depth measurement method and system for obtaining depth image |
| CN106504227A (en) * | 2016-09-26 | 2017-03-15 | 深圳奥比中光科技有限公司 | Demographic method and its system based on depth image |
| US20170295360A1 (en) * | 2016-04-07 | 2017-10-12 | Seiko Epson Corporation | Head-mounted display device and computer program |
| CN107341481A (en) * | 2017-07-12 | 2017-11-10 | 深圳奥比中光科技有限公司 | It is identified using structure light image |
| CN107424188A (en) * | 2017-05-19 | 2017-12-01 | 深圳奥比中光科技有限公司 | Structured light projection module based on VCSEL array light source |
| CN207802143U (en) * | 2018-02-01 | 2018-08-31 | 深圳奥比中光科技有限公司 | depth imaging mobile terminal |
-
2018
- 2018-09-05 CN CN201811030786.6A patent/CN109274864A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103198473A (en) * | 2013-03-05 | 2013-07-10 | 腾讯科技（深圳）有限公司 | Depth image generating method and device |
| US20170295360A1 (en) * | 2016-04-07 | 2017-10-12 | Seiko Epson Corporation | Head-mounted display device and computer program |
| CN105931240A (en) * | 2016-04-21 | 2016-09-07 | 西安交通大学 | Three-dimensional depth sensing device and method |
| CN106323190A (en) * | 2016-09-26 | 2017-01-11 | 深圳奥比中光科技有限公司 | Depth measurement range-customizable depth measurement method and system for obtaining depth image |
| CN106504227A (en) * | 2016-09-26 | 2017-03-15 | 深圳奥比中光科技有限公司 | Demographic method and its system based on depth image |
| CN107424188A (en) * | 2017-05-19 | 2017-12-01 | 深圳奥比中光科技有限公司 | Structured light projection module based on VCSEL array light source |
| CN107341481A (en) * | 2017-07-12 | 2017-11-10 | 深圳奥比中光科技有限公司 | It is identified using structure light image |
| CN207802143U (en) * | 2018-02-01 | 2018-08-31 | 深圳奥比中光科技有限公司 | depth imaging mobile terminal |
Non-Patent Citations (1)
| Title |
|---|
| 周杰等: "《飞行时间深度相机和彩色相机的联合标定》", 《信号处理》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112700482A (en) * | 2019-10-23 | 2021-04-23 | 深圳市优必选科技股份有限公司 | Camera depth resolution determination method and device, storage medium and intelligent device |
| CN112700482B (en) * | 2019-10-23 | 2023-12-29 | 深圳市优必选科技股份有限公司 | Camera depth resolution determination method and device, storage medium and intelligent equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200293754A1 (en) | Task execution method, terminal device, and computer readable storage medium | |
| CN109389632A (en) | Depth calculation System and method for | |
| US8773512B1 (en) | Portable remote control device enabling three-dimensional user interaction with at least one appliance | |
| CN102970548B (en) | Image depth sensing device | |
| US9432593B2 (en) | Target object information acquisition method and electronic device | |
| CN208874676U (en) | A kind of image acquiring device, identity recognition device and electronic equipment | |
| CN106454287A (en) | Combined camera shooting system, mobile terminal and image processing method | |
| CN103810708B (en) | A kind of laser speckle image depth perception method and device | |
| WO2019080580A1 (en) | 3d face identity authentication method and apparatus | |
| US11972618B2 (en) | System and method for identifying items | |
| CN112005548B (en) | Method of generating depth information and electronic device supporting the same | |
| WO2020243969A1 (en) | Facial recognition apparatus and method, and electronic device | |
| CN207802143U (en) | depth imaging mobile terminal | |
| CN109389631A (en) | Depth calculation System and method for | |
| CN208782911U (en) | Lens assembly, sensing device and imaging system | |
| CN106875465B (en) | RGBD image-based three-dimensional control space establishment method and device | |
| CN113341396A (en) | Robot and charging pile identification method and device thereof | |
| CN110533710A (en) | A kind of method and processing unit of the binocular ranging algorithm based on GPU | |
| US20150278582A1 (en) | Image Processor Comprising Face Recognition System with Face Recognition Based on Two-Dimensional Grid Transform | |
| CN109274864A (en) | Depth camera, depth calculation System and method for | |
| CN106323190B (en) | The depth measurement method of customizable depth measurement range and the system of depth image | |
| CN103033145A (en) | Method and system for identifying shapes of plurality of objects | |
| CN107330930A (en) | Depth of 3 D picture information extracting method | |
| CN110136203A (en) | The scaling method and its calibration system of TOF device | |
| US20230007160A1 (en) | Photographing method of electronic apparatus and electronic apparatus thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190125 |