CN112802068A - Device for synchronously acquiring depth, color and spectrum images and identification system - Google Patents
Device for synchronously acquiring depth, color and spectrum images and identification system Download PDFInfo
- Publication number
- CN112802068A CN112802068A CN202011552870.1A CN202011552870A CN112802068A CN 112802068 A CN112802068 A CN 112802068A CN 202011552870 A CN202011552870 A CN 202011552870A CN 112802068 A CN112802068 A CN 112802068A
- Authority
- CN
- China
- Prior art keywords
- image data
- depth
- camera module
- color
- spectrum
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001228 spectrum Methods 0.000 title claims abstract description 63
- 238000012545 processing Methods 0.000 claims abstract description 25
- 230000003595 spectral effect Effects 0.000 claims description 91
- 239000000126 substance Substances 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 description 14
- 238000000034 method Methods 0.000 description 12
- 238000013519 translation Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
Abstract
The invention discloses a device for synchronously acquiring depth, color and spectrum images and an identification system, comprising a depth camera module, an RGB camera module, a spectrum camera module and a control processing module; the depth camera module is used for acquiring depth image data of a target area; the RGB camera module is used for acquiring color image data of a target area; the spectrum camera module is used for acquiring spectrum image data of a target area; the control processing module is used for synchronously controlling the data acquisition of the depth camera module, the RGB camera module and the spectrum camera module, processing the acquired depth image data, color image data and spectrum image data, and outputting the depth image data, the color image data and the spectrum image data which are aligned in a registering manner. The invention provides the combination of depth information, color information and spectrum information by adding the spectrum information, thereby improving the application safety and expanding the application scene of data.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a device for synchronously acquiring depth, color and spectrum images and an identification system.
Background
Depth data information has become a key data in artificial intelligence technology, and many applications can be realized by acquiring depth information of a target, such as: ranging, three-dimensional modeling, and enabling human-computer interaction, among others. With the development and application of the artificial intelligence related technology, the requirements of numerous applications are difficult to meet only by means of depth information, so that the combination of the depth information and the color information appears, and more applications of the depth camera can be enriched by means of the combination of the depth information and the color information. However, in some fields of application, such as: in application scenarios with high security requirements such as face-brushing payment and identity authentication, the material attribute of the target object cannot be judged only by the depth information and the color information, and the risk of being broken by the prosthesis exists, so that if the spectral information can be combined, the security of identification can be greatly improved.
At present, a depth camera based on a structured light scheme or a TOF technical scheme is generally used for obtaining depth information of a target, an RGB camera is used for obtaining color information of the target, a hyperspectral camera is used for obtaining spectral information, and the depth camera, the RGB camera and the hyperspectral camera are independently used for obtaining data information and are difficult to combine together for application. Although the spectral lidar can synchronously acquire depth and spectral information of a target object, such as: and determining the depth information of the target object according to a flight time method, and simultaneously performing spectral splitting on the received reflection signals to obtain spectral information of each waveband. However, spectral lidar is too bulky, expensive, low resolution, and cannot be made into consumer grade products.
The above background disclosure is only for the purpose of assisting understanding of the inventive concept and technical solutions of the present invention, and does not necessarily belong to the prior art of the present patent application, and should not be used for evaluating the novelty and inventive step of the present application in the case that there is no clear evidence that the above content is disclosed at the filing date of the present patent application.
Disclosure of Invention
The present invention is directed to an apparatus and a system for acquiring depth, color and spectrum images synchronously, so as to solve at least one of the above-mentioned problems.
In order to achieve the above purpose, the technical solution of the embodiment of the present invention is realized as follows:
a device for synchronously acquiring depth, color and spectrum images comprises a depth camera module, an RGB camera module, a spectrum camera module and a control processing module; wherein the content of the first and second substances,
the depth camera module is used for acquiring depth image data of a target area;
the RGB camera module is used for acquiring color image data of a target area;
the spectrum camera module is used for acquiring spectrum image data of a target area;
the control processing module is used for synchronously controlling the data acquisition of the depth camera module, the RGB camera module and the spectrum camera module, processing the acquired depth image data, the color image data and the spectrum image data, and outputting the depth image data, the color image data and the spectrum image data which are aligned in a registering manner.
In some embodiments, the control processing module is to register the depth image data with the color image data.
In some embodiments, the control processing module is further configured to register the color image data with the spectral image data.
In some embodiments, the control processing module is configured to register the depth image data with the spectral image data using the color image data to establish a relationship between the depth image data and the spectral image data based on the registration of the depth image data with the color image data and the registration of the color image data with the spectral image data.
In some embodiments, the depth camera module is a structured light depth camera having an RGB camera module.
In some embodiments, the depth camera module and the spectral camera module are separately configured to acquire the depth image data and the hyperspectral image data, respectively.
In some embodiments, the spectral camera module includes a lens, a mount, and a spectral chip.
In some embodiments, the spectral camera module is configured to output 8 bands of spectral images; wherein the spectral image resolution is 640 x 480.
The technical scheme of the other embodiment of the invention is as follows:
a recognition system for synchronously acquiring depth, color and spectral images comprises a device for synchronously acquiring depth, color and spectral images and a face recognition device, which are described in the technical scheme of any one of the embodiments; the device for synchronously acquiring the depth image, the color image and the spectral image outputs the depth image data, the RGB image data and the spectral image data which are aligned in a registering way to the face recognition device, and the face recognition device carries out face recognition according to the received image data and outputs a recognition result.
In some embodiments, the face recognition device includes a depth face recognition model, an RGB face recognition model, and a spectral face recognition model; the depth face recognition model is used for carrying out face recognition on the input depth image data; the RGB face recognition model is used for carrying out face recognition on the input color image data; the spectrum face recognition model is used for carrying out face recognition on the input spectrum image data.
The technical scheme of the invention has the beneficial effects that:
compared with the prior art, the depth image data, the RGB image data and the spectral image data are registered and aligned by the aid of the pre-calibrated calibration information through adding the spectral camera module and synchronously controlling data acquisition of the depth camera module, the RGB camera module and the spectral camera module through the control processing module, and finally the registered and aligned depth image data, RGB image data and spectral image data are output. By adding the spectrum information, the combination of the depth information, the color information and the spectrum information is provided, so that the application safety is improved, and the application scene of the data can be expanded.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of an apparatus for simultaneously acquiring depth, color and spectral images in accordance with one embodiment of the present invention;
FIG. 2 is a flowchart illustrating an apparatus for synchronously acquiring depth, color and spectral images according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a system for simultaneous acquisition of depth, color and spectral image recognition according to another embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the embodiments of the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element. The connection may be for fixation or for circuit connection.
Referring to fig. 1, fig. 1 is a schematic diagram of an apparatus for synchronously acquiring depth information, color information and spectral information according to an embodiment of the present invention, where the apparatus 100 includes a depth camera module, an RGB camera module, a spectral camera module and a control processing module; the depth camera module is used for acquiring depth image data of a target area; the RGB camera module is used for acquiring color image data of a target area; the spectrum camera module is used for acquiring spectrum image data of a target area; the control processing module is used for synchronously controlling the data acquisition of the depth module, the RGB camera module and the spectrum camera module, processing the acquired depth image data, color image data and spectrum image data, and outputting the depth image data, the color image data and the spectrum image data which are aligned in a registering manner.
In one embodiment, the depth camera module may be a depth camera based on structured light, binocular, TOF (time of flight algorithm), etc. solutions. Taking the depth camera as a structured light depth camera for example, in general, the structured light depth camera includes a transmitting module and a receiving module. In one embodiment, the structured light pattern emitted by the emitting module is an infrared speckle image, the receiving module is an infrared camera, the structured light pattern is collected by the infrared camera and then output to the control processing module, and the control processing module calculates the structured light pattern to obtain the depth image data of the target object.
In one embodiment, the depth camera module is a structured light depth camera having an RGB camera module, thereby integrating the RGB camera module with the depth camera module.
In one embodiment, the depth camera module and the spectrum camera module may be configured on the same image sensor array to acquire depth image data and hyperspectral image data; wherein the image sensor array comprises a filter, optionally arranged above the image sensor array so as to optically cover the image sensor array. The optical filter selectively transmits the light beams with specific wave bands and prevents the light outside the specific wave bands from reaching the image sensor array; it should be noted that the blocked light may be absorbed, reflected and/or scattered, depending on the implementation of the filter, and is not limited herein.
In one filtering state, if the depth image is generated based on infrared light, then the filter may project infrared light and block light outside the infrared band; in another filtering state, where the spectral image is based on generation using a different wavelength band than the depth image, the filter may transmit light beams in a wavelength band contained in the hyperspectral image and block light beams outside the wavelength band. It will be appreciated that the spectral image contains a plurality of wavelength bands of light and the filter may be configured to switch between a plurality of filter states for different wavelength bands of light. It will be appreciated that the filter may be switched between any suitable number of different filter states to transmit light beams of any suitable wavelength band while blocking light beams outside of that wavelength band.
In one embodiment, the depth camera module and the spectral camera module are separately configured to acquire depth image data and hyperspectral image data, respectively.
In one embodiment, the spectral camera module acquires spectral image data, which does not define a particular spectral acquisition scheme; in some embodiments, the spectrum camera module may be a micro imaging spectrometer, a single-point spectrum detection device, a multi-channel/multi-spectrum imaging sensor, and the like, and the light splitting technology may be various schemes such as prism light splitting, grating light splitting, filter light splitting, and the like.
In one embodiment, a spectral camera module includes a lens, a holder (holder), and a spectral chip; the spectrum chip is divided into a group according to every 3 x 3 pixels based on a CMOS sensor, optical filters with different spectrum transmission characteristics are plated on different pixels in each group, original data are firstly obtained through the CMOS sensor, and spectrum image data are obtained through calculation according to the original data. In an embodiment of the present invention, the spectral camera module preferably outputs 8 wavelength band spectral images, each spectral image having a resolution of 640 × 480.
Specifically, a depth camera module, an RGB camera module and a spectrum camera module are respectively used to shoot a specific target image, an external parameter [ R | T ] (a rotation matrix and a translation matrix) between an internal parameter K (a camera calibration matrix) and any one of three camera modules is obtained by a camera calibration method, and an illumination light source corresponding to the target image covers corresponding working wave bands of the three camera modules. Wherein the camera scaling matrix K is:
fxand fyThe focal length in the X, Y direction is shown.
The rotation matrix R is:
α, β, γ are euler triangles: yaw, Pitch, Roll.
Registering the depth image data and the color image (also called RGB) data by a control processing module, specifically, selecting any point [ u ] on the depth imaged,vd]The corresponding point [ u ] on the RGB map can be obtained by the following equationr,vr]Wherein:
Kdis an internal reference of the depth camera module; krFor internal reference of the RGB camera module, the rotation matrix and translation vector from the depth camera module to the RGB camera module are respectively recordedIs Rd2rAnd Td2r；ddIs the point [ u ] in the depth mapd,vd]Corresponding depth values, obtainable from the depth map; drFor the corresponding point [ u ] in the RGB mapr,vr]The depth value of (2). According to the formula (1), the original depth map is traversed, and a depth map corresponding to the RGB image pixel by pixel is obtained through calculation, so that the registration of the depth image data and the RGB image data is realized.
The RGB image data and the spectral image data are registered through the control processing module, after the registration of the depth image data and the RGB image data is completed, any point [ u ] on the RGB image is registeredr,vr]Obtaining the corresponding depth value, and establishing the coordinate corresponding relation between the RGB image and the spectrum image according to the following formula, wherein:
Ksis an internal reference of the spectral camera module, KrIs an internal reference of the RGB camera module; the rotation matrix and translation vector from the RGB camera module to the spectrum camera module are respectively marked as Rr2sAnd Tr2s；drFor the point [ u ] in the RGB imager,vr]A corresponding depth value provided by the depth image registered with the RGB image; dsFor corresponding points [ u ] in the spectral camera modules,vs]The depth value of (2). For any point in the RGB image, the corresponding coordinate of the point in the spectral image can be calculated according to the formula (2), and the RGB image is traversed, so that the registration of the RGB image data and the spectral image data is realized.
The depth image data and the RGB image data are registered, the RGB image data and the spectral image data are registered, the relationship between the depth image and the spectral image is established by the RGB image, and the depth image data and the spectral image data can be aligned and registered, so that the registration of the depth image data, the RGB image data and the spectral image data is completed, the three kinds of registered data are output, and the synchronous acquisition of the three kinds of data is realized.
Specifically, a pairAt any point in the RGB image, its color components [ r, g, b ] are obtained]Obtaining the depth value d corresponding to the point by using the registered depth map, calculating the corresponding coordinate in the spectral image according to the formula (2), and if the spectral image contains k wave bands, extracting the component [ s ] of each wave band1,s2,...,sk]. Therefore, for an RGB image with a resolution of m × n, if the spectral camera module includes k wavebands, a matrix of m × n (1+3+ k) dimensions is output after registration, and the matrix includes registered depth image data, RGB image data, and spectral image data.
According to the invention, by adding the spectrum camera module, the control processing module synchronously controls the depth camera module, the RGB camera module and the spectrum camera module to acquire data, the pre-calibrated calibration information is utilized to realize the registration and alignment of the depth image data, the RGB image data and the spectrum image data, and finally the registered and aligned depth image data, RGB image data and spectrum image data of each wave band are output. By adding the spectrum information, the combination of the depth information, the color information and the spectrum information is provided, so that the application safety is improved, and the application scene of the data can be expanded.
Fig. 2 is a flowchart illustrating an implementation method of an apparatus for synchronously acquiring depth, color and spectrum images according to an embodiment of the present invention, the method including the following steps:
s20, the depth image data, the color image (RGB image) data and the spectral image data of the target area are collected by the depth camera module, the RGB camera module and the spectral camera module;
s21, registering the depth image data and the color image data;
specifically, any point [ u ] on the depth map is selectedd,vd]The corresponding point [ u ] on the RGB image can be obtained by the following formular,vr]Wherein:
Kdis an internal reference of the depth camera module; krFor the internal reference of the RGB camera module, the rotation matrix and the translation vector from the depth camera module to the RGB camera module are respectively marked as Rd2rAnd Td2r；ddIs the point [ u ] in the depth mapd,vd]Corresponding depth values, obtainable from the depth map data; drFor the corresponding point [ u ] in the RGB mapr,vr]The depth value of (2). According to the formula (1), the original depth map is traversed, and a depth map corresponding to the RGB image pixel by pixel is obtained through calculation, so that the registration of the depth image data and the color image data is realized.
S22, registering the color image data and the spectral image data;
specifically, according to step S21, for any point [ u ] on the RGB imager,vr]Obtaining the corresponding depth value, and establishing the coordinate corresponding relation between the RGB image and the spectrum image according to the following formula, wherein:
Ksis an internal reference of the spectral camera module, KrIs an internal reference of the RGB camera module; the rotation matrix and translation vector from the RGB camera module to the spectrum camera module are respectively marked as Rr2sAnd Tr2s；drFor the point [ u ] in the RGB imager,vr]A corresponding depth value provided by the depth image after registration with the color image; dsFor corresponding points [ u ] in the spectral camera modules,vs]The depth value of (2).
And S23, establishing a relation between the depth image data and the spectral image data based on the color image data, completing registration of the depth image data and the spectral image data, and outputting the registered depth image data, color image data and spectral image data.
In some embodiments, before acquiring image data of a target area, a depth camera module, an RGB camera module, and a spectrum camera module are used to capture a specific image target, and an external parameter [ R | T ] (rotation matrix and translation matrix) between an internal parameter K (camera calibration matrix) and any one of the three camera modules is obtained by a camera calibration method. And the illumination light source corresponding to the target image covers the corresponding working wave bands of the three camera modules.
Fig. 3 is a schematic diagram of a recognition system for synchronously acquiring depth, color and spectral images according to another embodiment of the present application, the system includes the device 100 for synchronously acquiring depth, color and spectral images and the face recognition device 300 according to the above-mentioned embodiment; the device 100 for synchronously acquiring depth, color and spectral images is used for outputting the depth image data, the RGB image data and the spectral image data after registration and alignment to the face recognition device 300, and the face recognition device 300 performs face recognition according to the received image data and outputs a recognition result.
In some embodiments, the face recognition apparatus 300 includes a depth face recognition model 301, an RGB face recognition model 302, and a spectral face recognition model 303; the depth face recognition model 301 is used for performing face recognition on input depth image data; the RGB face recognition model 302 is used to perform face recognition on the input color image data; the spectral face recognition model 303 is used to perform face recognition on the input spectral image data.
The identification system provided by the embodiment of the invention increases the spectral information and provides the combination of the depth information, the color information and the spectral information, thereby improving the safety of system application.
Embodiments of the present invention further provide a storage medium for storing a computer program, which when executed performs at least the method for synchronously acquiring depth information, color information and spectral image described in any of the above embodiments.
Embodiments of the present invention may comprise or utilize a special purpose or general-purpose computer including computer hardware, as discussed in greater detail below. Embodiments within the scope of the present invention also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. The computer-readable medium storing the computer-executable instructions is a physical storage medium. Computer-readable media carrying computer-executable instructions are transmission media. Thus, by way of example, and not limitation, embodiments of the invention can include at least two distinct computer-readable media: physical computer-readable storage media and transmission computer-readable media.
It is to be understood that the foregoing is a more detailed description of the invention, and that specific embodiments are not to be considered as limiting the invention. It will be apparent to those skilled in the art that various substitutions and modifications can be made to the described embodiments without departing from the spirit of the invention, and these substitutions and modifications should be considered to fall within the scope of the invention. In the description herein, references to the description of the term "one embodiment," "some embodiments," "preferred embodiments," "an example," "a specific example," or "some examples" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention.
In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. Although embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope of the invention as defined by the appended claims.
Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. One of ordinary skill in the art will readily appreciate that the above-disclosed, presently existing or later to be developed, processes, machines, manufacture, compositions of matter, means, methods, or steps, that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.
Claims (10)
1. An apparatus for simultaneously acquiring depth, color and spectral images, comprising: the system comprises a depth camera module, an RGB camera module, a spectrum camera module and a control processing module; wherein the content of the first and second substances,
the depth camera module is used for acquiring depth image data of a target area;
the RGB camera module is used for acquiring color image data of a target area;
the spectrum camera module is used for acquiring spectrum image data of a target area;
the control processing module is used for synchronously controlling the data acquisition of the depth camera module, the RGB camera module and the spectrum camera module, processing the acquired depth image data, the color image data and the spectrum image data, and outputting the depth image data, the color image data and the spectrum image data which are aligned in a registering manner.
2. The apparatus for simultaneously acquiring depth, color and spectral images of claim 1, wherein: the control processing module is used for registering the depth image data and the color image data.
3. The apparatus for simultaneously acquiring depth, color and spectral images of claim 2, wherein: the control processing module is further configured to register the color image data with the spectral image data.
4. The apparatus for simultaneously acquiring depth, color and spectral images of claim 3, wherein: the control processing module is configured to establish a relationship between the depth image data and the spectral image data by using the color image data according to the registration of the depth image data and the color image data and the registration of the color image data and the spectral image data, and to register the depth image data and the spectral image data.
5. The apparatus for simultaneous depth, color and spectral image acquisition according to any one of claims 1-3, wherein: the depth camera module is a structured light depth camera with an RGB camera module.
6. The apparatus for simultaneous depth, color and spectral image acquisition according to any one of claims 1-3, wherein: the depth camera module and the spectrum camera module are arranged separately to acquire the depth image data and the hyperspectral image data respectively.
7. The apparatus for simultaneous depth, color and spectral image acquisition according to any one of claims 1-3, wherein: the spectrum camera module comprises a lens, a bracket and a spectrum chip.
8. The apparatus for simultaneous depth, color and spectral image acquisition according to any one of claims 1-3, wherein: the spectral camera module is configured to output 8 bands of spectral images; wherein the resolution of the spectral image is 640 x 480.
9. An identification system for synchronously acquiring depth, color and spectrum images, characterized in that: comprising the device for synchronously acquiring depth, color and spectrum images and the face recognition device of any one of claims 1-3; the device for synchronously acquiring the depth image, the color image and the spectral image outputs the depth image data, the RGB image data and the spectral image data which are aligned in a registering way to the face recognition device, and the face recognition device carries out face recognition according to the received image data and outputs a recognition result.
10. An identification system for simultaneously acquiring depth, color and spectral images as defined in claim 9, wherein: the face recognition device comprises a depth face recognition model, an RGB face recognition model and a spectrum face recognition model; the depth face recognition model is used for carrying out face recognition on the input depth image data; the RGB face recognition model is used for carrying out face recognition on the input color image data; the spectrum face recognition model is used for carrying out face recognition on the input spectrum image data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011552870.1A CN112802068A (en) | 2020-12-24 | 2020-12-24 | Device for synchronously acquiring depth, color and spectrum images and identification system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011552870.1A CN112802068A (en) | 2020-12-24 | 2020-12-24 | Device for synchronously acquiring depth, color and spectrum images and identification system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112802068A true CN112802068A (en) | 2021-05-14 |
Family
ID=75804193
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011552870.1A Pending CN112802068A (en) | 2020-12-24 | 2020-12-24 | Device for synchronously acquiring depth, color and spectrum images and identification system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112802068A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113687369A (en) * | 2021-07-14 | 2021-11-23 | 南京大学 | Synchronous acquisition system and method for spectral information and depth information |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107105217A (en) * | 2017-04-17 | 2017-08-29 | 深圳奥比中光科技有限公司 | Multi-mode depth calculation processor and 3D rendering equipment |
| CN107103626A (en) * | 2017-02-17 | 2017-08-29 | 杭州电子科技大学 | A kind of scene reconstruction method based on smart mobile phone |
| CN109684924A (en) * | 2018-11-21 | 2019-04-26 | 深圳奥比中光科技有限公司 | Human face in-vivo detection method and equipment |
| CN109814718A (en) * | 2019-01-30 | 2019-05-28 | 天津大学 | A kind of multi-modal information acquisition system based on Kinect V2 |
| US20190295271A1 (en) * | 2017-04-17 | 2019-09-26 | Shenzhen Orbbec Co., Ltd. | Depth calculation processor, data processing method and 3d image device |
| US10452947B1 (en) * | 2018-06-08 | 2019-10-22 | Microsoft Technology Licensing, Llc | Object recognition using depth and multi-spectral camera |
| CN111091063A (en) * | 2019-11-20 | 2020-05-01 | 北京迈格威科技有限公司 | Living body detection method, device and system |
| CN111739031A (en) * | 2020-06-19 | 2020-10-02 | 华南农业大学 | Crop canopy segmentation method based on depth information |
| CN111767788A (en) * | 2020-05-12 | 2020-10-13 | 贵阳像树岭科技有限公司 | Non-interactive monocular in vivo detection method |
| CN112016495A (en) * | 2020-09-03 | 2020-12-01 | 福建库克智能科技有限公司 | Face recognition method and device and electronic equipment |
| CN112036500A (en) * | 2020-09-04 | 2020-12-04 | 广东弓叶科技有限公司 | Garbage classification method and device based on depth image and multispectral image fusion |
| CN112052830A (en) * | 2020-09-25 | 2020-12-08 | 北京百度网讯科技有限公司 | Face detection method, device and computer storage medium |
-
2020
- 2020-12-24 CN CN202011552870.1A patent/CN112802068A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103626A (en) * | 2017-02-17 | 2017-08-29 | 杭州电子科技大学 | A kind of scene reconstruction method based on smart mobile phone |
| CN107105217A (en) * | 2017-04-17 | 2017-08-29 | 深圳奥比中光科技有限公司 | Multi-mode depth calculation processor and 3D rendering equipment |
| US20190295271A1 (en) * | 2017-04-17 | 2019-09-26 | Shenzhen Orbbec Co., Ltd. | Depth calculation processor, data processing method and 3d image device |
| US10452947B1 (en) * | 2018-06-08 | 2019-10-22 | Microsoft Technology Licensing, Llc | Object recognition using depth and multi-spectral camera |
| CN109684924A (en) * | 2018-11-21 | 2019-04-26 | 深圳奥比中光科技有限公司 | Human face in-vivo detection method and equipment |
| CN109814718A (en) * | 2019-01-30 | 2019-05-28 | 天津大学 | A kind of multi-modal information acquisition system based on Kinect V2 |
| CN111091063A (en) * | 2019-11-20 | 2020-05-01 | 北京迈格威科技有限公司 | Living body detection method, device and system |
| CN111767788A (en) * | 2020-05-12 | 2020-10-13 | 贵阳像树岭科技有限公司 | Non-interactive monocular in vivo detection method |
| CN111739031A (en) * | 2020-06-19 | 2020-10-02 | 华南农业大学 | Crop canopy segmentation method based on depth information |
| CN112016495A (en) * | 2020-09-03 | 2020-12-01 | 福建库克智能科技有限公司 | Face recognition method and device and electronic equipment |
| CN112036500A (en) * | 2020-09-04 | 2020-12-04 | 广东弓叶科技有限公司 | Garbage classification method and device based on depth image and multispectral image fusion |
| CN112052830A (en) * | 2020-09-25 | 2020-12-08 | 北京百度网讯科技有限公司 | Face detection method, device and computer storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113687369A (en) * | 2021-07-14 | 2021-11-23 | 南京大学 | Synchronous acquisition system and method for spectral information and depth information |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11145071B2 (en) | Image processing method, non-transitory computer-readable storage medium, and electronic apparatus | |
| EP2721828B1 (en) | High resolution multispectral image capture | |
| EP3614659B1 (en) | Image processing method, electronic apparatus, and computer-readable storage medium | |
| US7477361B2 (en) | Range image producing method and apparatus for image taking | |
| CN111462128B (en) | Pixel-level image segmentation system and method based on multi-mode spectrum image | |
| US8199228B2 (en) | Method of and apparatus for correcting contour of grayscale image | |
| CN112288663A (en) | Infrared and visible light image fusion method and system | |
| CN102647941A (en) | Method and system for carrying out photoplethysmography | |
| CN113676628B (en) | Image forming apparatus and image processing method | |
| CN103201602A (en) | Digital multi-spectral camera system having at least two independent digital cameras | |
| CN106572339B (en) | A kind of image acquisition device and image capturing system | |
| CN109087395B (en) | Three-dimensional reconstruction method and system | |
| CN113534596B (en) | RGBD stereo camera and imaging method | |
| CN112672054A (en) | Focusing method and device and electronic equipment | |
| JP2006322795A (en) | Image processing device, image processing method and image processing program | |
| CN112802068A (en) | Device for synchronously acquiring depth, color and spectrum images and identification system | |
| US20220103797A1 (en) | Integrated Spatial Phase Imaging | |
| KR101994473B1 (en) | Method, apparatus and program sotred in recording medium for refocucing of planar image | |
| CN111344711A (en) | Image acquisition method and device | |
| JP2013024653A (en) | Distance measuring apparatus and program | |
| CN111272101A (en) | Four-dimensional hyperspectral depth imaging system | |
| CN112672021B (en) | Language identification method and device and electronic equipment | |
| CN112750157B (en) | Depth image generation method and device | |
| CN114529488A (en) | Image fusion method, device and equipment and storage medium | |
| CN109447925B (en) | Image processing method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |