CN115035234A - Depth reconstruction method, system, device and storage medium - Google Patents
Depth reconstruction method, system, device and storage medium Download PDFInfo
- Publication number
- CN115035234A CN115035234A CN202110239717.1A CN202110239717A CN115035234A CN 115035234 A CN115035234 A CN 115035234A CN 202110239717 A CN202110239717 A CN 202110239717A CN 115035234 A CN115035234 A CN 115035234A
- Authority
- CN
- China
- Prior art keywords
- infrared
- images
- image
- structured light
- frame frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a depth reconstruction method, a system, equipment and a storage medium, comprising the following steps: acquiring preset frame frequency values, and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency values; subtracting the gray values of corresponding pixels of the background image and the infrared structured light image to generate a target structured light image; and performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image. According to the depth camera and the method, the depth reconstruction result of the detected target under the condition of strong background light can be improved, the acquisition time interval between two adjacent frames of images can be shortened, the attack difficulty of the depth camera is improved, and the safety of the depth camera is higher.
Description
Technical Field
The present invention relates to structured light three-dimensional reconstruction, and in particular, to a depth reconstruction method, system, device, and storage medium.
Background
Mobile payment has become a mainstream payment mode in China, and by the fact that the number of mobile payment users reaches 10.5 billion at present, the annual mobile payment transaction amount reaches 30 trillion yuan, which becomes the financial support force; the mobile payment is spread in smart phones, with the 4G/5G, artificial intelligence, big data and biometric identification technology tend to be mature, a face-brushing payment original year representing payment of 4.0 is met by 2019, a commercial scene of the face-brushing payment terminal is gradually expanded on line, large-scale use is about to be met, and the market scale is expected to reach 185 trillion yuan in 2022.
As the core device of the face-brushing payment terminal, the face recognition camera module plays a very key role. At present, a relatively mature face recognition camera module adopts a structured light scheme.
The structured light three scheme is based on the optical triangulation method measuring principle. The optical projector projects the structured light with a certain mode on the surface of the object to form a light bar three-dimensional image modulated by the surface shape of the object to be measured on the surface. The three-dimensional image is detected by a camera at another location to obtain a two-dimensional distorted image of the light bars. The degree of distortion of the light bar depends on the relative position between the optical projector and the camera and the object surface profile (height). Intuitively, the displacement (or offset) displayed along the bar is proportional to the height of the object surface, the kink indicates a change in plane, and the discontinuity indicates a physical gap in the surface. When the relative position between the optical projector and the camera is fixed, the three-dimensional profile of the object surface can be reproduced by the distorted two-dimensional light bar image coordinates.
The degree of front end perception has been widened to the degree of depth camera module, the anti false body that solution 2D face identification met that can be fine attacks and the problem that under the extreme condition discernment rate of accuracy reduces, the effect has obtained market's recognition, and the demand is strong, can be applied to scenes such as lock, entrance guard and payment based on 3D face identification. When the human face needs to be recognized, not only the depth image of the target but also the gray image is needed, and the general gray image is divided into two types, one is the gray image shot by the same camera or the gray image shot by different cameras or both. In depth reconstruction applications, the acquired texture image contains a texture pattern projected by a projector, which corresponds to a valid signal, and background light, which corresponds to noise interference. In some cases, such as when the illumination intensity is strong, the background light ratio is strong, the interference is severe, and the signal-to-noise ratio is low.
Disclosure of Invention
In view of the defects in the prior art, the present invention provides a depth reconstruction method, system, device and storage medium.
The depth reconstruction method provided by the invention comprises the following steps
Step S1: acquiring preset frame frequency values, and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency values;
step S2: subtracting the gray values of corresponding pixels of the background image and the infrared structured light image to generate a target structured light image;
step S3: and performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
Preferably, the step S1 includes the steps of:
step S101: acquiring a preset frame frequency threshold value and an original frame frequency value, wherein the original frame frequency value is a frame frequency value of an infrared camera for acquiring an infrared structured light image and an infrared image;
step S102: determining a multiple value between the original frame frequency value and the frame frequency threshold, when the multiple value is less than or equal to a preset multiple threshold, determining that the preset frame frequency value is the product of the multiple threshold and the frame frequency threshold, and when the multiple value is greater than or equal to the preset multiple threshold, determining that the preset frame frequency value is the original frame frequency value;
step S103: and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target in a frame frequency threshold sampling period or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value.
Preferably, the step S103 includes the steps of:
step S1031: projecting structured light and floodlight through a light projector of a depth camera toward a target;
step S1032: when the multiple value is less than or equal to a preset multiple threshold value, continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target in a frame frequency threshold value sampling period or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value;
step S1033: and when the multiple value is larger than a preset multiple threshold value, sequentially acquiring background images, infrared structured light images and infrared images of the same target in any three continuous frames in each sampling period determined according to the frame frequency threshold value according to the frame frequency value, or sequentially acquiring the infrared images, the infrared structured light images and the background images of the same target.
Preferably, the step S2 includes the steps of:
step S201: determining a pixel value of each pixel in the background image and the infrared structured light image;
step S202: performing pixel level alignment on the background image and the infrared structured light image;
step S203: and subtracting the gray value of each pixel in the infrared structured light image from the gray value of the corresponding pixel in the background image to generate a target structured light image.
Preferably, the step S3 includes the steps of:
step S301: calculating the target structured light image and known calibration information to obtain a parallax image of the target structured light image;
step S302: determining the distance between the optical center of the infrared camera and each parallax value in the parallax map according to a triangulation principle, and generating depth information of each pixel;
step S303: and performing depth reconstruction or three-dimensional reconstruction according to the depth information of each pixel to generate a depth image.
Preferably, the infrared structured light image is an image containing encoded texture, and includes any one of the following structured light images:
-a speckle image;
-a fringe image;
-encoding the image;
-a raster image.
Preferably, the preset frame rate threshold is 15FPS, and the preset multiple threshold is 3.
The depth reconstruction system provided by the invention is used for realizing the depth reconstruction method, and comprises the following steps:
the image acquisition module is used for acquiring preset frame frequency values and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target according to the frame frequency values or continuously and sequentially acquiring the infrared images, the infrared structured light images and the background images of the same target;
the image enhancement module is used for generating a target structured light image according to the subtraction of the gray values of the corresponding pixels of the background image and the infrared structured light image;
and the depth reconstruction module is used for performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
According to the present invention, there is provided a depth reconstruction apparatus comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the depth reconstruction method via execution of the executable instructions.
According to the present invention, there is provided a computer-readable storage medium for storing a program which, when executed, implements the steps of the depth reconstruction method.
Compared with the prior art, the invention has the following beneficial effects:
according to the method, the background image, the infrared structured light image and the infrared image of the same target are continuously and sequentially acquired according to the preset frame frequency value, or the infrared image, the infrared structured light image and the background image of the same target are continuously and sequentially acquired, so that the continuous acquisition of three frames of images is realized, the acquisition time interval between two adjacent frames of images is shortened, the attack difficulty of the depth camera is improved, and the safety of the depth camera is higher;
according to the invention, the target structured light image is generated by subtracting the gray values of the corresponding pixels of the background image and the infrared structured light image, and then the depth image is generated by performing depth reconstruction or three-dimensional reconstruction according to the target structured light image, so that the interference of background light is reduced, and the depth camera can be suitable for the environment with strong illumination intensity.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts. Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flowchart illustrating steps of a depth reconstruction method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the steps of determining a frame rate value according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating the steps of acquiring images based on frame rate values according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating steps for generating a light image of a target structure according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating steps of performing depth reconstruction to generate a depth image according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a captured image of a depth camera in an embodiment of the invention;
FIG. 7 is a block diagram of a depth reconstruction system according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a depth reconstruction apparatus according to an embodiment of the present invention; and
fig. 9 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical means of the present invention will be described in detail with reference to specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The invention provides a depth reconstruction method, and aims to solve the problems in the prior art.
The following describes the technical solutions of the present invention and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
Fig. 1 is a flowchart illustrating steps of a depth reconstruction method according to an embodiment of the present invention, and as shown in fig. 1, the depth reconstruction method provided by the present invention includes the following steps
Step S1: acquiring preset frame frequency values, and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency values;
fig. 2 is a flowchart illustrating steps of determining a frame rate value according to an embodiment of the present invention, and as shown in fig. 2, the step S1 includes the following steps:
step S101: acquiring a preset frame frequency threshold value and an original frame frequency value, wherein the original frame frequency value is a frame frequency value of an infrared camera for acquiring an infrared structured light image and an infrared image;
step S102: determining a multiple value between the original frame frequency value and the frame frequency threshold, when the multiple value is less than or equal to a preset multiple threshold, determining that the preset frame frequency value is the product of the multiple threshold and the frame frequency threshold, and when the multiple value is greater than or equal to the preset multiple threshold, determining that the preset frame frequency value is the original frame frequency value;
step S103: and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target within a frame frequency threshold value sampling period according to the frame frequency value, or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target.
In this embodiment of the present invention, the preset frame frequency threshold is 15FPS, the preset multiple threshold is 3, when the original frame frequency value is 30FPS, the multiple value is 2, and since the multiple value 2 is smaller than the preset number threshold 3, the preset frame frequency value is determined to be the preset number threshold 3 multiplied by the preset frame frequency threshold 15, and the frame frequency value is determined to be 45; and when the original frame frequency value is 60FPS, the multiple value is 4, and the preset frame frequency value is determined to be 60FPS because the multiple value 4 is greater than a preset number threshold value 3.
Fig. 3 is a flowchart illustrating steps of acquiring an image according to a frame rate according to an embodiment of the present invention, and as shown in fig. 3, the step S103 includes the following steps:
step S1031: projecting structured light and floodlight through a light projector of a depth camera toward a target;
step S1032: when the multiple value is less than or equal to a preset multiple threshold value, continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target in a frame frequency threshold value sampling period or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value;
step S1033: and when the multiple value is larger than a preset multiple threshold value, sequentially acquiring background images, infrared structured light images and infrared images of the same target in any three continuous frames in each sampling period determined according to the frame frequency threshold value according to the frame frequency value, or sequentially acquiring the infrared images, the infrared structured light images and the background images of the same target.
In the embodiment of the present invention, when the multiple value is 2 and the multiple value is less than or equal to a preset multiple threshold, continuously and sequentially acquiring background images, infrared structured light images, and infrared images of the same target within 15 sampling periods of the frame frequency threshold according to the frame frequency value 45, or continuously and sequentially acquiring infrared images, infrared structured light images, and background images of the same target;
and analogizing in sequence, for example, when the multiple value is 4, and at this time, when the multiple value is greater than a preset multiple threshold, continuously and sequentially acquiring a background image, an infrared structured light image, and an infrared image of the same target every 4 frames in 15 sampling periods of the frame frequency threshold according to the frame frequency value 60, or continuously and sequentially acquiring an infrared image, an infrared structured light image, and a background image of the same target.
In an embodiment of the present invention, the infrared structured light image is an image including a coding texture, and includes any one of the following structured light images:
-a speckle image;
-a fringe image;
-encoding the image;
-a raster image.
Step S2: subtracting the gray values of corresponding pixels of the background image and the infrared structured light image to generate a target structured light image;
fig. 4 is a flowchart of the steps of generating the target structured light image according to the embodiment of the present invention, and as shown in fig. 4, the step S2 includes the following steps:
step S201: determining a gray value of each pixel in the background image and the infrared structured light image;
step S202: performing pixel level alignment on the background image and the infrared structured light image;
step S203: and subtracting the gray value of each pixel in the infrared structured light image from the gray value of the corresponding pixel in the background image to generate a target structured light image.
Step S3: and performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
Fig. 5 is a flowchart of a step of generating a depth image by performing depth reconstruction according to an embodiment of the present invention, and as shown in fig. 5, the step S3 includes the following steps:
step S301: calculating the target structured light image and known calibration information to obtain a parallax image of the target structured light image;
step S302: determining the distance between the optical center of the infrared camera and each parallax value in the parallax map according to a triangulation principle, and generating depth information of each pixel;
step S303: and performing depth reconstruction or three-dimensional reconstruction according to the depth information of each pixel to generate a depth image. Fig. 6 is a schematic diagram of an image collected by a depth camera in an embodiment of the present invention, as shown in fig. 6, when the depth camera provided by the present invention is used, a background image may be collected by an infrared camera first, then an infrared structured light image may be collected after structured light is projected toward a target by a structured light projector, and finally an infrared image may be collected after floodlight is projected toward the target by a floodlight projector; the method can also be used for collecting a background image through an infrared camera, collecting an infrared structural light image after projecting structural light towards a target through a structural light projector, and collecting an infrared image after projecting floodlight towards the target through a floodlight projector. The infrared camera adopts a 940nm infrared camera. The floodlight projector adopts an LED light source.
Fig. 7 is a schematic block diagram of a depth reconstruction system according to an embodiment of the present invention, and as shown in fig. 7, the depth reconstruction system provided in the present invention is configured to implement the depth reconstruction method, and includes:
the image acquisition module is used for acquiring preset frame frequency values and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency values;
the image enhancement module is used for generating a target structured light image according to the subtraction of the gray values of the corresponding pixels of the background image and the infrared structured light image;
and the depth reconstruction module is used for performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
The embodiment of the invention also provides depth reconstruction equipment which comprises a processor. A memory having stored therein executable instructions of the processor. Wherein the processor is configured to perform the steps of the depth reconstruction method via execution of executable instructions.
As described above, according to the embodiment, the background image, the infrared structured light image, and the infrared image of the same target can be continuously and sequentially acquired according to the preset frame frequency value, or the infrared image, the infrared structured light image, and the background image of the same target can be continuously and sequentially acquired, so that the continuous acquisition of three frames of images is realized, the time interval of acquisition between two adjacent frames of images is shortened, the difficulty of attacking the depth camera is improved, and the security of the depth camera is higher.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" platform.
Fig. 8 is a schematic structural diagram of a depth reconstruction apparatus in an embodiment of the present invention. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 600 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores a program code, which can be executed by the processing unit 610, such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above-mentioned deep reconstruction method section of the present specification. For example, processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in FIG. 8, other hardware and/or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
The embodiment of the invention also provides a computer readable storage medium for storing a program, and the program realizes the steps of the depth reconstruction method when being executed. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention described in the above-mentioned deep reconstruction method section of the present description, when the program product is run on the terminal device.
As shown above, when the program of the computer-readable storage medium of this embodiment is executed, the background image, the infrared structured light image, and the infrared image of the same target are continuously and sequentially acquired according to the preset frame frequency value, or the infrared image, the infrared structured light image, and the background image of the same target are continuously and sequentially acquired, so that the continuous acquisition of three frames of images is realized, the acquisition time interval between two adjacent frames of images is shortened, the difficulty of the depth camera being attacked is improved, and the security of the depth camera is higher.
Fig. 9 is a schematic structural diagram of a computer-readable storage medium in an embodiment of the present invention. Referring to fig. 9, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this respect, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).
According to the embodiment of the invention, the background image, the infrared structure light image and the infrared image of the same target are continuously and sequentially acquired according to the preset frame frequency value or the infrared image, the infrared structure light image and the background image of the same target are continuously and sequentially acquired, so that the continuous acquisition of three frames of images is realized, the acquisition time interval between two adjacent frames of images is shortened, the attack difficulty of the depth camera is improved, and the safety of the depth camera is higher; according to the embodiment of the invention, the target structured light image is generated by subtracting the gray values of the corresponding pixels of the background image and the infrared structured light image, and then the depth image is generated by performing depth reconstruction or three-dimensional reconstruction according to the target structured light image, so that the interference of background light is reduced, and the depth camera can be suitable for an environment with strong illumination intensity.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.
Claims (10)
1. A depth reconstruction method, comprising the steps of:
step S1: acquiring a preset frame frequency value, and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value;
step S2: subtracting the gray values of corresponding pixels of the background image and the infrared structured light image to generate a target structured light image;
step S3: and performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
2. The depth reconstruction method according to claim 1, wherein the step S1 includes the steps of:
step S101: acquiring a preset frame frequency threshold value and an original frame frequency value, wherein the original frame frequency value is a frame frequency value of an infrared camera for acquiring an infrared structured light image and an infrared image;
step S102: determining a multiple value between the original frame frequency value and the frame frequency threshold, when the multiple value is less than or equal to a preset multiple threshold, determining that the preset frame frequency value is the product of the multiple threshold and the frame frequency threshold, and when the multiple value is greater than or equal to the preset multiple threshold, determining that the preset frame frequency value is the original frame frequency value;
step S103: and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target in a frame frequency threshold sampling period or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value.
3. The depth reconstruction method according to claim 2, wherein the step S103 comprises the steps of:
step S1031: projecting structured light and floodlight through a light projector of a depth camera toward a target;
step S1032: when the multiple value is less than or equal to a preset multiple threshold value, continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target in a frame frequency threshold value sampling period or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency value;
step S1033: and when the multiple value is larger than a preset multiple threshold value, sequentially acquiring background images, infrared structured light images and infrared images of the same target in any three continuous frames in each sampling period determined according to the frame frequency threshold value according to the frame frequency value, or sequentially acquiring the infrared images, the infrared structured light images and the background images of the same target.
4. The depth reconstruction method according to claim 1, wherein the step S2 includes the steps of:
step S201: determining a pixel value of each pixel in the background image and the infrared structured light image;
step S202: performing pixel-level alignment on the background image and the infrared structured light image;
step S203: and subtracting the gray value of each pixel in the infrared structured light image from the gray value of the corresponding pixel in the background image to generate a target structured light image.
5. The depth reconstruction method according to claim 2, wherein the step S3 includes the steps of:
step S301: calculating the target structured light image and known calibration information to obtain a parallax image of the target structured light image;
step S302: determining the distance between the optical center of the infrared camera and each parallax value in the parallax map according to a triangulation principle, and generating depth information of each pixel;
step S303: and performing depth reconstruction or three-dimensional reconstruction according to the depth information of each pixel to generate a depth image.
6. The depth reconstruction method according to claim 1, wherein the infrared structured light image is an image containing encoded texture, and comprises any one of the following structured light images:
-a speckle image;
-a fringe image;
-encoding the image;
-a raster image.
7. The depth reconstruction method according to claim 2, wherein the preset frame rate threshold is 15FPS, and the preset multiple threshold is 3.
8. A depth reconstruction system for implementing the depth reconstruction method according to any one of claims 1 to 7, comprising:
the image acquisition module is used for acquiring preset frame frequency values and continuously and sequentially acquiring background images, infrared structured light images and infrared images of the same target or continuously and sequentially acquiring infrared images, infrared structured light images and background images of the same target according to the frame frequency values;
the image enhancement module is used for generating a target structured light image according to the subtraction of the gray values of the corresponding pixels of the background image and the infrared structured light image;
and the depth reconstruction module is used for performing depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.
9. A depth reconstruction apparatus, characterized by comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the depth reconstruction method of any one of claims 1 to 7 via execution of the executable instructions.
10. A computer-readable storage medium storing a program, wherein the program is configured to implement the steps of the depth reconstruction method according to any one of claims 1 to 7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239717.1A CN115035234A (en) | 2021-03-04 | 2021-03-04 | Depth reconstruction method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239717.1A CN115035234A (en) | 2021-03-04 | 2021-03-04 | Depth reconstruction method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115035234A true CN115035234A (en) | 2022-09-09 |
Family
ID=83117892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110239717.1A Pending CN115035234A (en) | 2021-03-04 | 2021-03-04 | Depth reconstruction method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035234A (en) |
-
2021
- 2021-03-04 CN CN202110239717.1A patent/CN115035234A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112771573A (en) | Depth estimation method and device based on speckle images and face recognition system | |
| US20220351390A1 (en) | Method for generating motion capture data, electronic device and storage medium | |
| Luo et al. | An intensity-based cooperative bidirectional stereo matching with simultaneous detection of discontinuities and occlusions | |
| CN107958446B (en) | Information processing apparatus, information processing method, and computer program | |
| WO2020134818A1 (en) | Image processing method and related product | |
| CN107452034B (en) | Image processing method and device | |
| CN103841406B (en) | A kind of depth camera device of plug and play | |
| US20170249053A1 (en) | Near Touch Interaction | |
| CN111192312B (en) | Depth image acquisition method, device, equipment and medium based on deep learning | |
| KR20220015964A (en) | Methods and systems for restoration of lost image features for visual odometry applications | |
| CN115272565A (en) | Head three-dimensional model reconstruction method and electronic equipment | |
| CN114037087B (en) | Model training method and device, depth prediction method and device, equipment and medium | |
| US20230260211A1 (en) | Three-Dimensional Point Cloud Generation Method, Apparatus and Electronic Device | |
| CN115035234A (en) | Depth reconstruction method, system, device and storage medium | |
| US20230115765A1 (en) | Method and apparatus of transferring image, and method and apparatus of training image transfer model | |
| CN113810654A (en) | Image video uploading method and device, storage medium and electronic equipment | |
| CN115410115A (en) | Action identification method and system based on multi-feature fusion | |
| CN114494574A (en) | Deep learning monocular three-dimensional reconstruction method and system based on multi-loss function constraint | |
| CN113920023A (en) | Image processing method and device, computer readable medium and electronic device | |
| CN116704001A (en) | Multi-frame depth reconstruction method, system, equipment and storage medium | |
| CN113673285B (en) | Depth reconstruction method, system, equipment and medium during capturing of depth camera | |
| CN113673286B (en) | Depth reconstruction method, system, equipment and medium based on target area | |
| CN113673287B (en) | Depth reconstruction method, system, equipment and medium based on target time node | |
| CN113673284B (en) | Depth camera snapshot method, system, equipment and medium | |
| CN111754543A (en) | Image processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |