WO2020199562A1

WO2020199562A1 - Depth information detection method, apparatus and electronic device

Info

Publication number: WO2020199562A1
Application number: PCT/CN2019/113434
Authority: WO
Inventors: 李彪; 苏显渝; 邵双运
Original assignee: 四川深瑞视科技有限公司
Priority date: 2019-04-01
Filing date: 2019-10-25
Publication date: 2020-10-08
Also published as: CN110009673B; CN110009673A

Abstract

A depth information detection method, an apparatus and an electronic device, related to the technical field of image processing. Wherein, the method comprising: obtaining a target speckle image group formed by projecting a k number of different reference speckle patterns onto a target object (S110); taking an m number of coarse matching template groups and respectively matching same to all of or to a portion of the target speckle image group, obtaining a coarse matching template group having the highest degree of similarity, and using same as a primary matching template group (S120); selecting precise matching template groups from a preset front/back range of the primary matching template group, and respectively matching same to all of or to a portion of the target speckle image group, obtaining a precise matching template group having the highest degree of similarity, and using same as a secondary matching template group (S130); in accordance with depth information of the secondary matching template group, determining depth information for a target speckle image (S140).

Description

In-depth information detection method, device and electronic equipment

This application claims the priority of the Chinese patent application with the application number CN201910258089.4 filed on April 1, 2019, the entire content of which is hereby incorporated by reference.

Technical field

This application relates to the field of image processing technology, and more specifically, to a depth information detection method, device, and electronic equipment.

Background technique

With the development of science and technology, in some image display scenes, two-dimensional images can no longer meet people's needs. 3D images are more real and accurate because they have more depth information than 2D images. In daily life, the use of 3D scenes is more and more common, such as face payment, motion sensing games, AR shopping, etc.

In a three-dimensional scene, it is necessary to obtain the depth information of the image, but the existing method of obtaining the depth information of the image is computationally intensive and time-consuming.

Summary of the invention

In view of the above problems, this application proposes a depth information detection method, device and electronic equipment to improve the above problems.

In a first aspect, an embodiment of the present application provides a depth information detection method, the method includes: obtaining a target speckle image group formed by projecting k different reference speckle patterns to a target object; and matching m coarse matching templates The group is matched with all or part of the target speckle image group, and the rough matching template group with the highest similarity is obtained as the primary matching template group. Each matching template group corresponds to its own depth information, and every two adjacent ones The interval of the rough matching template group is R, every two adjacent rough matching template groups include a fine matching template group, and the interval between every two adjacent fine matching template groups is r, R is greater than r, the same rough matching template The group or the same fine matching template group is formed by all or part of the k different reference speckle patterns respectively projected to the reference screen at the same position; the fine matching templates within the preset range before and after the primary matching template group are selected Groups, respectively matching all or part of the target speckle image group, obtaining the fine matching template group with the highest similarity as the secondary matching template group; determining the target speckle according to the depth information of the secondary matching template group The depth information of the image.

In a second aspect, an embodiment of the present application provides a depth information detection device, the device includes: an image acquisition module configured to acquire a target speckle image group formed by projecting k different reference speckle patterns to a target object; The rough matching module is used to match the m rough matching template groups with all or part of the target speckle image group respectively, and obtain the rough matching template group with the highest similarity as the primary matching template group, where each matching template The groups correspond to their respective depth information. The interval between every two adjacent rough matching template groups is R, every two adjacent rough matching template groups includes a fine matching template group, and every two adjacent fine matching template groups are separated Is r, R is greater than r, the same rough matching template group or the same fine matching template group is formed by all or part of the k different reference speckle patterns projected to the same position of the reference screen; the fine matching module, Used to select the fine matching template group within the preset range before and after the primary matching template group, respectively match all or part of the target speckle image group, and obtain the fine matching template group with the highest similarity as the secondary matching Template group; a depth information determination module for determining the depth information of the target speckle image according to the depth information of the secondary matching template group.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, the memory is coupled to the processor, and the memory stores instructions. When the instructions are executed by the processor, The processor executes the above-mentioned method.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 shows a schematic structural diagram of a matching template obtaining system provided by an embodiment of the present application.

Figures 2 to 4 show different schematic diagrams of projections in the matching template acquisition.

FIG. 5 shows a schematic diagram of speckle movement provided by an embodiment of the present application.

Fig. 6 shows a flowchart of a depth information detection method provided by an embodiment of the present application.

Fig. 7 shows a flowchart of a depth information detection method provided by another embodiment of the present application.

FIG. 8 shows a schematic diagram of a matching template provided by an embodiment of the present application.

FIG. 9 shows a schematic diagram of the matching template and the target speckle image area division provided by an embodiment of the present application.

FIG. 10 shows a schematic diagram of a specific area division method provided by an embodiment of the present application.

Fig. 11 shows a functional block diagram of a depth information detection device provided by an embodiment of the present application.

Fig. 12 shows a structural block diagram of an electronic device provided by an embodiment of the present application.

Fig. 13 is a storage medium for storing or carrying program codes for implementing the depth information detection method according to the embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

In various fields such as face payment, somatosensory games, and AR shopping, it is necessary to obtain the depth information of the image. For example, in the field of face payment, it is necessary to obtain the depth information of the face in the face image to accurately match whether the face for payment verification is a registered face.

Monocular speckle measurement can be used as a method to obtain image depth information. Generally, monocular speckle measurement can be divided into time correlation and space correlation. Time correlation generally refers to moving an equidistant reference screen in a space with a known depth. The transmitter projects a pattern with speckle patterns on the reference screen, and the collector records the speckle patterns at these positions, thereby recording the corresponding known depth. The speckle pattern of the equidistant reference screen in each position is moved in space. Taking advantage of the different shape of speckles at each position in space, when the target object (such as the face in face payment) is placed in this dimension, it is similar to speckles of different shapes at each position in this set of time series The degree of matching can search for the depth information of the target object. Spatial correlation generally refers to only using a reference screen speckle image and projecting a speckle image with a speckle shape to the target object. Only the two images are matched for similarity to compare the target object relative to the reference screen speckle image. The offset of each coordinate position is used to obtain the depth map of the target object with the help of the external geometric triangle relationship, and obtain the depth information of the target object.

Among the above methods, the time correlation method needs to load all speckle images and match them with all speckle images, and in addition to various complex forms of cross-correlation matching functions, the calculation is very time-consuming and not suitable for fast measurement methods. Such as the requirement of fast matching required in face payment scenarios. The spatial correlation method only uses two images, and needs to calculate the offset of each coordinate position, which is time-consuming. In addition, using only a small window in the calculation will bring about a large number of mismatches, while using a large window will result in a decrease in spatial resolution. When the surface of the measurement object is complex, the accuracy will be lower.

Therefore, an embodiment of the present application proposes a depth information detection method. By increasing the reference template group, the approximate depth information of the target object is found through coarse matching, and then the fine matching is performed. The partial fine matching image is scattered with the target. Spot image matching can obtain accurate depth information of the target object with a small amount of calculation.

Figure 1 shows a matching template acquisition system for acquiring a matching template used for depth information detection. As shown in Figure 1, the matching template acquisition system includes a projection unit, an image acquisition unit and a storage unit.

Among them, the projection unit may include a light source, a collimating lens, a diffractive optical element and other devices for projecting patterns. The projection unit can be used to project one pattern, and can also be used to project multiple patterns with different densities and/or shapes.

Optionally, the projection unit may be a visible light projector. Optionally, the projection unit may be an infrared laser module, and its light source may be a VCSEL array laser for projecting infrared patterns. When the light source of the projection unit is a VCSEL array laser, the VCSEL array laser with variable density and/or shape can be continuously projected to project different patterns; it can also be combined with multiple VCSEL array lasers to emit different densities And/or the shape of the pattern; or the relative position can be changed by multiple diffractive optical elements for emitting different patterns.

The specific shape and density of the pattern projected by the projection unit are not limited in the embodiments of the present application. When the same pattern is projected to different distances from the projection unit, different imaging can be achieved. For example, the characteristic of speckle is scattered round spots, which meets the irregular messy information required for matching. When the same speckle pattern is projected from the projection unit, the image is different at different distances from the projection unit, so that it can be obtained at different positions A unique speckle image. Therefore, in the embodiment of the present application, the projection unit can be used to project the speckle pattern, and the speckle pattern is taken as an example for description. The specific light source of the projection unit is not limited in the embodiments of the present application. The speckle pattern projected by the light source can be collected by the corresponding image acquisition unit. For example, the speckle image projected by the infrared light source is collected by the infrared image acquisition device. The visible light image acquisition device collects speckle images projected by the visible light source, etc.

The image acquisition unit maintains a certain baseline distance from the projection unit. It can be an image sensor that records the wavelength of the pattern emitted by the projection unit. It is used to collect the image of the speckle pattern projected by the projection unit. It can include photosensitive elements, filters, and lenses. The image acquisition unit may be an image sensor corresponding to the type of light source. For example, the light source of the projection unit is infrared light, and the image acquisition unit is an infrared light image acquisition device; if the light source is visible light, the image acquisition unit is a visible light image acquisition device. The positional relationship between the image acquisition unit and the projection unit is not limited in the embodiment of the present application. For example, the projection unit is placed horizontally and projected horizontally, and the image acquisition unit and the projection unit are placed at the same level.

The storage unit is connected to the image acquisition unit, and is used to store the speckle pattern acquired by the image acquisition unit as a matching template. The storage unit can be any of FLASH, ROM or hard disk.

In the embodiment of the present application, the matching template acquisition system may further include a processing unit, which is electrically connected to the image acquisition unit, the projection unit, and the storage unit. The platform of the processing unit can be one of ASIC, FPGA and DSP, which is used to process the collected images, and can also be used to control the projection of the projection unit and the image collection of the image acquisition unit. Optionally, the processing unit may include a controller for performing control, such as control by a synchronous sequential circuit and an asynchronous sequential circuit; and may also include a depth processor for performing depth information acquisition processing.

The units in the system can be independent of each other or integrated. For example, the system may be an electronic device that integrates a projection unit, an image acquisition unit, a storage unit, and a processing unit, such as a mobile phone, a tablet computer, and a notebook computer.

The matching template obtaining system can obtain the matching template for image depth information detection. When obtaining the matching template, as shown in Figure 2, a reference screen can be placed in the projection direction of the projection unit (as shown by the arrow in Figure 2). The reference screen is placed on the depth axis of the projection unit, and the reference screen and The distance between the projection units changes, such as increasing or decreasing sequentially. When the image acquisition unit acquires different distances between the reference screen and the projection unit, the speckle pattern projected by the projection unit is imaged on the reference screen. Among them, the reference screen is the projection plane used to carry the speckle pattern, the image projected by the projection unit can be imaged on the reference screen, and the image acquisition unit can acquire the pattern projected by the projection unit on the reference screen by collecting images on the reference screen. Into an image, which can be used as a matching template. It can be understood that FIG. 2 is only an exemplary illustration, and the reference screen at all positions is not drawn.

For example, as shown in FIG. 3, R11, R12 to Rpb are respectively equidistantly arranged position points in the projection direction, and the distance between every two adjacent position points of R11, R12 to Rpb is r. R11, R21, R31 and so on are deduced to Rp1, among the p position points, the distance between every two adjacent position points is R. When the reference screen is at each position corresponding to R11, R12 to Rpb, the speckle image projected by the projection unit is imaged on the reference screen at each position, and the image acquisition unit collects the imaging of the speckle image at each position on the reference screen . For example, when the reference screen is at the position point R11, the image acquisition unit collects the image projected on the reference screen by the projection unit; when the reference screen is at R12, the image acquisition unit collects the image projected on the reference screen by the projection unit; until the reference screen is at the position point In Rpb, the image acquisition unit collects the image projected on the reference screen by the projection unit. It is defined that the image projected to the reference screen by the speckle pattern collected at each position is a speckle image, so as to obtain a series of equidistant speckle images, and the distance between the speckle images represents the reference screen imaged by the speckle image. The spacing between. For example, the distance between the speckle image formed on the reference screen at R11 and the speckle image formed on the reference screen at R12 is the distance between the R11 position point and the R12 position point.

As an implementation manner, if the distance between the reference screen and the projection unit is small, such as less than a certain minimum preset threshold, the distance between the reference screen and the projection unit is increased sequentially, and the collection is performed as the distance increases. image.

In this embodiment, the way to increase the distance between the projection unit and the reference screen in sequence may be to move the reference screen in the projection direction and move away from the projection unit to form a series of equidistant reference screens. The speckle pattern at each position is imaged on the reference screen to obtain a series of equally spaced images. For example, in the scene shown in Figure 3, the reference screen is moved from R11 to R12 to Rpb, the speckle image projected by the projection unit is imaged on the reference screen at each position, and the image acquisition unit collects scatter at each position. The speckle image is imaged on the reference screen to obtain b*p equidistant speckle images.

In this embodiment, the way to increase the distance between the projection unit and the reference screen in sequence may also be to move the projection unit in the projection direction to the direction away from the reference screen at equal intervals to form a series of equidistant reference screens. Screen, obtain the imaging of the reference screen on the reference screen at each position, and obtain a series of equally spaced images. For example, the placement of the projection unit and the image acquisition unit and the reference screen is shown in Figure 4. Move the projection unit and the image acquisition unit from R11 to R12, R13 and Rpb at the same time as shown in Figure 4. The projection unit is in The speckle image projected at each position point is imaged on the reference screen, and the image acquisition unit collects the imaging of the speckle image projected by the projection unit at each position point on the reference screen, b*p speckle images with a spacing of r.

As an implementation manner, if the distance between the reference screen and the projection unit is large, such as greater than a certain maximum preset threshold, the distance between the reference screen and the projection unit is sequentially reduced, and the collection is performed as the distance decreases. image.

In this embodiment, the way to sequentially reduce the distance between the projection unit and the reference screen may be to move the reference screen to the projection unit in sequence to form a series of equidistant reference screens. For example, in the scene shown in Figure 3, the reference screen is moved from Rpb to Rp(b-1) to R11, the speckle image projected by the projection unit is imaged on the reference screen at each position, and the image acquisition unit collects each Refer to the speckle image on the screen at the position point to obtain b*p equidistant speckle images.

In this embodiment, the way to sequentially reduce the distance between the projection unit and the reference screen may also be to move the projection unit to the reference screen in sequence to form a series of equally spaced reference screens. For example, in the scene shown in Figure 4, the image acquisition unit and the projection unit are moved from Rpb to Rp(b-1) to R11 at the same time. The speckle image projected by the projection unit at each position point is imaged on the reference screen. The acquisition unit acquires the imaging of the speckle image projected by the projection unit at each position point on the reference screen, and obtains b*p speckle images with a spacing of r. It can be understood that when the relative displacement between the reference screen and the projection unit changes, the speckles in the speckle image will also move left and right. In the embodiment of the present application, the criterion for selecting the distance between every two adjacent position points may be such that the moving distance of the speckle is less than or equal to the radius of the speckle. That is to say, every time the distance between the reference screen and the projection unit increases or decreases by r, the moving distance of the speckle on the reference screen is less than or equal to the radius of the speckle, and the distance between the reference screen and the projection unit increases There is an intersection between the two positions where r and the reduced r speckle are located. For example, Fig. 5 shows an example of the movement of a speckle. In Fig. 5, the solid circle 101 represents the image of a speckle on the reference screen when it is at a certain position on the reference screen. When the distance between the reference screen and the projection unit decreases, the speckle will shift to the left, as shown by the dotted circle 102 in Figure 5; when the distance between the reference screen and the projection unit increases, the speckle will shift to the right The displacement is shown by the dotted circle 103 in FIG. 5. The selection criterion for the distance between every two adjacent points is that when the distance between the reference screen and the projection unit decreases by r, the solid circle 101 moves to the position of the dotted circle 102, and the solid circle 101 to the dashed circle 102 The moving distance of is smaller than the radius of the solid circle 101; when the distance between the reference screen and the projection unit increases by r, the solid circle 101 moves to the position of the dashed circle 103, and the moving distance of the solid circle 101 to the dashed circle 103 It is smaller than the radius of the solid circle 101. The solid circle 101 and the dashed circle 102 have an intersection, and the solid circle 101 and the dashed circle 103 also have an intersection.

Through the matching template obtaining system, multiple templates can be obtained for multi-template matching, or a single template can be obtained for single-template matching. Among them, the speckle pattern projected from the projection unit is defined as the reference speckle pattern. A single template is a set of matching templates obtained when a reference speckle pattern is projected; a multi-template is a set of matching templates obtained when multiple different reference speckle patterns are projected. Among them, different reference speckle patterns may have different shapes of speckle patterns, different densities of speckle patterns, or different shapes and densities of speckle patterns.

When a single template is acquired, a reference speckle pattern is projected from the projection unit. When the image acquisition unit acquires different distances between the reference screen and the projection unit, the speckle pattern projected by the projection unit is imaged on the reference screen to obtain the corresponding A series of matching templates for the reference speckle pattern. For example, as shown in Figures 2 and 3, the reference speckle pattern P1 is projected from the projection unit, and the images formed when the reference speckle pattern P1 is projected onto the reference screens at R11, R12, R13 to Rpb, respectively, are obtained to obtain b*p A matching template as a set of matching templates corresponding to the reference speckle pattern P1.

When acquiring multiple templates, a variety of different reference speckle patterns are projected from the projection unit, and the images of different reference speckle patterns on the reference screen are obtained at various positions at different distances from the projection unit. The images formed at different positions are used as a set of templates corresponding to the reference speckle patterns, and multiple sets of matching templates corresponding to multiple different reference speckle patterns are obtained. Wherein, the multiple sets of matching templates include multiple sets of matching templates, and each set of matching templates is an image formed by projecting a different reference speckle pattern onto the same position. It is understandable that in the embodiment of the present application, a set of matching templates can also be selected from multiple templates as a single template.

As an implementation manner, single templates corresponding to different reference speckle patterns may be obtained in a manner of obtaining single templates, as multiple templates. For example, k different reference speckle patterns include P1, P2, P3 to Pk, project the reference speckle pattern P1 from the projection unit to obtain a set of matching templates corresponding to P1; project the reference speckle pattern P2 from the projection unit, Obtain a set of matching templates corresponding to P2; project the reference speckle pattern P3 from the projection unit to obtain a set of matching templates corresponding to P3; until the reference speckle pattern Pk is projected from the projection unit to obtain a set of matching templates corresponding to Pk, Thus, k sets of matching templates corresponding to k different reference speckle patterns P1, P2, P3 to Pk are obtained. The k sets of templates include b*p matching templates, and each matching template includes k matching templates P1, P2, P3 to Pk respectively projected at the same position. Among them, the images formed by P1, P2, P3 to Pk projected on the reference screen at R11 are a set of matching templates, and the images formed by P1, P2, P3 to Pk projected on the reference screen at R12 are a set of matching templates, P1, The images formed by P2, P3 to Pk projected on the reference screen at R13 are a set of matching templates, and so on, forming a b*p set of matching templates.

As another implementation manner, different reference speckle patterns are respectively projected at each position at different distances from the projection unit, and different matching templates corresponding to different reference speckle patterns at that position are obtained. For example, the k different reference speckle patterns include P1, P2, P3 to Pk. As shown in Figure 3, when the reference screen is at R11, the reference speckle patterns P1, P2, P3 to Pk are respectively projected to obtain P1, P2, Each of P3 to Pk is imaged at the reference screen at R11 to obtain a set of matching templates at R11, including k matching templates corresponding to P1, P2, P3 to Pk; when the reference screen is at R12, Project the reference speckle patterns P1, P2, P3 to Pk respectively, obtain the imaging of the reference screen at R12 for each of P1, P2, P3 to Pk, and obtain a set of matching templates at R12, including corresponding P1, K matching templates of P2, P3 to Pk; until the reference screen is at Rpb, project the reference speckle patterns P1, P2, P3 to Pk respectively, and obtain the reference of each of P1, P2, P3 to Pk at Rpb The imaging of the scene, obtain a set of matching templates in Rpb, including k matching templates corresponding to P1, P2, P3 to Pk. So as to match the template in the b*p group of R11 to Rpb respectively. The b*p group of matching templates includes k sets of matching templates, corresponding to k different reference speckle patterns P1, P2, P3 to Pk, respectively.

In the embodiment of the present application, a fine matching template may be selected from a single template for fine matching, and a coarse matching template may be selected for coarse matching. Wherein, the distance between the coarse matching templates is greater than the distance between the fine matching templates. For example, from a single template, use R as the spacing, and select templates at equal intervals as a set of coarse matching templates, and use the remaining templates as fine matching templates. Specifically, taking the scenarios shown in Figures 2 and 3 as an example, matching templates formed at p positions of R11, R21, R31, R41, and so on to Rp1, respectively, can be selected as a set of rough matching templates. The remaining matching templates are used as fine matching templates. Among them, adjacent rough matching templates include (b-1) fine matching templates. For example, the rough matching template at R11 and the rough matching template at R21 include (b-1) at R12 to R1b, respectively. (B-1) fine matching templates at positions.

In the embodiment of the present application, a fine matching template group may be selected from multiple templates for fine matching, and a coarse matching template group may be selected for coarse matching. Among them, the spacing between the coarse matching template groups is greater than the spacing between the fine matching template groups, the matching templates in the same coarse matching template group are selected from the same set of matching templates, and the matching templates in the same fine matching template group are selected Match templates in the same group. For example, from a multi-template, using R as the spacing, multiple sets of matching templates are selected at equal intervals as multiple rough matching template groups, and the remaining sets of matching templates are used as multiple fine matching template groups. Specifically, taking the scenes shown in Figure 2 and Figure 3 as examples, p sets of matching templates formed at p positions of R11, R21, R31, R41, and so on to Rp1, respectively, can be selected as p rough Matching template group; other matching templates are used as fine matching template groups. Among them, the adjacent rough matching template groups include (b-1) fine matching template groups. For example, the rough matching template group at R11 and the rough matching template group at R21 include R12 to R1b respectively. (B-1) precise matching template groups at (b-1) positions.

In the embodiment of the present application, the number of matching templates in each rough matching template group is the same, and the corresponding reference speckle patterns are the same. For example, the reference speckle patterns corresponding to each rough matching template group are P1, P2, P3 to Pk ; The number of matching templates in each fine matching template group is the same, and the corresponding reference speckle patterns are the same. For example, the reference speckle patterns corresponding to each fine matching template group are P1, P2, P3 to Pk. In the selected rough matching template group, the number of matching templates in each rough matching template group is not limited to be equal to the number of matching templates in a set of matching templates, such as the selected rough matching template group in the scenario shown in Figure 2. The number of matching templates in a rough matching template group is not limited to k. In the selected fine matching template group, the number of matching templates in each fine matching template group is not limited to be equal to the number of matching templates in a set of matching templates, such as the fine matching template group selected in the scenario shown in Figure 2. The number of matching templates in a fine matching template group is not limited to k. The number of matching templates in each fine matching template group may be different from the number of matching templates in the rough matching template group. In the embodiment of the present application, when generating the matching template, only the coarse matching template for coarse matching and the fine matching template for fine matching may be generated; or when the matching template is obtained, only the coarse matching template for coarse matching may be obtained. Matching template and fine matching template for fine matching.

It is understandable that when the number of matching templates in a coarse matching template group is 1, the coarse matching template group can be regarded as the coarse matching template in single template matching; when the number of matching templates in a fine matching template group is 1, then The fine matching template group can be regarded as the fine matching template in single template matching.

In the embodiment of the present application, depth information can be defined for each location point, and the change relationship between the depth information corresponds to the change relationship between the location points. For example, in the scenes shown in Figures 2 to 4, with x as a depth unit, the depth information of R11 is defined as x0, then the depth information of R12 is (x0-x), and the depth information of R13 is (x0-2x), The depth information of R14 is (x0-3x), and the depth information of Rpb is (x0-(p*b-1)x). For another example, define the depth information of a location point as depth 0, and each location point before this location point is depth 1 unit, 2 unit, 3 unit, and so on from the adjacent location point; Each position point is -1 unit, -2 unit, and so on from the adjacent position point. Or directly use the distance r between the position points as a unit of depth information, or directly use the distance of each position point from the projection unit as the depth information of the position point. Therefore, in this embodiment of the present application, each matching template and each group of matching templates corresponds to depth information, and the depth information corresponding to each matching template is the depth information of the location where the matching template is obtained; the depth information corresponding to each group of matching templates is the acquisition The depth information corresponding to the position of the set of matching templates. In the same way, the selected coarse matching template, fine matching template, coarse matching template group, and fine matching template group all have depth information of corresponding positions.

In the embodiment of the present application, the depth information detection of the image can be performed through the matching template obtained by the matching template acquisition system. Fig. 6 shows a depth information detection method provided by an embodiment of the present application. This method can be applied to electronic devices. The electronic device can be a mobile phone, a tablet computer, a personal computer, and other smart devices that can be used for in-depth information detection. The depth information detection method can also be used in a depth information detection system. The depth information detection system may include a projection unit, an image acquisition unit, a storage unit, and a processing unit as shown in FIG. 1, or the depth information detection system and matching template acquisition The system is the same system. Among them, the target object of the measured depth needs to be placed between the distance range corresponding to the nearest and the farthest reference screen and within the field of view of the acquisition unit, that is, the effective measurement area.

In the exemplification process of the embodiments of the present application, usually k reference speckle patterns P1, P2, P3 to Pk are respectively projected to R11, R21, R31, and so on to the p set of matching templates formed by Rp1 as the multi-template rough matching The p rough matching template groups of, and k reference speckle patterns P1, P2, P3 to Pk are respectively projected into the b*p group of matching templates formed by R11 to Rpb, and the other groups except the rough matching template group As the multiple fine matching template groups for multi-template fine matching, the p matching templates formed by projecting the reference speckle pattern P1 to R11, R21, R31 and so on to Rp1 are used as the p coarse matching templates of single template coarse matching, and The reference speckle pattern P1 is projected into the b*p matching templates formed by R11 to Rpb, and the matching templates except the rough matching template are used as multiple fine matching templates of single template fine matching. Of course, it is understandable that in actual use, in multi-template matching, the number of coarse matching template groups and the number of matching templates in a coarse matching template group are not limited. The number of fine matching template groups and the number of fine matching template groups are not limited. The number of matching templates is not limited; in single template matching, the number of rough matching templates is not limited, and the number of fine matching templates is not limited.

Referring to Fig. 6, the depth information detection method may include:

Step S110: Obtain a target speckle image group formed by projecting k different reference speckle patterns to the target object.

When it is necessary to detect the depth information of a target object on the image, k different reference speckle patterns can be projected to the k images formed by the target object as a target speckle image group, and each image is defined as the target speckle image group. Spot image. Wherein, the k different reference speckle patterns are the same as the reference speckle patterns when the matching template group is obtained.

For example, to detect the depth information of a human face, k different reference speckle patterns of P1, P2, P3 to Pk are respectively projected on the human face, and then the images when each reference speckle pattern is projected onto the human face are collected to obtain k images, each of the k images includes a target speckle image formed by projecting a reference speckle pattern onto a human face. Optionally, if the captured image only includes the target speckle image formed by the projection of the speckle pattern, the k captured images are used as the target speckle image group; if the captured image also includes the image of the target object , Image processing is performed on the collected k images, and the target speckle image formed by projection of the reference speckle pattern is obtained as the target speckle image group.

In the embodiment of the present application, the device for detecting depth information may project k different reference speckle patterns onto the target object to form a target speckle image group; it may also be that other devices combine k different reference speckle patterns Projecting to the target object forms a target speckle image group, and the device for depth information detection obtains the target speckle image group from other devices. In addition, the preprocessing may also be performed by a device that performs depth information detection or by other devices, which is not limited in the embodiment of the present application.

Step S120: The m rough matching template groups are respectively matched with all or part of the target speckle image group, and the rough matching template group with the highest similarity is obtained as the primary matching template group.

Among them, each group of matching templates corresponds to its own depth information, the interval between every two adjacent rough matching template groups is R, and every two adjacent rough matching template groups includes a fine matching template group, and every two adjacent rough matching template groups include a fine matching template group. The interval of the fine matching template group is r, R is greater than r. The same rough matching template group or the same fine matching template group is formed by all or part of the k different reference speckle patterns projected to the reference screen at the same position. . Optionally, R may be a positive integer multiple of r, and the positive integer is greater than 1.

Define the number of rough matching template groups as m. The m rough matching template groups are respectively matched with all or part of the target speckle image group. m may be equal to or less than p. In the embodiment of the present application, m is equal to p as an example for description.

Wherein, a rough matching template group in the embodiment of the present application may include multiple matching templates for multi-template rough matching, or one rough matching template for single template matching. In addition, in this step, all or part of the target speckle image group, that is, the target speckle images in the target speckle image group that have the same number of matching templates as the rough matching template group and correspond to the same reference speckle pattern. For example, the rough matching template group is used for single template matching, and each rough matching template group has only one matching template corresponding to the reference speckle pattern P1, then the target speckle image corresponding to the reference speckle pattern P1 in the target speckle image group is combined with Matching of m rough matching template groups. If the coarse matching template group is used for multi-template matching, and each coarse matching template group has k matching templates corresponding to the reference speckle patterns P1 to Pk, then all target speckle images in the target speckle image group are combined with m coarse speckle images. Match template group matches.

Specifically, the number of rough matching templates in each rough matching template group can be defined as I. When I is equal to 1, the rough matching template group is used for single-template rough matching; when I is greater than 1, the rough matching template group is used for multiple rough matching. Rough template matching. In the embodiment of the present application, when I is greater than 1, I is equal to k as an example for description, that is, the number of matching templates in each rough matching template group is equal to the number of target speckle images in the target speckle image group. Description.

Then in this step, it may be that one target speckle image corresponding to the rough matching template group in the target speckle image group is used as the first target speckle image group, and the m rough matching template groups are respectively combined with The first target speckle image group is matched, and the rough matching template group with the highest similarity to the first target speckle image group is acquired as a primary matching template group.

Since the distance between the rough matching template groups is R, the distance is relatively large. If all or part of the target speckle image group is roughly matched with the rough matching template group, a rough matching template group with the highest similarity can be determined. , Less precise depth information. Therefore, in the embodiment of the present application, the fine matching template group with a small distance between each other can be further matched to obtain more accurate depth information.

Step S130: Select the fine matching template group within the preset range before and after the primary matching template group, and respectively match all or part of the target speckle image group, and obtain the fine matching template group with the highest similarity as the secondary Match template group.

The rough matching template with the highest similarity to the speckle image group is defined as the primary matching template group. Since the primary matching template group can determine an approximate depth information of the target speckle image group, according to the inherent properties of the object, its more accurate depth information usually fluctuates within the approximate depth information range. Therefore, in order to reduce the amount of calculation, the fine matching template group can be selected from before and after the primary matching template group.

Among them, before and after the primary matching template group, that is, the position before and after the position corresponding to the primary matching template group. The fine matching template group before the primary matching template group, that is, the fine matching template group corresponding to the position before the position of the primary matching template group; the fine matching template group after the primary matching template group, that is, the position after the primary matching template group The precise matching template group corresponding to the position. For example, for the primary matching template set at the position of R21, the previous fine matching template set is R1p, R1(p-1), R1(p-2) and then the fine matching template set by analogy; the subsequent fine matching template The group is the fine matching template group of R22, R23, R24 and so on.

A fine matching template group in the embodiment of the present application may include multiple matching templates for multi-template fine matching, or one fine matching template for single template matching. In addition, in this step, all or part of the target speckle image group, that is, the target speckle images in the target speckle image group that have the same number of matching templates as the fine matching template group and correspond to the same reference speckle pattern. For example, the fine matching template group is used for single template matching, and there is only one matching template corresponding to the reference speckle pattern P1, then the target speckle image corresponding to the reference speckle pattern P1 in the target speckle image group is combined with the selected fine matching template group match. If the fine matching template group is used for multi-template matching, and there are only k matching templates corresponding to the reference speckle patterns P1 to Pk, then all the target speckle image groups are matched with the selected fine matching template group.

Specifically, the number of fine matching templates in each fine matching template group can be defined as i. When i is equal to 1, the fine matching template group is used for single template fine matching; when i is greater than 1, the fine matching template group is used for multiple templates. Fine matching. In the embodiment of this application, when i is greater than 1, take i equal to k as an example for description, that is, take the number of matching templates in each fine matching template group equal to the number of target speckle images in the target speckle image group as an example. Description.

Then, in this step, the i target speckle images corresponding to the fine matching template group in the target speckle image group are used as the second target speckle image group, and the preset ranges before and after the primary matching template group are selected The fine matching template groups within are respectively matched with the second target speckle image group, and the fine matching template group with the highest similarity to the second target speckle image group is obtained as a secondary matching template group.

Step S140: Determine the depth information of the target speckle image according to the depth information of the secondary matching template group.

Since the reference speckle pattern is projected to different distances, the formed images are different, therefore, the distance between the precision matching template group with the highest similarity to the target speckle image group from the projection unit is the closest to the distance between the target object and the projection unit, Therefore, the depth information of the target speckle image can be determined according to the depth information of the fine matching template group with the highest similarity to the target speckle image.

In the solution provided in the embodiment of the present application, the target speckle image group is coarsely matched through the coarse matching template group with a larger distance from each other, and the fine matching template group is selected according to the rough matching result, and then according to the distance between each other. A smaller fine matching template group performs fine matching and obtains accurate image depth information through a smaller amount of calculation.

In the embodiment of the present application, in a rough matching template group, the number I of matching templates may be equal to 1 for single-template rough matching; the number I of matching templates may be greater than 1 for multi-template rough matching. In a fine matching template group, the number i of matching templates can be equal to 1 for single-template fine matching; the number i of matching templates can be greater than 1 for multi-template fine matching. Therefore, in the matching process, I can be greater than 1, i is equal to 1, for multi-template coarse matching, single template fine matching; it can also be I is greater than 1, i is greater than 1, for multi-template coarse matching, multi-template fine matching; Or if I is equal to 1, and i is greater than 1, a single-template coarse matching and multiple-template fine matching are performed. For the different situations where I and i are greater than 1 or equal to 1, the present application is described by the following embodiments.

FIG. 7 shows a depth information detection method provided by an embodiment, and the method includes:

Step S210: Obtain a target speckle image group formed by projecting k different reference speckle patterns to the target object.

Step S220: Use I target speckle images corresponding to the rough matching template group in the target speckle image group as the first target speckle image group, and disperse the m rough matching template groups with the first target speckle image group respectively. Spot image group matching, obtaining a rough matching template group with the highest similarity to the first target speckle image group as a primary matching template group.

When I is equal to 1, the rough matching template group has only one rough matching template, and the m rough matching template groups are m rough matching templates corresponding to the same reference speckle pattern at different positions. There is only one target speckle image in the first target speckle image group. In this step, the target speckle image is compared with m coarse matching templates one by one, and the coarse matching template with the highest similarity to the target speckle pattern is obtained as the primary matching template group. There is only one matching template in the primary matching template group.

Optionally, when calculating the similarity between the target speckle image and the coarse matching template, the m coarse matching templates and the target speckle image can be normalized first and then numerically calculated, for example, by template matching based on grayscale Algorithm SAD algorithm (Sum of absolute differences) and MAD algorithm (Mean Absolute Differences) and other algorithms perform numerical calculations to obtain m difference results corresponding to m coarse matching templates. Among them, the coarse matching template corresponding to the difference result with the smallest value is the coarse matching template with the highest similarity to the target speckle image.

Optionally, when calculating the similarity between the target speckle image and the coarse matching template, a logical operation of XOR can be used, and the m coarse matching templates are XORed with the target speckle image to obtain the corresponding m XOR results of m rough matching templates. The rough matching template corresponding to the XOR result with the smallest value is the rough matching template with the highest similarity to the target speckle image.

Optionally, when calculating the similarity between the target speckle image and the coarse matching template, the logical AND operation can be used, and the m coarse matching templates are respectively ANDed with the target speckle image to obtain the corresponding m Roughly match the m AND results of the template. Among them, the coarse matching template corresponding to the result of and with the largest value is the coarse matching template with the largest number of overlapping speckles of the target speckle image, and the coarse matching template with the highest similarity to the target speckle image.

Among them, the depth information of the rough matching template with the highest similarity to the target speckle image is close to the depth information of the target speckle image.

When I is greater than 1, the reference speckle patterns respectively corresponding to the target speckle images in the first target speckle image group are the same as the reference speckle patterns respectively corresponding to the rough matching template group. When I is equal to k, the first target speckle image group is the target speckle image group. In the embodiment of the present application, I is equal to k as an example for description.

In one embodiment, when the first target speckle image group is matched with the rough matching template group, each rough matching template group can be taken as a whole, and the first target speckle image group can be taken as a whole, and each rough matching can be calculated. The similarity between the template group and the first target speckle image group; the rough matching template group with the highest similarity to the first target speckle image group is used as the primary matching template group.

Specifically, as shown in FIG. 8, at each position of the space axis S, the k coarse matching templates on the time axis T are a group of coarse matching templates, which are regarded as a three-dimensional space, which is called a voxel. For example, the k coarse matching templates T1 to Tk on the space axis S1 are a coarse matching template group, which is regarded as a three-dimensional space, which is called a voxel. The k coarse matching templates T1 to Tk on the spatial axis S2 are a coarse matching template group, which is regarded as a three-dimensional space, which is called a voxel. Correspondingly, the first target speckle image group can also be regarded as a three-dimensional space, that is, voxels. The characteristics of the rough matching template group on the spatial axis are more obvious, and the difference in depth can be more sensitively distinguished with slight differences.

In this embodiment, when calculating the similarity between the rough matching template group and the first target speckle image group, the first target speckle image group may be used as a voxel, and each rough matching template As a voxel, the group is matched with the highest similarity through three-dimensional calculations, such as numerical operations, logical operations, or cross-correlation. In addition, voxels can also be split into two-dimensional matrices or one-dimensional sequences to simplify operations. For example, when calculating the similarity between the coarse matching template group and the first target speckle image group by using a 3-dimensional cross-correlation formula, the calculation formula may be:

Among them, A in the formula represents the voxel formed by the rough matching template group,

Is the average value of the voxel. B represents the voxel formed by the first target speckle image group,

Is the corresponding average value. m, n, and s represent the length, width, and height of the voxel, respectively, and i, j, and k are the control variables for the length, width, and height of the voxel, respectively. corr3 is the similarity coefficient of the voxel, and the value reflects the similarity between the two. It is understandable that each letter in the formula represents various meanings defined in this paragraph, and has nothing to do with the aforementioned i representing the number of matching templates, m, n representing the number of matching template groups, and so on.

In another embodiment, each target speckle image in the target speckle image group can be matched with a rough matching template group for single template matching, and then the target can be obtained according to the rough matching template with the highest similarity of each target speckle image. The primary matching template group with the highest pixel degree of the speckle image group.

Specifically, in this embodiment, the calculation method may be the same for each target speckle image in the first target speckle image group. For any target speckle image, determine m coarse matching templates corresponding to the target speckle image in the m coarse matching template group, that is, determine m coarse speckle patterns corresponding to the target speckle image. Match the template. Then calculate the similarity between the m rough matching templates and the target speckle image, and obtain the rough matching template with the highest similarity to the target speckle image.

Each target speckle image in the first target speckle image group corresponds to a rough matching template with the highest similarity, so that one target speckle image corresponding to one target speckle image in the first target speckle image group has the highest similarity. To determine the primary matching template group.

Specifically, a rough matching template with the highest similarity of each target speckle image in the first target speckle image group may be calculated to obtain a matching template, which is used as the primary matching template group.

Optionally, determining the primary matching template group according to one rough matching template with the highest similarity corresponding to one target speckle image in the first target speckle image group may include: obtaining one rough matching template with the highest similarity The similarity coefficients corresponding to the matching templates respectively obtain I similarity coefficients. Then, among the I similarity coefficients, the similarity coefficient representing the highest similarity is obtained, and the rough matching template corresponding to the similarity coefficient representing the highest similarity is used as the primary matching template group. That is, the primary matching template group is a rough matching template, and the similarity between the rough matching template and the corresponding target speckle image is greater than the similarity between other rough matching templates and the corresponding target speckle image. Wherein, before obtaining the similarity coefficient representing the highest similarity among the I similarity coefficients, the I similarity coefficients may be firstly subjected to a parallel multiplication, addition, or mean operation, so as to better distinguish the similarity.

Optionally, determining the primary matching template group according to one rough matching template with the highest similarity corresponding to one target speckle image in the first target speckle image group may include: determining one rough matching template with the highest similarity Match the depth information corresponding to the template to obtain I depth information. Then calculate the average value of the I depth information to obtain the average depth information to reduce accidental errors caused by noise in certain places. The matching template corresponding to the average depth information is used as the primary matching template group.

Step S230: Use i target speckle images corresponding to the fine matching template group in the target speckle image group as the second target speckle image group, and select fine matching template groups within a preset range before and after the primary matching template group , Respectively matching with the second target speckle image group, and obtaining the fine matching template group with the highest similarity with the second target speckle image group as the secondary matching template group.

Optionally, selecting a fine matching template group within a preset range before and after the primary matching template group, the preset range may be a preset range, such as the first number of fine matching template groups before the primary matching template group, The second number of fine matching templates after the primary matching template group.

Optionally, a fine matching template group within a preset range before and after the primary matching template group is selected, and the preset range can be calculated based on the similarity between the primary matching template group and the first target speckle image group. The higher the similarity, the closer the depth information of the target speckle image is to the primary matching template group, and the smaller the range that can be selected. Specifically, if the similarity between the primary matching template group and the first target speckle image group is less than θ ₁ , select γ ₁ fine matching template groups from before and after the primary matching template group; if the similarity is [θ1, θ2] In between, γ ₂ fine matching template groups are selected from the front and back of the primary matching template group; if the similarity is greater than θ ₂ , γ ₃ fine matching template groups are respectively selected from the front and back of the primary matching template group. Among them, θ ₁ <θ ₂ , r ₁ >r ₂ >r _{3 are selected} in order from the fine matching template group adjacent to the primary matching template group. It is understandable that if the primary matching template group is at the boundary, only one direction needs to be selected, that is, the direction of the matching template group before or after. For example, if the primary matching template group is obtained at position R11, and the similarity between the primary matching template group and the first target speckle image group is less than θ ₁ , then starting from R12, γ ₁ fine matching template group is selected in the direction of Rpb.

In the embodiment of the present application, the method of performing fine matching through the selected fine matching template group is similar to the coarse matching method of the coarse matching template group. The following describes the fine matching process, where the rough matching corresponds to the rough matching but is not described in detail Places can be cross-referenced.

Define the number of fine matching template groups in the preset range before and after the selected primary matching template group as n. In the following description process, the comparison with the second target speckle image group is all the n fine matching template groups.

When i is equal to 1, the fine matching template group has only one fine matching template, and the n fine matching template groups are n fine matching templates corresponding to the same reference speckle pattern. There is only one target speckle image in the second target speckle image group. In this step, the target speckle image is compared with the n fine matching templates one by one, and the fine matching template with the highest similarity to the target speckle pattern is obtained as the secondary matching template group. There is only one matching template in the secondary matching template group.

Optionally, during the fine matching, since the similarity with the target speckle image must be determined more accurately to determine the depth information of the target speckle image more accurately, the fine matching enables the matching algorithm used to be more accurate than the coarse More accurate when matching. For example, in the embodiment of the present application, the similarity is calculated by a cross-correlation method with higher accuracy. A cross-correlation operation is performed on the target speckle image and n fine matching templates one by one to obtain the fine matching template with the highest similarity to the target speckle pattern. The algorithm corresponding to the cross-correlation operation may be ZNCC (Zero Mean Normalization cross correlation) or NCC (Normalization cross correlation), etc., which is not limited in the embodiment of the present application.

When i is greater than 1, the reference speckle patterns respectively corresponding to the target speckle images in the second target speckle image group are the same as the reference speckle patterns respectively corresponding to the fine matching template group. When i is equal to k, the second target speckle image group is the target speckle image group. In the embodiment of the present application, i is equal to k as an example for description.

In one embodiment, when the second target speckle image group is matched with the selected fine matching template group, each fine matching template group may be taken as a whole, and the second target speckle image group may be taken as a whole, and each The degree of similarity between the fine matching template group and the second target speckle image group; the fine matching template group with the highest similarity to the second target speckle image group is used as the secondary matching template group.

Specifically, each fine matching template group can be regarded as a three-dimensional space, called a voxel. Correspondingly, the second target speckle image group can also be regarded as one voxel. The characteristics of the fine matching template group on the spatial axis are more obvious, and the difference in depth can be more sensitively distinguished with small differences.

When calculating the similarity between the fine matching template group and the second target speckle image group, the second target speckle image group may be used as a voxel, and each fine matching template group may be used as a voxel, Through three-dimensional calculations, such as numerical operations, logical operations or cross-correlation methods, the highest similarity is matched. In addition, voxels can also be split into two-dimensional matrices or one-dimensional sequences to simplify operations. For example, when calculating the similarity between the fine matching template group and the second target speckle image group by using a 3-dimensional cross-correlation formula, the calculation formula may be:

Among them, A in the formula represents the voxel formed by the fine matching template group,

Is the average value of the voxel. B represents the voxel formed by the second target speckle image group,

In another embodiment, each target speckle image in the target speckle image group can be matched with the fine matching template group for single template matching, and then the target is obtained according to the fine matching template with the highest similarity of each target speckle image. The secondary matching template group with the highest pixel degree of the speckle image group.

Specifically, in this embodiment, the calculation method may be the same for each target speckle image in the second target speckle image group. For any target speckle image, determine n fine matching templates corresponding to the target speckle image in the n fine matching template groups, that is, determine n fine matching templates corresponding to the same reference speckle pattern of the target speckle image. The similarity between the n fine matching templates and the target speckle image is calculated separately, and the fine matching template with the highest similarity to the target speckle image is obtained. Wherein, when calculating the similarity between the target speckle image and the fine matching template, the similarity calculation method described in the single template fine matching can be used, that is, a similarity calculation method with higher accuracy than the single template rough matching is used.

Each target speckle image in the second target speckle image group corresponds to a fine matching template with the highest similarity, so that the i target speckle images corresponding to the i target speckle images in the second target speckle image group have the highest similarity The fine matching template is determined to determine the secondary matching template group.

Specifically, in this embodiment, a fine matching template with the highest similarity of each target speckle image in the second target speckle image group may be calculated to obtain a matching template as the primary matching template group.

Optionally, determining the secondary matching template group according to the i precision matching templates corresponding to i target speckle images in the second target speckle image group with the highest similarity may include: obtaining i highest similarity templates The similarity coefficients corresponding to the finely matched templates respectively obtain i similarity coefficients. Then, among the i similarity coefficients, the similarity coefficient representing the highest similarity is obtained, and the fine matching template corresponding to the similarity coefficient representing the highest similarity is used as the secondary matching template group. That is, the secondary matching template group is a fine matching template, and the similarity between the fine matching template and the corresponding target speckle image is greater than the similarity between other fine matching templates and the corresponding target speckle image. Wherein, before obtaining the similarity coefficient representing the highest similarity among the i similarity coefficients, the i similarity coefficients may be first subjected to a bitwise multiplication, addition, or mean operation, so as to better distinguish the similarity.

Optionally, determining the secondary matching template group according to the i highest similarity fine matching templates corresponding to i target speckle images in the second target speckle image group may include: determining i highest similarity templates The depth information corresponding to the template is precisely matched to obtain i depth information. Then calculate the average value of the i depth information to obtain the average depth information. The fine matching template corresponding to the average depth information is used as the secondary matching template group. That is, the secondary matching template group is a fine matching template.

Step S240: Determine the depth information of the target speckle image according to the depth information of the secondary matching template group.

Each secondary template group corresponds to depth information, and the determined depth information of the secondary template group can be used as the depth information of the target speckle image.

In the embodiment of the present application, k different reference speckle patterns are projected to the target object to form a target speckle image group. First, the target speckle image group is matched through the rough matching template group, and the rough matching template group with the highest similarity is obtained, which is defined as the primary matching template group. Then, from the fine matching template groups that are closer to each other than the coarse matching template group, the fine matching template group in the preset range before and after the primary matching template group is selected to match the target speckle image group, and the target speckle image group is obtained. The precise matching template group with the highest similarity of the spot image group. The depth information of the target speckle image is determined according to the fine matching template group, and accurate image depth information is obtained through a smaller calculation amount.

Further, since the target object may not be a flat object, as shown in the target object in FIG. 2, the distance from the projection unit is different at different positions, so the depth information of different regions in the target speckle image may be different. In the embodiment of the present application, the component regions of the target speckle image may be subjected to coarse matching and fine matching to obtain the depth information of each region, and combine to form the depth information of the target speckle image. That is, the target speckle image is divided into multiple regions, and the region at the same position in the target speckle image group is used as an independent matching unit, and coarse matching and fine matching are performed in the manner of the foregoing embodiment.

Specifically, each target speckle image in the target speckle image group may be divided into multiple image regions according to the same region division method. Then, the image area at the same position in all target speckle images is taken as a sub-target speckle image group to obtain multiple sub-target speckle image groups. Wherein, the image area at the same position means that the pixel area of each image area in the corresponding target speckle image is the same. For example, in a certain target speckle image, the divided image area is a rectangle, and its upper left corner pixel coordinates are (x1, y1), and the lower right corner pixel coordinates are (x2, y2), then in other target speckle images In the image area, the image area at the same position of the image area is a rectangular area with the upper left corner pixel coordinates (x1, y1) and the lower right corner pixel coordinates (x2, y2).

Then, all rough matching templates in all rough matching template groups are divided into multiple image regions according to the same division method as the target speckle image. That is, all rough matching templates and target speckle images can find regions with the same size and the same position in the image. In a rough matching template group, the image area at the same position of each rough matching template is regarded as a sub rough matching template group.

All the fine matching templates of all fine matching template groups are divided into multiple image regions according to the same division method as the target speckle image. That is, all fine matching templates and target speckle images can find regions with the same size and the same position in the image. In a fine matching template group, the image area at the same position of each fine matching template is used as a sperm matching template group.

Traverse each area in the target speckle image for matching, that is, perform coarse matching and fine matching for each sub-target speckle image group. Wherein, each sub-target speckle image group is matched with the sub-rough matching template group and the sub-sperm matching template group at the same position in the image. The matching process refers to the matching process of the target speckle image group in the foregoing embodiment. The specific reference method can be understood as that for each sub-target speckle image group, the target speckle image group in the foregoing embodiment, and the sub-rough matching template group at the same position as the sub-target speckle image group in the foregoing embodiment are substituted into the coarse matching in the foregoing embodiment The template group and the fine matching template group at the same position as the sub-target speckle image group are substituted into the fine matching template group in the foregoing embodiment for matching, and the acquired depth information is the depth information of the image region corresponding to the sub-target speckle image group. The matching process will be briefly described below. It should be emphasized that, in the sub-region matching mode, the specific matching process can refer to the foregoing embodiment.

In an embodiment, for each sub-target speckle image group, first obtain the sub-rough matching template group with the highest similarity of each sub-target speckle image group, and then obtain the sub-rough matching template group according to the sub-rough matching template group. The sperm matching image group with the highest similarity of the target speckle image group. Specifically, for each sub-target speckle image group, the matching process can be:

Match the m sub-rough matching template groups with all or part of the sub-target speckle image group respectively, and obtain the sub-rough matching template group with the highest similarity as the primary sub-matching template group; before and after selecting the primary sub-matching template group The sperm matching template group within the preset range is matched with all or part of the sub-target speckle image group, and the sperm matching template group with the highest similarity is obtained as the secondary sub matching template group; The depth information of the secondary sub-matching template group determines the depth information of the image region corresponding to the sub-target speckle image group.

Among them, the number of rough matching templates in each rough matching template group is defined as I, and the number of fine matching templates in each fine matching template group is i. Matching the m sub-rough matching template groups with all or part of the sub-target speckle image group respectively, and obtaining the sub-rough matching template group with the highest similarity, as the primary sub-matching template group, may include: using the sub-target speckle I image regions corresponding to the rough matching template group in the image group are used as the first sub-target speckle image group, and the m sub-coarse matching template groups are matched with the first sub-target speckle image group respectively, and the The sub-rough matching template group with the highest similarity of the first sub-speckle image group is used as the primary sub-matching template group. Select the sperm matching template group within the preset range before and after the primary sub-matching template group, and respectively match all or part of the sub-target speckle image group, and obtain the sperm-sperm matching template group with the highest similarity as the secondary The first-level sub-matching template group includes: taking i image regions corresponding to the fine-matching template group in the sub-target speckle image group as the second sub-target speckle image group, and selecting a preset range before and after the primary sub-matching template group The sperm matching template group within is respectively matched with the second sub-target speckle image group, and the sperm matching template group with the highest similarity to the second sub-target speckle image group is obtained as the secondary sub-matching template group. The depth information of the secondary sub-matching template group is used as the depth information of the image region corresponding to the sub-target speckle image group.

Since all target speckle images are images formed by projecting the reference speckle pattern onto the same target object, theoretically all target speckle images have the same depth information, so the image area corresponding to the sub-target image group in any target speckle image can be obtained In-depth information. In the same way, the depth information of other image areas in any target speckle image can be obtained, so as to obtain the depth information of each area in the target speckle image.

In another embodiment, the sub-rough matching template group with the highest similarity of each sub-target speckle image group may be obtained first, and the sub-rough matching templates with the highest similarity corresponding to each sub-target speckle image group can be combined into a rough Match template group. Then, according to the rough matching template groups in the rough matching template group, the sperm matching template group is selected, and the sperm matching image group with the highest similarity to each sub-target speckle image group is obtained. The matching process can be:

For each sub-target speckle image group, the m sub-rough matching template groups are respectively matched with all or part of the sub-target speckle image group, and the sub-rough matching template group with the highest similarity is obtained as the primary sub-matching template group . Obtain the sub-rough matching template group with the highest similarity of all sub-target speckle image groups, and synthesize the primary matching template group.

For each sub-target speckle image group, determine the primary sub-matching template group corresponding to the same position, select the sub-prime sub-matching template group within a preset range before and after the primary sub-matching template group, respectively, and the sub-target speckle image group Match all or part of the sub-sperm matching template group with the highest similarity as the secondary sub-matching template group; determine the image area corresponding to the sub-target speckle image group according to the depth information of the secondary sub-matching template group In-depth information.

Obtain the depth information of all image areas in the target speckle image according to all sub-target speckle image groups, thereby obtaining the depth information of the target speckle image. In addition, the depth information of the target object can also be determined according to the position of the target object in the target speckle image.

Furthermore, during the matching process, the matching result may be inaccurate, so it needs to be corrected. Specifically, the secondary sub-matching template group whose similarity is higher than the preset similarity is taken as the effective secondary sub-matching template group, and the depth information of the corresponding image area is calculated; for the secondary sub-matching template whose similarity is not greater than the preset similarity The matching template group is regarded as an invalid secondary sub-matching template group for correction.

Specifically, for each sub-sperm matching template group, if the similarity between it and the corresponding secondary sub-matching template group is greater than the preset similarity β1, the depth result is considered to be accurate, and the depth information of the sub-matching template group is used as the The depth information of the image area corresponding to the sub-target speckle image group; if the similarity between the sub-sperm matching template group and the corresponding secondary sub-matching template group is less than or equal to the preset similarity β1, it is considered that the depth result is not accurate enough, Temporarily use the depth information of the sub-matching template group as the depth information of the image area corresponding to the sub-target speckle image group, but further classification and estimation are required to determine whether to correct the depth information of the image area. For the convenience of description, the image area is named the target image area.

The specific classification and estimation method can be, if there are image areas with similarity greater than β1 in the 8 neighborhoods of the target image area, the depth information of the image areas with similarity greater than β1 is averaged as the depth information of the target image area ; If there is no image area with similarity greater than β1 in the 8 neighborhood, the depth information of the target image area is not modified. For some problems that cannot be corrected based on the similarity correction method, the depth value correction method can be used, such as common image processing methods such as mean filtering and median filtering. It is understandable that the 8 neighborhoods may be 8 image areas adjacent to the target image area. Of course, the embodiment of the present application is not limited to 8 neighborhoods, and other numbers of neighborhoods may also be used.

The embodiment of the present application is described through a specific example of image area division.

Each image in the dashed frame 104 in FIG. 9 represents k target speckle images in the target speckle image group. The area of the target speckle image is divided as shown by the squares in the dashed frame 104 in FIG. 9. The position intervals of the image area in the image are A1, A2, A3 to A24, as shown in Fig. 10. Fig. 10 shows an example of an area division method including a matching template and a target speckle image. The image area at A1 of k target speckle images forms a sub-target speckle image group A1, and the image area at A2 of k target speckle images forms a sub-target speckle image group A2, until the number of k target speckle images The image area at A24 forms a sub-target speckle image group A24. The sub-target speckle image group A6 formed by the image area A6 of the k target speckle images is shown at the corresponding dotted frame 104 in FIG. 9.

Each image in the coordinate system of FIG. 9 represents each matching template, and the matching template corresponding to the same coordinate point on the space axis S is a group matching template. As shown by the squares in each matching template in Figure 9 and Figure 10, each matching template is divided into 24 locations of A1, A2, A3 to A24 according to the same area division method as the target speckle image. Image area. Thus, in each rough matching template group, the image area at A1 forms a sub-rough matching template group A1, the image area at A2 forms a sub-rough matching template group A2, and the image area at A24 forms a sub-rough matching template group A24. In each fine matching template group, the image area at A1 forms a sperm matching template group A1, the image area at A2 forms a sperm matching template group A2, and the image area at A24 forms a sperm matching template group A24. The voxel corresponding to each spatial axis coordinate point in FIG. 9 represents the sub-matching template group formed by the image area at A6 from T1 to Tk.

For each sub-target speckle image group, the rough matching template group matched with it is the sub rough matching template group at the same position in the image, and the fine matching template group matched with it is the fine matching template group at the same position in the image. For example, all sub-coarse matching template groups A6 are matched with the sub-target speckle image group A6, and the primary sub-matching template group closest to the sub-target speckle image group A6 is obtained. Then the sub-target speckle image group A6 is matched with the sperm matching template group A6 in the preset range before and after the primary sub matching template group, and the sperm matching template group A6 with the highest similarity is obtained as the secondary sub matching template group. The depth information of the secondary sub-matching template group is used as the depth information of the sub-target speckle image group A6. For the specific matching process, refer to the matching process of the target speckle image group in the foregoing embodiment.

Since all target speckle images are images formed by projecting the reference speckle pattern onto the same target object, theoretically all target speckle images have the same depth information, so the depth information of the image area at A6 in the target speckle image can be obtained. It is understandable that the sub-target speckle image group A6, the sub-rough matching template group A6, and the sub-target speckle image group A6 are all used to describe the location of each image area as A1.

In the same way, the depth information of the image area at other areas in the target speckle image can be obtained, thereby obtaining the depth information of the target speckle image.

Optionally, the depth information of the image area in each target speckle image can be corrected. Taking the image area at A6 in the target speckle image as an example, if the similarity between the sub-target speckle image group A6 and the secondary sub-matching template group with the highest similarity is greater than β1, then the depth information of the secondary sub-matching template group is used As the depth information of the image area at A6 in the target speckle image; if the similarity between the sub-target speckle image group A6 and the secondary sub-matching template group with the highest similarity is not greater than β1, it is judged that the image area of its 8 neighborhood corresponds to Whether the similarity of is greater than β1, that is, determine whether the similarity between the sub-target speckle image group and its corresponding secondary fine matching template group is greater than β1 at A1 to A3, A5, A7, and A9-A11, that is, determine the sub-target Whether the similarity between the speckle image group A1 and the corresponding secondary fine matching template group is greater than β1, and determining whether the similarity between the sub-target speckle image group A2 and the corresponding secondary fine matching template group is greater than β1, etc. The depth information of the image regions with similarity greater than β1 is averaged as the depth information of the image region A6 in the target speckle image.

In the embodiment of the present application, the target speckle image group is subjected to regional matching. Since the depth information of each target speckle image is the same, any target speckle image in any target speckle image group is used to represent the depth information. Image to obtain the depth information of each image area in the target speckle image. For the target object in the target speckle image, the depth information at each different position can be determined more accurately, so that the depth information detection method can be applied to planar or non-planar target objects for depth information detection.

The embodiment of the present application also provides a depth information detection device 400. Referring to FIG. 11, the device 400 includes: an image acquisition module 410, configured to acquire a target speckle image group formed by projecting k different reference speckle patterns to a target object; a coarse matching module 420, configured to combine m coarse speckle patterns The matching template group is matched with all or part of the target speckle image group, and the rough matching template group with the highest similarity is obtained as the primary matching template group, wherein each matching template group corresponds to its own depth information, and every two The interval between adjacent rough matching template groups is R, every two adjacent rough matching template groups includes a fine matching template group, and the interval between every two adjacent fine matching template groups is r, R is greater than r, and the same rough The matching template group or the same fine matching template group is formed by all or part of the k different reference speckle patterns projected to the same position of the reference screen; the fine matching module 430 is used to select the primary matching template group The fine matching template groups within the preset range before and after are respectively matched with all or part of the target speckle image group, and the fine matching template group with the highest similarity is obtained as the secondary matching template group; the depth information determining module 440, It is used to determine the depth information of the target speckle image according to the depth information of the secondary matching template group.

Specifically, the number of coarse matching templates in each coarse matching template group is defined as I, and the number of fine matching templates in each fine matching template group is i. The rough matching module 420 may be configured to use 1 target speckle images corresponding to the rough matching template group in the target speckle image group as the first target speckle image group, and combine the m rough matching template groups with the The first target speckle image group is matched, and the rough matching template group with the highest similarity to the first target speckle image group is obtained as the primary matching template group. The fine matching module 430 may be configured to: use i target speckle images corresponding to the fine matching template group in the target speckle image group as the second target speckle image group, and select the primary matching template group before and after the preset range The fine matching template group of is respectively matched with the second target speckle image group, and the fine matching template group with the highest similarity to the second target speckle image group is obtained as a secondary matching template group.

Among them, it can be that I is greater than 1 and i is equal to 1, or I is greater than 1, and i is greater than 1, or I is equal to 1, and i is greater than 1.

Optionally, when I is greater than 1, the rough matching module 420 may be configured to take each rough matching template group as a whole, and the first target speckle image group as a whole, and calculate each rough matching template group and the first target speckle image group. The similarity between a target speckle image group; the rough matching template group with the highest similarity to the first target speckle image group is used as the primary matching template group.

Optionally, when I is greater than 1, the coarse matching module 420 may be configured to determine, for each target speckle image in the first target speckle image group, that the m coarse matching template groups correspond to the target speckle M coarse matching templates of the image; respectively calculating the similarity between the m coarse matching templates and the target speckle image to obtain the coarse matching template with the highest similarity to the target speckle image; according to the first target speckle image One rough matching template with the highest similarity corresponding to one target speckle image in the group determines the primary matching template group.

Optionally, when i is greater than 1, the fine matching module 430 may be configured to take each selected fine matching template group as a whole, and the second target speckle image group as a whole, and calculate each fine matching template group and all The similarity between the first target speckle image group; the fine matching template group with the highest similarity to the first target speckle image group is used as the secondary matching template group.

Optionally, when i is greater than 1, the fine matching module 430 may be configured to determine, for each target speckle image in the second target speckle image group, that the n fine matching template groups correspond to the target speckle N fine matching templates of the image; respectively calculating the similarity between the n fine matching templates and the target speckle image to obtain the fine matching template with the highest similarity to the target speckle image; according to the second target speckle image The i precision matching templates with the highest similarity corresponding to i target speckle images in the group determine the secondary matching template group.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the device and module described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.

Please refer to FIG. 12, which shows a structural block diagram of an electronic device 600 according to an embodiment of the present application. The electronic device 600 may be an electronic device capable of in-depth information recognition, such as a mobile phone, a tablet computer, or an e-book. The electronic device includes a processor 610 and a memory 620, the memory is coupled to the processor, and the memory stores instructions. When the instructions are executed by the processor, the processor executes one or more of the above The method described in the embodiment.

The processor 610 may include one or more processing cores. The processor 610 uses various interfaces and lines to connect various parts of the entire electronic device 600, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 620, and calling data stored in the memory 620. Various functions and processing data of the electronic device 600. Optionally, the processor 610 may use at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). A kind of hardware form to realize. The processor 610 may be integrated with one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 610, but may be implemented by a communication chip alone.

The memory 620 may include random access memory (RAM) or read-only memory (Read-Only Memory). The memory 620 may be used to store instructions, programs, codes, code sets or instruction sets, such as instructions or code sets used to implement the deep information detection method provided in the embodiments of the present application. The memory 620 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing each of the foregoing method embodiments, and the like. The storage data area can also be data created by the electronic device in use (such as phone book, audio and video data, chat record data), etc.

Optionally, the electronic device may further include a projection unit for projecting the reference speckle pattern; and an image acquisition unit for acquiring an image projected by the projection unit.

Please refer to FIG. 13, which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable storage medium 700 stores program codes, and the program codes can be invoked by a processor to execute the methods described in the foregoing method embodiments.

The computer-readable storage medium 700 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has a storage space for the program code 710 for executing any method steps in the above methods. These program codes can be read out from or written into one or more computer program products. The program code 710 may be compressed in a suitable form, for example.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A depth information detection method, characterized in that the method includes:

Acquiring a target speckle image group formed by projecting k different reference speckle patterns onto the target object;

Match the m rough matching template groups with all or part of the target speckle image group respectively, and obtain the rough matching template group with the highest similarity as the primary matching template group, wherein each matching template group corresponds to its own depth information , The interval between every two adjacent rough matching template groups is R, every two adjacent rough matching template groups includes a fine matching template group, and the interval between every two adjacent fine matching template groups is r, R is greater than r , The same rough matching template group or the same fine matching template group is formed by all or part of the k different reference speckle patterns respectively projected to the reference screen at the same position;

Select the fine matching template group within the preset range before and after the primary matching template group, and respectively match all or part of the target speckle image group, and obtain the fine matching template group with the highest similarity as the secondary matching template group ；

The depth information of the target speckle image is determined according to the depth information of the secondary matching template group.
The method according to claim 1, wherein the number of coarse matching templates in each coarse matching template group is I, the number of fine matching templates in each fine matching template group is i, and the m coarse matching template groups are Respectively match all or part of the target speckle image group, and obtain the rough matching template group with the highest similarity as the primary matching template group, including: using one of the corresponding rough matching template groups in the target speckle image group The target speckle image is used as the first target speckle image group, the m coarse matching template groups are matched with the first target speckle image group respectively, and the one with the highest similarity to the first target speckle image group is obtained The rough matching template group is used as the primary matching template group;

The selection of the fine matching template group within a preset range before and after the primary matching template group is matched with all or part of the target speckle image group, and the fine matching template group with the highest similarity is obtained as the secondary matching The template group includes: taking i target speckle images corresponding to the fine matching template group in the target speckle image group as the second target speckle image group, and selecting fine matching within a preset range before and after the primary matching template group The template groups are respectively matched with the second target speckle image group, and the fine matching template group with the highest similarity to the second target speckle image group is obtained as a secondary matching template group.
The method according to claim 2, wherein I is greater than 1, and i is equal to 1.
The method according to claim 2, wherein I is greater than 1, and i is greater than 1.
The method according to claim 3 or 4, wherein the m rough matching template groups are respectively matched with the first target speckle image group to obtain the first target speckle image group The rough matching template group with the highest similarity, as the primary matching template group, includes:

Taking each rough matching template group as a whole and taking the first target speckle image group as a whole, calculating the similarity between each rough matching template group and the first target speckle image group;

The rough matching template group with the highest similarity to the first target speckle image group is used as the primary matching template group.
The method of claim 5, wherein the first target speckle image group is taken as a whole to calculate the similarity between each rough matching template group and the first target speckle image group ,include:

Taking the first target speckle image group as a voxel and each rough matching template group as a voxel, three-dimensional calculations are performed on each rough matching template group and the first target speckle image group to obtain each The similarity between a group of rough matching templates and the first target speckle image group, wherein the three-dimensional calculation includes: numerical operation or logical operation.
The method according to claim 3 or 4, wherein the m rough matching template groups are respectively matched with the first target speckle image group to obtain the first target speckle image group The rough matching template group with the highest similarity, as the primary matching template group, includes:

For each target speckle image in the first target speckle image group,

Determining m coarse matching templates corresponding to the target speckle image in the m coarse matching template group;

Calculate the similarity between the m coarse matching templates and the target speckle image respectively, and obtain the coarse matching template with the highest similarity to the target speckle image;

Determine the primary matching template group according to one rough matching template with the highest similarity corresponding to one target speckle image in the first target speckle image group.
The method according to claim 7, characterized in that, the primary matching template group is determined according to one rough matching template with the highest similarity corresponding to one target speckle image in the first target speckle image group, Including: obtaining the similarity coefficients corresponding to the rough matching templates with the highest similarity, and obtaining I similarity coefficients;

Among the I similarity coefficients, the rough matching template corresponding to the similarity coefficient representing the highest similarity is used as the primary matching template group.
The method according to claim 7, characterized in that, the primary matching template group is determined according to one rough matching template with the highest similarity corresponding to one target speckle image in the first target speckle image group, Including: determining the depth information corresponding to each rough matching template with the highest similarity, and obtaining one depth information;

The average value of the I pieces of depth information is calculated to obtain the average depth information, and the matching template corresponding to the average depth information is used as the primary matching template group.
The method according to claim 2, wherein I is equal to 1, and i is greater than 1.
The method according to claim 4 or 10, wherein the matching with the second target speckle image group respectively to obtain the fine matching template group with the highest similarity to the second target speckle image group, As a secondary matching template group, it includes:

Taking each selected fine matching template group as a whole, and taking the second target speckle image group as a whole, calculating the similarity between each fine matching template group and the second target speckle image group;

The fine matching template group with the highest similarity to the second target speckle image group is used as the secondary matching template group.
The method according to claim 11, wherein each selected fine matching template group is taken as a whole, and the second target speckle image group is taken as a whole, and each fine matching template group is calculated with respect to the The similarity between the second target speckle image group includes:

Using the second target speckle image group as a voxel, and each fine matching template group as a voxel, perform three-dimensional calculations on each fine matching template group and the second target speckle image group to obtain each The similarity between a group of fine matching templates and the second target speckle image group, wherein the three-dimensional calculation includes: a cross-correlation operation.
The method according to claim 6 or 12, wherein each voxel is split into a two-dimensional matrix, or split into a one-dimensional sequence for calculation.
The method according to claim 4 or 10, wherein the number of selected fine matching template groups is n, and the matching is respectively matched with the second target speckle image group, and the second target speckle image group is obtained. The fine matching template group with the highest image group similarity, as the secondary matching template group, includes:

For each target speckle image in the second target speckle image group,

Determine n fine matching templates corresponding to the target speckle image in the n fine matching template groups;

Respectively calculating the similarity between the n fine matching templates and the target speckle image, and obtaining the fine matching template with the highest similarity to the target speckle image;

Determine the secondary matching template group according to the i precision matching templates with the highest similarity corresponding to the i target speckle images in the second target speckle image group.
The method according to claim 14, wherein the secondary matching template group is determined according to the i fine matching templates with the highest similarity corresponding to the i target speckle images in the second target speckle image group , Including: obtaining the similarity coefficients corresponding to the i precision matching templates with the highest similarity, and obtaining i similarity coefficients;

Among the i similarity coefficients, the fine matching template corresponding to the similarity coefficient representing the highest similarity is used as the secondary matching template group.
The method according to claim 14, wherein the secondary matching template group is determined according to the i fine matching templates with the highest similarity corresponding to the i target speckle images in the second target speckle image group , Including: determining the depth information corresponding to the i precision matching templates with the highest similarity, and obtaining i depth information;

Calculate the average value of the i pieces of depth information to obtain average depth information, and use the fine matching template corresponding to the average depth information as the secondary matching template.
The method according to any one of claims 1 to 16, wherein R is a positive integer multiple of r.
A depth information detection device, characterized in that the device includes:

An image acquisition module for acquiring a target speckle image group formed by projecting k different reference speckle patterns to the target object;

The rough matching module is used to match the m rough matching template groups with all or part of the target speckle image group respectively, and obtain the rough matching template group with the highest similarity as the primary matching template group, where each matching template The groups correspond to their respective depth information. The interval between every two adjacent rough matching template groups is R, every two adjacent rough matching template groups includes a fine matching template group, and every two adjacent fine matching template groups are separated R, R is greater than r, the same rough matching template group or the same fine matching template group is formed by all or part of the k different reference speckle patterns projected to the same position of the reference screen;

The fine matching module is used to select fine matching template groups within a preset range before and after the primary matching template group, and respectively match all or part of the target speckle image group, to obtain the fine matching template group with the highest similarity, As a secondary matching template group;

The depth information determining module is configured to determine the depth information of the target speckle image according to the depth information of the secondary matching template group.
An electronic device, comprising a memory and a processor, the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor executes The method of any one of claims 1-17.
A computer readable storage medium, wherein the computer readable storage medium stores program code, and the program code can be called by a processor to execute the method according to any one of claims 1-17 .