CN115442590A - Performance analysis method and device, electronic equipment and computer readable storage medium - Google Patents

Abstract

The invention discloses a performance analysis method, a performance analysis device, electronic equipment and a computer-readable storage medium. The performance analysis method comprises the following steps: acquiring the optical signal power S of reflected light received by a light receiving camera, wherein the reflected light is light which is emitted to an object to be detected by a dot matrix projector and reflected to the light receiving camera by the object to be detected; light intensity I based on stray light at G point position _G Calculating the power N of the stray light at the position of the light receiving camera, wherein the stray light at the position of the G point is the light which is emitted to the protective cover by the dot matrix projector and is totally reflected by the protective cover; calculating the signal-to-noise ratio (SNR) of the depth imaging device according to the optical signal power S and the stray light power N; the performance of the depth imaging device is analyzed based on the signal-to-noise ratio, SNR. By analyzing the performance of the structural model of the depth imaging device by using the performance analysis method, the most suitable design parameters are found, and the elimination or reduction of the design parameters is realizedVeiling glare.

Description

Performance analysis method, device, electronic device, and computer-readable storage medium

technical field

The disclosure belongs to the technical field of depth imaging, and in particular relates to a performance analysis method, device, electronic equipment, and computer-readable storage medium.

Background technique

With the improvement of people's living standards, indoor robots based on intelligent navigation solutions gradually enter people's lives, and the 3D perception system is the core part of it, which is used to realize SLAM (synchronous positioning and mapping), obstacle avoidance and other functions. The 3D perception system here mostly adopts active, wide-angle structured light scheme to realize three-dimensional reconstruction of space.

Among them, the depth imaging device with large wide-angle structured light includes a large wide-angle dot matrix projector, a light-receiving camera and a protective cover. However, due to the speckle projection field angle of the dot matrix projector is large, in the actual use of the product, the images collected by the light receiving camera often have light leakage, glare and other stray light problems, which affect the quality of 3D depth reconstruction. These factors will greatly affect the Limiting the application of large wide-angle depth imaging devices on robots.

Contents of the invention

The purpose of the present disclosure is to provide a performance analysis method, device, electronic equipment and computer-readable storage medium. When designing a depth imaging device, the performance analysis method can be used to analyze the performance of the depth imaging device in order to find the most suitable Design parameters, so as to improve the product yield of the depth imaging device, and improve the problems of light leakage, glare and other stray light that often occur in images collected by the light-receiving camera.

The first aspect of the present disclosure provides a performance analysis method for a depth imaging device, the depth imaging device includes a substrate, a functional element and a protective cover, the functional element is arranged on the substrate, and the protective cover is supported on On the substrate, and located on the side of the functional element away from the substrate, the functional element includes dot matrix projectors and light-receiving cameras arranged at intervals; wherein the performance analysis method includes:

Obtaining the optical signal power S of the reflected light received by the light-receiving camera, the reflected light is the light emitted by the dot matrix projector to the object to be measured and reflected to the light-receiving camera by the object to be measured;

Based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position, calculate the stray light power N at the light receiving camera, the G The stray light at the dot position is the light emitted by the dot matrix projector to the protective cover and totally reflected by the protective cover;

calculating the signal-to-noise ratio SNR of the depth imaging device according to the optical signal power S and the stray light power N;

Based on the signal-to-noise ratio SNR, the performance of the depth imaging device is analyzed.

In an exemplary embodiment of the present disclosure, the optical signal power S, the stray light power N, and the signal-to-noise ratio SNR of the depth imaging device satisfy the relationship of the following formula (1), wherein:

The light intensity I _G of the stray light at the G point position, the diameter δ of the image formed by the stray light at the G point position at the light receiving camera, and the stray light power N at the light receiving camera satisfy the following The relation of formula (1), wherein:

In an exemplary embodiment of the present disclosure, the acquiring the optical signal power S of the reflected light received by the light-receiving camera specifically includes:

Obtain the focal length f of the light-receiving camera, the light divergence angle α of the dot matrix projector and the single-point light power Is, the reflectivity Rs of the object to be measured, and the path D of the reflected light, and based on the following Described formula (3) calculates described optical signal power S, wherein:

In an exemplary embodiment of the present disclosure, based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position, the Before receiving the stray light power N at the camera, the performance analysis method also includes:

Obtain the vertical distance d1 between the protective cover and the optical center of the light-receiving camera;

When it is determined that the vertical distance d1 between the protective cover and the optical center of the light-receiving camera is less than a preset distance value, the point G is calculated based on the lens aperture F, focus distance FL, focal length f, and the vertical distance d1 of the light-receiving camera The diameter δ of the image formed by stray light at the light receiving camera at the position, where:

In an exemplary embodiment of the present disclosure, the preset distance is 10 mm.

In an exemplary embodiment of the present disclosure, based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position, the Before receiving the stray light power N at the camera, the performance analysis method also includes:

Obtain the incident light intensity I _i of stray light in the light emitted by the dot matrix projector, and the attenuation coefficient u of the protective cover;

Determine the reflectivity R of the protective cover, the number of internal reflections M and the single internal reflection path L when the stray light in the protective cover is reflected to the G point position, and calculate the stray light at the G point position based on the following formula (5). Light intensity I _G , where:

I _G =I _i *R ^M *exp(-uML) Formula (5).

In an exemplary embodiment of the present disclosure, the determination of the reflectivity R of the protective cover, and the number of internal reflections M and the single internal reflection path L when the stray light in the protective cover is reflected to the G point position specifically includes:

Based on the incident angle θi of the stray light in the light emitted by the dot matrix projector, the refractive index n1 of the medium between the functional element and the protective cover, and the refractive index n2 of the protective cover, it is determined that the stray light enters The refraction angle θt of the protective cover and the reflectivity R of the protective cover;

determining a single internal reflection path L based on the thickness d2 of the protective cover and the refraction angle θt of the protective cover;

Based on the incident angle θi of the stray light, the refraction angle θt and the thickness d2 of the protective cover, the horizontal distance HG between the optical center of the dot matrix projector and the position of the G point, and the distance between the protective cover and the The vertical distance d1 between the optical centers of the light-receiving cameras determines the number of internal reflections M when the stray light in the protective cover is reflected to the G point position, wherein the optical center of the dot matrix projector and the light-receiving camera The optical centers of are located on the same horizontal line.

In an exemplary embodiment of the present disclosure, the incident angle θi of the stray light, the refractive index n1, the refractive index n2, the refraction angle θt of the protective cover, the reflectivity R, the Between the thickness d2, the horizontal distance HG, the vertical distance d1, and the number of internal reflections M, the following formulas (6) to (10) are correspondingly satisfied, wherein:

n1 sinθ _i =n2 sinθ _t formula (6);

R(R ₀ , θ _t )=R ₀ +(1-R ₀ )(1-cosθ _t ) ⁵ formula (7);

HG=d1* _tanθi +M*d2* _tanθt Formula (10).

In an exemplary embodiment of the present disclosure, the obtaining the attenuation coefficient u of the protective cover specifically includes:

Obtain the thickness d2 of the protective cover and its transmittance T, and calculate the attenuation coefficient u of the protective cover based on the following formula (11), where:

In an exemplary embodiment of the present disclosure, the analyzing the performance of the depth imaging device based on the signal-to-noise ratio (SNR) specifically includes:

Comparing the size relationship between the signal-to-noise ratio SNR and the preset ratio;

When the signal-to-noise ratio SNR is less than the preset ratio, it is determined that the performance of the depth imaging device is in a bad state;

When the performance of the depth imaging device is in a bad state, at least part of the parameter information used for calculating the optical signal power S and/or the stray light power N is called out to indicate the at least part of the parameter information The information is adjusted.

In an exemplary embodiment of the present disclosure, the preset ratio is 10dB.

The second aspect of the present disclosure provides a performance analysis device for analyzing the performance of a depth imaging device, the depth imaging device includes a substrate, a functional element and a protective cover, the functional element is arranged on the substrate, the The protective cover is supported on the substrate and is located on the side of the functional element away from the substrate, and the functional element includes dot matrix projectors and light-receiving cameras arranged at intervals; wherein the performance analysis device includes:

An acquisition module, configured to acquire the optical signal power S of the reflected light received by the light-receiving camera, the reflected light is emitted by the dot matrix projector to the object to be measured and reflected to the light-receiving camera by the object to be measured light;

The first calculation module is used to calculate the stray light at the light receiving camera based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position. Optical power N, the stray light at the G point is the light emitted by the dot matrix projector to the protective cover and totally reflected by the protective cover;

The second calculation module is used to calculate the signal-to-noise ratio SNR of the depth imaging device according to the optical signal power S and the stray light power N;

A performance analysis module, configured to analyze the performance of the depth imaging device based on the signal-to-noise ratio (SNR).

A third aspect of the present disclosure provides an electronic device, which includes:

a memory storing computer readable instructions;

The processor reads the computer-readable instructions stored in the memory to execute any one of the performance analysis methods described above.

A fourth aspect of the present disclosure provides a computer-readable storage medium, wherein computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is made to perform any one of the above-mentioned The performance analysis method described above.

The disclosed scheme has the following beneficial effects:

The performance of the depth imaging device is evaluated by the signal-to-noise ratio SNR. The calculation of the signal-to-noise ratio SNR is related to the optical signal power S and the stray light power N received by the optical receiving camera. The optical signal power S is the signal received by the optical receiving camera. The normal signal of the stray light power N refers to the light power totally reflected by the protective cover into the light-receiving camera. This stray light power N is the noise received by the light-receiving camera. When designing the depth imaging device, it can be used first This performance analysis method analyzes the performance of the structural model of the depth imaging device, based on the analysis results, in order to find the most suitable design parameters, avoid or improve the influence of the stray light totally reflected by the protective cover on the light receiving camera, so as to improve the light The images collected by the receiving camera often have light leakage, glare and other stray light problems, which can then improve the yield of the depth imaging device.

Other features and advantages of the present disclosure will become apparent from the following detailed description, or in part, be learned by practice of the present disclosure.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure. Apparently, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative efforts.

FIG. 1 shows a schematic structural diagram of a depth imaging device according to an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of the geometric relationship between the simplified structures of the depth imaging device shown in Fig. 1;

Fig. 3 shows a schematic flow chart of the performance analysis method described in an embodiment of the present disclosure;

4 shows a schematic diagram of the relationship between the signal-to-noise ratio SNR and the path D of reflected light when the number of internal reflections M of the protective cover is 4;

5 shows a schematic diagram of the relationship between the signal-to-noise ratio SNR and the path D of reflected light when the number of internal reflections M of the protective cover is 6;

Fig. 6 shows a structural block diagram of a performance analysis device according to an embodiment of the present disclosure;

Fig. 7 shows a structural block diagram of an electronic device according to an embodiment of the present disclosure.

detailed description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; Fully conveyed to those skilled in the art. The drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more example embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of example embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details being omitted, or other methods, components, steps, etc. may be adopted. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and/or processor means and/or microcontroller means.

Embodiments of the present disclosure provide a depth imaging device, which can be used to realize depth perception, and its main implementation mode is a visual mode based on dot matrix projection. Among them, the depth imaging device of this embodiment can be active monocular structured light or active binocular structured light, as shown in Figure 1, its core structure includes: a dot matrix projector 101 including a large wide angle, a light receiving camera 102 and Protective cover 104 .

It should be understood that, when the depth imaging device is an active binocular structured light, compared with an active monocular structured light solution, it may include two light-receiving cameras 102, which are equally spaced between the dot matrix projector 101 opposite sides.

For example, the dot matrix projector 101 can be an infrared speckle projector, which can be composed of an infrared vcsel (vertical cavity surface) laser, a collimating lens, and a diffractive optical element. To increase the characteristics and uniqueness of the object to be tested, in the design of the product field of view, it adopts a large and wide-angle design, such as 120°×100°, to ensure that the robot can realize the obstacle avoidance function at a short distance. The light-receiving camera 102 can be an infrared receiving camera, which is composed of a photosensitive chip CMOS, an imaging lens and an infrared narrow-band filter, and its resolution can be megapixels, such as 1280×1080, for collecting infrared speckle in the space to be measured picture.

The depth imaging device of this embodiment calculates the parallax offset of the point with the same name through the infrared speckle image of the space to be measured collected by the light-receiving camera 102, and finally performs depth calculation and depth compensation to generate high-resolution, high-precision image depth information.

Wherein, the dot matrix projector 101 and the light receiving camera 102 as a whole can be regarded as functional elements, which are installed on the same side of the same substrate 103, and the installation mode of the dot matrix projector 101 and the light receiving camera 102 can be fixed by screws or It is a glue fixing method. The center distance between the dot matrix projector 101 and the light-receiving camera 102 is called the baseline distance, and the choice of its size is determined by the application scenario. For example, when it is used at a short distance, a short baseline is usually used, and when it is used at a long distance, a long baseline is usually used. .

The protective cover 104 can be supported on the substrate 103 and is located on the side of the functional element away from the substrate 103, that is, on the side of the dot matrix projector 101 and the light receiving camera 102 away from the substrate 103. It should be understood that the protective cover 104 It can be supported on the substrate 103 by a supporting shell surrounding the functional element, and the protective cover 104 is located between the functional element and the object to be measured.

In this embodiment, the protective cover 104 mainly functions as dustproof and waterproof, so as to ensure the stability of the measurement environment of the depth imaging device. Further, the surface of the protective cover 104 can be coated with an AR coating to ensure that most of the light can be projected there without damage, so as to facilitate subsequent generation of high-resolution and high-precision image depth information.

For example, the protective cover 104 of this embodiment can be a glass structure, but it is not limited thereto, and it depends on specific conditions.

In this embodiment, the optical center of the dot matrix projector 101 and the optical center of the light-receiving camera 102 may be located on the same horizontal line, and the horizontal line may be parallel to the substrate 103 . Wherein, due to the existence of the protective cover 104, in the actual use of the product, the light with a large field of view in the dot matrix projector 101 is likely to be totally reflected in the protective cover 104, thereby generating stray light affecting the light receiving camera 102. Light will affect the quality of images captured by the light-receiving camera 102, that is, the images captured by the light-receiving camera 102 often have light leakage, glare and other stray light problems, which seriously affect the quality of 3D depth reconstruction. These factors will severely limit the application of depth imaging devices in the field of robot obstacle avoidance. Therefore, it is particularly important to establish a model for the stray light problem of a large wide-angle depth imaging device, so as to find the most suitable design parameters during product design.

Based on the above problems, this embodiment provides a performance analysis method, which can be used to perform performance analysis on the depth imaging device shown in FIG. Repeat it again.

Wherein, Fig. 2 shows a schematic diagram of the geometric relationship between the simplified structures of the depth imaging device shown in Fig. 1, O1 is the optical center of the dot matrix projector 101 in Fig. , n2 is the refractive index of the protective cover 104, d2 is the thickness of the protective cover 104, d1 is the vertical distance between the optical center O1, O2 and the protective cover 104, n1 is the refraction of the medium (for example: air) on both sides of the protective cover 104 rate, here n1<n2; the horizontal distance between O1 and O2 is the baseline, and a bunch of stray light is simulated in Fig. 2 from the optical center O1 of the dot matrix projector 101 and incident on the n2 medium through the n1 medium, the stray light The angle of incidence is θi, and the angle of refraction is θt, where the positions of points A, B, C, D, E, F, and G in Fig. 2 are the positions where stray light passes through the protective cover 104, points H and K in Fig. 2 The positions are respectively the positions where the optical centers O1 and O2 are orthographically projected on the protective cover 104 .

Specifically, in combination with the contents of FIG. 1 and FIG. 2, the performance analysis method of this embodiment can be specifically shown in FIG. 3, which includes step S200, step S202, step S204, and step S206, wherein:

In step S200, the optical signal power S of the reflected light received by the light-receiving camera 102 is obtained. The reflected light mentioned here is emitted by the dot matrix projector 101 to the object to be measured and reflected to the light-receiving camera by the object to be measured. 102 rays;

In step S202, based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera 102 at the G point position, the stray light at the light receiving camera 102 is calculated. Optical power N, the stray light at the G point is the light emitted by the dot matrix projector 101 to the protective cover 104 and totally reflected by the protective cover 104;

In step S204, according to the optical signal power S and the stray light power N, calculate the signal-to-noise ratio SNR of the depth imaging device;

In step S206, the performance of the depth imaging device is analyzed based on the SNR.

In this embodiment, the performance of the depth imaging device is evaluated by the signal-to-noise ratio SNR. The calculation of the signal-to-noise ratio SNR is related to the optical signal power S and the stray light power N received by the light-receiving camera 102. The optical signal power S It is the normal signal received by the light-receiving camera 102. The stray light power N refers to the light power that is totally reflected by the protective cover 104 and enters the light-receiving camera 102. This stray light power N is the noise received by the light-receiving camera 102, When designing a depth imaging device, this performance analysis method can be used to analyze the performance of the structural model of the depth imaging device, based on the analysis results, in order to find the most suitable design parameters to avoid or improve the stray light totally reflected by the protective cover 104 The impact on the light-receiving camera 102 can improve the problems of light leakage, glare and other stray light in images collected by the light-receiving camera 102, and then improve the yield rate of the depth imaging device.

In step S206, analyzing the performance of the depth imaging device based on the signal-to-noise ratio SNR may specifically include:

Step S2061, comparing the magnitude relationship between the signal-to-noise ratio SNR and a preset ratio;

Step S2062, when the signal-to-noise ratio SNR is less than the preset ratio, determine that the performance of the depth imaging device is in a bad state;

Step S2063. When the performance of the depth imaging device is in a bad state, call out at least part of the parameter information used for calculating the optical signal power S and/or the stray light power N, so as to indicate the At least some parameter information is adjusted.

That is to say, when the signal-to-noise ratio SNR is less than the preset ratio, it means that in the image information collected by the light-receiving camera 102, the stray light power N accounts for a large proportion, and the optical signal power S accounts for a small proportion. The image information has obvious light leakage, glare and other stray light problems, that is, it is determined that the performance of the depth imaging device is in a bad state. Therefore, it is necessary to adjust the structural model of the depth imaging device. In this performance analysis method, When it is determined that the performance of the depth imaging device is in a bad state, at least part of the parameter information used to calculate the optical signal power S and/or stray light power N is called out, so as to indicate that the called out part of the parameter information is performed. Adjust, quickly build the most suitable structural model of the depth imaging device, and improve the design efficiency of the depth imaging device.

Wherein, at least part of the parameter information used to calculate the optical signal power S and/or stray light power N can be called out in the form of displaying these parameters on the terminal, and the operator can manually modify these parameter values, and then Then perform performance analysis; or, call out at least part of the parameter information used to calculate the optical signal power S and/or stray light power N, then automatically adjust and modify the value of this part of parameter information based on the setting program, and then perform performance analysis , until the performance analysis result meets the requirements, that is, the signal-to-noise ratio SNR is greater than or equal to the preset ratio.

For example, the preset ratio mentioned in this embodiment may be 10dB, but it is not limited thereto, and the preset ratio may be adjusted according to actual needs.

In step S206, analyzing the performance of the depth imaging device based on the signal-to-noise ratio SNR may specifically include:

Step S2063. When the signal-to-noise ratio SNR is greater than or equal to the preset ratio, it is determined that the performance of the depth imaging device is in an excellent state. After determining that the performance of the depth imaging device is in an excellent state, the operator can send the parameter information corresponding to the depth imaging device at this time to as the final design parameter.

Wherein, in order to facilitate the operator to obtain these final design parameters, in step S206, analyzing the performance of the depth imaging device based on the signal-to-noise ratio SNR may specifically include:

Step S2064, when it is determined that the performance of the depth imaging device is in an excellent state, display all the parameter information used for calculating the optical signal power S and/or the stray light power N, and the operator can use this parameter information Design a depth imaging device that meets the requirements.

The parameter information combination formula used for calculating the optical signal power S and the stray light power N will be described in detail below.

In this embodiment, the optical signal power S, the stray light power N, and the signal-to-noise ratio SNR of the depth imaging device satisfy the relationship of the following formula (1), wherein:

And the light intensity I _G of the stray light at the position of the G point, the diameter δ of the image formed by the stray light at the light receiving camera 102 at the position of the G point, and the stray light power N at the light receiving camera 102 satisfy the relationship of the following formula (1), where:

Wherein, in step S200, acquiring the optical signal power S of the reflected light received by the light-receiving camera 102 may specifically include:

Obtain the focal length f of the light-receiving camera 102, the light divergence angle α of the dot matrix projector 101 and the single-point light power Is, the reflectivity Rs of the object to be measured, and the path D of the reflected light, and The optical signal power S is calculated based on the following formula (3), wherein:

In this embodiment, by using the focal length f of the light-receiving camera 102, the light divergence angle α of the dot matrix projector 101 and the single-point light power Is, the reflectivity Rs of the object to be measured, and the path D of the reflected light To calculate the optical signal power S obtained by the light-receiving camera 102, compared with the scheme of directly collecting the optical signal of the depth imaging device in use, the value of the optical signal power S is more accurate, so that the subsequent analysis results In addition, when the signal-to-noise ratio SNR does not meet the requirements, it is also convenient to use the focal length f of the light-receiving camera 102, the light divergence angle α of the dot matrix projector 101, the single-point optical power Is, and the reflection of the object to be measured Ratio Rs, and the path D of the reflected light are retrieved to facilitate the subsequent adjustment of these parameter information, so that the final designed depth imaging device can avoid or improve the stray light totally reflected by the protective cover 104 on the light receiving camera 102, so that the image captured by the light-receiving camera 102 often has light leakage, glare and other stray light problems, and then the yield rate of the depth imaging device can be improved.

In step S202, based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera 102 at the G point position, calculate the light intensity at the light receiving camera 102 Before the stray light power N, the performance analysis method also includes:

Step S2011, obtaining the vertical distance d1 between the protective cover 104 and the optical center of the light-receiving camera 102;

Step S2012, when it is determined that the vertical distance d1 between the protective cover 104 and the optical center of the light-receiving camera 102 is less than the preset distance value, based on the lens aperture F of the light-receiving camera 102, the focus distance FL, the focal length f and the vertical distance d1, calculate the diameter δ of the image formed by stray light at the light receiving camera 102 at the position of G point, wherein:

In this embodiment, when it is determined that the vertical distance d1 between the protective cover 104 and the optical center of the light-receiving camera 102 is less than the preset distance value, the position of the G point belongs to the macro distance for the light-receiving camera 102, which is in the light-receiving camera 102. The image formed by the camera 102 can be approximated as a circle of confusion, and the calculation formula of the diameter δ of the circle of confusion can refer to the above-mentioned formula (4), that is to say, the distance between the optical center of the protective cover 104 and the light-receiving camera 102 is determined. When the vertical distance d1 is less than the preset distance value, the diameter δ of the circle of confusion is related to parameters such as the vertical distance d1, the lens aperture F, the focus distance FL, and the focal length f. In order to adjust the diameter δ of the circle of confusion to reduce the stray light power N, Avoiding or improving the impact of stray light totally reflected by the protective cover 104 on the light-receiving camera 102 can be achieved by adjusting these parameters.

For example, the aforementioned preset distance may be 10 mm, but it is not limited thereto, and may also be adjusted according to actual conditions.

In step S202, based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera 102 at the G point position, calculate the light intensity at the light receiving camera 102 Before the stray light power N, the performance analysis method can also include:

Step S2013, obtaining the incident light intensity I _j of the stray light in the light emitted by the dot matrix projector 101 and the attenuation coefficient u of the protective cover 104;

Step S2014, determine the reflectivity R of the protective cover 104, and the number of internal reflections M and the single internal reflection path L when the stray light in the protective cover 104 is reflected to the G point position, and calculate the G point based on the following formula (5) The light intensity I _G of the stray light at the position, where:

I _G =I _i *R ^M *exp(-uML) Formula (5).

Based on the above, it can be known that the light intensity I _G of stray light at point G is related to the parameters of incident light intensity I _i , attenuation coefficient u, reflectivity R, number of internal reflections M and single internal reflection path L, in order to adjust the position of G Adjusting the light intensity I _G of the stray light to reduce the stray light power N, avoiding or improving the impact of the stray light totally reflected by the protective cover 104 on the light receiving camera 102 can be achieved by adjusting these parameters.

Wherein, a single internal reflection path L is taken as an example to illustrate the change of light intensity. For example, when the reflected light in the protective cover 104 is from point B to point C in FIG. In the internal reflection path L, the optical intensity I _c at point C and the optical intensity at point B can be expressed as: I _c = _IB exp(-uL).

Therefore, when going from point A (that is, the incident position of stray light) to point G (that is, the exit position of stray light), the light has been internally reflected M times in the protective cover 104. At this time, the light at point A For the relationship between the intensity (namely: the incident light intensity I _i ) and the light intensity I _G at the point G, reference can be made to the aforementioned formula (5).

In step S2014, determine the reflectivity R of the protective cover 104, and the number of internal reflections M and the single internal reflection path L when the stray light in the protective cover 104 is reflected to the G point position, which may specifically include:

Step S20141, based on the incident angle θi of stray light in the light emitted by the dot matrix projector 101, the refractive index n1 of the medium between the functional element and the protective cover 104, and the refractive index n2 of the protective cover 104, Determine the refraction angle θt of the stray light entering the protective cover 104 and the reflectivity R of the protective cover 104;

Step S20142, based on the thickness d2 of the protective cover 104 and the refraction angle θt of the protective cover 104, determine a single internal reflection path L;

Step S20143, based on the incident angle θi of the stray light, the refraction angle θt and the thickness d2 of the protective cover 104, the horizontal distance HG between the optical center of the dot matrix projector 101 and the position of the G point, and the The vertical distance d1 between the protective cover 104 and the optical center of the light-receiving camera 102 determines the number of internal reflections M when the stray light in the protective cover 104 is reflected to the G point.

In this embodiment, the incident angle θi of stray light, the refractive index n1, the refractive index n2 of the protective cover 104, the thickness d2 of the protective cover 104, the horizontal distance HG and the vertical distance d1 can reflect the reflection of the protective cover 104. The ratio R, the single internal reflection path L and the number of internal reflections M are adjusted, so that the light intensity I _G of the stray light at the G point can be adjusted to reduce the stray light power N, and avoid or improve the protective cover 104. The reflected stray light affects the light-receiving camera 102 .

Specifically, the incident angle θi of the stray light, the refractive index n1, the refractive index n2, the refraction angle θt of the protective cover 104, the reflectivity R, the thickness d2, the horizontal distance HG , the vertical distance d1, and the number of internal reflections M, correspondingly satisfy the following formula (6) to formula (10), wherein:

n1 sinθ _i =n2 sinθ _t formula (6);

R(R ₀ , θ _t )=R ₀ +(1-R ₀ )(1-cosθ _t ) ⁵ formula (7);

HG=d1* _tanθi +M*d2* _tanθt Formula (10).

For example, the medium between the functional element and the protective cover 104 can be air, and the refractive index n1 of the air is equal to 1.

For example, in step S2013, obtaining the attenuation coefficient u of the protective cover 104 may specifically include:

Obtain the thickness d2 of the protective cover 104 and the transmittance T of the protective cover 104, and calculate the attenuation coefficient u of the protective cover 104 based on the following formula (11), wherein:

That is to say, the attenuation coefficient u of the protective cover 104 can be adjusted by adjusting the transmittance T and the thickness d2 of the protective cover 104, so that the light intensity IG of the stray light at the _G point can be adjusted to reduce The stray light power N is used to avoid or improve the influence of the stray light totally reflected by the protective cover 104 on the light-receiving camera 102 .

It should be understood that, based on the content of formula (10) and the corresponding refraction theorem of formula (6), the horizontal distance between the optical center O1 of the dot matrix projector 101 and the optical center O2 of the light receiving camera 102 can be deduced HK, where

HK=2*d1* _tanθi +M*d2* _tanθt Formula (12).

Based on the above content, it can be seen that the parameters involved in the calculation of the signal-to-noise ratio SNR may include the aforementioned lens aperture F, focusing distance FL, focal length f, refractive index n1, refractive index n2, transmittance T, vertical distance d1, Thickness d2, reflectivity Rs, light divergence angle α, single-point optical power Is, path D of reflected light, incident light intensity I _j , incident angle θi, horizontal distance HK, that is, in order to make the signal-to-noise ratio SNR satisfy the design According to requirements, these parameters can be adjusted and designed to avoid or improve the impact of stray light totally reflected by the protective cover 104 on the light-receiving camera 102 .

The above is the manner of completing the depth imaging device by using the performance analysis method. The following is an example of how to eliminate and reduce stray light in the design.

In fact, in the design of depth imaging devices: lens aperture F, focusing distance FL, focal length f, refractive index n1, refractive index n2, transmittance T, vertical distance d1, thickness d2, light divergence angle α, single point light The power Is is a known value. In addition, when the application environment of the depth imaging device product is determined, that is, the reflectivity Rs of the object to be measured is also a known value. According to the aforementioned formulas (1) to (12), given some given parameters, such as f=1.65mm, FL=600mm, F=2, n1=1, n2=1.5, T=0.9, d1= 7mm, d2=1.5mm, Rs=0.6, α=1°, Is=100uW, stray light incident light intensity Ii=50mW, incident angle θi=50°, the change of SNR with the path D of reflected light can be obtained, as follows :

As shown in Figure 4, for the image quality collected by the light-receiving camera 102, when SNR≥10dB, the performance of three-dimensional reconstruction is more guaranteed, so from the perspective of the number of total reflections M=4, it cannot meet the requirements within the scope of use. The signal-to-noise ratio (SNR) specification is greater than 10dB, especially at a long distance, that is, when the path D of the reflected light is particularly large. Obviously, the horizontal distance HK=20 mm between the optical center O1 of the dot matrix projector 101 and the optical center O2 of the light-receiving camera 102 is not enough.

As shown in Figure 5, when the number of total reflections M=6, it can be seen that when the path D of the reflected light is at 4m, the SNR is still far greater than 10dB. The horizontal distance HK=22mm between the centers O2, that is to say, when the horizontal distance HK=22mm between the optical center O1 of the dot matrix projector 101 and the optical center O2 of the light-receiving camera 102, basically there will be no stray light problem .

This embodiment proposes a system modeling and stray light elimination method for a large wide-angle depth imaging device. By establishing a stray light model, that is, using a performance analysis method, using a formula to combine the requirements of the application scene and the modules of the depth imaging device relationship with the design parameters. Designers can eliminate the stray light problem caused by the protective cover 104 of the large wide-angle depth imaging device by adjusting the design parameters of different components. It can be widely used in robot navigation, obstacle avoidance and other fields.

Wherein, when adjusting parameters, the horizontal distance HK between the optical center O1 of the dot matrix projector 101 and the optical center O2 of the light-receiving camera 102 can be adjusted preferentially to adjust the number of internal reflections M, and the adjustment of the horizontal distance HK is also It cannot be that the SNR meets the design requirements, and the incident angle θi of stray light, the thickness d2 of the protective cover 104 , the transmittance T, etc. can be continuously adjusted.

The embodiment of the present disclosure also provides a performance analysis device for analyzing the performance of the aforementioned depth imaging device, and the specific structure of the depth imaging device will not be described in detail here.

Wherein, as shown in FIG. 6 , the performance analysis device includes: an acquisition module 400 , a first calculation module 410 , a second calculation module 420 and a performance analysis module 430 .

1, FIG. 2 and FIG. 6, the acquisition module 400 is used to acquire the optical signal power S of the reflected light received by the light receiving camera 102, the reflected light is emitted to the object to be measured by the dot matrix projector 101, and light reflected by the object to be measured to the light-receiving camera 102;

The first calculation module 410 is used to calculate the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera 102 at the G point position, and calculate the The stray light power N at the position of the G point is the light emitted by the dot matrix projector 101 to the protective cover 104 and totally reflected by the protective cover 104;

The second calculation module 420 is used to calculate the signal-to-noise ratio SNR of the depth imaging device according to the optical signal power S and the stray light power N;

The performance analysis module 430 is used for analyzing the performance of the depth imaging device based on the SNR. Specifically, the acquiring module 400 is used to acquire the focal length f of the light-receiving camera 102, the light divergence angle α and the single-point optical power Is of the dot matrix projector 101, the reflectivity Rs of the object to be measured, and the The path D of the reflected light is calculated, and the optical signal power S is calculated based on the aforementioned formula (3).

Exemplarily, the first calculation module 410 is used to calculate the diameter δ of the image formed by the stray light at the light-receiving camera 102 based on the light intensity I _G of the stray light at the G point position and the stray light at the G point position. Before the stray light power N at the light receiving camera 102, the acquiring module 400 is also used to: acquire the vertical distance d1 between the protective cover 104 and the optical center of the light receiving camera 102; When the vertical distance d1 between the centers is less than the preset distance value, based on the lens aperture F of the light-receiving camera 102, the focus distance FL, the focal length f and the vertical distance d1, the stray light at the G point position is calculated in the light-receiving For the diameter δ of the image formed by the camera 102 , refer to the foregoing formula (4) for details.

In addition, the obtaining module 400 is also used to obtain the incident light intensity I _i of the stray light in the light emitted by the dot matrix projector 101 and the attenuation coefficient u of the protective cover 104; The number of internal reflections M and the single internal reflection path L when the stray light is reflected to the G point position, and the light intensity I _G of the stray light at the G point position is calculated based on the aforementioned formula (5).

It should be understood that the performance analysis device corresponds to the performance analysis method mentioned above, and will not be described in detail here.

An electronic device 50 according to an embodiment of the present disclosure is described below with reference to FIG. 7 . The electronic device 50 shown in FIG. 7 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 7, electronic device 50 takes the form of a general-purpose computing device. Components of the electronic device 50 may include but not limited to: at least one processing unit 510 , at least one storage unit 520 , and a bus 530 connecting different system components (including the storage unit 520 and the processing unit 510 ).

Wherein, the storage unit stores program codes, and the program codes can be executed by the processing unit 510, so that the processing unit 510 executes various exemplary methods according to the present invention described in the description part of the above-mentioned exemplary methods in this specification. Implementation steps. For example, the processing unit 510 may execute various steps in the foregoing performance analysis method.

The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 5201 and/or a cache storage unit 5202 , and may further include a read-only storage unit (ROM) 5203 .

Storage unit 520 may also include a program/utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, Implementations of networked environments may be included in each or some combination of these examples.

Bus 530 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local area using any of a variety of bus structures. bus.

The electronic device 50 can also communicate with one or more external devices 600 (such as keyboards, pointing devices, Bluetooth devices, etc.), and can also communicate with one or more devices that enable the user to interact with the electronic device 50, and/or communicate with Any device (eg, router, modem, etc.) that enables the electronic device 50 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 550 . An input/output (I/O) interface 550 is connected to the display unit 540 . Moreover, the electronic device 50 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 560 . As shown, network adapter 560 communicates with other modules of electronic device 50 via bus 530 . It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of software products, and the software products can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor of a computer, the computer is made to perform the above method Methods described in the Examples section.

According to an embodiment of the present disclosure, there is also provided a program product for implementing the method in the above method embodiment, which may adopt a portable compact disk read-only memory (CD-ROM) and include program codes, and may be used in a terminal devices, such as personal computers. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus or device.

The program product may reside on any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

A computer readable signal medium may include a data signal carrying readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages—such as JAVA, C++, etc., as well as conventional procedural programming languages. Programming language - such as "C" or a similar programming language. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider). business to connect via the Internet).

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. Actually, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided to be embodied by a plurality of modules or units.

In addition, although steps of the methods of the present disclosure are depicted in the drawings in a particular order, there is no requirement or implication that the steps must be performed in that particular order, or that all illustrated steps must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, etc.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or can be implemented by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of software products, and the software products can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

Other embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with the true scope and spirit of the disclosure indicated by the appended claims.

Claims (14)

1. A performance analysis method for a depth imaging device, the depth imaging device comprising a substrate, a functional element and a protective cover, the functional element is arranged on the substrate, and the protective cover is supported on the substrate, And located on the side of the functional element away from the substrate, the functional element includes dot matrix projectors and light-receiving cameras arranged at intervals; it is characterized in that the performance analysis method includes: Obtaining the optical signal power S of the reflected light received by the light-receiving camera, the reflected light is the light emitted by the dot matrix projector to the object to be measured and reflected to the light-receiving camera by the object to be measured; Based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position, calculate the stray light power N at the light receiving camera, the G The stray light at the dot position is the light emitted by the dot matrix projector to the protective cover and totally reflected by the protective cover; calculating the signal-to-noise ratio SNR of the depth imaging device according to the optical signal power S and the stray light power N; Based on the signal-to-noise ratio SNR, the performance of the depth imaging device is analyzed. 2. performance analysis method according to claim 1, is characterized in that, The relationship between the optical signal power S, the stray light power N, and the signal-to-noise ratio SNR of the depth imaging device satisfies the following formula (1), wherein:

The light intensity I _G of the stray light at the G point position, the diameter δ of the image formed by the stray light at the G point position at the light receiving camera, and the stray light power N at the light receiving camera satisfy the following The relation of formula (1), wherein:

3. The performance analysis method according to claim 1, wherein said obtaining the optical signal power S of the reflected light received by the light-receiving camera specifically comprises: Obtain the focal length f of the light-receiving camera, the light divergence angle α of the dot matrix projector and the single-point light power Is, the reflectivity Rs of the object to be measured, and the path D of the reflected light, and based on the following Described formula (3) calculates described optical signal power S, wherein:

4. performance analysis method according to claim 1, is characterized in that, based on the light intensity I _G of stray light at the G point position and the diameter of the image formed by stray light at the light receiving camera at the G point position δ, before calculating the stray light power N at the light receiving camera, the performance analysis method also includes: Obtain the vertical distance d1 between the protective cover and the optical center of the light-receiving camera; When it is determined that the vertical distance d1 between the protective cover and the optical center of the light-receiving camera is less than a preset distance value, the point G is calculated based on the lens aperture F, focus distance FL, focal length f, and the vertical distance d1 of the light-receiving camera The diameter δ of the image formed by stray light at the light receiving camera at the position, where:

5. The performance analysis method according to claim 4, wherein the preset distance value is 10mm. 6. performance analysis method according to claim 1, is characterized in that, based on the light intensity I of stray light at _G point position and the diameter of the image formed by stray light at said light receiving camera at said G point position δ, before calculating the stray light power N at the light receiving camera, the performance analysis method also includes: Obtain the incident light intensity I _i of stray light in the light emitted by the dot matrix projector, and the attenuation coefficient u of the protective cover; Determine the reflectivity R of the protective cover, the number of internal reflections M and the single internal reflection path L when the stray light in the protective cover is reflected to the G point position, and calculate the stray light at the G point position based on the following formula (5). Light intensity I _G , where: I _G =I _i *R ^M *exp(-uML) formula (5). 7. The performance analysis method according to claim 6, characterized in that, the determination of the reflectivity R of the protective cover, and the number of internal reflections M and the single internal reflection path when the stray light in the protective cover is reflected to the G point position L, specifically including: Based on the incident angle θi of the stray light in the light emitted by the dot matrix projector, the refractive index n1 of the medium between the functional element and the protective cover, and the refractive index n2 of the protective cover, it is determined that the stray light enters The refraction angle θt of the protective cover and the reflectivity R of the protective cover; determining a single internal reflection path L based on the thickness d2 of the protective cover and the refraction angle θt of the protective cover; Based on the incident angle θi of the stray light, the refraction angle θt and the thickness d2 of the protective cover, the horizontal distance HG between the optical center of the dot matrix projector and the position of the G point, and the distance between the protective cover and the The vertical distance d1 between the optical centers of the light-receiving cameras determines the number of internal reflections M when the stray light in the protective cover is reflected to the G point position, wherein the optical center of the dot matrix projector and the light-receiving camera The optical centers of are located on the same horizontal line. 8. The performance analysis method according to claim 7, characterized in that, The incident angle θi of the stray light, the refractive index n1, the refractive index n2, the refraction angle θt of the protective cover, the reflectivity R, the thickness d2, the horizontal distance HG, the vertical Between the distance d1 and the number of internal reflections M, the following formulas (6) to (10) are correspondingly satisfied, wherein: _n1sinθi = _n2sinθt formula (6); R(R ₀ ,θ _t )＝R ₀ +(1-R ₀ )(1-cosθ _t ) ⁵ formula (7);

HG=d1* _tanθi +M*d2* _tanθt Formula (10). 9. The performance analysis method according to claim 7, wherein said obtaining the attenuation coefficient u of the protective cover specifically comprises: Obtain the thickness d2 of the protective cover and its transmittance T, and calculate the attenuation coefficient u of the protective cover based on the following formula (11), where:

10. The performance analysis method according to any one of claims 1 to 9, wherein the analyzing the performance of the depth imaging device based on the signal-to-noise ratio (SNR) specifically comprises: Comparing the size relationship between the signal-to-noise ratio SNR and the preset ratio; When the signal-to-noise ratio SNR is less than the preset ratio, it is determined that the performance of the depth imaging device is in a bad state; When the performance of the depth imaging device is in a bad state, at least part of the parameter information used for calculating the optical signal power S and/or the stray light power N is called out to indicate the at least part of the parameter information The information is adjusted. 11. The performance analysis method according to claim 10, wherein the preset ratio is 10dB. 12. A performance analysis device for analyzing the performance of a depth imaging device, the depth imaging device comprising a substrate, a functional element and a protective cover, the functional element is arranged on the substrate, and the protective cover is supported on On the substrate, and located on the side of the functional element away from the substrate, the functional element includes dot matrix projectors and light-receiving cameras arranged at intervals; it is characterized in that the performance analysis device includes: An acquisition module, configured to acquire the optical signal power S of the reflected light received by the light-receiving camera, the reflected light is emitted by the dot matrix projector to the object to be measured and reflected to the light-receiving camera by the object to be measured light; The first calculation module is used to calculate the stray light at the light receiving camera based on the light intensity I _G of the stray light at the G point position and the diameter δ of the image formed by the stray light at the light receiving camera at the G point position. Optical power N, the stray light at the G point is the light emitted by the dot matrix projector to the protective cover and totally reflected by the protective cover; The second calculation module is used to calculate the signal-to-noise ratio SNR of the depth imaging device according to the optical signal power S and the stray light power N; A performance analysis module, configured to analyze the performance of the depth imaging device based on the signal-to-noise ratio (SNR). 13. An electronic device, characterized in that it comprises: a memory storing computer readable instructions; The processor reads the computer-readable instructions stored in the memory to execute the performance analysis method described in any one of claims 1-11. 14. A computer-readable storage medium, characterized in that computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is made to perform any one of claims 1-11 The performance analysis method described.

Images