WO2023083274A1

WO2023083274A1 - Method for improving uneven light intensity of stripes, camera, storage medium, and program

Info

Publication number: WO2023083274A1
Application number: PCT/CN2022/131227
Authority: WO
Inventors: 王挺; 丁有爽; 邵天兰
Original assignee: 梅卡曼德（北京）机器人科技有限公司
Priority date: 2021-11-10
Filing date: 2022-11-10
Publication date: 2023-05-19
Also published as: CN116109492A

Abstract

The present application provides a method for improving uneven light intensity of stripes, a camera, a storage medium, and a program. The method comprises: forming grating stripes on a projection surface by using a structured light depth camera; taking a plurality of calibration points on a straight line which is perpendicular to the grating stripes, wherein the number of the calibration points is greater than the number of the grating stripes; and adjusting the rotation angle speed of a galvanometer of the structured light depth camera in a segmented manner according to a grayscale value of each calibration point. The accuracy of a subsequent image recognition result is improved.

Description

Method for improving uneven light intensity of stripes, camera, storage medium and program

This application claims the priority of the Chinese patent application submitted to the China Patent Office on November 10, 2021, with the application number 202111329098.1. The entire contents are incorporated by reference in this application.

technical field

The present application relates to the technical field of optical cameras, in particular to a method for improving uneven light intensity of grating stripes in a structured light depth camera, and in particular to a method for improving uneven light intensity of stripes, a camera, a storage medium, and a program.

Background technique

Due to its stable performance and low cost, the structured light depth camera can be widely used in 3D (three-dimensional) scanning, somatosensory interaction, gesture recognition and other aspects.

In the process of realizing the concept of the present application, the inventors found that the existing structured light cameras have at least the following problems: the ambient light and the reflectivity are not completely consistent, which will also lead to uneven brightness of the light stripes on the projection surface. The brightness unevenness is specifically manifested as the brightness of the brightest part of the grating stripe is 3 to 4 times the brightness of the darkest part of the grating stripe, of which 30% to 40% of the brightness unevenness is caused by the inconstant movement of the grating stripe, and the remaining The non-uniformity of brightness is due to the influence of ambient light and the inconsistency of the reflectivity when the galvanometer reflects the laser light at different angles. The uneven brightness will directly cause the uneven brightness of the grating stripes collected by the image sensor, which will directly affect the subsequent image processing and analysis results.

Therefore, those skilled in the art urgently need to develop a method for improving the uneven light intensity of stripes, so as to solve the above technical problems.

Contents of the invention

In view of this, the technical problem to be solved in this application is to provide a method for improving the uneven light intensity of stripes, a camera, a storage medium and a program, which solve the problem that the vibrating mirror of the existing structured light depth camera rotates at a constant angular speed, and the projection surface The brightness of the stripes generated on the image is uneven, which directly affects the subsequent image processing and analysis results.

In order to solve the above technical problems, the present application provides a method for improving uneven light intensity of stripes, including:

Using a structured light depth camera to form grating stripes on the projection surface;

Taking a plurality of calibration points along a straight line perpendicular to the grating stripes, wherein the number of the calibration points is greater than the number of the grating stripes;

The rotational angular velocity of the vibrating mirror of the structured light depth camera is adjusted segmentally according to the gray value of each calibration point.

The specific embodiment of the present application also provides a structured light depth camera, including: one or more processors; a storage device for storing executable instructions, and when the executable instructions are executed by the processors, the stripes are realized. Improvement method for uneven light intensity.

The specific embodiment of the present application also provides a computer-readable storage medium on which executable instructions are stored, and when the executable instructions are executed by a processor, a method for improving uneven light intensity of stripes is realized.

A specific embodiment of the present application further provides a computer program, the above-mentioned computer program includes computer-executable instructions, and the computer-executable instructions are used to implement a method for improving uneven light intensity of stripes when executed.

In this application, the rotational angular velocity of the galvanometer is adjusted in sections to compensate the darker areas of the grating stripes, so that the brightness of the grating stripes on the projection surface is uniform, which can at least partially solve the problem of existing structured light depth cameras (ie, grating projection cameras). ) of the galvanometer rotates at a constant angular velocity, and the brightness of the grating stripes generated on the projection surface is not uniform, which directly affects the subsequent image processing and analysis results. The technical effect of analyzing the precision of the results.

It should be understood that the above general description and the following specific embodiments are only exemplary and explanatory, and are not intended to limit the scope of the present application.

Description of drawings

The accompanying drawings that follow are a part of the specification of this application, illustrate example embodiments of the application, and together with the description serve to explain the principles of the application.

FIG. 1 is a schematic structural diagram of a structured light depth camera provided by the present application;

Fig. 2 is a schematic flow chart of a method for improving uneven light intensity of stripes provided by the present application;

FIG. 3 is a schematic flow chart of another method for improving uneven light intensity of stripes provided by the present application;

FIG. 4 is a schematic flow diagram of another method for improving uneven light intensity of stripes provided by the present application;

Fig. 5 is a schematic flow chart of another method for improving uneven light intensity of stripes provided by the present application;

FIG. 6 is a schematic structural diagram of a structured light depth camera provided by the present application;

FIG. 7 is a schematic diagram of multiple calibration points for obtaining grating stripes on the projection surface provided by the present application;

Fig. 8 is a schematic diagram of a plurality of calibration point gray values provided by the present application;

Fig. 9 is a schematic diagram of using a cosine curve to calculate the maximum and minimum values of the gray values of multiple calibration points in Fig. 8;

Fig. 10A is a schematic diagram of the modulated light intensity corresponding to multiple calibration points after screening the maximum value and minimum value in Fig. 9;

Figure 10B is a schematic diagram of the modulated light intensity corresponding to multiple calibration points without screening the maximum and minimum values in Figure 9;

Fig. 11A is a schematic diagram of clustering the modulated light intensities in Fig. 10A to obtain multiple cluster center values;

Fig. 11B is a schematic diagram of clustering the modulated light intensities in Fig. 10B to obtain multiple cluster center values;

Fig. 12A is a schematic diagram of adjusting the rotational angular velocity of the vibrating mirror in sections according to the ratio obtained in Fig. 11A to obtain gray values of multiple calibration points;

FIG. 12B is a schematic diagram of adjusting the rotational angular velocity of the vibrating mirror in sections according to the ratio obtained in FIG. 11B to obtain gray values of multiple calibration points.

Explanation of reference signs:

1 Laser Laser 2 Powell Lens

3 Galvanometer 4 Image Sensor

5 Image Processor 10 Laser Transmitter

20 lens 30 vibrating mirror

40 Image Collector 50 Image Processor

60 drive motor

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly illustrate the spirit of the content disclosed in the application with the accompanying drawings and detailed descriptions. After any person skilled in the art understands the embodiments of the content of the application , when it can be changed and modified by the technology taught in the content of the application, it does not depart from the spirit and scope of the content of the application.

The exemplary embodiments and descriptions of the present application are used to explain the present application, but not to limit the present application. In addition, elements/members with the same or similar numbers used in the drawings and embodiments are used to represent the same or similar parts.

The terms "first", "second", ... etc. used herein do not specifically refer to a sequence or order, nor are they used to limit the present application, but are only used to distinguish elements or operations described with the same technical terms .

Regarding the directional terms used herein, such as: up, down, left, right, front or rear, etc., only refer to the directions of the drawings. Accordingly, the directional terms used are for illustration and not for limitation of the present invention.

As used herein, "comprising", "comprising", "having", "comprising" and so on are all open terms, meaning including but not limited to.

As used herein, "and/or" includes any or all combinations of things.

The "plurality" herein includes "two" and "two or more"; the "multiple groups" herein includes "two groups" and "two or more groups".

The terms "approximately" and "about" used herein are used to modify any quantity or error that may vary slightly, but these slight changes or errors will not change its essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments or other numerical values. Those skilled in the art should understand that the aforementioned values can be adjusted according to actual needs, and are not limited thereto.

FIG. 1 is a schematic structural diagram of a structured light depth camera provided in the present application.

In the embodiment of the present application, as shown in Figure 1, the main components of the structured light depth camera include a laser 1, a Powell lens 2, a vibrating mirror 3, an image sensor 4, and an image processor 5, etc. The laser emits a laser point, and the Powell lens converts the laser The point is transformed into linear light, and the vibrating mirror reflects the linear light. Due to the periodic switching of the laser, as the vibrating mirror rotates at a constant angular velocity, the linear light reflected by the vibrating mirror can be formed on the projection surface such as the object, the ground or the wall. There are light and dark grating stripes, the image sensor collects the grating stripe image on the projection surface, and the image processor analyzes the grating stripe image collected by the image sensor.

When the galvanometer reflects linear light, if the angular velocity of the galvanometer is constant and the laser is switched on and off periodically, the linear light reflected by the galvanometer can form light and dark stripes on the projection surface. For grating stripes, due to the The distance between the different points and the galvanometer is different, so that the farther away from the galvanometer on the projection surface, the faster the grating stripes move and the lower the exposure, resulting in uneven brightness of the grating stripes on the projection surface.

FIG. 2 is a schematic flowchart of a method for improving uneven light intensity of stripes provided by the present application, and the method is applied to a structured light depth camera or a laser camera or an electronic device. Wherein, the electronic device may represent various forms of digital computers. Such as, cellular phones, smart phones, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. As shown in Figure 2, the method includes:

Step S201, using a structured light depth camera to form grating stripes on a projection surface.

In the embodiment of the present application, a structured depth camera is used to form light and dark grating stripes on projection surfaces such as objects, ground, or walls.

Optionally, there is no obstacle between the structured light depth camera and the projection surface, and the projection surface is a plane with no objects on it. It can be considered that the reflectivity of the projection surface is consistent, and the influence of the reflectivity of the projection surface on the brightness of the grating stripes can be basically ignored. Multiple calibration points can be selected within a window size of 3×3.

Step S202, taking a plurality of calibration points along a straight line perpendicular to the grating stripes, wherein the number of calibration points is greater than the number of grating stripes.

In the embodiment of the present application, multiple calibration points are selected along a straight line perpendicular to the grating stripes, the number of the calibration points is greater than the number of the grating stripes, the more the number of calibration points, the more accurate the result, but the greater the amount of data calculation; The fewer the number of calibration points, the less accurate the result and the smaller the calculation.

Step S203, adjusting the rotational angular velocity of the vibrating mirror of the structured light depth camera in sections according to the gray value of each calibration point.

In the embodiment of the present application, the gray value of each calibration point is obtained, specifically, the gray value of each calibration point is calculated based on the ambient light intensity corresponding to each calibration point and the processed modulated light intensity, wherein, after processing The modulated light intensity is calculated from the modulated light intensity corresponding to the calibration point and the cosine value of the preset phase field of the galvanometer, and the rotation of the galvanometer of the structured light depth camera is adjusted in sections according to the gray value of each calibration point angular velocity.

Due to the different angles between the laser emitted by the ambient light intensity and the structured light depth camera and the galvanometer, the reflectivity of the laser reflected by the galvanometer is different, and the distance between the galvanometer and the projection surface is different, so the exposure of the laser reflected by the galvanometer on the projection surface The brightness of different grating stripes is different, that is, the gray value of the calibration point is different, which will directly affect the subsequent image processing and image recognition results. Generally, the larger the distance between the galvanometer and the projection surface, the faster the grating stripes move on the projection surface, the smaller the exposure of the laser reflected by the galvanometer on the projection surface, and the smaller the distance between the galvanometer and the projection surface , the slower the moving speed of the grating stripes on the projection surface, the greater the exposure of the laser reflected by the galvanometer on the projection surface.

In the embodiment of the present application, adjusting the rotational angular velocity of the galvanometer of the structured light depth camera in sections according to the gray value can make the brightness of the grating stripes on the projection surface relatively balanced, that is, make the gray values of the calibration points relatively consistent, and improve the follow-up The accuracy of image recognition results.

Fig. 3 is a schematic flow chart of another method for improving uneven light intensity of stripes provided by the present application. This method is applied to a structured light depth camera or a laser camera or an electronic device. As shown in Fig. 3, the method includes:

Step S301, using a structured light depth camera to form grating stripes on a projection surface.

In this embodiment, step S301 has the same technical features as step S201, and for a specific description, refer to step 201, and details are not repeated here.

In step S302, a plurality of calibration points are uniformly or randomly selected along a straight line perpendicular to the grating stripes, wherein the number of calibration points is more than a preset multiple of the number of grating stripes.

In this embodiment, the number of calibration points is greater than the number of grating stripes, and the number of calibration points is more than a preset multiple of the number of grating stripes. For example, the number of calibration points is 3 times, 4 times, 5 times of the number of grating stripes, etc., which can ensure that each grating stripe has a calibration point. The larger the number of calibration points, the more accurate the result, but the greater the amount of data calculation; the fewer the number of calibration points, the less accurate the result, and the smaller the amount of calculation.

Step S303, adjusting the rotational angular velocity of the vibrating mirror of the structured light depth camera in sections according to the gray value of each calibration point.

In this embodiment, step S303 has the same technical features as step S203, and for a specific description, refer to step 203, which will not be repeated here.

In a possible implementation, the rotational angular velocity of the galvanometer of the structured light depth camera is adjusted in segments according to the gray value of each calibration point, including:

Step S3031, using the cosine curve to calculate the maximum value and minimum value of the gray value of each calibration point.

In the embodiment of the present application, the gray value of each calibration point can be calculated using a cosine curve, specifically, the gray value of each calibration point is calculated based on the ambient light intensity corresponding to each calibration point and the processed modulated light intensity, where , the processed modulated light intensity is calculated from the modulated light intensity corresponding to the calibration point and the cosine value of the preset phase field of the galvanometer, wherein, when the cosine value is equal to 1, the maximum value of the gray value of the calibration point is obtained; When the cosine value is equal to 0, the minimum value of the gray value of the calibration point is obtained.

Optionally, using a cosine curve to calculate the maximum value and minimum value of the gray value of each calibration point, including:

Step S20311, obtaining the ambient light intensity and modulated light intensity corresponding to each calibration point.

In this embodiment, the ambient light intensity corresponding to each calibration point and the corresponding modulated light intensity are acquired.

Step S20312, obtain the processed modulated light intensity according to the modulated light intensity and the cosine value of the preset phase field of the vibrating mirror.

In this embodiment, the product of the modulated light intensity corresponding to each calibration point and the cosine value of the preset phase field of the vibrating mirror is calculated to obtain the processed modulated light intensity.

Step S20313, according to the processed modulated light intensity and ambient light intensity, obtain the maximum value and minimum value of the gray value of the calibration point; wherein, when the preset phase field is 0, obtain the minimum value of the gray value of the calibration point , when the default phase field is 1, the maximum value of the gray value of the calibration point is obtained.

In this embodiment, the processed modulated light intensity and ambient light intensity are substituted into formula (1) to calculate the gray value of the calibration point, and formula (1) is expressed as:

I(x,y)=I′(x,y)+I″(x,y)cos[θ(x,y)] Formula (1)

Among them, I(x, y) represents the gray value of the calibration point; I′(x, y) represents the ambient light intensity corresponding to the calibration point; I″(x, y) represents the modulation light intensity corresponding to the calibration point, and the modulation light Intensity is the light intensity of the laser emitted by the structured light depth camera corresponding to the calibration point after adjusting the rotation angular velocity of the galvanometer of the structured light depth camera, reflected by the galvanometer and projected onto the projection surface; θ(x,y) represents the light intensity of the galvanometer The preset phase field of , cos[θ(x,y)] represents the cosine value of the preset phase field of the vibrating mirror, I″(x,y)cos[θ(x,y)] represents the modulated light intensity after processing .

When adjusting the rotation angular velocity of the galvanometer of the structured light depth camera, when the preset phase field is 1, that is, when cos[θ(x,y)] is equal to 1, the maximum value of the gray value of the calibration point is expressed as I _max (x ,y)=I′(x,y)+I″(x,y); when the default phase field is 0, that is, when cos[θ(x,y)] is equal to 0, the minimum gray value of the calibration point Values are expressed as I _min (x,y)=I'(x,y).

Step S3032: Calculate the modulated light intensity corresponding to each calibration point according to the maximum value and the minimum value.

In the embodiment of the present application, the modulated light intensity corresponding to each calibration point is calculated according to the maximum value of the gray value of each calibration point and the minimum value of the gray value of each calibration point, and the distance is found in the horizontal direction (X-axis direction). The calibration point corresponding to the nearest minimum value of a maximum value, the minimum value is used as the background light intensity of the calibration point corresponding to the maximum value, and the difference between the maximum value and the minimum value is obtained to obtain the corresponding calibration point Modulates light intensity.

In a possible implementation, the modulation light intensity corresponding to each calibration point is calculated according to the maximum value and the minimum value, including:

Step S30321, calculating the difference between the maximum value and the minimum value of the gray value of each calibration point, and determining the difference as the modulated light intensity corresponding to each calibration point.

In the present embodiment, the calibration point gray value is calculated according to the formula (2), and the formula (2) is expressed as:

I(x,y)=I′(x,y)+I″(x,y)cos[θ(x,y)] Formula (2)

The maximum value and the minimum value of the gray value of each calibration point are calculated by the formula (2), and the maximum value of the gray value of each calibration point and the minimum value of the gray value are substituted into the formula ( 3):

I _max (x,y)-I _min (x,y)=I′(x,y)+I″(x,y)-I′(x,y)=I″(x,y) Formula (3 )

Among them, the maximum value of the gray value of the calibration point is expressed as I _max (x, y) = I′ (x, y) + I″ (x, y), and the minimum value of the gray value of the calibration point is expressed as I _min (x, y)=I'(x, y), the difference between the maximum value and the minimum value is the modulated light intensity corresponding to the calibration point.

Step S3033, clustering the modulated light intensities corresponding to the multiple calibration points to obtain multiple cluster center values.

In the embodiment of the present application, the entire rotation cycle of the galvanometer of the structured light depth camera is adjusted in segments, so the modulated light intensities corresponding to multiple calibration points can be clustered to obtain multiple cluster center values. Then, the entire rotation period of the galvanometer can be divided into multiple segments by using multiple cluster center values.

Step S3034, adjusting the rotation angular velocity of the galvanometer of the structured light depth camera in segments according to multiple cluster center values, where each cluster center value corresponds to a rotation stage in the segment.

In the embodiment of this application, the ratio between the cluster center values can be calculated, for example, the following ratio can be obtained:

1:1.412251778:2.543979698:4.643095647:9.454755853:23.4681388

In the embodiment of the present application, the rotational angular velocity of the oscillating mirror of the structured light depth camera can be adjusted according to the above-mentioned ratio at different rotational stages of the galvanometer, wherein each cluster center value corresponds to a rotational stage in the segment, for example, a ratio of 1 corresponds to For the first rotation stage, the ratio 1.412251778 corresponds to the second rotation stage, the ratio 2.543979698 corresponds to the third rotation stage, the ratio 4.643095647 corresponds to the fourth rotation stage, the ratio 9.454755853 corresponds to the fifth rotation stage, and the ratio 23.4681388 corresponds to the sixth rotation stage. In the first rotation stage of the galvanometer, the rotational angular velocity of the galvanometer is ω. In the second rotational stage of the galvanometer, the rotational angular velocity of the galvanometer is 1.412251778ω. In the third rotational stage of the galvanometer, the rotational angular velocity of the galvanometer is 2.543979698 ω; in the fourth rotation stage of the vibrating mirror, the angular velocity of the vibrating mirror is 4.643095647ω; in the fifth rotating stage of the vibrating mirror, the angular velocity of the vibrating mirror is 9.454755853ω; The rotational angular velocity is 23.4681388ω.

In the embodiment of the present application, adjusting the rotational angular velocity of the oscillating mirror of the structured light depth camera according to the ratio of the above-mentioned clustering center values at different rotational stages of the galvanizing mirror can accurately improve the uneven light intensity of the fringes, laying a solid foundation for subsequent image recognition and analysis. good foundation.

Figure 4 is a schematic flow chart of another method for improving uneven light intensity of stripes provided by the present application. This method is applied to a structured light depth camera or a laser camera or an electronic device. Calculate the modulated light intensity corresponding to each calibration point, including the following steps:

Step S401, using the theoretical maximum value to remove invalid values in the maximum value to obtain an effective maximum value.

In this embodiment, a preset theoretical maximum value is obtained. For example, the theoretical maximum value can be set to 11. If the value in the maximum value is less than 11, the invalid maximum value is removed to obtain an effective maximum value. It should be noted that the theoretical maximum value can be set according to the actual situation, and is not limited to the above-mentioned value.

Step S402, using the theoretical minimum value to remove invalid values in the minimum value to obtain an effective minimum value.

In the embodiment of the present application, a preset theoretical minimum value is obtained. For example, the theoretical minimum value can be set to 12. If the value of the minimum value is greater than 12, the invalid minimum value is removed to obtain an effective minimum value. It should be noted that the theoretical minimum value can be set according to the actual situation, and is not limited to the above-mentioned values.

Step S403: Calculate the difference between the effective maximum value and the effective minimum value of the gray value of each calibration point, and determine the difference as the modulated light intensity corresponding to each calibration point.

In the embodiment of this application, the effective maximum value and the effective minimum value of the gray value of each calibration point are substituted into the formula (4):

I _max (x,y)-I _min (x,y)=I′(x,y)+I″(x,y)-I′(x,y)=I″(x,y) Formula (4 )

Among them, the effective maximum value of the gray value of the calibration point is expressed as I _max (x, y) = I′ (x, y) + I″ (x, y), and the effective minimum value of the gray value of the calibration point is expressed as I _min (x, y) = I'(x, y), the difference between the effective maximum value and the effective minimum value is the modulated light intensity corresponding to the calibration point, and the modulated light intensity corresponding to each calibration point is obtained.

In the embodiment of the present application, the theoretical maximum value is used to remove invalid values in the maximum value, and the theoretical minimum value is used to remove invalid values in the minimum value. Using the effective maximum value and the effective minimum value to calculate the modulated light intensity can improve the accuracy of subsequent analysis. After removing invalid values, calculate the modulated light intensity corresponding to each calibration point, and then cluster all modulated light intensities to obtain the ratio between all cluster center values, and adjust the structured light according to the ratio in different rotation stages of the galvanometer The rotational angular velocity of the galvanometer of the depth camera can further improve the uneven light intensity of the stripes, laying a good foundation for subsequent image recognition and analysis.

Fig. 5 is a schematic flow chart of another method for improving uneven light intensity of stripes provided by the present application. This method is applied to a structured light depth camera or a laser camera or an electronic device. As shown in Fig. 5, according to multiple cluster center values Adjusting the rotation angular velocity of the galvanometer of the structured light depth camera in sections includes the following steps:

Step S501, find the smallest cluster center value from multiple cluster center values as the reference cluster center value.

In the embodiment of this application, the smaller the cluster center value, the smaller the grayscale value, and the rotational angular velocity of the vibrating mirror should be reduced; the larger the clustering central value, the larger the grayscale value, and the rotational angular velocity of the vibrating mirror should be increased , find the smallest cluster center value from multiple cluster center values, and use it as the benchmark cluster center value.

Step S502, calculating the ratio of each cluster center value to the reference cluster center value.

In the embodiment of this application, the ratio of each cluster center value to the reference cluster center value is calculated, for example, the following values can be:

1:1.412251778:2.543979698:4.643095647:9.454755853:23.4681388

1: 1.237281749: 1.632876235: 2.308164404: 3.063120874: 4.711006726

Step S503, adjusting the rotation angular velocity of the vibration mirror of the structured light depth camera at different rotation stages of the vibration mirror according to each ratio.

In the embodiment of this application, the smaller the cluster center value, the smaller the grayscale value, and the rotational angular velocity of the vibrating mirror should be reduced; the larger the clustering central value, the larger the grayscale value, and the rotational angular velocity of the vibrating mirror should be increased . For example, in the first rotation stage of the vibrating mirror, the rotational angular velocity of the vibrating mirror is ω, then in the second rotating stage of the vibrating mirror, the rotational angular velocity of the vibrating mirror is 1.412251778ω; The rotation angular velocity is 2.543979698ω. In the fourth rotation stage of the galvanometer, the rotation angular velocity of the galvanometer is 4.643095647ω. In the fifth rotation stage of the galvanometer, the rotation angular velocity of the galvanometer is 9.454755853ω. In the sixth rotation stage of the galvanometer , the rotational angular velocity of the vibrating mirror is 23.4681388ω. For another example, in the first rotation stage of the vibrating mirror, the rotational angular velocity of the vibrating mirror is ω, then in the second rotating stage of the vibrating mirror, the rotational angular velocity of the vibrating mirror is 1.237281749ω, The rotation angular velocity of the vibration mirror is 1.632876235ω. In the fourth rotation stage of the vibration mirror, the rotation angular velocity of the vibration mirror is 2.308164404ω. In the fifth rotation stage of the vibration mirror, the rotation angular velocity of the vibration mirror is 3.063120874ω. stage, the rotational angular velocity of the galvanometer is 4.711006726ω.

In the embodiment of the present application, adjusting the rotation angular velocity of the vibration mirror of the structured light depth camera according to the ratio of all cluster center values to the reference cluster center value at different rotation stages of the vibration mirror can accurately improve the unevenness of the light intensity of the stripes, as Follow-up image recognition and analysis lay a good foundation.

In a possible implementation manner, adjusting the rotation angular velocity of the vibration mirror of the structured light depth camera at different rotation stages of the vibration mirror according to each ratio includes:

Step S5031, divide the rotation range of the vibrating mirror into rotation intervals corresponding to the ratios according to a plurality of ratios, wherein the rotation range is composed of rotation intervals.

In this embodiment, if the ratio is: 1:1.237281749:1.632876235:2.308164404:3.063120874, the above ratio is used to divide the rotation range of the galvanometer into rotation intervals corresponding to the ratio, and the rotation range is composed of rotation intervals.

Step S5032, adjust the rotational angular velocity of the oscillating mirror in the corresponding rotational interval according to the ratio.

In the embodiment of the present application, the rotational angular velocity of the vibrating mirror in the corresponding rotational interval is further adjusted according to the ratio, and the rotational angular velocity of the vibrating mirror of the structured light depth camera is adjusted at different rotational stages of the vibrating mirror, which can accurately improve the unevenness of the fringe light intensity, Lay a good foundation for subsequent image recognition and image analysis, and improve the accuracy of image recognition.

In a possible implementation manner, adjusting the rotational angular velocity of the vibrating mirror in the corresponding rotational interval according to the ratio includes:

Step S50321, sorting the N ratios according to a preset order to obtain ratio rankings M ₁ , M ₂ , M ₃ . . . M _N ; wherein, N is a positive integer; M is a ratio.

In this embodiment, the N ratios are sorted in descending order, and the ratios of the ratios sorted are: 1:1.237281749:1.632876235:2.308164404:3.063120874.

Step S50322, processing the ratio ranking to obtain the target ratio rankings M _N , M _N-1 , M _N -2 . . . M ₁ , . . . M _N-2 , M _N-1 , M _N .

In this embodiment, since the rotation trajectory of the galvanometer of the structured light depth camera is fan-shaped, for example, 1: 1.237281749: 1.632876235: 2.308164404: 3.063120874 is processed to obtain the target ratio sorting 3.063120874, 2.308164404, 1.632876235, 1.237 281749, 1, 1.237281749 , 1.632876235, 2.308164404, 3.063120874.

Step S50323, divide the rotation range into 2N-1 rotation intervals, and each rotation interval corresponds to the ratio in the target ratio sorting one by one.

In this embodiment, the rotation range is divided into 2N-1 rotation intervals. If the ratio number N is 5, the rotation interval is 9, and the above target ratios are used to sort 3.063120874, 2.308164404, 1.632876235, 1.237281749, 1, 1.237281749, 1.632876235, 2.308164404, 3.063120874 can divide a rotation period of the vibration mirror of the structured light depth camera into 9 intervals, such as [0,259), [259,455), [455,627), [627,855), [855.1272), [1272,1532) ,[1532,1616),[1616,1770),[1770,2048).

Further, each rotation interval is in one-to-one correspondence with the ratio in the target ratio sorting, the rotation angular velocity of the oscillating mirror of the structured light depth camera in the rotation interval [0,259) is 3.063120874ω, and the oscillating mirror of the structured light depth camera is in the rotation interval [259,455) The rotation angular velocity of the structured light depth camera is 2.308164404ω, the rotation angular velocity of the oscillating mirror of the structured light depth camera in the rotation interval [455,627) is 1.632876235ω, the rotation angular velocity of the oscillating mirror of the structured light depth camera in the rotation interval [627,855) is 1.237281749ω, the vibration mirror of the structured light depth camera is The rotation angular velocity of the mirror in the rotation interval [855.1272) is 1ω, the rotation angular velocity of the oscillating mirror of the structured light depth camera in the rotation interval [1272,1532) is 1.237281749ω, the rotation of the oscillating mirror of the structured light depth camera in the rotation interval [1532,1616) The angular velocity is 1.632876235ω, the rotational angular velocity of the vibrating mirror of the structured light depth camera in the rotation interval [1616,1770) is 2.308164404ω, and the rotational angular velocity of the vibrating mirror of the structured light depth camera in the rotational interval [1770,2048) is 3.063120874ω.

In the embodiment of the present application, adjusting the rotational angular velocity of the galvanometer of the structured light depth camera at different rotation stages of the galvanometer can accurately improve the uneven light intensity of stripes, lay a good foundation for subsequent image recognition and image analysis, and improve image recognition. the accuracy.

Step S503a, performing normalization processing on multiple ratios to obtain multiple normative ratios.

In the embodiment of the present application, for the convenience of engineering implementation, the comparison value may be normalized to obtain the standard ratio. For example, after normalizing the comparison value 1:1.237281749:1.632876235:2.308164404:3.063120874:4.711006726, we get 1:1.25:1.5:2.0:3.0, discarding 4.711006726.

Step S503b, dividing the rotation range of the vibrating mirror into rotation intervals corresponding to the standard ratios according to a plurality of standard ratios, wherein the rotation range is composed of rotation intervals.

In the embodiment of this application, since the rotation trajectory of the galvanometer of the structured light depth camera is fan-shaped, the standardized ratio of 3, 2, 1.5, 1.25, 1, 1.25, 1.5, 2, 3 can be used to convert the structured light depth camera to A rotation period of the galvanometer is divided into 9 intervals, which are [0,259), [259,455), [455,627), [627,855), [855.1272), [1272,1532), [1532,1616), [1616,1770 ), [1770, 2048).

Step S503c, adjusting the rotational angular velocity of the vibrating mirror in each rotational interval according to a plurality of standard ratios.

In the embodiment of this application, the rotational angular velocity of the oscillating mirror of the structured light depth camera is 3ω in the rotation interval [0,259), the rotational angular velocity of the oscillating mirror of the structured light depth camera is 2ω in the rotational interval [259,455), and the oscillating mirror of the structured light depth camera is at The rotation angular velocity of the rotation interval [455,627) is 1.5ω, the rotation angular velocity of the vibration mirror of the structured light depth camera in the rotation interval [627,855) is 1.25ω, the rotation angular velocity of the vibration mirror of the structured light depth camera is 1ω in the rotation interval [855.1272), the structured light The rotational angular velocity of the galvanometer of the depth camera is 1.25ω in the rotation interval [1272,1532), the rotational angular velocity of the galvanometer of the structured light depth camera is 1.5ω in the rotational interval [1532,1616), and the oscillating mirror of the structured light depth camera is in the rotational interval [1616, 1770) with a rotation angular velocity of 2ω, and the oscillating mirror of the structured light depth camera in the rotation interval [1770, 2048) with a rotation angular velocity of 3ω.

In an embodiment of the present application, a structured light depth camera may include: one or more processors; a storage device for storing executable instructions, and when the executable instructions are executed by the processors, the stripes in the above embodiments are realized Improvement method for uneven light intensity.

In an embodiment of the present application, a laser camera may include: one or more processors; a storage device for storing executable instructions, and when the executable instructions are executed by the processors, realize the fringe light intensity in the above-mentioned embodiments Uneven method of improvement.

In the embodiment of the present application, an electronic device may include: one or more processors; a storage device for storing executable instructions, and when the executable instructions are executed by the processors, realize the stripe light intensity in the above embodiments Uneven method of improvement.

In an embodiment of the present application, a computer-readable storage medium stores executable instructions thereon, and when the executable instructions are executed by a processor, the method for improving uneven light intensity of stripes disclosed in the above-mentioned embodiments is implemented. The computer-readable storage medium may be included in the device/apparatus/system described in the above embodiments; it may also exist independently without being assembled into the device/apparatus/system. The above-mentioned computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method for improving uneven light intensity of stripes disclosed in the above-mentioned embodiments is realized.

In an embodiment of the present application, a computer program includes computer-executable instructions, and the computer-executable instructions are used to implement the method for improving uneven light intensity of stripes disclosed in the above-mentioned embodiments when executed. The computer program contains program code for carrying out the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network, and/or from removable media. When the computer program is executed by the processor, the above-mentioned functions defined in the electronic device in the embodiment of the present application are executed. According to the embodiments of the present application, the above-described electronic devices, devices, devices, modules, units, etc. may be implemented by computer program modules.

FIG. 6 is a schematic structural diagram of a structured light depth camera provided by the present application.

In the embodiment of the present application, as shown in FIG. 6 , the structured light depth camera may include a laser emitter 10 , a lens 20 (for example, a Powell lens), a vibrating mirror 30 , an image collector 40 , an image processor 50 and a drive motor 60 . Wherein, the point laser light emitted by the laser transmitter 10 passes through the lens 20 and becomes a line laser, and the line laser is reflected by the vibrating mirror 30 and projected on the projection surface; the driving motor 60 drives the vibrating mirror 30 to rotate at a constant speed, and the laser transmitter 10 cycles switch, Form grating stripes on the projection surface; within an exposure time of the image collector 40, collect the grating stripes on the projection surface; the image processor 50 analyzes the gray value of the marked point on the grating stripe, and controls the drive motor according to the gray value 60, and then adjust the rotational angular velocity of the vibrating mirror 30 in sections.

FIG. 7 is a schematic diagram of obtaining multiple calibration points of grating stripes on a projection plane provided by the present application.

In the embodiment of the present application, as shown in FIG. 7 , the x-axis is the overall width of all the grating stripes, and y is the height of the grating stripes. The structured light depth camera forms grating stripes on the projection surface, and takes multiple calibration points on the straight line of y=1200. If there are no obstacles on the projection surface, it can be assumed that the reflectivity is relatively consistent, and the influence of reflectivity can be basically ignored.

FIG. 8 is a schematic diagram of gray values of multiple calibration points provided by the present application.

In the embodiment of the present application, as shown in FIG. 8 , the gray value of each calibration point in FIG. 7 is acquired, the y-axis represents the size of the gray value, and the x-axis represents a rotation period of the vibrating mirror. The gray value I(x,y) can be expressed by the following formula:

I(x,y)=I'(x,y)+I"(x,y)cos[θ(x,y)]

In the above formula, I(x,y) represents the gray value of the calibration point; I′(x,y) represents the ambient light intensity corresponding to the calibration point; I″(x,y) represents the modulation light intensity corresponding to the calibration point, The modulated light intensity is the light intensity of the laser light emitted by the structured light depth camera corresponding to the calibration point that is reflected by the galvanometer and projected onto the projection surface after adjusting the rotational angular velocity of the oscillating mirror of the structured light depth camera; θ(x,y) represents The preset phase field of the galvanometer,

cos[θ(x,y)] represents the cosine value of the preset phase field of the vibrating mirror, and I″(x,y)cos[θ(x,y)] represents the modulated light intensity after processing.

FIG. 9 is a schematic diagram of calculating the maximum and minimum values of the gray values of multiple calibration points in FIG. 8 by using a cosine curve.

In the embodiment of the present application, as shown in FIG. 9 , a cosine curve may be used to calculate the maximum value and the minimum value of the gray value of each calibration point. When cos[θ(x,y)] is equal to 1, the maximum value of the gray value of the calibration point is expressed as I _max (x,y)=I′(x,y)+I″(x,y); cos[ When θ(x, y)] is equal to 0, the minimum value of the gray value of the calibration point is expressed as I _min (x, y) = I'(x, y).

FIG. 10A is a schematic diagram of modulated light intensities corresponding to multiple calibration points after filtering the maximum value and minimum value in FIG. 9 .

In the embodiment of the present application, as shown in Figure 10A, the theoretical maximum value is used to remove invalid values in the maximum value, and the theoretical minimum value is used to remove invalid values in the minimum value. The theoretical minimum point basically satisfies I<12, and the theoretical maximum point basically satisfies I>11. On the premise of removing some invalid gray values, the accuracy of subsequent analysis can be improved.

FIG. 10B is a schematic diagram of the modulated light intensity corresponding to multiple calibration points without screening the maximum and minimum values in FIG. 9 .

In the embodiment of the present application, as shown in FIG. 10B , it is also possible to directly perform subsequent processing without removing invalid gray values, reduce the amount of data processing, improve the response speed of the structured light depth camera, and improve user experience.

FIG. 11A is a schematic diagram of clustering the modulated light intensities in FIG. 10A to obtain multiple cluster center values. FIG. 11B is a schematic diagram of clustering the modulated light intensities in FIG. 10B to obtain multiple cluster center values.

In the embodiment of the present application, as shown in FIG. 11A , on the premise of removing some invalid gray values, clustering can be performed on each modulated light intensity to obtain multiple cluster center values, which facilitates the implementation of the present application.

In the embodiment of the present application, as shown in FIG. 11B , on the premise of not removing some invalid gray values, each modulated light intensity may be directly clustered to obtain multiple cluster center values.

In the schematic diagrams shown in Figure 11A and Figure 11B, each modulated light intensity is clustered, and the rotation angular velocity of the vibrating mirror is adjusted in segments by using multiple clustering center values, instead of directly using the modulated light intensity to control the vibrating mirror. The angular velocity of rotation greatly reduces the complexity of engineering realization, thereby facilitating the realization of this application.

FIG. 12A is a schematic diagram of adjusting the rotational angular velocity of the vibrating mirror in sections according to the ratio obtained in FIG. 11A to obtain gray values of multiple calibration points. FIG. 12B is a schematic diagram of adjusting the rotational angular velocity of the vibrating mirror in sections according to the ratio obtained in FIG. 11B to obtain gray values of multiple calibration points.

In the embodiment of the present application, as shown in FIG. 12A and FIG. 12B , the ratio between each cluster center value can be calculated, and the rotational angular velocity of the vibrating mirror can be adjusted in sections according to the ratio to obtain gray values of multiple calibration points. The gray values of the obtained multiple calibration points are basically consistent, which can basically eliminate the background light, and the problem that the distance between the galvanometer and the projection surface is different, and the brightness of the grating stripes on the projection surface is uneven, which can improve the accuracy of subsequent image analysis. Spend.

In the embodiment of the present application, the rotational angular velocity of the galvanometer of the structured light depth camera is adjusted in sections according to the gray value, so that the brightness of the grating stripes on the projection surface is relatively balanced, that is, the gray value of the calibration point is relatively consistent or uniform, Improve the accuracy of subsequent image recognition results. Cluster the modulated light intensities corresponding to all the calibration points, and adjust the rotational angular velocity of the vibrating mirror in segments by using multiple cluster center values, instead of directly using the modulated light intensity corresponding to the calibration points to control the rotational angular velocity of the vibrating mirror, which can reduce the The complexity of engineering realization is beneficial to the realization and popularization of this application.

The flowcharts in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that includes one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions.

Those skilled in the art can understand that various combinations and/or combinations of the features described in the various embodiments and/or claims of the present application can be performed, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teaching of the present application, various combinations and/or combinations can be made of the features described in the various embodiments and/or claims of the present application. All such combinations and/or combinations fall within the scope of the present application.

The embodiments of the present application have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the application. Although the various embodiments have been described separately above, this does not mean that the measures in the various embodiments cannot be advantageously used in combination. The scope of the application is defined by the claims appended hereto and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present application, and these substitutions and modifications should all fall within the scope of the present application.

Claims

A method for improving uneven light intensity of stripes, comprising:

Using a structured light depth camera to form grating stripes on the projection surface;

Taking a plurality of calibration points along a straight line perpendicular to the grating stripes, wherein the number of the calibration points is greater than the number of the grating stripes;

The rotational angular velocity of the vibrating mirror of the structured light depth camera is adjusted segmentally according to the gray value of each calibration point.
The method for improving uneven light intensity of stripes according to claim 1, wherein said segmental adjustment of the rotational angular velocity of the vibrating mirror of the structured light depth camera according to the gray value of each of the calibration points comprises:

Using a cosine curve to calculate the maximum and minimum values of the gray value of each calibration point;

calculating the modulated light intensity corresponding to each of the calibration points according to the maximum value and the minimum value;

Clustering the modulated light intensities corresponding to multiple calibration points to obtain multiple cluster center values;

The rotational angular velocity of the galvanometer of the structured light depth camera is adjusted segmentally according to the plurality of cluster center values, wherein each cluster center value corresponds to a rotation stage in the segment.
The method for improving uneven light intensity of stripes according to claim 2, wherein said calculating the maximum value and minimum value of the gray value of each said calibration point by using a cosine curve comprises:

Obtain the ambient light intensity and modulation light intensity corresponding to each calibration point;

Obtain the processed modulated light intensity according to the modulated light intensity and the cosine value of the preset phase field of the vibrating mirror;

According to the processed modulated light intensity and the ambient light intensity, the maximum value and the minimum value of the gray value of the calibration point are obtained; wherein, when the preset phase field is 0, the gray value of the calibration point is obtained Minimum value, when the default phase field is 1, the maximum value of the gray value of the calibration point is obtained.
The method for improving uneven light intensity of stripes according to claim 2, wherein the calculation of the modulated light intensity corresponding to each of the calibration points according to the maximum value and the minimum value includes:

Calculate the difference between the maximum value and the minimum value of the gray value of each of the calibration points, and determine the difference as the modulated light intensity corresponding to each of the calibration points.
The method for improving uneven light intensity of stripes according to claim 2, wherein the calculation of the modulated light intensity corresponding to each of the calibration points according to the maximum value and the minimum value includes:

Using a theoretical maximum value to remove invalid values in the maximum value to obtain an effective maximum value;

using a theoretical minimum to remove invalid values from said minimum to obtain an effective minimum;

Calculate the difference between the effective maximum value and the effective minimum value of the gray value of each calibration point, and determine the difference as the modulated light intensity corresponding to each calibration point.
The method for improving uneven light intensity of stripes according to claim 2, wherein said adjusting the rotation angular velocity of the vibrating mirror of the structured light depth camera according to a plurality of said clustering center values, comprising:

Finding the smallest cluster center value from multiple cluster center values as the benchmark cluster center value;

calculating the ratio of each cluster center value to the reference cluster center value;

Adjusting the rotation angular velocity of the vibration mirror of the structured light depth camera at different rotation stages of the vibration mirror according to each of the ratios.
The method for improving uneven light intensity of stripes according to claim 6, wherein said adjusting the rotation angular velocity of the vibration mirror of the structured light depth camera at different rotation stages of the vibration mirror according to each of the ratios includes:

dividing the rotation range of the galvanometer into rotation intervals corresponding to the ratios according to the plurality of ratios, wherein the rotation range is composed of the rotation intervals;

The rotational angular velocity of the vibrating mirror in the corresponding rotational interval is adjusted according to the ratio.
The method for improving uneven light intensity of stripes according to claim 7, wherein said dividing the rotation range of the vibrating mirror into rotation intervals corresponding to the ratios according to the plurality of ratios includes:

Sorting the N ratios according to a preset order to obtain a ratio ranking M 1 , M 2 , M 3 ... M N ; wherein, the N is a positive integer; M is a ratio;

Processing the ratio rankings to obtain target ratio rankings M N , M N-1 , M N-2 ... M 1 , ... M N-2 , M N-1 , M N ;

The rotation range is divided into 2N-1 rotation intervals, and each rotation interval is in one-to-one correspondence with the ratios in the target ratio sorting.
The method for improving uneven light intensity of stripes according to claims 1 to 8, wherein said taking a plurality of calibration points along a straight line perpendicular to said grating stripes comprises:

A plurality of calibration points are uniformly or randomly selected along a straight line perpendicular to the grating stripes, wherein the number of the calibration points is more than a preset multiple of the number of the grating stripes.
A structured light depth camera, comprising:

one or more processors;

A storage device for storing executable instructions, the executable instructions implement the method according to any one of claims 1 to 9 when executed by the processor.
A computer-readable storage medium, on which executable instructions are stored, and the executable instructions implement the method according to any one of claims 1 to 9 when executed by a processor.
A computer program comprising computer executable instructions for implementing the method according to any one of claims 1 to 9 when executed.