CN115063368A

CN115063368A - Projector assembly abnormality detection method, projector assembly abnormality detection device, electronic apparatus, and storage medium

Info

Publication number: CN115063368A
Application number: CN202210681029.5A
Authority: CN
Inventors: 户磊; 曹天宇; 薛远; 季栋; 李东洋; 化雪诚
Original assignee: Hefei Dilusense Technology Co Ltd
Current assignee: Hefei Dilusense Technology Co Ltd
Priority date: 2022-06-16
Filing date: 2022-06-16
Publication date: 2022-09-16

Abstract

The application provides a projector assembly abnormity detection method, a projector assembly abnormity detection device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a speckle pattern projected by a projector of a depth camera module to be detected; detecting whether the speckle distribution of the speckle pattern meets a preset speckle distribution requirement or not, and/or detecting the assembly deflection angle of a projector of the depth camera module to be detected based on the speckle pattern; and when any one of the preset abnormal conditions is detected to occur, determining that the projector of the depth camera module to be detected is abnormally assembled. The method is applied to the production of the depth camera, and can effectively detect the abnormal assembly of the depth camera projector, thereby avoiding the generation of unqualified depth cameras.

Description

Projector assembly abnormality detection method, projector assembly abnormality detection device, electronic apparatus, and storage medium

Technical Field

The present disclosure relates to the field of depth camera technologies, and in particular, to a method and an apparatus for detecting projector assembly abnormality, an electronic device, and a storage medium.

Background

The depth camera based on structured light speckle mainly comprises a projector and an infrared camera, wherein the projector comprises an array light source, a collimating lens, a diffraction optical element and other main components and is mainly used for projecting structured light generated by the array light source to a shooting object; and the infrared camera is used for collecting a speckle pattern formed on the surface of the shot object by the structured light projected by the projector. In actual shooting, the projector projects structured light to a shot object, the structured light irradiates the surface of the shot object to form speckles, the infrared camera collects a speckle pattern covering the shot object, then depth calculation is carried out based on the collected speckle pattern, and depth information of the shot object is determined.

In the production process of the depth camera, due to unavoidable assembly tolerance, the projector can be abnormally assembled, so that the projector cannot accurately project speckles, and the infrared camera cannot acquire an effective speckle pattern, so that the shooting effect is influenced. Therefore, at the depth camera production stage, projector assembly anomalies need to be detected, thereby avoiding production of an unacceptable depth camera.

Disclosure of Invention

Based on the above requirements, the application provides a projector assembly abnormity detection method, a projector assembly abnormity detection device, an electronic device and a storage medium, and the assembly abnormity detection of the depth camera module projector can be realized.

The application provides a projector assembling abnormity detection method, which comprises the following steps:

acquiring a speckle pattern projected by a projector of a depth camera module to be detected;

detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not, and/or detecting the assembly deflection angle of a projector of the depth camera module to be detected based on the speckle pattern; wherein the fitting deflection angle represents a deviation between a pose of the projector in the depth camera module and a standard pose of the projector in the depth camera module;

when any one of preset abnormal conditions is detected, determining that the projector of the depth camera module to be tested is abnormally assembled;

the preset abnormal condition comprises that the speckle distribution of the speckle pattern does not accord with a preset speckle distribution requirement, the assembly deflection angle of the projector of the depth camera module to be detected is larger than a preset deflection angle threshold value, and the speckle distribution of the speckle pattern does not accord with the preset speckle distribution requirement and the assembly deflection angle of the projector of the depth camera module to be detected is larger than the preset deflection angle threshold value.

In one implementation manner of the first aspect, detecting whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement includes:

determining whether the speckle distribution in the speckle pattern meets the preset speckle distribution requirement by detecting whether an effective speckle area in the speckle pattern meets the preset effective speckle distribution requirement and/or comparing the speckle pattern with a standard speckle pattern;

wherein the effective speckle distribution requirement comprises at least one of a speckle pattern requirement, an effective speckle area proportion requirement and an effective speckle area distribution position requirement of an effective speckle area; and the standard speckle pattern is projected by a projector of the standard depth camera module in the working environment of the depth camera module to be detected.

In one implementation manner of the first aspect, comparing the speckle pattern with a standard speckle pattern to determine whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement includes:

and determining whether the speckle distribution in the speckle pattern meets the preset speckle distribution requirement by detecting whether the speckle pattern contains a set speckle area of a standard speckle pattern.

In one implementation manner of the first aspect, determining whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement by detecting whether the speckle pattern includes a set speckle region of a standard speckle pattern includes:

determining at least one target speckle region from the standard speckle pattern, wherein the target speckle region in the standard speckle pattern is more unique than a non-target speckle region in the standard speckle pattern;

determining whether the speckle pattern includes all target speckle areas in the standard speckle pattern by searching for speckle areas from the speckle pattern that match each target speckle area;

and if the speckle pattern does not contain all target speckle areas in the standard speckle pattern, determining that the speckle distribution of the speckle pattern does not meet the preset speckle distribution requirement.

In an implementation manner of the first aspect, detecting an assembly deflection angle of a projector of the depth camera module to be detected based on the speckle pattern includes:

performing speckle matching on the speckle pattern and a standard speckle pattern to determine an assembly deflection angle of a projector of the depth camera module to be detected; and the standard speckle pattern is projected by a projector of the standard depth camera module in the working environment of the depth camera module to be detected.

In an implementation manner of the first aspect, determining an assembly deflection angle of a projector of the depth camera module to be measured by performing speckle matching on the speckle pattern and a standard speckle pattern includes:

respectively searching speckle areas matched with each target speckle area from the speckle pattern;

and if speckle areas matched with the target speckle areas are searched from the speckle pattern respectively, calculating and determining the assembly deflection angle of the projector of the depth camera module to be measured according to the position information of each target speckle area in the standard speckle pattern and the position information of the speckle area matched with each target speckle area in the speckle pattern determined by searching.

In one implementation form of the first aspect, the method further comprises:

acquiring the projector deflection direction of the depth camera module to be detected, which is determined according to the distribution position of the effective speckle area in the speckle pattern;

and verifying whether the assembling deflection angle of the projector of the depth camera module to be tested is credible or not according to the deflection direction of the projector of the depth camera module to be tested.

In one implementation of the first aspect, determining at least one target speckle region from the standard speckle pattern comprises:

determining each speckle image sub-region from the standard speckle pattern by taking each speckle in the standard speckle pattern as a reference speckle, wherein each speckle image sub-region respectively takes one reference speckle as a center;

corresponding to each speckle image subregion, respectively calculating the similarity mean value of the speckle image subregion and other speckle image subregions;

selecting N speckle image sub-areas with the minimum similarity mean value with other speckle image sub-areas from each speckle image sub-area as a target speckle area in a standard speckle pattern; wherein N is a positive integer.

performing key speckle matching processing on the speckle pattern and a standard speckle pattern to obtain at least one group of matched key speckle;

and calculating and determining the assembly deflection angle of the projector of the depth camera module to be detected according to the position coordinates of the at least one group of matched key scattered spots.

The present application provides in a second aspect an apparatus for detecting projector assembly abnormality, including:

the data acquisition unit is used for acquiring a speckle pattern projected by a projector of the depth camera module to be detected;

the anomaly detection unit is used for detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not and/or detecting the assembly deflection angle of the projector of the depth camera module to be detected based on the speckle pattern; wherein the fitting deflection angle represents a deviation between a pose of the projector in the depth camera module and a standard pose of the projector in the depth camera module; when any one of preset abnormal conditions is detected, determining that the projector of the depth camera module to be tested is abnormally assembled; the preset abnormal condition comprises that the speckle distribution of the speckle pattern does not accord with a preset speckle distribution requirement, the assembly deflection angle of the projector of the depth camera module to be detected is larger than a preset deflection angle threshold value, and the speckle distribution of the speckle pattern does not accord with the preset speckle distribution requirement and the assembly deflection angle of the projector of the depth camera module to be detected is larger than the preset deflection angle threshold value.

A third aspect of the present application provides an electronic device comprising:

a memory and a processor;

the memory is connected with the processor and used for storing programs;

the processor is configured to execute the program in the memory to implement the method for detecting an assembling abnormality of a projector according to the first aspect or any one of the implementations of the first aspect.

A fourth aspect of the present application provides a storage medium, where a computer program is stored, and when the computer program is executed by a processor, the method for detecting an assembling abnormality of a projector according to the first aspect or any one of the implementation manners of the first aspect is implemented.

The projector assembling abnormity detection method comprises the steps of detecting whether speckle distribution of a speckle pattern projected by a projector of a depth camera module to be detected meets a preset speckle distribution requirement or not, and/or detecting an assembling deflection angle of the projector of the depth camera module to be detected based on the speckle pattern projected by the projector of the depth camera module to be detected, and performing assembling abnormity detection on the projector of the depth camera module to be detected. The method is applied to the production of the depth camera, and can effectively detect the abnormal assembly of the depth camera projector, thereby avoiding the generation of unqualified depth cameras.

In addition, the speckle pattern obtained by projection of the depth camera module also needs to be acquired in the production calibration process of the depth camera. Therefore, the projector assembling abnormity detection method provided by the embodiment of the application can be completed in the production calibration process of the depth camera, namely, the projector of the depth camera is directly subjected to assembling abnormity detection based on the speckle pattern acquired during the production calibration of the depth camera. Therefore, the projector assembly abnormity detection method provided by the embodiment of the application does not need to additionally build a detection environment, the projector assembly abnormity detection can be realized by adding the speckle pattern analysis algorithm in the original depth camera production environment, the calculated amount is small, and the production line blocking risk is not increased. Therefore, the projector assembly abnormality detection method is low in cost and easy to perform.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a structured light depth camera according to an embodiment of the present disclosure;

FIG. 2 is a schematic view of a speckle pattern provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a speckle pattern and an effective speckle area of the speckle pattern provided by an embodiment of the present application;

FIG. 4 is a schematic illustration of an effective speckle pattern provided by an embodiment of the present application;

fig. 5 is a schematic flowchart of a projector assembly abnormality detection method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a speckle pattern projected by an improperly assembled projector according to an embodiment of the present application;

fig. 7 is a schematic flowchart of another projector assembly abnormality detection method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a projector mounting abnormality detection apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The projector assembling anomaly detection method and device are suitable for application scenes of production and testing of the depth camera, and by means of the technical scheme, the projector of the depth camera module can be effectively subjected to assembling anomaly detection, and the assembling anomaly of the projector can be accurately identified.

In order to facilitate understanding of the technical solutions proposed in the embodiments of the present application, first, a structure and an operation principle of the structured light depth camera are briefly described.

Referring to fig. 1, the structured light depth camera is mainly composed of a structured light projector 1 and an infrared camera 2. The structured light projector 1 is mainly used for projecting structured light generated by an array light source to a shooting object; the infrared camera 2 is configured to collect a speckle pattern formed on the surface of the object by the structured light projected by the projector 1, and the specific speckle pattern can be seen in fig. 2. The distorted rectangular bright spot area shown in fig. 2 is the actual speckle area in the speckle pattern. In actual shooting, the projector 1 projects structured light to the object to be shot, the structured light irradiates the surface of the object to be shot to form speckles, and the infrared camera 2 collects speckle patterns covering the object to be shot, and then performs depth calculation based on the collected speckle patterns to determine the depth information of the object to be shot.

Theoretically, speckle patterns projected by projectors of different depth camera products have consistency in the same working environment for different product individuals of the same depth camera, that is, speckle patterns are consistent, for example, when different product individuals of the same depth camera are shot in a certain working environment, the speckle patterns obtained are all the speckle patterns shown in fig. 2. However, in practice, due to unavoidable assembly tolerances, the assembling positions of the projectors of different cameras are different, so that speckle patterns projected by different projectors are different in zooming, rotating, clipping and local contrast inconsistency.

Strictly speaking, however, the speckle patterns projected by the projectors of any two depth cameras correctly assembled according to the same standard in the same working environment should be highly consistent, and there will be no situations of scaling, rotation, clipping, brightness inconsistency, etc. It can be understood that if the speckle pattern projected by a certain depth camera product in a specific working environment is inconsistent with the pattern of the speckle pattern projected by a standard product in the specific working environment, or a local inconsistency exists, the projector of the depth camera product is considered to be abnormally assembled.

In addition, for a certain depth camera product, the projection area of the projector 1 should coincide with the image acquisition area of the infrared camera 2, so that it can be ensured that the speckle formed on the surface of the object to be photographed by the structured light projected by the projector 1 can be completely acquired by the infrared camera 2.

In general, in a standard depth camera assembling operation, the assembling position of the projector 1 should be adjusted so that an effective speckle region (i.e., a speckle region within an inscribed rectangle of an actual speckle region shown in fig. 2, specifically, a speckle region enclosed by a rectangular frame in fig. 3) formed by the structured light projected by the projector 1 can substantially cover the visual field range of the infrared camera 2 within a working distance range of the depth camera, i.e., a speckle pattern as shown in fig. 4 is formed, so that the effectiveness of the speckle pattern for depth calculation can be ensured. If the projector 1 cannot project an effective speckle pattern as shown in fig. 4, it is considered that the projector 1 is abnormally assembled, and the projector 1 needs to be assembled and adjusted.

Therefore, whether the projector is abnormally assembled or not can be determined in a reverse mode based on the speckle pattern projected by the projector of the depth camera module. The projector assembling abnormity detection method and device based on the principle are applied to projector assembling abnormity detection, and rapid, accurate and low-cost projector assembling abnormity detection can be achieved.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Exemplary method

The embodiment of the application provides a projector assembling abnormity detection method which can be exemplarily applied to intelligent processing equipment, so that automatic detection of projector assembling abnormity is realized. The intelligent processing device includes, but is not limited to, various types of electronic devices with data processing functions, such as a computer, an intelligent terminal, a single chip microcomputer, and the like, or high-performance data processing devices, such as a server (including a physical server and a cloud server), a workstation, and the like.

Referring to fig. 5, a projector assembly abnormality detection method according to an embodiment of the present application includes:

s501, obtaining a speckle pattern projected by a projector of the depth camera module to be measured.

The depth camera module to be tested is a depth camera module which needs to detect whether the projector is abnormally assembled. According to the speckle pattern projected by the projector of the depth camera module to be tested, whether the projector of the depth camera module to be tested is abnormally assembled or not is judged.

Specifically, in the embodiment of the present application, the speckle pattern obtained when the depth camera module to be measured works in a set working environment is obtained first. In the above-mentioned working environment that sets for, the structure light is projected to a big enough plane within its working distance scope perpendicularly to this depth camera module that awaits measuring's projector to form the speckle on this plane, shoot the speckle pattern that obtains through infrared camera to the speckle region on this plane.

The set working environment may be a test environment for testing the depth camera module. The test environment is generally designed according to the working distance of the camera and the production environment, for example, a depth camera module, the test environment is a dark box, in the dark box environment, the depth camera module can shoot a scene which is within the working distance range and does not shield the projection of the projector, the distance between each part of the scene and the depth camera module is known, and the speckle pattern formed in the scene can be obtained when the projector of the depth camera module is started.

In addition, the set working environment may further define the working environment of the depth camera module, for example, it is specified in the embodiment of the present application that the set working environment refers to a sufficiently large projection structured light within a range of a projection distance of a projector of the depth camera module vertically to the projector in a dark box test environment, and at the same time, an infrared camera of the depth camera module collects speckles formed on the plane to obtain a speckle pattern.

For example, the above-mentioned obtaining the speckle pattern projected by the projector of the depth camera module to be measured includes, but is not limited to, obtaining by:

firstly, the depth camera module to be measured in the set working environment is controlled to shoot a large enough plane in the working distance range, and a speckle pattern projected on the plane by a projector of the depth camera module to be measured is acquired and obtained through an infrared camera of the depth camera module to be measured.

Secondly, reading a prestored speckle pattern from a preset storage address or a preset storage area, wherein the read speckle pattern is the speckle pattern obtained by projecting a projector on a plane when a depth camera module to be measured working in the preset working environment shoots the large enough plane in the working distance range.

Thirdly, the speckle pattern transmitted by other equipment may be received, and similarly, the received speckle pattern is projected on a sufficiently large plane within the working distance range by the projector when the depth camera module to be measured working in the set working environment photographs the plane.

S502, detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not, and/or detecting the assembling deflection angle of the projector of the depth camera module to be detected based on the speckle pattern.

When any one of preset abnormal conditions is detected, step S503 is executed to determine that the projector of the depth camera module to be tested is abnormally assembled.

If any one of the preset abnormal conditions does not occur, step S504 may be executed to determine that the projector of the depth camera module to be tested is assembled successfully.

In particular, as can be seen from the above description of the embodiments, the projector is equipped with an acceptable depth camera module that determines and uniquely identifies the speckle distribution in the projected speckle pattern in a particular work environment. For example, the projector projects structured light to form a certain speckle pattern, the center of the speckle should be at the center of the speckle pattern, and the speckle pattern is substantially filled with speckle, for example, the speckle pattern shown in fig. 4 is obtained.

The speckle pattern determined, determined and unique based on the speckle pattern projected by the depth camera module qualified in projector assembly is used as a speckle distribution pattern for verifying whether the speckle pattern projected by the projector of any depth camera module meets the speckle distribution requirement of the standard speckle pattern. Illustratively, the speckle distribution requirements include, but are not limited to, the presence or absence of speckles, the pattern of speckles, the speckle distribution area, the speckle distribution location, the mutual positional relationship between speckle points, and the like.

On this basis, the embodiment of the application detects whether the speckle distribution of the speckle pattern meets the speckle distribution requirement after acquiring the speckle pattern projected by the projector of the depth camera module to be measured. The specific detection content can be determined according to the specific content of the above-mentioned speckle distribution requirement, and preferably, the specific detection content is consistent with the specific content of the above-mentioned speckle distribution requirement. For example, assuming that the above-mentioned speckle distribution requirements specifically include an effective speckle ratio and an effective speckle distribution position, when detecting whether the speckle distribution of the acquired speckle pattern meets the above-mentioned speckle distribution requirements, the above-mentioned speckle ratio and effective speckle distribution position of the acquired speckle pattern are specifically detected, and whether the requirements of the above-mentioned speckle distribution requirements on the effective speckle ratio and the effective speckle distribution position are met.

As an example, the speckle pattern obtained by projecting the depth camera module with the qualified projector in a specific working environment can also be directly used as the speckle distribution requirement. At the moment, whether the speckle distribution of the acquired speckle pattern meets the preset speckle distribution requirement or not is detected, and specifically, whether the acquired speckle pattern meets the speckle distribution of the speckle pattern projected by the depth camera module which is qualified in assembling the projector in the same working environment or not is detected.

When the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be measured in a certain working environment is not consistent with the speckle distribution of the speckle pattern projected by the correctly assembled projector in the same working environment and the same working mode, the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be measured can be considered to be inconsistent with the speckle distribution requirement, and therefore the abnormal assembly of the projector of the depth camera module to be measured can be determined.

For example, for a depth camera, the speckle pattern projected by a standard assembled depth camera product in a given work environment is in the form shown in fig. 4. For a certain depth camera product to be measured of the depth camera, if the speckle distribution in the speckle pattern projected in the set working environment is inconsistent with the speckle distribution shown in fig. 4, it can be determined that the speckle pattern projected by the projector of the depth camera to be measured does not meet the speckle distribution requirement, so that it can be determined that the assembly of the projector of the depth camera to be measured does not meet the assembly standard, namely the projector of the depth camera to be measured has abnormal assembly.

Besides detecting whether the speckle distribution of the acquired speckle pattern meets the preset speckle distribution requirement, the embodiment of the application also detects the assembly deflection angle of the projector of the depth camera module to be detected based on the acquired speckle pattern.

The assembling deflection angle of the projector of the depth camera module to be tested is used for representing the deviation between the actual pose of the projector in the depth camera module and the standard pose of the projector in the depth camera module.

For example, according to the predetermined imaging coordinate system of the projector and the corresponding relationship between the position coordinate systems of the projector in the depth camera module, the installation position and the installation pose of the projector in the depth camera module can be determined according to the distribution position of the speckle pattern projected by the projector, the inclination angle of the speckle pattern and the like.

The detected assembling deflection angle of the projector of the depth camera module to be detected can be used as a quantitative detection result for detecting the assembling abnormity of the projector, for example, when the speckle distribution of the acquired speckle pattern does not accord with the preset speckle distribution requirement, the assembling deflection angle of the projector of the depth camera module to be detected can be detected to determine the difference between the assembling pose of the projector and the standard assembling pose of the projector.

On the other hand, the detected assembling deflection angle of the projector of the depth camera module to be detected can also be directly used for judging whether the projector of the depth camera module to be detected is abnormally assembled or not, so that qualitative and quantitative detection results of abnormal projector assembling detection are obtained.

As an exemplary judgment mode, a deflection angle threshold is preset in the embodiment of the application, and when the assembling deflection angle of the projector of the depth camera module to be detected is detected to be larger than the preset deflection angle threshold, it can be determined that the projector of the depth camera module to be detected is abnormally assembled; if the assembling deflection angle of the projector of the depth camera module to be detected is not larger than the preset deflection angle threshold value, the projector of the depth camera module to be detected can be determined to be normally assembled.

According to the embodiment of the application, whether the projector of the depth camera module to be detected is assembled abnormally can be judged qualitatively and/or quantitatively through detection of the two aspects.

In the detection process, whether the projector of the depth camera module to be detected is abnormally assembled or not can be qualitatively judged by detecting whether the speckle distribution of the acquired speckle pattern meets the preset speckle distribution requirement or not; or qualitatively or quantitatively judging whether the projector of the depth camera module to be detected is abnormally assembled or not by detecting the assembly deflection angle of the projector of the depth camera module to be detected; or, qualitatively judging whether the projector of the depth camera module to be detected is abnormally assembled or not by detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not, and quantitatively judging the abnormal assembling condition of the projector of the depth camera module to be detected by detecting the assembling deflection angle of the projector of the depth camera module to be detected when the abnormal assembling is determined.

The embodiment of the application presets three preset abnormal conditions: one is that the speckle distribution of the acquired speckle pattern does not meet the preset speckle distribution requirement; secondly, the assembling deflection angle of a projector of the depth camera module to be detected is larger than a preset deflection angle threshold value; thirdly, the speckle distribution of the acquired speckle pattern does not accord with the preset speckle distribution requirement, and the assembly deflection angle of the projector of the depth camera module to be measured is larger than the preset deflection angle threshold value.

In the detection process, if any one of the preset abnormal conditions is detected, it is determined that the projector of the depth camera module to be detected is abnormally assembled.

It can be seen from the above description that, in the projector assembly anomaly detection method provided in the embodiment of the present application, it is detected whether the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be detected meets the preset speckle distribution requirement, and/or the assembly deflection angle of the projector of the depth camera module to be detected is detected based on the speckle pattern projected by the projector of the depth camera module to be detected, and the projector of the depth camera module to be detected is subjected to assembly anomaly detection, and if the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be detected does not meet the preset speckle distribution requirement, and/or the assembly deflection angle of the projector of the depth camera module to be detected is detected to be greater than the preset deflection angle threshold, it is determined that the projector of the depth camera module to be detected is assembled abnormally. The method is applied to the production of the depth camera, and can effectively detect the abnormal assembly of the depth camera projector, thereby avoiding the generation of unqualified depth cameras.

In addition, the calibration of the depth camera is an essential link in the production process of the depth camera, and a speckle pattern obtained by projection of the depth camera module also needs to be acquired in the production calibration process of the depth camera. Therefore, the projector assembling abnormity detection method provided by the embodiment of the application can be completed in the production calibration process of the depth camera, namely, the projector of the depth camera is directly subjected to assembling abnormity detection based on the speckle pattern acquired during the production calibration of the depth camera. Therefore, the projector assembly abnormity detection method provided by the embodiment of the application does not need to additionally build a detection environment, the projector assembly abnormity detection can be realized by adding the speckle pattern analysis algorithm in the original depth camera production environment, the calculated amount is small, and the production line blocking risk is not increased. Therefore, the projector assembly abnormality detection method is low in cost and easy to perform.

As an exemplary embodiment, whether the speckle distribution of the acquired speckle pattern meets a preset speckle distribution requirement may be detected by performing any one or two of the following two-aspect processes (when the following two-aspect processes are performed, the specific execution order is not limited):

in a first aspect, it is determined whether the speckle distribution of the acquired speckle pattern meets a preset speckle distribution requirement by detecting whether an effective speckle area in the acquired speckle pattern meets the preset effective speckle distribution requirement.

And in the second aspect, comparing the obtained speckle pattern with a standard speckle pattern to determine whether the speckle distribution of the obtained speckle pattern meets the preset speckle distribution requirement.

The effective speckle distribution requirement comprises at least one of a speckle pattern requirement, an effective speckle area proportion requirement and an effective speckle area distribution position requirement of the effective speckle area. The standard speckle pattern is obtained by projecting by a projector of a standard depth camera module in the working environment of the depth camera module to be detected.

For the first aspect, whether the speckle pattern integrally meets the speckle distribution requirement is determined mainly by detecting whether the pattern, distribution and the like of the effective speckle region of the projected speckle pattern per se meet the effective speckle distribution requirement when the depth camera module to be detected works in a set working environment.

For example, whether the speckle pattern projected by the projector of the depth camera module to be detected has effective speckles, whether the pattern of the effective speckles is correct, whether the effective speckles are clear, whether the effective speckles are seriously deformed, whether the effective speckle area is too large or too small, whether the effective speckles are seriously offset, and the like are detected, so that whether the effective speckle area of the speckle pattern projected by the projector of the depth camera module to be detected meets the distribution requirement of the effective speckles is determined. If the speckle pattern projected by the projector of the depth camera module to be detected has any one or more of the conditions of no effective speckles, incorrect effective speckle pattern, unclear effective speckles, seriously deformed effective speckles, seriously deviated effective speckles or overlarge or undersize effective speckle area, the effective speckle area of the speckle pattern can be determined not to meet the distribution requirement of the effective speckles, namely the speckle pattern does not meet the distribution requirement of the speckles, and then the projector of the depth camera module to be detected can be determined to be abnormally assembled.

As a preferred implementation manner, in the embodiment of the present application, it is mainly determined whether the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be measured meets the speckle distribution requirement by detecting whether the distribution position of the effective speckle area in the speckle pattern projected by the projector of the depth camera module to be measured meets the preset requirement for the distribution position of the effective speckle area.

As can be seen from the foregoing description of the embodiment, for a depth camera module with a correctly assembled projector, the effective speckle region in the projected speckle pattern (i.e. the inscribed rectangular region of the actual speckle region shown in fig. 2, specifically, the speckle region enclosed by the white rectangular frame in fig. 3) can substantially cover the field of view of the infrared camera 2, i.e. form the speckle pattern shown in fig. 4. In this case, the center of the effective speckle area coincides with the center of the speckle pattern, i.e., the effective speckle area is at the center of the speckle pattern, and the effective speckle area covers substantially the entire speckle pattern area. If the projector is improperly assembled, a speckle pattern in the canonical form shown in FIG. 4 is not available.

Based on the situation, whether the speckle pattern meets the speckle distribution requirement or not is judged by verifying whether the effective speckle area in the speckle pattern projected by the projector of the depth camera module to be detected is located in the center of the speckle pattern and covers the range of the speckle pattern, and then whether the projector of the depth camera module to be detected is abnormally assembled or not can be determined by reverse estimation.

In an exemplary embodiment, whether an effective speckle area of a speckle pattern meets an effective speckle distribution requirement is determined by detecting whether a large blank area exists in any side area of the speckle pattern projected by a projector of a depth camera module to be measured.

The blank area refers to an image area where no speckle exists. When the area proportion of the blank area of any side area of the speckle pattern (the proportion of the area of the blank area in the whole speckle pattern) is larger than the set area proportion, the distribution position of the effective speckle area in the speckle pattern is considered to be unreasonable, the effective speckle area is not positioned in the central position of the speckle pattern and does not completely cover the speckle pattern, and the projector of the depth camera module to be measured can be determined to be abnormally assembled at the moment.

For example, if a speckle pattern projected by a projector of a certain depth camera module to be measured is as shown in fig. 6, it can be determined that the entire effective speckle region in the speckle pattern is shifted to the right side of the speckle pattern, and a large blank region is formed on the left side of the speckle pattern, that is, the projector fails to project speckles to the center of the speckle pattern, and the speckles cannot cover the complete speckle pattern, so that it can be determined that the projector of the depth camera module to be measured is abnormally assembled.

Furthermore, when the effective speckle area is determined to be shifted according to a speckle pattern projected by a projector of the depth camera module to be detected, the deflection direction of the projector can be determined according to the shifting aspect and the relation between the speckle shifting direction and the installation pose of the projector, and a reference basis can be provided for correcting the abnormal assembly of the projector.

With respect to the second aspect described above, it may be determined whether the speckle distribution of the acquired speckle pattern meets a preset speckle distribution requirement by processing shown in a1 as follows:

a1, determining whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement by detecting whether the speckle pattern contains the set speckle area of the standard speckle pattern.

The set speckle region may be any speckle region in the standard speckle pattern, such as a main feature region and a vertex region of the standard speckle pattern.

If the speckle pattern projected by the projector of the depth camera module to be tested contains all the set speckle areas of the standard speckle pattern, the speckle pattern can be considered to contain all the standard speckle pattern, and then the speckle distribution of the speckle pattern can be determined to meet the preset speckle distribution requirement; conversely, the speckle distribution of the speckle pattern may be determined to not meet the preset speckle distribution requirement.

As an alternative embodiment of the above-mentioned a1, the following processing a11-a14 may be implemented to determine whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement by detecting whether the speckle pattern contains the set speckle region of the standard speckle pattern:

and A11, determining at least one target speckle area from the standard speckle pattern. Wherein the uniqueness of the target speckle area in the standard speckle pattern is stronger than the uniqueness of the non-target speckle area in the standard speckle pattern.

Specifically, the area with the strong uniqueness is selected from the standard speckle pattern to be used as the interested area for verifying whether the projector of the depth camera to be tested is abnormally assembled. The region with stronger uniqueness in the speckle pattern is the speckle pattern region which has more obvious and unique characteristics in the speckle pattern and can be used for representing or characterizing the overall characteristics of the speckle pattern.

As an exemplary implementation, the embodiment of the present application determines a target speckle region from a standard speckle pattern by the following processes:

firstly, each scattered spot in a standard scattered spot map is taken as a reference scattered spot, and each speckle image sub-area is determined from the standard scattered spot map, wherein each speckle image sub-area respectively takes one reference scattered spot as the center.

For example, with respect to each scattered spot in the standard speckle pattern, a speckle image sub-region having a size of 31 × 31 centered on the scattered spot is extracted. Assuming a total of 1000 scattered spots in the standard speckle pattern, 1000 sub-regions of 31 x 31 speckle images can be extracted by the above process.

Then, corresponding to each speckle image sub-area, calculating the similarity mean value of the speckle image sub-area and other speckle image sub-areas.

Specifically, for each speckle image sub-region, similarity calculation is performed by using the image characteristics of the region and the image characteristics of all other speckle image sub-regions to obtain the characteristic similarity of the region and each other speckle image sub-region, and then each similarity is averaged to obtain the similarity average value of the speckle image sub-region and each other speckle image sub-region.

Finally, selecting N speckle image sub-areas with the minimum similarity mean value with other speckle image sub-areas from each speckle image sub-area as a target speckle area in the standard speckle pattern; wherein N is a positive integer.

In particular, the similarity mean of a speckle image sub-region and other speckle image sub-regions can be used to represent the uniqueness of the speckle image sub-region in the whole speckle pattern. The smaller the similarity mean value of the speckle image subarea and other speckle image subareas is, the more unique the speckle image subarea is, and the stronger the uniqueness is. Therefore, from each speckle image sub-area, the N speckle image sub-areas with the minimum similarity mean value with other speckle image sub-areas are selected, so that the N speckle image sub-areas with the strongest uniqueness can be obtained, and the speckle image sub-areas can be respectively used as target speckle areas.

A12, determining whether the speckle pattern contains all target speckle areas in the standard speckle pattern by searching for speckle areas from the speckle pattern that match each target speckle area.

If the speckle pattern does not include all target speckle areas in the standard speckle pattern, executing step A13, and determining that the speckle distribution in the speckle pattern does not meet the preset speckle distribution requirement.

And if the speckle pattern comprises all target speckle areas in the standard speckle pattern, executing the step A14, and determining that the speckle distribution in the speckle pattern meets the preset speckle distribution requirement.

Specifically, each target speckle area is used as a search condition, and a speckle area matched with the target speckle area (the matching degree is greater than a set matching degree threshold value) is searched from a speckle pattern projected by a projector of the depth camera module to be detected.

If the speckle area matched with each target speckle area can be searched from the speckle pattern, each target speckle area in the standard speckle pattern can be considered to be in the speckle pattern, so that the speckle pattern can be considered to be consistent with the standard speckle pattern, and further the speckle distribution in the speckle pattern can be determined to meet the preset speckle distribution requirement; otherwise, the speckle distribution in the speckle pattern can be determined not to meet the preset speckle distribution requirement, and further the projector of the depth camera module to be measured can be determined to be abnormally assembled.

The speckle pattern obtained by projecting the projector of the depth camera module to be measured is compared with the standard speckle pattern in the speckle area by the processing, so that more detailed speckle pattern detail comparison is realized, and the abnormal projection assembly can be more accurately identified.

As a preferred implementation manner, when detecting the assembly deflection angle of the projector of the depth camera module to be tested based on the acquired speckle pattern projected by the projector of the depth camera module to be tested, the embodiment of the application determines the assembly deflection angle of the projector of the depth camera module to be tested by performing speckle matching on the speckle pattern and a standard speckle pattern.

And the standard speckle pattern is projected by a projector of the standard depth camera module in the working environment of the depth camera module to be detected.

As described in the above embodiments, the assembly deflection angle can be directly output as a result of quantitative detection on whether the projector of the depth camera module to be measured is abnormally assembled. Or, the assembly deflection angle can be used for judging whether the projector of the depth camera module to be detected is abnormally assembled.

When the assembling deflection angle is used for judging whether the projector of the depth camera module to be detected is abnormally assembled or not, if the assembling deflection angle of the projector of the depth camera module to be detected determined through the processing is larger than a preset deflection angle threshold value, it is determined that the projector of the depth camera module to be detected is abnormally assembled, otherwise, the projector of the depth camera module to be detected can be considered to be qualified in assembling.

As an implementation manner of the above "determining the assembly deflection angle of the projector of the depth camera module to be measured by performing speckle matching on the speckle pattern and the standard speckle pattern", the detection of the assembly deflection angle of the projector may be continued based on the processing results of the above steps a11-a 12:

after the steps a11-a12 are executed, if speckle areas matched with each target speckle area are respectively searched from the speckle pattern projected by the projector of the depth camera module to be measured, the assembling deflection angle of the projector of the depth camera module to be measured can be calculated and determined according to the position information of each target speckle area in the standard speckle pattern and the position information of the speckle area matched with each target speckle area in the speckle pattern where the target speckle area is located.

For example, in the above search matching process, for each target speckle region, the position information of the target speckle region in the standard speckle pattern can be specified, for example, the position coordinates of its central speckle in the standard speckle pattern can be determined, and when a speckle region matched with the standard speckle region is searched from the speckle pattern projected by the projector of the depth camera module to be measured, the position information of the speckle region can be determined, for example, the position coordinates of its central speckle can be determined, so that for a pair of mutually matched speckle regions, a group of matched coordinate points can be determined, and further, when matching of all speckle regions is completed, the matched coordinate points of all matched speckle regions can be obtained.

On the basis, the pose of the projector in the depth camera module to be detected can be calculated by using a pose calculation method, and the assembling deflection angle of the projector of the depth camera module to be detected is determined relative to the change relation of the pose of the projector in the standard depth camera module. The pose of the projector in the standard depth camera module can be regarded as the standard pose of the projector in the depth camera module.

If the speckle areas matched with each target speckle area are not searched in the speckle pattern projected by the projector of the depth camera module to be detected after the processing of the steps a11-a12, that is, when it is determined that the speckle pattern projected by the projector of the depth camera module to be detected does not meet the preset speckle distribution requirement, matching coordinate points corresponding to each standard speckle area cannot be obtained, and therefore, the assembling deflection angle cannot be calculated according to the description, at this time, the projector assembling abnormity detection process can be ended, and only the qualitative detection result of the projector assembling abnormity is output, or the projector assembling deflection angle can be calculated according to other schemes introduced in the subsequent embodiments of the present application.

Further, when the projector of the depth camera module to be detected has a large assembling deflection angle determined through the calculation, whether the assembling deflection angle of the projector obtained through calculation is credible or not can be verified through obtaining the projector deflection direction determined according to the distribution position of the effective speckle area in the speckle pattern of the projector.

The deflection direction of the projector determined according to the distribution position of the effective speckle region in the speckle pattern of the projector may be the deflection direction of the projector determined by detecting whether the effective speckle region in the acquired speckle pattern meets a preset effective speckle distribution requirement or not and determining whether the speckle distribution of the acquired speckle pattern meets the preset speckle distribution requirement or not, which is introduced in the related content of analyzing the distribution position of the effective speckle region of the speckle pattern projected by the projector of the depth camera module to be measured, and determining the effective speckle region of the speckle pattern in the acquired speckle pattern. For example, the direction of deflection of the projector is determined according to the effective speckle region shift direction shown in fig. 6.

And according to the deflection direction of the projector of the depth camera module to be tested, which is determined according to the effective speckle region offset direction of the speckle pattern, verifying whether the assembly deflection angle of the projector of the depth camera module to be tested, which is obtained through calculation, is credible.

Specifically, if the projector deflection direction determined by the speckle pattern is consistent with the deflection direction corresponding to the calculated projector assembly deflection angle, the calculated projector assembly deflection angle is considered to be credible, otherwise, the projector assembly deflection angle is considered to be unreliable.

By the judgment, whether the calculation result of the assembly deflection angle is correct or not can be further verified by combining with multi-aspect information, so that the projector assembly abnormity can be more accurately detected.

As another optional implementation manner of the aforementioned "determining the assembly deflection angle of the projector of the depth camera module to be measured by performing speckle matching on the speckle pattern and the standard speckle pattern", the implementation manner may specifically be implemented by the following processing:

firstly, key speckle matching processing is carried out on a speckle pattern projected by a projector of a depth camera module to be detected and a standard speckle pattern to obtain at least one group of matched key speckles.

Specifically, the key speckle matching specifically refers to selecting a batch of scattered speckles from a standard speckle pattern as key speckle, or selecting a batch of speckle regions as key speckle point regions, and then searching speckle points matched with the key speckle points or speckle regions matched with the key speckle point regions from the speckle pattern projected by the projector of the depth camera module to be measured. Hypothesis search results in n _d For the key scattered spots or key speckle point areas which are matched with each other, P is obtained according to the position coordinates of each key scattered spot or key scattered spot area _d And matching the coordinates.

The number of the key scattered spots or key speckle point areas selected from the standard scattered spot pattern can be determined according to actual conditions, and theoretically, the number of the selected key scattered spots or key speckle point areas is more, the density is higher, and the assembling deflection angle of the projector of the depth camera module to be measured can be calculated more favorably. Therefore, in the embodiment of the application, dense key scattered spots or key scattered spot areas are selected from the standard scattered spot image, and the scattered spot image projected by the projector of the depth camera module to be measured is subjected to speckle point matching processing. The dense critical scattered spots or critical speckle point areas may be partial scattered spots or speckle point areas in the standard speckle pattern, or all scattered spots or all speckle areas in the standard speckle pattern.

And then, calculating and determining the assembly deflection angle of the projector of the depth camera module to be detected according to the position coordinates of the at least one group of matched key scattered spots.

Specifically, according to the obtained position coordinates of the matching key scattered spots or the position coordinates of the matching key speckle point area, namely the above P _d For the matching coordinates, the relative position relation between the depth camera module and the shot scene is combined, the assembling pose of the projector in the depth camera module to be measured can be calculated through a pose calculation method, and the deviation of the assembling pose of the projector relative to the assembling pose of the projector in the standard depth camera module can be obtained through calculationAnd obtaining the assembling deflection angle of the projector of the depth camera module to be detected.

In the embodiment, the assembling pose of the projector in the depth camera module to be tested is calculated by performing speckle matching processing on the speckle pattern projected by the projector of the depth camera module to be tested and the standard speckle pattern, and the deviation of the assembling pose of the projector in the depth camera module to be tested relative to the assembling pose of the projector in the standard depth camera module (namely the standard pose of the projector in the depth camera module) is calculated, so that whether the projector of the depth camera module to be tested is abnormally assembled or not can be quantitatively judged, and the abnormal assembling degree of the projector can be determined. Furthermore, according to the calculated assembling deflection angle of the projector of the depth camera module to be detected, the adjusting direction and the adjusting amount of the projector of the depth camera module to be detected which are abnormally assembled can be determined, so that the qualified depth camera module can be produced.

As an exemplary implementation, the projector assembly abnormality detection after the qualitative determination can be realized by implementing various projector assembly abnormality detection schemes described in the above embodiments of the present application.

Specifically, for example, a result of qualitative or quantitative detection of the projector attachment abnormality may be obtained by performing the processing of steps S1-S5 shown in fig. 7:

s1, determining whether the projector of the depth camera module to be detected is abnormally assembled by detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement.

Specifically, whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement is determined by detecting whether the effective speckle area in the speckle pattern meets the preset speckle distribution requirement, and then whether the projector of the depth camera module to be measured is abnormally assembled is determined.

Referring to the embodiment, whether the speckle distribution of the speckle pattern projected by the projector of the depth camera module to be measured meets the speckle distribution requirement can be determined by detecting whether the distribution position of the effective speckle area in the speckle pattern meets the preset requirement of the distribution position of the effective speckle area, and further, whether the projector of the depth camera module to be measured is abnormally assembled can be determined.

This step makes it possible to obtain a qualitative detection result of the detection of the projector mounting abnormality, i.e., whether the projector mounting is abnormal or not.

S2, determining at least one target speckle region from the standard speckle pattern, and determining whether the projector of the depth camera module to be measured is abnormally assembled or not by searching the speckle region matched with each target speckle region from the speckle pattern.

Specifically, as described with reference to the above embodiment, it is determined whether the speckle pattern contains all target speckle areas in the standard speckle pattern by searching for a speckle area from the speckle pattern that matches each target speckle area. If all target speckle areas in the standard speckle pattern are not contained, determining that the speckle distribution of the speckle pattern does not meet the speckle distribution requirement, and further determining that the projector of the depth camera module to be measured is abnormally assembled; if all target speckle areas in the standard speckle pattern are contained, the speckle distribution of the speckle pattern can be determined to meet the speckle distribution requirement, and then the projector of the depth camera module to be measured can be determined to be normally assembled.

The processing of this step can also result in a qualitative detection result of the abnormal detection of the projector assembly, i.e. whether the projector assembly is abnormal or not.

If it is determined that the projector is abnormally mounted through the processing of step S2, the process proceeds to step S5, and if it is determined that the projector is normally mounted, the process proceeds to step S3.

And S3, calculating and determining the assembling deflection angle of the projector of the depth camera module to be measured according to the position information of each target speckle area in the standard speckle pattern and the position information of the speckle area matched with each target speckle area in the speckle pattern, which is determined by searching.

Referring to the description of the embodiment, the pose of the projector of the depth camera module to be measured can be calculated and determined by pose calculation according to the matching position information of the matching speckle region, and the assembling deflection angle of the projector of the depth camera module to be measured can be obtained relative to the difference of the poses of the projectors of the standard depth camera module.

Through the processing of this step, a quantitative detection result of the detection of the projector assembly abnormality, that is, the assembly deflection angle of the projector can be obtained. On this basis, it is possible to verify whether the calculated fitting deflection angle is authentic by continuing to perform step S4, or to jump to step S5, to calculate a more accurate fitting deflection angle.

S4, acquiring the projector deflection direction of the depth camera module to be tested determined according to the distribution position of the effective speckle region in the speckle pattern, and verifying whether the assembly deflection angle of the projector of the depth camera module to be tested is credible according to the projector deflection direction of the depth camera module to be tested.

The specific verification process is as described in the above embodiments. If the assembling deflection angle of the projector of the depth camera module to be detected calculated in the step S3 is verified and determined to be credible, outputting the quantitative detection result, and further jumping to the step S5 to execute; if not, directly jumping to step S5.

S5, performing key speckle matching processing on the speckle pattern and a standard speckle pattern to obtain at least one group of matched key speckles, and calculating and determining the assembly deflection angle of the projector of the depth camera module to be measured according to the position coordinates of the at least one group of matched key speckles.

As described with reference to the above-described embodiment, in the process of step S5, the dense key scatterers are used to perform the scatterer matching process, thereby calculating a more accurate fitting deflection angle.

Through the processing of the step, a more accurate quantitative detection result of the projector assembling abnormity detection can be obtained, and a more accurate assembling deflection angle calculation result can be obtained.

For the specific processing content of the above steps S1-S5, please refer to the specific results of the corresponding content in the above embodiments, which is not repeated here. The execution sequence of steps S1-S5 is not limited to the sequence described above, and may be flexibly combined and adjusted. It is understood that through the processing of the above steps S1-S5, an accurate projector mounting abnormality detection result can be obtained from qualitative to quantitative, from rough to precise.

Exemplary devices

Accordingly, an embodiment of the present application further provides a projector mounting abnormality detection apparatus, as shown in fig. 8, the apparatus including:

the data acquisition unit 100 is used for acquiring a speckle pattern projected by a projector of the depth camera module to be detected;

the anomaly detection unit 110 is used for detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not and/or detecting the assembly deflection angle of the projector of the depth camera module to be detected based on the speckle pattern; wherein the assembly deflection angle represents a deviation between a pose of the projector in the depth camera module and a standard pose of the projector in the depth camera module; when any one of preset abnormal conditions is detected, determining that the projector of the depth camera module to be tested is abnormally assembled; the preset abnormal condition comprises that the speckle distribution of the speckle pattern does not accord with the preset speckle distribution requirement, the assembly deflection angle of the projector of the depth camera module to be detected is larger than the preset deflection angle threshold value, and the speckle distribution of the speckle pattern does not accord with the preset speckle distribution requirement and the assembly deflection angle of the projector of the depth camera module to be detected is larger than the preset deflection angle threshold value.

As an optional implementation, detecting whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement includes:

As an optional implementation, comparing the speckle pattern with a standard speckle pattern to determine whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement includes:

As an optional implementation, determining whether the speckle distribution in the speckle pattern meets a preset speckle distribution requirement by detecting whether the speckle pattern contains a set speckle region of a standard speckle pattern includes:

As an optional implementation manner, when the anomaly detection unit detects the assembly deflection angle of the projector of the depth camera module to be measured based on the speckle pattern, the anomaly detection unit is specifically configured to:

As an optional implementation manner, the determining an assembly deflection angle of a projector of the depth camera module to be measured by performing speckle matching on the speckle pattern and a standard speckle pattern includes:

As an optional implementation, the deflection angle calculation unit is further configured to:

As an alternative embodiment, the determining at least one target speckle region from the normalized speckle pattern includes:

The projector assembly abnormity detection device provided by the embodiment belongs to the same application concept with the projector assembly abnormity detection method provided by the embodiment of the application, can execute the projector assembly abnormity detection method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects for executing the projector assembly abnormity detection method. For details of the technology not described in detail in this embodiment, reference may be made to the specific processing content of the projector assembly anomaly detection method provided in the foregoing embodiment of the present application, and details are not described here again.

Exemplary electronic device

Another embodiment of the present application further provides an electronic device, as shown in fig. 9, the electronic device including:

a memory 200 and a processor 210;

wherein, the memory 200 is connected to the processor 210 for storing programs;

the processor 210 is configured to execute the program stored in the memory 200 to implement the projector assembling abnormality detection method disclosed in any of the above embodiments.

Specifically, the electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

The processor 210, the memory 200, the communication interface 220, the input device 230, and the output device 240 are connected to each other through a bus. Wherein:

a bus may comprise a path that transfers information between components of a computer system.

The processor 210 may be a general-purpose processor, such as a general-purpose Central Processing Unit (CPU), microprocessor, etc., an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs in accordance with the present invention. But may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

The processor 210 may include a main processor and may also include a baseband chip, a modem, and the like.

The memory 200 stores programs for executing the technical solution of the present invention, and may also store an operating system and other key services. In particular, the program may include program code including computer operating instructions. More specifically, memory 200 may include a read-only memory (ROM), other types of static storage devices that may store static information and instructions, a Random Access Memory (RAM), other types of dynamic storage devices that may store information and instructions, a disk storage, a flash, and so forth.

The input device 230 may include a means for receiving data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor, among others.

Output device 240 may include equipment that allows output of information to a user, such as a display screen, a printer, speakers, and the like.

Communication interface 220 may include any device that uses any transceiver or the like to communicate with other devices or communication networks, such as an ethernet network, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), etc.

The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the steps of any one of the projector assembling abnormality detection methods provided in the above embodiments of the present application.

Exemplary computer program product and storage Medium

In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the projector assembly anomaly detection method described in the "exemplary methods" section above of this specification.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, an embodiment of the present application may also be a storage medium having stored thereon a computer program that is executed by a processor to perform the steps in the projector attachment abnormality detection method described in the above-mentioned "exemplary method" section of this specification.

While, for purposes of simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present application is not limited by the order of acts or acts described, as some steps may occur in other orders or concurrently with other steps in accordance with the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The steps in the method of each embodiment of the present application may be sequentially adjusted, combined, and deleted according to actual needs, and technical features described in each embodiment may be replaced or combined.

The modules and sub-modules in the device and the terminal in the embodiments of the application can be combined, divided and deleted according to actual needs.

In the several embodiments provided in the present application, it should be understood that the disclosed terminal, apparatus and method may be implemented in other manners. For example, the above-described terminal embodiments are merely illustrative, and for example, the division of a module or a sub-module is only one logical division, and there may be other divisions when the terminal is actually implemented, for example, a plurality of sub-modules or modules may be combined or integrated into another module, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The modules or sub-modules described as separate parts may or may not be physically separate, and parts that are modules or sub-modules may or may not be physical modules or sub-modules, may be located in one place, or may be distributed over a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional modules or sub-modules in the embodiments of the present application may be integrated into one processing module, or each module or sub-module may exist alone physically, or two or more modules or sub-modules are integrated into one module. The integrated modules or sub-modules may be implemented in the form of hardware, or may be implemented in the form of software functional modules or sub-modules.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software unit executed by a processor, or in a combination of the two. The software cells may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A projector assembly abnormality detection method, characterized by comprising:

detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not, and/or detecting the assembly deflection angle of a projector of the depth camera module to be detected based on the speckle pattern; wherein the assembly deflection angle represents a deviation between a pose of the projector in the depth camera module and a standard pose of the projector in the depth camera module;

when any one of preset abnormal conditions is detected, determining that the projector of the depth camera module to be detected is abnormally assembled;

2. The method of claim 1, wherein detecting whether the speckle distribution of the speckle pattern meets a predetermined speckle distribution requirement comprises:

determining whether the speckle distribution of the speckle pattern meets a preset speckle distribution requirement by detecting whether an effective speckle area in the speckle pattern meets the preset effective speckle distribution requirement and/or comparing the speckle pattern with a standard speckle pattern;

wherein the effective speckle distribution requirement comprises at least one of a speckle pattern requirement, an effective speckle area proportion requirement and an effective speckle area distribution position requirement of an effective speckle area; and the standard speckle pattern is obtained by projecting through a projector of the standard depth camera module in the working environment of the depth camera module to be detected.

3. The method of claim 2, wherein determining whether the speckle distribution of the speckle pattern meets a preset speckle distribution requirement by comparing the speckle pattern with a standard speckle pattern comprises:

and determining whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement by detecting whether the speckle pattern contains a set speckle area of a standard speckle pattern.

4. The method of claim 3, wherein determining whether the speckle distribution in the speckle pattern meets a predetermined speckle distribution requirement by detecting whether the speckle pattern contains a set speckle region of a standard speckle pattern comprises:

5. The method of any one of claims 1 to 4, wherein detecting the assembly deflection angle of the projector of the depth camera module under test based on the speckle pattern comprises:

6. The method of claim 5, wherein determining the fitting deflection angle of the projector of the depth camera module under test by speckle matching the speckle pattern with a standard speckle pattern comprises:

7. The method of claim 6, further comprising:

8. The method of claim 4 or 6, wherein determining at least one target speckle region from the normalized speckle pattern comprises:

9. The method of claim 5, wherein determining the fitting deflection angle of the projector of the depth camera module under test by speckle matching the speckle pattern with a standard speckle pattern comprises:

10. A projector mounting abnormality detection apparatus, characterized by comprising:

the anomaly detection unit is used for detecting whether the speckle distribution of the speckle pattern meets the preset speckle distribution requirement or not and/or detecting the assembly deflection angle of the projector of the depth camera module to be detected based on the speckle pattern; wherein the fitting deflection angle represents a deviation between a pose of the projector in the depth camera module and a standard pose of the projector in the depth camera module; when any one of preset abnormal conditions is detected, determining that the projector of the depth camera module to be detected is abnormally assembled; the preset abnormal condition comprises that the speckle distribution of the speckle pattern does not accord with a preset speckle distribution requirement, the assembly deflection angle of the projector of the depth camera module to be detected is larger than a preset deflection angle threshold value, and the speckle distribution of the speckle pattern does not accord with the preset speckle distribution requirement and the assembly deflection angle of the projector of the depth camera module to be detected is larger than the preset deflection angle threshold value.

11. An electronic device, comprising:

a memory and a processor;

the memory is connected with the processor and used for storing programs;

the processor is configured to implement the projector mounting abnormality detection method according to any one of claims 1 to 9 by executing the program in the memory.

12. A storage medium having stored thereon a computer program which, when executed by a processor, implements the projector fitting abnormality detection method according to any one of claims 1 to 9.