CN114708232A

CN114708232A - Image processing method and device, storage medium and electronic equipment

Info

Publication number: CN114708232A
Application number: CN202210375698.XA
Authority: CN
Inventors: 侯烨
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-04-11
Filing date: 2022-04-11
Publication date: 2022-07-05

Abstract

The embodiment of the application discloses an image processing method, an image processing device, a storage medium and electronic equipment, wherein the method comprises the following steps: the method comprises the steps of obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, obtaining pixel characteristic parameters corresponding to light spot pixel points in the light spot, and determining an interference light spot from the at least one light spot based on the pixel characteristic parameters. By adopting the embodiment of the application, the interference light spots can be accurately identified.

Description

Image processing method and device, storage medium and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to an image processing method and apparatus, a storage medium, and an electronic device.

Background

At present, in a three-dimensional imaging scene, the three-dimensional imaging scene is mainly realized based on a binocular camera, a result light method and a flight time method. Taking an indirect time-of-flight method (ietf method) in the time-of-flight method as an example, the ietf method is a method for determining a distance between a target to be measured by measuring a phase relationship between modulated emitted light and received light, and is widely applied to related application scenes such as gesture recognition related to three-dimensional depth vision, human face recognition, three-dimensional modeling, motion sensing games, machine vision, auxiliary focusing, security protection, automatic driving and the like.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, a storage medium and electronic equipment, wherein the technical scheme is as follows:

in a first aspect, an embodiment of the present application provides an image processing method, where the method includes:

acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map;

acquiring pixel characteristic parameters corresponding to light spot pixel points in the light spots;

determining an interference spot from the at least one spot based on the pixel characteristic parameter.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including:

the light spot determining module is used for acquiring a reflection intensity map and determining at least one light spot in the reflection intensity map;

the parameter acquisition module is used for acquiring pixel characteristic parameters corresponding to light spot pixel points in the light spots;

and the interference determining module is used for determining an interference light spot from the at least one light spot based on the pixel characteristic parameters.

In a third aspect, embodiments of the present application provide a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the above-mentioned method steps.

In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

The beneficial effects brought by the technical scheme provided by some embodiments of the application at least comprise:

in one or more embodiments of the present application, a reflection intensity map is obtained, at least one light spot in the reflection intensity map is determined, then a pixel characteristic parameter corresponding to a pixel point of the light spot in the light spot is obtained, based on a difference between a pixel characteristic parameter on an interference light spot and a pixel characteristic parameter on an effective light spot, the interference light spot can be accurately determined from the at least one light spot, and light spot interference caused by superposition of edge signals of a plurality of light spots can be eliminated in an auxiliary manner; the whole image interference processing process reduces the dependence on equipment hardware and complex algorithms, realizes accurate light spot interference identification, can assist in outputting high-precision depth measurement results, and improves the robustness in a depth measurement scene.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a partial region in a reflection intensity map involved in an image processing method provided in an embodiment of the present application;

fig. 3 is a schematic view of a local scene related to a light spot interference point, according to an embodiment of the present application, of an image processing method;

FIG. 4 is a schematic flowchart of another image processing method provided in the embodiments of the present application;

FIG. 5 is a schematic flowchart of another image processing method provided in the embodiments of the present application;

fig. 6 is a scene schematic diagram of a light spot provided in an embodiment of the present application;

fig. 7 is a schematic view of a scene related to interference light spots provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of an interference determination module according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another image processing apparatus provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of an interference verification module according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

FIG. 13 is a schematic structural diagram of an operating system and a user space provided in an embodiment of the present application;

FIG. 14 is an architectural diagram of the android operating system of FIG. 13;

FIG. 15 is an architectural diagram of the IOS operating system of FIG. 13.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present application, it is noted that, unless explicitly stated or limited otherwise, "including" and "having" and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art. Further, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

The present application will be described in detail with reference to specific examples.

In one embodiment, as shown in fig. 1, an image processing method is proposed, which can be implemented by means of a computer program and can be run on an image processing device based on a von neumann architecture. The computer program may be integrated into the application or may run as a separate tool-like application. The image processing device may be a terminal, including but not limited to: personal computers, tablet computers, handheld devices, in-vehicle devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and the like. The terminal devices in different networks may be called different names, for example: user equipment, access terminal, subscriber unit, subscriber station, mobile station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user equipment, cellular telephone, cordless telephone, terminal equipment in a 5G network or future evolution network, and the like.

Specifically, the image processing method includes:

s101: acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map;

the reflection intensity map is obtained based on a TOF camera carried by a terminal, and the TOF camera is a camera for 3D imaging by using a flight time method; the terminal emits a modulated light beam through the TOF camera, and reflects the modulated light beam after encountering an object, and the terminal can receive a reflected light signal reflected by the object through a light-sensing element (Sensor) such as the TOF camera to generate a reflection intensity map. The reflection intensity map can be used for feeding back the intensity of a pixel point corresponding to at least one pixel point, the received light energy corresponding to at least one pixel point is in direct proportion to the pixel intensity of the pixel point in the reflection intensity map, and the stronger the received light energy corresponding to the pixel point is, the higher the pixel intensity of the pixel point is. In an actual application scene, a terminal performs scene imaging through a TOF camera, and can obtain a Depth map (a target scene Depth distance map, also called a Depth map) and a reflection intensity map in a current scene.

As can be understood, the TOF camera is based on 3D imaging by a time-of-flight method, which can be classified into a direct time-of-flight method (dTOF) and an indirect time-of-flight method (ietf) according to different signal results obtained, where the signal result obtained by the direct time-of-flight method is a time difference and the signal result obtained by the indirect time-of-flight method is a phase difference;

as can be appreciated, a TOF camera can be understood as a TOF module integrated in a terminal, the TOF module generally comprising a light receiving end (RX) and a light emitting end (TX);

illustratively, the light receiving end typically includes a lens, a narrow-band filter (e.g., 940nm narrow-band filter), a TOF sensor, etc., the light Emitting end typically includes a Vertical-Cavity Surface-Emitting Laser (VCSEL) array, a collimating lens, a Diffractive Optical Element (DOE), etc., and the DOE may be a Diffuser. The VCSELs in the VCSEL array are uniformly arranged on one plane and used for emitting light beams with two-dimensional regular patterns; the collimating lens is used for receiving and converging the light beams and ensuring that the light beams have parallel light paths; the DOE is used for scattering the light beams processed by the collimating lens, projecting the light beams onto an object of a scene according to a certain field angle, and forming bright spots as speckles after the light beams are reflected on the object and received by the light receiving end. For example, a VCSEL array typically includes at least two VCSELs, and the at least two VCSELs are uniformly arranged, so that regular uniform speckle can be obtained when the uniformly arranged VCSELs are used to emit light beams.

In practical applications, the light intensity value received by each Pixel at the light emitting end (RX) is usually used to characterize the Pixel (point) and to calculate the Confidence of generating the depth value, i.e. Confidence. The reflection intensity map may be understood as a value for feeding back a light intensity received by each Pixel, and in some embodiments, the reflection intensity map may also be referred to as a Confidence map (Confidence map), where a parameter value of each Pixel in the Confidence map is characterized by an intensity of the received signal light and is used as a Confidence of a target distance measurement value of the point.

Optionally, a TOF camera carried by the terminal may be a device supporting a speckle ietf mode, that is, may be used as the speckle ietf camera, the terminal modulates infrared light emitted by a light emitting end (TX) in the speckle ietf mode through the TOF camera to form a light beam of one of a plurality of beams, after being reflected by a scene target object, a plurality of light spots are received at an RX end, that is, an effective depth value only exists in an area illuminated by the light spots on a depth map, and the remaining areas are invalid values. In the application, the light spot interference can be accurately identified based on the reflection intensity diagram by executing the image processing method.

It can be understood that, after being reflected by a scene target object, a plurality of light spots received at the RX end are actually a plurality of speckles, and usually at least one speckle covers a plurality of pixel points, but each speckle only generates an effective depth value in an application scene such as a depth image and a distance measurement.

In a possible implementation manner, after the terminal acquires the reflection intensity map, the terminal may determine at least one light spot based on intensity distribution information of at least one pixel point in the reflection intensity map.

Schematically, the light spot shape in the reflection intensity map is a two-dimensional gaussian shape, that is, the light spot has high central intensity and gradually weakens around the light spot; that is, at least one spot may be identified based on the spot morphology in the reflected intensity map.

S102: acquiring pixel characteristic parameters corresponding to light spot pixel points in the light spots;

illustratively, the region where at least one light spot is located in the reflection intensity map may cover a plurality of light spot pixel points. As shown in fig. 2, fig. 2 is a schematic view of a partial region in a reflection intensity diagram, in fig. 2, a dashed circular frame can be regarded as a light spot, and the light spot corresponding to the dashed circular frame is in a two-dimensional gaussian shape, that is, the central intensity of the light spot is high, and the periphery of the light spot is gradually weakened. Each pixel grid in fig. 2 can be regarded as a pixel point, and the area covered by the dotted circular frame usually includes a plurality of pixel points, which are also spot pixel points. The pixel characteristic parameters can be pixel intensity parameters, pixel position parameters and the like of the light spot pixel points. The pixel characteristic parameters can be directly obtained based on the reflection intensity map.

S103: determining an interference spot from the at least one spot based on the pixel characteristic parameter.

It can be understood that, since the reflected light beam diverges (i.e., a plurality of pixel points are covered by one spot in the reflection intensity map), in the reflection intensity map, the spot shape of the spot is usually two-dimensional gaussian, and the central light energy is strongest, i.e., the spot image shown in fig. 2. The interference light spots are generated by overlapping divergent edge signals of at least two light spots, for example, if the light spots are close, the divergent edge signals of the light spots are overlapped, a 'false light spot' is formed between the light spots, the 'false light spot' also conforms to a two-dimensional Gaussian form, but the received light energy is far smaller than a theoretical design value, the signal-to-noise ratio in a subsequent application scene is poor, the 'false light spots' in an actual application scene can be mistakenly identified as a light spot because of being capable of generating a rough depth value, but the interference to the rear-end application is caused by most of poor accuracy, and the interference is an interference point.

Illustratively, for example, due to the existence of objective factors such as lens distortion and optical signal divergence, when the distance between at least two light spots is close (for example, the distance between every two light spots is about 2 times of the distance between the central regions of the light spots), a "false light spot" that is, a light spot interference point may appear between the light spots that affect each other, for example, as shown in fig. 3, fig. 3 is a partial schematic view of a scene related to the light spot interference point, 4 light spots actually correspond to a certain object surface of the scene under a certain actual application scene, the light spot intensities of the generated 4 light spots are all very close (the values in a Depth map (Depth map) or a reflection intensity map are equivalent), the application scene at the back end such as virtual reality and ranging considers that the region surfaces corresponding to the 4 light spots are planar, but due to signal divergence between the 4 light spots (it can be understood that due to hardware design at the factory stage and the like, causing at least one light spot to actually cover a plurality of pixel points in the reflection intensity map) and a signal superposition phenomenon exists in the edge signal between at least two light spots, that is, an interference light spot exists due to the signal superposition phenomenon, as shown in fig. 3, the intermediate light spot shown in fig. 3 is actually an interference light spot generated due to the superposition of the edge signal between a plurality of light spots, and a pixel point with high pixel intensity in the area covered by the interference light spot can be generally used as an interference light spot, it can be understood that, because the parameter difference between the interference light spot and the actual light spot is large (for example, the depth value in the depth map and the pixel intensity value in the reflection intensity map), the surface of the 5 points may be fitted to a curved surface in the back-end application scene, which causes serious errors. Speckle interference may be reduced based on the elimination of such interfering speckle points to depth pixels, such as corresponding locations in a depth map.

As can be understood, the interference light spot detection is performed on at least one light spot based on the pixel characteristic parameters to detect the difference of the pixel characteristic parameters (such as pixel intensity parameters and pixel position parameters) corresponding to the pixel points between the actual light spot and the interference light spot generated by overlapping the edge light signals of the actual light spots, so that the terminal can determine the interference light spot from the at least one light spot based on the difference; for example, the interference light spot intensity detection can be performed on the at least one light spot based on the pixel intensity parameter, and the terminal can determine the interference light spot from the at least one light spot; for example, based on the pixel position parameter, the at least one light spot can be subjected to interference light spot intensity detection, and the terminal can determine an interference light spot from the at least one light spot; in an actual application scenario, the terminal may generally determine an interference light spot from the at least one light spot, and based on the actual application scenario, at least one pixel covered by the interference light spot is used as the interference light spot.

In one or more embodiments related to the application, at least one light spot in a reflection intensity map is determined by obtaining the reflection intensity map, then a pixel characteristic parameter corresponding to a pixel point of the light spot in the light spot is obtained, based on the difference between the pixel characteristic parameter on an interference light spot and the pixel characteristic parameter on an effective light spot, the interference light spot can be accurately determined from the at least one light spot, and light spot interference caused by superposition of edge signals of a plurality of light spots can be eliminated in an auxiliary manner; the whole image interference processing process reduces the dependence on equipment hardware and complex algorithms, realizes accurate light spot interference identification, can assist in outputting high-precision depth measurement results, and improves the robustness in a depth measurement scene.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating another embodiment of an image processing method according to the present application. Specifically, the method comprises the following steps:

s201: acquiring a reflection intensity map, and determining at least one light spot in the reflection intensity map;

reference may be made in detail to method steps of one or more embodiments of the present application, which are not described in detail herein.

S202: acquiring the number of light spots corresponding to the light spots; and if the number of the light spots is larger than the number threshold, adjusting the exposure time of the camera.

According to some embodiments, after the terminal acquires the reflection intensity map, the terminal may determine at least one light spot based on intensity distribution information of at least one pixel point in the reflection intensity map.

Schematically, the light spot shape in the reflection intensity map is a two-dimensional gaussian shape, that is, the light spot has high central intensity and gradually weakens around the light spot; that is to say, at least one light spot can be identified based on the shape of the light spot in the reflected intensity map, and then the number of all the light spots is calculated, so that the number of the light spots corresponding to the light spot, that is, the number of the light spots in the reflected intensity map can be obtained. It will be appreciated that the counted number of spots may have interfering spots.

In the present specification, the number threshold is a threshold set for measuring the influence of the number of interference light spots. If the number of the detected light spots is larger than the number threshold, the number of the interference light spots is considered to be large, the energy of the interference light spots is generally large, and the depth value of the interference light spot corresponding to the interference light spot in the depth map is also high.

It can be understood that, in an actual application scenario, the TOF camera mounted on the terminal may be a device supporting a speckle ietf mode, such as a speckle ietf camera; in the factory stage of equipment, a TOF camera carried by a terminal is usually customized and calibrated according to actual scenes (target detection distance, detection precision, target object reflectivity, whether the TOF camera is used outdoors and the like), and the TOF camera includes Sensor model selection, light source design (including light source optical power and luminous point arrangement), lens design and other software and hardware aspects. In the terminal using stage after leaving the factory, the terminal works based on a certain speckle iTOF mode, and the theoretical quantity of light spots received by a light receiving end in the certain speckle iTOF mode, the theoretical quantity of pixel points in a light spot coverage area and the like are determined. For example, after the actual application stage of the speckle ietf camera, how many spots theoretically exist in the field of view, each spot theory covers several pixel points, the number threshold of the spots corresponding to the theory is usually set, and the terminal can acquire the spots accordingly. Alternatively, the threshold for the number of spots is typically greater than the theoretical number of spots (i.e. the actual number of spots).

In a specific implementation scenario, if the number of detected light spots is greater than the number threshold, the number of interference light spots is considered to be large, the energy of the interference light spots is generally large, and the depth value of the interference light spot corresponding to the interference light spot in the depth map is also high. In this case, a possible scenario is that the entire field of view of the TOF camera of the terminal is a target object in a short distance, and even though the target object is a region that is not a spot originally in the depth map or the reflected intensity map, scattered light due to an optical signal of the target object in the short distance can be received, and interference spots or interference spots generated between the target object in the short distance are large in number and high in energy and intensity. The terminal may adjust the camera exposure time. A camera module, such as an Auto Focus (AE) algorithm module, is usually present in the terminal, and the terminal may send an indication signal to the AE (Auto Focus automatic exposure) algorithm module based on the spot number fed back by the reflection intensity map, assist an overexposure determination process of the AE (Auto Focus automatic exposure) algorithm module, and reduce exposure time when determining overexposure, so as to ensure measurement of the depth distance of the target object in a short distance. Therefore, the number of the light spots in the reflection intensity map is used as an overexposure judgment condition, the automatic exposure process is optimized, and the practicability of the terminal such as an iTOF camera is further improved.

Optionally, after the terminal performs adjustment on the camera exposure time, the step of obtaining the reflection intensity map and determining at least one light spot in the reflection intensity map may be performed, that is, after the camera exposure time is completed, the reflection intensity map is obtained again, so as to implement interference optimization and optimize the optimization effect in the actual application scene.

S203: acquiring pixel characteristic parameters corresponding to light spot pixel points in the light spots;

S204: at least one spot disturbance detection mode for the reflected intensity map is determined.

In a specific implementation scenario, a current application scenario may be obtained, where different application scenarios correspond to different combinations of speckle interference detection manners (the combination includes at least one speckle interference detection manner), different requirements for measurement accuracy, interference adaptability and the like under different application scenes can be set in advance based on different reference application scenes, namely, an interference detection combination mapping relation of at least one reference application scene and the corresponding reference light spot interference detection mode combination is established, after the current application scene is determined, the light spot interference detection mode combination under the current application scene can be determined based on the interference detection combination mapping relation, at least one speckle interference detection mode for the reflection intensity map is determined based on the speckle interference detection mode combination.

Illustratively, the interference detection combination mapping relationship may be characterized in the form of a mapping combination, a mapping list, a mapping array, and the like, and is not limited in detail here.

Illustratively, the different reference application scenarios may be application scenarios in the types of remote depth measurement, virtual reality application, visual environment reconstruction, virtual reality game, plane detection, and the like, which are involved in the terminal.

Optionally, the light spot interference detection mode includes, but is not limited to, fitting of one or more of a pixel point intensity detection mode, an energy ratio detection mode, a pixel point mean value detection mode, an energy total intensity detection mode, a light spot position detection mode, and the like.

Optionally, when it is determined that there are at least two light spot interference detection modes in the current application scenario, a detection timing sequence between at least one light spot interference detection mode is not limited, and the at least one light spot interference detection mode may be executed synchronously or asynchronously.

Optionally, when it is determined that the light spot interference detection mode in the current application scene is at least one, a corresponding interference light spot may be obtained based on each light spot interference detection mode, and whether the interference light spot or an interference light spot point corresponding to the interference light spot is removed may be removed by combining an interference light spot removal strategy set in the actual scene. For example, a plurality of interference light spots (e.g., a first interference light spot, a second interference light spot, a third interference light spot, a fourth interference light spot, a fifth interference light spot, etc.) can be obtained by combining a plurality of light spot interference detection methods, and an intersection can be taken for the interference light spots, for example, based on an interference light spot removal strategy set in an actual application scenario, if a certain light spot is detected as belonging to the "first interference light spot, the second interference light spot, the third interference light spot, the fourth interference light spot, and the fifth interference light spot", the certain light spot is removed.

S205: and performing light spot interference detection processing on at least one light spot by adopting the light spot interference detection mode based on the pixel characteristic parameters to obtain interference light spots.

In a feasible implementation manner, the light spot interference detection manner may be a pixel point intensity detection manner, and the interference light spots are distinguished from the plurality of identified light spots by detecting the pixel point intensity of the light spot pixel point. In an actual application scenario, for example, a reflection intensity map under a current field of view is obtained through a light receiving end of a speckle ietf camera, where a ratio of the number of pixels covered by all actual light spots in the reflection intensity map to the number of pixels in the entire reflection intensity map is usually small, for example, about several percent to ten percent. In the area not covered by the actual light spot, the value of the pixel point or the pixel intensity value (i.e. the Confidence value) in the reflection intensity map or the depth map is not 0, which is caused by the existence of noise such as ambient light, a circuit, a dark circuit, and the like. However, the pixel intensity value (i.e. the Confidence value) of the area not covered by the actual spot is usually very low (even if there is an interference spot, the pixel intensity value of the interference spot is usually much lower than that of the actual spot), and in some scenes, the pixel intensity value (i.e. the Confidence value) of the interference spot or the edge area is lower than that of the actual spot signal light by one to several orders of magnitude. Based on the detection result, the light spot intensity parameters corresponding to the light spot pixel points covered by all the light spots identified in the reflection intensity map can be detected, so that the interference light spots are determined from the identified light spots, namely the interference light spots are determined.

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is a pixel point intensity detection mode, the terminal can perform pixel point intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a first interference light spot; the first interference light spot is also the interference light spot detected by the pixel point intensity detection mode.

Optionally, the pixel intensity detection method may specifically be to draw an intensity statistical graph, such as an intensity histogram representing intensities of pixels, where the intensity histogram may be used to represent pixel intensity conditions of light spot pixels covered by all light spots, for example, light spot pixels with the same pixel intensity value may be fed back, and for example, intensity priorities of the light spot pixels may be fed back.

It can be understood that, in the terminal use stage after leaving the factory, the terminal works based on a certain speckle ietf mode, and the theoretical number of light spots received by the light receiving end in the certain speckle ietf mode, the theoretical number of pixel points in the light spot coverage area, and the like are determined. For example, after the actual application stage of the speckle ietf camera, the theory of how many light spots exist in the field of view, and the theory of the light spots covers several pixel points; further, the number of target spot points of theoretical spot pixel points (without interference points) can be determined in advance, or the target spot point ratio (the ratio of the theoretical spot pixel points to the total pixel points of the map) of the theoretical spot pixel points (without interference points) can be determined. Based on this, when detecting the intensity of the pixel point, only the reference light spot (some embodiments may refer to as a first light spot or a second light spot) indicated by the target light spot proportion or the target light spot quantity needs to be determined from all the light spot pixel points, and the interference point screening is performed by using the light spot intensity parameter corresponding to the reference light spot as a threshold.

In a possible implementation manner, the light spot interference detection manner may be an energy ratio detection manner, and the interference light spots are distinguished from the plurality of identified light spots by detecting the energy ratio of the pixel points included in the light spots. Illustratively, the energy ratio detection mode may traverse at least one light spot, and calculate an energy ratio between a pixel intensity parameter (i.e., a Confidence value) of a central pixel point of the light spot and Confidence values of other pixel points of the light spot except the central point of the light spot, where usually, signal light energy of an actual light spot is concentrated and the energy ratio is larger than that of an interference light spot.

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is an energy ratio detection mode, the terminal may perform energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; the second interference light spot is also the interference light spot detected by the energy ratio detection mode. Illustratively, it is common to calculate the energy ratio of two adjacent spots and then compare the energy ratios of the two spots, and if a spot is an interference spot, the energy ratio of the actual spot is much larger than that of the adjacent interference spot.

In a feasible implementation manner, the light spot interference detection manner may be a pixel point mean value detection manner, and the pixel point mean value detection manner may be a ratio of "the sum of pixel intensity parameters (such as Confidence values) of all pixels covered by a certain light spot in the reflection intensity map" to "the sum of pixel intensity parameters (such as Confidence values) of all pixels within a certain neighborhood range of the light spot", so as to use the ratio as a pixel point mean value;

optionally, the pixel point mean value detection mode may be a ratio of "the sum of the light spot pixel intensity parameters (such as the Confidence value) divided by the mean value of the number of covered pixels" to "the light spot pixel intensity parameters (such as the Confidence value) of all the pixel points within a certain neighborhood range of the light spot divided by the mean value of the number of covered pixels", so as to use the ratio as the pixel point mean value; in general, the smaller the ratio, the greater the likelihood that the spot will be an interference spot

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is a pixel point mean value detection mode, the terminal performs pixel point mean value detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a third interference light spot from the plurality of light spots; for example, the pixel point mean value of the light spot is obtained after the pixel point mean value detection processing, and then the pixel point mean value is compared with the mean value threshold value, so that the interference light spot is determined from the plurality of light spots, namely the interference light spot is determined.

In a possible embodiment, the light spot interference detection manner may be an energy total intensity detection manner, and generally, in at least one small range region corresponding to the reflection intensity map, the number of theoretical light spots in the small range region is preset, and the total energy of a plurality of actual light spots (without interference light spots) in the small range region should be equal to each other, for example, the small range region corresponds to a certain two-dimensional plane of a target object in the visual scene, and the total energy of the generated interference light spots is much smaller than the actual light spots in the small range region. Illustratively, a certain small range area may not be determined, and usually the number of light spots in the reflected energy map is large, and one light spot is usually processed and then corresponds to a parameter value such as one pixel point in the depth map, that is, any n (n is usually a small value) adjacent light spots may be set for comparison.

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the total energy intensity detection mode, the terminal performs total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot; for example, the total energy intensity of the plurality of light spots is obtained after the total energy intensity detection processing, then the total energy intensity of at least one light spot is compared, when the total energy intensity of a certain light spot is smaller, the light spot indicated by the smaller total energy intensity can be used as an interference light spot, that is, at least one of all pixel points covered by the interference light spot can be determined as an interference light spot, for example, the pixel point indicated by the maximum pixel intensity parameter covered by the interference light spot is used as the interference light spot.

In a possible implementation manner, the light spot interference detection manner may be a light spot position detection manner, and may also be understood as fixed position detection, according to some embodiments, in a factory stage of the speckle ietf camera, a position of a light spot on a reflection intensity map obtained by a light receiving end is generally relatively fixed, but a shift of several pixels may occur due to objective reasons such as far-near distance parallax, information divergence, and the like.

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the light spot position detection mode, the terminal may perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. For example, the spot positions of a plurality of spots are obtained during the spot position detection process, and the spot positions of the spots are matched with the corresponding position ranges, so that the interference spots can be determined from the spots.

In some embodiments, after determining the interference light spot, the terminal performs interference cancellation processing on the interference light spot. Illustratively, after the terminal determines the interference light spot based on the Confidence information in the reflection intensity map, at least one pixel point included in the interference light spot may be removed in the reflection intensity map, for example, the entire interference light spot is removed. Illustratively, after the terminal determines the interference light spot, the interference position of the interference light spot in the depth map can be determined based on the position of the interference light spot, so as to perform interference light spot cancellation on the depth map.

In one possible implementation scenario, after determining the interference spot, interference verification may be performed on the interference spot in combination with color image and depth image such as RGB to determine whether the interference spot is a correct interference point.

It can be understood that the terminal may acquire a color image and a depth image, where the reflection intensity map, the depth image and the color image are different types of images for the same target object; that is, when the terminal images a target object in the current scene, the terminal may simultaneously capture the reflection intensity map, the depth map, and the color image for the same target object, and the color image may be an RGB image in general. After the determination of the interference spots, a captured color image is then acquired.

As can be appreciated, the terminal may determine a first position of an interference spot in the depth image, and may determine a second position of at least one neighboring spot corresponding to the interference spot in the depth image; obtaining a depth pixel validation result for the interference spot based on the first location and the second location.

As can be understood, the terminal determines the third position of the interference light spot in the color image and determines the fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; obtaining a color pixel verification result for the interference light spot based on the third position and the fourth position;

schematically, in a reflection intensity map, at least one light spot usually covers a plurality of light spot pixel points, and a depth image and a color image usually correspond to only one pixel point;

illustratively, a light spot may cover a plurality of pixel points, but the light spot is only calculated at a certain pixel position of the light spot (such as a light spot center-of-gravity pixel point) to generate a depth value, and the depth value represents a depth distance value of a small area adjacent to the light spot in a depth image; for example: the original information of all pixel points covered by the light spot can be combined in an original image (RAW image, original image output by TOF camera), and a light spot gravity center pixel point corresponding to the gravity center position of the light spot is taken to generate an effective depth value, and the effective depth value can calculate a depth value based on the original information of all pixel points covered by the light spot.

Illustratively, after the interference light spot is determined, its position in the depth image, i.e. the first position, is usually determined, i.e. the position of the interference light spot corresponding to the effective depth value is taken as the first position (usually the position of the pixel point indicated by the maximum pixel intensity of the interference light spot corresponding to the depth image).

Further, the depth image is usually captured by a TOF camera of the device, the color image such as an RGB image is usually captured by an RGB camera of the device, by calibrating the RGB camera and the TOF camera, a coordinate transformation relation (which may exist in the form of an image coordinate transformation) between the depth image and the color image may be determined, after the terminal determines the interference light spot, a corresponding interference light spot of the interference light spot in the depth image is obtained, and a position of the interference light spot in the depth image is taken as a first position; then determining a third position of the interference light spot in the color image according to the coordinate transformation relation based on the first position in the depth image; similarly, at least one adjacent light spot corresponding to the interference light spot is obtained, then, with reference to the foregoing manner, a corresponding adjacent light spot of the at least one adjacent light spot in the depth image is obtained, and a position of the adjacent light spot in the depth image is taken as a second position; then determining a fourth position of the adjacent light spot in the color image according to the coordinate transformation relation based on the second position; it can be understood that, in the case that the number of adjacent light spots corresponding to the interference light spot is multiple, the second position of the at least one adjacent light spot in the depth image and the fourth position in the color image are obtained in the manner described above.

Optionally, the at least one neighboring light spot and the interference light spot are in a neighboring relationship, and the number of neighboring light spots may be a preset number. The second position of the adjacent pixel point corresponding to the adjacent light spot can be obtained in the depth image, and the fourth position of the adjacent light spot in the color image is determined according to the image coordinate system transformation relation based on the second position of the adjacent pixel point;

in practical application: color images such as RGB images are high in resolution and can provide color information; the depth image may provide depth information. And if the interference light spot is judged in the previous step, performing interference verification on the interference light spot. In the specific implementation: a first position where the interference light spot corresponds to the interference light spot point can be found in the depth image, a second position where the adjacent light spot corresponds to the adjacent light spot can be found in the depth image, and then a third position where the interference light spot corresponds to the interference light spot point and a fourth position where the adjacent light spot corresponding to the interference light spot is located in the adjacent light spot of the RGB image are found on the RGB image;

illustratively, the depth value indicated by the interference light spot at the first position is greatly different from the depth values indicated by other adjacent light spots at the second position, for example, the depth difference between the "depth value of the first position" and the "depth value of the second position" is larger than a certain depth difference threshold, and the color information given by the RGB image is that the interference light spot is similar to the color information corresponding to the "interference light spot and the adjacent light spot" in the RGB image, for example, the color pixel difference (for example, RGB difference) between the "color pixel value (for example, RGB value) at the third position" and the "color pixel value (for example, RGB value) at the third position" is smaller than a certain color difference threshold (for example, a threshold value or a threshold value set for the RGB value); the possibility that the points are possibly the surface of the same object is high, so that the possibility that the judgment of the interference light spot is correct is high; on the contrary, the interference light spot corresponding to the interference light spot may be from a long distance or other low-reflectivity target or a smooth mirror-reflected target, that is, it is possible that the interference light spot may belong to a valid light spot type rather than the interference light spot type.

In a specific implementation, based on the first position and the second position, obtaining a depth pixel verification result for the interference light spot may be: acquiring a first depth value corresponding to the first position and a second depth value corresponding to the second position, calculating a depth value difference value between the first depth value and the second depth value, if the depth value difference value is greater than a depth threshold value, the depth pixel verification result is a first result, such as True, otherwise, the depth pixel verification result is a second result, such as False;

further, based on the third position and the fourth position, obtaining a color pixel verification result for the interference light spot may be: obtaining a first color value (such as an RGB value) corresponding to the third position and a second color value corresponding to the fourth position, calculating a color value difference value between the first color value and the second color value, if the color value difference value is smaller than a color threshold value, the color information of the interference light spot and the color information of the adjacent light spot are generally similar, and the color pixel verification result is a first result, such as True; otherwise, the color pixel verification result is a second result, such as False;

and then combining the depth pixel verification result and the color pixel verification result to determine an interference verification result aiming at the interference light spot. Illustratively, if the depth pixel verification result and the color pixel verification result are both the first results, such as True, the interference verification result for the interference light spot is: the interference verification is passed, otherwise, the interference verification result for the interference light spot is as follows: the interference validation fails.

In the embodiment of the application, at least one light spot in the reflection intensity map is determined by obtaining the reflection intensity map, then the pixel characteristic parameter corresponding to the light spot pixel point in the light spot is obtained, based on the difference of the pixel characteristic parameter between the interference light spot and the effective light spot, the interference light spot can be accurately determined from the at least one light spot, and the light spot interference caused by the superposition of edge signals of a plurality of light spots can be eliminated in an auxiliary manner; the whole image interference processing process reduces the dependence on equipment hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting a high-precision depth measurement result, and improves the robustness in a depth measurement scene; the verification of the interference light spots can be carried out by combining the depth map and the color map, so that the accuracy of the identification of the interference light spots is further improved; and the detection process of the interference light spots is optimized by combining different detection modes or different time sequence detection modes determined based on practical application scenes.

Referring to fig. 5, fig. 5 is a schematic flowchart illustrating an image processing method according to another embodiment of the present disclosure. Specifically, the method comprises the following steps:

s301: acquiring a reflection intensity map, determining at least one light spot in the reflection intensity map, and acquiring pixel characteristic parameters corresponding to light spot pixel points in the light spot;

s302: at least one spot disturbance detection mode for the reflected intensity map is determined.

S303: if the light spot interference detection mode is a pixel point intensity detection mode, determining a first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and acquiring a target light spot proportion aiming at a reflection intensity graph;

according to some embodiments, the light spot interference detection mode may be a pixel point intensity detection mode, and the interference light spots are distinguished from the plurality of identified light spots by detecting the pixel point intensity of the light spot pixel point. In an actual application scenario, for example, a reflection intensity map under a current field of view is obtained through a light receiving end of a speckle ietf camera, where a ratio of the number of pixels covered by all actual light spots in the reflection intensity map to the number of pixels in the entire reflection intensity map is usually small, for example, about several percent to ten percent. In the area not covered by the actual light spot, the value of the pixel point in the reflection intensity map or the depth map or the pixel intensity value (i.e. the Confidence value) is not 0, which is caused by the existence of noise such as ambient light, circuits, dark circuits, etc. However, the pixel intensity value of the area not covered by the actual light spot (which may also be understood as an effective light spot other than the interference light spot) (that is, the Confidence value of the pixel point in the area not covered) is usually very low (even if the interference light spot exists, the pixel intensity value in the interference light spot is usually much lower than the pixel intensity value in the actual light spot), and in some scenarios, the pixel intensity value of the interference light spot or the edge area (i.e., the Confidence value) is lower than that of the signal light of the actual light spot by one to several orders of magnitude. Based on this, the light spot intensity parameters (such as the Confidence values corresponding to the light spot pixel points) corresponding to the light spot pixel points covered by all the light spots identified in the reflection intensity map can be detected, so as to determine the interference light spots from the plurality of identified light spots, that is, determine the interference light spots.

It can be understood that, after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the pixel point intensity detection mode, the terminal may perform pixel intensity detection processing on at least one of the light spots based on the pixel intensity parameter, and in particular implementations, based on the pixel intensity parameter (such as the Confidence value of the pixel point of the light spot covered by the light spot) of the pixel point of the light spot covered by the light spot in the reflection intensity map (also called Confidence map), the priority ranking is carried out according to the value of the pixel intensity parameter of all the light spot covering pixel points (the number of the light spot covering pixel points can be multiple), the first intensity priority corresponding to at least one light spot pixel point (namely the light spot covering pixel point) is determined, it will be appreciated that the pixel intensity parameter is positively correlated with the first intensity priority and the target spot fraction is obtained for the reflected intensity map.

Optionally, the determining of the first intensity priority corresponding to at least one spot pixel (that is, a pixel covered by a spot) may be determined by drawing an intensity statistical graph, for example, the intensity statistical graph may be an intensity histogram representing pixel intensity parameters (Confidence values corresponding to the spot pixels) of the spot pixels, and the intensity histogram may be used to represent pixel intensity conditions of the spot pixels covered by all spots, for example, spot pixels having the same pixel intensity parameter (the same Confidence value) may be fed back, and for example, the intensity priority of at least one spot pixel may be fed back.

It can be understood that, in the terminal use stage after leaving the factory, the terminal works based on a certain speckle ietf mode, and the theoretical number of light spots received by the light receiving end in the certain speckle ietf mode, the number of theoretical light spot pixel points covered by the light spots, and the like are determined. For example, after the actual application stage of the speckle ietf camera, how many light spots theoretically exist in a field of view, and at least one light spot theory covers several light spot pixel points; further, the number of theoretical light spot pixel points covered by the light spot (in some embodiments, the number of theoretical light spot pixel points may also be referred to as reference pixel points) may be determined, or the target light spot occupancy ratio (the ratio of the theoretical light spot pixel points to the total image pixel points) of the theoretical light spot pixel points (without interference points) may be determined. Based on this, when detecting the intensity of the pixel point, only the reference light spot (which may be referred to as a first light spot or a second light spot in some embodiments) needs to be determined from all the light spot pixel points, and the interference screening is performed by using the pixel intensity parameter corresponding to the reference light spot as a threshold.

Schematically, histogram statistics is to count a histogram of the whole Confidence image, and according to prior knowledge, how many light spots are in a field of view after a speckle iTOF camera is designed, and it can be derived from theory that at least one light spot covers several pixel points.

S304: determining an intensity threshold based on the target light spot proportion, the first intensity priority of the light spot pixel points and the pixel intensity parameters of the light spot pixel points; a first interference spot is determined from at least one of the spots based on the intensity threshold.

In a possible implementation manner, a first light spot point may be determined based on the target light spot proportion and a first intensity priority of the light spot pixel point, and a first intensity threshold corresponding to the first light spot point is obtained, and a first interference light spot is determined from at least one light spot based on the first intensity threshold;

illustratively, assuming that the target light spot proportion (that is, the light spot pixel point number proportion) is x%, the first intensity priority of the light spot pixel points may be represented in a histogram form corresponding to the Confidence map, the terminal may determine x% number points before the histogram goes from high to low, so as to determine the first light spot (the x% number point may be used as the first light spot), and the pixel intensity parameter of the first light spot is used as the first intensity threshold, if the Confidence value corresponding to the first light spot is Cx, that is, the Confidence value of the light spot pixel points covered by all the light spots should be larger than Cx theoretically. The interference light spots can be screened by taking Cx as a threshold value, namely, the Confidence values of all the light spot pixel points can be compared with Cx, if the Confidence value of the light spot pixel point is greater than Cx, the light spot pixel point is a non-interference light spot, if the Confidence value of the light spot pixel point is less than or equal to Cx, the light spot pixel point can be taken as an interference light spot, or the light spot pixel point can be taken as a suspected interference light spot, and then the detection is further determined by combining other light spot interference detection modes, for example, the light spot pixel point is taken as an interference light spot if the light spot pixel point is determined to be in accordance with the judgment condition of the interference point by adopting other light spot interference modes. After the interference light spots are screened out, the first interference light spot is determined, for example, a light spot corresponding to the interference light spot can be used as the first interference light spot, and in actual light spot interference detection, most of light spot pixel points covered by the first interference light spot may be the interference light spot.

In a feasible implementation manner, the number of reference pixels corresponding to the light spot is obtained, the number of reference pixels is the theoretical number of light spot pixels covered by the light spot, a target ratio of the target light spot to the number of reference pixels is determined, a second light spot is determined and a second intensity threshold corresponding to the second light spot is obtained based on the target ratio and the first intensity priority of the light spot pixels, and a first interference light spot is determined from at least one light spot based on the second intensity threshold;

schematically, in an actual application scene, the Confidence value (which can be understood as pixel intensity of a pixel point) of a pixel point in a spot edge intersection region of an actual spot is usually greater than the Confidence value of a pixel point such as ambient light noise or circuit noise; in a general scene, there may be a long-distance or low-reflectivity object beyond the design range, and at this time, the energy of the light spot of the signal light reflected by the long-distance object on the light receiving end Sensor is very weak, that is, the value of the pixel intensity parameter is low, and may be lower than the Confidence value of the pixel point in the interference light spot at the intersection of the short-distance light spot edges, that is to say: the number of effective light spots (which can be understood as the number of actual light spots except for interference light spots) on the current reflection intensity map may be lower than a set threshold, for example, lower than the number of pixel points corresponding to the target light spot ratio. In order to improve the accuracy of the identification of speckle interference. The number of reference pixels corresponding to the combined light spot can be used for detecting the interference light spot.

The number of reference pixel points corresponding to the light spots can be understood as the number of light spot pixel points theoretically covered by at least one light spot. It can be understood that, for example, in a terminal use stage after leaving a factory, the terminal operates based on a certain speckle ietf mode, and the theoretical number of light spots received by the light receiving end in the certain speckle ietf mode, the target light spot proportion, the theoretical number of pixel points (that is, the number of reference pixel points) in at least one light spot coverage area, and the like have been determined or can be calculated.

It can be understood that the second intensity threshold may be determined based on a combination of the number of reference pixels corresponding to the at least one light spot and the ratio of target light spots, so as to improve the accuracy of identifying the light spot interference.

Illustratively, if one light spot covers n light spot pixel points (that is, the number of reference pixel points is n), taking the histogram as an example, the first intensity priority of the light spot pixel point may be obtained in a form of a histogram corresponding to a Confidence map, the terminal may determine (x/n)% number points of the histogram from top to bottom, so as to determine the second light spot (the (x/n)% number point may be used as the second light spot), the pixel intensity parameter of the second light spot is used as the second intensity threshold, and for example, the Confidence value corresponding to the second light spot is used as the second intensity threshold Cxna. Further, theoretically, the Confidence value of all the spot coverage pixel points should be larger than Cxna. That is, Cxna can be used as a threshold value to screen interference light spots, if the Confidence value of a light spot pixel point is greater than Cxna, the light spot pixel point is a non-interference light spot, and if the Confidence value of the light spot pixel point is less than or equal to Cxna, the light spot pixel point can be used as an interference light spot. The first interference spot may be determined from the spot corresponding to the interference spot point.

In a possible implementation manner, the first intensity threshold and the second intensity threshold may also be combined to perform balancing to avoid an error when the light spot interferes with the decision, to achieve accuracy when the light spot interferes with the decision, and to avoid missing the interference light spot. Illustratively, the Confidence threshold value is from small to large in the first intensity threshold value and the second intensity threshold value, that is, Cx to Cxna, and correspondingly, the number of interference points is less and less, and a suitable threshold value can be determined by combining the first intensity threshold value and the second intensity threshold value with an actual application scenario to determine the light spot interference.

As can be appreciated, the terminal may perform determining a first light spot point and acquiring a first intensity threshold corresponding to the first light spot point based on the target light spot proportion and the first intensity priority of the light spot pixel point; determining a first light spot point and obtaining a first intensity threshold corresponding to the first light spot point based on the target light spot proportion and the first intensity priority of the light spot pixel point, obtaining the number of reference pixel points corresponding to the light spot, determining a target proportion of the target light spot proportion and the number of the reference pixel points, determining a second light spot point and obtaining a second intensity threshold corresponding to the second light spot point based on the target proportion and the first intensity priority of the light spot pixel point, and determining a first interference light spot from at least one light spot based on the first intensity threshold and the second intensity threshold.

It can be understood that, from the first intensity threshold and the second intensity threshold, that is, from Cx to Cxna, the Confidence threshold is from small to large, and the number of interference points is smaller and smaller, and a threshold can be determined by combining the first intensity threshold and the second intensity threshold to perform interference filtering; in specific implementation, the terminal determines a threshold reference range based on the first intensity threshold and the second intensity threshold, where the threshold reference range is: [ Cx, Cxna ], then a target intensity threshold is obtained from the threshold reference range, a first interference spot is determined from at least one of the spots based on the target intensity threshold.

Alternatively, the target intensity threshold may be any value randomly selected from the threshold reference range as the target intensity threshold.

Optionally, a threshold may also be determined from the first intensity threshold and the second intensity threshold in combination with the current actual application scenario for interference filtering.

Illustratively, under different application scenarios, different requirements for measurement accuracy, interference adaptability and the like may be set, and reference adjustment factors may be set for a plurality of reference application scenarios, where the reference factors are used to determine the target intensity threshold from the threshold reference range. In the actual application stage, an adjustment factor mapping relationship between a plurality of reference application scenes and the reference adjustment factors corresponding to the reference application scenes is established, and after the current application scene is determined, the adjustment factors corresponding to the application scenes can be determined based on the adjustment factor mapping relationship. Determining a target intensity threshold value from a threshold reference range by adopting a preset calculation formula based on the adjusting factor, wherein the threshold reference range is as follows: [ Cx, Cxna ], assuming the adjustment factor is a, the target intensity threshold b is (Cxna-Cx) × a + Cx;

s305: if the light spot interference detection mode is an energy proportion detection mode, acquiring a central point intensity value and an edge point intensity value corresponding to at least one light spot;

according to some embodiments, the energy ratio detection method detects the energy ratio of the pixel points included in the light spot to distinguish the interference light spot from the plurality of identified light spots. Illustratively, the energy ratio detection mode may traverse at least one light spot, and calculate an energy ratio between a pixel intensity parameter (i.e., a Confidence value) of a central pixel point of the light spot and Confidence values of other pixel points of the light spot except the central point of the light spot, where usually, signal light energy of an actual light spot is concentrated and the energy ratio is larger than that of an interference light spot.

The central point intensity value may be understood as a pixel intensity value (equivalent to a Confidence value) of a central pixel point of the light spot, and in some embodiments, the central point intensity value may be regarded as a central pixel point of the light spot, and the pixel intensity value of the central pixel point of the light spot may be regarded as a central point intensity value. The edge point intensity value can be understood as a pixel intensity value (i.e., a Confidence value) of an edge pixel point except for a center pixel point in the light spot. The center point intensity value and the edge point intensity value can be directly obtained based on the reflection intensity map.

Schematically, supposing that a light spot covers m light spot pixel points, theoretically, from the center to the edge of the light spot, the intensity of the light spot pixel points is gradually attenuated, that is, the corresponding Confidence values are gradually attenuated, the attenuation rule basically accords with a Gaussian curved surface, and the Confidence values of one row or one column of light spot pixel points in an area covered by the light spot accord with a Gaussian curve. As shown in fig. 6, fig. 6 is a schematic view of a scene related to light spots, a relationship between the Confidence value of a certain row of light spot pixel points of the light spots and the position of the light spot pixel points in fig. 6 is shown in fig. 6, the light spots on both sides in fig. 6 can be regarded as actual light spots or effective signal light spots, the light spots in the middle are generated by overlapping edge signal light of the light spots on both sides, the effective signal light spots are shown in (a) in fig. 6, the energy intensity of the effective signal light spots is concentrated, the Confidence value of the central pixel is high, and the energy of the light spots on the periphery of the effective signal light spots is rapidly attenuated. That is to say, the ratio of the Confidence value of the central pixel point of the effective light spot to the Confidence value around the light spot is large. As shown in fig. 6 (b), the interference signal light spots are also distributed in a gaussian curved surface, but the energy of the interference signal light spots is dispersed, and the ratio of the Confidence value of the central pixel point to the Confidence value of the edge of the interference signal light spot is far smaller than the ratio of the two adjacent effective light spots.

S306: and performing energy proportion detection processing on at least one light spot based on the central point intensity value and the edge point intensity value, and determining a second interference light spot from at least one light spot.

According to some embodiments, the terminal may perform energy ratio detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a second interference light spot; the second interference light spot is also the interference light spot detected by the energy ratio detection mode. Illustratively, it is common to calculate the energy ratio of two adjacent spots, and then compare the energy ratios of the two spots, and if a spot is an interference spot, the energy ratio of the actual spot is much larger than that of the adjacent interference spot.

It is understood that 1, the terminal may determine the first intensity ratio of the light spot based on the central point intensity value and the edge point intensity value;

the first intensity ratio is also the ratio of the spot center intensity value to the edge point intensity value.

Illustratively, an edge distance may be set, and an edge point of the same light spot indicated by the edge distance is taken by using a light spot center point as a reference, for example, the edge distance may be 2 pixel points, and then the pixel point of the light spot center point spaced by 2 pixel points is taken as the edge point.

2. The terminal respectively acquires first light spots corresponding to the light spots and determines a second intensity ratio of the first light spots;

illustratively, a neighboring light spot adjacent to the light spot may be obtained, and the neighboring light spot is taken as a first light spot corresponding to the light spot; allowing for comparability of the ratio of adjacent spots. The adjacent is understood to mean that the light spot is particularly close to the first light spot corresponding to the light spot, and the two light spots may be from adjacent positions on the surface of the same object in the real scene, that is, the energy and the calculated distance of the two light spots are similar, that is, the two light spots are comparable. If the light spot is far from the first light spot corresponding to the light spot, one light spot may come from the surface of a short-distance object, and the other light spot may come from a long-distance background, and the reflected light of the long-distance background itself may have a weak energy or a non-concentrated energy due to the optical path length, so that the comparability between the light spots far from each other is poor. Based on this, a neighboring spot adjacent to the spot may be acquired as the first spot corresponding to the spot.

Illustratively, a fixed distance may be set, and a spot within the fixed distance from at least one of the spots may be acquired as the first spot.

For example, a distance threshold may be set, and the terminal may obtain a target distance between the light spot and at least one second light spot (the second light spot is a light spot other than the light spot), and determine a first light spot corresponding to the light spot from the at least one second light spot based on the target distance and the distance threshold, that is, obtain the first light spot corresponding to a target distance between the second light spot and the light spot smaller than or equal to the distance threshold.

It will be appreciated that the second intensity ratio is determined in a similar manner to the first intensity ratio, i.e. the ratio of the center point intensity value to the edge point intensity value of the first spot. Illustratively, the "selection of the center point and the edge point" of the first light spot should be consistent with the "selection of the center point and the edge point" of the light spot, for example, the edge point corresponding to the center point of the light spot and the edge point corresponding to the center point of the first light spot are selected based on the same edge distance.

3. The terminal determines a second interference spot from at least one of said spots based on said first intensity ratio and said second intensity ratio of said spots.

In one possible embodiment, a target difference value may be determined based on the first intensity ratio and the second intensity ratio; determining a second interference spot from at least one of the spots based on the target difference and a difference threshold;

illustratively, the target difference is also the difference between the first intensity ratio and the second intensity ratio. The difference threshold is a threshold value determined for a target difference based on pre-determined hardware requirements and application scenarios. And if the target difference is larger than the difference threshold value, determining that the first light spot is an interference light spot. If the target difference is less than or equal to the difference threshold, then the first spot may generally not be an interference spot.

In a possible embodiment, the fluctuation range of the intensity ratio for the first light spot may be determined based on at least one of the second intensity ratios with reference to the first intensity ratio; determining a second interference spot from at least one of the spots based on the intensity ratio fluctuation range and a reference fluctuation range.

For example, in a certain small area including the current light spot, there are k × k adjacent first light spots, a fluctuation range of a ratio between a central pixel Confidence value and an edge Confidence value (that is, a fluctuation range of intensity ratios corresponding to the second intensity ratios and the current first intensity ratio) of the k × k first light spots should not exceed a reference fluctuation range, which may be preset, and if the fluctuation range deviates from the reference fluctuation range, the light spot is regarded as a second interference light spot.

In one or more embodiments, a first intensity ratio of the spot may be determined based on the center point intensity value and the edge point intensity value; illustratively, a threshold value for the first intensity ratio, i.e. an intensity ratio threshold value, is determined based on pre-determined hardware requirements and application scenarios; then, based on the first intensity ratio and an intensity ratio threshold, determining a second interference light spot from at least one of the light spots, for example, if the first intensity ratio is greater than the intensity ratio threshold, the light spot is a valid light spot; and if the first intensity ratio is smaller than or equal to the intensity ratio threshold value, the light spot is an interference light spot.

S307: if the light spot interference detection mode is a pixel point mean value detection mode, determining a target pixel area corresponding to at least one light spot, wherein the light spot is located in the target pixel area;

according to some embodiments, the light spot interference detection mode may be a pixel point mean value detection mode, and the pixel point mean value detection mode may be a mode of detecting a ratio of a sum of pixel intensity parameters (such as Confidence values) of all pixel points of a light spot covered by a certain light spot in a reflection intensity map to a sum of pixel intensity parameters (such as Confidence values) of all pixel points in all regions within a certain neighborhood range of the light spot, so as to use the ratio as a pixel point mean value;

the target pixel region can be a rectangular region, and the rectangular region includes pixel points covered by one or more light spots. The target pixel region may be a circular region. The target pixel region may be all pixels with intensity values greater than the maximum value of intensity in a certain rectangular region by a certain ratio, for example, if the maximum value of Confidence in a certain rectangular region is 100 and a certain ratio is a (if a is 40%), it is determined that all pixels with Confidence values greater than (a × 100) in the certain rectangular region meet the requirement.

S308: determining a spot pixel point mean value aiming at least one spot based on the pixel intensity parameter, and determining a region pixel point mean value corresponding to at least one target pixel region;

schematically, a first total intensity and a total number of spot pixels corresponding to all spot pixels of at least one spot may be obtained, and a quotient of the first total intensity and the total number of spot pixels is used as a spot pixel mean value of the spot;

the first total intensity is the sum of pixel intensity parameters (Confidenc values) of all the light spot pixel points covered by one light spot. The total number of the spot pixel points is also the total number of all the spot pixel points of the spot. Assuming that the light spot a covers 6 light spot pixel points, the first total intensity is the sum of the Confidenc values of the 6 light spot pixel points, and the total number of the light spot pixel points is 6.

Illustratively, the second total intensity and the total number of regional pixels corresponding to all regional pixel points of at least one of the target pixel regions may be obtained, and a quotient of the second total intensity and the total number of regional pixels is used as a regional pixel mean value corresponding to the target pixel region.

The second total intensity is the sum of pixel intensity parameters of all pixel points in the target pixel region. The total number of the regional pixel points is also the number of all the spot pixel points in the region. Assuming that the target pixel region b covers 8 regional pixel points, the second total intensity is the sum of the confident values of the 8 regional pixel points, and the total number of the regional pixel points is 8.

S309: and performing pixel point mean value detection processing on at least one light spot based on the light spot pixel point mean value and the regional pixel point mean value, and determining a third interference light spot from at least one light spot.

Optionally, the pixel point mean value detection method may be: for a certain light spot, calculating the average value C of the sum of the intensity parameters (such as Confidence value) of the pixels of the light spot (namely, the first total intensity) divided by the number of the covered pixels (namely, the total number of the pixels of the light spot)₁", average value C₁Can be used as the mean value of the light spot pixel points; and calculating the average value C of the pixel intensity parameters (namely the second total intensity) of all the pixel points in the target pixel region divided by the number of covered pixel points (namely the total number of the pixel points in the region)₂", average value C₂Can be used as the average value of regional pixel points; average value C₁With the average value C₂Ratio of (i.e. C)₁/C₂) In general, the smaller the ratio, the more likely that the certain spot is an interference spot. Wherein the ratio (i.e. C)₁/C₂) May be referred to as a reference ratio.

It can be understood that, if the light spot interference detection mode is a pixel point mean value detection mode, the terminal performs pixel point mean value detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a third interference light spot from the plurality of light spots; for example, the ratio (i.e. C) for at least one light spot is obtained after the pixel point mean detection processing₁/C₂) Then the ratio (i.e. C)₁/C₂) Comparing with a certain ratio threshold, if the ratio (i.e. C)₁/C₂) If the ratio is less than a certain ratio threshold value, the light spot is taken as an interference light spot, otherwise, the light spot is usually taken as an effective light spot; in the case of a plurality of spots, the plurality of spots may be compared in sequence in the manner described above, so as to determine the interference spot from the plurality of spots, and the interference spot may be determined based on the interference spot.

Wherein the ratio threshold is for a reference ratio (i.e., C)₁/C₂) A set threshold or threshold value.

Schematically, determining a reference ratio corresponding to the spot pixel point mean value and the area pixel point mean value; the terminal may perform pixel mean value detection processing on the at least one light spot based on the reference ratio and the ratio threshold, and determine a third interference light spot from the at least one light spot.

It can be understood that the basis of the pixel point mean value detection mode is that the effective light spot light energy is strong and concentrated, and the energy and Confidence are rapidly attenuated from the light spot center to the edge. For effective spot, C₁/C₂The ratio of (a) to (b), i.e. the reference ratio, is much larger than the ratio corresponding to the interference point. This is because the spot center of the interference spot is itself a result of the superposition of at least two valid spot edges, which is very low in energy. Illustratively, the determination of the interference light spot can be realized by setting a threshold value for the reference ratio.

S310: if the light spot interference detection mode is an energy total intensity detection mode, determining the light spot intensity of a third light spot in a reference area and the light spot intensity mean value corresponding to all fourth light spots based on the pixel intensity parameter;

according to some embodiments, the speckle interference detection method may be an energy total intensity detection method, generally, in at least one reference region (the reference region may be a small region range of a preset size specification) corresponding to the reflection intensity map, the theoretical number of the speckle in the reference region may be preset or calculated and derived, and the total energy (also understood as speckle energy) of the speckle among a plurality of actual speckles (without interference speckle) in the reference region should be equal, for example, the reference region corresponds to a certain two-dimensional plane of a target object in a visual scene, and the speckle energy of the interference speckle generated at this time is usually much smaller than the speckle energy of the actual speckle in the reference region.

The light spot energy can be understood as the sum of the intensity parameters of the pixels of the light spot covering all the pixel points of the light spot, that is, the sum of the Confidence values of the pixels of the light spot covering all the pixel points of the light spot, and in some embodiments, the light spot energy may also be referred to as the light spot intensity or the light spot energy intensity.

The reference area can be generally understood as a small area range with a preset size specification; the method comprises the following steps: the reference region may be a region of r × c pixels, and it can be understood that r × c pixels are in a preset size specification.

In a specific implementation scenario, it is assumed that the reference region should theoretically contain 2 × 2 spots.

One possible implementation may be: the third light spot may be any one of all light spots in the reference area; the third light spot can also be the light spot with the minimum light spot intensity in all the light spots in the reference area;

optionally, the relationship between the third light spot and the fourth light spot may be: all light spots included in the reference area are fourth light spots, and the third light spot is one of all the fourth light spots;

for example, assume that there are 4 spots in the reference area: spot a1, spot a2, spot A3, spot a 4; fourth spot, spot a1, spot a2, spot A3, spot a 4: the third spot, i.e. one of spot a1, spot a2, spot A3, spot a4, for example, the third spot may be the spot with the smallest spot intensity in spot a1, spot a2, spot A3, spot a4, assuming: the spot intensity of spot a1 is: 45. the spot intensity of spot a2 is: 99. the spot intensity of spot a3 is: 100. the spot intensity of spot a4 is: 101, the spot a1 in the reference area may be taken as the third spot.

Optionally, the relationship between the third light spot and the fourth light spot may be: the reference area comprises the third light spot and all fourth light spots

For example, assume that there are 4 spots in the reference area: spot B1, spot B2, spot B3, spot B4; the third spot may be one of "spot B1, spot B2, spot B3, spot B4"; the fourth spot, namely, the spot other than the third spot among "spot B1, spot B2, spot B3, and spot B4": for example, the third spot may be spot B3 in "spot B1, spot B2, spot B3, spot B4", and the fourth spot is: "spot B1, spot B2, spot B4" except for spot B3; schematically, assume: the spot intensity of spot B1 is: 45. the spot intensity of spot B2 is: 99. the spot intensity of spot B3 is: 100. the spot intensity of spot B4 is: 101, the spot B1 with the smallest spot intensity in the reference area can be used as the third spot.

The light spot intensity is the sum of pixel intensity parameters (Confidence values) of all light spot pixel points covered by one light spot; for example: the light spot has 3 light spot pixel points, and the Confidence value of at least one light spot pixel point is 1: x of the light spot pixel point respectively₁2, x of light spot pixel point₂1: x of light spot pixel point₃(ii) a The spot intensity is then: (x)₁₊x₂₊x₃)/3。

The "spot intensity mean value" in the spot intensity mean values corresponding to all the fourth spots can be understood as: the sum of the spot intensities of all the fourth spots (i.e. the sum of the Confidence values of all the fourth spots) to the number of fourth spots.

The average value of the light spot intensities corresponding to all the fourth light spots is also the ratio of the sum of the light spot intensities of all the fourth light spots to the total number of the fourth light spots. For example: the fourth spot has 4 spots, spot a1, spot a2, spot A3, and spot a4, assuming: the spot intensity of spot a1 is: 45. the spot intensity of spot a2 is: 99. the spot intensity of spot a3 is: 100. the spot intensity of spot a4 is: 101, a first electrode and a second electrode; the total number of the fourth light spots is 4, and the average value of the intensities of the light spots corresponding to all the fourth light spots is: (45+99+100+101)/4.

In a feasible implementation manner, the energy total intensity detection manner may also be a non-maximum suppression detection manner, according to some embodiments, such as how many effective light spots exist in a reference area theory in a field of view in an actual application stage of the speckle ietf camera, at least one effective light spot theory covers several pixel points, and a quantity threshold value of the light spots corresponding to the theory is usually determined; non-maxima suppression detection can be based on the actual number of spots in the reference area: when the total number of light spots corresponding to all the light spots in the reference area is less than or equal to the threshold value of the number of light spots corresponding to the reference area (in some embodiments, the threshold value of the number of light spots can be determined based on the theoretical number of light spots), the total energy intensity detection is not required, and at this time, the actual light spots in the reference area are usually valid light spots; when the actual number of light spots is larger than the theoretical number of light spots corresponding to the reference area, the interference light spots can be generally considered to exist.

Illustratively, the reference region may be a region of r × c pixel points, and it can be understood that r × c pixel points are of a preset size specification, and it is assumed that the reference region theory should include 2 × 2 light spots, that is, the light spot number threshold of the reference region is: 4, in practice, the number of spots detected in the reference area in the reflection intensity map is: 5, namely the total number of the light spots corresponding to the reference area is 5; at this time: and if the total number of the light spots is greater than the threshold value of the number of the light spots, performing total energy intensity detection, that is, performing S311 and determining the light spot intensity of the third light spot and the light spot intensity mean value corresponding to all the fourth light spots in the reference region based on the pixel intensity parameter.

Alternatively, the light spot with the minimum spot intensity of all the light spots in the reference area may be regarded as the interference light spot.

Optionally, the light spot number threshold may be determined based on the theoretical light spot number, for example, the light spot number threshold may be equal to the theoretical light spot number; the threshold value of the number of the light spots can be slightly larger than the number of the theoretical light spots so as to consider the light spot error in an actual scene and improve the robustness.

In a possible embodiment, it is predetermined how many effective spots should theoretically appear in the above-mentioned reference area, thus, the distance between the spots or the range of the change in the distance between the spots, which occurs theoretically, can also be determined, i.e. a non-maximum suppression detection can be performed for further determination on the basis of the spot separation distance between two spots in the reference area, by setting a threshold value for the spot separation distance, i.e. a spot separation distance threshold value, after the light spot spacing distance between every two light spots in the reference area is obtained, if the light spot spacing distance is smaller than the spacing distance threshold value, then there is a possibility that an interference spot appears in the reference region of the field of view, S311 and "determining the spot intensity of the third spot and the average of the spot intensities corresponding to all the fourth spots in the reference region based on the pixel intensity parameters" in S310 can be performed.

Optionally, if the separation distance of the light spots is greater than or equal to the separation distance threshold, the ignoring process may be performed.

Optionally, the position relationship between every two of the foregoing light spots may be an adjacent position relationship.

Illustratively, if the light spot separation distance of each two light spots between the adjacent 3 light spots is smaller than the light spot separation distance threshold value, the light spot intensity of the middle light spot is smaller than the light spot intensity average value of the two light spots, or the light spot intensity of the middle light spot is smaller than the light spot intensities of the light spots on the two sides, the energy intensity of the middle light spot is weaker, the intermediate spot may be used as an interference spot, as shown in fig. 7, fig. 7 is a schematic view of a scene involving an interference spot, in fig. 7, the distance between every two adjacent light spots between 3 adjacent light spots is smaller than the distance threshold, the light spot intensity of the middle light spot is smaller than the light spot intensities of the light spots on the two sides, that is, the energy intensity of the middle light spot is weaker, the intermediate light spot can be used as an interference light spot, at least one of all pixel points covered by the interference light spot can be used as an interference light spot, for example, a point indicated by the maximum value of the pixel intensity parameter (Confidence value) in all the pixel points covered by the interference light spot is used as the interference point.

S311: and carrying out energy total intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity mean value, and determining a fourth interference light spot.

In a possible embodiment, the intensity difference between the spot intensity of the third light spot and the average value of the spot intensities, that is, the difference between the spot intensity and the average value of the spot intensities, is determined; the decision is realized by setting a first difference threshold for the intensity difference;

and if the intensity difference is smaller than or equal to a first difference threshold value, taking the third light spot of which the intensity difference is smaller than or equal to the first difference threshold value as a fourth interference light spot.

And if the intensity difference is larger than a first difference threshold value, performing neglect processing.

In a possible embodiment, the intensity ratio of the spot intensity of the third light spot to the average of the spot intensities, i.e. the ratio of the spot intensity to the average of the spot intensities, is determined;

and if the intensity ratio is smaller than or equal to the first ratio threshold, taking the third light spot as a fourth interference light spot. Wherein the first scale threshold is a threshold set for a strength scale.

And if the intensity ratio is larger than a first ratio threshold, performing ignoring processing.

In a specific implementation scenario, it can be understood that if a plurality of light spots are included in a reference region, the intensities of the light spots respectively corresponding to the plurality of light spots may be obtained, a third light spot indicated by the minimum light spot intensity is selected based on the intensities of the light spots of the plurality of light spots, and then all fourth light spots in the reference region are determined. One way is that: all the light spots in the reference area are taken as fourth light spots; one way is that: taking all light spots in the reference area except the third light spot as fourth light spots; and then calculating the light spot intensity mean value corresponding to all the fourth light spots, then calculating the quotient of the light spot intensity of the third light spot and the light spot intensity mean value as an intensity ratio, and if the intensity ratio is smaller than a first ratio threshold value, taking the third light spot as a fourth interference light spot.

S312: if the light spot interference detection mode is a light spot position detection mode, determining the light spot position aiming at the at least one light spot based on the pixel position parameter; acquiring a reference position range corresponding to at least one light spot;

according to some embodiments, the position of the spot on a reflection intensity map obtained by the light receiving end of the speckle ietf camera is usually fixed in the factory stage, but the position of the spot may shift by several pixels due to objective reasons such as distance parallax, information divergence and the like.

S313: and if the light spot position of the light spot is not matched with the reference position range, determining the light spot as a fifth interference light spot.

It can be understood that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the light spot position detection mode, the terminal may perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. For example, the light spot positions of a plurality of light spots are obtained during light spot position detection processing, the light spot positions of the light spots are matched with the position range corresponding to the light spots to detect whether the light spot positions fall into the reference position range, and the light spots are determined as fifth interference light spots; by analogy, a fifth interference spot may be determined from the plurality of spots.

In the embodiment of the application, at least one light spot in the reflection intensity map is determined by obtaining the reflection intensity map, then the pixel characteristic parameter corresponding to the light spot pixel point in the light spot is obtained, based on the difference of the pixel characteristic parameter between the interference light spot and the effective light spot, the interference light spot can be accurately determined from the at least one light spot, and the light spot interference caused by the superposition of edge signals of a plurality of light spots can be eliminated in an auxiliary manner; the whole image interference processing process reduces the dependence on equipment hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting a high-precision depth measurement result, and improves the robustness in a depth measurement scene; and the detection flow of the interference light spots can be optimized by combining different light spot interference detection modes or light spot interference detection modes with different time sequences determined based on practical application scenes.

The image processing apparatus provided in the embodiment of the present application will be described in detail below with reference to fig. 8. It should be noted that the image processing apparatus shown in fig. 8 is used for executing the method of the embodiment shown in fig. 1 to 6 of the present application, and for convenience of description, only the portion related to the embodiment of the present application is shown, and details of the specific technology are not disclosed, please refer to the embodiment shown in fig. 1 to 7 of the present application.

Please refer to fig. 8, which illustrates a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. The image processing apparatus 1 may be implemented as all or a part of a user terminal by software, hardware, or a combination of both. According to some embodiments, the image processing apparatus 1 includes a light spot determining module 11, a parameter obtaining module 12, and an interference determining module 13, and is specifically configured to:

the light spot determining module 11 is configured to acquire a reflection intensity map and determine at least one light spot in the reflection intensity map;

the parameter acquisition module 12 is configured to acquire a pixel characteristic parameter corresponding to a light spot pixel point in the light spot;

an interference determination module 13, configured to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter.

Optionally, as shown in fig. 9, the interference determining module 13 includes:

a detection determining unit 131, configured to determine at least one light spot interference detection manner for the reflection intensity map;

and the interference detection unit 132 is configured to perform light spot interference detection processing on at least one light spot in the light spot interference detection manner based on the pixel characteristic parameters to obtain an interference light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

if the light spot interference detection mode is a pixel point intensity detection mode, performing pixel point intensity detection processing on at least one light spot based on pixel intensity parameters to obtain a first interference light spot;

if the light spot interference detection mode is an energy proportion detection mode, performing energy proportion detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot;

if the light spot interference detection mode is a pixel point mean value detection mode, performing pixel point mean value detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot;

if the light spot interference detection mode is an energy total intensity detection mode, performing energy total intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot;

and if the light spot interference detection mode is a light spot position detection mode, performing light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a first intensity priority corresponding to the spot pixel point based on the pixel intensity parameter, and obtaining a target spot fraction for the reflected intensity map;

and determining an intensity threshold value based on the target light spot occupation ratio, the first intensity priority of the light spot pixel point and the pixel intensity parameter of the light spot pixel point.

A first interference spot is determined from at least one of the spots based on the intensity threshold.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a first light spot point and acquiring a first intensity threshold corresponding to the first light spot point based on the target light spot ratio and the first intensity priority of the light spot pixel point, and determining a first interference light spot from at least one light spot based on the first intensity threshold; or the like, or, alternatively,

acquiring the number of reference pixels corresponding to the light spots, determining a target ratio of the target light spot occupation ratio to the number of the reference pixels, determining a second light spot and acquiring a second intensity threshold corresponding to the second light spot based on the target ratio and the first intensity priority of the light spot pixels, and determining a first interference light spot from at least one light spot based on the second intensity threshold; or the like, or, alternatively,

determining a first light spot point and obtaining a first intensity threshold corresponding to the first light spot point based on the target light spot proportion and the first intensity priority of the light spot pixel point, obtaining the number of reference pixel points corresponding to the light spot, determining a target proportion of the target light spot proportion and the number of the reference pixel points, determining a second light spot point and obtaining a second intensity threshold corresponding to the second light spot point based on the target proportion and the first intensity priority of the light spot pixel point, and determining a first interference light spot from at least one light spot based on the first intensity threshold and the second intensity threshold.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a threshold reference range based on the first intensity threshold and the second intensity threshold;

obtaining a target intensity threshold from the threshold reference range, and determining a first interference light spot from at least one of the light spots based on the target intensity threshold.

Optionally, the interference detecting unit 132 is specifically configured to:

obtaining a central point intensity value and an edge point intensity value corresponding to at least one light spot;

and performing energy proportion detection processing on at least one light spot based on the central point intensity value and the edge point intensity value, and determining a second interference light spot from at least one light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determining a second interference spot from at least one of the spots based on the first intensity ratio and an intensity ratio threshold; and/or the presence of a gas in the gas,

determining a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; respectively acquiring first light spots corresponding to the light spots, and determining a second intensity ratio of the first light spots; determining a second interference spot from at least one of the spots based on the first and second intensity ratios of the spots.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a target difference value based on the first intensity ratio and the second intensity ratio; determining a second interference spot from at least one of the spots based on the target difference and a difference threshold; or the like, or, alternatively,

determining an intensity ratio fluctuation range for the first spot based on at least one of the second intensity ratios with the first intensity ratio as a reference; determining a second interference spot from at least one of the spots based on the intensity ratio fluctuation range and a reference fluctuation range.

Optionally, the interference detecting unit 132 is specifically configured to:

acquiring an adjacent light spot adjacent to the light spot, and taking the adjacent light spot as a first light spot corresponding to the light spot; or the like, or, alternatively,

and acquiring a target distance between the light spot and at least one second light spot, and determining a first light spot corresponding to the light spot from the at least one second light spot based on the target distance and a distance threshold.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a target pixel area corresponding to at least one light spot, wherein the light spot is positioned in the target pixel area;

determining a spot pixel point mean value aiming at least one spot based on the pixel intensity parameter, and determining a region pixel point mean value corresponding to at least one target pixel region;

and performing pixel point mean value detection processing on at least one light spot based on the light spot pixel point mean value of the light spot and the regional pixel point mean value, and determining a third interference light spot from at least one light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

acquiring first total intensity and total number of spot pixel points corresponding to all spot pixel points of at least one spot, and taking the quotient of the first total intensity and the total number of the spot pixel points as the mean value of the spot pixel points of the spot;

and acquiring second total intensity and total number of regional pixels corresponding to all regional pixel points of at least one target pixel region, and taking the quotient of the second total intensity and the total number of the regional pixels as the mean value of the regional pixels corresponding to the target pixel region.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a reference ratio corresponding to the light spot pixel point mean value and the area pixel point mean value;

and performing pixel point mean value detection processing on at least one light spot based on the reference ratio and a ratio threshold value, and determining a third interference light spot from at least one light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

determining the light spot intensity of a third light spot in the reference area and the light spot intensity mean value corresponding to all fourth light spots based on the pixel intensity parameters;

and carrying out energy total intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity mean value, and determining a fourth interference light spot.

Optionally, all the light spots included in the reference region are fourth light spots, and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all fourth light spots.

Optionally, the interference detecting unit 132 is specifically configured to:

acquiring the total number of light spots corresponding to a reference area; if the total number of the light spots is larger than the threshold value of the number of the light spots, the step of determining the light spot intensity of the third light spot in the reference area and the light spot intensity mean value corresponding to all the fourth light spots based on the pixel intensity parameter is executed; or the like, or, alternatively,

and obtaining the light spot spacing distance between every two light spots in the reference area, and if the light spot spacing distance is smaller than the spacing distance threshold value, executing the step of determining the light spot intensity of the third light spot and the light spot intensity mean value corresponding to all the fourth light spots in the reference area based on the pixel intensity parameter.

Optionally, the interference detecting unit 132 is specifically configured to:

determining the intensity ratio of the light spot intensity of the third light spot to the average value of the light spot intensities;

and if the intensity ratio is smaller than or equal to a first ratio threshold value, taking the third light spot as a fourth interference light spot.

Optionally, the interference detecting unit 132 is specifically configured to:

determining a spot location for the at least one spot based on the pixel location parameter; acquiring a reference position range corresponding to at least one light spot;

and if the light spot position of the light spot is not matched with the reference position range, determining the light spot as a fifth interference light spot.

Optionally, as shown in fig. 10, the apparatus 1 further includes:

an image obtaining module 14, configured to obtain a color image and a depth image, where the reflection intensity map, the depth image, and the color image are all different types of images for a same target object;

and an interference verification module 15, configured to perform interference verification on the interference light spot based on the color image and the depth image.

Optionally, as shown in fig. 11, the interference verification module 15 includes:

a color verification unit 151, configured to determine a third position of the interference light spot in the color image, and determine a fourth position of at least one neighboring light spot corresponding to the interference light spot in the color image; obtaining a color pixel verification result for the interference light spot based on the third position and the fourth position;

a depth verification unit 152, configured to determine a first position of the interference light spot in the depth image, and determine a second position of at least one neighboring light spot corresponding to the interference light spot in the depth image; obtaining a depth pixel verification result for the interference light spot based on the first position and the second position;

a result generating unit 153, configured to obtain an interference verification result for the interference light spot based on the color pixel verification result and the depth pixel verification result.

Optionally, as shown in fig. 10, the apparatus 1 further includes:

and the exposure adjusting module 15 is configured to obtain the number of light spots corresponding to the light spots, and adjust the exposure time of the camera if the number of light spots is greater than a number threshold.

It should be noted that, when the image processing apparatus provided in the foregoing embodiment executes the image processing method, only the division of the at least one functional module is illustrated, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the above described functions. In addition, the image processing apparatus and the image processing method provided by the above embodiments belong to the same concept, and details of implementation processes thereof are referred to in the method embodiments and are not described herein again.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

In the embodiment of the application, at least one light spot in the reflection intensity map is determined by obtaining the reflection intensity map, then the pixel characteristic parameter corresponding to the pixel point in the light spot is obtained, the interference light spot can be accurately determined from the at least one light spot based on the difference of the pixel characteristic parameter between the interference light spot and the effective light spot, and the light spot interference caused by the superposition of edge signals of a plurality of light spots can be eliminated in an auxiliary manner; the whole image interference processing process reduces the dependence on equipment hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting a high-precision depth measurement result, and improves the robustness in a depth measurement scene; and the detection flow of the interference light spots can be optimized by combining different light spot interference detection modes or light spot interference detection modes with different time sequences determined based on practical application scenes.

An embodiment of the present application further provides a computer storage medium, where the computer storage medium may store multiple instructions, and the instructions are suitable for being loaded by a processor and being executed by the image processing method according to the embodiment shown in fig. 1 to fig. 7, and specific execution processes may refer to specific descriptions of the embodiment shown in fig. 1 to fig. 7, which are not described herein again.

The present application further provides a computer program product, where at least one instruction is stored, and the at least one instruction is loaded by the processor and executes the image processing method according to the embodiment shown in fig. 1 to 7, where a specific execution process may refer to specific descriptions of the embodiment shown in fig. 1 to 7, and is not described herein again.

Referring to fig. 12, a block diagram of an electronic device according to an exemplary embodiment of the present application is shown. The electronic device in the present application may comprise one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected by a bus 150.

Processor 110 may include one or more processing cores. The processor 110 is connected to at least one of the various components throughout the electronic device using at least one of the interfaces and lines, and performs at least one function of the electronic device 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120 and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), field-programmable gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.

The Memory 120 may include a Random Access Memory (RAM) or a read-only Memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium. The memory 120 may be used to store instructions, programs, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, and the like), instructions for implementing at least one embodiment of the method, and the like, and the operating system may be an Android (Android) system, including a system based on Android system depth development, an IOS system developed by apple, including a system based on IOS system depth development, or other systems. The data storage area may also store data created by the electronic device during use, such as phone books, audio and video data, chat log data, and the like.

Referring to fig. 13, the memory 120 may be divided into an operating system space, in which an operating system runs, and a user space, in which native and third-party applications run. In order to ensure that different third-party application programs can achieve a better operation effect, the operating system allocates corresponding system resources to the different third-party application programs. However, the requirements of different application scenarios in the same third-party application program on system resources also differ, for example, in a local resource loading scenario, the third-party application program has a higher requirement on the disk reading speed; in the animation rendering scene, the third-party application program has a high requirement on the performance of the GPU. The operating system and the third-party application program are independent from each other, and the operating system cannot sense the current application scene of the third-party application program in time, so that the operating system cannot perform targeted system resource adaptation according to the specific application scene of the third-party application program.

In order to enable the operating system to distinguish a specific application scenario of the third-party application program, data communication between the third-party application program and the operating system needs to be opened, so that the operating system can acquire current scenario information of the third-party application program at any time, and further perform targeted system resource adaptation based on the current scenario.

Taking an operating system as an Android system as an example, programs and data stored in the memory 120 are as shown in fig. 14, and a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360, and an application layer 380 may be stored in the memory 120, where the Linux kernel layer 320, the system runtime library layer 340, and the application framework layer 360 belong to an operating system space, and the application layer 380 belongs to a user space. The Linux kernel layer 320 provides underlying drivers for at least one hardware of the electronic device, such as a display driver, an audio driver, a camera driver, a bluetooth driver, a Wi-Fi driver, power management, and the like. The system runtime library layer 340 provides a main feature support for the Android system through some C/C + + libraries. For example, the SQLite library provides support for a database, the OpenGL/ES library provides support for 3D drawing, the Webkit library provides support for a browser kernel, and the like. Also provided in the system runtime library layer 340 is an Android runtime library (Android runtime), which mainly provides some core libraries that can allow developers to write Android applications using the Java language. The application framework layer 360 provides at least one API that may be used when building an application, and developers may build their own applications by using the API, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application program runs in the application layer 380, and the application programs may be native application programs carried by the operating system, such as a contact program, a short message program, a clock program, a camera application, and the like; or a third-party application developed by a third-party developer, such as a game application, an instant messaging program, a photo beautification program, and the like.

Taking an operating system as an IOS system as an example, programs and data stored in the memory 120 are shown in fig. 15, and the IOS system includes: a Core operating system Layer 420(Core OS Layer), a Core Services Layer 440(Core Services Layer), a Media Layer 460(Media Layer), and a touchable Layer 480(Cocoa Touch Layer). The kernel operating system layer 420 includes an operating system kernel, drivers, and underlying program frameworks that provide functionality closer to hardware for use by program frameworks located in the core services layer 440. The core services layer 440 provides system services and/or program frameworks such as a Foundation (Foundation) framework, an account framework, an advertisement framework, a data storage framework, a network connection framework, a geographic location framework, a motion framework, and the like, as needed by the application. The media layer 460 provides audiovisual related interfaces for applications, such as graphics image related interfaces, audio technology related interfaces, video technology related interfaces, audio video transmission technology wireless playback (AirPlay) interfaces, and the like. Touchable layer 480 provides at least one common interface-related framework for application development, and touchable layer 480 is responsible for user touch interaction operations on the electronic device. Such as a local notification service, a remote push service, an advertising framework, a game tool framework, a messaging User Interface (UI) framework, a User Interface UIKit framework, a map framework, and so forth.

In the framework shown in FIG. 15, the framework associated with most applications includes, but is not limited to: a base framework in the core services layer 440 and a UIKit framework in the touchable layer 480. The base framework provides many basic object classes and data types, provides the most basic system services for all applications, and is UI independent. While the class provided by the UIKit framework is a basic library of UI classes for creating touch-based user interfaces, iOS applications can provide UIs based on the UIKit framework, so it provides an infrastructure for applications for building user interfaces, drawing, processing and user interaction events, responding to gestures, and the like.

The Android system can be referred to as a mode and a principle for realizing data communication between the third-party application program and the operating system in the IOS system, and details are not repeated herein.

The input device 130 is used for receiving input instructions or data, and the input device 130 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used for outputting instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 are touch display screens for receiving touch operations of a user on or near the touch display screens by using any suitable object such as a finger, a touch pen, and the like, and displaying a user interface of at least one application program. Touch displays are typically provided on the front panel of an electronic device. The touch display screen may be designed as a full-face screen, a curved screen, or a profiled screen. The touch display screen can also be designed to be a combination of a full-face screen and a curved-face screen, and a combination of a special-shaped screen and a curved-face screen, which is not limited in the embodiment of the present application.

In addition, those skilled in the art will appreciate that the configurations of the electronic devices illustrated in the above-described figures do not constitute limitations on the electronic devices, which may include more or fewer components than illustrated, or some components may be combined, or a different arrangement of components. For example, the electronic device further includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a bluetooth module, and other components, which are not described herein again.

In an embodiment of the present application, the main body of execution of at least one step may be the electronic device described above. Optionally, the execution subject of at least one step is an operating system of the electronic device. The operating system may be an android system, an IOS system, or another operating system, which is not limited in this embodiment of the present application.

The electronic device of the embodiment of the application may further include a display device, and the display device may be at least one device capable of implementing a display function, for example: a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink panel, a Liquid Crystal Display (LCD), a Plasma Display Panel (PDP), and the like. A user may utilize a display device on the electronic device 101 to view information such as displayed text, images, video, and the like. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality) device, an automobile, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device, a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality) device, an automobile, a desktop computing device, a smart phone, an AR (Augmented Reality) device, an automobile, a data storage device, a smart phone, an AR (Augmented Reality, a data storage device, an AR (data storage device, a data,

In the electronic device shown in fig. 12, which may be a terminal, the processor 110 may be configured to invoke the network optimization application stored in the memory 120 and specifically perform the following operations:

In one embodiment, the processor 110 specifically performs the following operations when executing the determining of the interference light spot from the at least one light spot based on the pixel characteristic parameter:

and determining at least one light spot interference detection mode aiming at the reflection intensity map, and performing light spot interference detection processing on at least one light spot by adopting the light spot interference detection mode based on the pixel characteristic parameters to obtain an interference light spot.

In an embodiment, when the processor 110 performs the light spot interference detection processing on at least one light spot in the light spot interference detection manner based on the pixel characteristic parameters to obtain an interference light spot, the following operations are specifically performed:

if the light spot interference detection mode is an energy proportion detection mode, performing energy proportion detection processing on at least one light spot based on the pixel intensity parameters to obtain a second interference light spot;

In an embodiment, when the processor 110 performs the pixel intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a first interference light spot, the following steps are specifically performed:

and determining an intensity threshold value based on the target light spot proportion, the first intensity priority of the light spot pixel point and the pixel intensity parameter of the light spot pixel point.

In one embodiment, the processor 110 specifically performs the following steps when performing the determining of the intensity threshold based on the target light spot proportion, the first intensity priority of the light spot pixel point, and the pixel intensity parameter of the light spot pixel point, and determining the first interference light spot from at least one of the light spots based on the intensity threshold:

determining a first light spot point and acquiring a first intensity threshold corresponding to the first light spot point based on the target light spot ratio and the first intensity priority of the light spot pixel point, and determining a first interference light spot from at least one light spot based on the first intensity threshold; or the like, or a combination thereof,

acquiring the number of reference pixel points corresponding to the light spots, determining a target ratio of the target light spot ratio to the number of the reference pixel points, determining a second light spot and acquiring a second intensity threshold corresponding to the second light spot based on the target ratio and the first intensity priority of the light spot pixel points, and determining a first interference light spot from at least one light spot based on the second intensity threshold; or the like, or, alternatively,

In an embodiment, the processor 110, when performing the determining a first interference light spot from at least one of the light spots based on the first intensity threshold and the second intensity threshold, specifically performs the following steps:

In an embodiment, when the processor 110 performs the energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot, the following steps are specifically performed:

In an embodiment, when the processor 110 performs the energy ratio detection processing on at least one light spot based on the center point intensity value and the edge point intensity value, and determines a second interference light spot from the at least one light spot, the following steps are specifically performed:

In an embodiment, the processor 110 specifically performs the following steps when performing the determining a second interference spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots:

determining a target difference value based on the first intensity ratio and the second intensity ratio; determining a second interference light spot from at least one of the light spots based on the target difference and a difference threshold; or the like, or a combination thereof,

In an embodiment, when the processor 110 performs the acquiring of the first light spot corresponding to the light spot, the following steps are specifically performed: acquiring an adjacent light spot adjacent to the light spot, and taking the adjacent light spot as a first light spot corresponding to the light spot; or the like, or, alternatively,

In an embodiment, when the processor 110 performs the pixel mean value detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot, the following steps are specifically performed: determining a target pixel area corresponding to at least one light spot, wherein the light spot is positioned in the target pixel area;

and performing pixel point mean value detection processing on at least one light spot based on the light spot pixel point mean value and the regional pixel point mean value, and determining a third interference light spot from at least one light spot.

In an embodiment, when the processor 110 determines the light spot pixel point mean value for at least one light spot based on the pixel intensity parameter, and determines the regional pixel point mean value corresponding to at least one target pixel region, the following steps are specifically performed:

and acquiring second total intensity and total number of regional pixels corresponding to all regional pixel points of at least one target pixel region, and taking the quotient of the second total intensity and the total number of the regional pixels as a regional pixel mean value corresponding to the target pixel region.

In an embodiment, when the processor 110 performs the pixel mean value detection processing on at least one light spot based on the light spot pixel mean value and the regional pixel mean value, and determines a third interference light spot from the at least one light spot, the following steps are specifically performed: determining a reference ratio corresponding to the light spot pixel point mean value and the regional pixel point mean value;

In an embodiment, when the processor 110 performs the total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot, the following steps are specifically performed: determining the light spot intensity of a third light spot in the reference area and the light spot intensity mean value corresponding to all fourth light spots based on the pixel intensity parameters; the fourth light spot is a light spot in the reference region other than the third light spot;

In one embodiment, all the light spots included in the reference area are fourth light spots, and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all fourth light spots.

In one embodiment, the processor 110 further performs the following steps before performing the step of determining the spot intensity of the third light spot and the average value of the spot intensities corresponding to all the fourth light spots in the reference region based on the pixel intensity parameter: acquiring the total number of light spots corresponding to a reference area; if the total number of the light spots is larger than the threshold value of the number of the light spots, the step of determining the light spot intensity of the third light spot in the reference area and the light spot intensity mean value corresponding to all the fourth light spots based on the pixel intensity parameter is executed; or the like, or a combination thereof,

and obtaining the light spot spacing distance between every two light spots in the reference area, and if the light spot spacing distance is smaller than a spacing distance threshold value, executing the step of determining the light spot intensity of the third light spot and the light spot intensity average value corresponding to all the fourth light spots in the reference area based on the pixel intensity parameters.

In an embodiment, when the processor 110 performs the total energy intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity average value to determine a fourth interference light spot, specifically perform the following steps:

In an embodiment, when the processor 110 performs the spot position detection processing on at least one spot based on the pixel position parameter to obtain a fifth interference spot, the following steps are specifically performed:

In one embodiment, the processor 110 further performs the following steps after performing the determining of the interference spot from the at least one spot:

acquiring a color image and a depth image, wherein the reflection intensity image, the depth image and the color image are all different images aiming at the same target object;

and performing interference verification on the interference light spots based on the color image and the depth image.

In an embodiment, when performing the interference verification on the interference light spot based on the color image and the depth image, the processor 110 specifically performs the following steps:

determining a first position of the interference light spot in the depth image, and determining a second position of at least one adjacent light spot corresponding to the interference light spot in the depth image; obtaining a depth pixel verification result for the interference light spot based on the first position and the second position;

determining a third position of the interference light spot in the color image, and determining a fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; obtaining a color pixel verification result for the interference light spot based on the third position and the fourth position;

and obtaining an interference verification result aiming at the interference light spot based on the color pixel verification result and the depth pixel verification result.

In one embodiment, the processor 110 further performs the following steps after performing the acquiring the reflected intensity map and determining at least one spot in the reflected intensity map:

acquiring the number of light spots corresponding to the light spots;

if the number of the light spots is larger than the number threshold value, adjusting the exposure time of the camera

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of at least one of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and should not be taken as limiting the scope of the present application, so that the present application will be covered by the appended claims.

Claims

1. An image processing method, characterized in that the method comprises:

2. The method of claim 1, wherein the determining an interference spot from the at least one spot based on the pixel characterization parameter comprises:

and determining at least one light spot interference detection mode aiming at the reflection intensity graph, and performing light spot interference detection processing on at least one light spot by adopting the light spot interference detection mode based on the pixel characteristic parameters to obtain interference light spots.

3. The method according to claim 2, wherein the performing, based on the pixel characteristic parameter, spot interference detection processing on at least one spot in the spot interference detection manner to obtain an interference spot comprises:

4. The method of claim 3, wherein the performing pixel intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a first interference light spot comprises:

5. The method of claim 4, wherein determining an intensity threshold based on the target spot fraction, the first intensity priority of the spot pixel, and the pixel intensity parameter of the spot pixel, and determining a first interference spot from at least one of the spots based on the intensity threshold comprises:

6. The method of claim 5, wherein determining a first interference spot from at least one of the spots based on the first intensity threshold and the second intensity threshold comprises:

7. The method of claim 3, wherein the performing an energy ratio detection process on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot comprises:

8. The method of claim 7, wherein the performing an energy scale detection process on at least one light spot based on the center point intensity value and the edge point intensity value to determine a second interference light spot from the at least one light spot comprises:

9. The method of claim 8, wherein determining a second interference spot from at least one of the spots based on the first intensity ratio and the second intensity ratio of the spot comprises:

determining a target difference value based on the first intensity ratio and the second intensity ratio; determining a second interference spot from at least one of the spots based on the target difference and a difference threshold; or the like, or a combination thereof,

10. The method according to claim 8, wherein the acquiring the first light spot corresponding to the light spot comprises:

11. The method of claim 3, wherein the performing pixel mean detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot comprises:

12. The method of claim 11, wherein the determining a spot pixel mean for at least one of the spots based on the pixel intensity parameters and determining a local pixel mean corresponding to at least one of the target pixel regions comprises:

13. The method of claim 11, wherein the performing pixel mean detection processing on at least one light spot based on the light spot pixel mean and the local pixel mean to determine a third interference light spot from the at least one light spot comprises:

determining a reference ratio corresponding to the light spot pixel point mean value and the regional pixel point mean value;

14. The method of claim 3, wherein the performing a total energy intensity detection process on the at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot comprises:

15. The method according to claim 14, wherein the reference area includes all of the light spots as fourth light spots and the third light spot is one of all of the fourth light spots; or, the reference area includes the third light spot and all fourth light spots.

16. The method according to claim 14, wherein before determining the spot intensities of the third light spot and the average of the spot intensities corresponding to all the fourth light spots in the reference area based on the pixel intensity parameter, the method further comprises:

17. The method of claim 14, wherein the performing an energy total intensity detection process on the third light spot based on the light spot intensity of the third light spot and the light spot intensity mean value to determine a fourth interference light spot comprises:

18. The method according to claim 3, wherein the performing the spot position detection processing on the at least one spot based on the pixel position parameter to obtain a fifth interference spot comprises:

19. The method of claim 1, wherein after determining the interference spot from the at least one spot, further comprising:

20. The method of claim 19, wherein the performing interference verification on the interference spot based on the color image and the depth image comprises:

21. The method of claim 1, wherein after obtaining the reflected intensity map and determining at least one spot in the reflected intensity map, further comprising:

acquiring the number of light spots corresponding to the light spots;

and if the number of the light spots is larger than the number threshold, adjusting the exposure time of the camera.

22. An image processing apparatus, characterized in that the apparatus comprises:

23. A computer storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to carry out the method steps according to any one of claims 1 to 20.

24. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1 to 21.