WO2022205827A1

WO2022205827A1 - Structured light image processing method, obstacle detection method, module and device

Info

Publication number: WO2022205827A1
Application number: PCT/CN2021/122723
Authority: WO
Inventors: 耿文峰; 薄慕婷; 孙佳佳; 吴军
Original assignee: 追觅创新科技(苏州)有限公司
Priority date: 2021-03-29
Filing date: 2021-10-09
Publication date: 2022-10-06
Also published as: CN113221910A

Abstract

A structured light image processing method, an obstacle detection method, a module and a device. The method comprises: acquiring a structured light image to be processed (S110); acquiring a reference image of a target area corresponding to the structured light image, the reference image being an image when the target area is not irradiated by structured light (S120); and carrying out image filtering processing on the structured light image by using the reference image to obtain an optimized structured light image (S130). The described method may reduce or even eliminate the interference of ambient light noise in a structured light image.

Description

Structured light image processing method, obstacle detection method, module and equipment

【Technical field】

The invention relates to the technical field of image processing, in particular to a structured light image processing method, an obstacle detection method, a module and equipment.

【Background technique】

With the rapid development of laser technology, laser detection technology has been gradually applied in various fields. Among them, structured light, as an effective means of laser detection, has been widely used in object recognition, ranging and so on. Specifically, by emitting structured light with a specific shape (such as a line laser, a cross beam, etc.) to the area to be detected, it can be detected whether there is an obstacle in the area according to the structured light pattern in the captured image of the area to be detected , and the relevant information of obstacles can also be determined through the features of the structured light image.

However, in the prior art, in the actual implementation environment of object detection using structured light images, various disturbances often exist, resulting in low reliability and accuracy of obstacle detection.

[Content of the invention]

The present invention provides a structured light image processing method, an obstacle detection method, a module and equipment, so as to reduce or even eliminate the interference of ambient light noise in the structured light image, thereby improving the reliability and accuracy of obstacle detection.

According to a first aspect of the present invention, a structured light image processing method is provided, the method comprising:

Obtain the structured light image to be processed;

acquiring a reference image of the target area corresponding to the structured light image, where the reference image is an image of the target area not irradiated by structured light;

Using the reference image, image filtering is performed on the structured light image to obtain an optimized structured light image.

In a possible implementation manner, the use of the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image includes:

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the to-be-processed structured light image to obtain the optimized structured light image.

In a possible implementation manner, the ambient light related information includes image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image to be processed includes:

The optimized structured light image is obtained by subtracting the image parameter value of each pixel point of the structured light image to be processed by the corresponding image parameter value of each pixel point of the reference image; or,

The optimized structured light image is obtained by subtracting the image parameter value of the local area of the structured light image to be processed by the image parameter value of the corresponding local area of the reference image.

In a possible implementation, the method further includes:

Perform image enhancement processing on the optimized structured light image to obtain a further optimized structured light image.

According to a second aspect of the present invention, an obstacle detection method is provided, the method comprising:

Obtain a structured light image of the target area;

acquiring a reference image of the target area, where the reference image is an image when the target area is not illuminated by structured light;

Using the reference image, image filtering is performed on the structured light image to obtain an optimized structured light image;

According to the optimized structured light image, the obstacle information of the target area is determined.

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the structured light image to obtain the optimized structured light image.

In a possible implementation manner, the acquiring the structured light image of the target area includes:

projecting structured light to the target area;

When the target area is illuminated by the structured light, acquiring an image of the target area as the structured light image;

Correspondingly, the obtaining the reference image of the target area includes:

When the target is not illuminated by the structured light, a reference image of the target area under the ambient light condition corresponding to the structured light image is acquired.

In a possible implementation manner, the ambient light related information includes image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image includes:

In a possible implementation manner, after using the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image, the method further includes:

performing image enhancement processing on the optimized structured light image to obtain a further optimized structured light image;

Correspondingly, determining the obstacle information of the target area according to the optimized structured light image includes:

According to the further optimized structured light image, the obstacle information of the target area is determined.

According to a third aspect of the present invention, a structured light module is provided, the module comprising:

an image acquisition unit, configured to acquire a structured light image to be processed, and to acquire a reference image of a target area corresponding to the structured light image, wherein the reference image is an image when the target area is not irradiated by structured light;

The processing unit is configured to use the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image.

In a possible implementation, the processing unit is configured to:

obtaining ambient light related information from the reference image;

According to a fourth aspect of the present invention, an autonomous mobile device is provided, the device comprising:

device body;

A structured light module, disposed on the main body of the device, configured to acquire a structured light image of the target area, and to acquire a reference image of the target area, wherein the reference image is when the target area is not irradiated by structured light Image;

The processing unit is configured to use the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image; and to determine obstacle information of the target area according to the optimized structured light image.

In a possible implementation, the processing unit is configured to:

obtaining ambient light related information from the reference image;

In a possible implementation, the structured light module includes:

a light-emitting unit for projecting structured light to the target area;

an image acquisition unit, configured to acquire an image of the target area as the structured light image when the target area is illuminated by the structured light; and configured to obtain an image of the target area when the target area is not illuminated by the structure In the case of light irradiation, a reference image of the target area under ambient light conditions corresponding to the structured light image is acquired.

According to the embodiments provided by various aspects of the present invention, a reference image of a target area irradiated by unstructured light can be used to perform image filtering processing on the structured light image, so as to reduce or even eliminate the interference of ambient light, so that better image quality can be obtained. Highly optimized structured light images. Further, using the optimized structured light image to detect obstacles in the target area can effectively improve the reliability and accuracy of obstacle detection.

Other features and aspects of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

【Description of drawings】

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features and aspects of the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a schematic flowchart of a method for processing a structured light image provided by an embodiment of the present invention.

FIG. 2 is a schematic flowchart of a method for detecting an obstacle according to an embodiment of the present invention.

FIG. 3 is a schematic flowchart of a method for processing a structured light image provided by another embodiment of the present invention.

FIG. 4 is a schematic structural diagram of a structured light module according to an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of a module of an autonomous mobile device according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of an implementation scenario of an obstacle detection method provided by an embodiment of the present invention.

FIG. 7 is a schematic diagram of an obstacle detection scene of an autonomous mobile device according to an embodiment of the present invention.

FIG. 8 is the structured light image obtained in an embodiment of the present invention.

FIG. 9 is the reference image obtained in an embodiment of the present invention.

FIG. 10 is the optimized structured light image obtained in an embodiment of the present invention.

FIG. 11 is a schematic flowchart of an obstacle detection method provided by another embodiment of the present invention.

FIG. 12 is a schematic diagram of a device structure of an autonomous mobile device according to an embodiment of the present invention.

【Detailed ways】

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

It is to be understood that the terms "comprising" and "comprising" used in the description and claims of the present invention indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or sets thereof.

It should also be understood that the terminology used in this specification of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the present specification and claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise. It will be further understood that, as used in the present specification and claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

The "plurality" in the embodiments of the present invention refers to two or more. The descriptions of the first, second, etc. appearing in the embodiments of the present invention are only used for illustration and distinguishing the description objects, and have no order, nor do they represent a special limitation on the number of devices in the embodiments of the present invention, and do not constitute a description of the present invention. any limitations of the examples.

FIG. 1 is a schematic flowchart of a method for processing a structured light image provided in an embodiment of the present invention. The method can be applied to any structured light module. Generally, the structured light module can be composed of a structured light emission unit and an image acquisition unit. Specifically, as shown in FIG. 1 , the method may include:

S110: Acquire the structured light image to be processed.

Among them, structured light refers to a laser beam projected onto the surface of an object that can form an optical pattern of a certain shape. For example, a planar laser beam projected onto the surface of an object will form a linear optical pattern, which can be called a line laser. A type of structured light. Of course, the specific shape of the structured light beam and the shape of the optical pattern formed by the light beam are not limited in the present invention. Any shape such as round, square, etc. Through the optical pattern formed by projecting structured light onto a certain area, the information of whether there is an object in the area, as well as the distance, shape, size and other information of the object can be obtained.

In this example, an image containing the optical pattern can be acquired by an image acquisition device such as a camera, a camera, etc., to be used to subsequently determine whether there is an obstacle in the projected target area according to the image, and to calculate the obstacle according to the image. relevant physical information. The image containing the optical pattern is the structured light image to be processed.

However, in some implementation scenarios of the present invention, in addition to the optical pattern, the structured light image to be processed also includes optical noise caused by ambient light in the target area, and the ambient light may include, for example, sunlight, lights, etc. , light reflected from an object, etc. In this way, in the subsequent object recognition process using this structured light image, the object recognition error will be caused by the interference of ambient light noise, resulting in low object recognition accuracy and reliability.

S120: Acquire a reference image of the target area corresponding to the structured light image, where the reference image is an image of the target area when the structured light is not irradiated.

The target area refers to the real physical area mapped by the structured light image, that is, the actual photographing area of the photographing device that photographed the structured light image. Correspondingly, the reference image is a physical area that is not structured light in this physical area. When irradiating, the photographing device photographs the image obtained by the physical area.

In this example, the reference image can be used to obtain ambient light related information when the structured light image is captured, and the ambient light related information can be used to eliminate the interference of ambient light noise in the structured light image. However, in some implementation scenarios of the present invention, there may be situations in which the ambient light changes more complexly or changes frequently. Therefore, further, in some embodiments of the present invention, by controlling the frequency of structured light irradiation and image shooting, the ambient light-related information corresponding to the reference image and the ambient light-related information corresponding to the structured light image can be equal or as close as possible, thereby The referenceability of the ambient light related information is improved. For example, in an embodiment of the present invention, a structured light image and a reference image of a similar time period can be obtained by controlling the time of structured light irradiation and image shooting, so that the shooting moment of the structured light image is the same as the reference image. The time difference between the shooting moments is within 0.1s, so that the ambient light-related information of the structured light image and the reference image is basically the same. Of course, the specific value of the above time difference is only exemplary, and in other embodiments of the present invention, the time difference can also be controlled to be smaller or larger. Specifically, the time difference can be determined according to the change of the actual ambient light and the actual accuracy requirement, which is not limited in the present invention.

S130: Using the reference image, perform image filtering processing on the structured light image to obtain an optimized structured light image.

Image filtering is to suppress the noise of the target image under the condition of preserving the details of the image as much as possible. The quality of the processing effect will directly affect the effectiveness and reliability of the subsequent image analysis.

In this example, the image filtering process mainly removes the noise generated by the ambient light in the structured light image to obtain the optimized structured light image, and then in the subsequent object recognition process using this structured light image, the Minimize the error of object recognition caused by the interference of ambient light noise, and improve the accuracy and reliability of object recognition.

In an embodiment of the present invention, the described use of the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image may include:

S131: Acquire ambient light related information from the reference image.

S132: Perform ambient light filtering processing on the structured light image to be processed according to the ambient light related information to obtain the optimized structured light image.

Wherein, the ambient light related information may include image parameters that can characterize ambient light characteristics, for example, may include any one or more of parameters such as brightness, grayscale, RGB value, saturation, hue, image intensity, etc., or It is a parameter obtained by combining various parameters according to preset weights.

In another embodiment of the present invention, the ambient light related information may include image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image to be processed may include:

Wherein, the image parameter value may be any one or more of parameter values such as brightness, grayscale, RGB value, saturation, hue, image intensity, etc., or may be parameters obtained by combining multiple parameters according to preset weights value of .

For example, in some embodiments of the present invention, the image parameter value may include a brightness value, and correspondingly, performing ambient light filtering processing on the structured light image to be processed may include: filtering the structured light image to be processed The brightness value of each pixel point of the image is subtracted from the brightness value of each pixel point of the corresponding reference image, that is, the brightness value of the corresponding pixel point of the structured light image to be processed and the reference image are subtracted to obtain the optimized Structured light image. Of course, in other embodiments of the present invention, the image parameter value may also be image intensity, RGB value, grayscale, saturation, hue, and other parameter values that can characterize image pixel characteristics. The implementer can select the type of image parameter value according to the actual application scenarios and requirements of the subsequent structured light image. For example, for the application requirements of object recognition, the brightness of the optical pattern has a great influence on the recognition accuracy, and the brightness value can be selected as the The image parameter value is not limited in the present invention. Through image filtering processing, the ambient light noise in the structured light image can be suppressed or even eliminated, so that the optical pattern formed by the structured light irradiation in the image has higher definition and more obvious features.

FIG. 3 is a schematic flowchart of a method for processing a structured light image provided by another embodiment of the present invention. Specifically, as shown in FIG. 3, the method may include:

S310: Acquire the structured light image to be processed.

S320: Acquire a reference image of the target area corresponding to the structured light image, where the reference image is an image of the target area when the structured light is not irradiated.

S330: Using the reference image, perform image filtering processing on the structured light image to obtain an optimized structured light image.

S340: Perform image enhancement processing on the optimized structured light image to obtain a further optimized structured light image.

Among them, image enhancement is an enhancement of useful information in an image, which can be a distortion process, the purpose of which is to improve the visual effect of an image, and is purposeful for the application of a given image. Emphasize the overall or local characteristics of the image, make the original unclear image clear or emphasize some interesting features, expand the difference between the features of different objects in the image, suppress the uninteresting features, and improve the image quality and richness. The amount of information, strengthen the effect of image interpretation and recognition, to meet the needs of some special analysis.

In this example, by performing image enhancement processing on the optimized structured light image, the optical pattern formed by the irradiation of structured light in the image can have higher definition and more obvious features.

Using the above reference image of the target area that can be irradiated with unstructured light, image filtering is performed on the structured light image to reduce or even eliminate the interference of ambient light, so that an optimized structured light image with higher image quality can be obtained.

FIG. 2 is a schematic flowchart of a method for detecting an obstacle according to an embodiment of the present invention. FIG. 6 is a schematic diagram of an implementation scenario of an obstacle detection method provided by an embodiment of the present invention. The method can be applied to an autonomous mobile device, and the autonomous mobile device can be any robot that can automatically move or Electronic devices or smart devices that work automatically. Specifically, as shown in FIG. 2, the method may include:

S210: Acquire a structured light image of the target area.

Among them, structured light refers to a laser beam projected on the surface of an object that can form an optical pattern of a certain shape. For example, as shown in Figure 6, the laser emitters E and F both emit a planar line laser, and a planar laser The beam projected on the obstacle will form a linear optical pattern, such as the linear pattern AB and the linear pattern CD as shown in Figure 6. The beams emitted by the laser emitters E and F can be called line lasers, which belong to structured light. A sort of. Of course, the specific shape of the structured light beam and the shape of the optical pattern formed by it are not limited in the present invention. In some other embodiments of the present invention, the shape of the formed optical pattern may also be any shape such as a line, a cross, a triangle, a circle, and a square. Through the optical pattern formed by projecting structured light onto a certain area, the information of whether there is an object in the area, as well as the distance, shape, size and other information of the object can be obtained.

Wherein, the target area may be the direction of the autonomous mobile device in which obstacles need to be detected, the photographable area of the image acquisition device of the autonomous mobile device, and the size of the photographable area depends on the image acquisition device the visible range (field of view, etc.). Wherein, the image acquisition device may be a camera, a camera, etc. In the case where the structured light is a non-visible light emitter, the image acquisition device may also be a corresponding non-visible light camera, such as an infrared camera.

FIG. 7 is a schematic diagram of an obstacle detection scene of an autonomous mobile device according to an embodiment of the present invention. As shown in FIG. 7 , the visual range of the camera C of the autonomous mobile device is the θ angle range in front of the camera, and the visual range can be regarded as a target area. For autonomous mobile devices, the target area is generally in the In the traveling direction of the device, the two structured light lasers A and B of the device can emit structured light into the visible range. The camera C can obtain the structured light image by capturing an image within the time period when the target area is illuminated by the structured light. If an obstacle appears on the propagation path of the structured light (such as an obstacle in the traveling direction of the autonomous mobile device), a corresponding optical pattern will be formed, and the structured light image of the target area captured by the camera C will contain The optical pattern will be included. According to the optical pattern, the obstacle can be detected, and the distance, shape, size and other related information of the obstacle can also be analyzed.

However, in some implementation scenarios of the present invention, in addition to the optical pattern, the structured light image also includes optical noise caused by ambient light in the target area, and the ambient light may include, for example, sunlight, lights, Light reflected from objects, etc. For example, FIG. 8 is the structured light image obtained in an embodiment of the present invention. As shown in Figure 8, in the structured light image, in addition to the optical pattern formed by the structured light irradiating the obstacle, there is also ambient light noise caused by the existence of various ambient lights. Object recognition will cause recognition errors due to the interference of ambient light noise (for example, there may also be optical patterns generated by similar structured light irradiation in ambient light noise, or the optical patterns formed by structured light are covered by ambient light noise, etc., which will lead to subsequent recognition error), resulting in low recognition accuracy and reliability.

S220: Acquire a reference image of the target area, where the reference image is an image of the target area when the target area is not irradiated by structured light.

The reference image is an image obtained by the photographing device photographing the target area when the target area is not illuminated by structured light.

However, in some implementation scenarios of the present invention, there may be situations in which the ambient light changes more complexly or changes frequently. Therefore, further, in some embodiments of the present invention, by controlling the frequency of structured light irradiation and image shooting, the ambient light-related information corresponding to the reference image and the ambient light-related information corresponding to the structured light image can be equal or as close as possible, thereby The referenceability of the ambient light related information is improved. For example, in an embodiment of the present invention, a structured light image and a reference image of a similar time period can be obtained by controlling the time of structured light irradiation and image shooting, so that the shooting moment of the structured light image is the same as the reference image. The time difference between the shooting moments is within 0.1s, so that the ambient light-related information of the structured light image and the reference image is basically the same. Of course, the specific value of the above time difference is only exemplary, and in other embodiments of the present invention, the time difference can also be controlled to be smaller or larger. Specifically, the time difference can be determined according to the change of the actual ambient light and the actual accuracy requirement, which is not limited in the present invention.

FIG. 9 is the reference image obtained in an embodiment of the present invention. As shown in FIG. 9 , the reference image and the structured light image shown in FIG. 8 correspond to the same target area. In FIG. 9 , there is no linear pattern formed by structured light irradiation, and the ambient light information is related to the environment of the structured light image. The light information is the same.

S230: Using the reference image, perform image filtering processing on the structured light image to obtain an optimized structured light image.

Among them, image filtering is to suppress the noise of the target image under the condition of preserving the image details as much as possible, and the quality of its processing effect will directly affect the effectiveness and reliability of subsequent image analysis.

In an embodiment of the present invention, the use of the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image may include:

S231: Acquire ambient light related information from the reference image.

S232: Perform ambient light filtering processing on the structured light image according to the ambient light related information to obtain the optimized structured light image.

In another embodiment of the present invention, the ambient light related information may include image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image may include:

For example, in some embodiments of the present invention, the image parameter value may include a brightness value, and correspondingly, performing ambient light filtering processing on the structured light image may include: value, subtract the corresponding brightness value of each pixel of the reference image, that is, subtract the brightness value of the corresponding pixel of the structured light image and the reference image, to obtain the optimized structured light image. Of course, in other embodiments of the present invention, the image parameter value may also be image intensity, RGB value, grayscale, saturation, hue, and other parameter values that can characterize image pixel characteristics. The implementer can select the type of image parameter value according to the actual application scenarios and requirements of the subsequent structured light image. For example, for the application requirements of object recognition, the brightness of the optical pattern has a great influence on the recognition accuracy, and the brightness value can be selected as the The image parameter value is not limited in the present invention. Through image filtering processing, the ambient light noise in the structured light image can be suppressed or even eliminated, so that the optical pattern formed by the structured light irradiation in the image has higher definition and more obvious features.

FIG. 10 is the optimized structured light image obtained in an embodiment of the present invention. FIG. 10 is an optimized structured light image obtained after performing image filtering processing on the structured light image shown in FIG. 8 using the reference image shown in FIG. 9 . As shown in Figure 10, in the optimized structured light image, only the optical pattern formed by the illumination of the structured light, without the interference of ambient light noise, can be used to identify obstacles more accurately and reliably and determine the relevant information of obstacles , so that the control system of the autonomous mobile device can take accurate obstacle avoidance or obstacle crossing actions according to the obstacle information.

S240: Determine obstacle information of the target area according to the optimized structured light image.

The obstacle information may include one or more of information such as whether there is an obstacle, the distance (position) of the obstacle, the size, shape, and category of the obstacle.

In another embodiment of the present invention, the acquiring the structured light image of the target area may include:

projecting structured light to the target area;

Correspondingly, the obtaining the reference image of the target area includes:

Further, in an example of the present invention, the autonomous mobile device may have at least two structured light emitters and one camera. In order to make the ambient light information of the reference image and the structured light image the same or have a small difference, during the moving process of the device, the controller of the device can control the two structured light emitters to repeatedly emit light alternately, And there is a period of time when neither of the two transmitters emits light in the period of alternating light emission, and the shooting frequency of the camera is controlled cooperatively. The specific process may be: controlling the first structured light emitter to emit light, taking a structured light image of the target area during the lighting process, and then controlling the first structured light emitter to turn off, and when the two structured light emitters do not emit light In the case of taking the unstructured light image of the target area, the unstructured light image can be used as a reference image of the structured light image for performing image filtering processing on the structured light image. Then control the second structured light emitter to emit light, take a structured light image of the target area, then control the second structured light emitter to turn off, and take an unstructured light image as a reference image for the structured light image, and so on and so forth. Then, the structured light image and the corresponding reference image can be continuously photographed during the movement of the autonomous mobile device, so as to identify obstacles. In this example, by increasing the frequency of light emission and shooting, the acquisition time interval between the structured light image and the corresponding reference image can be shortened, and the ambient light information of the reference image and the ambient light information of the structured light image can be the same or as close as possible. Of course, the above control process is only exemplary. In other embodiments of the present invention, the specific light-emitting and shooting sequence and frequency settings can be set by the implementer according to the actual ambient light changes and/or obstacle recognition accuracy requirements It should be confirmed that the present invention is not limited thereto.

FIG. 11 is a schematic flowchart of an obstacle detection method provided by another embodiment of the present invention. As shown in Figure 11, the method may include:

S410: Acquire a structured light image of the target area.

S420: Acquire a reference image of the target area, where the reference image is an image of the target area when the structured light is not irradiated.

S430: Using the reference image, perform image filtering processing on the structured light image to obtain an optimized structured light image.

S440: Perform image enhancement processing on the optimized structured light image to obtain a further optimized structured light image.

S450: Determine obstacle information of the target area according to the further optimized structured light image.

Using the implementation of an obstacle detection method provided by the above embodiments, the reference image of the target area without structured light irradiation can be used to perform image filtering processing on the structured light image, so as to reduce or even eliminate the interference of ambient light, Thereby, an optimized structured light image with higher image quality can be obtained. Further, using the optimized structured light image to detect obstacles in the target area can effectively improve the reliability and accuracy of obstacle detection.

Based on the structured light image processing methods described in the above embodiments corresponding to FIGS. 1 and 3 , the present invention further provides a structured light module. 4 is a schematic structural diagram of a structured light module according to an embodiment of the present invention. As shown in Figure 4, the module may include:

The image acquisition unit 101 can be configured to acquire a structured light image to be processed, and acquire a reference image of a target area corresponding to the structured light image, wherein the reference image is when the target area is not irradiated by structured light. image.

The processing unit 102 is configured to use the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image.

In an embodiment of the present invention, the processing unit 102 may be configured as:

obtaining ambient light related information from the reference image;

In an embodiment of the present invention, the ambient light related information includes image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image to be processed includes:

The optimized structured light image is obtained by subtracting the image parameter value of the local area of the structured light image to be processed by the image parameter value of the corresponding local area of the reference image. In another embodiment of the present invention, the processing unit 102 may be further configured as:

For the processes that are the same or similar to those in the embodiments shown in FIG. 1 and FIG. 3 involved in the above-mentioned embodiments of the structured light module, the specific implementation methods can be as provided in the respective embodiments corresponding to FIG. 1 and FIG. 3 . The execution method is executed, which will not be repeated here.

Based on the obstacle detection method described in each of the embodiments corresponding to FIG. 2 , FIG. 4 , and FIG. 6 to FIG. 11 , the present invention further provides an autonomous mobile device. FIG. 5 is a schematic structural diagram of a module of an autonomous mobile device according to an embodiment of the present invention. FIG. 12 is a schematic diagram of a device structure of an autonomous mobile device according to an embodiment of the present invention. Specifically, as shown in FIG. 5 and FIG. 12 , the autonomous mobile device may include:

Device body 201 .

The structured light module 202, which can be arranged on the main body of the device 201, can be configured to obtain a structured light image of the target area and obtain a reference image of the target area, wherein the reference image is the target area that has not been Image when illuminated by structured light.

The processing unit 203 may be configured to perform image filtering processing on the structured light image by using the reference image to obtain an optimized structured light image; and determine the obstacle information of the target area according to the optimized structured light image .

In an embodiment of the present invention, the processing unit 203 may be configured as:

obtaining ambient light related information from the reference image;

In another embodiment of the present invention, the structured light module 202 may include:

The light-emitting unit 2021 can be used to project structured light to the target area.

The image acquisition unit 2022 may be configured to acquire an image of the target area as the structured light image when the target area is illuminated by the structured light; and may also be configured to obtain an image of the target area when the target area is not illuminated by the structured light. In the case of the structured light irradiation, a reference image of the target area under ambient light conditions corresponding to the structured light image is acquired.

projecting structured light to the target area;

Correspondingly, the obtaining the reference image of the target area includes:

For the processes that are the same or similar to those in the embodiments shown in FIG. 2 and FIG. 4 to FIG. 11 involved in the above-mentioned embodiments of the structured light module, the specific execution methods may be as follows: The execution modes provided in the various embodiments are executed, and details are not described here.

The processing unit described in the above embodiments may be, for example, but not limited to, a CPU, a GPU, an MCU, a processing chip implemented based on an FPGA or a CPLD, or a single-chip microcomputer.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

Various embodiments of the present invention have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

A structured light image processing method, characterized in that the method comprises:

Obtain the structured light image to be processed;

acquiring a reference image of the target area corresponding to the structured light image, where the reference image is an image of the target area not irradiated by structured light;

Using the reference image, image filtering is performed on the structured light image to obtain an optimized structured light image.
The method according to claim 1, wherein the using the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image comprises:

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the to-be-processed structured light image to obtain the optimized structured light image.
The method according to claim 2, wherein the ambient light related information includes image parameter values of each pixel of the reference image, and correspondingly, the ambient light filtering is performed on the structured light image to be processed. Processing includes:

The optimized structured light image is obtained by subtracting the image parameter value of each pixel point of the structured light image to be processed by the corresponding image parameter value of each pixel point of the reference image; or,

The optimized structured light image is obtained by subtracting the image parameter value of the local area of the structured light image to be processed by the image parameter value of the corresponding local area of the reference image.
The method of claim 1 or 2, wherein the method further comprises:

Perform image enhancement processing on the optimized structured light image to obtain a further optimized structured light image.
An obstacle detection method, characterized in that, applied to an autonomous mobile device, the method comprising:

Obtain a structured light image of the target area;

acquiring a reference image of the target area, where the reference image is an image when the target area is not illuminated by structured light;

Using the reference image, image filtering is performed on the structured light image to obtain an optimized structured light image;

According to the optimized structured light image, the obstacle information of the target area is determined.
The method according to claim 5, wherein, using the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image comprising:

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the structured light image to obtain the optimized structured light image.
The method of claim 5, wherein the acquiring the structured light image of the target area comprises:

projecting structured light to the target area;

When the target area is illuminated by the structured light, acquiring an image of the target area as the structured light image;

Correspondingly, the obtaining the reference image of the target area includes:

When the target is not illuminated by the structured light, a reference image of the target area under the ambient light condition corresponding to the structured light image is acquired.
The method according to claim 6, wherein the ambient light related information includes image parameter values of each pixel of the reference image, and correspondingly, performing ambient light filtering processing on the structured light image comprises:

The optimized structured light image is obtained by subtracting the image parameter value of each pixel point of the structured light image to be processed by the corresponding image parameter value of each pixel point of the reference image; or,

The optimized structured light image is obtained by subtracting the image parameter value of the local area of the structured light image to be processed by the image parameter value of the corresponding local area of the reference image.
The method according to any one of claims 5 to 8, wherein after the reference image is used to perform image filtering processing on the structured light image to obtain an optimized structured light image, the method Also includes:

performing image enhancement processing on the optimized structured light image to obtain a further optimized structured light image;

Correspondingly, determining the obstacle information of the target area according to the optimized structured light image includes:

According to the further optimized structured light image, the obstacle information of the target area is determined.
A structured light module, characterized in that the module comprises:

an image acquisition unit, configured to acquire a structured light image to be processed, and to acquire a reference image of a target area corresponding to the structured light image, wherein the reference image is an image when the target area is not irradiated by structured light;

The processing unit is configured to use the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image.
The module of claim 10, wherein the processing unit is configured to:

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the to-be-processed structured light image to obtain the optimized structured light image.
An autonomous mobile device, characterized in that the device comprises:

device body;

A structured light module, disposed on the main body of the device, configured to acquire a structured light image of the target area, and to acquire a reference image of the target area, wherein the reference image is when the target area is not irradiated by structured light Image;

The processing unit is configured to use the reference image to perform image filtering processing on the structured light image to obtain an optimized structured light image; and to determine obstacle information of the target area according to the optimized structured light image.
13. The apparatus of claim 12, wherein the processing unit is configured to:

obtaining ambient light related information from the reference image;

According to the ambient light related information, ambient light filtering processing is performed on the structured light image to obtain the optimized structured light image.
The device of claim 12, wherein the structured light module comprises:

a light-emitting unit for projecting structured light to the target area;

an image acquisition unit, configured to acquire an image of the target area as the structured light image when the target area is illuminated by the structured light; and configured to obtain an image of the target area when the target area is not illuminated by the structure In the case of light irradiation, a reference image of the target area under ambient light conditions corresponding to the structured light image is acquired.