WO2020019345A1

WO2020019345A1 - Coherent light-based obstacle avoidance device and method

Info

Publication number: WO2020019345A1
Application number: PCT/CN2018/097659
Authority: WO
Inventors: 王星泽; 舒远
Original assignee: 合刃科技（深圳）有限公司
Priority date: 2018-07-27
Filing date: 2018-07-27
Publication date: 2020-01-30
Also published as: CN111213069B; CN111213069A

Abstract

A coherent light-based obstacle avoidance device and method. The method comprises: obtaining, by means of ultrasound, the distance d between a detected object (10) and an obstacle avoidance device (20) (S701); obtaining, on the basis of the coherent light, n vibrating speckle images of the detected object (10) under the ultrasonic stimulation (S702); obtaining, according to the n vibrating speckle images, a vibration waveform signal of the detected object (10); and determining, according to the vibration waveform signal, the category of the detected object (10) (S703); and prompting a user of the distance d between the user and the detected object (10) and the category thereof (S704). The method can comprehensively detect the object in the environment, and improves the object recognition precision, thereby improving the precision of the object recognition-based navigation for the blind, security monitoring level, and automotive navigation system.

Description

Obstacle avoidance device and method based on coherent light

Technical field

The present application relates to the field of electronic technology, and in particular, to an obstacle avoidance device and method based on coherent light.

Background technique

Due to physical defects, blind people have many inconveniences in life and work. How to walk safely is the biggest problem in the life of blind people.

Existing auxiliary blind guide devices to ensure the safety of the blind are mainly based on a single sensor hardware, such as ultrasound and infrared, to detect obstacle information, and then prompt the user to avoid collision danger through sound or vibration. The ultrasonic sensor emits ultrasonic waves. When the ultrasonic waves encounter obstacles in the air, they will be reflected back and converted into electrical signals by the ultrasonic receiving probe. It can only be calculated by measuring the time difference between the transmitted sound wave and the received sound wave and multiplying the propagation speed The distance from the launch point to the obstacle. When laser and infrared sensors work, they emit laser pulses or infrared light aiming at obstacles. After being reflected by the obstacle, they are scattered in all directions, so that part of the scattered light is returned to the receiving sensor to receive its weak optical signal, so that the light pulse is recorded and processed Determine the distance from the time elapsed from the launch to the return. However, the solution for judging the position and distance of obstacles by using the time difference between signals transmitted and received by the sensor has a single function and poor accuracy, and cannot comprehensively detect environmental information.

And in security surveillance and car navigation systems, both visual recognition is used to realize object recognition, and then security surveillance and navigation are implemented; however, in a dark environment, visual imaging cannot accurately identify objects, which leads to security surveillance. Security loopholes, car navigation systems cannot accurately navigate.

Summary of the Invention

The embodiments of the present application provide an obstacle avoidance device and method based on coherent light. The embodiments of the present application can comprehensively detect objects in the surrounding environment, improve the accuracy of object recognition, and further improve blind navigation and security monitoring based on the object recognition. Accuracy of the rating system.

In a first aspect, an embodiment of the present application provides an obstacle avoidance device based on coherent light, including:

An ultrasonic sensor, a coherent light sensor, a high-speed camera connected to the coherent light sensor, and a processing device connected to both the ultrasonic sensor and the high-speed camera;

The ultrasonic sensor is configured to obtain a distance d between the detected object and the obstacle avoidance device, and transmit the distance d to the processing device;

The coherent light sensor is configured to emit coherent light to the detected object, receive reflected coherent light, and transmit the reflected coherent light to the high-speed camera;

The high-speed camera is configured to obtain n vibrational speckle images based on the reflected coherent light, and the vibrational speckle images are speckle images of the detected object generating vibrations under the stimulation of the ultrasonic wave. ; N is an integer greater than 1;

The processing device is configured to obtain a vibration waveform signal of the detected object according to the n speckle images of the vibration; and determine a category of the detected object according to the vibration waveform signal.

In a possible embodiment, the processing device acquiring the vibration waveform signal of the detected object according to the n speckle images of the vibration includes:

The processing device acquires M speckle comparison maps according to the n speckle images of vibration; the M is an integer greater than 1 and less than or equal to the n;

The processing device performs a clustering operation on the M spotted images according to a K-means clustering algorithm to obtain k clustered images, where k is an integer greater than 1 and less than M;

The processing device acquires a vibration waveform signal of the detected object according to the k cluster images.

In a possible embodiment, the processing device determining the type of the detected object according to the vibration waveform signal includes:

The processing device performs a fast Fourier transform on the vibration waveform signal to obtain a vibration spectrum of the detected object;

The processing device inputs the vibration frequency spectrum, the distance d, the ultrasonic frequency spectrum and the measurement environment information into an object recognition model and performs a neural network operation to obtain a calculation result;

Obtaining an object category corresponding to the calculation result from a correspondence table between the calculation result and the object category to determine the category of the detected object.

In a possible embodiment, the obstacle avoidance device further includes: an environmental information detection module and a reminder device connected to the processing device;

The environmental information detection module is configured to detect and obtain information of the measurement environment, and the information of the measurement environment includes a temperature value, a wind speed value, and a humidity value;

The reminding device is used to remind a user of the distance d between the detection object and the type of the detection object.

In a possible embodiment, before the processing device determines the category of the detected object according to the vibration waveform signal, the processing device is further configured to:

Acquiring multiple sets of training parameters, each set of training data of the plurality of sets of training data corresponding to an object category;

Performing a neural network training according to the plurality of sets of training parameters to obtain the object recognition model;

Inputting the plurality of sets of training parameters to the object recognition model for calculation respectively to obtain a plurality of sets of calculation results, and each set of calculation results in the plurality of sets of calculation results corresponds to an object category;

According to the plurality of sets of calculation results, a correspondence table between the calculation results and the object category is obtained, and the correspondence table between the calculation results and the object category includes a calculation result range and a corresponding object category, and an upper limit of the calculation result range and The lower limit is the maximum and minimum of a set of calculation results corresponding to the object category.

In a second aspect, an embodiment of the present application provides a method for avoiding obstacles based on coherent light, including:

Acquiring the distance d between the detected object and the obstacle avoidance device through ultrasonic waves;

Speckle images of n vibrations of the detected object under the ultrasound stimulation are acquired based on the coherent light; the n is an integer greater than 1.

Acquiring a vibration waveform signal of the detected object according to the n vibration speckle images; and determining a category of the detected object according to the vibration waveform signal;

The user is reminded of the distance d from the detected object and the type of the detected object.

In a possible embodiment, acquiring the vibration waveform signal of the detected object according to the n speckle images of the vibration includes:

Obtaining M speckle comparison maps according to the n vibrational speckle images; the M is an integer greater than 1 and less than or equal to the n;

From the M speckle contrast images, arbitrarily select k speckle contrast images as k initial cluster centers, where k is an integer smaller than the M;

For any speckle contrast image p in the Mk speckle contrast images, calculate a distance value from each of the initial cluster centers of the k initial cluster centers to obtain k distance values; where the Mk sheets The speckle contrast image is a speckle contrast image among the M speckle contrast images except for the k speckle contrast images that serve as the initial cluster center;

The initial clustering center corresponding to the smallest distance value among the k distance values is selected as the cluster described in the speckle contrast image p; according to this method, k clustered images are obtained, where k is greater than 1 and An integer less than said M;

According to the k cluster images, a vibration waveform signal of the detected object is obtained.

In a possible embodiment, determining the category of the detected object according to the vibration waveform signal includes:

Performing a fast Fourier transform on the vibration waveform signal to obtain a vibration spectrum of the detected object;

Input the vibration frequency spectrum, the distance d, the ultrasonic frequency spectrum and the measurement environment information into an object recognition model and perform a neural network operation to obtain a calculation result;

In a possible embodiment, the method further includes:

Detecting and acquiring information of the measurement environment, where the information of the measurement environment includes a temperature value, a wind speed value, and a humidity value;

Acquire the frequency spectrum of the ultrasound.

In a possible embodiment, before the determining the type of the detected object according to the vibration waveform signal, the processing device is further configured to:

According to a third aspect, an embodiment of the present application further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, includes part or all of the steps of the method described in the second aspect above

It can be seen that in the solution of the embodiment of the present application, the distance d between the detected object and the obstacle avoidance device is obtained by ultrasonic waves; n speckle images of the vibration of the detected object under ultrasonic stimulation are acquired based on coherent light; n vibration speckle images to obtain the vibration waveform signal of the detected object; and determine the type of the detected object based on the vibration waveform signal; remind the user of the distance d between the detected object and the type of the detected object. The embodiments of the present application can comprehensively detect objects in the surrounding environment, improve the accuracy of object recognition, and further improve the accuracy of blind navigation, security monitoring level, and car navigation system based on the object recognition.

These or other aspects of the present application will be more concise and easy to understand in the description of the following embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solutions in the embodiments of the present application or the prior art more clearly, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without paying creative labor.

1 is a schematic diagram of an application scenario of an obstacle avoidance device based on coherent light according to an embodiment of the present application;

Figure 2 is a speckle image of the vibration of the detected object;

3 is a schematic diagram of a vibration waveform obtained from a speckle image;

FIG. 4 is a vibration waveform diagram and a corresponding spectrum diagram according to an embodiment of the present application; FIG.

5 is a vibration spectrum corresponding to vibration waveforms of different objects under the same ultrasonic stimulation;

6 is a schematic diagram of an object recognition model according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an obstacle avoidance process based on coherent light according to an embodiment of the present application.

detailed description

The embodiments of the present application are described below with reference to the drawings.

Referring to FIG. 1, FIG. 1 is a schematic diagram of an application scenario of an obstacle avoidance device based on coherent light according to an embodiment of the present application. As shown in FIG. 1, the application scenario includes: a detected object 10 and an obstacle avoidance device 20.

The detected object 10 may be a pedestrian, glass, tree, metal, plastic, or other object.

The obstacle avoidance device 20 includes an ultrasonic sensor 201, a coherent light sensor 202, a high-speed camera 203 connected to the coherent light sensor 202, a processing device 204 connected to both the ultrasonic sensor 201 and the high-speed camera 203, and The reminder device 205 is connected to the processing device 204.

The above-mentioned ultrasonic sensor 201 includes an ultrasonic transmitter 2012, an ultrasonic receiver 2011, and a first processor 2013 connected to the ultrasonic transmitter 2012 and the ultrasonic receiver 2011. The ultrasonic transmitter 2012 transmits ultrasonic waves to the object 10 to be detected, and the ultrasonic receiver 2011 receives ultrasonic waves reflected by the object 10 to be detected. The first processor 2013 determines the flying time of the ultrasonic wave based on the time when the ultrasonic transmitter 2012 transmits the ultrasonic wave and the ultrasonic receiver 2011 receives the reflected ultrasonic wave, and then determines the detected object 10 and the ultrasonic wave based on the ultrasonic flight time and the ultrasonic speed. The distance d between the obstacle avoidance devices 20. The first processor 2013 includes a communication unit, and the first processor 2013 sends the distance d to the processing device 204 through the communication unit.

The above-mentioned coherent light sensor 202 includes a coherent light transmitter 2022, a coherent light receiver 2021, a first lens 2023, and a second lens 2024. The coherent light emitter 2022 generates incident coherent light, and the incident coherent light is irradiated onto the object 10 to be detected through the second lens. The coherent light receiver receives coherent light reflected by the detected object 10 and passing through the first lens 2023, that is, reflected coherent light. The coherent light receiver 2021 transmits the reflected coherent light to the high-speed camera 203 after receiving the reflected coherent light.

The high-speed camera 203 obtains n speckle images of the detected object according to the received coherent light, and transmits the n speckle images to the processing device 204, where n is an integer greater than 1.

It should be noted that after the active ultrasonic signal encounters the object, it will cause a weak vibration of the object. At this time, a speckle image of the vibration of the detected object 10 can be obtained according to the coherent light emitted in the same direction as the above-mentioned ultrasonic emission direction, according to the reflected or scattered coherent light; and then, according to the obtained n pieces of the detected object at the same interval, 10, a speckle image of vibration, to obtain a vibration waveform signal of the vibrating object 10 vibrating under the action of the ultrasonic wave.

Due to the different material properties, structure distribution and distance from the obstacle avoidance device of the detected object, when the ultrasonic waves of the same frequency are irradiated on different detected objects, the vibration waveform signals of different detected objects are different, so they can The type of the detected object is determined according to the vibration waveform signal of the detected object.

It should be noted that the above-mentioned ultrasonic sensor 2041 and the coherent light sensor 202 start to work at the same time, and the transmission angle is the same. That is, the above-mentioned ultrasonic sensor 201 transmits ultrasonic waves to the detected object 10 and receives reflected ultrasonic waves. The detected object 10 emits coherent light and receives reflected coherent light.

Because the propagation speed of sound waves in the air is much slower than the speed of light, the time at which the coherent light transmitter emits coherent light lags behind the time at which the ultrasound transmitter emits ultrasound, so that the ultrasound emitted by the ultrasound transmitter is reflected by the detected object. The above-mentioned coherent light transmitter starts to emit coherent light.

Specifically, the processing device 204 obtains n speckle images of the vibration of the detected object 10 according to the above method, and then according to the time interval between any two adjacent speckle images in the n vibration speckle images Is Δt. As shown in FIG. 2, the speckle image includes a plurality of spots, and the collection times of the four speckle images from left to right are t, t + Δt, t + 2Δt, and t + 3Δt, respectively. As shown in FIG. 2, the speckle position in the speckle image acquired at different times will change.

The processing device 204 determines the information on the change of the spot with time displacement in the speckle image within the time period nΔt according to the n speckle images of vibration, and further obtains the detected object based on the information about the change of the spot with time displacement in the speckle image. A vibration waveform signal of the object 10.

Specifically, as shown in FIG. 3, the processing device 204 acquires n speckle images according to the foregoing method, and the speckle images are speckle images of vibrations of the detected object. The time interval between the collection times of any two adjacent speckle images in the above n speckle images is Δt, as shown in FIG. 3 a. The processing device 204 obtains M speckle contrast images according to the n speckle images, as shown in FIG. 3B, where M is an integer greater than 1 and less than or equal to n. Then, the processing device 204 performs clustering operation on the M spot contrast images according to the K-means clustering algorithm to obtain k clustered images. As shown in FIG. 3C, the above k is an integer greater than 1 and less than M. .

Further, the processing device 204 arbitrarily selects k speckle contrast images from the M speckle contrast images as k initial cluster centers; and then the processing device 204 calculates each speckle contrast image in the Mk speckle comparison images. A distance value from each of the k initial clustering centers, wherein the Mk spotted contrast images are among the M spotted contrast images except for the k pieces of spotted contrast images as the initial clustering centers. Contrast image of outer spots.

For any speckle contrast image p in the above M-k speckle contrast images, after performing the above calculation, k distance values are obtained, and each distance value corresponds to an initial clustering center. The processing device 204 selects the initial clustering center corresponding to the smallest distance value among the k distance values as the cluster to which the blob contrast image belongs. According to the above method, the above processing device 204 obtains k cluster images, as shown in FIG. 3C. The processing device 13 obtains a vibration waveform signal of the detected object based on the k cluster images, as shown in FIG. 4A.

After obtaining the vibration waveform signal of the detected object 10, the processing device 204 performs a fast Fourier transform on the vibration waveform signal to obtain the vibration spectrum of the detected object, as shown in FIG. 4B. The vibration spectrum contains rich information, such as the ultrasonic signals emitted by the ultrasonic transmitter, the structure and material properties of the object to be detected, and the movement of the obstacle avoidance device itself.

Under the stimulation of the same fixed frequency ultrasonic wave, the vibration spectrum corresponding to the vibration waveform signals of different objects is different. As shown in FIG. 5, FIG. 5A is a vibration spectrum diagram when the detected object is a tree, FIG. 5B is a vibration spectrum diagram when the detected object is a pedestrian, and FIG. 5C is a detected object. The vibration spectrum diagram when glass is used, FIG. 5D is the vibration spectrum diagram when the detected object is metal, and FIG. 5E is the vibration spectrum diagram when the detected object is plastic.

In a possible embodiment, the obstacle avoidance device 20 further includes an environmental information detection module, which is configured to detect information of a current measurement environment. After the environmental information detection module acquires the information of the measurement environment, the measurement environment information is transmitted to the processing device 204.

The environmental information measurement module includes sensors such as a temperature sensor, a wind speed sensor, and a humidity sensor, and the measurement environment information includes temperature, wind speed, and humidity values.

After the processing device 204 obtains the vibration spectrum of the detected object 10, the processing device 204 converts the vibration spectrum of the detected object, the distance d between the detected object and the obstacle avoidance device 20, and the ultrasonic wave generated by the ultrasonic transmitter. The corresponding ultrasonic spectrum and the information of the above-mentioned measurement environment are input into an object recognition model for neural network operation. The object recognition model is a neural network model. The object recognition model is used to perform a neural network operation on the information of the vibration spectrum, the ultrasonic spectrum, the distance d, and the measurement environment to obtain at least one calculation result, and each calculation result corresponds to an object type. The processing device 204 may according to the calculation result, that is, The type of the detected object can be determined. As shown in FIG. 6, the above object recognition model includes an input layer, an intermediate layer, and an output layer; after the information of the vibration spectrum, the ultrasonic spectrum, the distance d, and the measurement environment is input from the input layer and after the intermediate layer operation, the output layer can output The five kinds of calculation results include a first calculation result, a second calculation result, a third calculation result, a fourth calculation result, and a fifth calculation result, and the corresponding object categories are trees, pedestrians, glass, metal, and plastic.

Further, the output layer of the object recognition model may output any one or any combination of the four calculation results, that is, the object category output by the object recognition model may be any of trees, pedestrians, glass, metal, and plastic. Species; or any combination.

Further, the processing device 204 determining the type of the detected object according to the calculation result includes:

The processing device 204 determines an object type corresponding to the calculation result according to a correspondence table between the calculation result and the object type.

The correspondence table between the calculation result and the object type is shown in Table 1.

计算结果范围Calculation result range	物体类型Object type
(a1,a2](a1, a2)	树木Trees
(a2,a3](a2, a3)	行人pedestrian
(a3,a4](a3, a4)	玻璃glass
(a4,a5](a4, a5)	金属metal
(a5,a6)(a5, a6)	塑料plastic

Table 1

Specifically, five types of objects are listed in the correspondence table between the calculation result and the object type, which are tree, pedestrian, glass, metal, and plastic. When the calculation result is greater than a1 and less than or equal to a2, the processing device 204 determines that the type of the detected object is a tree; when the settlement result is greater than a2 and is less than or equal to a3, the processing device 13 determines that the type of the detected object is Pedestrians; when the settlement result is greater than a3 and less than or equal to a4, the processing device 13 determines that the type of the detected object is glass; when the settlement result is greater than a4 and less than a5, the processing device 13 determines that the type of the detected object is Metal; when the settlement result is greater than a5 and less than or equal to a6, the processing device 204 determines that the type of the detected object is plastic.

In a possible embodiment, after the processing device 204 obtains the vibration spectrum of the detected object, the processing device 204 extracts a frequency intensity distribution corresponding to a feature vector characterizing the detected object from the vibration spectrum. The vector includes the material and internal structure of the detected object. The processing device 204 inputs the extracted frequency intensity distribution (that is, a part of the vibration spectrum) into the object recognition model. By excluding the frequency spectrum of the detected object that cannot be characterized in the vibration frequency spectrum, compression of the vibration frequency spectrum is achieved, that is, data compression, and the calculation amount of the neural network operation performed by the processing device 204 is reduced.

In a possible embodiment, before the processing device 204 inputs the information of the vibration spectrum, the ultrasonic spectrum, the distance d, and the measurement environment into the object recognition model, the processing device 204 obtains multiple sets of training data, and the multiple sets Each set of training data in the training data corresponds to a type of object. For any group of training data i in the plurality of sets of training data, the corresponding object type is object O, and the training data i includes the vibration spectrum, ultrasonic spectrum of the object O, between the obstacle avoidance device 20 and the object O. Distance and measurement environment information. The processing device 204 performs a neural network operation on the plurality of sets of training data to obtain the object recognition model.

Further, after the processing device 204 performs a neural network operation on the plurality of sets of training data to obtain the object recognition model, the plurality of sets of training data is input into the object recognition model to obtain a plurality of sets of calculation results. Each group of calculation results corresponds to an object category; each group of calculations includes at least two calculation results; according to the above-mentioned multiple groups of calculation results, a correspondence table between the calculation results and the object categories is obtained, and the correspondence between the calculation results and the object categories The table includes the calculation result range and the corresponding object category, and the upper and lower limits of the calculation result range are the maximum and minimum values of a set of calculation results corresponding to the object category, respectively.

In a possible embodiment, before the processing device 204 inputs the information of the vibration spectrum, the ultrasonic spectrum, the distance d, and the measurement environment into the object recognition model, the processing device 204 further includes a communication module, and the processing device 204 A request message is sent to the third-party server through the communication module, and the request message is used to request to obtain the object recognition model and the correspondence table between the calculation result and the object category. The communication module of the processing device 204 receives a response message sent by the third-party server for responding to the request message, and the response message carries the object recognition model and a correspondence table between the calculation result and the object type.

In a possible embodiment, the processing device 204 is further configured to retrain the object recognition model and update the correspondence table between the calculation result and the object category to ensure the accuracy of the object recognition model, as follows:

Each time the object category recognition is performed N times using the above object recognition model and the correspondence table between the calculation result and the object type, multiple sets of training data are acquired again, where N is an integer greater than 1;

Re-training the object recognition model according to the reacquired sets of training data to obtain the re-trained object recognition model;

Input the re-obtained multiple sets of training data to the trained object recognition model for calculation to obtain multiple sets of calculation results, and each of the multiple sets of calculation results corresponds to an object category;

According to the multiple sets of calculation results, a correspondence table between the calculation results and the object type is obtained again.

It should be noted that the above-mentioned retraining of the object recognition model and the update of the correspondence table between the calculation result and the object category may be performed by the third-party server. Each time the processing device 204 uses the object recognition model and the calculation result and the object category, After performing the object recognition N times in the correspondence relationship table, a request message is re-sent to the third-party server for requesting to obtain a correspondence table between the retrained object recognition model and the updated calculation result and the object category.

After the processing device 204 determines the type of the detected object 10, the processing device 204 transmits the distance d and the type of the detected object to the reminder device 205, and the reminder device 205 sends out a voice message to inform the user in advance Object information, including the type of object and the distance between the object and the user.

In a possible embodiment, after the processing device obtains the vibration speckle image of the detected object 10, the processing device sends the vibration speckle image to the third-party device, and the third-party device determines the subject according to the vibration speckle image. For the type of the detected object, refer to the related description of the processing device 204 above. The three-party device sends the type of the detected object to the processing device 204.

Optionally, the third-party device may be a smart phone, a smart watch, a smart bracelet, a notebook computer, a desktop computer, or other devices.

It should be noted that the obstacle avoidance device 20 can obtain relevant information of the user's surrounding environment according to the above-mentioned related description, including relative position information of surrounding pedestrians, trees, buildings, and vehicles, so that the user can avoid the obstacles more flexibly. Above obstacles.

In another specific application scenario, the obstacle avoidance device 20 includes a rotation structure, and the rotation mechanism can realize a 360-degree rotation of the obstacle avoidance device, thereby achieving classification and recognition of objects in the entire scene, which is suitable for panoramic security monitor.

Specifically, the rotation structure is fixedly connected to the ultrasonic sensor 201 and the coherent light sensor 202 in the obstacle avoidance device 20 to achieve synchronous rotation of the ultrasonic sensor 201 and the coherent light sensor 202, and the rotation angle range is 0-360. Degrees to achieve surveillance within a panoramic range.

Further, the obstacle avoidance device 20 is used in combination with a security system, for example, when the obstacle avoidance device 20 detects a pedestrian in a preset detection area, such as 23:00 to 5:00, the security system reports to the security The personnel sends alarm information to inform the security personnel that there is an abnormality in the preset detection area, and the alarm information carries the position information of the preset detection area. The security personnel can further check the preset detection area according to the position information of the preset detection area.

In another specific application scenario, the obstacle avoidance device 20 may be applied to a car navigation system. The obstacle avoidance device may obtain the surrounding objects (including pedestrians, trees, Buildings, vehicles, etc.) location information, the above-mentioned obstacle avoidance device 20 can perform real-time planning of the route according to the destination information of the user and the location information of the surrounding objects of the current location, and the road condition information between the current location and the destination This enables users to reach their destinations quickly and safely, solves the problem that navigation through visual imaging cannot be applied in the case of low light, and avoids contact with transparent glass objects.

It can be seen that in the solution of the embodiment of the present application, the distance d between the detected object and the obstacle avoidance device is obtained by ultrasonic waves; n speckle images of the vibration of the detected object under ultrasonic stimulation are acquired based on coherent light; n vibration speckle images to obtain the vibration waveform signal of the detected object; and determine the type of the detected object based on the vibration waveform signal; remind the user of the distance d between the detected object and the type of the detected object. The embodiments of the present application have the following advantages: 1. Active ultrasonic waves and coherent light are irradiated onto the surface of the object in the same direction at the same time, so that different objects will be actively affected by the ultrasonic wave and generate vibration information with different frequency and different amplitude distributions, and these vibrations The frequency spectrum of the signal is capable of classifying and identifying objects with information about the objects themselves. 2. The non-imaging detection method is adopted. The structure and material information of the object is reflected in the speckle image. Compared with the traditional optical imaging detection method, there is no need to design very complicated lighting and imaging optics, which is especially suitable for scenes in dark environments Recognition, as well as the use of transparent or highly reflective scenes where imaging is difficult. 3.Using artificial neural network-based training and recognition algorithms for sound signals excited by sound waves, through deep learning neural networks, collect a variety of different object types and object-free occlusion spaces under a variety of active ultrasonic signal stress The vibration spectrum can be directly trained and identified without the need for complex image analysis algorithms and various detection parameters set by the user. The algorithm is simple and efficient, and is suitable for various industries such as automatic navigation, unmanned driving, and security monitoring.

In short, the embodiments of the present application can comprehensively detect obstacles in the surrounding environment, and improve the accuracy of blind navigation. This type of scene object recognition is required in many industries, such as scene reconstruction recognition in autonomous driving or assisted driving, scene monitoring in the security field, and equipment operating status monitoring in industrial production. This method solves applications such as no lighting or poor lighting, transparent glass detection, etc.

Referring to FIG. 7, FIG. 7 is a schematic flowchart of a method for avoiding obstacles based on coherent light according to an embodiment of the present application. As shown in Figure 7,

S701. The obstacle avoidance device obtains a distance d between the detected object and the obstacle avoidance device through ultrasonic waves.

S702. The obstacle avoidance device acquires n speckle images of the vibration of the detected object under the ultrasound stimulation based on coherent light; the n is an integer greater than 1.

S703. The obstacle avoidance device acquires a vibration waveform signal of the detected object according to the n speckle images of the vibration; and determines a category of the detected object according to the vibration waveform signal.

In a possible embodiment, the method further includes:

Acquire the frequency spectrum of the ultrasound.

S704. The obstacle avoidance device reminds the user of the distance d from the detected object and the type of the detected object.

It should be noted that, for the detailed description of the above steps S701-S704, reference may be made to the related description of the embodiment shown in FIG. 1 to FIG. 6 above, which will not be described here.

An embodiment of the present application further provides a computer storage medium, where the computer storage medium may store a program, and when the program is executed, includes part or all steps of any one of the obstacle avoidance methods described in the foregoing method embodiments.

The embodiments of the present application have been described in detail above. Specific examples have been used in this document to explain the principles and implementation of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. Persons of ordinary skill in the art may change the specific implementation and application scope according to the idea of the present application. In summary, the content of this description should not be construed as a limitation on the present application.

Claims

An obstacle avoidance device based on coherent light, comprising:

An ultrasonic sensor, a coherent light sensor, a high-speed camera connected to the coherent light sensor, and a processing device connected to both the ultrasonic sensor and the high-speed camera;

The ultrasonic sensor is configured to obtain a distance d between the detected object and the obstacle avoidance device, and transmit the distance d to the processing device;

The coherent light sensor is configured to emit coherent light to the detected object, receive reflected coherent light, and transmit the reflected coherent light to the high-speed camera;

The high-speed camera is configured to obtain n vibrational speckle images based on the reflected coherent light, and the vibrational speckle images are speckle images of the detected object generating vibrations under the stimulation of the ultrasonic wave. ; N is an integer greater than 1;

The processing device is configured to obtain a vibration waveform signal of the detected object according to the n speckle images of the vibration; and determine a category of the detected object according to the vibration waveform signal.
The device according to claim 1, wherein the processing device obtains a vibration waveform signal of the detected object according to the n speckle images of vibrations, comprising:

The processing device acquires M speckle comparison maps according to the n speckle images of vibration; the M is an integer greater than 1 and less than or equal to the n;

The processing device performs a clustering operation on the M spotted images according to a K-means clustering algorithm to obtain k clustered images, where k is an integer greater than 1 and less than M;

The processing device acquires a vibration waveform signal of the detected object according to the k cluster images.
The device according to claim 1 or 2, wherein the processing device determines the type of the detected object according to the vibration waveform signal, comprising:

The processing device performs a fast Fourier transform on the vibration waveform signal to obtain a vibration spectrum of the detected object;

The processing device inputs the vibration frequency spectrum, the distance d, the frequency spectrum of the ultrasonic wave, and measurement environment information into an object recognition model and performs a neural network operation to obtain a calculation result;

Obtaining an object category corresponding to the calculation result from a correspondence table between the calculation result and the object category to determine the category of the detected object.
The device according to claim 3, wherein the obstacle avoidance device further comprises: an environmental information detection module and a reminder device connected to the processing device;

The environmental information detection module is configured to detect and obtain information of the measurement environment, and the information of the measurement environment includes a temperature value, a wind speed value, and a humidity value;

The reminding device is used to remind a user of the distance d between the detection object and the type of the detection object.
The device according to claim 3, wherein before the processing device determines a category of the detected object according to the vibration waveform signal, the processing device is further configured to:

Acquiring multiple sets of training parameters, each set of training data of the plurality of sets of training data corresponding to an object category;

Performing a neural network training according to the plurality of sets of training parameters to obtain the object recognition model;

Inputting the plurality of sets of training parameters to the object recognition model for calculation respectively to obtain a plurality of sets of calculation results, and each set of calculation results in the plurality of sets of calculation results corresponds to an object category;

According to the plurality of sets of calculation results, a correspondence table between the calculation results and the object category is obtained, and the correspondence table between the calculation results and the object category includes a calculation result range and a corresponding object category, and an upper limit of the calculation result range and The lower limit is the maximum and minimum of a set of calculation results corresponding to the object category.
A method for avoiding obstacles based on coherent light includes:

Acquiring the distance d between the detected object and the obstacle avoidance device through ultrasonic waves;

Acquiring speckle images of n vibrations of the detected object under the ultrasound stimulation based on coherent light; the n is an integer greater than 1;

Acquiring a vibration waveform signal of the detected object according to the n vibration speckle images; and determining a category of the detected object according to the vibration waveform signal;

The user is reminded of the distance d from the detected object and the type of the detected object.
The method according to claim 6, wherein the obtaining a vibration waveform signal of the detected object according to the n speckle images of vibrations comprises:

Obtaining M speckle comparison maps according to the n vibrational speckle images; the M is an integer greater than 1 and less than or equal to the n;

From the M speckle contrast images, arbitrarily select k speckle contrast images as k initial cluster centers, where k is an integer smaller than the M;

For any speckle contrast image p in the Mk speckle contrast images, calculate a distance value from each of the initial cluster centers of the k initial cluster centers to obtain k distance values; where the Mk sheets The speckle contrast image is a speckle contrast image among the M speckle contrast images except for the k speckle contrast images that serve as the initial cluster center;

The initial clustering center corresponding to the smallest distance value among the k distance values is selected as the cluster described in the speckle contrast image p; according to this method, k clustered images are obtained, where k is greater than 1 and An integer less than said M;

According to the k cluster images, a vibration waveform signal of the detected object is obtained.
The method according to claim 6 or 7, wherein determining the category of the detected object according to the vibration waveform signal comprises:

Performing a fast Fourier transform on the vibration waveform signal to obtain a vibration spectrum of the detected object;

Input the vibration frequency spectrum, the distance d, the ultrasonic frequency spectrum and the measurement environment information into an object recognition model and perform a neural network operation to obtain a calculation result;

Obtaining an object category corresponding to the calculation result from a correspondence table between the calculation result and the object category to determine the category of the detected object.
The method according to claim 8, further comprising:

Detecting and acquiring information of the measurement environment, where the information of the measurement environment includes a temperature value, a wind speed value, and a humidity value;

Acquire the frequency spectrum of the ultrasound.
The method according to claim 8, wherein before the determining the type of the detected object according to the vibration waveform signal, the processing device is further configured to:

Acquiring multiple sets of training parameters, each set of training data of the plurality of sets of training data corresponding to an object category;

Performing a neural network training according to the plurality of sets of training parameters to obtain the object recognition model;

Inputting the plurality of sets of training parameters to the object recognition model for calculation respectively to obtain a plurality of sets of calculation results, and each set of calculation results in the plurality of sets of calculation results corresponds to an object category;

According to the plurality of sets of calculation results, a correspondence table between the calculation results and the object category is obtained, and the correspondence table between the calculation results and the object category includes a calculation result range and a corresponding object category, and an upper limit of the calculation result range and The lower limit is the maximum and minimum of a set of calculation results corresponding to the object category.