CN112307994A - Obstacle identification method based on sweeper, electronic device and storage medium - Google Patents
Obstacle identification method based on sweeper, electronic device and storage medium Download PDFInfo
- Publication number
- CN112307994A CN112307994A CN202011216808.5A CN202011216808A CN112307994A CN 112307994 A CN112307994 A CN 112307994A CN 202011216808 A CN202011216808 A CN 202011216808A CN 112307994 A CN112307994 A CN 112307994A
- Authority
- CN
- China
- Prior art keywords
- preset
- recognition result
- obstacle
- image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a data processing technology and provides an obstacle identification method based on a sweeper, an electronic device and a storage medium. The method comprises the steps of obtaining an original image collected by pre-configured image collection equipment, carrying out preprocessing operation on the original image to obtain a target image, extracting a feature vector of the target image based on a preset feature extraction rule, inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image, carrying out similarity recognition on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second recognition result of the target image, obtaining the target recognition result according to the first recognition result, the second recognition result and a preset judgment rule, and carrying out obstacle avoidance control on a sweeper based on the target recognition result. By using the invention, the sweeper can accurately identify whether the obstacle can cross, and the obstacle avoidance efficiency is improved.
Description
Technical Field
The invention relates to the field of data processing, in particular to an obstacle identification method based on a sweeper, an electronic device and a storage medium.
Background
With the continuous progress of science and technology, the technology of the intelligent sweeper is rapidly developed, and the detection and avoidance of obstacles are important embodiments of the intelligent level of the intelligent sweeper.
The good obstacle avoidance function is an important guarantee for the safe walking of the sweeper. At present, the intelligent sweeper can not intelligently judge the front terrain in the working process, so that the front terrain can not be avoided in a targeted manner, the working efficiency of the sweeper is influenced, and most of the existing methods are that a front baffle is additionally arranged on the front side of the sweeper to avoid the damage caused by collision on the front side of the sweeper.
Therefore, how to enable the sweeper to automatically and accurately judge whether the obstacle can cross has become a technical problem to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of the above, the present invention provides an obstacle recognition method based on a sweeper, an electronic device and a storage medium, and aims to solve the technical problem that in the prior art, the sweeper cannot determine whether the recognized obstacle can be crossed, so that the obstacle cannot be effectively avoided.
In order to achieve the above object, the present invention provides an obstacle identification method based on a sweeper, which comprises:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
Preferably, the extracting the feature vector of the target image based on the preset feature extraction rule includes:
and constructing a MobileNet V2 network, inputting the target image into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to the target image.
Preferably, the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the specific training process includes:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule;
generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion;
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing training when the accuracy is greater than a first preset threshold value to obtain the obstacle recognition model.
Preferably, the second identifying step includes:
and respectively calculating the similarity values of the feature vectors of the target image and the feature vectors of the reference obstacle images in the first preset database by using a cosine similarity algorithm, sequencing the calculated similarity values from large to small, and selecting the maximum value as the second identification result.
Preferably, the method further comprises:
and when the second recognition result is smaller than a second preset threshold value, performing similarity recognition on the feature vector of the target image and the feature vectors of the reference obstacle images in a second preset database, and taking the recognition result as the second recognition result.
Preferably, the obtaining of the target recognition result based on the first recognition result, the second recognition result and a preset determination rule includes:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
To achieve the above object, the present invention also provides an electronic device, including: the obstacle identification program based on the sweeper is executed by the processor, and the following steps are realized:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
Preferably, the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the specific training process includes:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule;
generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion;
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing training when the accuracy is greater than a first preset threshold value to obtain the obstacle recognition model.
Preferably, the obtaining of the target recognition result based on the first recognition result, the second recognition result and a preset determination rule includes:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
In order to achieve the above object, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes an obstacle identification program based on a sweeper, and when the obstacle identification program based on the sweeper is executed by a processor, any step of the obstacle identification method based on the sweeper is implemented.
The invention provides an obstacle recognition method based on a sweeper, an electronic device and a storage medium, which are characterized in that an original image acquired by a pre-configured image acquisition device is acquired, a target image is acquired by performing preprocessing operation on the original image, a characteristic vector of the target image is extracted based on a preset characteristic extraction rule, the characteristic vector of the target image is input into a pre-trained obstacle recognition model to acquire a first recognition result of the target image, similarity recognition is performed on the characteristic vector of the target image and the characteristic vector of each reference obstacle image in a first preset database to acquire a second recognition result of the target image, a target recognition result is acquired according to the first recognition result, the second recognition result and a preset judgment rule, obstacle avoidance control is performed on the sweeper based on the target recognition result, and the sweeper can accurately recognize whether obstacles can cross, and the obstacle avoidance efficiency is improved.
Drawings
FIG. 1 is a diagram of an electronic device according to a preferred embodiment of the present invention;
FIG. 2 is a block diagram of a preferred embodiment of the obstacle identification process based on the sweeper shown in FIG. 1;
FIG. 3 is a flowchart illustrating a method for recognizing obstacles based on a sweeper according to a preferred embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a schematic diagram of an electronic device 1 according to a preferred embodiment of the invention is shown.
The electronic device 1 includes but is not limited to: memory 11, processor 12, display 13, and network interface 14. The electronic apparatus 1 is connected to a network through a network interface 14. The network may be a wireless or wired network such as the Internet, a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, a communication network, and the like.
The memory 11 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 11 may be an internal storage unit of the electronic device 1, such as a hard disk or a memory of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic apparatus 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided in the electronic apparatus 1. Of course, the memory 11 may also comprise both an internal memory unit of the electronic apparatus 1 and an external memory device thereof. In this embodiment, the memory 11 is generally used for storing an operating system installed in the electronic device 1 and various application software, such as a program code of the obstacle identification program 10 based on a sweeper. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is generally used for controlling the overall operation of the electronic device 1, such as performing data interaction or communication related control and processing. In this embodiment, the processor 12 is configured to run the program code or the processing data stored in the memory 11, for example, the program code of the obstacle identification program 10 based on the sweeper.
The display 13 may be referred to as a display screen or display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch screen, or the like. The display 13 is used for displaying information processed in the electronic device 1 and for displaying a visual work interface, for example, results of data statistics.
The network interface 14 may optionally comprise a standard wired interface, a wireless interface (e.g. WI-FI interface), the network interface 14 typically being used for establishing a communication connection between the electronic apparatus 1 and other electronic devices.
Fig. 1 shows only the electronic device 1 with the components 11-14 and the sweeper-based obstacle identification program 10, but it will be understood that not all of the shown components are required and that more or fewer components may alternatively be implemented.
Optionally, the electronic device 1 may further comprise a user interface, the user interface may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic apparatus 1 and for displaying a visualized user interface.
The electronic device 1 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, and the like, which are not described in detail herein.
In the above embodiment, the processor 12, when executing the sweeper-based obstacle recognition program 10 stored in the memory 11, may implement the following steps:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
The storage device may be the memory 11 of the electronic apparatus 1, or may be another storage device communicatively connected to the electronic apparatus 1.
For the detailed description of the above steps, please refer to the following description of fig. 2 regarding a block diagram of an embodiment of the obstacle identification procedure 10 based on the sweeper and fig. 3 regarding a flowchart of an embodiment of the obstacle identification method based on the sweeper.
In other embodiments, the sweeper-based obstacle identification program 10 may be divided into a plurality of modules that are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The modules referred to herein are referred to as a series of computer program instruction segments capable of performing specified functions.
Referring to fig. 2, a block diagram of an embodiment of the obstacle identification process 10 based on the sweeper shown in fig. 1 is shown. In this embodiment, the sweeper-based obstacle identification program 10 may be divided into: a receiving module 110, a first identifying module 120, a second identifying module 130 and a control module 140.
The receiving module 110 is configured to obtain an original image acquired by a preconfigured image acquisition device, and perform a preprocessing operation on the original image to obtain a target image.
In this embodiment, an original image collected by a pre-configured image collection device (e.g., a camera) is received, and the original image is an image directly collected and photographed in an actual application scene without any processing, so that the original image collected by the image collection device may be pre-processed to obtain a target image, so as to eliminate interference factors such as noise existing in the original image.
In one embodiment, the preprocessing mode may include performing image enhancement processing on the original image, enhancing useful information in the image for the purpose of improving the visual effect of the image, purposefully emphasizing the overall or local characteristics of the image for a given image application, making the original unclear image clear or emphasizing some interesting features, expanding the difference between different object features in the image, suppressing the uninteresting features, improving the image quality, enriching the information content, and enhancing the image interpretation and recognition effect, wherein the image enhancement algorithm may utilize a spatial domain method and a frequency domain method.
The first recognition module 120 is configured to extract a feature vector of the target image based on a preset feature extraction rule, and input the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image.
In this embodiment, in order to recognize the target image input after the original image preprocessing, it is necessary to extract the feature vector of the target image, and input the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image. The feature vector of the target image can be extracted by adopting a preset feature extraction rule, further, a MobileNetV2 network is constructed, the target image is input into the MobileNetV2 network, and the output feature vector of the MobileNetV2 network is used as the feature vector corresponding to the target image.
The mobilenetV2 is a lightweight convolutional neural network structure, the mobilenetV2 network can efficiently and quickly identify images with low resolution, and the calculation occupies a small memory and can be carried on a micro device for use. The mobilenetV2 network comprises 53 convolutional layers, 1 pooling layer and 1 full-connection layer which are sequentially connected, wherein the 53 convolutional layers comprise 1 input layer, 17 bottleneck building blocks and 1 output layer which are sequentially connected, each bottleneck building block comprises 3 convolutional layers respectively, and the convolutional cores of the 53 convolutional layers are all 3 x 3. Since the features of the image only need to be extracted by using MobileNetV2, the present embodiment uses the feature vector output after removing the convolutional layer used for classification by MobileNetV2 as the feature vector corresponding to the target image.
In one embodiment, the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the specific training process includes:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule; the acquired sample image comprises a first preset type of obstacle image (an obstacle which can pass through the sweeper) and a second preset type of obstacle image (an obstacle which cannot pass through the sweeper), and the first preset type of obstacle image can be marked with 1, and the second preset type of obstacle image is marked with 0.
Generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion; wherein, the preset proportion can be 4: 1.
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing the training when the accuracy is greater than a first preset threshold (for example, 90%), and obtaining the obstacle recognition model.
After the feature vector of the target image is input into a pre-trained obstacle recognition model, a first recognition result of the target image can be obtained, and the first recognition result is the confidence coefficient that the target image belongs to a first preset type of obstacle image.
The second identifying module 130 is configured to perform similarity identification on the feature vector of the target image and the feature vectors of the reference obstacle images in the first preset database to obtain a second identifying result of the target image.
In this embodiment, in order to improve the identification accuracy of the target image, after the target image is identified by the pre-trained obstacle identification model, the similarity between the reference obstacle image stored in the first preset database and the target image may be identified to obtain a second identification result of the target image, where the first preset database is a local database, the local database may store a plurality of first preset types of obstacle images in advance, and the identification accuracy of the target image may be improved by comparing the plurality of first preset types of obstacle images in the local database with the target image.
In one embodiment, the similarity values of the feature vectors of the target image and the feature vectors of the reference obstacle images in the first preset database are respectively calculated by using a cosine similarity algorithm, the calculated similarity values are sorted from large to small, and the maximum value is selected as the second identification result.
Further, when the second recognition result is smaller than a second preset threshold, performing similarity recognition on the feature vector of the target image and the feature vectors of the reference obstacle images in a second preset database, and taking the recognition result as the second recognition result.
When a second recognition result obtained by comparing a plurality of first preset types of obstacle images in the local database with the target image is smaller than a second preset threshold (for example, 30%), it is indicated that there may be no image similar to the target image in the local database, and therefore, the feature vector of the target image and the feature vector of each reference obstacle image in a second preset database (for example, a database in communication connection with a cloud server) may be subjected to similarity recognition, and the recognition result is used as the second recognition result, where the second preset database stores more obstacle images of the first preset types.
And the control module 140 is configured to obtain a target identification result based on the first identification result, the second identification result and a preset determination rule, and perform obstacle avoidance control on the sweeper based on the target identification result.
In this embodiment, a target recognition result is obtained based on the first recognition result, the second recognition result and a preset determination rule, obstacle avoidance control is performed on the sweeper based on the target recognition result, the target recognition result is obtained by associating the first recognition result and the second recognition result, the recognition accuracy of a target image can be improved, and when the sweeper performs sweeping, the condition of a front terrain can be determined, so that obstacles which are difficult to cross are intelligently avoided.
In one embodiment, the obtaining a target recognition result based on the first recognition result, the second recognition result and a preset determination rule includes:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
In addition, the invention also provides an obstacle identification method based on the sweeper. Fig. 3 is a schematic method flow diagram of an embodiment of the obstacle identification method based on the sweeper according to the present invention. When the processor 12 of the electronic device 1 executes the obstacle recognition program 10 based on the sweeper stored in the memory 11, the following steps of the obstacle recognition method based on the sweeper are realized:
step S10: the method comprises the steps of obtaining an original image collected by a pre-configured image collecting device, and executing preprocessing operation on the original image to obtain a target image.
In this embodiment, an original image collected by a pre-configured image collection device (e.g., a camera) is received, and the original image is an image directly collected and photographed in an actual application scene without any processing, so that the original image collected by the image collection device may be pre-processed to obtain a target image, so as to eliminate interference factors such as noise existing in the original image.
In one embodiment, the preprocessing mode may include performing image enhancement processing on the original image, enhancing useful information in the image for the purpose of improving the visual effect of the image, purposefully emphasizing the overall or local characteristics of the image for a given image application, making the original unclear image clear or emphasizing some interesting features, expanding the difference between different object features in the image, suppressing the uninteresting features, improving the image quality, enriching the information content, and enhancing the image interpretation and recognition effect, wherein the image enhancement algorithm may utilize a spatial domain method and a frequency domain method.
Step S20: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image.
In this embodiment, in order to recognize the target image input after the original image preprocessing, it is necessary to extract the feature vector of the target image, and input the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image. The feature vector of the target image can be extracted by adopting a preset feature extraction rule, further, a MobileNetV2 network is constructed, the target image is input into the MobileNetV2 network, and the output feature vector of the MobileNetV2 network is used as the feature vector corresponding to the target image.
The mobilenetV2 is a lightweight convolutional neural network structure, the mobilenetV2 network can efficiently and quickly identify images with low resolution, and the calculation occupies a small memory and can be carried on a micro device for use. The mobilenetV2 network comprises 53 convolutional layers, 1 pooling layer and 1 full-connection layer which are sequentially connected, wherein the 53 convolutional layers comprise 1 input layer, 17 bottleneck building blocks and 1 output layer which are sequentially connected, each bottleneck building block comprises 3 convolutional layers respectively, and the convolutional cores of the 53 convolutional layers are all 3 x 3. Since the features of the image only need to be extracted by using MobileNetV2, the present embodiment uses the feature vector output after removing the convolutional layer used for classification by MobileNetV2 as the feature vector corresponding to the target image.
In one embodiment, the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the specific training process includes:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule; the acquired sample image comprises a first preset type of obstacle image (an obstacle which can pass through the sweeper) and a second preset type of obstacle image (an obstacle which cannot pass through the sweeper), and the first preset type of obstacle image can be marked with 1, and the second preset type of obstacle image is marked with 0.
Generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion; wherein, the preset proportion can be 4: 1.
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing the training when the accuracy is greater than a first preset threshold (for example, 90%), and obtaining the obstacle recognition model.
After the feature vector of the target image is input into a pre-trained obstacle recognition model, a first recognition result of the target image can be obtained, and the first recognition result is the confidence coefficient that the target image belongs to a first preset type of obstacle image.
Step S30: and carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image.
In this embodiment, in order to improve the identification accuracy of the target image, after the target image is identified by the pre-trained obstacle identification model, the similarity between the reference obstacle image stored in the first preset database and the target image may be identified to obtain a second identification result of the target image, where the first preset database is a local database, the local database may store a plurality of first preset types of obstacle images in advance, and the identification accuracy of the target image may be improved by comparing the plurality of first preset types of obstacle images in the local database with the target image.
In one embodiment, the similarity values of the feature vectors of the target image and the feature vectors of the reference obstacle images in the first preset database are respectively calculated by using a cosine similarity algorithm, the calculated similarity values are sorted from large to small, and the maximum value is selected as the second identification result.
Further, when the second recognition result is smaller than a second preset threshold, performing similarity recognition on the feature vector of the target image and the feature vectors of the reference obstacle images in a second preset database, and taking the recognition result as the second recognition result.
When a second recognition result obtained by comparing a plurality of first preset types of obstacle images in the local database with the target image is smaller than a second preset threshold (for example, 30%), it is indicated that there may be no image similar to the target image in the local database, and therefore, the feature vector of the target image and the feature vector of each reference obstacle image in a second preset database (for example, a database in communication connection with a cloud server) may be subjected to similarity recognition, and the recognition result is used as the second recognition result, where the second preset database stores more obstacle images of the first preset types.
Step S40: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
In this embodiment, a target recognition result is obtained based on the first recognition result, the second recognition result and a preset determination rule, obstacle avoidance control is performed on the sweeper based on the target recognition result, the target recognition result is obtained by associating the first recognition result and the second recognition result, the recognition accuracy of a target image can be improved, and when the sweeper performs sweeping, the condition of a front terrain can be determined, so that obstacles which are difficult to cross are intelligently avoided.
In one embodiment, the obtaining a target recognition result based on the first recognition result, the second recognition result and a preset determination rule includes:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
Furthermore, the embodiment of the present invention also provides a computer-readable storage medium, which may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, and the like. The computer-readable storage medium includes a sweeper-based obstacle identification program 10, and when executed by a processor, the sweeper-based obstacle identification program 10 implements the following operations:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the obstacle identification method based on the sweeper, and will not be described herein again.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention essentially or contributing to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (such as a mobile phone, a computer, an electronic device, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The obstacle identification method based on the sweeper is applied to an electronic device, and is characterized by comprising the following steps:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
2. The obstacle recognition method based on a sweeper of claim 1, wherein the extracting the feature vector of the target image based on the preset feature extraction rule comprises:
and constructing a MobileNet V2 network, inputting the target image into the MobileNet V2 network, and taking the output characteristic vector of the MobileNet V2 network as the characteristic vector corresponding to the target image.
3. The obstacle recognition method based on the sweeper of claim 1, wherein the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the specific training process comprises:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule;
generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion;
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing training when the accuracy is greater than a first preset threshold value to obtain the obstacle recognition model.
4. The sweeper-based obstacle identification method of claim 1, wherein the second identification step comprises:
and respectively calculating the similarity values of the feature vectors of the target image and the feature vectors of the reference obstacle images in the first preset database by using a cosine similarity algorithm, sequencing the calculated similarity values from large to small, and selecting the maximum value as the second identification result.
5. The sweeper-based obstacle identification method of claim 4, further comprising:
and when the second recognition result is smaller than a second preset threshold value, performing similarity recognition on the feature vector of the target image and the feature vectors of the reference obstacle images in a second preset database, and taking the recognition result as the second recognition result.
6. The obstacle recognition method based on the sweeper according to any one of claims 1 to 5, wherein the obtaining of the target recognition result based on the first recognition result, the second recognition result and a preset judgment rule comprises:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
7. An electronic device, comprising: the obstacle identification program based on the sweeper is executed by the processor, and the following steps are realized:
a receiving step: acquiring an original image acquired by a pre-configured image acquisition device, and performing preprocessing operation on the original image to obtain a target image;
a first identification step: extracting a feature vector of the target image based on a preset feature extraction rule, and inputting the feature vector of the target image into a pre-trained obstacle recognition model to obtain a first recognition result of the target image;
a second identification step: carrying out similarity identification on the feature vector of the target image and the feature vector of each reference obstacle image in a first preset database to obtain a second identification result of the target image;
the control steps are as follows: and obtaining a target identification result based on the first identification result, the second identification result and a preset judgment rule, and executing obstacle avoidance control on the sweeper based on the target identification result.
8. The electronic device of claim 7, wherein the pre-trained obstacle recognition model is obtained by training a convolutional neural network model, and the training process includes:
acquiring a preset number of sample images, labeling a preset label on each sample image, and extracting a feature vector of each sample image by using a preset feature extraction rule;
generating a sample set by taking the characteristic vector of each sample image as a variable and taking a preset label of each sample image as a dependent variable;
dividing the sample set into a training set and a verification set according to a preset proportion;
training the convolutional neural network model by using each variable and each dependent variable in the training set, verifying the convolutional neural network model by using the verification set every other preset period, and verifying the accuracy of the convolutional neural network model by using each variable and each dependent variable in the verification set; and
and finishing training when the accuracy is greater than a first preset threshold value to obtain the obstacle recognition model.
9. The electronic device of claim 7, wherein obtaining a target recognition result based on the first recognition result, the second recognition result and a preset determination rule comprises:
judging whether the confidence of the first recognition result is greater than a third preset threshold, judging whether the second recognition result is greater than a fourth preset threshold when the confidence of the first recognition result is greater than the third preset threshold, and if the second recognition result is greater than the fourth preset threshold, judging that the target recognition result is a first preset type of obstacle;
when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is larger than a fifth preset threshold, the target recognition result is a first preset type of obstacle;
and when the confidence of the first recognition result is smaller than or equal to a third preset threshold and the second recognition result is smaller than or equal to a fourth preset threshold, the target recognition result is an obstacle of a second preset type.
10. A computer-readable storage medium, wherein the computer-readable storage medium includes a sweeper-based obstacle identification program, and when the sweeper-based obstacle identification program is executed by a processor, the steps of the sweeper-based obstacle identification method according to any one of claims 1 to 6 are realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216808.5A CN112307994A (en) | 2020-11-04 | 2020-11-04 | Obstacle identification method based on sweeper, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216808.5A CN112307994A (en) | 2020-11-04 | 2020-11-04 | Obstacle identification method based on sweeper, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112307994A true CN112307994A (en) | 2021-02-02 |
Family
ID=74325628
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011216808.5A Pending CN112307994A (en) | 2020-11-04 | 2020-11-04 | Obstacle identification method based on sweeper, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112307994A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528972A (en) * | 2021-02-08 | 2021-03-19 | 常州微亿智造科技有限公司 | Positioning method and device for flying shooting point |
| CN113469000A (en) * | 2021-06-23 | 2021-10-01 | 追觅创新科技(苏州)有限公司 | Regional map processing method and device, storage medium and electronic device |
| CN114090568A (en) * | 2022-01-24 | 2022-02-25 | 深圳市慧为智能科技股份有限公司 | Dirty data clearing method and device, terminal equipment and readable storage medium |
-
2020
- 2020-11-04 CN CN202011216808.5A patent/CN112307994A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112528972A (en) * | 2021-02-08 | 2021-03-19 | 常州微亿智造科技有限公司 | Positioning method and device for flying shooting point |
| CN112528972B (en) * | 2021-02-08 | 2021-06-04 | 常州微亿智造科技有限公司 | Positioning method and device for flying shooting point |
| CN113469000A (en) * | 2021-06-23 | 2021-10-01 | 追觅创新科技(苏州)有限公司 | Regional map processing method and device, storage medium and electronic device |
| CN114090568A (en) * | 2022-01-24 | 2022-02-25 | 深圳市慧为智能科技股份有限公司 | Dirty data clearing method and device, terminal equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110738101B (en) | Behavior recognition method, behavior recognition device and computer-readable storage medium | |
| CN109325964B (en) | Face tracking method and device and terminal | |
| CN112307994A (en) | Obstacle identification method based on sweeper, electronic device and storage medium | |
| US10534957B2 (en) | Eyeball movement analysis method and device, and storage medium | |
| JP6719457B2 (en) | Method and system for extracting main subject of image | |
| CN110033018B (en) | Graph similarity judging method and device and computer readable storage medium | |
| CN108009466B (en) | Pedestrian detection method and device | |
| CN110689535B (en) | Workpiece identification method and device, electronic equipment and storage medium | |
| CN111639653B (en) | False detection image determining method, device, equipment and medium | |
| CN106484837A (en) | The detection method of similar video file and device | |
| US9633272B2 (en) | Real time object scanning using a mobile phone and cloud-based visual search engine | |
| CN113255557B (en) | Deep learning-based video crowd emotion analysis method and system | |
| CN102306287A (en) | Method and equipment for identifying sensitive image | |
| CN111160169A (en) | Face detection method, device, equipment and computer readable storage medium | |
| CN104951440B (en) | Image processing method and electronic equipment | |
| CN111263955A (en) | Method and device for determining movement track of target object | |
| CN111126254A (en) | Image recognition method, device, equipment and storage medium | |
| CN112926601A (en) | Image recognition method, device and equipment based on deep learning and storage medium | |
| CN116311214B (en) | License plate recognition method and device | |
| CN111488798B (en) | Fingerprint identification method, fingerprint identification device, electronic equipment and storage medium | |
| CN111553241B (en) | Palm print mismatching point eliminating method, device, equipment and storage medium | |
| CN111814776A (en) | Image processing method, device, server and storage medium | |
| CN113221601A (en) | Character recognition method, device and computer readable storage medium | |
| CN114168768A (en) | Image retrieval method and related equipment | |
| CN108109164B (en) | Information processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |