CN115877847A - Obstacle avoidance method and device based on machine vision - Google Patents
Obstacle avoidance method and device based on machine vision Download PDFInfo
- Publication number
- CN115877847A CN115877847A CN202211634929.0A CN202211634929A CN115877847A CN 115877847 A CN115877847 A CN 115877847A CN 202211634929 A CN202211634929 A CN 202211634929A CN 115877847 A CN115877847 A CN 115877847A
- Authority
- CN
- China
- Prior art keywords
- obstacle
- obstacle avoidance
- destination
- information
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Traffic Control Systems (AREA)
Abstract
The application provides an obstacle avoidance method and device based on machine vision, and the method comprises the following steps: responding to the destination instruction, acquiring destination information, and planning a traveling path from the current place to the destination; advancing according to the planned advancing path, and acquiring an environment image in front of an advancing body in real time in the advancing process; identifying an environment image through a pre-constructed obstacle identification model to acquire obstacle information; and replanning the current travel path to the destination according to the obstacle information. The obstacle recognition accuracy on the advancing path of the advancing body is improved, the obstacle avoiding path is planned in advance, and the advancing efficiency is improved.
Description
Technical Field
The application relates to the technical field of mobile communication, in particular to an obstacle avoidance method and device based on machine vision.
Background
The traveling body of the unmanned automobile, the robot, the blind and the like makes a traveling decision depending on the obstacle recognition result to ensure that the unmanned automobile, the robot, the blind and the like avoid the obstacle and do not rub or collide with the obstacle during normal traveling. The robots comprise a sweeping robot, an inspection robot, a blind guiding robot and the like. The blind guiding robot is an auxiliary tool for providing environment guidance for the visually impaired, belongs to the field of service robots, detects the surrounding environment through various sensors, feeds detected information back to the visually impaired, and helps the visually impaired to make up for the lack of visual information.
The presence of unknown obstacles in the environment poses a significant threat to the body of travel (e.g., driverless cars, robots, or the blind population). At present, the obstacle is identified and positioned through the object identification and positioning device, so that the obstacle existing in the advancing body is reminded, however, the identification accuracy of the obstacle is low at present, the obstacle avoiding path is not planned in advance, other obstacles can be touched in the obstacle avoiding process, and the advancing efficiency is low.
Therefore, the technical problems which are continuously solved at present are as follows: how to improve the obstacle identification accuracy on the advancing path of the advancing body, plan the obstacle avoidance path in advance and improve the advancing efficiency.
Disclosure of Invention
The application aims to provide an obstacle avoidance method and device based on machine vision, which can be used for improving the obstacle identification accuracy of a travelling path of a travelling body, planning an obstacle avoidance path in advance and improving the travelling efficiency.
In order to achieve the above object, the present application provides an obstacle avoidance method based on machine vision, which includes the following steps: responding to the destination instruction, acquiring destination information, and planning a traveling path from the current place to the destination; advancing according to the planned advancing path, and acquiring an environment image in front of an advancing body in real time in the advancing process; identifying an environment image through a pre-constructed obstacle identification model to obtain obstacle information; and replanning the current travel path to the destination according to the obstacle information.
The obstacle avoidance method based on the machine vision, wherein the method for constructing the obstacle identification model in advance comprises the following steps: acquiring a training data set; inputting the training data set into a convolutional neural network model for training to obtain an obstacle identification model; and carrying out optimization verification on the obstacle identification model.
The obstacle avoidance method based on the machine vision, wherein the method for performing optimization verification on the obstacle identification model comprises the following steps: acquiring images of a plurality of known obstacles as a verification set; inputting the images in the verification set into the obstacle identification model for verification, and obtaining the identification result of the obstacle identification model; calculating the recognition accuracy of the obstacle recognition model according to the recognition result of the obstacle recognition model; and comparing the calculated recognition accuracy with a preset threshold, and if the calculated recognition accuracy is smaller than the preset threshold, repeatedly optimizing and training the obstacle recognition model until the recognition accuracy of the obstacle recognition model is larger than the preset threshold.
The obstacle avoidance method based on machine vision as described above, wherein the method for replanning the current travel path to the destination according to the obstacle information includes: acquiring the position information of a next path turning node closest to the obstacle in the original traveling path according to the obstacle information; and inserting an obstacle avoidance node between the current point and the next path turning node closest to the obstacle.
The obstacle avoidance method based on the machine vision, wherein the method for inserting the obstacle avoidance node between the current point and the next path turning node closest to the obstacle comprises the following steps: establishing an obstacle avoidance node optimization model; optimizing the positions of the obstacle avoidance nodes according to the obstacle avoidance node optimization model; and inserting the obstacle avoidance nodes after the optimized positions on the basis of the original traveling path so as to replan the traveling path from the current place to the destination.
The obstacle avoidance method based on machine vision as described above, wherein the method for obtaining the destination information in response to the destination instruction and planning the traveling path to the destination currently includes: responding to the destination instruction, and acquiring destination information; returning the obtained destination information to the traveling body for confirmation, and judging whether the destination information is correct or not; if the destination information is correct, planning a traveling path from the current place to the destination according to the destination information after the correctness is confirmed, and otherwise, acquiring the destination information again.
This application still provides an obstacle avoidance device based on machine vision, installs on advancing the body, should keep away the obstacle device and include: the route planning module is used for responding to the destination instruction, acquiring destination information and planning a traveling route from the current place to the destination; the advancing body advances according to the planned advancing path; the image acquisition device is used for acquiring an environment image in front of the advancing body in real time in the advancing process; the obstacle information acquisition module is used for identifying the environment image through a pre-constructed obstacle identification model to acquire obstacle information; and the path correction module is used for replanning the current travel path to the destination according to the obstacle information.
The obstacle avoidance device based on machine vision further comprises an operation monitoring module, wherein the operation monitoring module is used for monitoring the operation state of the obstacle avoidance device.
The obstacle avoidance device based on machine vision as described above, wherein the path correction module includes: the acquisition module is used for acquiring the position information of the turning node of the next path closest to the obstacle in the original traveling path according to the obstacle information; and the obstacle avoidance node inserting module is used for inserting an obstacle avoidance node between the current point and the next path turning node closest to the obstacle.
The obstacle avoidance device based on machine vision further comprises an alarm for giving an alarm when the obstacle information is acquired.
The beneficial effect that this application realized is as follows:
(1) This application trains barrier recognition model in advance, calculates the recognition accuracy of barrier recognition model, obtains the barrier recognition model that recognition accuracy accords with preset threshold to the barrier to the current route of travel orientation is discerned, improves the recognition accuracy of the preceding barrier of the body of marcing.
(2) According to the method and the device, the obstacle avoidance node is inserted between the current point and the next path turning point, the obstacle avoidance path is planned in advance, and the advancing efficiency of the advancing body is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1 is a flowchart of an obstacle avoidance method based on machine vision according to an embodiment of the present application.
Fig. 2 is a flowchart of a method for planning a travel path from a current location to a destination according to an embodiment of the present disclosure.
Fig. 3 is a flowchart of a method for acquiring an environmental image according to an embodiment of the present disclosure.
Fig. 4 is a flowchart of a method for constructing an obstacle identification model according to an embodiment of the present application.
Fig. 5 is a flowchart of a method for replanning a current travel path to a destination according to obstacle information according to an embodiment of the present application.
Fig. 6 is a flowchart of a method for inserting an obstacle avoidance node according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of an obstacle avoidance device based on machine vision according to an embodiment of the present application.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Reference numerals: 10-a path planning module; 20-an image acquisition device; 30-obstacle information acquisition module; 40-a path correction module; 41-an acquisition module; 42-inserting an obstacle avoidance node into a module; 43-high precision map module; 50-a device operation monitoring module; 60-a data processor; 70-a data comparator; 80-alarm device; 100-obstacle avoidance device based on machine vision; 200-an electronic device; 201-a memory; 202-a processor.
Detailed Description
The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments obtained by a person skilled in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Example one
As shown in fig. 1, the present application provides an obstacle avoidance method based on machine vision, which includes the following steps:
and S1, responding to a destination instruction, acquiring destination information and planning a traveling path from the current place to the destination.
As shown in fig. 2, step S1 includes the following sub-steps:
step S110, in response to the destination instruction, acquires destination information.
As an embodiment of the present invention, the travel body issues a voice destination instruction to the obstacle avoidance device. The obstacle avoidance device acquires a destination instruction through a voice recognition function, and recognizes destination information in response to the destination instruction. The destination information includes a destination type, a destination name, and the like. The traveling body can be an inspection robot, a sweeping robot, a blind person, an unmanned automobile and the like.
As a specific embodiment of the present invention, a voice destination instruction is sent to an obstacle avoidance device through a remote control terminal, or a destination instruction speech is sent to the obstacle avoidance device, and the obstacle avoidance device recognizes destination information through voice recognition or semantic recognition. Specifically, the voice recognition is used for recognizing a voice destination instruction and obtaining destination information, and the semantic recognition is used for recognizing a destination instruction segment and obtaining destination information through a semantic recognition technology, wherein the destination information includes, for example: go to a restaurant, bedroom or some other location (bank, supermarket or station, etc.).
Step S120, returning the acquired destination information to the travel body for confirmation, and determining whether the destination information is correct.
Specifically, the destination information recognized by the obstacle avoidance device is fed back to the travel body in a voice form or a text (sentence) form. It can be understood that: the obstacle avoidance device sends the recognized destination information to the traveling body in a voice form or a text (sentence) form so that the traveling body can confirm the destination information recognized by the obstacle avoidance device. The travel body may have a voice recognition function for recognizing destination information in a voice form transmitted by the obstacle avoidance device, or may have a semantic recognition function for recognizing destination information in a text (sentence) form transmitted by the obstacle avoidance device by a semantic recognition technique.
As a specific embodiment of the present invention, the obstacle avoidance device broadcasts the destination information in the form of voice, after the travel body recognizes the broadcasted voice, it determines whether the broadcasted destination information is correct, if so, it returns the correct confirmation information to the obstacle avoidance device, otherwise, it returns the wrong confirmation information, and the obstacle avoidance device re-recognizes and feeds back the destination information.
As another specific embodiment of the present invention, the obstacle avoidance device feeds back the destination information in the form of characters to the traveling body, and the traveling body compares the destination information fed back by the obstacle avoidance device with the characters in the voice destination instruction sent by the obstacle avoidance device, and determines whether the destination information is consistent with the characters in the voice destination instruction, if so, the obstacle avoidance device returns the correct confirmation information, otherwise, the obstacle avoidance device returns the wrong confirmation information, and the obstacle avoidance device re-identifies and feeds back the destination information.
Step S130, if the destination information is correct, planning a traveling path from the current place to the destination according to the destination information after the correctness is confirmed, otherwise, acquiring the destination information again.
Specifically, the current position is located through a satellite navigation system, the current position information of the traveling body is obtained, the position information of the destination is obtained according to the destination information, and the traveling path from the current position to the destination is planned. The satellite navigation system is installed on keeping away the barrier device, keeps away the barrier device and installs on advancing the body.
And S2, advancing according to the planned advancing path, and acquiring an environment image in front of the advancing body in real time in the advancing process.
Specifically, the advancing body advances according to a planned advancing path, and an environment image in front of the advancing body is acquired in real time in the advancing process of the advancing body through the image acquisition device. Image acquisition devices such as: video camera.
As shown in fig. 3, step S2 includes the steps of:
and step S210, traveling according to the planned traveling path.
Specifically, the planning of the route of marcing is responded to, real-time voice broadcast is carried out on the planned route of marcing, and the body of marcing recognizes the content of the voice broadcast and then marches according to the content of the voice broadcast. The content of voice broadcast includes: travel 10 meters forward, 50 meters left, 10 meters right, etc.
Step S220, an environmental image in front of the traveling body is collected in real time during the traveling process.
Specifically, an environment image of the advancing direction of the advancing body is collected through a video camera. The environment image comprises images of the ground, objects on the ground and objects in a height range higher than the traveling body (such as a desktop or objects on the desktop).
And S3, identifying the environment image through a pre-constructed obstacle identification model to acquire obstacle information.
As a specific embodiment of the present invention, the information of the obstacle in the traveling process is acquired by the laser radar, and the obstacle avoidance reminder is sent out according to the information of the obstacle to modify the moving direction.
As shown in fig. 4, the method of constructing the obstacle recognition model in advance includes the steps of:
and step Y1, acquiring a training data set.
The method for acquiring the training data set comprises the following steps:
multiple images of common obstacles are acquired.
Common obstacles to the collection are, for example, tables, chairs, characters, ground stones, poles and wall extensions, etc., as embodiments of the invention.
And carrying out image enhancement processing on the acquired image.
And Y2, inputting the training data set into the convolutional neural network model for training to obtain an obstacle recognition model.
Specifically, the image of the common obstacle is input into a convolutional neural network model for training to obtain an obstacle recognition model, and the obstacle recognition model is used for recognizing different obstacles in the image to obtain obstacle data.
And Y3, optimizing and verifying the obstacle identification model.
Step Y3 includes the following substeps:
and step Y310, acquiring a plurality of images of the known obstacles as a verification set.
And step Y320, inputting the images in the verification set into the obstacle recognition model for verification, and obtaining the recognition result of the obstacle recognition model.
The type of the obstacle in the image is identified as the identification result.
And step Y330, calculating the recognition accuracy of the obstacle recognition model according to the recognition result of the obstacle recognition model.
The calculation method of the identification accuracy of the obstacle identification model comprises the following steps: q = Ks/Kz; wherein Q represents the recognition accuracy of the obstacle recognition model; ks represents the number of recognized obstacles of the obstacle recognition model that match the known obstacle types; kz represents the total number of images input into the obstacle recognition model.
And step Y340, comparing the calculated recognition accuracy with a preset threshold, if the calculated recognition accuracy is smaller than the preset threshold, repeatedly optimizing and training the obstacle recognition model, and improving the recognition accuracy of the obstacle recognition model until the recognition accuracy of the obstacle recognition model is larger than the preset threshold, otherwise, repeatedly optimizing and training the obstacle recognition model.
The method comprises the following steps of optimizing and training a barrier recognition model by adopting the conventional loss function optimization method.
Step S3 also comprises that after the obstacle information is identified, the alarm gives a voice alarm to the obstacle information, for example, the voice broadcast obstacle information comprises the obstacle in front, the type of the obstacle or the distance between the obstacle and the traveling body.
As a specific embodiment of the present invention, a pre-constructed obstacle identification model is placed in an obstacle avoidance device for identifying an obstacle in an image acquired by an image acquisition device.
And S4, replanning the current travel path to the destination according to the obstacle information, and returning to execute the step S2 until the destination is reached.
As shown in fig. 5, the method for replanning the current travel path to the destination while avoiding the obstacle according to the acquired obstacle information includes:
step S410, according to the obstacle information, obtaining the position information of the turning node of the next path closest to the obstacle in the original traveling path.
Step S420, an obstacle avoidance node is inserted between the current point and the next path turning node closest to the obstacle.
Specifically, the traveling path from the current point to the obstacle avoidance node and from the obstacle avoidance node to the next turning node is used for replacing the traveling paths of the original current point and the next turning node. It can be understood that the obstacle avoidance node is inserted, so that the current ground travels to the obstacle avoidance node, and then travels from the obstacle avoidance node according to the original planned path, thereby realizing the shortest traveling path while safely avoiding the obstacle.
Specifically, according to the obtained image of the obstacle in the obstacle information, the maximum width of the outline of the obstacle from the center point is obtained; and inserting an obstacle avoidance node near the obstacle by combining the position information of the obstacle and a preset obstacle avoidance safe distance.
As a specific embodiment of the present invention, the category of the obstacle is obtained from the obtained obstacle information, when the category of the obstacle is an object that is likely to move, for example, a person; and the traveling body waits for a preset time (for example, 3S, 5S, 8S, or the like) in a stationary manner, the image acquisition device mounted on the traveling body acquires an environment image in front of the traveling body once again, an obstacle in the environment image is identified and compared with the obstacle image identified before 5S, and if the obstacle is identified to be in the original position, the judgment result is that: the obstacle is not moved (stationary), and if the identified obstacle does not exist or has changed position, the determination result is: the obstacle is active.
As a specific embodiment of the present invention, after the obstacle is detected to be an immobile object or person, an obstacle avoidance node is inserted between the current point and the next path turning node closest to the obstacle, otherwise, the vehicle still travels according to the original traveling path without inserting the obstacle avoidance node.
As shown in fig. 6, the method for inserting an obstacle avoidance node includes the following steps:
and step S421, establishing an obstacle avoidance node optimization model.
The obstacle avoidance node optimization model is established as follows:
D＝max(DL、DR)；
d represents the obstacle avoidance node with the maximum adaptive value; max (DL, DR) represents the maximum value of the selected DL and DR; DL represents an adaptive value of an obstacle avoidance node when the vehicle travels from the left side of the obstacle; DR represents an adaptive value of an obstacle avoidance node when the vehicle travels from the right side of the obstacle; SL represents the barrier value of the left-side barrier-avoiding node; SR represents the barrier value of the right obstacle avoidance node; o is 12 Representing a distance between the current point and the obstacle along a direction from the current point to a next path turning point; dz i The width distance from the ith pixel point in the left outline of the barrier to the original path (the connection direction from the current point to the turning point of the next path) is represented; dz j Representing the width distance from the jth pixel point in the right outline of the obstacle to the original path; l represents the total number of pixel points in the left contour of the obstacle; r represents the total number of pixel points in the right contour of the barrier; delta i Representing the obstacle factor of the ith pixel point in the left outline of the obstacle; delta j Representing the barrier factor of the jth pixel point in the right contour of the barrier; a represents the obstacle avoidance safety distance; bx representsA width of the traveling body; max (delta) i ×dz i ) When i is 1 to L, δ is calculated i ×dz i And obtaining L numerical values, and taking the maximum value of the L numerical values. The left contour of the obstacle refers to a contour located on the left side of the connection direction from the current point to the next path turning point, and the right contour of the obstacle refers to a contour located on the right side of the connection direction from the current point to the next path turning point.
As a specific embodiment of the present invention, the environmental image is collected by a camera, and a shooting extending direction of the camera is along a path direction where a current point and a next path turning point are located. And recognizing the existence of the obstacle on the path where the current point and the next path turning point are located through the obstacle recognition model, and outputting the obstacle image. And dividing the obstacle in the obstacle image by using a vertical line of a cross point of the path where the current point and the next path turning point are located and the obstacle, extracting pixel points of the obstacle outline positioned on the left side of the dividing line (namely the vertical line) to be used as left outline pixel points, and extracting pixel points of the obstacle outline positioned on the right side of the dividing line to be used as right outline pixel points.
Wherein, the barrier factor delta of the ith pixel point in the left contour of the barrier i The calculation formula of (a) is as follows:
wherein Hzi represents the height of the ith pixel point in the left contour of the obstacle from the ground; hx represents the height of the traveling body. Obstacle factor delta of jth pixel point in right outline of obstacle j Barrier factor delta of ith pixel point in left contour of barrier i The calculation method of (2) is the same.
As a specific embodiment of the present invention, the ground in the image of the obstacle is recognized by the pre-trained ground recognition model, so as to obtain the vertical height of the pixel point of the obstacle from the ground, and the direction of the pre-trained ground recognition model is the same as the direction of the pre-trained obstacle recognition model.
As a specific embodiment of the invention, the image acquisition device is aligned to the position of the left obstacle avoidance node, and the environment image of the left obstacle avoidance node is acquired; identifying a new obstacle in the environment image according to the obstacle identification model, and acquiring a new obstacle outline image; and calculating the barrier value of the left obstacle avoidance node according to the distance between the new obstacle and the left obstacle avoidance node in the new obstacle outline image.
The calculation method of the obstruction value SL of the left obstacle avoidance node comprises the following steps:
k represents the total number of right contour pixel points of the new obstacle at the left obstacle avoidance node; delta z The obstacle factor of the z-th pixel point in the right outline of the new obstacle at the left obstacle avoidance node is represented; delta z Calculation method and delta i The calculation methods are the same; db z And the width distance between the right contour pixel point of the new obstacle at the left obstacle avoidance node and the left obstacle avoidance node is represented, namely the distance between the current point and the left obstacle avoidance node is perpendicular to the width distance.
As a specific real-time example of the invention, the image acquisition device is aligned to the position of the right obstacle avoidance node, and the environment image of the right obstacle avoidance node is acquired; identifying a new obstacle in the environment image according to the obstacle identification model, and acquiring a new obstacle outline image; and calculating the barrier value of the right obstacle avoidance node according to the distance between the new obstacle and the right obstacle avoidance node in the new obstacle outline image.
The calculation method of the barrier value SR of the right side barrier-avoiding node comprises the following steps:
wherein, P represents the total number of left contour pixel points of a new obstacle at the right obstacle avoidance node; delta h Representing the barrier factor of the h pixel point in the left contour of the new barrier at the right barrier avoiding node; delta h Calculation method and delta i The calculation methods are the same; dc h And the width distance between the left contour pixel point of the new obstacle at the right obstacle avoidance node and the right obstacle avoidance node is represented, namely the distance between the current point and the right obstacle avoidance node is perpendicular to the width distance.
And S422, optimizing the positions of the obstacle avoidance nodes according to the obstacle avoidance node optimization model.
Calculating an adaptive value DL of an obstacle avoidance node when the vehicle travels from the left side of the obstacle and an adaptive value DR of the obstacle avoidance node when the vehicle travels from the right side of the obstacle according to the path optimization model; and selecting the position of the obstacle avoidance node corresponding to the larger adaptive value in the DL and the DR as the optimized position of the obstacle avoidance node.
And step S423, inserting the obstacle avoidance node after the optimized position on the basis of the original traveling path to re-plan the traveling path from the current point to the destination.
Specifically, the moving main body moves from the current position to an obstacle avoidance node, then moves from the obstacle avoidance node to the next node in the originally planned moving path, continues to move according to the originally planned moving path, and carries out obstacle avoidance in real time, and the obstacle avoidance method is based on the steps S2-S4.
As a specific embodiment of the present invention, the positions of the inserted obstacle avoidance nodes are: when the obstacle avoidance node is positioned on the left side of the obstacle, the distance between the obstacle avoidance node and the original path (the current point and the next node connecting path of the original planned path) is equal to
Namely, the position of the obstacle avoidance node on the left side, or when the obstacle avoidance node is positioned on the right side of the obstacle, the distance between the obstacle avoidance node and the original path (the current point and the next node connecting path of the original planned path) is
Namely the right obstacle avoidance node position.
And S5, acquiring running state data of the obstacle avoidance device in real time.
Preferably, the operation state data of the obstacle avoidance device is collected in real time, and the operation state data of the obstacle avoidance device comprises: remaining power and fault data. The fault data includes low or high voltage, high temperature, etc.
And S6, calculating the operation reliability value of the obstacle avoidance device according to the operation state data of the obstacle avoidance device.
The calculation formula of the operation reliability value of the obstacle avoidance device is as follows:
wherein, W represents the operation reliability value of the obstacle avoidance device; α 1 represents a remaining capacity usage time influence weight; e represents the remaining capacity; ec represents the power consumption per hour; alpha 2 represents the weight of successful influence of obstacle avoidance; v1 represents the number of times of successfully avoiding the obstacle; v represents the total obstacle avoidance times; α 3 represents a fault data influence weight; g represents the total variety number of the fault data; q. q of a An impact factor representing the a-th failure data; u shape a An actual measurement value representing the a-th failure data; u shape e A standard value representing the a-th failure data; α 1+ α 2+ α 3=1.
And S7, comparing the operation reliability value of the obstacle avoidance device with a preset safety threshold, if the operation reliability value of the obstacle avoidance device is smaller than the preset safety threshold, performing alarm reminding, and otherwise, continuously monitoring the operation state of the obstacle avoidance device.
If the operation reliability value of the obstacle avoidance device is smaller than the preset safety threshold value, the alarm is given out through the alarm, and the alarm can be in a voice alarm broadcasting mode, a vibration response mode and the like.
Example two
As shown in fig. 7, the present application provides a machine vision-based obstacle avoidance device 100, which is installed on a traveling body, and preferably, the machine vision-based obstacle avoidance device 100 is installed right in front of the traveling body, and the machine vision-based obstacle avoidance device 100 includes:
the path planning module 10 is configured to, in response to a destination instruction, obtain destination information and plan a current travel path to the destination; the traveling body travels according to the planned traveling path.
And the image acquisition device 20 is used for acquiring an environment image in front of the travelling body in real time in the travelling process. The image acquisition device is a video camera.
And the obstacle information acquisition module 30 is configured to identify the environment image through a pre-established obstacle identification model, and acquire obstacle information.
And the path correction module 40 is used for replanning the current travel path to the destination according to the obstacle information.
As shown in fig. 7, the path correction module 40 includes:
the obtaining module 41 is configured to obtain, according to the obstacle information, position information of a next path turning node closest to the obstacle in the original traveling path.
And an obstacle avoidance node inserting module 42, configured to insert an obstacle avoidance node between the current point and a next path turning node closest to the obstacle.
And the high-precision map module 43 is configured to perform high-precision map acquisition on a fixed place by using an RTK device to generate a site map, and simultaneously project position information of an obstacle or high-precision GPS information to obtain coordinate information of the obstacle on the site map. The method for acquiring the high-precision GPS information of the obstacle comprises the following steps: the method comprises the steps of obtaining an image of an obstacle in a fixed place through a camera, identifying a central point of the obstacle through an image identification algorithm, obtaining coordinates of the central point in the image, inquiring the image calibrated in advance through RTK equipment according to the coordinates, and further obtaining high-precision GPS information of the obstacle.
As shown in fig. 7, the obstacle avoidance apparatus 100 based on machine vision further includes:
and the device operation monitoring module 50 is used for acquiring the operation state data of the obstacle avoidance device in real time.
And the data processor 60 is configured to calculate an operation reliability value of the obstacle avoidance device according to the operation state data of the obstacle avoidance device.
And the data comparator 70 is used for comparing the operation reliability value of the obstacle avoidance device with the preset safety threshold value, if the operation reliability value of the obstacle avoidance device is smaller than the preset safety threshold value, performing alarm reminding, and otherwise, the device operation monitoring module continues to monitor the operation state of the obstacle avoidance device.
And an alarm 80 for giving an alarm when the obstacle information is acquired.
The battery module is used for providing electric energy for the electricity utilization module in the obstacle avoidance device, and the electricity utilization module is a module which works by using the electric energy, such as the path planning module 10, the image acquisition device 20, the obstacle information acquisition module 30, the path correction module 40, the device operation monitoring module 50, the data processor 60, the data comparator 70 and the alarm 80, which belong to the electricity utilization module.
As a specific embodiment of the present invention, the obstacle avoidance apparatus 100 based on machine vision further includes:
and the communication module is used for being in communication connection with the external equipment and sending alarm information or related position information to the external equipment. The external device may be a cell phone or a computer, etc.
As another embodiment of the present invention, the obstacle avoidance apparatus 100 based on machine vision further includes:
the electronic compass is mainly driven by an execution controller, and initialization, data reading and angle coordinate integration of the electronic compass are realized. After the electronic compass direction sensing function is started, the main control transmits starting information to the execution controller, the execution controller is responsible for driving the electronic compass module to start direction identification, X, Y and Z coordinates of an azimuth angle are obtained, and the azimuth deflection angle is calculated according to the existing formula. The existing electronic compass senses the azimuth deflection angle of the advancing body in the advancing process, is used for effectively compensating GPS signals and improving the accuracy of GPS navigation orientation information.
EXAMPLE III
As shown in fig. 8, the present application provides an electronic device 200, which includes a memory 201, a processor 202, and a computer program stored in the memory 201 and executable on the processor 202, wherein the processor 202 implements the steps of the obstacle avoidance method based on machine vision in the first embodiment when executing the computer program. Alternatively, the processor 202, when executing the computer program, implements the functions of each module/unit in the obstacle avoidance apparatus 100 based on machine vision according to the second embodiment.
The beneficial effect that this application realized as follows:
(1) This application trains barrier recognition model in advance, calculates the recognition accuracy of barrier recognition model, obtains the barrier recognition model that recognition accuracy accords with preset threshold to the barrier to the current route of travel orientation is discerned, improves the recognition accuracy of the preceding barrier of the body of marcing.
(2) According to the method and the device, the obstacle avoidance node is inserted between the current point and the next path turning point, the obstacle avoidance path is planned in advance, and the advancing efficiency of the advancing body is improved.
As will be appreciated by one of ordinary skill in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, devices and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by a computer program, which may be stored in a computer readable storage medium and executed by a computer to implement the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only an embodiment of the present invention, and is not intended to limit the present invention. Various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (10)
1. An obstacle avoidance method based on machine vision is characterized by comprising the following steps:
responding to the destination instruction, acquiring destination information, and planning a traveling path from the current place to the destination;
advancing according to the planned advancing path, and acquiring an environment image in front of an advancing body in real time in the advancing process;
identifying an environment image through a pre-constructed obstacle identification model to acquire obstacle information;
and replanning the current travel path to the destination according to the obstacle information.
2. A machine vision-based obstacle avoidance method according to claim 1, wherein the method of pre-constructing the obstacle identification model comprises the following steps:
acquiring a training data set;
inputting the training data set into a convolutional neural network model for training to obtain an obstacle identification model;
and carrying out optimization verification on the obstacle identification model.
3. The obstacle avoidance method based on machine vision according to claim 2, wherein the method for performing optimization verification on the obstacle identification model comprises:
acquiring images of a plurality of known obstacles as a verification set;
inputting the images in the verification set into the obstacle recognition model for verification to obtain a recognition result of the obstacle recognition model;
calculating the recognition accuracy of the obstacle recognition model according to the recognition result of the obstacle recognition model;
and comparing the calculated recognition accuracy with a preset threshold, and if the calculated recognition accuracy is smaller than the preset threshold, repeatedly optimizing and training the obstacle recognition model until the recognition accuracy of the obstacle recognition model is larger than the preset threshold.
4. The obstacle avoidance method based on machine vision according to claim 1, wherein the method of replanning the traveling path to the destination at present according to the obstacle information comprises:
acquiring the position information of a next path turning node closest to the obstacle in the original traveling path according to the obstacle information;
and inserting an obstacle avoidance node between the current point and the next path turning node closest to the obstacle.
5. An obstacle avoidance method based on machine vision according to claim 4, wherein the method of inserting an obstacle avoidance node between a current point and a next path turning node closest to an obstacle comprises:
establishing an obstacle avoidance node optimization model;
optimizing the positions of the obstacle avoidance nodes according to the obstacle avoidance node optimization model;
and inserting the obstacle avoidance nodes after the optimized positions on the basis of the original traveling path so as to replan the traveling path from the current place to the destination.
6. The obstacle avoidance method based on machine vision according to claim 1, wherein, in response to a destination instruction, destination information is acquired, and the method of planning a traveling path to a destination at present comprises:
responding to a destination instruction, and acquiring destination information;
returning the obtained destination information to the traveling body for confirmation, and judging whether the destination information is correct or not;
if the destination information is correct, planning a traveling path from the current place to the destination according to the destination information after the correctness is confirmed, and otherwise, acquiring the destination information again.
7. An obstacle avoidance apparatus based on machine vision for performing the method of any one of claims 1 to 6, mounted on a travelling body, the obstacle avoidance apparatus comprising:
the route planning module is used for responding to the destination instruction, acquiring destination information and planning a traveling route from the current place to the destination; the advancing body advances according to the planned advancing path;
the image acquisition device is used for acquiring an environment image in front of the advancing body in real time in the advancing process;
the obstacle information acquisition module is used for identifying the environment image through a pre-constructed obstacle identification model to acquire obstacle information;
and the path correcting module is used for replanning the current traveling path to the destination according to the obstacle information.
8. An obstacle avoidance device based on machine vision according to claim 7, further comprising an operation monitoring module for monitoring an operation state of the obstacle avoidance device.
9. The obstacle avoidance device based on machine vision according to claim 7, wherein the path correction module comprises:
the acquisition module is used for acquiring the position information of a turning node of the next path closest to the obstacle in the original traveling path according to the obstacle information;
and the obstacle avoidance node inserting module is used for inserting an obstacle avoidance node between the current point and the next path turning node closest to the obstacle.
10. An obstacle avoidance apparatus according to claim 7, wherein the obstacle avoidance apparatus further comprises an alarm for giving an alarm when acquiring obstacle information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211634929.0A CN115877847A (en) | 2022-12-19 | 2022-12-19 | Obstacle avoidance method and device based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211634929.0A CN115877847A (en) | 2022-12-19 | 2022-12-19 | Obstacle avoidance method and device based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115877847A true CN115877847A (en) | 2023-03-31 |
Family
ID=85754013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211634929.0A Pending CN115877847A (en) | 2022-12-19 | 2022-12-19 | Obstacle avoidance method and device based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115877847A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116203970A (en) * | 2023-04-27 | 2023-06-02 | 泰坦(天津)能源技术有限公司 | Intelligent obstacle avoidance method and system for inspection robot |
-
2022
- 2022-12-19 CN CN202211634929.0A patent/CN115877847A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116203970A (en) * | 2023-04-27 | 2023-06-02 | 泰坦(天津)能源技术有限公司 | Intelligent obstacle avoidance method and system for inspection robot |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109164809B (en) | Autonomous following control system and method for vehicle formation | |
| EP3526737B1 (en) | Neural network system for autonomous vehicle control | |
| CN110418743B (en) | Autonomous vehicle operation management obstruction monitoring | |
| CN108027242B (en) | Automatic driving navigation method, device and system, vehicle-mounted terminal and server | |
| US20200209857A1 (en) | Multimodal control system for self driving vehicle | |
| CN111426326A (en) | Navigation method, device, equipment, system and storage medium | |
| CN112509355B (en) | Vehicle searching method, device and system based on parking lot and storage medium | |
| CN110553660A (en) | unmanned vehicle trajectory planning method based on A-star algorithm and artificial potential field | |
| CN110705385B (en) | Method, device, equipment and medium for detecting angle of obstacle | |
| CN115877847A (en) | Obstacle avoidance method and device based on machine vision | |
| US11921506B2 (en) | Belief state determination for real-time decision-making | |
| US20220297732A1 (en) | Train dispatching control method, platform, and system, intelligent carriage, and medium | |
| CN107506733B (en) | Obstacle identification method, fog node and system | |
| US20210003410A1 (en) | Route generation device, route generation method and travel control device | |
| CN114475585A (en) | Automatic driving method and device at intersection, electronic equipment and automatic driving vehicle | |
| US11499833B2 (en) | Inferring lane boundaries via high speed vehicle telemetry | |
| KR102425741B1 (en) | Autonomous Driving Method Adapted for a Recognition Failure of Road Line and a Method for Building Driving Guide Data | |
| CN111986232B (en) | Target object detection method, target object detection device, robot and storage medium | |
| CN113125795A (en) | Obstacle speed detection method, device, equipment and storage medium | |
| CN116481541A (en) | Vehicle autonomous return control method, device and medium without satellite navigation | |
| Xia et al. | CyberC3: A prototype cybernetic transportation system for urban applications | |
| CN114179805B (en) | Driving direction determining method, device, equipment and storage medium | |
| CN114942639A (en) | Self-adaptive path planning method and device for mobile robot | |
| CN114184203A (en) | Path planning method, electronic equipment and mobile device | |
| CN112685517B (en) | Method and apparatus for identifying diverging/converging regions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |