CN109993106A - Barrier-avoiding method and device - Google Patents
Barrier-avoiding method and device Download PDFInfo
- Publication number
- CN109993106A CN109993106A CN201910247682.9A CN201910247682A CN109993106A CN 109993106 A CN109993106 A CN 109993106A CN 201910247682 A CN201910247682 A CN 201910247682A CN 109993106 A CN109993106 A CN 109993106A
- Authority
- CN
- China
- Prior art keywords
- avoidance
- model
- sample
- action
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the present invention provides a kind of barrier-avoiding method and device, and wherein method includes: to obtain realtime graphic based on the monocular vision sensing module installed in avoidance equipment；Realtime graphic is input to Obstacle avoidance model, obtains the action command of Obstacle avoidance model output；Wherein, Obstacle avoidance model is obtained based on sample image and the corresponding sample action instruction of sample image and the mark training of sample avoidance；Avoidance equipment avoidance is controlled based on action command.Method and apparatus provided in an embodiment of the present invention obtain realtime graphic by monocular vision sensing module, and compared to traditional binocular camera, cost is cheaper.The Obstacle avoidance model obtained by training, it is instructed based on realtime graphic output action and carries out automatic obstacle-avoiding, computing resource consumption is smaller in actual operation, it is simple and convenient, without optional equipment acceleration equipment, can be realized the real-time output of action command, guarantee avoidance equipment under complex environment can automatic obstacle-avoiding, improve the safety and stability of avoidance equipment application.
Description
Technical field
The present embodiments relate to technical field of computer vision more particularly to a kind of barrier-avoiding methods and device.
Background technique
With the continuous development of artificial intelligence technology, the detection of barrier and avoidance as unmanned plane, it is unmanned and
The key technology in the fields such as mobile robot, it is receive more and more attention.
Currently, automatic obstacle-avoiding technology depends on the binocular camera installed in avoidance equipment, by judging avoidance equipment
Whether front has barrier, and calculates accurate spatial correlation between barrier and avoidance equipment, and then to avoidance equipment
The movement of subsequent time is planned, and then realizes the action control of avoidance equipment.However, binocular camera needed for automatic obstacle-avoiding
It is with high costs and extremely complex to the data progress vision calculating of binocular camera acquisition, need to consume a large amount of computing resources, to place
The requirement for managing device is high, it usually needs adds acceleration equipment to guarantee to export in real time.
Therefore, during the actual motion of avoidance equipment, how to realize cheap, easy automatic obstacle-avoiding be one very
It is challenging to study a question.
Summary of the invention
The embodiment of the present invention provides a kind of barrier-avoiding method and device, and to solve, existing automatic obstacle-avoiding is with high costs, meter
Calculate complicated, computationally intensive problem.
In a first aspect, the embodiment of the present invention provides a kind of barrier-avoiding method, comprising:
Realtime graphic is obtained based on the monocular vision sensing module installed in avoidance equipment；
The realtime graphic is input to Obstacle avoidance model, obtains the action command of the Obstacle avoidance model output；Wherein, described
Obstacle avoidance model is obtained based on sample image and the corresponding sample action instruction of the sample image and the mark training of sample avoidance
's；
The avoidance equipment avoidance is controlled based on the action command.
Second aspect, the embodiment of the present invention provide a kind of obstacle avoidance apparatus, comprising:
Image acquisition unit, for obtaining realtime graphic based on the monocular vision sensing module installed in avoidance equipment；
Instruction acquisition unit obtains the Obstacle avoidance model output for the realtime graphic to be input to Obstacle avoidance model
Action command；Wherein, the Obstacle avoidance model is based on sample image and the corresponding sample action instruction of the sample image and sample
The mark training of this avoidance obtains；
Action control unit, for controlling the avoidance equipment avoidance based on the action command.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, including processor, communication interface, memory and total
Line, wherein processor, communication interface, memory complete mutual communication by bus, and processor can call in memory
Logical order, to execute as provided by first aspect the step of method.
Fourth aspect, the embodiment of the present invention provide a kind of non-transient computer readable storage medium, are stored thereon with calculating
Machine program is realized as provided by first aspect when the computer program is executed by processor the step of method.
A kind of barrier-avoiding method and device provided in an embodiment of the present invention obtain figure in real time by monocular vision sensing module
Picture, compared to traditional binocular camera, cost is cheaper.The Obstacle avoidance model obtained by training, it is dynamic based on realtime graphic output
Make instruction and carries out automatic obstacle-avoiding, computing resource consumption is smaller in actual operation, and it is simple and convenient, it is set without optional equipment acceleration
It is standby, can be realized the real-time output of action command, guarantee avoidance equipment under complex environment can automatic obstacle-avoiding, improve avoidance
The safety and stability of equipment application.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the flow diagram of barrier-avoiding method provided in an embodiment of the present invention；
Fig. 2 be another embodiment of the present invention provides barrier-avoiding method flow diagram；
Fig. 3 is the structural schematic diagram of obstacle avoidance apparatus provided in an embodiment of the present invention；
Fig. 4 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Binocular camera needed for existing automatic obstacle-avoiding method is with high costs, and algorithm is complicated, and real-time is poor.In this regard, this hair
Bright embodiment provides a kind of barrier-avoiding method, calculating simple effective avoidance cheap with cost of implementation.Fig. 1 is the embodiment of the present invention
The flow diagram of the barrier-avoiding method of offer, as shown in Figure 1, this method comprises:
Step 110, realtime graphic is obtained based on the monocular vision sensing module installed in avoidance equipment.
Specifically, avoidance equipment can be the equipment that any needs carry out avoidance during traveling, such as unmanned plane, nothing
People's driving or mobile robot etc..Monocular vision sensing module, that is, monocular photographic device, specifically can be camera or
Person's aerial camera etc., the embodiment of the present invention do not make specific limit to this.Monocular vision sensing module is installed in avoidance equipment, is used
The image of environment in front of acquisition characterization avoidance device action path.In step 110, obtained based on monocular vision sensing module
The image of realtime graphic, i.e. monocular vision sensing module in front of the avoidance device action path that current time collects.
Step 120, realtime graphic is input to Obstacle avoidance model, obtains the action command of Obstacle avoidance model output；Wherein, avoidance
Model is obtained based on sample image and the corresponding sample action instruction of sample image and the mark training of sample avoidance.
Specifically, Obstacle avoidance model is used for based in environment in front of real-time image analysis current time avoidance device action path
Whether there are obstacles, and how to carry out automatic obstacle-avoiding if there is barrier.It, can after realtime graphic is input to Obstacle avoidance model
To obtain the action command of Obstacle avoidance model output, i.e., the current time avoidance equipment that is obtained based on realtime graphic of Obstacle avoidance model is dynamic
It instructs, action command is used to indicate the movement that current time avoidance equipment needs to be implemented, such as to the left, to the right or turn over
Deng the present invention is not especially limit this.
In addition, can also train in advance before executing step 120 and obtain Obstacle avoidance model, can specifically instruct in the following way
It gets: firstly, collecting the sample action instruction and sample avoidance mark of great amount of samples image and sample image；Wherein, sample
Image is by manually controlling avoidance device action or avoidance equipment auto-action in the process by being installed in avoidance equipment
The sample of the image for being used to characterize environment in front of avoidance device action path that monocular vision sensing module obtains, sample image is dynamic
Make instruction as the action command that executes under environment in front of the path of motion that characterize for the sample image, sample action instructs can be with
It is the action command that operator manually controls sending, is also possible to the action command executed in avoidance equipment auto-action；Sample
The sample avoidance mark of image is used to indicate at the time of acquiring the sample image whether succeed based on the instruction of corresponding sample action
Avoidance, if there are barrier and avoidance equipment in the sample image at clear or the moment in the sample image at the moment
Successfully avoidance is instructed based on sample action, sample avoidance can be identified and be set as avoidance success, if the sample image at the moment
In there are barrier and avoidance equipment is based on sample action instruction avoidance failure, hit with barrier, then by sample avoidance
Mark is set as avoidance failure.
Immediately based on the sample action of sample image and sample image instruction and sample avoidance mark to initial model into
Row training, to obtain Obstacle avoidance model.Wherein, initial model can be single neural network model, be also possible to multiple nerves
The combination of network model, the embodiment of the present invention do not make specific limit to the type of initial model and structure.
Step 130, avoidance equipment avoidance is controlled based on action command.
Specifically, after the action command for obtaining Obstacle avoidance model output, the movement of avoidance equipment is controlled based on action command,
And then realize the automatic obstacle-avoiding of avoidance equipment.
Method provided in an embodiment of the present invention obtains realtime graphic by monocular vision sensing module, compared to traditional pair
Mesh camera, cost are cheaper.The Obstacle avoidance model obtained by training is kept away automatically based on the instruction of realtime graphic output action
Barrier, computing resource consumption is smaller in actual operation, simple and convenient, is not necessarily to optional equipment acceleration equipment, movement can be realized and refer to
Enable real-time output, guarantee avoidance equipment under complex environment can automatic obstacle-avoiding, improve the safety of avoidance equipment application
And stability.
Based on the above embodiment, before step 120 further include:
Step 101, right respectively based on sample image and the corresponding sample action instruction of sample image and sample avoidance mark
Several initial models are trained.
Specifically, during obtaining Obstacle avoidance model, several initial models, different introductory dies can be preset
Type can be the neural network model of the same type under identical structure, also has different structures, can also be different type
Neural network model, the present invention is not especially limit this.
Based on sample image and the corresponding sample action instruction of sample image and sample avoidance mark respectively at the beginning of several
Beginning model is trained, and then obtains the initial model after several different training.In the training process, it can be directed to
Different initial models is using the identical or different corresponding sample avoidance mark of sample image and sample image and sample action
Instruction is trained, and the present invention is not especially limit this.
Step 102, Obstacle avoidance model is chosen from the initial model after all training.
Specifically, it in obtaining the initial model after several training, is selected from the initial model after above-mentioned all training
Take Obstacle avoidance model.Herein, the basis for selecting of Obstacle avoidance model can be the accuracy rate of the initial model after each training, can also be
The factors such as the accuracy rate of the initial model after each training and scale of model, the present invention is not especially limit this.
Method provided in an embodiment of the present invention is protected by choosing Obstacle avoidance model from the initial model after several training
The accuracy rate and operational efficiency of Obstacle avoidance model are demonstrate,proved, to realize that accurate avoidance equipment automatic obstacle-avoiding in real time is laid a good foundation.
Based on any of the above-described embodiment, step 102 is specifically included:
Step 1021, test image is inputted into the initial model after any training, the initial model after obtaining the training is defeated
Test action instruction out.
Specifically, test image is by passing through during remote manual control avoidance device action or avoidance equipment auto-action
The figure for being used to characterize environment in front of avoidance device action path that the monocular vision sensing module being installed in avoidance equipment obtains
Picture, test image is for testing the initial model after training.Test action instruction is that the initial model after training is based on
The action command of test image output.
Step 1022, based on test action instruction deliberate action instruction corresponding with test image, after obtaining the training
The test result of initial model.
Specifically, the corresponding deliberate action instruction of test image is ring in front of the path of motion characterized for the test image
The action command actually executed under border, deliberate action instruction can be the action command that operator manually controls sending, can also be with
It is the action command executed in avoidance equipment auto-action.
After the test action instruction of the initial model output after being trained, test action instruction is referred to deliberate action
Order is compared, to obtain the test result of the initial model after training, test result is used to characterize first after training herein
The accuracy rate of beginning model.For example, when deliberate action instruction is the successful action command of avoidance, test action instruction and default
The consistent ratio of action command is higher, then the accuracy rate of the initial model after training is higher, and test result is better.In another example
When deliberate action instruction in comprising the successful action command of avoidance and avoidance failure action command when, then test action instruction with
The consistent ratio of the successful action command of avoidance is higher, and the ratio consistent with the action command of avoidance failure is lower, test
As a result better.
Step 1023, the test result based on the initial model after each training, from the initial model after all training
Choose Obstacle avoidance model.
Specifically, the initial model after the test result for obtaining the initial model after each training, after all training
Initial model after the best training of middle selection test result is as Obstacle avoidance model.
Method provided in an embodiment of the present invention, the test result based on the initial model after each training choose avoidance mould
Type can effectively improve the accuracy rate of Obstacle avoidance model.
Based on any of the above-described embodiment, before step 101 further include: pre-processed to sample image；Pretreatment includes
Go mean value.
Specifically, it before sample image is applied to the training of initial model, needs to pre-process sample image.
Herein, pretreatment includes going mean value, can also include normalization, PCA (principal components analysis, it is main at
Analysis) dimensionality reduction etc..Wherein, by carrying out mean value to sample image, i.e., pixel is removed to each pixel in sample image
Mean value, so that individual difference is highlighted, acceleration model convergence.
Based on any of the above-described embodiment, step 120 is specifically included: realtime graphic being input to Obstacle avoidance model, obtains avoidance
The action command and avoidance mark of model output；If avoidance is identified as there are barrier, avoidance prompt is issued.
Specifically, in the training process of Obstacle avoidance model, sample avoidance is identified in addition to for characterizing capturing sample image
Moment is based on the whether successful avoidance of corresponding sample action instruction, and can be also used for characterizing whether there is in corresponding sample image
Barrier.Accordingly, after realtime graphic being input to Obstacle avoidance model, Obstacle avoidance model can not only be based on realtime graphic output action
Instruction is also based on realtime graphic output avoidance mark.Herein, avoidance mark is for characterizing in realtime graphic with the presence or absence of barrier
Hinder object, i.e., whether there are obstacles in environment in front of path of motion.
After obtaining the avoidance mark of Obstacle avoidance model output, if avoidance is identified as, there are barriers, i.e., in front of path of motion
There are barriers in environment, then issue avoidance prompt.Herein, hedging prompt can be used for reminding ring in front of operator's path of motion
There are barriers in border, inform operator currently just in automatic obstacle-avoiding.
Based on any of the above-described embodiment, after step 130 further include: based on optimization image, the corresponding optimization of optimization image
Action command and optimization avoidance mark are trained Obstacle avoidance model.
Specifically, during avoidance equipment automatic obstacle-avoiding, available optimization image, and optimization image are corresponding excellent
Change action command and optimization avoidance mark is trained Obstacle avoidance model.Herein, optimization image is avoidance equipment in automatic obstacle-avoiding
Period is used to characterize in front of avoidance device action path by what the monocular vision sensing module being installed in avoidance equipment obtained
The image of environment, optimization action command are action command of the former Obstacle avoidance model based on optimization image output, and optimization avoidance mark is used
In instruction based on optimization action command progress avoidance as a result, i.e. avoidance success or avoidance failure.Based on above-mentioned optimization image, excellent
Change the corresponding optimization action command of image and optimization avoidance mark is iterated tuning to Obstacle avoidance model, can further increase and keep away
The accuracy rate for hindering model, makes up Obstacle avoidance model loophole.
Especially in the case where avoidance failure, corresponding optimization image, optimization image are corresponding excellent during avoidance failure
Change action command leakage loophole action command corresponding with optimization avoidance mark respectively loophole image, loophole image and loophole is kept away
Barrier mark.Herein, the monocular vision during loophole image is the failure of avoidance equipment automatic obstacle-avoiding by being installed in avoidance equipment
The image for being used to characterize environment in front of avoidance device action path that sensing module obtains, loophole action command are former Obstacle avoidance model
Based on the action command for leading to avoidance failure of loophole image output, loophole avoidance is identified as avoidance failure.Based on loophole image,
The corresponding loophole action command of loophole image and loophole avoidance mark are trained update to Obstacle avoidance model, can effectively make up
Loophole further increases the performance of Obstacle avoidance model.
Based on any of the above-described embodiment, Fig. 2 be another embodiment of the present invention provides barrier-avoiding method flow diagram, such as
Shown in Fig. 2, the avoidance equipment in this method is unmanned plane, this includes the following steps:
Step 210, sample collection.
The sample action instruction and sample avoidance for collecting great amount of samples image and sample image identify；Wherein, sample image
For by being passed in remote manual control unmanned plane during flying or the automatic flight course of unmanned plane by the monocular vision being installed on unmanned plane
Sense module obtain for characterizing the image of environment in front of unmanned plane path of motion, the sample action instruction of sample image for for
The action command executed under environment in front of the path of motion of sample image characterization, it is manual that sample action instruction can be operator
The action command issued is controlled, the action command executed in unmanned plane auto-action is also possible to；The sample avoidance of sample image
Mark is used to indicate at the time of acquiring the sample image and instructs the avoidance that whether succeeds based on corresponding sample action, if the moment
In sample image in the sample image at clear or the moment there are barrier and unmanned plane be based on sample action instruct at
Function avoidance, can by sample avoidance identify be set as avoidance success, if in the sample image at the moment there are barrier and nobody
Machine is based on sample action instruction avoidance failure, hits with barrier, then sets avoidance failure for sample avoidance mark.
Then execute step 220.
Step 220, model training.
In order to improve trained speed and precision, mean value is carried out to each sample image.After going mean value
The instruction of the sample action of sample image and sample image and sample avoidance mark to the initial models of several different structures into
Row training.After training, by the inclusion of the test set test for having test image and the corresponding deliberate action instruction of test image
Initial model after each training, and select the work of the initial model after a best training of accuracy rate highest, effect
For Obstacle avoidance model.
Then execute step 230.
Step 230, iteration tuning.
After obtaining Obstacle avoidance model, judge whether to need to carry out tuning iteration to Obstacle avoidance model.
If necessary to carry out tuning iteration, then during Obstacle avoidance model being used in unmanned plane automatic obstacle-avoiding, obtain online
Optimization image, and the corresponding optimization action command of optimization image and optimization avoidance identify.Herein, optimization image is unmanned plane
Road is acted for characterizing unmanned plane by what the monocular vision sensing module being installed on unmanned plane obtained during automatic obstacle-avoiding
The image of environment in front of diameter, optimization action command are action command of the former Obstacle avoidance model based on optimization image output, optimize avoidance
Mark is used to indicate based on optimization action command progress avoidance as a result, i.e. avoidance success or avoidance failure.Then execute step
220, Obstacle avoidance model is instructed based on optimization image, and the corresponding optimization action command of optimization image and optimization avoidance mark
Practice, realizes the iteration optimization of Obstacle avoidance model.
If you do not need to carrying out tuning iteration, 240 are thened follow the steps.
Step 240, system deployment.
Monocular vision sensing module is installed on unmanned plane, the deployment of unmanned plane obstacle avoidance system is completed.Herein, monocular vision
Sensing module can be Haikang USB camera.The realtime graphic of monocular vision sensing module acquisition is obtained, and realtime graphic is defeated
Enter to Obstacle avoidance model, obtain the action command of Obstacle avoidance model output, unmanned plane avoidance is controlled based on action command.
Method provided in an embodiment of the present invention obtains realtime graphic by monocular vision sensing module, compared to traditional pair
Mesh camera, cost are cheaper.The Obstacle avoidance model obtained by training is kept away automatically based on the instruction of realtime graphic output action
Barrier, computing resource consumption is smaller in actual operation, simple and convenient, is not necessarily to optional equipment acceleration equipment, movement can be realized and refer to
The real-time output enabled, guarantee unmanned plane under complex environment can automatic obstacle-avoiding, improve the safety of unmanned plane application and steady
It is qualitative.
Based on any of the above-described embodiment, this method is tested.Experiment scene size is about 200m*30m, including one
Piece hurst and a piece of lawn.In experimentation, progress data collection task first, using being installed in monocular vision on unmanned plane
Sensing module is acquired the path data to be flown, obtain include sample image and sample image sample action instruction and
Sample set including sample avoidance mark, and the test including the instruction of the deliberate action of test image and test image
Collection.Initial model is trained subsequently, based on sample set, obtains Obstacle avoidance model.Then unmanned plane is controlled based on Obstacle avoidance model
Automatic obstacle-avoiding.It realizes to above-mentioned Success in Experiment unmanned plane to fly in the woods and get around trees, and can in another woods
To fly well and avoidance, there is relatively good robustness to the variation of illumination.
Based on any of the above-described embodiment, this method is tested.Experiment scene size is about 200m*100m, including
The corridor of one 50m*4m and a piece of bamboo grove.In experimentation, progress data collection task first, using being installed on unmanned plane
Monocular vision sensing module is acquired the path data to be flown, obtain include sample image and sample image sample it is dynamic
Make the sample set including instruction and sample avoidance mark, and including the instruction of the deliberate action of test image and test image
Test set.Initial model is trained subsequently, based on sample set, obtains Obstacle avoidance model.Then it is controlled based on Obstacle avoidance model
Unmanned plane automatic obstacle-avoiding.It realizes to above-mentioned Success in Experiment unmanned plane to fly in corridor and get around barrier, and in bamboo grove road
It can also fly well in road and avoidance, and there is relatively good robustness to the variation of illumination.
Based on any of the above-described embodiment, this method is tested.Experiment scene size is about 500m*200m, including
The a piece of woods and lawn.In experimentation, progress data collection task first is sensed using monocular vision on unmanned plane is installed in
Module is acquired the path data to be flown, obtain include sample image and sample image sample action instruction and sample
Sample set including avoidance mark, and the test set including the instruction of the deliberate action of test image and test image.With
Afterwards, initial model is trained based on sample set, obtains Obstacle avoidance model.Then it is kept away automatically based on Obstacle avoidance model control unmanned plane
Barrier.It realizes to above-mentioned Success in Experiment unmanned plane to fly in corridor and get around barrier, and can also be well in the woods
It flies and gets around the trees encountered, at the same time in different light application time sections, can successfully act and avoidance, the change to illumination
Changing has relatively good robustness.
Based on any of the above-described embodiment, Fig. 3 is the structural schematic diagram of obstacle avoidance apparatus provided in an embodiment of the present invention, such as Fig. 3
Shown, which includes image acquisition unit 310, instruction acquisition unit 320 and action control unit 330.
Wherein, image acquisition unit 310 is used to obtain based on the monocular vision sensing module installed in avoidance equipment real-time
Image；
Instruction acquisition unit 320 is used to the realtime graphic being input to Obstacle avoidance model, obtains the Obstacle avoidance model output
Action command；Wherein, the Obstacle avoidance model be based on sample image and the instruction of the sample image corresponding sample action and
The mark training of sample avoidance obtains；
Action control unit 330 is used to control the avoidance equipment avoidance based on the action command.
Device provided in an embodiment of the present invention obtains realtime graphic by monocular vision sensing module, compared to traditional pair
Mesh camera, cost are cheaper.The Obstacle avoidance model obtained by training is kept away automatically based on the instruction of realtime graphic output action
Barrier, computing resource consumption is smaller in actual operation, simple and convenient, is not necessarily to optional equipment acceleration equipment, movement can be realized and refer to
Enable real-time output, guarantee avoidance equipment under complex environment can automatic obstacle-avoiding, improve the safety of avoidance equipment application
And stability.
Based on any of the above-described embodiment, which further includes training unit and selection unit；
Wherein, training unit is used for based on the sample image and the corresponding sample action instruction of the sample image and sample
This avoidance mark is respectively trained several initial models；
Selection unit is for choosing the Obstacle avoidance model from the initial model after all training.
Based on any of the above-described embodiment, selection unit is specifically used for:
The initial model after test image to be inputted to any training, the initial model after obtaining any training are defeated
Test action instruction out；
Deliberate action instruction corresponding with the test image is instructed based on the test action, obtains any training
The test result of initial model afterwards；
Based on the test result of the initial model after each training, selected from the initial model after all training
Take the Obstacle avoidance model.
Based on any of the above-described embodiment, which further includes pretreatment unit；
Pretreatment unit is for pre-processing the sample image；The pretreatment includes going mean value.
Based on any of the above-described embodiment, instruction acquisition unit 320 is specifically used for:
The realtime graphic is input to the Obstacle avoidance model, obtain Obstacle avoidance model output the action command and
Avoidance mark；
If the avoidance is identified as there are barrier, avoidance prompt is issued.
Based on any of the above-described embodiment, which further includes iterative optimization unit；
Iterative optimization unit is used for based on optimization image, the corresponding optimization action command of the optimization image and optimization avoidance
Mark is trained the Obstacle avoidance model.
Based on any of the above-described embodiment, in the device, avoidance equipment is unmanned plane.
Fig. 4 is the entity structure schematic diagram of electronic equipment provided in an embodiment of the present invention, as shown in figure 4, the electronic equipment
It may include: processor (processor) 401,402, memory communication interface (Communications Interface)
(memory) 403 and communication bus 404, wherein processor 401, communication interface 402, memory 403 pass through communication bus 404
Complete mutual communication.Processor 401 can call the meter that is stored on memory 403 and can run on processor 401
Calculation machine program, to execute the barrier-avoiding method of the various embodiments described above offer, for example, based on the monocular view installed in avoidance equipment
Feel that sensing module obtains realtime graphic；The realtime graphic is input to Obstacle avoidance model, obtains the dynamic of the Obstacle avoidance model output
It instructs；Wherein, the Obstacle avoidance model is based on sample image and the corresponding sample action instruction of the sample image and sample
Avoidance mark training obtains；The avoidance equipment avoidance is controlled based on the action command.
In addition, the logical order in above-mentioned memory 403 can be realized by way of SFU software functional unit and conduct
Independent product when selling or using, can store in a computer readable storage medium.Based on this understanding, originally
The technical solution of the inventive embodiments substantially part of the part that contributes to existing technology or the technical solution in other words
It can be embodied in the form of software products, which is stored in a storage medium, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes the present invention respectively
The all or part of the steps of a embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory
(ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk
Etc. the various media that can store program code.
The embodiment of the present invention also provides a kind of non-transient computer readable storage medium, is stored thereon with computer program,
The computer program is implemented to carry out the barrier-avoiding method of the various embodiments described above offer when being executed by processor, for example, be based on
The monocular vision sensing module installed in avoidance equipment obtains realtime graphic；The realtime graphic is input to Obstacle avoidance model, is obtained
Take the action command of the Obstacle avoidance model output；Wherein, the Obstacle avoidance model is based on sample image and the sample image pair
What the sample action instruction and the mark training of sample avoidance answered obtained；The avoidance equipment is controlled based on the action command to keep away
Barrier.
The apparatus embodiments described above are merely exemplary, wherein described, unit can as illustrated by the separation member
It is physically separated with being or may not be, component shown as a unit may or may not be physics list
Member, it can it is in one place, or may be distributed over multiple network units.It can be selected according to the actual needs
In some or all of the modules achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creativeness
Labour in the case where, it can understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be embodied in the form of software products in other words, should
Computer software product may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
Method described in certain parts of example or embodiment.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；
And these are modified or replaceed, technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (10)
1. a kind of barrier-avoiding method characterized by comprising
Realtime graphic is obtained based on the monocular vision sensing module installed in avoidance equipment；
The realtime graphic is input to Obstacle avoidance model, obtains the action command of the Obstacle avoidance model output；Wherein, the avoidance
Model is obtained based on sample image and the corresponding sample action instruction of the sample image and the mark training of sample avoidance；
The avoidance equipment avoidance is controlled based on the action command.
2. being obtained the method according to claim 1, wherein described be input to Obstacle avoidance model for the realtime graphic
The action command of the Obstacle avoidance model output is taken, before further include:
Based on the sample image and the corresponding sample action instruction of the sample image and sample avoidance mark respectively to several
A initial model is trained；
The Obstacle avoidance model is chosen from the initial model after all training.
3. according to the method described in claim 2, it is characterized in that, being chosen in the initial model from after all training
The Obstacle avoidance model, specifically includes:
The initial model after test image to be inputted to any training, what the initial model after obtaining any training exported
Test action instruction；
Deliberate action instruction corresponding with the test image is instructed based on the test action, after obtaining any training
The test result of initial model；
Based on the test result of the initial model after each training, institute is chosen from the initial model after all training
State Obstacle avoidance model.
4. according to the method described in claim 2, it is characterized in that, described be based on the sample image and the sample image pair
The sample action instruction answered respectively is trained several initial models, before further include:
The sample image is pre-processed；The pretreatment includes going mean value.
5. being obtained the method according to claim 1, wherein described be input to Obstacle avoidance model for the realtime graphic
The action command for taking the Obstacle avoidance model output, specifically includes:
The realtime graphic is input to the Obstacle avoidance model, obtains the action command and avoidance of the Obstacle avoidance model output
Mark；
If the avoidance is identified as there are barrier, avoidance prompt is issued.
6. the method according to claim 1, wherein described control the avoidance equipment based on the action command
Avoidance, later further include:
Based on optimization image, the corresponding optimization action command of the optimization image and optimize avoidance identify to the Obstacle avoidance model into
Row training.
7. method according to any one of claim 1 to 6, which is characterized in that the avoidance equipment is unmanned plane.
8. a kind of obstacle avoidance apparatus characterized by comprising
Image acquisition unit, for obtaining realtime graphic based on the monocular vision sensing module installed in avoidance equipment；
Instruction acquisition unit obtains the movement of the Obstacle avoidance model output for the realtime graphic to be input to Obstacle avoidance model
Instruction；Wherein, the Obstacle avoidance model is kept away based on sample image and the corresponding sample action instruction of the sample image and sample
Barrier mark training obtains；
Action control unit, for controlling the avoidance equipment avoidance based on the action command.
9. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and bus, wherein processor leads to
Believe that interface, memory complete mutual communication by bus, processor can call the logical order in memory, to execute
Method as described in claim 1 to 7 is any.
10. a kind of non-transient computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer
The method as described in claim 1 to 7 is any is realized when program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910247682.9A CN109993106A (en) | 2019-03-29 | 2019-03-29 | Barrier-avoiding method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910247682.9A CN109993106A (en) | 2019-03-29 | 2019-03-29 | Barrier-avoiding method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109993106A true CN109993106A (en) | 2019-07-09 |
Family
ID=67131774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910247682.9A Pending CN109993106A (en) | 2019-03-29 | 2019-03-29 | Barrier-avoiding method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109993106A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365299A (en) * | 2013-08-02 | 2013-10-23 | 中国科学院自动化研究所 | Method and device for avoiding obstacle of unmanned aerial vehicle |
| CN105760894A (en) * | 2016-03-16 | 2016-07-13 | 哈尔滨伟方智能科技开发有限责任公司 | Robot navigation method based on machine vision and machine learning |
| CN106155091A (en) * | 2016-08-31 | 2016-11-23 | 众芯汉创（北京）科技有限公司 | A kind of unmanned plane barrier-avoiding method and device |
| CN106168808A (en) * | 2016-08-25 | 2016-11-30 | 南京邮电大学 | A kind of rotor wing unmanned aerial vehicle automatic cruising method based on degree of depth study and system thereof |
| CN107065890A (en) * | 2017-06-02 | 2017-08-18 | 北京航空航天大学 | A kind of unmanned vehicle intelligent barrier avoiding method and system |
| CN107491072A (en) * | 2017-09-05 | 2017-12-19 | 百度在线网络技术（北京）有限公司 | Vehicle obstacle-avoidance method and apparatus |
| CN107553490A (en) * | 2017-09-08 | 2018-01-09 | 深圳市唯特视科技有限公司 | A kind of monocular vision barrier-avoiding method based on deep learning |
| CN108068107A (en) * | 2016-11-09 | 2018-05-25 | 哈尔滨工大天才智能科技有限公司 | A kind of deicing high-voltage power transmission line robot vision control method |
| CN108255182A (en) * | 2018-01-30 | 2018-07-06 | 上海交通大学 | A kind of service robot pedestrian based on deeply study perceives barrier-avoiding method |
| CN109238288A (en) * | 2018-09-10 | 2019-01-18 | 电子科技大学 | Autonomous navigation method in a kind of unmanned plane room |
-
2019
- 2019-03-29 CN CN201910247682.9A patent/CN109993106A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365299A (en) * | 2013-08-02 | 2013-10-23 | 中国科学院自动化研究所 | Method and device for avoiding obstacle of unmanned aerial vehicle |
| CN105760894A (en) * | 2016-03-16 | 2016-07-13 | 哈尔滨伟方智能科技开发有限责任公司 | Robot navigation method based on machine vision and machine learning |
| CN106168808A (en) * | 2016-08-25 | 2016-11-30 | 南京邮电大学 | A kind of rotor wing unmanned aerial vehicle automatic cruising method based on degree of depth study and system thereof |
| CN106155091A (en) * | 2016-08-31 | 2016-11-23 | 众芯汉创（北京）科技有限公司 | A kind of unmanned plane barrier-avoiding method and device |
| CN108068107A (en) * | 2016-11-09 | 2018-05-25 | 哈尔滨工大天才智能科技有限公司 | A kind of deicing high-voltage power transmission line robot vision control method |
| CN107065890A (en) * | 2017-06-02 | 2017-08-18 | 北京航空航天大学 | A kind of unmanned vehicle intelligent barrier avoiding method and system |
| CN107491072A (en) * | 2017-09-05 | 2017-12-19 | 百度在线网络技术（北京）有限公司 | Vehicle obstacle-avoidance method and apparatus |
| CN107553490A (en) * | 2017-09-08 | 2018-01-09 | 深圳市唯特视科技有限公司 | A kind of monocular vision barrier-avoiding method based on deep learning |
| CN108255182A (en) * | 2018-01-30 | 2018-07-06 | 上海交通大学 | A kind of service robot pedestrian based on deeply study perceives barrier-avoiding method |
| CN109238288A (en) * | 2018-09-10 | 2019-01-18 | 电子科技大学 | Autonomous navigation method in a kind of unmanned plane room |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102362744B1 (en) | Method for recognizing face using multiple patch combination based on deep neural network with fault tolerance and fluctuation robustness in extreme situation | |
| CN106709461B (en) | Activity recognition method and device based on video | |
| CN109658418A (en) | Learning method, device and the electronic equipment of scene structure | |
| US11100646B2 (en) | Future semantic segmentation prediction using 3D structure | |
| KR20200093425A (en) | Learning method and learning device for generation of virtual feature maps whose characteristics are same as or similar to those of real feature maps by using gan capable of being applied to domain adaptation to be used in virtual driving environments | |
| CN107784294B (en) | Face detection and tracking method based on deep learning | |
| CN107481292A (en) | The attitude error method of estimation and device of vehicle-mounted camera | |
| US20230419521A1 (en) | Unsupervised depth prediction neural networks | |
| CN114415628A (en) | Automatic driving test method and device, electronic equipment and storage medium | |
| CN107741781A (en) | Flight control method, device, unmanned plane and the storage medium of unmanned plane | |
| WO2020211242A1 (en) | Behavior recognition-based method, apparatus and storage medium | |
| CN110465089B (en) | Map exploration method, map exploration device, map exploration medium and electronic equipment based on image recognition | |
| CN109109863A (en) | Smart machine and its control method, device | |
| Sizkouhi et al. | RoboPV: An integrated software package for autonomous aerial monitoring of large scale PV plants | |
| CN114120253A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN112990222B (en) | Image boundary knowledge migration-based guided semantic segmentation method | |
| KR102372687B1 (en) | Learning method and learning device for heterogeneous sensor fusion by using merging network which learns non-maximum suppression | |
| CN108960412A (en) | Image-recognizing method, device and computer readable storage medium | |
| WO2021194490A1 (en) | Method and system for improved attention map guidance for visual recognition in images | |
| CN106778910A (en) | Deep learning system and method based on local training | |
| CN113392793A (en) | Method, device, equipment, storage medium and unmanned vehicle for identifying lane line | |
| Teng et al. | Clickbait: click-based accelerated incremental training of convolutional neural networks | |
| CN109685805A (en) | A kind of image partition method and device | |
| CN115294539A (en) | Multitask detection method and device, storage medium and terminal | |
| CN109977884B (en) | Target following method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190709 |
|
| RJ01 | Rejection of invention patent application after publication |