CN110244710A - Automatic Track Finding method, apparatus, storage medium and electronic equipment - Google Patents
Automatic Track Finding method, apparatus, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN110244710A CN110244710A CN201910407609.3A CN201910407609A CN110244710A CN 110244710 A CN110244710 A CN 110244710A CN 201910407609 A CN201910407609 A CN 201910407609A CN 110244710 A CN110244710 A CN 110244710A
- Authority
- CN
- China
- Prior art keywords
- angle
- rgb
- avoidance
- network model
- measuring angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000000007 visual effect Effects 0.000 claims abstract description 17
- 230000011218 segmentation Effects 0.000 claims description 30
- 238000005259 measurement Methods 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004888 barrier function Effects 0.000 abstract description 14
- 238000004891 communication Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- KLDZYURQCUYZBL-UHFFFAOYSA-N 2-[3-[(2-hydroxyphenyl)methylideneamino]propyliminomethyl]phenol Chemical compound OC1=CC=CC=C1C=NCCCN=CC1=CC=CC=C1O KLDZYURQCUYZBL-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 201000001098 delayed sleep phase syndrome Diseases 0.000 description 1
- 208000033921 delayed sleep phase type circadian rhythm sleep disease Diseases 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Image Analysis (AREA)
Abstract
This disclosure relates to a kind of Automatic Track Finding method, apparatus, storage medium and electronic equipment, comprising: obtain the RGB figure and depth map in the target visual field；RGB is schemed into the input as the first predetermined angle network model, to obtain the first pre- measuring angle of the first predetermined angle network model output；Using RGB figure and depth map as the input of default avoidance network model, to obtain the avoidance type and accuracy corresponding with avoidance type of default avoidance network model output；The second pre- measuring angle is determined according to avoidance type and accuracy corresponding with avoidance type；Target travel angle is determined according to the first pre- measuring angle and the second pre- measuring angle, is tracked with realizing.The travel angle got can be made more accurate in this way, and therefore the avoidance due to combining the output of avoidance network model when tracking as a result, can also avoid the barrier in travelling route, so that tracking for equipment can be more accurate and intelligent.
Description
Technical field
This disclosure relates to computer vision field, and in particular, to a kind of Automatic Track Finding method, apparatus, storage medium and
Electronic equipment.
Background technique
In real life, environment locating for the various smart machines such as intelligent robot or automatic driving vehicle is usually dynamic
State, variable, and these smart machines usually may require that and be moved in the environment of variation, therefore these smart machines exist
It has to overcome the problems, such as to be how that optimal travelling route can be found in the process of moving in use process, so that intelligence
Energy equipment can not only be completed to track, that is, reach specified place, but also can be avoided the travelling route during tracking
In be likely to occur various barriers, this just needs one perfect to track algorithm to assist smart machine to track to realize.
Currently, common some algorithms that track are some traditional path planning algorithms in the market, it is multiple to implement comparison
It is miscellaneous, higher cost, and mostly that barrier avoiding function is not added, so that smart machine can not handle the obstacle occurred on programme path
Object, irregular and while comparing the barrier being difficult to especially occur will lead to smart machine and can not handle, such as on road surface
Some short small barriers etc..
Summary of the invention
Purpose of this disclosure is to provide a kind of Automatic Track Finding method, apparatus, storage medium and electronic equipment, can make to obtain
The travel angle arrived is more accurate, and therefore the avoidance due to combining the output of avoidance network model is as a result, can also seek
The barrier in travelling route is avoided when mark, so that tracking for equipment can be more accurate and intelligent.
To achieve the goals above, the disclosure provides a kind of method of Automatic Track Finding, which comprises
Obtain the RGB figure and depth map in the target visual field；
The RGB is schemed into the input as the first predetermined angle network model, to obtain the first predetermined angle network
First pre- measuring angle of model output；
Using RGB figure and the depth map as the input of default avoidance network model, to obtain the default avoidance
The avoidance type and accuracy corresponding with the avoidance type of network model output；
The second pre- measuring angle is determined according to the avoidance type and accuracy corresponding with the avoidance type；
Target travel angle is determined according to the described first pre- measuring angle and the second pre- measuring angle, is tracked with realizing.
Optionally, for training the training data of the first predetermined angle network model to obtain in accordance with the following methods:
According to target travelling route obtain the continuous multiple frames RGB in the target travelling route figure, and with described in each frame
RGB schemes the measurement angle in corresponding horizontal direction；
The RGB is schemed into the difference of corresponding with the neighbor map measurement angle as the angle true value of the RGB figure,
The neighbor map is the RGB figure got after the RGB figure；
The RGB of the angled true value of the band is schemed into the training data as the first predetermined angle network model.
Optionally, the default avoidance network model includes:
Semantic segmentation sub-network model is used to be schemed according to RGB figure and depth map output with the RGB and described
The corresponding semantic segmentation figure of depth map；
Avoidance sub-network model, for by the RGB figure, the depth map and by the semantic segmentation sub-network model it is defeated
The semantic segmentation figure corresponding with RGB figure and the depth map out as inputting, with export the avoidance type and with
The corresponding accuracy of the avoidance type.
Optionally, the avoidance type includes straight trip, turns left and turn right, described to keep away according to the avoidance type and with described
The corresponding accuracy of barrier type determines that the second pre- measuring angle includes:
In the case where the avoidance type is straight trip, the second pre- measuring angle is calculated according to following formula:
θ=90 ° (1-P)；
In the case where the avoidance type is to turn left, the second pre- measuring angle is calculated according to following formula:
θ=- 90 ° × P；
In the case where the avoidance type is to turn right, the second pre- measuring angle is calculated according to following formula:
θ=90 ° × P；
Wherein, θ is the described second pre- measuring angle, and P is the accuracy.
Optionally, described that target travel angle packet is determined according to the described first pre- measuring angle and the second pre- measuring angle
It includes:
The average value of the first pre- measuring angle and the second pre- measuring angle is determined as the target angle of travel
Degree.
Optionally, described that target travel angle packet is determined according to the described first pre- measuring angle and the second pre- measuring angle
It includes:
Described first pre- measuring angle and the second pre- measuring angle are inputted into the second predetermined angle network model to obtain
State target travel angle.
The disclosure also provides a kind of Automatic Track Finding device, and described device includes:
Module is obtained, the RGB for obtaining the target visual field schemes and depth map；
First angle obtains module, for the RGB to be schemed to the input as the first predetermined angle network model, to obtain
First pre- measuring angle of the first predetermined angle network model output；
Second angle obtains module, for using RGB figure and the depth map as the defeated of default avoidance network model
Enter, to obtain the avoidance type and accuracy corresponding with the avoidance type of the default avoidance network model output；
The second angle obtains module and is also used to, according to the avoidance type and corresponding with the avoidance type correct
Rate determines the second pre- measuring angle；
Target travel angle obtains module, for determining mesh according to the described first pre- measuring angle and the second pre- measuring angle
Travel angle is marked, is tracked with realizing.
Optionally, for training the training data of the first predetermined angle network model to obtain in accordance with the following methods:
According to target travelling route obtain the continuous multiple frames RGB in the target travelling route figure, and with described in each frame
RGB schemes the measurement angle in corresponding horizontal direction；
The RGB is schemed into the difference of corresponding with the neighbor map measurement angle as the angle true value of the RGB figure,
The neighbor map is the RGB figure got after the RGB figure；
The RGB of the angled true value of the band is schemed into the training data as the first predetermined angle network model.
Optionally, the default avoidance network model includes:
Semantic segmentation sub-network model is used to be schemed according to RGB figure and depth map output with the RGB and described
The corresponding semantic segmentation figure of depth map；
Avoidance sub-network model, for by the RGB figure, the depth map and by the semantic segmentation sub-network model it is defeated
The semantic segmentation figure corresponding with RGB figure and the depth map out as inputting, with export the avoidance type and with
The corresponding accuracy of the avoidance type.
Optionally, the avoidance type includes straight trip, turns left and turn right, described to keep away according to the avoidance type and with described
The corresponding accuracy of barrier type determines that the second pre- measuring angle includes:
In the case where the avoidance type is straight trip, the second pre- measuring angle is calculated according to following formula:
θ=90 ° (1-P)；
In the case where the avoidance type is to turn left, the second pre- measuring angle is calculated according to following formula:
θ=- 90 ° × P；
In the case where the avoidance type is to turn right, the second pre- measuring angle is calculated according to following formula:
θ=90 ° × P；
Wherein, θ is the described second pre- measuring angle, and P is the accuracy.
Optionally, the target travel angle obtains module and is also used to:
The average value of the first pre- measuring angle and the second pre- measuring angle is determined as the target angle of travel
Degree.
Optionally, the target travel angle obtains module and is also used to:
Described first pre- measuring angle and the second pre- measuring angle are inputted into the second predetermined angle network model to obtain
State target travel angle.
The disclosure also provides a kind of computer readable storage medium, is stored thereon with computer program, and the program is processed
The step of Automatic Track Finding method described above is realized when device executes.
The disclosure also provides a kind of electronic equipment, comprising:
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize Automatic Track Finding side described above
The step of method.
Through the above technical solutions, get the target visual field RGB figure and depth map after, can using RGB figure as
The input of first predetermined angle network model obtains the first pre- measuring angle, and using RGB figure and depth map as default avoidance network
The input of model obtains the accuracy of avoidance type and the avoidance type, and then obtains the second pre- measuring angle, finally combines first
Pre- measuring angle and the second pre- measuring angle determine final target travel angle, and travel angle when enabling to track in this way is more
Add precisely, and therefore the avoidance due to combining the output of avoidance network model is as a result, can also avoid the barrier in travelling route
Hinder object, so that tracking for equipment can be more accurate and intelligent.
Other feature and advantage of the disclosure will the following detailed description will be given in the detailed implementation section.
Detailed description of the invention
Attached drawing is and to constitute part of specification for providing further understanding of the disclosure, with following tool
Body embodiment is used to explain the disclosure together, but does not constitute the limitation to the disclosure.In the accompanying drawings:
Fig. 1 is the flow chart according to a kind of method of Automatic Track Finding shown in one exemplary embodiment of the disclosure.
Fig. 2 is in a kind of Automatic Track Finding method shown according to disclosure another exemplary embodiment for training described the
The flow chart of the acquisition methods of the training data of one predetermined angle network model.
Fig. 3 is the structural block diagram according to a kind of device of Automatic Track Finding shown in one exemplary embodiment of the disclosure.
Fig. 4 is the block diagram of a kind of electronic equipment shown according to an exemplary embodiment.
Fig. 5 is the block diagram of a kind of electronic equipment shown according to an exemplary embodiment.
Specific embodiment
It is described in detail below in conjunction with specific embodiment of the attached drawing to the disclosure.It should be understood that this place is retouched
The specific embodiment stated is only used for describing and explaining the disclosure, is not limited to the disclosure.
Fig. 1 is a kind of flow chart of Automatic Track Finding method shown according to one exemplary embodiment of the disclosure.Such as Fig. 1 institute
Show, the method includes the steps 101 to step 105.
In a step 101, the RGB figure and depth map in the target visual field are obtained.The target visual field can be to be installed on intelligence to set
The position and the instruction institute received that the image acquiring devices such as video camera, video camera, visual sensor in standby are arranged according to it
The environmental field for the image information that can be got.For example, in intelligent robot, which can be with to be set to the intelligence
The environmental field that visual sensor in energy robot can be seen.RGB figure and the depth map can be directly through the view
Feel that sensor, the existing image acquiring device such as RGB-D camera directly acquire, be also possible to by other data come
What source was obtained by post-processing, be how to acquire with no restriction for RGB figure and the depth map in the disclosure, only
Will RGB figure and the depth map all correspond in the same target visual field included same frame picture frame.
In a step 102, the RGB is schemed into the input as the first predetermined angle network model, to obtain described first
First pre- measuring angle of predetermined angle network model output.The first predetermined angle network model can be preparatory trained use
In the network model for exporting the angle that tracks, can be obtained advancing for RGB figure institute's show surroundings according to the RGB figure of input
Angle.In the disclosure with no restrictions to the first predetermined angle network model, it is gone as long as being that by according to RGB figure
Into the network model of angle.
In step 103, using RGB figure and the depth map as the input of default avoidance network model, to obtain
The avoidance type and accuracy corresponding with the avoidance type of the default avoidance network model output.The default avoidance network
Model can in advance it is trained can according to input correspond to same frame picture frame RGB scheme and depth map, output keeping away
Hinder the arbitrary network model of type and accuracy corresponding with the avoidance type.The avoidance type can be for example " straight
Row ", " left-hand rotation ", " right-hand rotation " etc., the accuracy are confidence level of the default avoidance network model to the avoidance type of output.Example
Such as, which can scheme according to the RGB of input and depth map all judges to obtain one to each avoidance type
A confidence level, and using the highest avoidance type of confidence level as output, in the disclosure, while exporting the avoidance type,
Accuracy of the confidence level of the avoidance type as the avoidance type for the RGB figure and depth map of input can also be exported.
At step 104, the second prediction is determined according to the avoidance type and accuracy corresponding with the avoidance type
Angle.After the avoidance type and corresponding accuracy for obtaining the default avoidance network model output, it will be able to calculate
Corresponding pre- measuring angle is obtained as the second pre- measuring angle.In the disclosure for how according to avoidance type and with avoidance type
Corresponding accuracy determines the method for the second pre- measuring angle without limitation, as long as can be obtained characterizing according to above-mentioned factor
The angle of the avoidance type and corresponding accuracy.But preferably, in a kind of possible embodiment, in avoidance
Type include " straight trip ", " left-hand rotation " and " right-hand rotation " in the case where, it is described according to the avoidance type and with the avoidance type pair
The accuracy answered determines that the second pre- measuring angle may include the following contents:
1) in the case where the avoidance type is straight trip, the second pre- measuring angle is calculated according to following formula: θ=
90°(1-P)；
2) in the case where the avoidance type is to turn left, the second pre- measuring angle is calculated according to following formula: θ=-
90°×P；
3) in the case where the avoidance type is to turn right, the second pre- measuring angle is calculated according to following formula: θ=
90°×P。
Wherein, θ is the described second pre- measuring angle, and P is the accuracy.It can be obtained by a kind of possible basis as a result,
The method that avoidance type and accuracy corresponding with avoidance type determine the second pre- measuring angle.
Above-mentioned step 102 and step 103 is when being executed each other without specific sequencing, i.e., in addition to such as Fig. 1
Shown in except execution sequence, step 102 can also be executed again after executing step 103, as long as guarantee step 104 be
It is executed after step 103, and step 105 is executed after step 102, step 103, step 104.
In step 105, target travel angle is determined according to the described first pre- measuring angle and the second pre- measuring angle, with
Realization tracks.After determining the first pre- measuring angle and the second pre- measuring angle, it is thus necessary to determine that one eventually for tracking when
The target travel angle used, for how according to the first pre- measuring angle and the second pre- measuring angle to obtain target line in the disclosure
Into angle method with no restrictions, as long as finally obtained target travel angle can all consider two pre- measuring angles i.e.
It can.
For example, in a kind of possible embodiment, it is described according to the described first pre- measuring angle and the second pre- angle measurement
It spends and determines that target travel angle may include: that the average value of the first pre- measuring angle and the second pre- measuring angle is true
It is set to the target travel angle.
It is described true according to the described first pre- measuring angle and the second pre- measuring angle in alternatively possible embodiment
The travel angle that sets the goal may include: that the described first pre- measuring angle and the second pre- measuring angle are inputted the second predetermined angle net
Network model is to obtain the target travel angle.The second predetermined angle network model can be preparatory trained, Neng Gougen
The output of avoidance network model is preset according to the first pre- measuring angle of the first predetermined angle network model output and corresponding to this
Second pre- measuring angle of avoidance type and accuracy obtains the network model of the target travel angle.
Through the above technical solutions, get the target visual field RGB figure and depth map after, can using RGB figure as
The input of first predetermined angle network model obtains the first pre- measuring angle, and using RGB figure and depth map as default avoidance network
The input of model obtains the accuracy of avoidance type and the avoidance type, and then obtains the second pre- measuring angle, finally combines first
Pre- measuring angle and the second pre- measuring angle determine final target travel angle, and travel angle when enabling to track in this way is more
Add precisely, and therefore the avoidance due to combining the output of avoidance network model is as a result, can also avoid the barrier in travelling route
Hinder object, so that tracking for equipment can be more accurate and intelligent.
Fig. 2 is in a kind of Automatic Track Finding method shown according to disclosure another exemplary embodiment for training described the
The flow chart of the acquisition methods of the training data of one predetermined angle network model.As shown in Fig. 2, the method includes the steps 201
To step 203.
In step 201, the figure of the continuous multiple frames RGB in the target travelling route is obtained according to target travelling route, and
The measurement angle in corresponding horizontal direction is schemed with RGB described in each frame.Measurement angle in the horizontal direction can pass through example
As Inertial Measurement Unit (Inertial measurement unit, IMU) is obtained.When the measurement angle in the horizontal direction by
When the Inertial Measurement Unit obtains, the relative position for obtaining between the equipment of the RGB figure and the Inertial Measurement Unit should
Keep relatively fixed.Equipment for obtaining the RGB figure can be that obtaining for RGB figure can be arbitrarily obtained such as visual sensor
Device is taken to be obtained.
In step 202, the RGB is schemed into the difference of the measurement angle corresponding with neighbor map as the RGB
The angle true value of figure, the neighbor map are the RGB figure got after the RGB figure.In a kind of possible embodiment,
The neighbor map is the first frame RGB figure got after the RGB.When due in step 201, obtaining each frame RGB figure
Corresponding a measurement angle will be got, it therefore, can be by calculating the corresponding measurement angle of each frame RGB figure and at this
The difference of the corresponding measurement angle of a certain frame RGB figure that RGB is got after scheming will corresponding to each frame RGB figure to determine
The angle to be changed, the angle that will change are that the RGB schemes corresponding angle true value.
In step 203, the RGB of the angled true value of the band is schemed to the instruction as the first predetermined angle network model
Practice data.The RGB with angled true value is schemed to the training data as the first predetermined angle network model, it will be able to so that
The first predetermined angle network model can provide after getting any RGB figure in order to closest to after the RGB figure
It is possible that measurement angle, need the angle change that carries out for RGB figure, i.e., provided in the above Automatic Track Finding method
First pre- measuring angle.Thus, it will be able to which training obtains the first predetermined angle network model.
In a kind of possible embodiment, the default avoidance network model includes: semantic segmentation sub-network model, is used
According to RGB figure and depth map output semantic segmentation figure corresponding with RGB figure and the depth map；Avoidance
Sub-network model, for by the RGB figure, the depth map and by the semantic segmentation sub-network model output with the RGB
Scheme the semantic segmentation figure corresponding with the depth map as input, with export the avoidance type and with the avoidance type
The corresponding accuracy.By presetting the judgment basis of increase semantic segmentation figure in avoidance network model at this, enable to
The avoidance type of final output can be more accurate, and due to increasing semantic segmentation figure, additionally it is possible to identify and be not easy really
The short small barrier recognized, to further improve the output accuracy of default avoidance network model.
It can also include such as convolution sub-network in the default avoidance network model in a kind of possible embodiment
Model for carrying out completion processing to the depth map according to the RGB figure, and the RGB is schemed and the depth map after completion
It inputs in the semantic segmentation sub-network model；The semantic segmentation sub-network model be also used to according to the RGB scheme and it is described
Depth map after completion exports semantic segmentation figure corresponding with RGB figure.The depth map usually directly passes through visual sensor
Equal depth map acquisition device is come the depth map that acquires, since the depth map that the device for obtaining depth map is got is usual
Will appear it is sufficiently complete, have cavity, inaccurate problem, therefore after getting the depth map, it is also necessary to the depth
Figure carries out certain optimization processing, such as completion processing.Depth map after handling by completion can be more accurately to figure
In the depth of each object be indicated, increase convolution sub-network model in the default avoidance network model to input
Depth map carries out completion processing, thus can according to after completion depth map and RGB figure generate more accurate semanteme point
Cut figure, so according to after more accurate semantic segmentation figure, the completion depth map and the RGB figure carry out sentencing for avoidance type
It is disconnected, the output accuracy of the default avoidance network model is more further increased in this way.
Fig. 3 is a kind of Automatic Track Finding device shown according to one exemplary embodiment of the disclosure.As shown in figure 3, the dress
Setting includes: to obtain module 10, and the RGB for obtaining the target visual field schemes and depth map；First angle obtains module 20, is used for institute
State RGB and scheme input as the first predetermined angle network model, with obtain that the first predetermined angle network model exports the
One pre- measuring angle；Second angle obtains module 30, for using RGB figure and the depth map as default avoidance network model
Input, to obtain the avoidance type and accuracy corresponding with the avoidance type of the default avoidance network model output；
The second angle obtains module 30 and is also used to, and is determined according to the avoidance type and accuracy corresponding with the avoidance type
Second pre- measuring angle；Target travel angle obtains module 40, for according to the described first pre- measuring angle and the second pre- angle measurement
It spends and determines target travel angle, tracked with realizing.
Through the above technical solutions, get the target visual field RGB figure and depth map after, can using RGB figure as
The input of first predetermined angle network model obtains the first pre- measuring angle, and using RGB figure and depth map as default avoidance network
The input of model obtains the accuracy of avoidance type and the avoidance type, and then obtains the second pre- measuring angle, finally combines first
Pre- measuring angle and the second pre- measuring angle determine final target travel angle, and travel angle when enabling to track in this way is more
Add precisely, and therefore the avoidance due to combining the output of avoidance network model is as a result, can also avoid the barrier in travelling route
Hinder object, so that tracking for equipment can be more accurate and intelligent.
In a kind of possible embodiment, for train the training data of the first predetermined angle network model according to
Following methods obtain:
1) according to target travelling route obtain the continuous multiple frames RGB in the target travelling route figure, and with each frame institute
It states RGB and schemes measurement angle in corresponding horizontal direction；
2) difference for the RGB being schemed the measurement angle corresponding with neighbor map is true as the angle of the RGB figure
Value, the neighbor map are the RGB figure got after the RGB figure；
3) RGB of the angled true value of the band is schemed to the training data as the first predetermined angle network model.
In a kind of possible embodiment, the default avoidance network model includes: semantic segmentation sub-network model, is used
According to RGB figure and depth map output semantic segmentation figure corresponding with RGB figure and the depth map；Avoidance
Sub-network model, for by the RGB figure, the depth map and by the semantic segmentation sub-network model output with the RGB
Scheme the semantic segmentation figure corresponding with the depth map as input, with export the avoidance type and with the avoidance type
The corresponding accuracy.
In a kind of possible embodiment, the avoidance type includes straight trip, turns left and turn right, described to keep away according to
Barrier type and accuracy corresponding with the avoidance type determine that the second pre- measuring angle includes the following contents:
1) in the case where the avoidance type is straight trip, the second pre- measuring angle is calculated according to following formula: θ=
90°(1-P)；
2) in the case where the avoidance type is to turn left, the second pre- measuring angle is calculated according to following formula: θ=-
90°×P；
3) in the case where the avoidance type is to turn right, the second pre- measuring angle is calculated according to following formula: θ=
90°×P。
Wherein, θ is the described second pre- measuring angle, and P is the accuracy.
In a kind of possible embodiment, the target travel angle obtains module 40 and is also used to: by described the
The average value of one pre- measuring angle and the second pre- measuring angle is determined as the target travel angle.
In a kind of possible embodiment, the target travel angle obtains module 40 and is also used to: pre- by described first
Measuring angle and the second pre- measuring angle input the second predetermined angle network model to obtain the target travel angle.
Those skilled in the art can be understood that, for convenience and simplicity of description, only with above-mentioned each function mould
The division progress of block can according to need and for example, in practical application by above-mentioned function distribution by different functional modules
It completes, i.e., the internal structure of device is divided into different functional modules, to complete all or part of the functions described above.
The specific work process of foregoing description functional module, can refer to corresponding processes in the foregoing method embodiment, no longer superfluous herein
It states.
RGB can be schemed after the RGB figure and depth map for getting the target visual field by above-mentioned Automatic Track Finding device
Input as the first predetermined angle network model obtains the first pre- measuring angle, and using RGB figure and depth map as default avoidance
The input of network model obtains the accuracy of avoidance type and the avoidance type, and then obtains the second pre- measuring angle, final to combine
First pre- measuring angle and the second pre- measuring angle determine final target travel angle, angle of travel when enabling to track in this way
Spend it is more accurate, and due to combine avoidance network model output avoidance as a result, therefore can also avoid in travelling route
Barrier so that tracking for equipment can be more accurate and intelligent.
The embodiment of the present disclosure also provides a kind of calculation machine readable storage medium storing program for executing, is stored thereon with computer program, the program quilt
The step of Automatic Track Finding method that above method embodiment provides is realized when processor executes.
The embodiment of the present disclosure also provides a kind of electronic equipment, which can be provided as a kind of server, should
Electronic equipment includes: memory, is stored thereon with computer program；Processor, by executing based on described in the memory
Calculation machine program, with realize above method embodiment provide Automatic Track Finding method the step of.
Fig. 4 is the block diagram of a kind of electronic equipment 400 shown according to an exemplary embodiment.As shown in figure 4, the electronics is set
Standby 400 may include: processor 401, memory 402.The electronic equipment 400 can also include multimedia component 403, input/
Export one or more of (I/O) interface 404 and communication component 405.
Wherein, processor 401 is used to control the integrated operation of the electronic equipment 400, to complete above-mentioned Automatic Track Finding side
All or part of the steps in method.Memory 402 is for storing various types of data to support the behaviour in the electronic equipment 400
To make, these data for example may include the instruction of any application or method for operating on the electronic equipment 400, with
And the relevant data of application program, such as contact data, the message of transmitting-receiving, picture, audio, video etc..The memory 402
It can be realized by any kind of volatibility or non-volatile memory device or their combination, such as static random-access is deposited
Reservoir (Static Random Access Memory, abbreviation SRAM), electrically erasable programmable read-only memory
(Electrically Erasable Programmable Read-Only Memory, abbreviation EEPROM), erasable programmable
Read-only memory (Erasable Programmable Read-Only Memory, abbreviation EPROM), programmable read only memory
(Programmable Read-Only Memory, abbreviation PROM), and read-only memory (Read-Only Memory, referred to as
ROM), magnetic memory, flash memory, disk or CD.Multimedia component 403 may include screen and audio component.Wherein
Screen for example can be touch screen, and audio component is used for output and/or input audio signal.For example, audio component may include
One microphone, microphone is for receiving external audio signal.The received audio signal can be further stored in storage
Device 402 is sent by communication component 405.Audio component further includes at least one loudspeaker, is used for output audio signal.I/O
Interface 404 provides interface between processor 401 and other interface modules, other above-mentioned interface modules can be keyboard, mouse,
Button etc..These buttons can be virtual push button or entity button.Communication component 405 is for the electronic equipment 400 and other
Wired or wireless communication is carried out between equipment.Wireless communication, such as Wi-Fi, bluetooth, near-field communication (Near Field
Communication, abbreviation NFC), 2G, 3G, 4G, NB-IOT, eMTC or other 5G etc. or they one or more of
Combination, it is not limited here.Therefore the corresponding communication component 405 may include: Wi-Fi module, bluetooth module, NFC mould
Block etc..
In one exemplary embodiment, electronic equipment 400 can be by one or more application specific integrated circuit
(Application Specific Integrated Circuit, abbreviation ASIC), digital signal processor (Digital
Signal Processor, abbreviation DSP), digital signal processing appts (Digital Signal Processing Device,
Abbreviation DSPD), programmable logic device (Programmable Logic Device, abbreviation PLD), field programmable gate array
(Field Programmable Gate Array, abbreviation FPGA), controller, microcontroller, microprocessor or other electronics member
Part is realized, for executing above-mentioned Automatic Track Finding method.
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned Automatic Track Finding method is realized when program instruction is executed by processor.For example, the computer readable storage medium
It can be the above-mentioned memory 402 including program instruction, above procedure instruction can be executed by the processor 401 of electronic equipment 400
To complete above-mentioned Automatic Track Finding method.
Fig. 5 is the block diagram of a kind of electronic equipment 500 shown according to an exemplary embodiment.For example, electronic equipment 500 can
To be provided as a server.Referring to Fig. 5, electronic equipment 500 includes processor 522, and quantity can be one or more, with
And memory 532, for storing the computer program that can be executed by processor 522.The computer program stored in memory 532
May include it is one or more each correspond to one group of instruction module.In addition, processor 522 can be configured as
The computer program is executed, to execute above-mentioned Automatic Track Finding method.
In addition, electronic equipment 500 can also include power supply module 526 and communication component 550, which can be with
It is configured as executing the power management of electronic equipment 500, which, which can be configured as, realizes electronic equipment 500
Communication, for example, wired or wireless communication.In addition, the electronic equipment 500 can also include input/output (I/O) interface 558.Electricity
Sub- equipment 500 can be operated based on the operating system for being stored in memory 532, such as Windows ServerTM, Mac OS
XTM, UnixTM, LinuxTM etc..
In a further exemplary embodiment, a kind of computer readable storage medium including program instruction is additionally provided, it should
The step of above-mentioned Automatic Track Finding method is realized when program instruction is executed by processor.For example, the computer readable storage medium
It can be the above-mentioned memory 532 including program instruction, above procedure instruction can be executed by the processor 522 of electronic equipment 500
To complete above-mentioned Automatic Track Finding method.
The preferred embodiment of the disclosure is described in detail in conjunction with attached drawing above, still, the disclosure is not limited to above-mentioned reality
The detail in mode is applied, in the range of the technology design of the disclosure, a variety of letters can be carried out to the technical solution of the disclosure
Monotropic type, these simple variants belong to the protection scope of the disclosure.
It is further to note that specific technical features described in the above specific embodiments, in not lance
In the case where shield, it can be combined in any appropriate way.In order to avoid unnecessary repetition, the disclosure to it is various can
No further explanation will be given for the combination of energy.
In addition, any combination can also be carried out between a variety of different embodiments of the disclosure, as long as it is without prejudice to originally
Disclosed thought equally should be considered as disclosure disclosure of that.
Claims (10)
1. a kind of method of Automatic Track Finding, which is characterized in that the described method includes:
Obtain the RGB figure and depth map in the target visual field；
The RGB is schemed into the input as the first predetermined angle network model, to obtain the first predetermined angle network model
First pre- measuring angle of output；
Using RGB figure and the depth map as the input of default avoidance network model, to obtain the default avoidance network
The avoidance type and accuracy corresponding with the avoidance type of model output；
The second pre- measuring angle is determined according to the avoidance type and accuracy corresponding with the avoidance type；
Target travel angle is determined according to the described first pre- measuring angle and the second pre- measuring angle, is tracked with realizing.
2. the method according to claim 1, wherein for training the instruction of the first predetermined angle network model
Practice data to obtain in accordance with the following methods:
The figure of the continuous multiple frames RGB in the target travelling route is obtained according to target travelling route, and is schemed with RGB described in each frame
Measurement angle in corresponding horizontal direction；
It is described using the difference of the RGB figure measurement angle corresponding with neighbor map as the angle true value of the RGB figure
Neighbor map is the RGB figure got after the RGB figure；
The RGB of the angled true value of the band is schemed into the training data as the first predetermined angle network model.
3. the method according to claim 1, wherein the default avoidance network model includes:
Semantic segmentation sub-network model, for according to RGB figure and depth map output and RGB figure and the depth
Scheme corresponding semantic segmentation figure；
Avoidance sub-network model, for by RGB figure, the depth map and by semantic segmentation sub-network model output
The semantic segmentation figure corresponding with RGB figure and the depth map as inputting, with export the avoidance type and with it is described
The corresponding accuracy of avoidance type.
4. method according to claim 3, which is characterized in that the avoidance type includes straight trip, turns left and turn right, and described
Determine that the second pre- measuring angle includes: according to the avoidance type and accuracy corresponding with the avoidance type
In the case where the avoidance type is straight trip, the second pre- measuring angle is calculated according to following formula:
θ=90 ° (1-P)；
In the case where the avoidance type is to turn left, the second pre- measuring angle is calculated according to following formula:
θ=- 90 ° × P；
In the case where the avoidance type is to turn right, the second pre- measuring angle is calculated according to following formula:
θ=90 ° × P；
Wherein, θ is the described second pre- measuring angle, and P is the accuracy.
5. method described in any claim in -4 according to claim 1, which is characterized in that described according to first prediction
Angle and the second pre- measuring angle determine that target travel angle includes:
The average value of the first pre- measuring angle and the second pre- measuring angle is determined as the target travel angle.
6. method described in any claim in -4 according to claim 1, which is characterized in that described according to first prediction
Angle and the second pre- measuring angle determine that target travel angle includes:
Described first pre- measuring angle and the second pre- measuring angle are inputted into the second predetermined angle network model to obtain the mesh
Mark travel angle.
7. a kind of Automatic Track Finding device, which is characterized in that described device includes:
Module is obtained, the RGB for obtaining the target visual field schemes and depth map；
First angle obtains module, described to obtain for the RGB to be schemed to the input as the first predetermined angle network model
First pre- measuring angle of the first predetermined angle network model output；
Second angle obtain module, for using the RGB figure and the depth map as preset avoidance network model input, with
Obtain the avoidance type and accuracy corresponding with the avoidance type of the default avoidance network model output；
The second angle obtains module and is also used to, true according to the avoidance type and accuracy corresponding with the avoidance type
Fixed second pre- measuring angle；
Target travel angle obtains module, for determining target line according to the described first pre- measuring angle and the second pre- measuring angle
Into angle, tracked with realizing.
8. device according to claim 7, which is characterized in that the default avoidance network model includes:
Semantic segmentation sub-network model, for according to RGB figure and depth map output and RGB figure and the depth
Scheme corresponding semantic segmentation figure；
Avoidance sub-network model, for by RGB figure, the depth map and by semantic segmentation sub-network model output
The semantic segmentation figure corresponding with RGB figure and the depth map as inputting, with export the avoidance type and with it is described
The corresponding accuracy of avoidance type.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of any one of claim 1-6 the method is realized when row.
10. a kind of electronic equipment characterized by comprising
Memory is stored thereon with computer program；
Processor, for executing the computer program in the memory, to realize described in any one of claim 1-6
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910407609.3A CN110244710B (en) | 2019-05-16 | 2019-05-16 | Automatic tracing method, device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910407609.3A CN110244710B (en) | 2019-05-16 | 2019-05-16 | Automatic tracing method, device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110244710A true CN110244710A (en) | 2019-09-17 |
| CN110244710B CN110244710B (en) | 2022-05-31 |
Family
ID=67884114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910407609.3A Active CN110244710B (en) | 2019-05-16 | 2019-05-16 | Automatic tracing method, device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110244710B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110954122A (en) * | 2019-12-09 | 2020-04-03 | 东风商用车有限公司 | Automatic driving track generation method under high-speed scene |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102059978A (en) * | 2009-11-16 | 2011-05-18 | 财团法人工业技术研究院 | Assisted method and system for driving |
| CN106384353A (en) * | 2016-09-12 | 2017-02-08 | 佛山市南海区广工大数控装备协同创新研究院 | Target positioning method based on RGBD |
| US20170368686A1 (en) * | 2016-06-28 | 2017-12-28 | Qihan Technology Co., Ltd. | Method and device for automatic obstacle avoidance of robot |
| CN107818333A (en) * | 2017-09-29 | 2018-03-20 | 爱极智（苏州）机器人科技有限公司 | Robot obstacle-avoiding action learning and Target Searching Method based on depth belief network |
| CN108510062A (en) * | 2018-03-29 | 2018-09-07 | 东南大学 | A kind of robot irregular object crawl pose rapid detection method based on concatenated convolutional neural network |
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
| CN109389621A (en) * | 2018-09-11 | 2019-02-26 | 淮阴工学院 | RGB-D method for tracking target based on the fusion of multi-mode depth characteristic |
| CN109658418A (en) * | 2018-10-31 | 2019-04-19 | 百度在线网络技术（北京）有限公司 | Learning method, device and the electronic equipment of scene structure |
-
2019
- 2019-05-16 CN CN201910407609.3A patent/CN110244710B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102059978A (en) * | 2009-11-16 | 2011-05-18 | 财团法人工业技术研究院 | Assisted method and system for driving |
| US20170368686A1 (en) * | 2016-06-28 | 2017-12-28 | Qihan Technology Co., Ltd. | Method and device for automatic obstacle avoidance of robot |
| CN106384353A (en) * | 2016-09-12 | 2017-02-08 | 佛山市南海区广工大数控装备协同创新研究院 | Target positioning method based on RGBD |
| CN107818333A (en) * | 2017-09-29 | 2018-03-20 | 爱极智（苏州）机器人科技有限公司 | Robot obstacle-avoiding action learning and Target Searching Method based on depth belief network |
| CN108510062A (en) * | 2018-03-29 | 2018-09-07 | 东南大学 | A kind of robot irregular object crawl pose rapid detection method based on concatenated convolutional neural network |
| CN109074668A (en) * | 2018-08-02 | 2018-12-21 | 深圳前海达闼云端智能科技有限公司 | Method for path navigation, relevant apparatus and computer readable storage medium |
| CN109389621A (en) * | 2018-09-11 | 2019-02-26 | 淮阴工学院 | RGB-D method for tracking target based on the fusion of multi-mode depth characteristic |
| CN109658418A (en) * | 2018-10-31 | 2019-04-19 | 百度在线网络技术（北京）有限公司 | Learning method, device and the electronic equipment of scene structure |
Non-Patent Citations (2)
| Title |
|---|
| KRISHNAM GUPTA等: "MergeNet: A Deep Net Architecture for Small Obstacle Discovery", 《2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)》 * |
| 白辰甲: "基于计算机视觉和深度学习的自动驾驶方法研究", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110954122A (en) * | 2019-12-09 | 2020-04-03 | 东风商用车有限公司 | Automatic driving track generation method under high-speed scene |
| CN110954122B (en) * | 2019-12-09 | 2021-07-06 | 东风商用车有限公司 | Automatic driving track generation method under high-speed scene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110244710B (en) | 2022-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109085608A (en) | Obstacles around the vehicle detection method and device | |
| JP2020077372A (en) | Data collection method and system therefor | |
| WO2018068771A1 (en) | Target tracking method and system, electronic device, and computer storage medium | |
| JP2020132154A (en) | Driving route planning method, device, vehicle, and program | |
| US9081384B2 (en) | Autonomous electronic apparatus and navigation method thereof | |
| CN105116994A (en) | Intelligent robot tracking method and tracking device based on artificial intelligence | |
| CN107957266A (en) | Localization method, device and storage medium | |
| CN111338361A (en) | Obstacle avoidance method, device, equipment and medium for low-speed unmanned vehicle | |
| US11282164B2 (en) | Depth-guided video inpainting for autonomous driving | |
| CN110245552A (en) | Interaction processing method, device, equipment and the client of vehicle damage image taking | |
| WO2021135813A1 (en) | Robot joint mapping method and device, and computer-readable storage medium | |
| CN110245567A (en) | Barrier-avoiding method, device, storage medium and electronic equipment | |
| CN113907663B (en) | Obstacle map construction method, cleaning robot, and storage medium | |
| WO2020010841A1 (en) | Autonomous vacuum cleaner positioning method and device employing gyroscope calibration based on visual loop closure detection | |
| CN108072385A (en) | Space coordinates localization method, device and the electronic equipment of mobile target | |
| CN107782304A (en) | Mobile robot positioning method and device, mobile robot and storage medium | |
| CN108121347A (en) | For the method, apparatus and electronic equipment of control device movement | |
| CN111192303B (en) | Point cloud data processing method and device | |
| CN115471731A (en) | Image processing method, image processing apparatus, storage medium, and device | |
| CN110244710A (en) | Automatic Track Finding method, apparatus, storage medium and electronic equipment | |
| CN114972485A (en) | Positioning accuracy testing method, positioning accuracy testing apparatus, storage medium, and program product | |
| CN113158779B (en) | Walking method, walking device and computer storage medium | |
| WO2021027966A1 (en) | Traveling method, traveling device, and storage medium | |
| CN111899277A (en) | Moving object detection method and device, storage medium and electronic device | |
| CN116952277A (en) | Potential position estimation method and system based on artificial intelligence and behavior characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210301 Address after: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant after: Dalu Robot Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: Shenzhen Qianhaida Yunyun Intelligent Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 201111 Building 8, No. 207, Zhongqing Road, Minhang District, Shanghai Applicant after: Dayu robot Co.,Ltd. Address before: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant before: Dalu Robot Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |