CN110006435A - A kind of Intelligent Mobile Robot vision navigation system method based on residual error network - Google Patents
A kind of Intelligent Mobile Robot vision navigation system method based on residual error network Download PDFInfo
- Publication number
- CN110006435A CN110006435A CN201910329956.9A CN201910329956A CN110006435A CN 110006435 A CN110006435 A CN 110006435A CN 201910329956 A CN201910329956 A CN 201910329956A CN 110006435 A CN110006435 A CN 110006435A
- Authority
- CN
- China
- Prior art keywords
- layers
- semantic segmentation
- network
- output
- built
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/2163—Partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The Intelligent Mobile Robot vision navigation system method based on residual error network that the invention discloses a kind of, substation's road scene image including S1, acquisition crusing robot inspection, road scene image is screened, and data set mark is manually carried out using semantic segmentation model-aided；S2, semantic segmentation model of the building based on residual error network；S3, the semantic segmentation model built in step S2 is trained using SGD method, and saves the semantic segmentation model of last time iteration acquisition；S4, the Generalization Capability for determining semantic segmentation model input new inspection road scene image, export semantic segmentation as a result, the object type distribution in semantic segmentation result in area-of-interest is transmitted to crusing robot navigation system.
Description
Technical field
The invention belongs to the technical fields of crusing robot navigation, and in particular to a kind of substation based on residual error network patrols
Examine robot vision auxiliary navigation method.
Background technique
With the rapid development of electric power network technique, gradually substitution is artificial for intelligent, automation power product.Wherein become
Power station crusing robot is usually the Important Platform for realizing various types of patrol tasks, navigation positioning system decision it is reliable
Property is one of important link.During robot inspection, the unstructuredness of outdoor substation's road scene, randomness
And the features such as complexity, makes the navigation system of crusing robot there are still certain when decision goes out the inspection route of robot
Problem.Such as the road hole occurred in inspection road environment, ponding etc., which may result in laser radar system, can not obtain current road conditions
Feedback information can not be made so as to cause robot next step traveling strategy；On relatively narrow inspection road, roadside is miscellaneous
The objects such as grass, stone may make Algorithms of Robots Navigation System be regarded as the decision that the stopping that barrier is made moves ahead or detours,
Eventually leading to robot, there may be the risks fallen.Therefore, during inspection, if crusing robot can be made to be similar to
The intuitivism apprehension ability of human vision assists the navigation system of crusing robot to understand and analyze itself local environment,
And the contextual data that provides of navigation data combination vision means for obtaining script crusing robot sensor, then crusing robot
Navigation system more decision factors can be comprehensively considered when decision goes out polling path, thus be conducive to cook up more rationally
Inspection route.
In existing Intelligent Mobile Robot airmanship, seldom occur combining the navigation system of the semantic information of image
System, and the case where system of laser radar navigation is in the presence of the type that can not identify detection barrier is nowadays relied on, certain
The feedback of radar signal can not even be received in region, this be for the decision process of crusing robot navigation system it is unfavorable,
Therefore it provides it is to be of great significance that a kind of intuitive semantic understanding information secondary navigation system, which makes reasonable decision path,
's.
With the rapid development of artificial intelligence technology in recent years, the various products for covering depth learning technology emerge one after another,
Such as the automated driving system and fatigue detecting system of automobile, the intelligent recommendation system of service robot, speech recognition system and machine
The application of the technologies such as device translation system.Depth learning technology has shown good study with ability to express and with extensive
Application scenarios, such as divide field in image, semantic, depth learning technology shown the excellent of remote ultra-traditional semantic segmentation technology
More performance.This becomes the semantic segmentation for carrying out image using road scene of the depth learning technology to substation's complexity can
Can, to set up the intuitivism apprehension ability that crusing robot is similar to human vision, itself navigation system decision is assisted to go out more
For reasonable navigation routine.
At present the common airmanship of crusing robot have magnetic navigation, rail mounted navigation, the navigation based on machine vision and
The technologies such as laser radar navigation.If document [1] proposes a kind of magnetic navigation technology for crusing robot, technology navigation is fixed
Position precision is high, it is strong to shoulder interference performance, but need to lay in advance magnetic stripe for inspection road, and there are at high cost, later maintenance inconvenience
The problems such as.Document [2] proposes a kind of rail mounted crusing robot navigation system, which need to be according to power equipment position distribution
Situation designs corresponding inspection track, is mostly used for indoor scene, and the outdoor scene in face of complexity can not also preferably meet
Its application demand.Document [3] [4] proposes the airmanship based on machine vision and navigates for Intelligent Mobile Robot
System, the technology identify that inspection mark and vision measurement technology are navigated and positioned by vision algorithm, and location navigation is reliable
Property dependent on the precision and robustness of vision algorithm taken, and be laid with accessory ID also bring along additional economic cost and after
Phase maintenance issues.With the rapid development of laser radar airmanship in recent years, the precision of the navigation system based on laser radar and
Reliability is increasingly promoted, and the navigator fix of the crusing robot of Foreign & Domestic Substations mostly uses laser radar at present, such as text
Offer [5] propose it is a kind of crusing robot is positioned and is navigated using two-dimensional laser radar, document [6] propose a kind of base
In the airmanship of Beidou RTK and laser radar, this technology is not required to that inspection route is transformed again, and there is good environment to adapt to
Property the advantages that, but laser radar navigation can not judge the type of barrier detected, under the conditions of certain complicated inspections, power transformation
It is possible that some weeds, the puddles of water, rubble and fence etc influence the reliability of navigation system decision in road scene of standing.
To sum up, the prior art has the disadvantage that or insufficient:
1, the problems such as magnetic navigation need to expend a large amount of resource and be laid with related inspection road, inconvenient there are later maintenance；
2, orbital navigation is suitable for indoor some equipment routing inspections, and outdoor complex environment increases the difficulty for being laid with track,
And it may be because that the reasons such as weather affect to track；
3, the navigator fix technology of view-based access control model needs to be laid with additional accessory ID, and navigator fix reliability is depended on and adopted
, there is later maintenance in the precision and robustness of the vision algorithm taken；
4, laser radar navigation can not judge the type of barrier detected, for may in substation's road scene
The object for some weeds, the puddles of water, stone crusher fence etc occur will affect the reliability of navigation system decision.
Bibliography is as follows:
[1] Wang Jinchai Intelligent Mobile Robot magnetic navigation systematical design idea [D] Southwest Jiaotong University, 2015.
[2] Pei Wenliang, Zhang Shusheng, Cen Qiang, Rao Yi rail mounted crusing robot system design and apply [J] coal mining machine
Tool, 2016,37 (06): 142-144.
[3] Intelligent Mobile Robot navigator fix technology [D] Anhui University of Science and Technology of the Shi Zehua based on machine vision,
2018.
[4] Fan Shaosheng, Zhang Shaohai, Hu Wentao wait the Intelligent Mobile Robot navigator fix control method of view-based access control model
The Hunan [P]: CN105700532A, 2016-06-22.
[5] Xiao Peng, Wang Haipeng, Li Rong wait Intelligent Mobile Robot location navigation two-dimensional laser Radar Calibration device
And the Shandong method [P]: CN106556826A, 2017-04-05.
[6] Peng Daogang, Qi Erjiang, Xia Fei, wait crusing robot navigation system of the based on RTK Beidou and laser radar and
The Shanghai method [P]: CN107817509A, 2018-03-20.
Summary of the invention
It is an object of the invention to be directed to above-mentioned deficiency in the prior art, a kind of substation based on residual error network is provided
Crusing robot vision navigation system method, to solve or improve the above problem.
In order to achieve the above objectives, the technical solution adopted by the present invention is that:
A kind of Intelligent Mobile Robot vision navigation system method based on residual error network comprising:
S1, the substation's road scene data for acquiring crusing robot inspection, screen road scene data, and adopt
Data set mark is carried out with semantic parted pattern indirect labor；
S2, semantic segmentation model of the building based on residual error network；
S3, the semantic segmentation model built in step S2 is trained using SGD method, and saves last time iteration
The semantic segmentation network model of acquisition；
The new inspection road scene image of S4, input exports semantic segmentation as a result, by language to the semantic segmentation model saved
Object type distribution in adopted segmentation result in area-of-interest is transmitted to crusing robot navigation system.
Further, in step S1 specifically includes the following steps:
S1.1, the substation's road scene video for acquiring crusing robot inspection, extract each frame of video, remove repetition
Higher video frame is spent, part raw sample data is filtered out and is manually marked；
S1.2, data set mark is manually carried out using semantic segmentation model-aided.
Further, the specific steps packet of data set mark is manually carried out in step S1.2 using semantic segmentation model-aided
It includes:
S1.2.1, artificial mark part raw sample data；
S1.2.2, the raw sample data training semantic segmentation model marked is utilized；
S1.2.3, semantic segmentation is carried out to the remaining sample in raw sample data using trained semantic segmentation model,
And export segmentation result；
S1.2.4, the sample of segmentation result difference is marked again, finally the segmentation result after readjustment is added
The data set marked.
Further, the specific steps of semantic segmentation network model of the building based on residual error network include: in step S2
S2.1, the coding for building semantic segmentation model, decoding, multitiered network fused layer three parts frame；
S2.2, according to the network frame in step S2.1, building is directed to the network structure of substation data collection.
Further, the network structure for being directed to substation data collection, including coding network implementation steps are constructed in step S2.2:
Network input layer is built, input layer input includes substation's road scene image and corresponding mark label data；
Convolutional layer conv1 is built, convolution operation is carried out to input picture；
Pond layer pool1 is built, 2 × down-sampling is carried out to output in conv1；
Residual block Res2 is built, is carried out continuously convolution twice in residual block Res2,2 × down-sampling is defeated after continuous convolution
Fusion pool1 is operated with Eltwise out and exports result；
Residual block Res3 is built, is carried out continuously convolution twice in residual block Res3,2 × down-sampling is defeated after continuous convolution
Fusion Res2 is operated with Eltwise out and exports result；
Residual block Res4 is built, is carried out continuously convolution twice in residual block Res4,2 × down-sampling is defeated after continuous convolution
Fusion Res3 is operated with Eltwise out and exports result；
Residual block Res5 is built, is carried out continuously convolution twice in residual block Res5,2 × down-sampling is defeated after continuous convolution
Fusion Res4 is operated with Eltwise out and exports result；
Pond layer pool5 is built, Res5 is exported and carries out 2 × down-sampling；
Convolutional layer Conv6 is built, output channel number is consistent with classification is marked in substation's road scene data set.
Further, building further includes decoding and multitiered network for the network structure of substation data collection in step S2.2
Converged network layer implementation steps:
Build Upscore2_1 layers, Upscore2_1 layers operated with Deconvolution conv6 output is carried out 2 × on
Sampling；
Build score4 layers, score4 layers Res4 output channel number is compressed to it is consistent with conv6 output channel number；
Crop4 layers are built, Crop4 layers are cut to score4 layers of output unanimously with Upscore2_1 layers of Output Size；
Fuse4 layers are built, Fuse4 layers operate fusion score4 and conv6 with Eltwise and export；
Upscore2_2 layers are built, fuse4 is carried out 2 × up-sampling by Upscore2_2 layers；
Build score3 layers, score3 layers Res3 output channel number is compressed to it is consistent with Fuse4 layers of output channel number；
Crop3 layers are built, Crop3 layers consistent with score3 layers of Output Size clipped value by Upscore2_2 layers；
Fuse3 layers are built, Fuse3 layers are merged Upscore2_2 layers with score3 layers of output with Elewise operation；
Upscore8 layers are built, Upscore8 layers are operated with Deconvolution by Fuse3 output 8 × up-sampling of progress；
Crop8 layers are built, Crop8 layers are cut to Upscore8 layers of output and the same size of input picture；
Classification layer specific implementation step are as follows: build classification layer, Crop8 layers of input are carried out using softmaxWithloss
Classify pixel-by-pixel.
Further, semantic segmentation model is trained using SGD method in step S3, and saves last time iteration and obtains
The specific steps of semantic segmentation model include:
S3.1, data set is divided into training set, verifying collection and test set；
S3.2, setting network optimized approach, select SGD to optimize；
S3.3, fixed Study rate parameter；
S3.4, setting moment of momentum parameter；
S3.5, setting weight attenuation parameter；
S3.6, setting Optimized Iterative step-length；
S3.7, random initializtion network of relation layer parameter；
S3.8, when network layer softmaxWithloss output loss value level off to balance or model verifying precision it is basic
Deconditioning when remaining unchanged；
S3.9, the semantic segmentation network model that last time iteration obtains is saved.
Preferably, step S4 inputs new inspection road scene image to the semantic segmentation model saved, and output is semantic to divide
It cuts as a result, the object type distribution in semantic segmentation result in area-of-interest is transmitted to crusing robot navigation system
Specific steps include:
S4.1, input test collection to the semantic segmentation network model saved；
S4.2, the corresponding semantic segmentation of forward calculation output test set is carried out as a result, determining the Generalization Capability of network；
The new substation's road scene image of S4.3, input exports semantic segmentation result figure, selection to semantic segmentation model
Area-of-interest；
S4.4, object type distribution situation in area-of-interest is determined；
S4.5, connection object category distribution state to crusing robot navigation system.
Intelligent Mobile Robot vision navigation system method provided by the invention based on residual error network has with following
Beneficial effect:
The present invention constructs and trains semantic segmentation model, most by carrying out semantic segmentation to substation's road scene image
Object type distributed intelligence in semantic segmentation result figure in area-of-interest is transmitted to the navigation system of crusing robot afterwards；
The present invention can directly provide some scene types and object distribution feelings in current crusing robot visual field using semantic segmentation figure
Condition, so that its navigation system decision be assisted to go out more reasonable driving path.
Detailed description of the invention
Fig. 1 is the flow chart of the Intelligent Mobile Robot vision navigation system method based on residual error network.
Fig. 2 is in the semantic segmentation model of the Intelligent Mobile Robot vision navigation system method based on residual error network
Residual block.
Fig. 3 is the Intelligent Mobile Robot vision navigation system method semantic segmentation network structure based on residual error network.
Specific embodiment
A specific embodiment of the invention is described below, in order to facilitate understanding by those skilled in the art this hair
It is bright, it should be apparent that the present invention is not limited to the ranges of specific embodiment, for those skilled in the art,
As long as various change is in the spirit and scope of the present invention that the attached claims limit and determine, these variations are aobvious and easy
See, all are using the innovation and creation of present inventive concept in the column of protection.
According to one embodiment of the application, with reference to Fig. 1, the Intelligent Mobile Robot based on residual error network of this programme
Vision navigation system method, comprising:
S1, the substation's road scene data for acquiring crusing robot inspection, screen road scene data, and adopt
Data set mark is carried out with semantic parted pattern indirect labor；
S2, semantic segmentation network model of the building based on residual error network；
S3, the semantic segmentation network model built in step S2 is trained using SGD method, and saves last time
The semantic segmentation network model that iteration obtains；
The new inspection road scene image of S4, input exports semantic segmentation as a result, by language to the semantic segmentation model saved
Object type distribution in adopted segmentation result in area-of-interest is transmitted to crusing robot navigation system.
Above-mentioned steps are described in detail below
S1, the substation's road scene data for acquiring crusing robot inspection, screen road scene data, and adopt
Data set mark is carried out with semantic parted pattern indirect labor；
S1.1, acquisition substation's road scene data, filter out part raw sample data and are manually marked；
Substation's road scene image is obtained to be acquired by the Visible Light Camera that machine itself is built.In collection process
In, the mounting height and angle of fixed camera, crusing robot are maked an inspection tour according to normal path of making an inspection tour, and camera shooting is related
The road scene video of inspection extracts each frame of video, removes the higher video frame of multiplicity, it is original that finishing screen selects part
Sample data is manually marked, and mark object includes road, grass, stone, six class of fence, puddle and background.
S1.2, data set mark is manually carried out using semantic segmentation model-aided, specifically includes the following steps:
S1.2.1, artificial mark part raw sample data；
S1.2.2, the raw sample data training semantic segmentation model marked is utilized；
S1.2.3, semantic segmentation is carried out to the remaining sample in raw sample data using trained semantic segmentation model,
And export segmentation result；
S1.2.4, the sample of segmentation result difference is marked again, finally the segmentation result after readjustment is added
The data set marked.
S2, semantic segmentation network model of the building based on residual error network；
S2.1, the coding for building semantic segmentation model, decoding, multitiered network fused layer three parts frame.
With reference to Fig. 3, three parts frame is merged comprising coding, decoding, multitiered network in network；
Coding network is basic residual error network, is responsible for taking out the object information in substation's road scene image；
Decoding network is some up-sampling layers, and major function is to up-sample to coding network output, to realize end
Substation's road scene image semantic segmentation of opposite end；
Multitiered network fused layer is mainly jumper connection structure, which is to be exported in conjunction with depth layer network to promote network
Semantic segmentation precision.
It include tetra- residual blocks of Res2, Res3, Res4 and Res5 in coding network, each residual block includes two volumes
Product network layer, the output of first convolutional network layer input second convolutional network layer of superposition, superimposed result is as next
The input of a residual block.
In decoding network, the variation Upscore2 and two kinds of up-sampling of Upscore8, two types respectively correspond 2 ×
Up-sampling and 8 × up-sampling, up-sampling operation layer use Deconvolution.
In multitiered network fusion results, which is merged twice, and first layer is fused to Res4 output result and compiles
The fusion of code network output is fused to Res3 output result for the second time and merges the output after carrying out 2 × up-sampling with first time
It is merged, it is in this course, respectively that its is defeated using 1 × 1 convolutional layer before Res3 is merged with the output of Res4
Port number is compressed to the classification number (substation's road scene is 6 classes) of training dataset out, finally in decoding network, will carry out
The output merged twice carries out the straight original input picture size of 8 × up-sampling, using softmaxWithloss to each of which
Pixel class is classified.
S2.2, according to the network frame in step S2.1, building is directed to the network structure of substation's road scene data set,
It specifically includes coding network implementation steps:
Network input layer is built, input layer input includes substation's road scene image and corresponding mark label data；
Convolutional layer conv1 is built, convolution operation is carried out to input picture；
Pond layer pool1 is built, 2 × down-sampling is carried out to output in conv1；
Residual block Res2 is built, is carried out continuously convolution twice in residual block Res2,2 × down-sampling is defeated after continuous convolution
Fusion pool1 is operated with Eltwise out and exports result；
Residual block Res3 is built, is carried out continuously convolution twice in residual block Res3,2 × down-sampling is defeated after continuous convolution
Fusion Res2 is operated with Eltwise out and exports result；
Residual block Res4 is built, is carried out continuously convolution twice in residual block Res4,2 × down-sampling is defeated after continuous convolution
Fusion Res3 is operated with Eltwise out and exports result；
Residual block Res5 is built, is carried out continuously convolution twice in residual block Res5,2 × down-sampling is defeated after continuous convolution
Fusion Res4 is operated with Eltwise out and exports result；
Pond layer pool5 is built, Res5 is exported and carries out 2 × down-sampling；
Convolutional layer Conv6 is built, output channel number is consistent with classification is marked in substation's road scene data set.
Further include decoding and multitiered network converged network layer implementation steps:
Build Upscore2_1 layers, Upscore2_1 layers operated with Deconvolution conv6 output is carried out 2 × on
Sampling；
Build score4 layers, score4 layers Res4 output channel number is compressed to it is consistent with conv6 output channel number；
Crop4 layers are built, Crop4 layers are cut to score4 layers of output unanimously with Upscore2_1 layers of Output Size；
Fuse4 layers are built, Fuse4 layers operate fusion score4 and conv6 with Eltwise and export；
Upscore2_2 layers are built, fuse4 is carried out 2 × up-sampling by Upscore2_2 layers；
Build score3 layers, score3 layers Res3 output channel number is compressed to it is consistent with Fuse4 layers of output channel number；
Crop3 layers are built, Crop3 layers consistent with score3 layers of Output Size clipped value by Upscore2_2 layers；
Fuse3 layers are built, Fuse3 layers are merged Upscore2_2 layers with score3 layers of output with Elewise operation；
Upscore8 layers are built, Upscore8 layers are operated with Deconvolution by Fuse3 output 8 × up-sampling of progress；
Crop8 layers are built, Crop8 layers are cut to Upscore8 layers of output and the same size of input picture；
Classification layer specific implementation step are as follows: build classification layer, Crop8 layers of input are carried out using softmaxWithloss
Classify pixel-by-pixel.
S3, the semantic segmentation model built in step S2 is trained using SGD method, and saves last time iteration
The semantic segmentation model of acquisition, the specific steps are that:
S3.1, data set is divided into training set, verifying collection and test set；Wherein training set is used to be fitted the semanteme point built
Network parameter is cut, verifying collection obtains the segmentation performance of model in the training process for evaluating to instruct phase in its training process
Close the fitting direction of parameter；The model finally saved is tested using test set to detect the model in face of new road
Generalization Capability when scene image；By training set and verifying collection input network input layer；
S3.2, setting network optimized approach, select SGD to optimize；
S3.3, fixed Study rate parameter；
S3.4, setting moment of momentum parameter；
S3.5, setting weight attenuation parameter；
S3.6, setting Optimized Iterative step-length；
S3.7, random initializtion network of relation layer parameter；
S3.8, when network layer softmaxWithloss output loss value level off to balance or model verifying precision it is basic
Deconditioning when remaining unchanged；
S3.9, the semantic segmentation model that last time iteration obtains is saved.
The new inspection road scene image of S4, input exports semantic segmentation as a result, by language to the semantic segmentation model saved
Object type distribution in adopted segmentation result in area-of-interest is transmitted to crusing robot navigation system, specific steps
Are as follows:
S4.1, input test collection to the semantic segmentation network model saved；
S4.2, the corresponding semantic segmentation of forward calculation output test set is carried out as a result, determining the Generalization Capability of network；
The Generalization Capability of semantic segmentation model directly influences the reliability of the vision auxiliary information of output in step S3, is
Determine that the semantic segmentation model finally saved in step S3 has good Generalization Capability to new substation's road scene image,
The model is tested with the test set of data set in step S3 in step 4.2, using pixel precision (Pixel
Accuracy, PA), mean accuracy (MA), hand over and than (Mean Intersection over Union, Mean IU) and frequency
Power is handed over and than (Frequency Weighted Intersection over Union, Fw IU) four precision indexs to its property
It can be carried out evaluation.Wherein PA, MA, Mean IU and Fw IU precision index is defined as:
PA=∑inii/∑iti
MA=(1/nc1)∑inii/ti
Mean IU=(1/nc1)∑inii/(ti+∑jnji-nii)
Fw IU=(∑ktk)-1∑itinii/(ti+∑jnji-nii)
nijTo represent the number of pixels for belonging to classification i and being but predicted to be classification j, nc1Represent total classification number, ti=∑jnij
Represent total pixel of classification i
Testing procedure are as follows:
S4.2.1, input test collection to the semantic segmentation model saved；
S4.2.2, the corresponding semantic segmentation result of forward calculation output test set is carried out；
S4.2.3, PA, MA, Mean IU are calculated with the label of image in corresponding test set according to output semantic segmentation result
With Fw IU precision；
In the test set, the measuring accuracy for the semantic segmentation model built is as shown in table 4-1
Table 4-1 semantic segmentation model measurement precision
| Precision | PA | MA | Mean IU | Fw IU |
| Test result | 84.0 | 56.1 | 42.9 | 74.4 |
As seen from the above table, in the test process, the PA precision of the network can reach 84.0%, MA precision and can reach
56.1%, Mean IU precision, which can reach 42.9%, Fw IU precision, can reach 74.4%, face new road scene data
When collection, the semantic segmentation network built has good Generalization Capability, can be acquisition substation's road scene figure in subsequent step
The reliability of the vision auxiliary information of picture lays a good foundation.
The new substation's road scene image of S4.3, input exports semantic segmentation result figure, selection to semantic segmentation model
Area-of-interest；
S4.4, object type distribution situation in area-of-interest is determined；
S4.5, connection object category distribution state to crusing robot navigation system.
The present invention constructs and trains semantic segmentation model, most by carrying out semantic segmentation to substation's road scene image
Posteriority model of a syndrome carries out the generalization ability of semantic segmentation to new substation's road scene image；The present invention utilizes semantic segmentation figure
The some scene types and object distribution situation in current crusing robot visual field can be directly provided, to assist its navigation system
Decision goes out more reasonable driving path.
Although being described in detail in conjunction with specific embodiment of the attached drawing to invention, should not be construed as to this patent
Protection scope restriction.In range described by claims, those skilled in the art are without creative work
The various modifications and deformation made still belong to the protection scope of this patent.
Claims (8)
1. a kind of Intelligent Mobile Robot vision navigation system method based on residual error network characterized by comprising
S1, the substation's road scene image for acquiring crusing robot inspection, screen road scene image, and use language
Adopted parted pattern indirect labor carries out data set mark；
S2, semantic segmentation model of the building based on residual error network；
S3, the semantic segmentation model built in step S2 is trained using SGD method, and saves convergent semantic segmentation mould
Type；
S4, the Generalization Capability for determining semantic segmentation model, input new inspection road scene image, output semantic segmentation as a result,
Object type distribution in semantic segmentation result in area-of-interest is transmitted to crusing robot navigation system.
2. the Intelligent Mobile Robot vision navigation system method according to claim 1 based on residual error network, special
Sign is, in the step S1 specifically includes the following steps:
S1.1, acquire crusing robot inspection substation's road scene video, extract each frame of video, remove multiplicity compared with
High video frame filters out part raw sample data and is manually marked；
S1.2, data set mark is manually carried out using semantic segmentation model-aided.
3. the Intelligent Mobile Robot vision navigation system method according to claim 2 based on residual error network, special
Sign is, includes: using the specific steps that semantic segmentation model-aided manually carries out data set mark in the step S1.2
S1.2.1, artificial mark part raw sample data；
S1.2.2, the raw sample data training semantic segmentation model marked is utilized；
S1.2.3, semantic segmentation is carried out to the remaining sample in raw sample data using trained semantic segmentation model, and defeated
Segmentation result out；
S1.2.4, the sample of segmentation result difference is marked again, has finally been marked the segmentation result addition after readjustment
The data set of note.
4. the Intelligent Mobile Robot vision navigation system method according to claim 1 based on residual error network, special
Sign is that the specific steps that the semantic segmentation model based on residual error network is constructed in the step S2 include:
S2.1, the coding for building semantic segmentation model, decoding, multitiered network fused layer three parts frame；
S2.2, according to the network frame in step S2.1, building is directed to the network structure of substation's road scene data set.
5. the Intelligent Mobile Robot vision navigation system method according to claim 4 based on residual error network, special
Sign is that building is directed to the network structure of substation's road scene data set in the step S2.2, including coding network is implemented
Step:
Network input layer is built, input layer input includes substation's road scene image and corresponding mark label data；
Convolutional layer conv1 is built, convolution operation is carried out to input picture；
Pond layer pool1 is built, 2 × (sampling step length 2) down-samplings are carried out to output in conv1；
Build residual block Res2, be carried out continuously convolution twice in residual block Res2,2 × down-sampling, output after continuous convolution with
Eltwise operation fusion pool1 exports result；
Build residual block Res3, be carried out continuously convolution twice in residual block Res3,2 × down-sampling, output after continuous convolution with
Eltwise operation fusion Res2 exports result；
Build residual block Res4, be carried out continuously convolution twice in residual block Res4,2 × down-sampling, output after continuous convolution with
Eltwise operation fusion Res3 exports result；
Build residual block Res5, be carried out continuously convolution twice in residual block Res5,2 × down-sampling, output after continuous convolution with
Eltwise operation fusion Res4 exports result；
Pond layer pool5 is built, Res5 is exported and carries out 2 × down-sampling；
Convolutional layer Conv6 is built, output channel number is consistent with classification is marked in substation's road scene data set.
6. the Intelligent Mobile Robot vision navigation system method according to claim 4 based on residual error network, special
Sign is that it further includes decoding and multilayer that building, which is directed to the network structure of substation's road scene data set, in the step S2.2
Network integration implementation steps:
Upscore2_1 layers are built, Upscore2_1 layers are operated with Deconvolution by conv6 output 2 × up-sampling of progress；
Build score4 layers, score4 layers Res4 output channel number is compressed to it is consistent with conv6 output channel number；
Crop4 layers are built, Crop4 layers are cut to score4 layers of output unanimously with Upscore2_1 layers of Output Size；
Fuse4 layers are built, Fuse4 layers operate fusion score4 and conv6 with Eltwise and export；
Upscore2_2 layers are built, fuse4 is carried out 2 × up-sampling by Upscore2_2 layers；
Build score3 layers, score3 layers Res3 output channel number is compressed to it is consistent with Fuse4 layers of output channel number；
Crop3 layers are built, Crop3 layers consistent with score3 layers of Output Size clipped value by Upscore2_2 layers；
Fuse3 layers are built, Fuse3 layers are merged Upscore2_2 layers with score3 layers of output with Elewise operation；
Upscore8 layers are built, Upscore8 layers are operated with Deconvolution by Fuse3 output 8 × up-sampling of progress；
Crop8 layers are built, Crop8 layers are cut to Upscore8 layers of output and the same size of input picture；
Classification layer specific implementation step are as follows: build classification layer, Crop8 layers of input are carried out by picture using softmaxWithloss
Element classification.
7. the Intelligent Mobile Robot vision navigation system method according to claim 1 based on residual error network, special
Sign is, is trained using SGD method to semantic segmentation network model in the step S3, and save last time iteration and obtain
The specific steps of semantic segmentation network model include:
S3.1, data set is divided into training set, verifying collection and test set；
S3.2, setting network optimized approach, select SGD to optimize；
S3.3, fixed Study rate parameter；
S3.4, setting moment of momentum parameter；
S3.5, setting weight attenuation parameter；
S3.6, setting Optimized Iterative step-length；
S3.7, random initializtion network of relation layer parameter；
S3.8, it is kept substantially when network layer softmaxWithloss output loss value levels off to the verifying precision of balance or model
Deconditioning when constant；
S3.9, the semantic segmentation network model that last time iteration obtains is saved.
8. the Intelligent Mobile Robot vision navigation system method according to claim 1 based on residual error network, special
Sign is that the step S4 determines the Generalization Capability of semantic segmentation model, inputs new inspection road scene image, and output is semantic
Object type distribution in semantic segmentation result in area-of-interest is transmitted to crusing robot navigation system by segmentation result
The specific steps of system include:
S4.1, input test collection image to the semantic segmentation model saved；
S4.2, progress forward calculation export corresponding semantic segmentation as a result, determining the Generalization Capability of network；
The new substation's road scene image of S4.3, input exports semantic segmentation result figure, selection sense is emerging to semantic segmentation model
Interesting region；
S4.4, object type distribution situation in area-of-interest is determined；
S4.5, connection object category distribution state to crusing robot navigation system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910329956.9A CN110006435A (en) | 2019-04-23 | 2019-04-23 | A kind of Intelligent Mobile Robot vision navigation system method based on residual error network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910329956.9A CN110006435A (en) | 2019-04-23 | 2019-04-23 | A kind of Intelligent Mobile Robot vision navigation system method based on residual error network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110006435A true CN110006435A (en) | 2019-07-12 |
Family
ID=67173765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910329956.9A Pending CN110006435A (en) | 2019-04-23 | 2019-04-23 | A kind of Intelligent Mobile Robot vision navigation system method based on residual error network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110006435A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110490729A (en) * | 2019-08-16 | 2019-11-22 | 南京汇银迅信息技术有限公司 | A kind of financial user classification method based on user's portrait model |
| CN111144418A (en) * | 2019-12-31 | 2020-05-12 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111367282A (en) * | 2020-03-09 | 2020-07-03 | 山东大学 | Robot navigation method and system based on multimode perception and reinforcement learning |
| CN111645073A (en) * | 2020-05-29 | 2020-09-11 | 武汉理工大学 | Robot visual semantic navigation method, device and system |
| CN111917178A (en) * | 2020-06-23 | 2020-11-10 | 国网浙江乐清市供电有限公司 | Patrol method of operation and maintenance integrated transformer substation |
| CN111958595A (en) * | 2020-08-13 | 2020-11-20 | 成都理工大学 | Multi-sensor asynchronous information fusion system and method for transformer substation inspection robot |
| CN112598003A (en) * | 2020-12-18 | 2021-04-02 | 燕山大学 | Real-time semantic segmentation method based on data expansion and full-supervision preprocessing |
| CN112669316A (en) * | 2021-01-29 | 2021-04-16 | 南方电网调峰调频发电有限公司 | Power production abnormity monitoring method and device, computer equipment and storage medium |
| CN113743330A (en) * | 2021-09-08 | 2021-12-03 | 新疆大学 | Transformer substation intelligent unmanned aerial vehicle inspection method based on visual navigation |
| CN114005097A (en) * | 2020-07-28 | 2022-02-01 | 株洲中车时代电气股份有限公司 | Train operation environment real-time detection method and system based on image semantic segmentation |
| CN114152937A (en) * | 2022-02-09 | 2022-03-08 | 西南科技大学 | External parameter calibration method for rotary laser radar |
| CN115222799A (en) * | 2021-08-12 | 2022-10-21 | 达闼机器人股份有限公司 | Method and device for acquiring image gravity direction, electronic equipment and storage medium |
| CN115249261A (en) * | 2021-08-12 | 2022-10-28 | 达闼机器人股份有限公司 | Method and device for acquiring image gravity direction, electronic equipment and storage medium |
| CN116228774A (en) * | 2023-05-10 | 2023-06-06 | 国网山东省电力公司菏泽供电公司 | Substation inspection image defect identification method and system based on image quality evaluation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426930A (en) * | 2015-11-09 | 2016-03-23 | 国网冀北电力有限公司信息通信分公司 | Substation property segmentation method based on convolutional neural network |
| CN108174165A (en) * | 2018-01-17 | 2018-06-15 | 重庆览辉信息技术有限公司 | Electric power safety operation and O＆M intelligent monitoring system and method |
| CN108319972A (en) * | 2018-01-18 | 2018-07-24 | 南京师范大学 | A kind of end-to-end difference online learning methods for image, semantic segmentation |
| CN108664974A (en) * | 2018-04-03 | 2018-10-16 | 华南理工大学 | A kind of semantic segmentation method based on RGBD images Yu Complete Disability difference network |
| CN109446970A (en) * | 2018-10-24 | 2019-03-08 | 西南交通大学 | A kind of Intelligent Mobile Robot road scene recognition methods based on deep learning |
-
2019
- 2019-04-23 CN CN201910329956.9A patent/CN110006435A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105426930A (en) * | 2015-11-09 | 2016-03-23 | 国网冀北电力有限公司信息通信分公司 | Substation property segmentation method based on convolutional neural network |
| CN108174165A (en) * | 2018-01-17 | 2018-06-15 | 重庆览辉信息技术有限公司 | Electric power safety operation and O＆M intelligent monitoring system and method |
| CN108319972A (en) * | 2018-01-18 | 2018-07-24 | 南京师范大学 | A kind of end-to-end difference online learning methods for image, semantic segmentation |
| CN108664974A (en) * | 2018-04-03 | 2018-10-16 | 华南理工大学 | A kind of semantic segmentation method based on RGBD images Yu Complete Disability difference network |
| CN109446970A (en) * | 2018-10-24 | 2019-03-08 | 西南交通大学 | A kind of Intelligent Mobile Robot road scene recognition methods based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 王晨: "基于深度学习的红外图像语义分割技术研究", 《中国博士学位论文全文数据库•信息科技辑》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110490729A (en) * | 2019-08-16 | 2019-11-22 | 南京汇银迅信息技术有限公司 | A kind of financial user classification method based on user's portrait model |
| CN111144418A (en) * | 2019-12-31 | 2020-05-12 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111144418B (en) * | 2019-12-31 | 2022-12-02 | 北京交通大学 | Railway track area segmentation and extraction method |
| CN111367282A (en) * | 2020-03-09 | 2020-07-03 | 山东大学 | Robot navigation method and system based on multimode perception and reinforcement learning |
| CN111645073A (en) * | 2020-05-29 | 2020-09-11 | 武汉理工大学 | Robot visual semantic navigation method, device and system |
| CN111917178A (en) * | 2020-06-23 | 2020-11-10 | 国网浙江乐清市供电有限公司 | Patrol method of operation and maintenance integrated transformer substation |
| CN111917178B (en) * | 2020-06-23 | 2023-07-14 | 国网浙江乐清市供电有限公司 | Inspection method of operation and maintenance integrated transformer substation |
| CN114005097A (en) * | 2020-07-28 | 2022-02-01 | 株洲中车时代电气股份有限公司 | Train operation environment real-time detection method and system based on image semantic segmentation |
| CN111958595A (en) * | 2020-08-13 | 2020-11-20 | 成都理工大学 | Multi-sensor asynchronous information fusion system and method for transformer substation inspection robot |
| CN111958595B (en) * | 2020-08-13 | 2021-09-17 | 成都理工大学 | Multi-sensor asynchronous information fusion system and method for transformer substation inspection robot |
| CN112598003A (en) * | 2020-12-18 | 2021-04-02 | 燕山大学 | Real-time semantic segmentation method based on data expansion and full-supervision preprocessing |
| CN112669316A (en) * | 2021-01-29 | 2021-04-16 | 南方电网调峰调频发电有限公司 | Power production abnormity monitoring method and device, computer equipment and storage medium |
| CN112669316B (en) * | 2021-01-29 | 2023-05-30 | 南方电网调峰调频发电有限公司 | Power production abnormality monitoring method, device, computer equipment and storage medium |
| CN115222799A (en) * | 2021-08-12 | 2022-10-21 | 达闼机器人股份有限公司 | Method and device for acquiring image gravity direction, electronic equipment and storage medium |
| CN115249261A (en) * | 2021-08-12 | 2022-10-28 | 达闼机器人股份有限公司 | Method and device for acquiring image gravity direction, electronic equipment and storage medium |
| CN115222799B (en) * | 2021-08-12 | 2023-04-11 | 达闼机器人股份有限公司 | Method and device for acquiring image gravity direction, electronic equipment and storage medium |
| CN113743330A (en) * | 2021-09-08 | 2021-12-03 | 新疆大学 | Transformer substation intelligent unmanned aerial vehicle inspection method based on visual navigation |
| CN114152937A (en) * | 2022-02-09 | 2022-03-08 | 西南科技大学 | External parameter calibration method for rotary laser radar |
| CN116228774A (en) * | 2023-05-10 | 2023-06-06 | 国网山东省电力公司菏泽供电公司 | Substation inspection image defect identification method and system based on image quality evaluation |
| CN116228774B (en) * | 2023-05-10 | 2023-09-08 | 国网山东省电力公司菏泽供电公司 | Substation inspection image defect identification method and system based on image quality evaluation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110006435A (en) | A kind of Intelligent Mobile Robot vision navigation system method based on residual error network | |
| Jiang et al. | Real‐time crack assessment using deep neural networks with wall‐climbing unmanned aerial system | |
| CN102147260B (en) | Electronic map matching method and device | |
| CN105241465B (en) | A kind of method of road renewal | |
| CN108389256B (en) | Two-three-dimensional interactive unmanned aerial vehicle electric power tower inspection auxiliary method | |
| CN109829908B (en) | Binocular image-based method and device for detecting safety distance of ground object below power line | |
| CN109766746A (en) | A kind of track foreign matter detecting method of unmanned plane video | |
| CN114998744B (en) | Agricultural machinery track field dividing method and device based on motion and vision dual-feature fusion | |
| CN110910440B (en) | Power transmission line length determination method and system based on power image data | |
| CN109816780B (en) | Power transmission line three-dimensional point cloud generation method and device of binocular sequence image | |
| CN108984755A (en) | A kind of geological geographic information system processing method | |
| CN111080609A (en) | Brake shoe bolt loss detection method based on deep learning | |
| CN111295666A (en) | Lane line detection method, device, control equipment and storage medium | |
| CN114003035A (en) | Method, device, equipment and medium for autonomous navigation of robot | |
| CN115690153A (en) | Intelligent agent track prediction method and system | |
| CN113284144A (en) | Tunnel detection method and device based on unmanned aerial vehicle | |
| CN115547003A (en) | External damage prevention alarm method and system for power transmission line | |
| CN111507189A (en) | Insulator string defect rapid detection method based on image processing technology | |
| Manninen et al. | Multi-stage deep learning networks for automated assessment of electricity transmission infrastructure using fly-by images | |
| CN113985406A (en) | Target track splicing method for marine radar | |
| CN106990724B (en) | Environmental monitoring cloud service platform and method of environmental monitoring | |
| Guo et al. | Detection of insulator defects based on YOLO V3 | |
| Li et al. | Automated bridge crack detection based on improving encoder–decoder network and strip pooling | |
| CN116627126A (en) | Intelligent inspection method and system based on big data | |
| Chen et al. | Defect digital twinning: A technical framework to integrate robotics, AI and BIM for facility management and renovation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |