CN108764115A - A kind of truck danger based reminding method - Google Patents
A kind of truck danger based reminding method Download PDFInfo
- Publication number
- CN108764115A CN108764115A CN201810507764.8A CN201810507764A CN108764115A CN 108764115 A CN108764115 A CN 108764115A CN 201810507764 A CN201810507764 A CN 201810507764A CN 108764115 A CN108764115 A CN 108764115A
- Authority
- CN
- China
- Prior art keywords
- truck
- frame
- image
- training
- acquiescence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000009429 distress Effects 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 38
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 3
- 238000005065 mining Methods 0.000 abstract description 8
- 238000004891 communication Methods 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000002779 inactivation Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 239000003721 gunpowder Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to truck safe early warning fields, and in particular to a kind of truck danger based reminding method.Method includes：For the truck in the region of mine, the image information of each image acquisition device corresponding region on truck；The image processing center being connect with all image collecting devices on truck handles image information in real time, determines and whether there is target to be identified in the preset range of current truck；If there are targets to be identified in preset range, send distress signal to the driver of current truck.The truck danger based reminding method of the present invention does not use network communication, all image informations all to handle in real time, and early warning speed is fast.The truck danger based reminding method of the present invention is handled image information using lightweight SSD models in real time, which needs especially to know that others and vehicle target recognition speed are fast, recognition efficiency is high to mining area truck.
Description
Technical field
The invention belongs to truck safe early warning fields, and in particular to a kind of truck danger based reminding method.
Background technology
With being constantly progressive for mining technique and being continuously increased for mining rate, equipment enlarging is also constantly being sent out
Exhibition.Large truck transport is one of the main form of current large-scale mine production and transport.
The different safety pre-warning system with general-utility car of mine truck safety pre-warning system, there are two main causes：Its
One, the travel of mine vehicle and the travel of general car are entirely different, and it is by various ditches that mine vehicle, which runs road,
Road and step composition, the complexity of road are significantly larger than general car；Second, mine vehicle is bulky, length, width and
Height is all as many as the several times of general car, therefore there is blind area, in contrast danger coefficient also higher.
The large truck of Mine haul has its own feature, is mainly shown as：Path space is complicated, due to limit of mining
Centainly, it and needs constantly to develop in depth, haul road conditions are complicated, and bend and ramp are more, and sight is bad, and road is again
It often changes, operating environment is severe；Production auxiliary vehicle class is various, such as command car, gunpowder vehicle, bull-dozer, sprinkling truck etc.；Card
Vehicle driver's great work intensity, it is continuous to drive a few houres, it is inevitably uninteresting；And large truck is bulky, driver's cabin only accounts for upper left side
Sub-fraction, there are blind areas for the overwhelming majority and front and rear part region on the right side of car body, and driver can't see truck within the scope of blind area
The people or object of surrounding；In addition mine road work double tides, size vehicle mixes the reasons such as row, vehicle and vehicle, vehicle and people it
Between collision accident happen occasionally, become threaten mine safety transport major hidden danger.
Invention content
(1) technical problems to be solved
In order to solve the problems, such as prior art mining area truck due to causing the accident and take place frequently there are blind area, the present invention provides
A kind of truck danger based reminding method.
(2) technical solution
To achieve the above object, the main technical schemes that the present invention uses include：
A kind of truck danger based reminding method, includes the following steps：
S1, for the truck in the region of mine, each image acquisition device corresponding region on the truck
Image information；
The image processing center being connect with all image collecting devices on S2, the truck carries out described image information real
When handle, determine and whether there is target to be identified in the preset range of current truck；
If there are the targets to be identified in S3, the preset range, danger is sent out to the driver of current truck
Signal.
Further, the step S2 includes：
S21, the image of each image acquisition device is pre-processed, obtains pretreated image；
S22, using lightweight SSD (Single Shot MultiBox Detector, abbreviation SSD) model after training
Pretreated image is handled in real time；
Wherein, the lightweight SSD models after the training are advance use with the relevant training image of current truck to light
Magnitude SSD models are trained acquisition.
Further, before the step S21, further include：
Training lightweight SSD models, obtain the lightweight SSD models after training；
The lightweight SSD models include eight modules：
First module includes two 3 × 3 convolutional layers and a maximum pond layer；Second module includes two 3 × 3 convolutional layers
With a maximum pond layer；Third module includes three 3 × 3 convolutional layers and a maximum pond layer；4th module includes three 3
× 3 convolutional layers and a maximum pond layer；5th module includes three 3 × 3 convolutional layers and a maximum pond layer；6th module
Including one 3 × 3 convolutional layer and a maximum pond layer；7th module includes 1 × 1 convolutional layer；8th module includes
One 1 × 1 convolutional layer and one 3 × 3 convolutional layer；
It is transmitted into row information by the convolutional layer and the pond layer between module.
Further, the training lightweight SSD models specifically include：
L01：The pretreated training image is inputted into the lightweight SSD models and obtains fisrt feature figure；
L02：Calculate the first acquiescence frame of the fisrt feature figure；
It is described first acquiescence frame dimension calculation formula be：
Wherein, m is characterized map number；sminThe scale of frame is given tacit consent to for bottom characteristic pattern first；smaxFor top feature
Figure first gives tacit consent to the scale of frame；
Last basisWithCalculate the width and height of each first acquiescence frame
Degree, wherein a is the first acquiescence frame ratio value；
L03：Determine whether to encode previously given reference block according to decision condition；
The decision condition is：Compare the size of the value and jaccard values of first threshold, the jaccard values are by described
First acquiescence frame coding and the reference block calculate；If the jaccard values are more than the first threshold, to the ginseng
Examine frame coding；
The reference block after coding includes：Position offset (g=(cx, cy, w, h)), target fractional (p ∈ [0,1]) and
Label (x ∈ { 0,1 }),
The calculation formula of position offset is：
Cx=(cxg-cxd)/wd
Cy=(cyg-cyd)/hd
Wherein, (cx, cy) indicates the center of the reference block, and (w, h) represents the width and height of the reference block；Subscript rope
Draw g and indicates that the reference block, d indicate the first acquiescence frame, cxg、cyg、wg、hgIt is directly read from the training image；
L04：First convolution algorithm without activation primitive is carried out to the fisrt feature figure, obtains four of the first acquiescence frame
Four position offsets of position offset, the first acquiescence frame are used for the location prediction of the target；
Second convolution algorithm without activation primitive is carried out to the fisrt feature figure, obtains three classification confidence levels, it is described
Classification confidence level is used for the class prediction of the target；
It is handled using classification confidence level described in softmax function pairs, obtains the prediction classification of the first acquiescence frame
Probability.
Further, the loss function of the lightweight SSD models is：
Wherein, N is the quantity of matched first default boundary frame, if N=0, L 0；
Position offset loss is defined as：
Wherein, i is the index for the first acquiescence frame that label is 1, and l and g are respectively the inclined of the training image prediction block
The offset of reference block after moving and encoding；
Target fractional loss is calculated by binary cross entropy：
Wherein, p is the target fractional of the training image prediction block, and x ∈ { 0,1 } are after the reference block encodes
Reference label.
Further, the third module of the lightweight SSD models after porous process of convolution with the 4th mould
Block is connected.
Further, the step S22 is specifically included：
M1：The lightweight SSD models after load training, the pretreated image information is after the training
Second feature figure is obtained after the processing of lightweight SSD models；
M2：Calculate the second acquiescence frame of the second feature figure；
M3：To the reference block decoding after coding；
Give tacit consent to frame and reference block fusion by described second, obtains third acquiescence frame；
The calculation formula of the fusion is：
xc=loc [0 [× wref×sacling[0[+xref
yc=loc [1 [× href×sacling[1[+yref
W=wref×e(loc[2[×sacling[2[)
H=href×e(loc[3[×sacling[3[)
Wherein, xc、ycThe center point coordinate of frame is given tacit consent to for the third, w, h are the width and height that the third gives tacit consent to frame, loc
For four position offsets of the second acquiescence frame that convolution obtains, xref、yref、wref、hrefTo be calculated according to ratio value
Four position offsets of the prediction block of the pretreated image information arrived, scaling is default parameters；
M4：Third acquiescence frame is screened, the position offset that the third filtered out is given tacit consent to frame marks
In the pretreated image information, then the image information of the mark is exported.
Further, the target to be identified includes personnel and vehicle.
(3) advantageous effect
The beneficial effects of the invention are as follows：
A kind of truck danger based reminding method, including step：For the truck in the region of mine, each figure on truck
As harvester acquires the image information of corresponding region；The image processing center pair being connect with all image collecting devices on truck
Image information is handled in real time, is determined and be whether there is target to be identified in the preset range of current truck；If in preset range
There are targets to be identified, then send distress signal to the driver of current truck.The truck danger based reminding method of the present invention is not
Using network communication, all image informations are all handled in real time, and early warning speed is fast.
The truck danger based reminding method of the present invention is handled image information using lightweight SSD models in real time, the mould
Type needs especially to know that others and vehicle target recognition speed are fast, recognition efficiency is high to mining area truck.
Description of the drawings
Fig. 1 is a kind of flow chart of truck danger based reminding method of the embodiment of the present invention；
Fig. 2 is the schematic diagram of lightweight SSD models of the embodiment of the present invention；
Fig. 3 is the schematic diagram of 8 × 8 second feature figure of the embodiment of the present invention；
Fig. 4 is the schematic diagram of 4 × 4 second feature figure of the embodiment of the present invention.
Specific implementation mode
In order to preferably explain the present invention, in order to understand, below in conjunction with the accompanying drawings, by specific implementation mode, to this hair
It is bright to be described in detail.
Embodiment 1
It is as shown in Figure 1 a kind of truck danger based reminding method of the embodiment of the present invention, includes the following steps：
S1, for the truck in the region of mine, the figure of each image acquisition device corresponding region on truck
As information.
The image processing center being connect with all image collecting devices on S2, truck handles image information in real time,
It determines and whether there is target to be identified in the preset range of current truck；
Wherein, step S2 is specifically included：
S21, the image of each image acquisition device is pre-processed, obtains pretreated image；
S22, pretreated image is handled in real time using the lightweight SSD models after training；
Wherein, the lightweight SSD models after training are advance use with the relevant training image of current truck to lightweight
SSD models are trained acquisition.
If there are targets to be identified in S3, preset range, send distress signal to the driver of current truck.
As shown in Fig. 2, for the schematic diagram of lightweight SSD models of the embodiment of the present invention, lightweight SSD models include eight moulds
Block：
First module includes two 3 × 3 convolutional layers and a maximum pond layer；Second module includes two 3 × 3 convolutional layers
With a maximum pond layer；Third module includes three 3 × 3 convolutional layers and a maximum pond layer；4th module includes three 3
× 3 convolutional layers and a maximum pond layer；5th module includes three 3 × 3 convolutional layers and a maximum pond layer；6th module
Including one 3 × 3 convolutional layer and a maximum pond layer；7th module includes 1 × 1 convolutional layer；8th module includes
One 1 × 1 convolutional layer and one 3 × 3 convolutional layer；It is transmitted into row information by convolutional layer and pond layer between module.
Specific steps include as follows：
First module：Input picture size is 300 × 300 × 3, is carried out to original image using 64 3 × 3 convolution kernels
Convolution twice, i.e. two convolutional layers；The pond core for using step-length to be 2 × 2 for 2 sizes again carries out pond；
Second module：Convolution is carried out twice using 128 3 × 3 the first module of convolution kernel pair output, i.e. two convolution
Layer；The pond core for using step-length to be 2 × 2 for 2 sizes again carries out pond；
Third module：The second module of the convolution kernel pair output adopted 256 3 × 3 carries out convolution three times, i.e. three convolutional layers；
The pond core for using step-length to be 2 × 2 for 2 sizes again carries out pond；
4th module：The convolution kernel adopted 512 3 × 3, which exports third module, carries out convolution three times, i.e. three convolutional layers；
The pond core for using step-length to be 2 × 2 for 2 sizes again carries out pond；
5th module：The 4th module of the convolution kernel pair output adopted 512 3 × 3 carries out convolution three times, i.e. three convolutional layers；
The pond core for using step-length to be 3 × 3 for 1 size again carries out pond；
6th module：The 5th module of the convolution kernel pair output for being 6 using 1024 3 × 3, rate (rate of spread) carries out convolution
Once, i.e. a convolutional layer；
7th module：Primary, an i.e. convolution using 1024 1 × 1 the 6th module of convolution kernel pair output progress convolution
Layer；
8th module：Primary, an i.e. convolution using 256 1 × 1 the 7th module of convolution kernel pair output progress convolution
Layer；Export that carry out convolution primary with the 7th module of convolution kernel pair adopted 256 3 × 3 again, totally two convolutional layers.
It should be noted that the lightweight SSD models that the present embodiment uses use multilayer feature fusion method, by third mould
Block, the 4th module, the 7th module and the characteristic pattern of the 8th module output carry out Fusion Features, wherein third module is through porous volume
It is connected with the 4th module after product processing, as new module, then by the convolution before third module and the 4th module pond layer
Layer output characteristic pattern and the output characteristic pattern of the 7th module, the 8th module do multi-scale feature fusion, the specific steps are：
First, the output characteristic pattern of third convolutional layer uses step-length for 1 in third module, the Chi Huahe that size is 2 × 2
Carry out maximum pond；The convolution kernel for using 512 sizes to be 2 for 3 × 3, rate again carries out convolution, is obtained using normalization layer
Export characteristic pattern；By in the 4th module third convolutional layer output characteristic pattern by normalization layer, then with third module
Output characteristic pattern connects together, that is to say, that the output feature bitmap-format of new module and the output feature bitmap-format of third module
It is identical；
Then, the characteristic pattern by the output of new module, the 7th module and the 8th module carries out Fusion Features.
Embodiment 2
When operation for the first time is loaded with a kind of program of truck danger based reminding method of the present invention, need to lightweight SSD models
It is trained, training lightweight SSD models specifically include following steps：
L01：Pretreated training image input lightweight SSD models are obtained into fisrt feature figure；
It should be noted that in order to be detected on multiple scales, new module, the 7th module and the 8th mould are extracted here
The fisrt feature figure of block.Each fisrt feature figure includes certain amount of default boundary frame, that is, gives tacit consent to frame, these acquiescence frames have
Certain ratio and aspect ratio, by two distinct types of filter (i.e. position and score) be applied to each fisrt feature figure with
The position offset and target fractional of prediction acquiescence frame.
L02：Calculate the first acquiescence frame of each fisrt feature figure；
First, fisrt feature figure is divided into the grid of the block with 1 × 1 size, and determines first in these blocks
Give tacit consent to the center of frame.
First acquiescence frame dimension calculation formula be：
Wherein, m is characterized map number；sminThe scale of frame is given tacit consent to for bottom characteristic pattern first；smaxFor top feature
Figure first gives tacit consent to the scale of frame；
Since personnel's image under truck visual angle is often that length is more than width, vehicle image is that width is more than length
, according to this attribute, to each fisrt feature figure layer use different the ratio of width to height (such as the ratio of width to height of 38 × 38 characteristic patterns forSince the personnel under truck visual angle are less than common personnel and vehicle with vehicle, to first
Give tacit consent to frame and uses smaller scale (such as smin=0.1) Small object, can preferably be detected.
Last basisWithThe width and height of each first acquiescence frame are calculated,
In, a is the first acquiescence frame ratio value.
L03：Determine whether to encode previously given reference block according to decision condition；
It after obtaining the first acquiescence frame, needs to encode previously given reference block, in this way, can incite somebody to action
Reference block is converted into that the form that lightweight SSD models are trained can be input to.Decision condition is：Compare the value of first threshold
With the size of jaccard values, jaccard values are from the first acquiescence frame coding and reference block calculating；If jaccard values are more than the
One threshold value then encodes reference block；Wherein, jaccard values can regard the ratio between the friendship union of two set A, B as：
JaccardSim=(A ∩ B)/(A ∪ B) is exactly to calculate the A degree Chong Die with B generations in fact.In the present embodiment, first threshold
It is 0.45, if jaccard values are more than first threshold, the first acquiescence frame is exactly corresponding coding result.In such case
Under, the object tag of the first acquiescence frame will be arranged to 1, and jaccard values are left to the object score of the first acquiescence frame, and
And the position offset between reference block and the first acquiescence frame will also be recorded.
Reference block after coding includes：Position offset (g=(cx, cy, w, h)), target fractional (p ∈ [0,1]) and label
(x ∈ { 0,1 }),
The calculation formula of position offset is：
Cx=(cxg-cxd)/wd
Cy=(cyg-cyd)/hd
Wherein, (cx, cy) indicates the center of reference block, and (w, h) represents the width and height of reference block；Subscript indexes g and indicates
Reference block, d indicate the first acquiescence frame, cxg、cyg、wg、hgIt is directly read from training image.
L04：First convolution algorithm without activation primitive is carried out to fisrt feature figure, obtains four positions of the first acquiescence frame
Four position offsets of offset, the first acquiescence frame are used for the location prediction of target；
Second convolution algorithm without activation primitive is carried out to fisrt feature figure, obtains three classification confidence levels, classification confidence
Class prediction of the degree for target；
It is handled using softmax function pair classification confidence levels, obtains the probability of the prediction classification of the first acquiescence frame.
During being trained to lightweight SSD models, the loss function of lightweight SSD models is：
Wherein, N is the quantity of matched first default boundary frame, if N=0, L 0；
Position offset loss is defined as：
Wherein, the index for the first acquiescence frame that i is that label is 1, and l and g be respectively training image prediction block offset and
The offset of reference block after coding；
Target fractional loss is calculated by binary cross entropy：
Wherein, p is the target fractional of training image prediction block, and x ∈ { 0,1 } are the reference marks after reference block coding
Label.
It should be noted that also needing to determining parameter in training process has：Sliding average undated parameter, learning rate etc..
Trained network parameter needs save, subsequently to use.
A kind of truck danger based reminding method of the present invention can realize the safety monitoring of large truck surrounding people and vehicle.Due to
Image under truck visual angle is less, and the similarity degree of these images and PASCAL VOC2012 data sets is higher, therefore first adopts
(PASCAL VOC data sets are handled, only retained personnel and vehicle with the PASCAL VOC2012 data sets for only remaining two classes
Two class target images) model is finely adjusted, then model is finely tuned again with truck multi-view image.The instruction of the present embodiment
Practicing image is acquired in thousand mining industry of saddle.
Embodiment 3
Lightweight SSD model structures after training are：
The port number of the image that the input of lightweight SSD models is 300 × 300 × 3, image is 3, is RGB (color of image
Three primary colours：Red, green, blue) image.First module：It is two convolutional layers first, convolution kernel size is 3 × 3, convolution
Core number is 64, and step-length 1, filling mode is ' SAME ' (zero padding keeps the image size of input and output identical), and activation primitive is
Line rectification function (Rectified Linear Unit, ReLU), followed by maximum pond layer, sampling core size are 2 × 2, step
A length of 2, export the characteristic pattern for 64 150 × 150；
Second module：Two convolutional layers before this, convolution kernel size are 3 × 3, and convolution kernel number is 128, followed by maximum pond
Change layer, sampling core size is 2 × 2, exports the characteristic pattern for 128 75 × 75；
Third module：Three convolutional layers before this, convolution kernel size are 3 × 3, and convolution kernel number is 256, followed by maximum pond
Change layer, sampling core size is 2 × 2, exports the characteristic pattern for 256 38 × 38；
4th module：Three convolutional layers before this, convolution kernel size are 3 × 3, and convolution kernel number is 512, followed by maximum pond
Change layer, sampling core size is 2 × 2, exports the characteristic pattern for 512 19 × 19；
Three convolutional layers, convolution kernel size are 3 × 3 to 5th module before this, and convolution kernel number is 512, followed by maximum pond
Change layer, sampling core size is 3 × 3, and step-length 1 exports the characteristic pattern for 512 19 × 19；
6th module is a convolutional layer, and convolution kernel size is 3 × 3, and the coefficient of expansion 6, convolution kernel number is 1024, defeated
Go out the characteristic pattern for 1024 19 × 19, this feature figure is random deactivating layer, inactivation rate 0.5 in training network；
7th module is a convolutional layer, and convolution kernel size is 1 × 1, and convolution kernel number is 1024, and it is 1024 19 to export
× 19 characteristic pattern, this feature figure are also random deactivating layer, inactivation rate 0.5 in training network；
8th module is that first a convolutional layer, convolution kernel size are 1 × 1, and convolution kernel number is 256, and ' 1 ' is carried out to output
Then filling is a convolutional layer again, convolution kernel size is 3 × 3, and convolution kernel number is 512, step-length 2, and filling mode is
' VALID ' (will not add new pixel) on the basis of original input, export the characteristic pattern for 512 10 × 10.
After lightweight SSD model trainings are good, you can commencement of commercial operation is loaded with a kind of truck danger based reminding method of the present invention
Program specifically includes following steps：
M1：Lightweight SSD models after load training, the lightweight SSD moulds after pretreated image information is trained
Second feature figure is obtained after the processing of type.
Wherein, image information is pre-processed in order to enhance data, includes mainly：A random interception image part is gone forward side by side
Row torsional deformation, random left and right flipped image, random distortion color (such as brightness, saturation degree, coloration, contrast), then carry out
Image normalization, the specification for being 300 × 300 by image scaling.
It should be noted that pretreated image obtains the second of 8 modules output by lightweight SSD model treatments
Characteristic pattern, in the present embodiment, the second feature figure that is exported using three modules：The new module of 38 × 38 pixels, 19 × 19 pixels
The 7th module, the 8th module of 10 × 10 pixels, the second feature figure of these three modules has different scale sizes, from difference
The prediction that the second feature figure of scale generates different scale can ensure that network can identify different size of target.
M2：Calculate the second acquiescence frame of second feature figure.
For each second feature figure of three modules, the center of the second acquiescence frame, the second acquiescence are confirmed with each pixel
The center of frame is arranged toWherein l is characteristic pattern
Size, generate k second acquiescence frame (k 6) according to different sizes and length-width ratio.For shown in Fig. 3, the second acquiescence frame
The second feature figure of number k=6,8 × 8 pixels share 8 × 8 × 6=384 second acquiescence frame.
Each second, which gives tacit consent to frame size calculation formula, is：
Wherein, m is second feature map number, and m is 3 here；Here sminIt is 0.10；smaxIt is 0.5.Each second acquiescence frame
Length-width ratio is calculated according to ratio value：
Wherein, a is acquiescence frame ratio value.
By taking the second feature figure of the 7th module as an example, in the second feature figure of the 7th module, a be 1,2,0.5,3,
1.0/3}.The acquiescence frame for being 1 for ratio, one acquiescence frame width of additional addition are a height of：
Finally, the acquiescence frame number of the 7th module is 6.
3 × 3 × (k × 4) convolution algorithms without activation primitive are carried out respectively to the second feature figure of three modules, obtain
Four of two acquiescence frames are used for target location prediction deviation post, wherein 3 × 3 be convolution kernel size, k is each second feature figure
Second acquiescence frame number of upper each pixel, ' 4 ' four deviation posts for giving tacit consent to frame for second：Starting point transverse and longitudinal coordinate and it is wide,
It is high；
Then the second feature figure of three modules is subjected to 3 × 3 × (k × 3) convolution algorithms without activation primitive respectively, obtained
Classification confidence level is predicted for target classification, wherein 3 × 3 be convolution kernel size, k is each pixel on each characteristic pattern to three
The acquiescence frame number of point, ' 3 ' the class numbers that may belong to for each acquiescence frame, here 3 refer to is personnel targets, vehicle target
And background.
By taking the second feature figure of the 7th module as an example, the second feature figure size of the 7th module is 19 × 19, k 6, therefore
Last output is (19 × 19) × 6 × (3+4).Three classification confidence level outputs are handled by softmax functions, are obtained
The probability (value range 0-1) of classification is predicted to each second acquiescence frame, as shown in Figure 4.
M3：To the reference block decoding after coding.
By the second acquiescence frame and reference block fusion, third acquiescence frame is obtained；
The calculation formula of fusion is：
xc=loc [0] × wref×sacling[0]+xref
yc=loc [1] × href×sacling[1]+yref
W=wref×e(loc[2]×sacling[2])
H=href×e(loc[[3]×sacling[3])
Wherein, xc、ycThe center point coordinate of frame is given tacit consent to for third, w, h are the width and height that third gives tacit consent to frame, and loc obtains for convolution
Four position offsets of the second acquiescence frame arrived, xref、yref、wref、hrefFor after the pretreatment that is calculated according to ratio value
Image information prediction block four position offsets, scaling is default parameters, and size is [0.1,0.1,0.2,0.2]；
M4：Third acquiescence frame is screened, the position offset that the third filtered out is given tacit consent to frame is labeled in pretreatment
In image information afterwards, then the image information of mark is exported.
Third acquiescence frame is screened, frame is given tacit consent to for each third, if prediction third acquiescence frame belongs to certain class and (belongs to
In personnel or vehicle) probability be more than second threshold, here the value of second threshold be 0.5, then retain the third acquiescence frame, and
Preserve the class and prediction score belonging to it.Satisfactory third acquiescence frame is cut：It calculates third acquiescence frame and gives
Determine the intersection between reference block；Then the prediction category score for giving tacit consent to each third frame carries out descending arrangement, takes highest scoring
N third give tacit consent to frame, take here n be 400.Remaining third acquiescence frame is sieved again using non-maxima suppression method
Choosing：The jaccard values for giving tacit consent to calculating any two third frame retain when two third acquiescence frames are that inhomogeneity is predicted
Two thirds give tacit consent to frames, and otherwise the high third of retention forecasting score value gives tacit consent to frame, to satisfactory third acquiescence frame according to giving
Determine reference block to be finely adjusted.Third acquiescence frame is labeled in image information by four position offsets that frame is given tacit consent to according to third, and
The classification that mark third acquiescence frame is predicted, that is, be personnel or vehicle.Finally the image information marked is synthesized and is regarded
Frequently, display screen is sent to be shown.
By a kind of lightweight SSD models of truck danger based reminding method of the present invention and the comparison of SSD models, comparing result is as follows
Shown in table：
In table, wherein fps is detection speed：The frame number of display per second；MAP is accuracy of detection, is commonly used in target detection
Accuracy of detection Measure Indexes.
As can be seen from the above table, a kind of truck danger based reminding method structure of the invention simplifies, while detection target being determined
For people and vehicle, this makes model more simply refine.Use a kind of early warning system of truck danger based reminding method of the present invention can be with
Environment around intuitive display truck, emphasis mark personnel and vehicle, all image informations are all handled in real time, early warning speed
Soon, the safety of mining area large truck is improved.
A kind of effective detecting distance of truck danger based reminding method of the present invention is 50 meters, and under large truck full load conditions
Braking distance be about 45 meters or so, meet the requirement of safe early warning.Compared to the network before improvement, the detection of the invention
Speed improves 7% or so.
It is to be appreciated that describing the skill simply to illustrate that the present invention to what specific embodiments of the present invention carried out above
Art route and feature, its object is to allow those skilled in the art to can understand the content of the present invention and implement it accordingly, but
The present invention is not limited to above-mentioned particular implementations.Every various change made within the scope of the claims is repaiied
Decorations should all be covered within the scope of the present invention.
Claims (8)
1. a kind of truck danger based reminding method, which is characterized in that include the following steps：
S1, for the truck in the region of mine, the figure of each image acquisition device corresponding region on the truck
As information；
The image processing center being connect with all image collecting devices on S2, the truck locates described image information in real time
Reason determines and whether there is target to be identified in the preset range of current truck；
If sending distress signal to the driver of current truck there are the target to be identified in S3, the preset range.
2. according to the method described in claim 1, it is characterized in that, the step S2 includes：
S21, the image of each image acquisition device is pre-processed, obtains pretreated image；
S22, pretreated image is handled in real time using the lightweight SSD models after training；
Wherein, the lightweight SSD models after the training are advance use with the relevant training image of current truck to lightweight
SSD models are trained acquisition.
3. according to the method described in claim 2, it is characterized in that, before the step S21, further include：
Training lightweight SSD models, obtain the lightweight SSD models after training；
The lightweight SSD models include eight modules：
First module includes two 3 × 3 convolutional layers and a maximum pond layer；Second module includes two 3 × 3 convolutional layers and one
A maximum pond layer；Third module includes three 3 × 3 convolutional layers and a maximum pond layer；4th module includes three 3 × 3
Convolutional layer and a maximum pond layer；5th module includes three 3 × 3 convolutional layers and a maximum pond layer；6th module packet
Include one 3 × 3 convolutional layer and a maximum pond layer；7th module includes 1 × 1 convolutional layer；8th module includes one
A 1 × 1 convolutional layer and one 3 × 3 convolutional layer；
It is transmitted into row information by the convolutional layer and the pond layer between module.
4. according to the method described in claim 3, it is characterized in that, the training lightweight SSD models specifically include：
L01：The pretreated training image is inputted into the lightweight SSD models and obtains fisrt feature figure；
L02：Calculate the first acquiescence frame of the fisrt feature figure；
It is described first acquiescence frame dimension calculation formula be：
Wherein, m is characterized map number；sminThe scale of frame is given tacit consent to for bottom characteristic pattern first；smaxFor top characteristic pattern
The scale of one acquiescence frame；
Last basisWithThe width and height of each first acquiescence frame are calculated,
In, a is the first acquiescence frame ratio value；
L03：Determine whether to encode previously given reference block according to decision condition；
The decision condition is：Compare the size of the value and jaccard values of first threshold, the jaccard values are by described first
Give tacit consent to frame coding and the reference block calculates；If the jaccard values are more than the first threshold, to the reference block
Coding；
The reference block after coding includes：Position offset (g=(cx, cy, w, h)), target fractional (p ∈ [0,1]) and label
(x ∈ { 0,1 }),
The calculation formula of position offset is：
Cx=(cxg-cxd)/wd
Cy=(cyg-cyd)/hd
Wherein, (cx, cy) indicates the center of the reference block, and (w, h) represents the width and height of the reference block；Subscript indexes g
Indicate that the reference block, d indicate the first acquiescence frame, cxg、cyg、wg、hgIt is directly read from the training image；
L04：First convolution algorithm without activation primitive is carried out to the fisrt feature figure, obtains four positions of the first acquiescence frame
Four position offsets of offset, the first acquiescence frame are used for the location prediction of the target；
Second convolution algorithm without activation primitive is carried out to the fisrt feature figure, obtains three classification confidence levels, the classification
Confidence level is used for the class prediction of the target；
It is handled using classification confidence level described in softmax function pairs, obtains the general of the prediction classification of the first acquiescence frame
Rate.
5. according to the method described in claim 4, it is characterized in that, the loss function of the lightweight SSD models is：
Wherein, N is the quantity of matched first default boundary frame, if N=0, L 0；
Position offset loss is defined as：
Wherein, the index for the first acquiescence frame that i is that label is 1, and l and g be respectively the training image prediction block offset and
The offset of reference block after coding；
Target fractional loss is calculated by binary cross entropy：
Wherein, p is the target fractional of the training image prediction block, and x ∈ { 0,1 } are the references after the reference block coding
Label.
6. according to the method described in claim 3, it is characterized in that, the third module of the lightweight SSD models is through more
It is connected with the 4th module after the process of convolution of hole.
7. according to the method described in claim 2, it is characterized in that, the step S22 is specifically included：
M1：The lightweight SSD models after load training, light weight of the pretreated image information after the training
Second feature figure is obtained after the processing of grade SSD models；
M2：Calculate the second acquiescence frame of the second feature figure；
M3：To the reference block decoding after coding；
Give tacit consent to frame and reference block fusion by described second, obtains third acquiescence frame；
The calculation formula of the fusion is：
xc=loc [0 [× wref×sacling[0[+xref
yc=loc [1 [× href×sacling[1[+yref
W=wref×e(loc[2[×sacling[2[)
H=href×e(loc[3[×sacling[3[)
Wherein, xc、ycThe center point coordinate of frame is given tacit consent to for the third, w, h are the width and height that the third gives tacit consent to frame, and loc is volume
Four position offsets of the second acquiescence frame that product obtains, xref、yref、wref、hrefIt is calculated according to ratio value
Four position offsets of the prediction block of the pretreated image information, scaling is default parameters；
M4：Third acquiescence frame is screened, the position offset that the third filtered out is given tacit consent to frame is labeled in institute
It states in pretreated image information, then exports the image information of the mark.
8. according to the method described in claim 1-7, which is characterized in that the target to be identified includes personnel and vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810507764.8A CN108764115B (en) | 2018-05-24 | 2018-05-24 | Truck danger reminding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810507764.8A CN108764115B (en) | 2018-05-24 | 2018-05-24 | Truck danger reminding method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108764115A true CN108764115A (en) | 2018-11-06 |
| CN108764115B CN108764115B (en) | 2021-12-14 |
Family
ID=64005140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810507764.8A Expired - Fee Related CN108764115B (en) | 2018-05-24 | 2018-05-24 | Truck danger reminding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764115B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109506628A (en) * | 2018-11-29 | 2019-03-22 | 东北大学 | Object distance measuring method under a kind of truck environment based on deep learning |
| CN109740463A (en) * | 2018-12-21 | 2019-05-10 | 沈阳建筑大学 | A kind of object detection method under vehicle environment |
| CN110459074A (en) * | 2019-08-12 | 2019-11-15 | 东北大学 | Mine truck collision early warning system and method based on radar and WIFI |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN111046734A (en) * | 2019-11-12 | 2020-04-21 | 重庆邮电大学 | Multi-modal fusion sight line estimation method based on expansion convolution |
| CN112183460A (en) * | 2020-10-20 | 2021-01-05 | 武汉光谷联合集团有限公司 | Method and device for intelligently identifying environmental sanitation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530865A (en) * | 2012-07-05 | 2014-01-22 | 现代自动车株式会社 | Apparatus and methods for use in detecting three-dimensional object around the image of the vehicle |
| CN103679838A (en) * | 2012-09-20 | 2014-03-26 | 上海科沁机电有限公司 | Vehicle monitoring system and method |
| CN103863190A (en) * | 2012-12-12 | 2014-06-18 | 现代自动车株式会社 | Apparatus and method for providing information of blind spot |
| CN105711501A (en) * | 2016-04-19 | 2016-06-29 | 深圳市航盛电子股份有限公司 | Car look-around camera-based car monitoring method and system in dead zone |
| CN105835878A (en) * | 2015-01-29 | 2016-08-10 | 丰田自动车工程及制造北美公司 | Autonomous vehicle operation in obstructed occupant view and sensor detection environments |
| CN107487333A (en) * | 2016-06-13 | 2017-12-19 | 福特全球技术公司 | Blind area detecting system and method |
| CN107487258A (en) * | 2016-06-13 | 2017-12-19 | 福特全球技术公司 | Blind area detecting system and method |
| CN109145769A (en) * | 2018-08-01 | 2019-01-04 | 辽宁工业大学 | The target detection network design method of blending image segmentation feature |
-
2018
- 2018-05-24 CN CN201810507764.8A patent/CN108764115B/en not_active Expired - Fee Related
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530865A (en) * | 2012-07-05 | 2014-01-22 | 现代自动车株式会社 | Apparatus and methods for use in detecting three-dimensional object around the image of the vehicle |
| CN103679838A (en) * | 2012-09-20 | 2014-03-26 | 上海科沁机电有限公司 | Vehicle monitoring system and method |
| CN103863190A (en) * | 2012-12-12 | 2014-06-18 | 现代自动车株式会社 | Apparatus and method for providing information of blind spot |
| CN105835878A (en) * | 2015-01-29 | 2016-08-10 | 丰田自动车工程及制造北美公司 | Autonomous vehicle operation in obstructed occupant view and sensor detection environments |
| CN105711501A (en) * | 2016-04-19 | 2016-06-29 | 深圳市航盛电子股份有限公司 | Car look-around camera-based car monitoring method and system in dead zone |
| CN107487333A (en) * | 2016-06-13 | 2017-12-19 | 福特全球技术公司 | Blind area detecting system and method |
| CN107487258A (en) * | 2016-06-13 | 2017-12-19 | 福特全球技术公司 | Blind area detecting system and method |
| CN109145769A (en) * | 2018-08-01 | 2019-01-04 | 辽宁工业大学 | The target detection network design method of blending image segmentation feature |
Non-Patent Citations (3)
| Title |
|---|
| ROSS GIRSHICK 等: "Rich feature hierarchies for accurate object detection and semantic segmentation", 《ARXIV:1311.2524V5 [CS.CV]》 * |
| WEI LIU 等: "SSD: Single Shot MultiBox Detector", 《ARXIV:1512.02325V5 [CS.CV]》 * |
| 张天琦: "基于深度学习的行人流量统计算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109506628A (en) * | 2018-11-29 | 2019-03-22 | 东北大学 | Object distance measuring method under a kind of truck environment based on deep learning |
| CN109740463A (en) * | 2018-12-21 | 2019-05-10 | 沈阳建筑大学 | A kind of object detection method under vehicle environment |
| CN110459074A (en) * | 2019-08-12 | 2019-11-15 | 东北大学 | Mine truck collision early warning system and method based on radar and WIFI |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN111046734A (en) * | 2019-11-12 | 2020-04-21 | 重庆邮电大学 | Multi-modal fusion sight line estimation method based on expansion convolution |
| CN111046734B (en) * | 2019-11-12 | 2022-10-18 | 重庆邮电大学 | Multi-modal fusion sight line estimation method based on expansion convolution |
| CN112183460A (en) * | 2020-10-20 | 2021-01-05 | 武汉光谷联合集团有限公司 | Method and device for intelligently identifying environmental sanitation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108764115B (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764115A (en) | A kind of truck danger based reminding method | |
| CN107871119B (en) | Target detection method based on target space knowledge and two-stage prediction learning | |
| CN109740463A (en) | A kind of object detection method under vehicle environment | |
| CN104751634B (en) | The integrated application method of freeway tunnel driving image acquisition information | |
| CN103761529B (en) | A kind of naked light detection method and system based on multicolour model and rectangular characteristic | |
| CN109508710A (en) | Based on the unmanned vehicle night-environment cognitive method for improving YOLOv3 network | |
| CN104766071B (en) | A kind of traffic lights fast algorithm of detecting applied to pilotless automobile | |
| CN103886308B (en) | A kind of pedestrian detection method of use converging channels feature and soft cascade grader | |
| CN108009524A (en) | A kind of method for detecting lane lines based on full convolutional network | |
| CN108108761A (en) | A kind of rapid transit signal lamp detection method based on depth characteristic study | |
| CN103824081B (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
| CN109506628A (en) | Object distance measuring method under a kind of truck environment based on deep learning | |
| CN107808133A (en) | Oil-gas pipeline safety monitoring method, system and software memory based on unmanned plane line walking | |
| CN107274445A (en) | A kind of image depth estimation method and system | |
| CN104408424B (en) | A kind of multi signal lamp recognition methods based on image procossing | |
| CN110245663A (en) | One kind knowing method for distinguishing for coil of strip information | |
| CN105678218B (en) | A kind of method of mobile object classification | |
| CN110232316A (en) | A kind of vehicle detection and recognition method based on improved DSOD model | |
| CN109117788A (en) | A kind of public transport compartment crowding detection method merging ResNet and LSTM | |
| CN107301378A (en) | The pedestrian detection method and system of Multi-classifers integrated in image | |
| CN108304786A (en) | A kind of pedestrian detection method based on binaryzation convolutional neural networks | |
| CN104143077B (en) | Pedestrian target search method and system based on image | |
| CN109935080A (en) | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time | |
| CN107025796A (en) | Automobile assistant driving vision early warning system and its method for early warning | |
| CN113327255A (en) | Power transmission line inspection image processing method based on YOLOv3 detection, positioning and cutting and fine-tune |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20211214 |