CN110059582A - Driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks - Google Patents

Driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks Download PDF

Info

Publication number
CN110059582A
CN110059582A CN201910242262.1A CN201910242262A CN110059582A CN 110059582 A CN110059582 A CN 110059582A CN 201910242262 A CN201910242262 A CN 201910242262A CN 110059582 A CN110059582 A CN 110059582A
Authority
CN
China
Prior art keywords
attention
multiple dimensioned
indicate
characteristic pattern
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910242262.1A
Other languages
Chinese (zh)
Other versions
CN110059582B (en
Inventor
路小波
胡耀聪
陆明琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201910242262.1A priority Critical patent/CN110059582B/en
Publication of CN110059582A publication Critical patent/CN110059582A/en
Application granted granted Critical
Publication of CN110059582B publication Critical patent/CN110059582B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks, include the following steps: the image data set of (1) shooting driving behavior identification；(2) the driving behavior data set obtained to shooting does data enhancing and will enhance obtained sample while being included in training data；(3) neural network model, including three modules are constructed, is respectively as follows: multiple dimensioned convolution module, pays attention to power module and categorization module；(4) the multiple dimensioned attention convolutional neural networks of training；Network model is built using Pytorch Open-Source Tools, optimizes network parameter using stochastic gradient descent method；(5) multiple row convolutional neural networks are tested.Multiple dimensioned model and attention mechanism are introduced by the present invention to be used to extract the behavior fine granularity character representation with discrimination in driving behavior identification mission, can further improve driving behavior recognition accuracy.

Description

Driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks
Technical field
It is especially a kind of based on multiple dimensioned attention convolution mind the present invention relates to image procossing and mode identification technology Driving behavior recognition methods through network.
Background technique
In recent years, with the continuous improvement of scientific and technological level and living standard, automobile has come into huge numbers of families, at present state Interior car ownership reaches 3.25 hundred million, is only second to the U.S..It is many convenient that the popularizing of automobile brings to the trip of people, together When potential hidden danger also is caused to traffic safety.According to the ASSOCIATE STATISTICS in Chinese transportation portion, sent out altogether in the whole country in 2017 212846 traffic accidents have been given birth to, have caused 63093 people dead, wherein 80% or more traffic accident and the violation of driver drive Behavior is closely bound up.Thin due to traffic law consciousness, it is relatively conventional that driver plays mobile phone, the bad steerings behavior such as smoking. In real life, unsafe driver behavior seriously disperses the attention of driver, reduces reaction and movement speed, gently then causes Traffic jam, it is heavy then cause traffic accident.Therefore the research of driving behavior recognizer is to road safety management and traffic intelligence Energyization has important meaning.
(Advanced Driver Assistance System, ADAS), built-in vehicle in advanced driving assistance system The behavior state of driver can be captured by carrying camera, however the driving behavior recognition accuracy that automatically analyzes of view-based access control model compared with It is low, but still there are a series of challenges:
(1) different driving behaviors such as normal driving, off-direction disk, smoking etc. of driving belong to and drive this big row For classification, and inter-class variance of these subclasses in image level is very small, and similarity is very high on global characteristics, only certain Different from local detail feature；
(2) different drivers has a variety of different driving habits, and such as holding the mode of steering wheel, there are notable differences.This It allows for driver and is presented biggish variance within clusters in image posture, while illumination blocks etc. and also to cause difficulty to accurately identifying Degree.
Summary of the invention
Technical problem to be solved by the present invention lies in provide a kind of driving based on multiple dimensioned attention convolutional neural networks Multiple dimensioned model and attention mechanism are introduced into and are used to extract tool in driving behavior identification mission by the person's of sailing Activity recognition method There is the behavior fine granularity character representation of discrimination, can further improve driving behavior recognition accuracy.
In order to solve the above technical problems, the present invention provides a kind of driver based on multiple dimensioned attention convolutional neural networks Activity recognition method, includes the following steps:
(1) image data set of shooting driving behavior identification；
(2) the driving behavior data set obtained to shooting does data enhancing and will enhance obtained sample while being included in instruction Practice in data；
(3) neural network model, including three modules are constructed, is respectively as follows: multiple dimensioned convolution module, pays attention to power module and divide Generic module；
(4) the multiple dimensioned attention convolutional neural networks of training；Network model is built using Pytorch Open-Source Tools, is used Stochastic gradient descent method optimizes network parameter；
(5) multiple row convolutional neural networks are tested.
Preferably, in step (1), driving behavior covers 6 kinds of different driving behaviors, including C0: safe driving；C1: Off-direction disk drives；C2: it makes a phone call to drive；C3: it bows and sees the mobile phone；C4: it smokes and drives；C5: it is talked with passenger.
Preferably, in step (2), data enhancing is done to the obtained driving behavior data set of shooting and obtains enhancing Sample is included in training data simultaneously to be specifically comprised the following steps:
(21) image normalization of input is 256 × 256, randomly selected by the data enhancement methods for using random cropping 224 × 224 image block is as training sample；
(22) it using the data enhancement methods of image content-based transformation, comprising small angle rotation, mirror image plus makes an uproar and Gauss Smoothly；
(23) if including K training sample in training set, X={ χ can be denoted as1,χ2,...χN, and in training set N-th of sample can be expressed as χk={ Ik,lk, wherein IkIndicate k-th three to lead to image, having a size of for 224 × 224 × 3, lkIndicate its corresponding class label.
Preferably, in step (3), multiple dimensioned convolution module is input with original image, using the convolution kernel of different scale Image is successively filtered, excitation function of the maximum selection rule unit as each multiple dimensioned convolution block is melted with adaptive It closes layer-by-layer multi-scale information and tentatively extracts behavioural characteristic；Notice that power module refines behavioural characteristic, which passes through Learn Pixel-level weight matrix and channel level weight matrix obtains the Pixel-level conspicuousness and channel level conspicuousness of behavioural characteristic, and Behavioural characteristic is refined using the strategy of soft attention；Categorization module is by full articulamentum and softmax layers to driver's row To classify.
Preferably, in step (3), building neural network model specifically comprises the following steps:
(31) for the network frame designed using 224 × 224 × 3 original image as input, first layer is basic convolutional layer, Original image is filtered with 64 7 × 7 × 3 convolution kernels, maximum value pond layer will input dimensionality reduction at 56 × 56 × 64 Characteristic pattern is specifically expressed as follows:
xbc=σ (I*W+b) (1)
Fbc=down (xbc) (2)
Wherein * indicates convolution operation, θbc={ W, b } indicates that basic convolutional layer weight and threshold parameter, σ () indicate ReLU Excitation function, down () indicate the operation of maximum value pondization, FbcIndicate the output characteristic pattern of basic convolutional layer；
Remaining convolutional layer is stacked by 8 multiple dimensioned convolution blocks, multiple dimensioned convolution block by 4 kinds of different scales (1 × 1, 3 × 3,5 × 5,7 × 7) filtering core the parallel combined forms, and each multiple dimensioned convolution block is realized certainly by maximum selection rule unit The multi-scale information of adaptation merges, and inhibits gradient explosion and gradient diffusing phenomenon using residual error learning method；
First of multiple dimensioned convolution block carries out convolution to the characteristic pattern that a upper block exports, and may be expressed as:
x(l)=F(l-1)*W(l)+b(l), l=1,2 ..., 8 } (3)
WhereinIndicate the weight and threshold parameter of first of multiple dimensioned convolution block, F(l-1)Indicate upper one The output of secondary multiple dimensioned convolution block, x(l)Indicate first piece of multiple dimensioned convolution characteristic pattern, first multiple dimensioned convolution block it is defeated Enter the output characteristic pattern for basic convolution；
For given lot sample sheet, first piece of trellis diagram output can be denoted asThe phase of batch data Prestige and variance can be denoted as:
Wherein K indicates the quantity of lot sample sheet,Indicate the multiple dimensioned convolution output on first piece of k-th of sample, E () The expectation and variance of lot sample sheet are respectively indicated with Var ()；
Feature after criticizing standardization can indicate are as follows:
Wherein ε, which takes, is similar to 0 normal number to improve the generalization ability of characteristic criterion, and α and β indicate that scale and offset become Parameter is changed,Feature after indicating standardization；
Maximum selection rule unit is used to the adaptive multiple dimensioned convolution characteristic pattern of fusion, first piece of standardization characteristic value It can be expressed asWherein (c, i, j) indicates that the channel of standardization feature and coordinate, scale have recorded corresponding Convolution kernel size (1 × 1,3 × 3,5 × 5,7 × 7), the output of maximum selection rule unit can indicate are as follows:
The wherein output y of maximum selection rule unit(l)It is different scale characteristic pattern in the position (c, i, j) in the value of (c, i, j) On maximum value；
The output of multiple dimensioned convolution block can indicate are as follows:
F(l)=σ (F(l-1)+y(l)) (8)
Wherein F(l-1)And F(l)The output and first piece of output of a block are respectively indicated, σ () indicates ReLU excitation Function；
By 8 multiple dimensioned convolution blocks, the output of multiple dimensioned convolution module can be denoted as F(8), the size of characteristic pattern is 7 × 7 ×512；
(32) pay attention to power module with the characteristic pattern F of the last one multiple dimensioned convolution block(8)As input, attention mechanism is drawn Wire guide network is concerned about conspicuousness characterization to realize that feature refines；
Pixel-level attention mechanism and channel level attention mechanism are used in a model, and wherein pixel attention layer is with convolution spy Sign figure is weighed the importance of each pixel in characteristic pattern by one pixel weight matrix of study, can indicated as input Are as follows:
αp=tanh (WpaU+bpa) (9)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θpa={ Wpa,bpaIndicate weight and threshold value Parameter, tanh () indicate hyperbolic tangent function,Indicate that the Pixel-level weight matrix being calculated, the matrix are used To reflect each pixel for the significance level of Activity recognition；
The pixel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of Pixel-level weight, tool Body may be expressed as:
WhereinRepresenting matrix multiplication, PA (|) indicate reflecting from input feature vector figure to output attention characteristic pattern It penetrates, the pixel attention characteristic pattern finally exported is
Channel attention layer is using convolution characteristic pattern as input, by one channel weight matrix learning characteristic figure of study The contribution margin that each channel classifies to behavior, may be expressed as:
αc=tanh (WcaV+bca) (12)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θca={ Wca,bcaIndicate weight and threshold value Parameter, tanh () indicate hyperbolic tangent function,Indicate the channel level weight matrix being calculated, the matrix For reflecting each channel of characteristic pattern for the significance level of Activity recognition；
The channel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of channel level weight, tool Body may be expressed as:
WhereinRepresenting matrix multiplication, CA (|) indicate reflecting from input feature vector figure to output attention characteristic pattern It penetrates, the channel attention characteristic pattern finally exported is
Pixel attention and channel are carried out note that the attention finally exported to convolution characteristic pattern using the mode of parallel connection Characteristic pattern is the addition fusion of the two, be may be expressed as:
Fatt=PA (F(8))+CA(F(8)) (15)
Wherein F(8)Indicate the characteristic pattern of the last one multiple dimensioned convolution block of input, PA () and CA () are respectively indicated Pixel and channel are note that FattIndicate the attention characteristic pattern finally exported；
(33) module is composed of a full articulamentum and one softmax layers respectively, and the module is with attention feature Scheme FattAs input, last output is the probability of different driving behavior classifications；
Full articulamentum will specifically can may be used having a size of 7 × 7 × 512 attention characteristic pattern dimensionality reductions at 1000 dimensional feature vectors It indicates are as follows:
F=WfcFatt+bfc (16)
Wherein θfc={ Wbc,bbcIndicate the weight and threshold parameter of full articulamentum, f indicate 1000 dimensional features of output to Amount；
In softmax layers, output unit number is identical as behavior classification number, and output valve is softmax classifier The different classes of probability being calculated, specifically may be expressed as:
Wherein P (j) indicates that feature f belongs to the posterior probability of jth class, θcls={ Wcls,bclsWeight and threshold parameter, Score={ s1,s2,...,snIndicate softmax layer export different behavior classifications probability distribution.
Preferably, in step (4), the multiple dimensioned attention convolutional neural networks of training；It is built using Pytorch Open-Source Tools Network model, using stochastic gradient descent method optimization network parameter, using intersection loss entropy function measurement true tag and in advance The distance between result is surveyed, specifically may be expressed as:
Wherein l indicates classification true value label, and P (j) i.e. softmax layers of output indicates that the posteriority for belonging to jth classification is general Rate；
For batch data, the parameter of whole network can be lost by softmax to be optimized as supervision, specifically may be used It indicates are as follows:
Wherein | | θ | | indicate loss function regularization term, for mitigate be likely to occur in network training process it is excessively quasi- It closes.
Preferably, in step (5), multiple row convolutional neural networks are tested specifically: given driver identification Test image is normalized to 224 × 224 size as the input of multiple row fusion convolutional neural networks, passes through multiple row by image The propagated forward of converged network obtains the Activity recognition result of test image.
The invention has the benefit that (1) present invention employs multiple dimensioned convolution modules to be filtered to original image, most The Analysis On Multi-scale Features of the adaptive each convolution block of fusion of big value selecting unit；(2) present invention employs the tradeoffs of attention mechanism The channel conspicuousness and pixel significance of characteristic pattern are refined for feature and behavior fine granularity character representation.
Detailed description of the invention
Fig. 1 is the sample schematic diagram of different driving behaviors in the present invention.
Fig. 2 is that data enhance schematic diagram in the present invention.
Fig. 3 is the configuration diagram of multiple dimensioned attention convolutional neural networks model in the present invention.
Fig. 4 is the multiple dimensioned convolution block schematic diagram of the present invention.
Fig. 5 is attention schematic diagram of mechanism in the present invention.
Specific embodiment
A kind of driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks, includes the following steps:
Step 1: the image data set of shooting driving behavior identification.All images are by built-in vehicle-mounted camera in difference It is recorded under angle and different light conditions.Driving behavior data set shares 42816 pictures, covers 6 kinds of different driving rows For as shown in Figure 1, being respectively as follows:
C0: safe driving；
C1: off-direction disk drives；
C2: it makes a phone call to drive；
C3: it bows and sees the mobile phone；
C4: it smokes and drives；
C5: it is talked with passenger；
The obtained image data collection of shooting be divided into training set and test set respectively include 17087 trained pictures with 25729 test pictures.
Step 2: data enhancing being done to the driving behavior data set that shooting obtains and obtained sample will be enhanced while being received Enter in training data, wherein mainly include two kinds of data enhancement methods, it is specific as follows:
Step 201: using the data enhancement methods of random cropping: by the image normalization of input for 256 × 256, at random The image block of selection 224 × 224 is as training sample.
Step 202: the data enhancement methods converted using image content-based include small angle rotation, and mirror image adds and makes an uproar, Gaussian smoothing etc. as shown in Fig. 2, these enhancing samples, which are added, can be improved the anti-noise ability of algorithm, and effectively raises depth Spend the robust ability of neural network.
Step 203: if including K training sample in training set, X={ χ can be denoted as1,χ2,...χN}.And for training N-th of the sample concentrated can be expressed as χk={ Ik,lk, wherein IkIndicate k-th three to lead to image, having a size of for 224 × 224 × 3, lkIndicate its corresponding class label.
Step 3: building neural network model, designed model include three modules, are respectively as follows: multiple dimensioned convolution mould Block pays attention to power module and categorization module.The structural diagrams of network are intended to as shown in Figure 3.Wherein multiple dimensioned convolution module is with original Image is input, is successively filtered using the convolution collecting image of different scale, maximum selection rule unit is as each more rulers The excitation function of convolution block is spent, behavioural characteristic is extracted tentatively with the layer-by-layer multi-scale information of adaptive fusion.Attention mould Block refines behavioural characteristic, which obtains behavioural characteristic by study Pixel-level weight matrix and channel level weight matrix Pixel-level conspicuousness and channel level conspicuousness, and using soft attention strategy behavioural characteristic is refined.Categorization module is logical It crosses full articulamentum and softmax layers is classified to driving behavior.It is described in detail below:
Step 301: the network frame of design is rolled up based on first layer using 224 × 224 × 3 original image as input Lamination is filtered original image with 64 7 × 7 × 3 convolution kernels.Maximum value pond layer will input dimensionality reduction at 56 × 56 × 64 characteristic pattern, is specifically expressed as follows:
xbc=σ (I*W+b) (1)
Fbc=down (xbc) (2)
Wherein * indicates convolution operation, θbc={ W, b } indicates basic convolutional layer weight and threshold parameter.σ () indicates ReLU Excitation function.Down () indicates the operation of maximum value pondization, FbcIndicate the output characteristic pattern of basic convolutional layer.
Remaining convolutional layer is stacked by 8 multiple dimensioned convolution blocks.Multiple dimensioned convolution block by 4 kinds of different scales (1 × 1, 3 × 3,5 × 5,7 × 7) filtering core the parallel combined forms, and each multiple dimensioned convolution block is realized certainly by maximum selection rule unit The multi-scale information of adaptation merges, and inhibits gradient explosion and gradient diffusing phenomenon using residual error learning method.Multiple dimensioned convolution block Structural schematic diagram it is as shown in Figure 4.
Specifically, first of multiple dimensioned convolution block carries out convolution to the characteristic pattern that a upper block exports, and may be expressed as:
x(l)=F(l-1)*W(l)+b(l), l=1,2 ..., 8 } (3)
WhereinIndicate the weight and threshold parameter of first of multiple dimensioned convolution block, F(l-1)Indicate upper one The output of secondary multiple dimensioned convolution block, x(l)Indicate first piece of multiple dimensioned convolution characteristic pattern.Particularly, first multiple dimensioned convolution The input of block is the output characteristic pattern of basic convolution.
It criticizes standardization to follow after each convolution operation, to increase the generalization of e-learning.For given lot sample This, first piece of trellis diagram output can be denoted asThe expectation of batch data and variance can be denoted as:
Wherein K indicates the quantity of lot sample sheet,Indicate the multiple dimensioned convolution output on first piece of k-th of sample, E () The expectation and variance of lot sample sheet are respectively indicated with Var ().
Feature after criticizing standardization can indicate are as follows:
Wherein ε, which takes, is similar to 0 normal number to improve the generalization ability of characteristic criterion.α and β indicates that scale and offset become Parameter is changed,Feature after indicating standardization.
Maximum selection rule unit is used to the adaptive multiple dimensioned convolution characteristic pattern of fusion.First piece of standardization characteristic value It can be expressed asWherein (c, i, j) indicates that the channel of standardization feature and coordinate, scale have recorded corresponding Convolution kernel size (1 × 1,3 × 3,5 × 5,7 × 7), the output of maximum selection rule unit can indicate are as follows:
The wherein output y of maximum selection rule unit(l)It is different scale characteristic pattern in the position (c, i, j) in the value of (c, i, j) On maximum value.
Residual error study is introduced in multiple dimensioned convolution block for improving the convergence capabilities of network.Residual unit uses The identical mapping of an input is added in the output of the connection type of shortcut, as residual unit.Multiple dimensioned convolution block it is defeated It can indicate out are as follows:
F(l)=σ (F(l-1)+y(l)) (8)
Wherein F(l-1)And F(l)The output and first piece of output of a block are respectively indicated, σ () indicates ReLU excitation Function.
By 8 multiple dimensioned convolution blocks, the output of multiple dimensioned convolution module can be denoted as F(8), the size of characteristic pattern is 7 × 7 ×512。
Step 302: paying attention to power module with the characteristic pattern F of the last one multiple dimensioned convolution block(8)As input, attention machine System guidance network is concerned about conspicuousness characterization to realize that feature refines, and specific attention model can automatically emphasize that part is thin Information is saved, inhibits the global context information of redundancy, attention model used in the design is as shown in Figure 5.
Pixel-level attention mechanism and channel level attention mechanism are used in a model.Wherein pixel attention layer is with convolution spy Sign figure is weighed the importance of each pixel in characteristic pattern by one pixel weight matrix of study, can indicated as input Are as follows:
αp=tanh (WpaU+bpa) (9)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θpa={ Wpa,bpaIndicate weight and threshold value Parameter, tanh () indicate hyperbolic tangent function,Indicate that the Pixel-level weight matrix being calculated, the matrix are used To reflect each pixel for the significance level of Activity recognition.
The pixel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of Pixel-level weight, tool Body may be expressed as:
WhereinRepresenting matrix multiplication, PA (|) indicate reflecting from input feature vector figure to output attention characteristic pattern It penetrates, the pixel attention characteristic pattern finally exported is
Similarly, attention layer in channel passes through one channel weight matrix study of study using convolution characteristic pattern as input The contribution margin that each channel classifies to behavior in characteristic pattern, may be expressed as:
αc=tanh (WcaV+bca) (12)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θca={ Wca,bcaIndicate weight and threshold value Parameter, tanh () indicate hyperbolic tangent function,Indicate the channel level weight matrix being calculated, the matrix For reflecting each channel of characteristic pattern for the significance level of Activity recognition.
The channel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of channel level weight, tool Body may be expressed as:
WhereinRepresenting matrix multiplication, CA (|) indicate reflecting from input feature vector figure to output attention characteristic pattern It penetrates, the channel attention characteristic pattern finally exported is
Pixel attention and channel are carried out note that the attention finally exported to convolution characteristic pattern using the mode of parallel connection Characteristic pattern is the addition fusion of the two, be may be expressed as:
Fatt=PA (F(8))+CA(F(8)) (15)
Wherein F(8)Indicate the characteristic pattern of the last one multiple dimensioned convolution block of input, PA () and CA () are respectively indicated Pixel and channel are note that FattIndicate the attention characteristic pattern finally exported.
Step 303: module is composed of a full articulamentum and one softmax layers respectively, and the module is with attention Characteristic pattern FattAs input, last output is the probability of different driving behavior classifications.
Full articulamentum will specifically can may be used having a size of 7 × 7 × 512 attention characteristic pattern dimensionality reductions at 1000 dimensional feature vectors It indicates are as follows:
F=WfcFatt+bfc (16)
Wherein θfc={ Wbc,bbcIndicate the weight and threshold parameter of full articulamentum, f indicate 1000 dimensional features of output to Amount.
In softmax layers, output unit number is identical as behavior classification number, and output valve is softmax classifier The different classes of probability being calculated, specifically may be expressed as:
Wherein P (j) indicates that feature f belongs to the posterior probability of jth class, θcls={ Wcls,bclsWeight and threshold parameter, Score={ s1,s2,...,snIndicate softmax layer export different behavior classifications probability distribution.
Step 4: the multiple dimensioned attention convolutional neural networks of training.Network model is built using Pytorch Open-Source Tools.Make Optimize network parameter with stochastic gradient descent method.
The distance between true tag and prediction result are measured using loss entropy function is intersected, specifically may be expressed as:
Wherein l indicates classification true value label, and P (j) i.e. softmax layers of output indicates that the posteriority for belonging to jth classification is general Rate.
For batch data, the parameter of whole network can be lost by softmax to be optimized as supervision, specifically may be used It indicates are as follows:
Wherein | | θ | | indicate loss function regularization term, for mitigate be likely to occur in network training process it is excessively quasi- It closes.
Step 5: multiple row convolutional neural networks are tested.A given driver identifies image, and test image is returned One size for turning to 224 × 224 merges the input of convolutional neural networks as multiple row, passes through the propagated forward of multiple row converged network Obtain the Activity recognition result of test image.

Claims (7)

1. a kind of driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks, which is characterized in that including such as Lower step:
(1) image data set of shooting driving behavior identification；
(2) the driving behavior data set obtained to shooting does data enhancing and will enhance obtained sample while being included in trained number In；
(3) neural network model, including three modules are constructed, is respectively as follows: multiple dimensioned convolution module, pays attention to power module and classification mould Block；
(4) the multiple dimensioned attention convolutional neural networks of training；Network model is built using Pytorch Open-Source Tools, using random Gradient descent method optimizes network parameter；
(5) multiple row convolutional neural networks are tested.
2. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (1), driving behavior covers 6 kinds of different driving behaviors, including C0: safe driving；C1: off-direction Disk drives；C2: it makes a phone call to drive；C3: it bows and sees the mobile phone；C4: it smokes and drives；C5: it is talked with passenger.
3. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (2), data enhancing is done to the obtained driving behavior data set of shooting and the sample that obtains enhancing simultaneously It is included in training data and specifically comprises the following steps:
(21) image normalization of input is 256 × 256 by the data enhancement methods for using random cropping, randomly select 224 × 224 image block is as training sample；
(22) it using the data enhancement methods of image content-based transformation, comprising small angle rotation, mirror image plus makes an uproar flat with Gauss It is sliding；
(23) if including K training sample in training set, it is denoted as X={ χ1,χ2,...χN, and for n-th in training set Sample is expressed as χk={ Ik,lk, wherein IkIt indicates k-th three to lead to image, having a size of for 224 × 224 × 3, lkIndicate that its is right The class label answered.
4. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (3), multiple dimensioned convolution module with original image be input, using different scale convolution collecting image into Row successively filtering, excitation function of the maximum selection rule unit as each multiple dimensioned convolution block, layer-by-layer with adaptive fusion Multi-scale information tentatively extracts behavioural characteristic；Notice that power module refines behavioural characteristic, which passes through study pixel Grade weight matrix and channel level weight matrix obtain the Pixel-level conspicuousness and channel level conspicuousness of behavioural characteristic, and use soft note The strategy of meaning refines behavioural characteristic；Categorization module divides driving behavior by full articulamentum and softmax layers Class.
5. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (3), building neural network model specifically comprises the following steps:
(31) network frame designed is using 224 × 224 × 3 original image as input, and first layer is basic convolutional layer, with 64 A 7 × 7 × 3 convolution kernel is filtered original image, and maximum value pond layer will input dimensionality reduction into 56 × 56 × 64 feature Figure, is specifically expressed as follows:
xbc=σ (I*W+b) (1)
Fbc=down (xbc) (2)
Wherein * indicates convolution operation, θbc={ W, b } indicates that basic convolutional layer weight and threshold parameter, σ () indicate ReLU excitation Function, down () indicate the operation of maximum value pondization, FbcIndicate the output characteristic pattern of basic convolutional layer；
Remaining convolutional layer is stacked by 8 multiple dimensioned convolution blocks, multiple dimensioned convolution block by 4 kinds of different scales (1 × 1,3 × 3,5 × 5,7 × 7) filtering core the parallel combined forms, and each multiple dimensioned convolution block is realized adaptive by maximum selection rule unit Multi-scale information fusion, use residual error learning method inhibit gradient explosion and gradient diffusing phenomenon；
First of multiple dimensioned convolution block carries out convolution to the characteristic pattern that a upper block exports, and indicates are as follows:
x(l)=F(l-1)*W(l)+b(l), l=1,2 ..., 8 } (3)
WhereinIndicate the weight and threshold parameter of first of multiple dimensioned convolution block, F(l-1)Indicate last more The output of scale convolution block, x(l)Indicate first piece of multiple dimensioned convolution characteristic pattern, the input of first multiple dimensioned convolution block is The output characteristic pattern of basic convolution；
For given lot sample sheet, first piece of trellis diagram output is denoted asThe expectation and variance of batch data It is denoted as:
Wherein K indicates the quantity of lot sample sheet,Indicate on first piece of k-th of sample multiple dimensioned convolution output, E () and Var () respectively indicates the expectation and variance of lot sample sheet；
Character representation after criticizing standardization are as follows:
Wherein ε, which takes, is similar to 0 normal number to improve the generalization ability of characteristic criterion, and α and β indicate scale and offset transformation ginseng Number,Feature after indicating standardization；
Maximum selection rule unit is used to the adaptive multiple dimensioned convolution characteristic pattern of fusion, and first piece of standardization characteristic value indicates ForWherein (c, i, j) indicates that the channel of standardization feature and coordinate, scale have recorded corresponding convolution kernel ruler The output of very little (1 × 1,3 × 3,5 × 5,7 × 7), maximum selection rule unit indicates are as follows:
The wherein output y of maximum selection rule unit(l)It is different scale characteristic pattern on the position (c, i, j) in the value of (c, i, j) Maximum value；
The output of multiple dimensioned convolution block indicates are as follows:
F(l)=σ (F(l-1)+y(l)) (8)
Wherein F(l-1)And F(l)The output and first piece of output of a block are respectively indicated, σ () indicates ReLU excitation function；
By 8 multiple dimensioned convolution blocks, the output of multiple dimensioned convolution module is denoted as F(8), the size of characteristic pattern is 7 × 7 × 512；
(32) pay attention to power module with the characteristic pattern F of the last one multiple dimensioned convolution block(8)As input, attention mechanism guiding net Network is concerned about conspicuousness characterization to realize that feature refines；
Pixel-level attention mechanism and channel level attention mechanism are used in a model, and wherein pixel attention layer is with convolution characteristic pattern As input, the importance of each pixel in characteristic pattern is weighed by one pixel weight matrix of study, is indicated are as follows:
αp=tanh (WpaU+bpa) (9)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θpa={ Wpa,bpaIndicate weight and threshold value ginseng Number, tanh () indicate hyperbolic tangent function,Indicate that the Pixel-level weight matrix being calculated, the matrix are used to Reflect each pixel for the significance level of Activity recognition；
The pixel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of Pixel-level weight, specific table It is shown as:
WhereinRepresenting matrix multiplication, PA (|) indicate one from input feature vector figure to output attention characteristic pattern mapping, most The pixel attention characteristic pattern exported afterwards is
Channel attention layer is using convolution characteristic pattern as input, by each in one channel weight matrix learning characteristic figure of study The contribution margin that a channel classifies to behavior indicates are as follows:
αc=tanh (WcaV+bca) (12)
WhereinFor the Two-Dimensional Moment array form of input feature vector figure, θca={ Wca,bcaIndicate weight and threshold parameter, Tanh () indicates hyperbolic tangent function,Indicate that the channel level weight matrix being calculated, the matrix are used to anti- The each channel of characteristic pattern is reflected for the significance level of Activity recognition；
The channel attention characteristic pattern of final output is the convolution characteristic pattern inputted and the matrix multiple of channel level weight, specific table It is shown as:
WhereinRepresenting matrix multiplication, CA (|) indicate one from input feature vector figure to output attention characteristic pattern mapping, most The channel attention characteristic pattern exported afterwards is
Pixel attention and channel are carried out note that the attention feature finally exported to convolution characteristic pattern using the mode of parallel connection Figure is the addition fusion of the two, is indicated are as follows:
Fatt=PA (F(8))+CA(F(8)) (15)
Wherein F(8)Indicate the characteristic pattern of the last one multiple dimensioned convolution block of input, PA () and CA () respectively indicate pixel With channel note that FattIndicate the attention characteristic pattern finally exported；
(33) module is composed of a full articulamentum and one softmax layers respectively, and the module is with attention characteristic pattern Fatt As input, last output is the probability of different driving behavior classifications；
Full articulamentum will be embodied as having a size of 7 × 7 × 512 attention characteristic pattern dimensionality reductions at 1000 dimensional feature vectors:
F=WfcFatt+bfc (16)
Wherein θfc={ Wbc,bbcIndicate that the weight and threshold parameter of full articulamentum, f indicate 1000 dimensional feature vectors of output；
In softmax layers, output unit number is identical as behavior classification number, and output valve is softmax classifier calculated Obtained different classes of probability, is embodied as:
Wherein P (j) indicates that feature f belongs to the posterior probability of jth class, θcls={ Wcls,bclsWeight and threshold parameter, score ={ s1,s2,...,snIndicate softmax layer export different behavior classifications probability distribution.
6. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (4), the multiple dimensioned attention convolutional neural networks of training；Network mould is built using Pytorch Open-Source Tools Type optimizes network parameter using stochastic gradient descent method, measures true tag and prediction result using loss entropy function is intersected The distance between, it is embodied as:
Wherein l indicates classification true value label, and P (j) i.e. softmax layers of output indicates the posterior probability for belonging to jth classification；
For batch data, the parameter of whole network is lost by softmax to be optimized as supervision, is embodied as:
Wherein | | θ | | the regularization term of loss function is indicated, for mitigating the over-fitting being likely to occur in network training process.
7. the driving behavior recognition methods as described in claim 1 based on multiple dimensioned attention convolutional neural networks, special Sign is, in step (5), tests multiple row convolutional neural networks specifically: a given driver identifies image, will survey Attempt input of the size as being normalized to 224 × 224 as multiple row fusion convolutional neural networks, passes through multiple row converged network The Activity recognition result of propagated forward acquisition test image.
CN201910242262.1A 2019-03-28 2019-03-28 Driver behavior identification method based on multi-scale attention convolution neural network Active CN110059582B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910242262.1A CN110059582B (en) 2019-03-28 2019-03-28 Driver behavior identification method based on multi-scale attention convolution neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910242262.1A CN110059582B (en) 2019-03-28 2019-03-28 Driver behavior identification method based on multi-scale attention convolution neural network

Publications (2)

Publication Number Publication Date
CN110059582A true CN110059582A (en) 2019-07-26
CN110059582B CN110059582B (en) 2023-04-07

Family

ID=67317536

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910242262.1A Active CN110059582B (en) 2019-03-28 2019-03-28 Driver behavior identification method based on multi-scale attention convolution neural network

Country Status (1)

Country Link
CN (1) CN110059582B (en)

Cited By (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110393519A (en) * 2019-08-19 2019-11-01 广州视源电子科技股份有限公司 Analysis method, device, storage medium and the processor of electrocardiosignal
CN110688986A (en) * 2019-10-16 2020-01-14 南京林业大学 Attention branch guided 3D convolution behavior recognition network method
CN110728219A (en) * 2019-09-29 2020-01-24 天津大学 3D face generation method based on multi-column multi-scale graph convolution neural network
CN110751957A (en) * 2019-09-25 2020-02-04 电子科技大学 Speech enhancement method using stacked multi-scale modules
CN110781814A (en) * 2019-10-24 2020-02-11 中国民用航空总局第二研究所 Signal classification method, device and medium based on Gaussian mixture neural network model
CN110796109A (en) * 2019-11-05 2020-02-14 哈尔滨理工大学 Driver distraction behavior identification method based on model fusion
CN110807734A (en) * 2019-10-30 2020-02-18 河南大学 SAR image super-resolution reconstruction method
CN110991219A (en) * 2019-10-11 2020-04-10 东南大学 Behavior identification method based on two-way 3D convolutional network
CN111046962A (en) * 2019-12-16 2020-04-21 中国人民解放军战略支援部队信息工程大学 Sparse attention-based feature visualization method and system for convolutional neural network model
CN111079795A (en) * 2019-11-21 2020-04-28 西安工程大学 Image classification method based on CNN (content-centric networking) fragment multi-scale feature fusion
CN111178304A (en) * 2019-12-31 2020-05-19 江苏省测绘研究所 High-resolution remote sensing image pixel level interpretation method based on full convolution neural network
CN111208821A (en) * 2020-02-17 2020-05-29 李华兰 Automobile automatic driving control method and device, automatic driving device and system
CN111242168A (en) * 2019-12-31 2020-06-05 浙江工业大学 Human skin image lesion classification method based on multi-scale attention features
CN111402274A (en) * 2020-04-14 2020-07-10 上海交通大学医学院附属上海儿童医学中心 Processing method, model and training method for magnetic resonance left ventricle image segmentation
CN111414932A (en) * 2020-01-07 2020-07-14 北京航空航天大学 Classification identification and fault detection method for multi-scale signals of aircraft
CN111461039A (en) * 2020-04-07 2020-07-28 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111460892A (en) * 2020-03-02 2020-07-28 五邑大学 Electroencephalogram mode classification model training method, classification method and system
CN111507281A (en) * 2020-04-21 2020-08-07 中山大学中山眼科中心 Behavior recognition system, device and method based on head movement and gaze behavior data
CN111553500A (en) * 2020-05-11 2020-08-18 北京航空航天大学 Railway traffic contact net inspection method based on attention mechanism full convolution network
CN111563468A (en) * 2020-05-13 2020-08-21 电子科技大学 Driver abnormal behavior detection method based on attention of neural network
CN111582044A (en) * 2020-04-15 2020-08-25 华南理工大学 Face recognition method based on convolutional neural network and attention model
CN111860427A (en) * 2020-07-30 2020-10-30 重庆邮电大学 Driving distraction identification method based on lightweight class eight-dimensional convolutional neural network
CN112215241A (en) * 2020-10-20 2021-01-12 西安交通大学 Image feature extraction device based on small sample learning
CN112257601A (en) * 2020-10-22 2021-01-22 福州大学 Fine-grained vehicle identification method based on data enhancement network of weak supervised learning
CN112307847A (en) * 2019-08-01 2021-02-02 复旦大学 Multi-scale attention pedestrian re-recognition deep learning system based on guidance
CN112347977A (en) * 2020-11-23 2021-02-09 深圳大学 Automatic detection method, storage medium and device for induced pluripotent stem cells
CN112464765A (en) * 2020-09-10 2021-03-09 天津师范大学 Safety helmet detection algorithm based on single-pixel characteristic amplification and application thereof
CN112527915A (en) * 2020-11-17 2021-03-19 北京科技大学 Linear cultural heritage knowledge graph construction method, system, computing device and medium
CN112613405A (en) * 2020-12-23 2021-04-06 电子科技大学 Method for recognizing actions at any visual angle
CN112733652A (en) * 2020-12-31 2021-04-30 深圳赛安特技术服务有限公司 Image target identification method and device, computer equipment and readable storage medium
CN112800834A (en) * 2020-12-25 2021-05-14 温州晶彩光电有限公司 Method and system for positioning colorful spot light based on kneeling behavior identification
CN112800871A (en) * 2021-01-13 2021-05-14 南京邮电大学 Automatic driving image recognition method based on attention mechanism and relation network
CN112837360A (en) * 2021-01-07 2021-05-25 北京百度网讯科技有限公司 Depth information processing method, apparatus, device, storage medium, and program product
CN113033448A (en) * 2021-04-02 2021-06-25 东北林业大学 Remote sensing image cloud-removing residual error neural network system, method and equipment based on multi-scale convolution and attention and storage medium
CN113029155A (en) * 2021-04-02 2021-06-25 杭州申昊科技股份有限公司 Robot automatic navigation method and device, electronic equipment and storage medium
CN113095479A (en) * 2021-03-22 2021-07-09 北京工业大学 Method for extracting ice-below-layer structure based on multi-scale attention mechanism
CN113283338A (en) * 2021-05-25 2021-08-20 湖南大学 Method, device and equipment for identifying driving behavior of driver and readable storage medium
CN113281029A (en) * 2021-06-09 2021-08-20 重庆大学 Rotating machinery fault diagnosis method and system based on multi-scale network structure
CN113516028A (en) * 2021-04-28 2021-10-19 南通大学 Human body abnormal behavior identification method and system based on mixed attention mechanism
CN113537003A (en) * 2021-07-02 2021-10-22 安阳工学院 Method and device for visually detecting external environment of vehicle for assisting driving
CN113642571A (en) * 2021-07-12 2021-11-12 中国海洋大学 Fine-grained image identification method based on saliency attention mechanism
CN113642634A (en) * 2021-08-12 2021-11-12 南京邮电大学 Shadow detection method based on mixed attention
CN113657534A (en) * 2021-08-24 2021-11-16 北京经纬恒润科技股份有限公司 Classification method and device based on attention mechanism
CN113762251A (en) * 2021-08-17 2021-12-07 慧影医疗科技（北京）有限公司 Target classification method and system based on attention mechanism
CN113780385A (en) * 2021-08-30 2021-12-10 武汉理工大学 Driving risk monitoring method based on attention mechanism
CN113989862A (en) * 2021-10-12 2022-01-28 天津大学 Texture recognition platform based on embedded system
CN114419558A (en) * 2022-03-31 2022-04-29 华南理工大学 Fire video image identification method, fire video image identification system, computer equipment and storage medium
CN114639169A (en) * 2022-03-28 2022-06-17 合肥工业大学 Human body action recognition system based on attention mechanism feature fusion and position independence
CN114782931A (en) * 2022-04-22 2022-07-22 电子科技大学 Driving behavior classification method for improved MobileNetv2 network
CN114818991A (en) * 2022-06-22 2022-07-29 西南石油大学 Running behavior identification method based on convolutional neural network and acceleration sensor
CN115082698A (en) * 2022-06-28 2022-09-20 华南理工大学 Distracted driving behavior detection method based on multi-scale attention module
CN115432331A (en) * 2022-10-10 2022-12-06 浙江绿达智能科技有限公司 Intelligent classification dustbin
CN115964360A (en) * 2023-03-14 2023-04-14 山东省地震工程研究院 Earthquake safety evaluation database construction method and system
CN116758631A (en) * 2023-06-13 2023-09-15 孟冠宇 Big data driven behavior intelligent analysis method and system
CN116977969A (en) * 2023-08-11 2023-10-31 中国矿业大学 Driver two-point pre-aiming identification method based on convolutional neural network
CN117173422A (en) * 2023-08-07 2023-12-05 广东第二师范学院 Fine granularity image recognition method based on graph fusion multi-scale feature learning
US11886199B2 (en) 2021-10-13 2024-01-30 Toyota Motor Engineering & Manufacturing North America, Inc. Multi-scale driving environment prediction with hierarchical spatial temporal attention
CN117576666A (en) * 2023-11-17 2024-02-20 合肥工业大学 Dangerous driving behavior detection method based on multi-scale dynamic convolution attention weighting

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108875674A (en) * 2018-06-29 2018-11-23 东南大学 A kind of driving behavior recognition methods based on multiple row fusion convolutional neural networks
CN109284670A (en) * 2018-08-01 2019-01-29 清华大学 A kind of pedestrian detection method and device based on multiple dimensioned attention mechanism

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108875674A (en) * 2018-06-29 2018-11-23 东南大学 A kind of driving behavior recognition methods based on multiple row fusion convolutional neural networks
CN109284670A (en) * 2018-08-01 2019-01-29 清华大学 A kind of pedestrian detection method and device based on multiple dimensioned attention mechanism

Cited By (94)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112307847A (en) * 2019-08-01 2021-02-02 复旦大学 Multi-scale attention pedestrian re-recognition deep learning system based on guidance
CN110393519A (en) * 2019-08-19 2019-11-01 广州视源电子科技股份有限公司 Analysis method, device, storage medium and the processor of electrocardiosignal
CN110393519B (en) * 2019-08-19 2022-06-24 广州视源电子科技股份有限公司 Electrocardiosignal analysis method and device, storage medium and processor
CN110751957A (en) * 2019-09-25 2020-02-04 电子科技大学 Speech enhancement method using stacked multi-scale modules
CN110728219A (en) * 2019-09-29 2020-01-24 天津大学 3D face generation method based on multi-column multi-scale graph convolution neural network
CN110728219B (en) * 2019-09-29 2023-09-26 天津大学 3D face generation method based on multi-column multi-scale graph convolution neural network
CN110991219B (en) * 2019-10-11 2024-02-06 东南大学 Behavior identification method based on two-way 3D convolution network
CN110991219A (en) * 2019-10-11 2020-04-10 东南大学 Behavior identification method based on two-way 3D convolutional network
CN110688986B (en) * 2019-10-16 2023-06-23 南京林业大学 3D convolution behavior recognition network method guided by attention branches
CN110688986A (en) * 2019-10-16 2020-01-14 南京林业大学 Attention branch guided 3D convolution behavior recognition network method
CN110781814A (en) * 2019-10-24 2020-02-11 中国民用航空总局第二研究所 Signal classification method, device and medium based on Gaussian mixture neural network model
CN110807734A (en) * 2019-10-30 2020-02-18 河南大学 SAR image super-resolution reconstruction method
CN110796109A (en) * 2019-11-05 2020-02-14 哈尔滨理工大学 Driver distraction behavior identification method based on model fusion
CN111079795A (en) * 2019-11-21 2020-04-28 西安工程大学 Image classification method based on CNN (content-centric networking) fragment multi-scale feature fusion
CN111046962A (en) * 2019-12-16 2020-04-21 中国人民解放军战略支援部队信息工程大学 Sparse attention-based feature visualization method and system for convolutional neural network model
CN111242168B (en) * 2019-12-31 2023-07-21 浙江工业大学 Human skin image lesion classification method based on multi-scale attention features
CN111242168A (en) * 2019-12-31 2020-06-05 浙江工业大学 Human skin image lesion classification method based on multi-scale attention features
CN111178304A (en) * 2019-12-31 2020-05-19 江苏省测绘研究所 High-resolution remote sensing image pixel level interpretation method based on full convolution neural network
CN111414932B (en) * 2020-01-07 2022-05-31 北京航空航天大学 Classification identification and fault detection method for multi-scale signals of aircraft
CN111414932A (en) * 2020-01-07 2020-07-14 北京航空航天大学 Classification identification and fault detection method for multi-scale signals of aircraft
CN111208821A (en) * 2020-02-17 2020-05-29 李华兰 Automobile automatic driving control method and device, automatic driving device and system
WO2021174618A1 (en) * 2020-03-02 2021-09-10 五邑大学 Training method for electroencephalography mode classification model, classification method and system
CN111460892A (en) * 2020-03-02 2020-07-28 五邑大学 Electroencephalogram mode classification model training method, classification method and system
CN111461039B (en) * 2020-04-07 2022-03-25 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111461039A (en) * 2020-04-07 2020-07-28 电子科技大学 Landmark identification method based on multi-scale feature fusion
CN111402274A (en) * 2020-04-14 2020-07-10 上海交通大学医学院附属上海儿童医学中心 Processing method, model and training method for magnetic resonance left ventricle image segmentation
CN111402274B (en) * 2020-04-14 2023-05-26 上海交通大学医学院附属上海儿童医学中心 Processing method, model and training method for segmentation of magnetic resonance left ventricle image
CN111582044A (en) * 2020-04-15 2020-08-25 华南理工大学 Face recognition method based on convolutional neural network and attention model
CN111582044B (en) * 2020-04-15 2023-06-20 华南理工大学 Face recognition method based on convolutional neural network and attention model
CN111507281A (en) * 2020-04-21 2020-08-07 中山大学中山眼科中心 Behavior recognition system, device and method based on head movement and gaze behavior data
CN111553500A (en) * 2020-05-11 2020-08-18 北京航空航天大学 Railway traffic contact net inspection method based on attention mechanism full convolution network
CN111563468A (en) * 2020-05-13 2020-08-21 电子科技大学 Driver abnormal behavior detection method based on attention of neural network
CN111563468B (en) * 2020-05-13 2023-04-07 电子科技大学 Driver abnormal behavior detection method based on attention of neural network
CN111860427A (en) * 2020-07-30 2020-10-30 重庆邮电大学 Driving distraction identification method based on lightweight class eight-dimensional convolutional neural network
CN111860427B (en) * 2020-07-30 2022-07-01 重庆邮电大学 Driving distraction identification method based on lightweight class eight-dimensional convolutional neural network
CN112464765B (en) * 2020-09-10 2022-09-23 天津师范大学 Safety helmet detection method based on single-pixel characteristic amplification and application thereof
CN112464765A (en) * 2020-09-10 2021-03-09 天津师范大学 Safety helmet detection algorithm based on single-pixel characteristic amplification and application thereof
CN112215241A (en) * 2020-10-20 2021-01-12 西安交通大学 Image feature extraction device based on small sample learning
CN112257601A (en) * 2020-10-22 2021-01-22 福州大学 Fine-grained vehicle identification method based on data enhancement network of weak supervised learning
CN112527915B (en) * 2020-11-17 2021-08-27 北京科技大学 Linear cultural heritage knowledge graph construction method, system, computing device and medium
CN112527915A (en) * 2020-11-17 2021-03-19 北京科技大学 Linear cultural heritage knowledge graph construction method, system, computing device and medium
CN112347977B (en) * 2020-11-23 2021-07-20 深圳大学 Automatic detection method, storage medium and device for induced pluripotent stem cells
CN112347977A (en) * 2020-11-23 2021-02-09 深圳大学 Automatic detection method, storage medium and device for induced pluripotent stem cells
CN112613405A (en) * 2020-12-23 2021-04-06 电子科技大学 Method for recognizing actions at any visual angle
CN112613405B (en) * 2020-12-23 2022-03-25 电子科技大学 Method for recognizing actions at any visual angle
CN112800834A (en) * 2020-12-25 2021-05-14 温州晶彩光电有限公司 Method and system for positioning colorful spot light based on kneeling behavior identification
CN112800834B (en) * 2020-12-25 2022-08-12 温州晶彩光电有限公司 Method and system for positioning colorful spot light based on kneeling behavior identification
CN112733652B (en) * 2020-12-31 2024-04-19 深圳赛安特技术服务有限公司 Image target recognition method, device, computer equipment and readable storage medium
CN112733652A (en) * 2020-12-31 2021-04-30 深圳赛安特技术服务有限公司 Image target identification method and device, computer equipment and readable storage medium
CN112837360B (en) * 2021-01-07 2023-08-11 北京百度网讯科技有限公司 Depth information processing method, apparatus, device, storage medium, and program product
CN112837360A (en) * 2021-01-07 2021-05-25 北京百度网讯科技有限公司 Depth information processing method, apparatus, device, storage medium, and program product
CN112800871A (en) * 2021-01-13 2021-05-14 南京邮电大学 Automatic driving image recognition method based on attention mechanism and relation network
CN112800871B (en) * 2021-01-13 2022-08-26 南京邮电大学 Automatic driving image recognition method based on attention mechanism and relation network
CN113095479A (en) * 2021-03-22 2021-07-09 北京工业大学 Method for extracting ice-below-layer structure based on multi-scale attention mechanism
CN113095479B (en) * 2021-03-22 2024-03-12 北京工业大学 Multi-scale attention mechanism-based extraction method for ice underlying structure
CN113029155A (en) * 2021-04-02 2021-06-25 杭州申昊科技股份有限公司 Robot automatic navigation method and device, electronic equipment and storage medium
CN113033448A (en) * 2021-04-02 2021-06-25 东北林业大学 Remote sensing image cloud-removing residual error neural network system, method and equipment based on multi-scale convolution and attention and storage medium
CN113516028A (en) * 2021-04-28 2021-10-19 南通大学 Human body abnormal behavior identification method and system based on mixed attention mechanism
CN113516028B (en) * 2021-04-28 2024-01-19 南通大学 Human body abnormal behavior identification method and system based on mixed attention mechanism
CN113283338A (en) * 2021-05-25 2021-08-20 湖南大学 Method, device and equipment for identifying driving behavior of driver and readable storage medium
CN113281029A (en) * 2021-06-09 2021-08-20 重庆大学 Rotating machinery fault diagnosis method and system based on multi-scale network structure
CN113537003B (en) * 2021-07-02 2022-10-21 安阳工学院 External environment visual detection method and device for driving assistance
CN113537003A (en) * 2021-07-02 2021-10-22 安阳工学院 Method and device for visually detecting external environment of vehicle for assisting driving
CN113642571A (en) * 2021-07-12 2021-11-12 中国海洋大学 Fine-grained image identification method based on saliency attention mechanism
CN113642571B (en) * 2021-07-12 2023-10-10 中国海洋大学 Fine granularity image recognition method based on salient attention mechanism
CN113642634A (en) * 2021-08-12 2021-11-12 南京邮电大学 Shadow detection method based on mixed attention
CN113762251B (en) * 2021-08-17 2024-05-10 慧影医疗科技(北京)股份有限公司 Attention mechanism-based target classification method and system
CN113762251A (en) * 2021-08-17 2021-12-07 慧影医疗科技（北京）有限公司 Target classification method and system based on attention mechanism
CN113657534A (en) * 2021-08-24 2021-11-16 北京经纬恒润科技股份有限公司 Classification method and device based on attention mechanism
CN113780385A (en) * 2021-08-30 2021-12-10 武汉理工大学 Driving risk monitoring method based on attention mechanism
CN113989862A (en) * 2021-10-12 2022-01-28 天津大学 Texture recognition platform based on embedded system
CN113989862B (en) * 2021-10-12 2024-05-14 天津大学 Texture recognition platform based on embedded system
US11886199B2 (en) 2021-10-13 2024-01-30 Toyota Motor Engineering & Manufacturing North America, Inc. Multi-scale driving environment prediction with hierarchical spatial temporal attention
CN114639169B (en) * 2022-03-28 2024-02-20 合肥工业大学 Human motion recognition system based on attention mechanism feature fusion and irrelevant to position
CN114639169A (en) * 2022-03-28 2022-06-17 合肥工业大学 Human body action recognition system based on attention mechanism feature fusion and position independence
CN114419558B (en) * 2022-03-31 2022-07-05 华南理工大学 Fire video image identification method, fire video image identification system, computer equipment and storage medium
CN114419558A (en) * 2022-03-31 2022-04-29 华南理工大学 Fire video image identification method, fire video image identification system, computer equipment and storage medium
CN114782931B (en) * 2022-04-22 2023-09-29 电子科技大学 Driving behavior classification method for improving mobilenet v2 network
CN114782931A (en) * 2022-04-22 2022-07-22 电子科技大学 Driving behavior classification method for improved MobileNetv2 network
CN114818991B (en) * 2022-06-22 2022-09-27 西南石油大学 Running behavior identification method based on convolutional neural network and acceleration sensor
CN114818991A (en) * 2022-06-22 2022-07-29 西南石油大学 Running behavior identification method based on convolutional neural network and acceleration sensor
CN115082698A (en) * 2022-06-28 2022-09-20 华南理工大学 Distracted driving behavior detection method based on multi-scale attention module
CN115082698B (en) * 2022-06-28 2024-04-16 华南理工大学 Distraction driving behavior detection method based on multi-scale attention module
CN115432331A (en) * 2022-10-10 2022-12-06 浙江绿达智能科技有限公司 Intelligent classification dustbin
CN115964360B (en) * 2023-03-14 2023-05-16 山东省地震工程研究院 Method and system for building earthquake safety evaluation database
CN115964360A (en) * 2023-03-14 2023-04-14 山东省地震工程研究院 Earthquake safety evaluation database construction method and system
CN116758631B (en) * 2023-06-13 2023-12-22 杭州追形视频科技有限公司 Big data driven behavior intelligent analysis method and system
CN116758631A (en) * 2023-06-13 2023-09-15 孟冠宇 Big data driven behavior intelligent analysis method and system
CN117173422B (en) * 2023-08-07 2024-02-13 广东第二师范学院 Fine granularity image recognition method based on graph fusion multi-scale feature learning
CN117173422A (en) * 2023-08-07 2023-12-05 广东第二师范学院 Fine granularity image recognition method based on graph fusion multi-scale feature learning
CN116977969B (en) * 2023-08-11 2023-12-26 中国矿业大学 Driver two-point pre-aiming identification method based on convolutional neural network
CN116977969A (en) * 2023-08-11 2023-10-31 中国矿业大学 Driver two-point pre-aiming identification method based on convolutional neural network
CN117576666A (en) * 2023-11-17 2024-02-20 合肥工业大学 Dangerous driving behavior detection method based on multi-scale dynamic convolution attention weighting
CN117576666B (en) * 2023-11-17 2024-05-10 合肥工业大学 Dangerous driving behavior detection method based on multi-scale dynamic convolution attention weighting

Also Published As

Publication number Publication date
CN110059582B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
CN110059582A (en) Driving behavior recognition methods based on multiple dimensioned attention convolutional neural networks
CN110619369B (en) Fine-grained image classification method based on feature pyramid and global average pooling
Lu et al. Driver action recognition using deformable and dilated faster R-CNN with optimized region proposals
CN108875674B (en) Driver behavior identification method based on multi-column fusion convolutional neural network
CN104408440B (en) A kind of facial expression recognizing method merged based on two step dimensionality reductions and Concurrent Feature
WO2022083784A1 (en) Road detection method based on internet of vehicles
CN103258204B (en) A kind of automatic micro-expression recognition method based on Gabor and EOH feature
CN101944174B (en) Identification method of characters of licence plate
CN105354986A (en) Driving state monitoring system and method for automobile driver
CN111401148A (en) Road multi-target detection method based on improved multilevel YO L Ov3
CN111460919B (en) Monocular vision road target detection and distance estimation method based on improved YOLOv3
CN110097109A (en) A kind of road environment obstacle detection system and method based on deep learning
CN104866810A (en) Face recognition method of deep convolutional neural network
CN205230272U (en) Driver drive state monitoring system
CN106446954A (en) Character recognition method based on depth learning
CN109670392A (en) Based on mixing autocoder road image semantic segmentation method
CN106845387A (en) Pedestrian detection method based on self study
CN110363093A (en) A kind of driver's action identification method and device
CN105787466A (en) Vehicle type fine identification method and system
CN110796109A (en) Driver distraction behavior identification method based on model fusion
CN113283338A (en) Method, device and equipment for identifying driving behavior of driver and readable storage medium
CN112036520A (en) Panda age identification method and device based on deep learning and storage medium
CN112052829B (en) Pilot behavior monitoring method based on deep learning
Mammeri et al. Design of a semi-supervised learning strategy based on convolutional neural network for vehicle maneuver classification
CN108960005A (en) The foundation and display methods, system of subjects visual label in a kind of intelligent vision Internet of Things

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant