CN108961675A - Fall detection method based on convolutional neural networks - Google Patents
Fall detection method based on convolutional neural networks Download PDFInfo
- Publication number
- CN108961675A CN108961675A CN201810614024.4A CN201810614024A CN108961675A CN 108961675 A CN108961675 A CN 108961675A CN 201810614024 A CN201810614024 A CN 201810614024A CN 108961675 A CN108961675 A CN 108961675A
- Authority
- CN
- China
- Prior art keywords
- model
- convolutional neural
- neural networks
- training
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
- G08B21/043—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis detecting an emergency event, e.g. a fall
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0438—Sensor means for detecting
- G08B21/0476—Cameras to detect unsafe condition, e.g. video cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Gerontology & Geriatric Medicine (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Psychology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of fall detection methods based on convolutional neural networks, it include: training convolutional neural networks, the training convolutional neural networks specifically include: pre-processing to each frame image acquired, pretreated work includes being followed successively by foreground extraction and normalization, whitening operation；Pre-training first is carried out to ResNet network on ImageNet data set, to obtain pre-training model.Classification method based on convolutional neural networks is applied in fall detection method, simultaneously, in order to improve the precision of system, reduce the complexity of operation, extract the personage under complex background using a kind of improved foreground detection method, then will treated that image is put into convolutional neural networks carries out model training.
Description
Technical field
The present invention relates to fall detection methods, more particularly to the fall detection method based on convolutional neural networks.
Background technique
Present social senilization's trend increasingly aggravates, the decline of the elderly's physical function and more and more common solitary existing
As making one of the main reason for causing injury as the elderly of falling, so carrying out detection to tumble behavior has highly important meaning
Justice.
Traditional fall detection method based on computer vision is usually manual extraction feature, and project amount is huge, and
Generalization ability is poor, and precision is not high.Different from traditional feature extraction, convolutional neural networks can automatically extract feature, training
Model afterwards has geometric invariance, can overcome the problems, such as the variation because of illumination and shooting angle due to generate.
There are following technical problems for traditional technology:
Current fall detection system is broadly divided into two classes: the first kind is sensor-based wearable detection system；It is another
Class is the detection system based on video.Currently, the wearable identifying system research based on three-dimensional acceleration or trunk angular speed
It has been relatively mature.However, wearable device, which generally requires, is worn on neck or waist, wearing will use family and feel not for a long time
It is suitable.And vision-based inspection system is then to pass through specific image by the movement of one or several cameras capture targets
Processing Algorithm determines characteristics of image when falling, so that tumble be distinguished with daily routines.It is currently used to be based on
The fall detection algorithm of vision is mainly threshold method and intelligent algorithm.Threshold method is usually the head position or center of gravity to human body
It is detected.Diraco, which passes through, judges that human body center is considered as falling lower than specified altitude assignment and when being maintained for more than 4s.
Rougier et al. estimates head position in next frame image by positioning head position, then by particle filter, calculates horizontal
The speed in direction and vertical direction, and the mode being compared with threshold value determines whether to fall.The realization of these methods is simple,
But precision is easy to be influenced by extraneous factors such as environment.And the method based on machine learning mainly first carries out people to image
Object extracts, then manual extraction feature again, and the feature input model of acquisition is realized that the detection to tumble behavior identifies.This
Kind of method need it is artificial extract feature, project amount is huge, and most of only rests in two classification problems, it is contemplated that it is following for
The requirement of smart home is got higher, and the various postures of human body are identified also into indispensable part.
Summary of the invention
Based on this, it is necessary in view of the above technical problems, a kind of fall detection method based on convolutional neural networks is provided,
Classification method based on convolutional neural networks is applied in fall detection method, meanwhile, in order to improve the precision of system, reduce
The complexity of operation extracts the personage under complex background using a kind of improved foreground detection method, then by treated
Image, which is put into convolutional neural networks, carries out model training.
A kind of fall detection method based on convolutional neural networks, comprising:
Training convolutional neural networks, the training convolutional neural networks specifically include:
The each frame image acquired is pre-processed, pretreated work includes being followed successively by foreground extraction and normalizing
Change, whitening operation；
Pre-training first is carried out to ResNet network on ImageNet data set, to obtain pre-training model；
Step " pre-processes each frame image acquired, pretreated work includes being followed successively by prospect to mention
It takes and normalizes, whitening operation；" treated, and picture is put into the pre-training model carries out model training, obtain model
Parameter；And
Test set is input to the model after training, is detected with the precision of test the set pair analysis model；
Picture is detected using the convolutional neural networks after trained；
Wherein, the method for the foreground extraction specifically includes:
Image is handled using background subtraction；
Image is handled using mixed Gauss model；
The result of image after being handled using the result after background subtraction processing image and using mixed Gauss model is asked
With.
The above-mentioned fall detection method based on convolutional neural networks, the classification method based on convolutional neural networks is applied to
In fall detection method, meanwhile, in order to improve the precision of system, reduce the complexity of operation, is examined using a kind of improved prospect
Survey method is to extract the personage under complex background, then image is put into progress model instruction in convolutional neural networks by treated
Practice.
In other one embodiment, step " each frame image acquired is pre-processed, it is pretreated
Work includes being followed successively by foreground extraction and normalization, whitening operation；" in, each frame image acquired is regarded by reading
What frequency file obtained.
In other one embodiment, after step " detecting picture using the convolutional neural networks after training ",
The training convolutional neural networks are specific further include: show the detection effect figure of each frame picture, and the convolution of implementation model
Core visualization.
In other one embodiment, step " shows the detection effect figure of each frame picture, and the volume of implementation model
The detection effect figure of each frame picture is shown in product core visualization " on matlab platform, and the convolution kernel of implementation model can
Depending on changing.
In other one embodiment, step " handles image using background subtraction；" specifically include:
The average value for taking former frame images, is used as initial background image Bt；
The gray scale of prior image frame and background image carries out subtracting operation, and taking its absolute value is Nt(x, y) formula is
Nt(x, y)=| It(x,y)-Bt(x,y)|
To the pixel (x, y) of present frame, if having | It(x,y)-Bt(x, y) | >=T, then the pixel is foreground point, that is, is updated
Current picture frame is；
Background image is updated with current frame image.
In other one embodiment, step " handles image using mixed Gauss model；" specifically include:
When using gauss hybrid models to background constructing model, the pixel value of each pixel of sequence image can use k
A Gauss model simulation, therefore in moment t, the probability density function of some pixel value can indicate are as follows:
Wherein, wi,tIndicate the weight of Gauss model, and the probability density function of Gauss model indicates are as follows:
Then, K gauss hybrid models are sorted according to weight divided by the size of the quotient of standard deviation, then selects previous B
A Gauss model differentiates that wherein the value of B is indicated for distinguishing are as follows:
Each pixel in new picture frame is put into respectively in sorted K Gauss model and is judged, is judged
Condition are as follows:
||Xt-μt||≤β∑1/2
In B Gauss model in front, meet above-mentioned condition if wherein had in a Gauss model, this
Pixel is it is determined that background, if above-mentioned condition is all unsatisfactory in B Gauss model, before determining that this pixel belongs to
Scape.
For each Gauss model, it is assumed that the invalid weight it is necessary to reduce this Gauss model of above-mentioned condition, if
Above-mentioned condition is set up it is necessary to update this gauss hybrid models, shown in the following formula of the method for concrete operations:
wi,t=(1- λ) wi,t-1+λBMt
μi,t=(1- α) μi,t-1+αXi,t
∑i,t=(1- α) ∑i,t-1+α(Xi,t-μi,t)(Xi,t-μi,t)T
α=λ/wi,t
Wherein, pixel is prospect then BM=0, and otherwise BM=1, finally replaces weight most with an initial Gauss model
That small Gauss model, threshold value T, learning rate λ, parameter beta are all constants specified in advance.
In other one embodiment, test set " is input to the model after training, with test set to mould by step
The precision of type is detected；" in, the test set comes from UR Fall Detection Dataset.
A kind of computer equipment can be run on a memory and on a processor including memory, processor and storage
The step of computer program, the processor realizes any one the method when executing described program.
A kind of computer readable storage medium, is stored thereon with computer program, realization when which is executed by processor
The step of any one the method.
A kind of processor, the processor is for running program, wherein described program executes described in any item when running
Method.
Detailed description of the invention
Fig. 1 is a kind of process signal of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Figure.
Fig. 2 is the residual error study in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Construct module diagram.
Fig. 3 is the penalty values letter in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Number curve schematic diagram.
Fig. 4 is the test of model in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Flow chart.
Fig. 5 is the background difference in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
The effect diagram of method.
Fig. 6 is the Gaussian Mixture in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
The effect diagram of environmental model.
Before Fig. 7 is improved in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
The effect diagram of scape detection method.
Fig. 8 is the foreground extraction in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
RGB figure.
Fig. 9 is that the convolution kernel in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application can
Depending on changing.
Figure 10 is the first layer in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Characteristic pattern.
Figure 11 is the second layer in a kind of fall detection method based on convolutional neural networks provided by the embodiments of the present application
Characteristic pattern.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Refering to fig. 1, a kind of fall detection method based on convolutional neural networks, comprising:
Training convolutional neural networks, the training convolutional neural networks specifically include:
The each frame image acquired is pre-processed, pretreated work includes being followed successively by foreground extraction and normalizing
Change, whitening operation；
Pre-training first is carried out to ResNet network on ImageNet data set, to obtain pre-training model；
Step " pre-processes each frame image acquired, pretreated work includes being followed successively by prospect to mention
It takes and normalizes, whitening operation；" treated, and picture is put into the pre-training model carries out model training, obtain model
Parameter；And
Test set is input to the model after training, is detected with the precision of test the set pair analysis model；
Picture is detected using the convolutional neural networks after trained；
Wherein, the method for the foreground extraction specifically includes:
Image is handled using background subtraction；
Image is handled using mixed Gauss model；
The result of image after being handled using the result after background subtraction processing image and using mixed Gauss model is asked
With.
The above-mentioned fall detection method based on convolutional neural networks, the classification method based on convolutional neural networks is applied to
In fall detection method, meanwhile, in order to improve the precision of system, reduce the complexity of operation, is examined using a kind of improved prospect
Survey method is to extract the personage under complex background, then image is put into progress model instruction in convolutional neural networks by treated
Practice.
In other one embodiment, step " each frame image acquired is pre-processed, it is pretreated
Work includes being followed successively by foreground extraction and normalization, whitening operation；" in, each frame image acquired is regarded by reading
What frequency file obtained.
In other one embodiment, after step " detecting picture using the convolutional neural networks after training ",
The training convolutional neural networks are specific further include: show the detection effect figure of each frame picture, and the convolution of implementation model
Core visualization.
In other one embodiment, step " shows the detection effect figure of each frame picture, and the volume of implementation model
The detection effect figure of each frame picture is shown in product core visualization " on matlab platform, and the convolution kernel of implementation model can
Depending on changing.
In other one embodiment, step " handles image using background subtraction；" specifically include:
The average value for taking former frame images, is used as initial background image Bt；
The gray scale of prior image frame and background image carries out subtracting operation, and taking its absolute value is Nt(x, y) formula is
Nt(x, y)=| It(x,y)-Bt(x,y)|
To the pixel (x, y) of present frame, if having | It(x,y)-Bt(x, y) | >=T, then the pixel is foreground point, that is, is updated
Current picture frame is；
Background image is updated with current frame image.
In other one embodiment, step " handles image using mixed Gauss model；" specifically include:
When using gauss hybrid models to background constructing model, the pixel value of each pixel of sequence image can use k
A Gauss model simulation, therefore in moment t, the probability density function of some pixel value can indicate are as follows:
Wherein, wi,tIndicate the weight of Gauss model, and the probability density function of Gauss model indicates are as follows:
Then, K gauss hybrid models are sorted according to weight divided by the size of the quotient of standard deviation, then selects previous B
A Gauss model differentiates that wherein the value of B is indicated for distinguishing are as follows:
Each pixel in new picture frame is put into respectively in sorted K Gauss model and is judged, is judged
Condition are as follows:
||Xt-μt||≤β∑1/2
In B Gauss model in front, meet above-mentioned condition if wherein had in a Gauss model, this
Pixel is it is determined that background, if above-mentioned condition is all unsatisfactory in B Gauss model, before determining that this pixel belongs to
Scape.
For each Gauss model, it is assumed that the invalid weight it is necessary to reduce this Gauss model of above-mentioned condition, if
Above-mentioned condition is set up it is necessary to update this gauss hybrid models, shown in the following formula of the method for concrete operations:
wi,t=(1- λ) wi,t-1+λBMt
μi,t=(1- α) μi,t-1+αXi,t
∑i,t=(1- α) ∑i,t-1+α(Xi,t-μi,t)(Xi,t-μi,t)T
α=λ/wi,t
Wherein, pixel is prospect then BM=0, and otherwise BM=1, finally replaces weight most with an initial Gauss model
That small Gauss model, threshold value T, learning rate λ, parameter beta are all constants specified in advance.
In other one embodiment, test set " is input to the model after training, with test set to mould by step
The precision of type is detected；" in, the test set comes from URFall Detection Dataset.
A kind of computer equipment can be run on a memory and on a processor including memory, processor and storage
The step of computer program, the processor realizes any one the method when executing described program.
A kind of computer readable storage medium, is stored thereon with computer program, realization when which is executed by processor
The step of any one the method.
A kind of processor, the processor is for running program, wherein described program executes described in any item when running
Method.
A concrete application scene of the invention is given below:
1. image preprocessing
In the particular embodiment, personage's prospect is extracted using improved foreground detection method.Foreground extraction at present
Method mainly has frame differential method, background subtraction, optical flow method, mixed Gauss model.
1) background subtraction
Wherein, if setting It, BtRespectively current picture frame and background frames, T are the gray threshold of foreground detection, background subtraction
The algorithm steps of point-score are as follows:
1) average value for taking former frame images, is used as initial background image Bt；
2) gray scale of prior image frame and background image carries out subtracting operation, and taking its absolute value is Nt(x,y)；Formula is
Nt(x, y)=| It(x,y)-Bt(x,y)|
3) to the pixel (x, y) of present frame, if having | It(x,y)-Bt(x, y) | >=T, then the pixel is foreground point, i.e., more
New current picture frame is
4) background image is updated with current frame image.
2) mixed Gauss model
Mixed Gauss model is a kind of ADAPTIVE MIXED Gauss based on to background modeling proposed by Stauffer et al.
Background extracting method.When using gauss hybrid models to background constructing model, the pixel value of each pixel of sequence image
K Gauss model simulation can be used, therefore in moment t, the probability density function of some pixel value can be indicated are as follows:
Wherein, wi,tIndicate the weight of Gauss model, and the probability density function of Gauss model can indicate are as follows:
Then, K gauss hybrid models are sorted according to weight divided by the size of the quotient of standard deviation, then selects previous B
A Gauss model differentiates that wherein the value of B is indicated for distinguishing are as follows:
Each pixel in new picture frame is put into respectively in sorted K Gauss model and is judged, is judged
Condition are as follows:
||Xt-μt||≤β∑1/2
In B Gauss model in front, meet above-mentioned condition if wherein had in a Gauss model, this
Pixel is it is determined that background, if above-mentioned condition is all unsatisfactory in B Gauss model, before determining that this pixel belongs to
Scape.
For each Gauss model, it is assumed that the invalid weight it is necessary to reduce this Gauss model of above-mentioned condition, if
Above-mentioned condition is set up it is necessary to update this gauss hybrid models, shown in the following formula of the method for concrete operations:
wi,t=(1- λ) wi,t-1+λBMt
μi,t=(1- α) μi,t-1+αXi,t
∑i,t=(1- α) ∑i,t-1+α(Xi,t-μi,t)(Xi,t-μi,t)T
α=λ/wi,t
Wherein, pixel is prospect then BM=0, otherwise BM=1.Finally weight is replaced most with an initial Gauss model
That small Gauss model, threshold value T, learning rate λ, parameter beta are all constants specified in advance.
3) improved foreground detection method
Although background subtraction method is simple, calculation amount is small, can generate " ghost " phenomenon.And mixed Gauss model is mixed
Gaussian processes is closed not only to background modeling, while also modeling to prospect, therefore very sensitive to the suddenly change of global brightness.In order to
Solve simple using limitation brought by a kind of method, the present invention proposes a kind of improvement foreground detection method, i.e., by the two
Treatment effect carry out with operation can be good at solving the problems, such as " ghost " and light sensitive.For the pixel of picture matrix
Point (x, y) assumes that the output of background subtraction is D (x, y), the output G (x, y) of mixed Gaussian method, improved foreground detection method
Output be R (x, y), then
Again with the mode of some binary conversion treatment pictures, the binary image of clearly target person can be obtained.
The selection in operation and largest connected domain is opened and closed to substantially determine after foreground extraction, then to image
The position of personage's prospect intercepts corresponding position and obtains the RGB image of corresponding personage's prospect.
2. the selection of model
Using champion's model ResNet in 2015 as network model of the invention.ResNet network can be solved effectively
Certainly as network depth increases, the accuracy of algorithm tends to saturation and the problem of rapid decrease.Meanwhile its parameter amount ratio
VGGNet is also low, effect highly significant.While the accuracy rate of greatly lift scheme, training speed has also obtained very big
It improves.This is mainly attributed to the building thought of its used residual error module, as shown in Figure 2
Following table is comparison of the ResNet-34 and VGG-16 in ImageNet2012
| Network | RESNET-34 | VGG-16 |
| Calculation amount | 3600000000 FLOPs | 15300000000 FLOPs |
| Precision top-1 | 0.733 | 0.715 |
The network architecture of ResNet is as shown in the table:
Pre-training model
Before formally carrying out model training with pretreated image, first with ImageNet to the ginseng of convolutional neural networks
Number carries out pre-training.Bengio professor et al. points out often model to be entered using the method for carrying out random initializtion to model
Probability to local minimum is very high, and enables to the effect of model more preferable by the way of pre-training.In actual operation,
The effect of pre-training model is exactly the model parameter for training the parameter initialization of network on network for ImageNet.But
Be that because ImageNet is divided into 1000 classes, and the present invention only needs to divide the image into two classes, thus need to full articulamentum into
Row modification, is changed to 2 by the 1000 of script for the value of num_output, and modify the name of full articulamentum.
Model training
The training process of network is completed based on Caffe platform, and Caffe assumes to construct according to one of neural network:
All calculating is wherein indicated with the form of layer, the effect of layer is exactly according to the input data, so that output is after calculating
Result.For convolution, if input is piece image, then carries out convolution algorithm operation with the parameter of this layer, finally again
Export the result of convolution.Each layer requires two kinds of operations: 1) through path, from enter data to calculate output data；2) anti-
To access, the gradient relative to input is calculated according to gradient value above.When each layer is completed the two functions
Afterwards, it will be able to many layers are all connected into a network, the function of network be exactly according to input data (image, voice or its
His message form) calculate desired output.During training, output can be calculated according to known label result
The loss function and gradient of model, then further update the parameter of network further according to gradient value.
Before being trained to model, need the database i.e. image and corresponding label of first package image, i.e., it will figure
Image set is encapsulated into the form of database: Lmdb and Leveldb.It is specifically exactly that convert_imageset order is called
Change data format.Building for ResNet34 model is to realize that it is defined by defining trian_val.prototxt
The specific network structure of ResNet.The organizational structure of this document is the form of class formation body.It is all wrapped in every layer of layer structural body
Include many parameters.Such as bottom parameter indicate bottom input, top parameter indicate be output to next layer as a result,
Param parameter indicates the parameter of this layer, and the number of filter is indicated including num_output, and kernel_size is indicated
The size of filter, stride indicate step-length.The mode of the training pattern of ResNet is by this text of solver.prototxt
What part was write, every row in this file all indicates a training parameter.Wherein, some common training parameters, as
Net parameter is used to specify the model used, and max_iter parameter indicates the maximum times of setting repeatedly reached, then
Snapshot_prefix parameter then indicates its prefix title etc. when model preservation.
It is also possible to the loss function curve in rendering model training on Matlab platform.Every iteration is once just drawn
As soon as making time penalty values, precision of every iteration 100 times draftings.As shown in Figure 3.
The test of model
Using trained model to single picture classify and output category as a result, the step for equally exist
It is realized on Matlab.Detailed process participates in Fig. 4.
When obtaining propagated forward result, it is only necessary to which the function net.forward for calling matlab included can be obtained figure
As the probability under each label.Data are existing in three dimensions in each convolutional layer.It can be regarded as by very
Multiple two-dimension pictures stack, and wherein each is known as a characteristic pattern (feature map).If input layer is grayscale image
Piece, then with regard to only one feature map；It generally will be that 3 characteristic patterns are (red green if input layer is color image
It is blue).There are many convolution kernels (kernel) between layers, upper one layer of each feature map is rolled up with each convolution kernel
Product operation, can all generate next layer of a characteristic pattern.
1. first person extraction is come out with improved foreground detection method, wherein background subtraction and mixed Gauss model
The effect picture of extraction prospect is as shown in Figure 5 and Figure 6:
It, i.e., will be after the result phase "AND" of both background subtraction and mixed Gauss model with improved foreground detection method
Effect picture is as shown in Figure 7.
Then, it then carries out the extraction in simple binary conversion treatment and largest connected domain and determines the range of personage's prospect, thus
Interception obtains the corresponding RGB image of personage in original image
2. carrying out the test of model training and performance on disclosed data set UR Fall Detection Dataset.
Wherein training set has 7381 pictures, and test set has 1326 pictures.Picture all in data set is all first carried out to prospect to mention
The operation taken is put into the training that model is carried out in network after obtaining picture as shown in Figure 8.
3. first pre-training is carried out to the ResNet network that the present invention uses on ImageNet data set, to obtain pre- instruction
Practice model.It training set is pre-processed into the picture after (foreground detection, albefaction, normalization) is input in network and is trained.
4. and accuracy test is carried out to trained model with test set, every iteration once tests a penalty values, often changes
A generation precision of 100 tests, finally result precision is 96.7% when convergence
5. by the output of the independent picture of Matlab platform test, and realizing the visualization of convolution kernel and characteristic pattern.
Wherein the visualization picture of convolution kernel is as shown in Figure 9
Matlab is called to realize characteristic pattern visualization from tape function feature_map, as shown in Figure 10 and Figure 11.
The convolution kernel that can be seen that bottom from Figure 10 and Figure 11 is mainly used to extract the feature on the basis such as profile of personage.
Training set and test set of the invention exists data set both from UR Fall Detection Dataset
The training of model and the test of precision are carried out on caffe platform, and is accelerated using cuDNN, and final display precision reaches
96.7%, time complexity 49ms.
Each technical characteristic of embodiment described above can be combined arbitrarily, for simplicity of description, not to above-mentioned reality
It applies all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, all should be considered as described in this specification.
The embodiments described above only express several embodiments of the present invention, and the description thereof is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
It says, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to protection of the invention
Range.Therefore, the scope of protection of the patent of the invention shall be subject to the appended claims.
Claims (10)
1. a kind of fall detection method based on convolutional neural networks characterized by comprising
The training convolutional neural networks, the training convolutional neural networks specifically include:
The each frame image acquired is pre-processed, pretreated work include be followed successively by foreground extraction and normalization,
Whitening operation；
Pre-training first is carried out to ResNet network on ImageNet data set, to obtain pre-training model；
By step " each frame image acquired is pre-processed, pretreated work include be followed successively by foreground extraction and
Normalization, whitening operation；" treated, and picture is put into the pre-training model carries out model training, obtain the parameter of model；
And
Test set is input to the model after training, is detected with the precision of test the set pair analysis model；
Picture is detected using the convolutional neural networks after trained；
Wherein, the method for the foreground extraction specifically includes:
Image is handled using background subtraction；
Image is handled using mixed Gauss model；
To the result of image after handling the result after image using background subtraction and being handled using mixed Gauss model ask with.
2. the fall detection method according to claim 1 based on convolutional neural networks, which is characterized in that " right in step
The each frame image acquired is pre-processed, and pretreated work includes being followed successively by foreground extraction and normalization, albefaction behaviour
Make；" in, each frame image acquired is obtained by reading video file.
3. the fall detection method according to claim 1 based on convolutional neural networks, which is characterized in that in step " benefit
Picture is detected with the convolutional neural networks after trained " after, the training convolutional neural networks are specific further include: show every
The detection effect figure of one frame picture, and the convolution kernel visualization of implementation model.
4. the fall detection method according to claim 3 based on convolutional neural networks, which is characterized in that step " display
It is shown on matlab platform in the detection effect figure of each frame picture out, and the convolution kernel visualization of implementation model " each
The detection effect figure of frame picture, and the convolution kernel visualization of implementation model.
5. the fall detection method according to claim 1 based on convolutional neural networks, which is characterized in that step " utilizes
Background subtraction handles image；" specifically include:
The average value for taking former frame images, is used as initial background image Bt；
The gray scale of prior image frame and background image carries out subtracting operation, and taking its absolute value is Nt(x, y) formula is
Nt(x, y)=| It(x,y)-Bt(x,y)|
To the pixel (x, y) of present frame, if having | It(x,y)-Bt(x, y) | >=T, then the pixel is foreground point, that is, is updated current
Picture frame be；
Background image is updated with current frame image.
6. the fall detection method according to claim 1 based on convolutional neural networks, which is characterized in that step " utilizes
Mixed Gauss model handles image；" specifically include:
When using gauss hybrid models to background constructing model, the pixel value of each pixel of sequence image can be used k high
This modeling, therefore in moment t, the probability density function of some pixel value can indicate are as follows:
Wherein, wi,tIndicate the weight of Gauss model, and the probability density function of Gauss model indicates are as follows:
Then, K gauss hybrid models are sorted according to weight divided by the size of the quotient of standard deviation, is then selected previous B high
This model differentiates that wherein the value of B is indicated for distinguishing are as follows:
Each pixel in new picture frame is put into respectively in sorted K Gauss model and is judged, Rule of judgment
Are as follows:
||Xt-μt||≤β∑1/2
In B Gauss model in front, meet above-mentioned condition if wherein had in a Gauss model, this pixel
It is determined that background determines that this pixel belongs to prospect if above-mentioned condition is all unsatisfactory in B Gauss model.
For each Gauss model, it is assumed that the invalid weight it is necessary to reduce this Gauss model of above-mentioned condition, if above-mentioned
Condition set up it is necessary to update this gauss hybrid models, shown in the following formula of the method for concrete operations:
wi,t=(1- λ) wi,t-1+λBMt
μi,t=(1- α) μi,t-1+αXi,t
∑i,t=(1- α) ∑i,t-1+α(Xi,t-μi,t)(Xi,t-μi,t)T
α=λ/wi,t
Wherein, pixel is prospect then BM=0, and otherwise BM=1, finally replaces weight the smallest with an initial Gauss model
That Gauss model, threshold value T, learning rate λ, parameter beta are all constants specified in advance.
7. the fall detection method according to claim 1 based on convolutional neural networks, which is characterized in that
Test set " is input to the model after training, is detected with the precision of test the set pair analysis model by step；" in, it is described
Test set comes from UR Fall Detection Dataset.
8. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor realizes any one of claims 1 to 7 the method when executing described program
Step.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
The step of any one of claims 1 to 7 the method is realized when row.
10. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit requires 1 to 7 described in any item methods.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810614024.4A CN108961675A (en) | 2018-06-14 | 2018-06-14 | Fall detection method based on convolutional neural networks |
| PCT/CN2018/107975 WO2019237567A1 (en) | 2018-06-14 | 2018-09-27 | Convolutional neural network based tumble detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810614024.4A CN108961675A (en) | 2018-06-14 | 2018-06-14 | Fall detection method based on convolutional neural networks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108961675A true CN108961675A (en) | 2018-12-07 |
Family
ID=64488772
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810614024.4A Pending CN108961675A (en) | 2018-06-14 | 2018-06-14 | Fall detection method based on convolutional neural networks |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108961675A (en) |
| WO (1) | WO2019237567A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871788A (en) * | 2019-01-30 | 2019-06-11 | 云南电网有限责任公司电力科学研究院 | A kind of transmission of electricity corridor natural calamity image recognition method |
| CN112489368A (en) * | 2020-11-30 | 2021-03-12 | 安徽国广数字科技有限公司 | Intelligent falling identification and detection alarm method and system |
| CN113269105A (en) * | 2021-05-28 | 2021-08-17 | 西安交通大学 | Real-time faint detection method, device, equipment and medium in elevator scene |
| CN113435306A (en) * | 2021-06-24 | 2021-09-24 | 三峡大学 | Fall detection method and device based on hybrid cascade convolution |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111209848B (en) * | 2020-01-03 | 2023-07-21 | 北京工业大学 | Real-time falling detection method based on deep learning |
| CN111353394B (en) * | 2020-02-20 | 2023-05-23 | 中山大学 | Video behavior recognition method based on three-dimensional alternate update network |
| CN111523492B (en) * | 2020-04-26 | 2023-04-18 | 安徽皖仪科技股份有限公司 | Detection method of black smoke vehicle |
| CN111598042B (en) * | 2020-05-25 | 2023-04-07 | 西安科技大学 | Visual statistical method for underground drill rod counting |
| CN111680614B (en) * | 2020-06-03 | 2023-04-14 | 安徽大学 | Abnormal behavior detection method based on video monitoring |
| CN111782857B (en) * | 2020-07-22 | 2023-11-03 | 安徽大学 | Footprint image retrieval method based on mixed attention-dense network |
| CN112541403B (en) * | 2020-11-20 | 2023-09-22 | 中科芯集成电路有限公司 | Indoor personnel falling detection method by utilizing infrared camera |
| CN112528775A (en) * | 2020-11-28 | 2021-03-19 | 西北工业大学 | Underwater target classification method |
| CN113947612B (en) * | 2021-09-28 | 2024-03-29 | 西安电子科技大学广州研究院 | Video anomaly detection method based on foreground and background separation |
| CN114049585B (en) * | 2021-10-12 | 2024-04-02 | 北京控制与电子技术研究所 | Mobile phone operation detection method based on motion prospect extraction |
| CN116469132B (en) * | 2023-06-20 | 2023-09-05 | 济南瑞泉电子有限公司 | Fall detection method, system, equipment and medium based on double-flow feature extraction |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134068A (en) * | 2014-08-12 | 2014-11-05 | 江苏理工学院 | Monitored vehicle characteristic representation and classification method based on sparse coding |
| CN107220604A (en) * | 2017-05-18 | 2017-09-29 | 清华大学深圳研究生院 | A kind of fall detection method based on video |
| CN108090458A (en) * | 2017-12-29 | 2018-05-29 | 南京阿凡达机器人科技有限公司 | Tumble detection method for human body and device |
| CN108124119A (en) * | 2016-11-28 | 2018-06-05 | 天津市军联科技有限公司 | Intelligent video monitoring system based on built-in Linux |
| CN108154113A (en) * | 2017-12-22 | 2018-06-12 | 重庆邮电大学 | Tumble event detecting method based on full convolutional network temperature figure |
-
2018
- 2018-06-14 CN CN201810614024.4A patent/CN108961675A/en active Pending
- 2018-09-27 WO PCT/CN2018/107975 patent/WO2019237567A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134068A (en) * | 2014-08-12 | 2014-11-05 | 江苏理工学院 | Monitored vehicle characteristic representation and classification method based on sparse coding |
| CN108124119A (en) * | 2016-11-28 | 2018-06-05 | 天津市军联科技有限公司 | Intelligent video monitoring system based on built-in Linux |
| CN107220604A (en) * | 2017-05-18 | 2017-09-29 | 清华大学深圳研究生院 | A kind of fall detection method based on video |
| CN108154113A (en) * | 2017-12-22 | 2018-06-12 | 重庆邮电大学 | Tumble event detecting method based on full convolutional network temperature figure |
| CN108090458A (en) * | 2017-12-29 | 2018-05-29 | 南京阿凡达机器人科技有限公司 | Tumble detection method for human body and device |
Non-Patent Citations (1)
| Title |
|---|
| 雷帮军等: "《视频目标跟踪系统分步详解》", 31 December 2015 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871788A (en) * | 2019-01-30 | 2019-06-11 | 云南电网有限责任公司电力科学研究院 | A kind of transmission of electricity corridor natural calamity image recognition method |
| CN112489368A (en) * | 2020-11-30 | 2021-03-12 | 安徽国广数字科技有限公司 | Intelligent falling identification and detection alarm method and system |
| CN113269105A (en) * | 2021-05-28 | 2021-08-17 | 西安交通大学 | Real-time faint detection method, device, equipment and medium in elevator scene |
| CN113435306A (en) * | 2021-06-24 | 2021-09-24 | 三峡大学 | Fall detection method and device based on hybrid cascade convolution |
| CN113435306B (en) * | 2021-06-24 | 2022-07-19 | 三峡大学 | Fall detection method and device based on hybrid cascade convolution |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019237567A1 (en) | 2019-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961675A (en) | Fall detection method based on convolutional neural networks | |
| CN111126472B (en) | SSD (solid State disk) -based improved target detection method | |
| CN108648191B (en) | Pest image recognition method based on Bayesian width residual error neural network | |
| CN111444828B (en) | Model training method, target detection method, device and storage medium | |
| CN103810490B (en) | A kind of method and apparatus for the attribute for determining facial image | |
| CN106504064A (en) | Clothes classification based on depth convolutional neural networks recommends method and system with collocation | |
| CN111754396B (en) | Face image processing method, device, computer equipment and storage medium | |
| CN109299716A (en) | Training method, image partition method, device, equipment and the medium of neural network | |
| CN108229330A (en) | Face fusion recognition methods and device, electronic equipment and storage medium | |
| CN109214298B (en) | Asian female color value scoring model method based on deep convolutional network | |
| CN109598234A (en) | Critical point detection method and apparatus | |
| CN106778852A (en) | A kind of picture material recognition methods for correcting erroneous judgement | |
| Qu et al. | A pedestrian detection method based on yolov3 model and image enhanced by retinex | |
| CN108647625A (en) | A kind of expression recognition method and device | |
| CN108363997A (en) | It is a kind of in video to the method for real time tracking of particular person | |
| CN112464865A (en) | Facial expression recognition method based on pixel and geometric mixed features | |
| CN109711283A (en) | A kind of joint doubledictionary and error matrix block Expression Recognition algorithm | |
| CN107194937A (en) | Tongue image partition method under a kind of open environment | |
| CN109359515A (en) | A kind of method and device that the attributive character for target object is identified | |
| CN104298974A (en) | Human body behavior recognition method based on depth video sequence | |
| CN109886153A (en) | A kind of real-time face detection method based on depth convolutional neural networks | |
| CN109753864A (en) | A kind of face identification method based on caffe deep learning frame | |
| CN109902613A (en) | A kind of human body feature extraction method based on transfer learning and image enhancement | |
| CN110008853A (en) | Pedestrian detection network and model training method, detection method, medium, equipment | |
| CN107818299A (en) | Face recognition algorithms based on fusion HOG features and depth belief network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181207 |