CN109598197A - The design method of hourglass model based on intensive link block - Google Patents
The design method of hourglass model based on intensive link block Download PDFInfo
- Publication number
- CN109598197A CN109598197A CN201811283711.9A CN201811283711A CN109598197A CN 109598197 A CN109598197 A CN 109598197A CN 201811283711 A CN201811283711 A CN 201811283711A CN 109598197 A CN109598197 A CN 109598197A
- Authority
- CN
- China
- Prior art keywords
- response
- link block
- intensive
- design method
- stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013461 design Methods 0.000 title claims abstract description 9
- 230000004044 response Effects 0.000 claims abstract description 47
- 238000010586 diagram Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 abstract description 7
- 238000012549 training Methods 0.000 abstract description 5
- 238000000605 extraction Methods 0.000 abstract description 4
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000004927 fusion Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 5
- 101000742346 Crotalus durissus collilineatus Zinc metalloproteinase/disintegrin Proteins 0.000 description 3
- 101000872559 Hediste diversicolor Hemerythrin Proteins 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to Attitude estimation field, the design method of the hourglass model based on intensive link block is proposed, the prediction of the human joint points of the Attitude estimation based on bottom-up structure is realized.The present invention realizes the extraction of joint point feature using intensive connection network module, and the pattern image extracted by fusion Gaussian template function and residual error module is at overall response figure.By the abundant study to each artis spatial information feature being realized, to promote the predictablity rate of artis using overall response figure on last stage as the input of next stage.Model is trained in deep learning frame, training result proves that the model compares master mould and has promotion on the predictablity rate of partial joint point.Illustrate the validity of the model of neotectonics in the invention.
Description
Technical field
The present invention relates to Attitude estimation fields, are a kind of design methods of hourglass model based on intensive link block.
Specifically related to the neural network model for constructing a kind of similar hourglass configuration using intensive link block, pass through realization pair after training
The prediction of human joint points.
Background technique
Since depth convolutional neural networks have a great advantage in terms of feature extraction, more and more scholars are by convolution
In processing Computer Vision Task, human body attitude estimates that a branch as visual task results in greatly for network application
Progress, in today of image processing techniques high speed development, Attitude estimation is due to image background, shooting visual angle, and human body clothing hides
The factors such as gear influence artis precision of prediction, so human body attitude estimation is still a great problem of visual task.
Tompson et al. is by estimating joint deviation post with one " position accurate adjustment " in some zonule of image
Method constructs model, this network model cascades using advanced convolutional network training, and that human synovial can be improved in this way is fixed
Position precision；The CPM method that Wei, Shih-En et al. are proposed has used ordered convolution framework to come expression of space information and texture
Information, this method supervise anti-training using relaying and have stopped gradient disperse problem, improve joint to all supervised trainings of each stage
The precision of point location；Newell et al. proposes Stacked Hourglass structure, using similar hourglass-shaped network shape,
The top-down artis position that human body is inferred to bottom-up method is reused, so that positioning progress is greatly mentioned
It rises.The intensive connection network model that G.Huang et al. is proposed all interconnects all layers in network, so that every in network
One layer all receive it before all layers of connection may learn richer to maximise the information flow in network
Feature.It is not high as precision in complicated intra-articular irrigation in current human's pose presentation to solve the problems, such as.
Although forefathers devise a variety of models in the human body attitude estimation based on deep neural network, propose much in
After supervision thought be the model algorithm instructed, but in human body attitude estimation field, to changing for the hourglass prototype network of forefathers
Into the research of the new network architecture is still of great importance.
Summary of the invention
It is an object of the invention to propose a kind of a kind of neural network for intensive link block being constructed to similar hourglass configuration
Intensive connection network model is combined with network model of skipping a grade, prediction knot less to parameter is realized by Fusion Features by model
The accurate neural network structure model of fruit.
The technical solution of the present invention is as follows: extracting the image conduct of 64*64 by convolutional layer to pre-processing image data first
Input, divides two-way to handle the image, retains the raw information of image by pattern handling of skipping a grade all the way, all the way to passing through
Intensive connection e-learning characteristics of image, carries out image reasoning, by the processing result of two-way and the Gaussian template letter that generates in advance
Number is merged to obtain overall response figure, then using obtained overall response figure as the input of next stage, passes through operation before
Step, the overall response figure finally obtained realize the function of joint point prediction.
The design method of hourglass model based on intensive link block, specifically includes the following steps:
Step 1: load human body attitude image data set；
Step 2: data prediction being carried out to data set, processing method is to carry out process of convolution, filter to image data
Size be 7*7, step-length 2；
Step 3: calculating the response diagram of the different parts under different scale, response diagram is divided into two parts: by intensive connection
A liter sampling operation is carried out after network processes again and is formed by characteristic response figure, and intensive link block is extracted in full size
Characteristic response figure；
Step 4: the response diagram for merging different parts is global response figure, and finds the peak response point at each position as prediction
Value；The global response figure is that the characteristic response figure extracted by a Gaussian template function and step 3 is added gained；
Step 5: using the characteristic pattern predicted on last stage as the input of next stage, repeating step 2,3,4；
Step 6: global response figure of every output is a stage, and this method is total up to four-stage.
The beneficial effects of the present invention are:
1, the present invention is shared using weight compared to master mould in trained each stage, reduces trained parameter
Amount, reduces the demand to computing resource.
2, the present invention uses intensive connection network module in feature extraction phases, available compared to master mould richer
Feature and each joint spatial information.
3, by the improvement to former hourglass model in the present invention, overall response figure on last stage is defeated as next stage
Enter, reduces calculating process, and promote the accuracy rate of joint point prediction.
Detailed description of the invention
Fig. 1 is the global procedures flow chart of the hourglass model based on intensive link block；
Fig. 2 is the characteristic extracting module explanatory diagram of hourglass model；
Fig. 3 is intensive connection network module explanatory diagram.
Specific embodiment
Present invention will be further explained below with reference to the attached drawings and examples.
In conjunction with the global procedures flow chart of Fig. 1 and the intensive connection network mould of the characteristic extracting module figure of Fig. 2 and Fig. 3
The step introduction in detail below of block figure:
Step 1: load human body attitude image data set Z；
Step 2: data prediction being carried out to data set Z, inputs and is expressed as H*W*3, mistake for the high * wide * port number of image
The size of filter is 7*7, and step-length 2 obtains the characteristic pattern of 64*64；
Step 3: primitive character figure is input in the hourglass model sub-network of intensive link block after down-sampled,
Result is exported after linear unit processing, the characteristic pattern that output channel is 256 is obtained, finally passes through convolution nuclear volume k=
1, the response diagram of H/2*W/2*K is obtained after the convolution filter of convolution step-length s=1；
Step 4: by the response diagram that step 3 obtains and the characteristic response figure that residual error module is extracted in full size and in advance
The Gaussian response stencil function of generation is added, and obtains overall response figure, by minimizing function ftFind the peak response at each position
Point is predicted value.
Wherein z is the subset of data set Z, and p is p-th of artis,The response diagram of some artis,For the pass
The true value in stage.
Step 5: this method is total up to four-stage, and global response figure of every output is a stage, will on last stage
The characteristic pattern of prediction as next stage input such as using the output result of stage1 as the input of stage2, repeatedly step 2,
3、4。
Embodiment 1
The embodiment of the present invention is implemented under the premise of the technical scheme of the present invention, in deep learning frame
It is run under TensorFlow, the detailed implementation method and specific operation process are given, but protection scope of the present invention is unlimited
In following embodiments.The present embodiment at data set FLIC and treated MPII data set to run
Step 1: load human body attitude image data set Z；
Step 2: data prediction is carried out to data set Z, is inputted as H*W*3, the size of filter is 7*7, step-length 2,
Obtain the characteristic pattern of 64*64；
Step 3: primitive character figure is input in the hourglass model sub-network of intensive link block after down-sampled,
Result is exported after linear unit processing, the characteristic pattern that output channel is 256 is obtained, finally passes through k=1, the volume of s=1
The response diagram of H/2*W/2*K is obtained after product filtering；
Step 4: by the response diagram that step 3 obtains and the characteristic response figure that residual error module is extracted in full size and in advance
The Gaussian response stencil function of generation is added, and obtains overall response figure, by minimizing function ftFind the peak response at each position
Point is predicted value.
Wherein p is p-th of artis,For the response diagram of some artis,For the true value in the pass stage.
Step 5: this method is total up to four-stage, and global response figure of every output is a stage, will on last stage
The characteristic pattern of prediction as next stage input such as using the output result of stage1 as the input of stage2, repeatedly step 2,
3、4。
Step 6: EP (end of program).
In conclusion being carried out using the hourglass model of intensive connection network module construction by intensively connecting network module
Feature extraction, and being merged with the feature for retaining artis raw information, by minimizing the true value of all artis and pre-
The L2 normal form of measured value composition generates total characteristic response diagram, and by deep learning frame TensorFlow in FLIC and MPII
It is trained on data set, experimental result further demonstrates the validity of improved hourglass model, thus proves this method
It is effective and feasible.
Table 1: the predictablity rate of artis is compared for this method on FLIC data set and original method
Table 2: the predictablity rate of artis is compared for this method on MPII data set and original method
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Anyone skilled in the art within the technical scope of the present disclosure, according to the technique and scheme of the present invention and its
Inventive concept is subject to equivalent substitution or change, should be covered by the scope of protection of the present invention.
Claims (4)
1. the design method of the hourglass model based on intensive link block, which comprises the following steps:
Step 1: load human body attitude image data set；
Step 2: data prediction is carried out to data set；
Step 3: calculating the response diagram of the different parts under different scale；
Step 4: the response diagram for merging different parts is global response figure, and the peak response point for finding each position is predicted value；
Step 5: using the characteristic pattern predicted on last stage as the input of next stage, repeating step 2,3,4；
Step 6: global response figure of every output is a stage, and this method is total up to four-stage.
2. the design method of the hourglass model according to claim 1 based on intensive link block, it is characterised in that: step
Data preprocessing method described in 2 is to carry out process of convolution to image data, the size of filter is 7*7, step-length 2.
3. the design method of the hourglass model according to claim 1 based on intensive link block, it is characterised in that: step
The response diagram of the different size lower components of calculating described in 3 is divided into two parts: being risen again after intensive connection network processes
Sampling operation is formed by characteristic response figure, and the characteristic response figure that intensive link block is extracted in full size.
4. the design method of the hourglass model according to claim 1 based on intensive link block, it is characterised in that: step
4 concrete operation method are as follows: by response diagram that step 3 obtains and the characteristic response figure that residual error module is extracted in full size and
The Gaussian response stencil function being generated in advance is added, and obtains overall response figure；
By minimizing function ftThe peak response point for finding each position is predicted value:
Wherein z is the subset of data set Z, and p is p-th of artis,For the response diagram of some artis,For the pass rank
The true value of section.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811283711.9A CN109598197A (en) | 2018-10-31 | 2018-10-31 | The design method of hourglass model based on intensive link block |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811283711.9A CN109598197A (en) | 2018-10-31 | 2018-10-31 | The design method of hourglass model based on intensive link block |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109598197A true CN109598197A (en) | 2019-04-09 |
Family
ID=65958338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811283711.9A Pending CN109598197A (en) | 2018-10-31 | 2018-10-31 | The design method of hourglass model based on intensive link block |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109598197A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110503063A (en) * | 2019-08-28 | 2019-11-26 | 东北大学秦皇岛分校 | Fall detection method based on hourglass convolution autocoding neural network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106991646A (en) * | 2017-03-28 | 2017-07-28 | 福建帝视信息科技有限公司 | A kind of image super-resolution method based on intensive connection network |
| CN107818302A (en) * | 2017-10-20 | 2018-03-20 | 中国科学院光电技术研究所 | Non-rigid multiple dimensioned object detecting method based on convolutional neural networks |
| CN108710830A (en) * | 2018-04-20 | 2018-10-26 | 浙江工商大学 | A kind of intensive human body 3D posture estimation methods for connecting attention pyramid residual error network and equidistantly limiting of combination |
-
2018
- 2018-10-31 CN CN201811283711.9A patent/CN109598197A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106991646A (en) * | 2017-03-28 | 2017-07-28 | 福建帝视信息科技有限公司 | A kind of image super-resolution method based on intensive connection network |
| CN107818302A (en) * | 2017-10-20 | 2018-03-20 | 中国科学院光电技术研究所 | Non-rigid multiple dimensioned object detecting method based on convolutional neural networks |
| CN108710830A (en) * | 2018-04-20 | 2018-10-26 | 浙江工商大学 | A kind of intensive human body 3D posture estimation methods for connecting attention pyramid residual error network and equidistantly limiting of combination |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110503063A (en) * | 2019-08-28 | 2019-11-26 | 东北大学秦皇岛分校 | Fall detection method based on hourglass convolution autocoding neural network |
| CN110503063B (en) * | 2019-08-28 | 2021-12-17 | 东北大学秦皇岛分校 | Falling detection method based on hourglass convolution automatic coding neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106127684B (en) | Image super-resolution Enhancement Method based on forward-backward recutrnce convolutional neural networks | |
| CN104978580B (en) | A kind of insulator recognition methods for unmanned plane inspection transmission line of electricity | |
| CN108090403A (en) | Face dynamic identification method and system based on 3D convolutional neural network | |
| CN109271933A (en) | The method for carrying out 3 D human body Attitude estimation based on video flowing | |
| CN106897714A (en) | A kind of video actions detection method based on convolutional neural networks | |
| CN106683048A (en) | Image super-resolution method and image super-resolution equipment | |
| CN106780543A (en) | A kind of double framework estimating depths and movement technique based on convolutional neural networks | |
| CN107609638A (en) | A kind of method based on line decoder and interpolation sampling optimization convolutional neural networks | |
| CN110889343A (en) | Crowd density estimation method and device based on attention type deep neural network | |
| CN107808129A (en) | A kind of facial multi-characteristic points localization method based on single convolutional neural networks | |
| CN109902548A (en) | A kind of object properties recognition methods, calculates equipment and system at device | |
| CN108776971A (en) | A kind of variation light stream based on layering nearest-neighbor determines method and system | |
| CN109299732A (en) | The method, apparatus and electronic equipment of unmanned behaviour decision making and model training | |
| CN109783887A (en) | A kind of intelligent recognition and search method towards Three-dimension process feature | |
| CN108154104A (en) | A kind of estimation method of human posture based on depth image super-pixel union feature | |
| CN110473284A (en) | A kind of moving object method for reconstructing three-dimensional model based on deep learning | |
| CN110276768A (en) | Image partition method, image segmentation device, image segmentation apparatus and medium | |
| CN110246171A (en) | A kind of real-time monocular video depth estimation method | |
| CN111881716A (en) | Pedestrian re-identification method based on multi-view-angle generation countermeasure network | |
| Zhang et al. | Dilated hourglass networks for human pose estimation | |
| Jiang et al. | Multi-level memory compensation network for rain removal via divide-and-conquer strategy | |
| CN104091364B (en) | Single-image super-resolution reconstruction method | |
| CN113449878B (en) | Data distributed incremental learning method, system, equipment and storage medium | |
| Zhang et al. | FCHP: Exploring the discriminative feature and feature correlation of feature maps for hierarchical DNN pruning and compression | |
| CN109598197A (en) | The design method of hourglass model based on intensive link block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190409 |