CN109002783A - Rescue the human testing in environment and gesture recognition method - Google Patents
Rescue the human testing in environment and gesture recognition method Download PDFInfo
- Publication number
- CN109002783A CN109002783A CN201810710454.6A CN201810710454A CN109002783A CN 109002783 A CN109002783 A CN 109002783A CN 201810710454 A CN201810710454 A CN 201810710454A CN 109002783 A CN109002783 A CN 109002783A
- Authority
- CN
- China
- Prior art keywords
- human body
- network
- human
- body frame
- gesture recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Probability & Statistics with Applications (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention proposes a kind of human testing rescued in environment and gesture recognition methods, belong to the human testing and gesture recognition technical field in rescue environment, it is mainly used in intelligent detecting robot, the personnel's detection for completing the disaster scene of the accident and rescue task.Detailed process includes: that this method acquires rescue site real-time image information by camera and is transferred in server, and server intercepts video frame first, carries out primary defogging in real time using mean filter and handles.Then, the relevant calculation of human testing is completed using mist elimination image to detect human region.Finally, being directed to each human body image, the relevant calculation for completing gesture recognition detects human body key point and attitude vectors, and posture judgement is completed by the ratio of width to height of human body frame and human body attitude vector.
Description
Technical field:
The invention belongs to rescue the human testing in environment and gesture recognition technical field.It is existing to rescue more particularly to one kind
Field picture carry out defogging after detect human body and carry out gesture recognition so that it is determined that the wounded method.
Background technique
In recent years, due to natural calamity, terrorist activity and all kinds of emergency situations, disaster accident emerges one after another, to economy
Great loss is brought with social life, becomes the great subject under discussion of current safety guarantee.In rescue after the accident
Cheng Zhong, top priority are exactly to detect and succour the wounded, ensure the life security of all personnel.But since scene of the accident environment is disliked
It is bad, and be easy there are a large amount of toxic and harmful gas, the failure of convention security detection means, rescue work is difficult to carry out.This
Under dangerous and complex environment, the human testing of view-based access control model and gesture recognition technology can make intelligent robot assist rescue people
Member completes the task that the wounded search and rescue, to avoid the secondary injures and deaths during rescue.
Currently, the human testing algorithm of view-based access control model is broadly divided into two classes: human testing based on manual feature and being based on
The human testing of deep learning.Wherein, most common two kinds of manual feature methods are HOG algorithm and DPM algorithm.Based on manual spy
The shortcomings that sign method is poor robustness, and verification and measurement ratio is low.Simultaneously as its detection time is long, it is difficult to meet practical application.Depth
Habit technology can learn the internal characteristics of image automatically, make up the problem of manual feature generalization ability difference, nowadays also start to be answered
For human testing problem.Comparing outstanding algorithm includes ConvNet, JointDeep etc..But network model is not deep enough, inspection
It is low to survey precision.Detection time is long simultaneously, is not able to satisfy requirement of real-time.
On the other hand, it since rescue site resource is limited, needs that resource stress according to the emergency of the state of affairs
Distribution, it is possible to the injured situation of Field Force is judged by gesture recognition technology, thus reasonable distribution field resources.Mesh
Preceding estimation method of human posture can be mainly divided into the frame based on component and two kinds of frame based on entirety.Based on component
Frame detects human body first, then assembles human body and forms multiple human postures, and that representative is openpose,
Disadvantage is exactly when densely populated place, and the position that can will not belong to the same person is spliced, and is lost from global carry out portion
The ability of position identification.It is exactly that key point inspection is integrally carried out to everyone on the basis of pedestrian detection based on whole frame
It surveys, the disadvantage is that examined frame influences greatly, the accuracy of Attitude estimation depends primarily on the quality of human body bounding box.
Simultaneously as rescue site often has a large amount of smog dusts, cause that the image viewability got is poor, details mould
Paste, greatly affected the detection accuracy of vision-based inspection system.And existing gesture recognition algorithms are only to export
Key point coordinate, therefore, it is also desirable to which the specific posture to people is analyzed.
Summary of the invention
The main object of the present invention be propose it is a kind of rescue environment in human testing and gesture recognition method, this method
It can be applied in intelligent detecting robot, the personnel's detection for completing the disaster scene of the accident and rescue task.Mainly face as
Lower problem:
1. rescue site often has a large amount of smog dusts, cause that the image viewability got is poor, details is fuzzy, greatly
Ground affects the detection accuracy of vision-based inspection system.
2. the requirement rescued in scene for human testing real-time is high, while meeting high-accuracy, detection speed
Degree is also required to reach requirement of real-time.
3. existing single gesture recognition algorithms depend critically upon the human body bounding box quality detected, only human body is located at
The high-precision bounding box in image center can satisfy the needs of gesture recognition.
4. existing gesture recognition algorithms are only to export key point coordinate, it is unable to get the specific posture of people.
In order to solve the above technical problems, the invention proposes human testings and gesture recognition in a kind of rescue environment
Method, this method acquire rescue site real-time image information by camera and are transferred in server, and server intercepts first
Video frame carries out primary defogging processing in real time.Then, human testing is completed using mist elimination image.Finally, being directed to each human figure
Picture completes human body attitude identification, and the human body attitude vector identified is combined to complete people with the human body frame the ratio of width to height detected
The judgement of body posture.The specific workflow of the method for the present invention is as follows:
S1 realtime graphic defogging
This method obtains original image, mist elimination image, environment light and global atmosphere by the physical model of foggy image
Relationship between light estimates the value of environment light and global atmosphere light, finally recovers clearly then by mean filter
Fog free images.
S2 completes human testing using mist elimination image
Algorithm uses multi-scale prediction, respectively in the enterprising pedestrian's body of characteristic pattern of 8 × 8,16 × 16 and 32 × 32 sizes
The prediction of frame.On each scale, input picture is divided into N × N number of grid, in each grid, passes through fine tuning human body default
Frame is predicted, for each default frame, the top left co-ordinate of neural network forecast human body frame, the width of human body frame and height and human body frame
Confidence level.Finally, the human body frame of neural network forecast is obtained final prediction result by a non-maxima suppression.
S3 carries out gesture recognition using the human body image that detection obtains
The gesture recognition process of this method is individually cut out the human body block diagram that human testing obtains first to obtain a width
Independent human body image carries out people using the method that RMPE gesture recognition frame is combined with Stacked Hourglass network
Body gesture recognition.Specific step is as follows:
S3.1 cuts out human body picture
Come in order to ensure that can be cut out complete human body, on the basis of the human body frame that human testing obtains, wide extension
30%, height extends 20% and is cut out.Verified, the human body in most of picture can completely be cut out by the method
Out.
The human region of S3.2 use space converting network extraction high quality
S3.3 carries out single Attitude estimation using Stacked Hourglass network
Attitude estimation result is mapped back original image by S3.4 use space contravariant switching network
S3.5 eliminates posture redundancy using PNMS, obtains final carriage vector.
S4 judges human body attitude
In rescue scene, it can determine if to need to succour immediately by judging whether personnel are in sprawl.
Therefore, this method passes through obtained human body frame and attitude vectors, is sentenced using the completion of simple geometrical constraint to human body attitude
It is disconnected.The geometrical constraint used includes the ratio of width to height of human body frame and the deflection of trunk.Finally judge human body whether be
Sprawl, and posture label is assigned for everyone.
The present invention has the advantage that
This method replaces tradition k-means clustering algorithm in YOLOv3 detection framework to obtain using k-means++ clustering algorithm
The width and height of default human body frame are taken, and the detection framework is applied to the human testing in rescue environment, to meet height
The requirement of accuracy and real-time.Meanwhile being combined using RMPE gesture recognition frame with Stacked Hourglass network
Method carries out human body attitude identification, so that also can preferably complete gesture recognition task when human testing frame quality deficiency.And
Human body attitude judgement is carried out by human testing and gesture recognition result, to obtain the specific posture of people.Finally, using letter
Single mean filter completes real-time defogging processing, so that method may operate in the rescue site of bad environments.
Detailed description of the invention
Fig. 1 rescues human testing and gesture recognition method flow chart in environment；
The flow diagram of Fig. 2 realtime graphic defogging processing；
Fig. 3 human detection result exemplary diagram, wherein figure (a) is testing result under sprawl, figure (b) is under other postures
Testing result；
Fig. 4 human body attitude identification process figure；
Fig. 5 human body attitude recognition result schematic diagram, wherein figure (a) is recognition result under sprawl, figure (b) is other appearances
Recognition result under gesture；
Fig. 6 human body attitude judging result schematic diagram, wherein figure (a) is judging result under sprawl, figure (b) is other appearances
Judging result under gesture
Specific embodiment
With reference to the accompanying drawing with example, elaborate for this method.
Fig. 1 is human testing and gesture recognition method flow chart in rescue environment proposed by the present invention.This method is logical
It crosses camera acquisition rescue site real-time image information and is transferred in server, server intercepts video frame first, using equal
Value filtering carries out primary defogging processing in real time.Then, the relevant calculation of human testing is completed using mist elimination image to detect
Human region.Finally, be directed to each human body image, complete gesture recognition relevant calculation detect human body key point and posture to
Amount, and posture judgement is completed by the ratio of width to height of human body frame and human body attitude vector.Specific step is as follows:
1, realtime graphic defogging
Fig. 2 is the flow diagram of realtime graphic defogging processing.In rescue site, often there are a large amount of smog dusts.Cause
This, can describe the original foggy image of rescue site using physical model formula shown in formula (1).
H (x)=F (x) t (x)+A (1-t (x)) (1)
Wherein, H (x) is the original image of rescue site；F (x) is treated mist elimination image；A indicates global atmosphere light
Ingredient；T (x) indicates transmissivity.A (1-t (x)) is referred to as environment light, is the factor for influencing the offset of image chroma-luminance.
In formula (1), L (x)=A (1-t (x)) is enabled, then formula (1) can be converted into formula (2).
Therefore, it can be seen that mist elimination image can be calculated by estimation overall situation atmosphere light ingredient A and environment light L (x).
Formula (3) can be obtained by formula (1).
A(1-t(x))≤H(x) (3)
The minimum value of calculation formula (3) two sides primary display channels enablesFormula (4) can be obtained.
Wherein, { r, g, b } is red green blue tricolor channel, and M (x) is the minimum value of original image primary display channels.
This method is to progress mean filter estimation transmissivity t (x) on the right side of formula (4).It enablesAnd an estimated value is added, in result for making up mean filter result and true
The difference of value.The estimated value of transmissivityIt is calculated by formula (5).
Wherein, Mave(x) for the minimum values of original image primary display channels carry out mean filter operation as a result,For mean filter operation, saFor the size of mean filter window,For the coefficient of estimated value, and meetIt enablesFormula (5) can be converted to formula (6).
In order to control the bright-dark degree of mist elimination image, coefficient is setWherein, ρ is adjustable parameter.At this
In example, ρ takes 1.3.mavFor the mean value of all elements in M (x).Also, coefficient δ is defined using formula (7).
δ=min (ρ mav,0.9) (7)
Formula (8) can be obtained according to formula (4), formula (6) and formula (7).
Then this method ambient light estimation method such as formula (9) is described.
L (x)=min (min (ρ mav,0.9)Mave(x),M(x)) (9)
Due toTherefore formula (10) can be obtained.
Wherein, ε is the preset ratio factor.In this example, it is contemplated that the rapidity of algorithm, ε=0.5 is arranged in we, benefit
Use Mave(x) and H (x) can obtain formula (11).
Finally, calculating mist elimination image by estimation overall situation atmosphere light ingredient A and environment light L (x) according to formula (2).
2, real-time body detects
Human testing is carried out using improved YOLOv3 algorithm herein.Specific step is as follows:
The training of 2.1 human testing networks
The preparation of sample database
Human sample is acquired as positive sample.In this example based on COCO public data collection, using wherein including
The picture of human sample uses the markup information of human body classification in original data set as training set positive sample, markup information.
The training of neural network
All training samples are inputted network in batches to be trained.Since batch size is bigger higher to hsrdware requirements
The training time is more long simultaneously, and the too small arithmetic accuracy of batch size is low.Comprehensively consider factors above, in instances, every batch of uses
64 samples pictures.When training, the loss function using error of sum square function as training process.
Meanwhile in the training process, BN layers of acceleration convergence have been used.Also, it is every to pass through 10 training, it randomly selects new
Size is inputted, size is 32 multiple, and minimum 320 × 320, it is up to 608 × 608.
2.2k-means++ obtains human body and defaults frame
In order to improve the detection accuracy of YOLOv3 algorithm, this method is estimated in extraction default and is improved in method.It uses
K-means++ clustering algorithm seeks the width and height of n default frame needed for YOLOv3 algorithm.Wherein, the value of n is bigger, calculates
The method precision the slower at high speed.In this example, n value is 9 by the requirement for comprehensively considering precision and speed.K-means++ is poly-
Class algorithm steps are as follows:
(1) sample point is selected as first cluster centre at random；
(2) each sample point and the cluster centre distance d (x) nearest from it in data set are calculated；
(3) select a new sample point as new cluster centre, selection principle are as follows: d (x) big point is selected several
Rate is big；
(4) step (2) and step (3) are repeated until selecting n cluster centre；
(5) it calculates each sample point and arrives the distance of cluster centre, and be divided into the smallest one kind cluster；
(6) the coordinate average value for calculating all the points in each cluster, as new cluster centre；
(7) step (5) and step (6) are repeated, until cluster centre does not move.
In cluster process, using distance metric formula shown in formula (13).
D (box, centroid)=1-IOU (box, centroid) (13)
Wherein, box is the human body frame in sample database, and centroid is cluster centre, and IOU (box, centroid) is two
The friendship in region and ratio, i.e. two regions intersection part divided by two region union parts result.
2.3 complete human testing using mist elimination image
Multi-scale prediction is used during human testing, respectively in the characteristic pattern of 8 × 8,16 × 16 and 32 × 32 sizes
The prediction of enterprising pedestrian's body frame.On each scale, input picture is divided into N × N number of grid, in each grid, by micro-
Mediator's body default frame is predicted, for each default frame, the width and height of the top left co-ordinate of neural network forecast human body frame, human body frame
And the confidence level of human body frame.Finally, the human body frame of neural network forecast is obtained final prediction by a non-maxima suppression
As a result.For human detection result as shown in figure 3, wherein figure (a) is testing result under sprawl, figure (b) is to examine under other postures
Survey result.
3, human body attitude identifies
Fig. 4 is human body gesture recognition flow chart.Specific step is as follows:
The preparation of 3.1 sample databases
User's physical examination survey grid network carries out human testing to MPII human body attitude database first, obtains a series of human body
Frame.According to the relationship of offset and human body attitude between the obtained human body frame of detection and real human body frame, Gaussian Mixture point is used
Cloth is modeled, and distribution P (δ B | atom (P)) is obtained, and wherein δ B is the offset of human body frame, and atom (P) is the atom appearance of human body
State.Finally, according to the human body attitude of mark, sample is extended using distribution P (δ B | atom (P)).
The training of 3.2 neural networks
It is anti-to spatial alternation network, Stacked Hourglass network, space using above-mentioned samples pictures in training
Converting network is finely adjusted simultaneously, so that spatial alternation network and spatial inverse transform network can adapt to gesture recognition automatically
Task.
Meanwhile a parallel Stacked Hourglass network is increased in training to promote accurately estimating for posture
Meter.The network and original Stacked Hourglass network share spatial alternation network, but space contravariant is not added after network
Switching network.Each branch uses mean square error as loss function, and total losses is the sum of two branch penalties.
3.3 complete gesture recognition using the human body frame detected
Firstly, use space converting network extracts the human region of high quality, formula is defined as shown in formula (14).
Wherein, θ1,θ2,θ3For spatial transform coefficient,For the coordinate value before spatial alternation,For space change
Coordinate value after changing.
After the completion of based on the single gesture recognition of Stacked Hourglass network, use space contravariant switching network will
Attitude estimation result maps back original image, and formula is defined as shown in formula (15).
Wherein, γ1,γ2,γ3For spatial inverse transform coefficient, formula is defined as shown in formula (16), (17).
[γ1γ2]=[θ1θ2]-1 (16)
γ3=-1 × [γ1γ2]θ3 (17)
Due to having bulk redundancy information in obtained gesture recognition result, so using PNMS to eliminate posture superfluous for this method
It is remaining, obtain final carriage vector.Steps are as follows:
The highest posture of confidence level is chosen first, then removes posture similar with it according to the standard of elimination as a result, constantly
The process is repeated until only remaining 1 unique posture.In the method, the elimination standard defined using formula (18).
f(Pi,Pj| Λ, η)=I [d (Pi,Pj|Λ,λ)≤η] (18)
Wherein, d (Pi,Pj| Λ) it is posture PiWith posture PjDistance, Λ be function d () parameter set, η be whether disappear
The threshold value standard removed, I [d (Pi,Pj| Λ, λ)≤η] it indicates to work as posture PiWith posture PjDistance be less than threshold value when, f (Pi,Pj|
Λ, η) output be 1, posture PiIt should be eliminated.In elimination standard, posture PiWith posture PjDistance definition such as formula (19)
It is shown.
Wherein,WithRespectively posture PiWith posture PjIn n-th of key point confidence level,WithRespectively posture
PiWith posture PjIn n-th of key point coordinate value,Expression center is located at ki nPosture frame, σ1For soft match parameter, σ2
For space length parameter, λ is balance factor.
For human body attitude recognition result as shown in figure 5, wherein figure (a) is testing result under sprawl, figure (b) is other appearances
Testing result under gesture.
The judgement of 4 postures
In this method, by obtained human body frame and attitude vectors, completed using simple geometrical constraint to human body appearance
The judgement of state.The geometrical constraint used includes the ratio of width to height of human body frame and the deflection of trunk.
By multiple contrast test, finds when the ratio of width to height of human body frame is less than 0.5, can tentatively conclude the people detected
Body is in sprawl, meanwhile, when trunk angle is less than ± 30 °, it may further determine that human body is in sprawl.
Therefore, in this example, posture judgement is carried out using following constraint condition:
(1) the ratio of width to height of human body frame is less than 0.5；
(2) trunk angle is less than ± 30 °；
When the conditions are satisfied, then determine the posture for lying position.
For human body attitude judging result as shown in fig. 6, wherein figure (a) is testing result under sprawl, figure (b) is other appearances
Testing result under gesture.
Claims (4)
1. rescuing the human testing in environment and gesture recognition method, which is characterized in that specific workflow is as follows:
S1 realtime graphic defogging
The pass between original image, mist elimination image, environment light and global atmosphere light is obtained by the physical model of foggy image
Then system uses mean filter, estimate the value of environment light and global atmosphere light, finally recover clearly fog free images；
S2 completes human testing using defogging treated image
Algorithm uses multi-scale prediction, respectively in the enterprising pedestrian's body frame of characteristic pattern of 8 × 8,16 × 16 and 32 × 32 sizes
Prediction；On each scale, input picture is divided into N × N number of grid, in each grid, by fine tuning human body default frame into
Row prediction, for each default frame, the top left co-ordinate of neural network forecast human body frame, the width of human body frame and height and human body frame are set
Reliability；Finally, the human body frame of neural network forecast is obtained final prediction result by a non-maxima suppression；
S3 carries out gesture recognition using the human body image that detection obtains
Gesture recognition process the human body block diagram that human testing obtains individually is cut out to obtain an independent human body image,
Human body attitude identification is carried out using the method that RMPE gesture recognition frame is combined with Stacked Hourglass network；Specifically
Steps are as follows:
S3.1 cuts out human body picture
Come in order to ensure that can be cut out complete human body, it is wide to extend 30% on the basis of the human body frame that human testing obtains,
Height extends 20% and is cut out；
The human region of S3.2 use space converting network extraction high quality
S3.3 carries out single Attitude estimation using Stacked Hourglass network
Attitude estimation result is mapped back original image by S3.4 use space contravariant switching network
S3.5 eliminates posture redundancy using PNMS, obtains final carriage vector；
S4 judges human body attitude
In rescue scene, it can determine if to need to succour immediately by judging whether personnel are in sprawl；Therefore,
By obtained human body frame and attitude vectors, the judgement to human body attitude is completed using simple geometrical characteristic；What is used is several
What feature includes the ratio of width to height of human body frame and the deflection of trunk；Posture judgement is carried out using following constraint condition:
(1) the ratio of width to height of human body frame is less than 0.5；
(2) trunk angle is less than ± 30 °；
When the conditions are satisfied, then determine that the posture for lying position, and assigns posture label for everyone.
2. human testing and gesture recognition method in rescue environment according to claim 1, it is characterised in that: mentioning
It takes default to estimate method using k-means++ clustering algorithm, seeks the width and height of n default frame needed for YOLOv3 algorithm；?
In cluster process, using distance metric formula shown in formula (1)；
D (box, centroid)=1-IOU (box, centroid) (1)
Wherein, box is the human body frame in sample database, and centroid is cluster centre, and d (box, centroid) is each human body frame
At a distance from cluster centre；IOU (box, centroid) is friendship and the ratio of human body frame and cluster centre, i.e. two regions intersection portion
It is divided to the result divided by two region union parts.
3. human testing and gesture recognition method in rescue environment according to claim 1, it is characterised in that: human body
The training of network is detected, detailed process is as follows:
The preparation of sample database
Human sample is acquired as positive sample；Based on COCO public data collection, the picture for wherein including human sample is used
As training set positive sample, markup information uses the markup information of human body classification in original data set；
The training of neural network
All training samples are inputted network in batches to be trained；When training, using error of sum square function as training
The loss function of process；
Meanwhile in the training process, BN layers of acceleration convergence have been used；Also, it is every to pass through 10 training, randomly select new input
Size, size are 32 multiple, and minimum 320 × 320, it is up to 608 × 608.
4. human testing and gesture recognition method in rescue environment according to claim 1, it is characterised in that: human body
The training of gesture recognition network, detailed process are as follows:
The preparation of sample database
User's physical examination survey grid network carries out human testing to MPII human body attitude database first, obtains a series of human body frame；
According to detection obtained human body frame and real human body frame between offset and human body attitude relationship, using Gaussian Mixture be distributed into
Row modeling, obtains distribution P (δ B | atom (P)), and wherein δ B is the offset of human body frame, and atom (P) is the atom posture of human body；Most
Afterwards, according to the human body attitude of mark, sample is extended using distribution P (δ B | atom (P))；
The training of neural network
In training, using above-mentioned samples pictures to spatial alternation network, Stacked Hourglass network, spatial inverse transform
Network is finely adjusted simultaneously, and spatial alternation network and spatial inverse transform network is allowed to adapt to gesture recognition task automatically；
Meanwhile a parallel Stacked Hourglass network is increased in training to promote the accurate estimation of posture；It should
Network and original Stacked Hourglass network share spatial alternation network, but spatial inverse transform net is not added after network
Network；Each branch uses mean square error as loss function, and total losses is the sum of two branch penalties.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810710454.6A CN109002783A (en) | 2018-07-02 | 2018-07-02 | Rescue the human testing in environment and gesture recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810710454.6A CN109002783A (en) | 2018-07-02 | 2018-07-02 | Rescue the human testing in environment and gesture recognition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109002783A true CN109002783A (en) | 2018-12-14 |
Family
ID=64598055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810710454.6A Pending CN109002783A (en) | 2018-07-02 | 2018-07-02 | Rescue the human testing in environment and gesture recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109002783A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815886A (en) * | 2019-01-21 | 2019-05-28 | 南京邮电大学 | A kind of pedestrian and vehicle checking method and system based on improvement YOLOv3 |
| CN109815901A (en) * | 2019-01-24 | 2019-05-28 | 杭州电子科技大学 | A kind of more people's Attitude estimation methods based on YOLOv3 algorithm |
| CN110084166A (en) * | 2019-04-19 | 2019-08-02 | 山东大学 | Substation's smoke and fire intelligent based on deep learning identifies monitoring method |
| CN110502965A (en) * | 2019-06-26 | 2019-11-26 | 哈尔滨工业大学 | A kind of construction safety helmet wearing monitoring method based on the estimation of computer vision human body attitude |
| CN110605724A (en) * | 2019-07-01 | 2019-12-24 | 青岛联合创智科技有限公司 | Intelligence endowment robot that accompanies |
| CN110811630A (en) * | 2019-10-31 | 2020-02-21 | 瞬联软件科技（北京）有限公司 | Pregnant woman sleeping posture detection method and device |
| CN110852162A (en) * | 2019-09-29 | 2020-02-28 | 深圳云天励飞技术有限公司 | Human body integrity data labeling method and device and terminal equipment |
| CN111046840A (en) * | 2019-12-26 | 2020-04-21 | 天津理工大学 | Personnel safety monitoring method and system based on artificial intelligence in pollution remediation environment |
| CN111062356A (en) * | 2019-12-26 | 2020-04-24 | 沈阳理工大学 | Method for automatically identifying human body action abnormity from monitoring video |
| CN111091110A (en) * | 2019-12-24 | 2020-05-01 | 山东仁功智能科技有限公司 | Wearing identification method of reflective vest based on artificial intelligence |
| CN111368810A (en) * | 2020-05-26 | 2020-07-03 | 西南交通大学 | Sit-up detection system and method based on human body and skeleton key point identification |
| CN111428703A (en) * | 2020-06-15 | 2020-07-17 | 西南交通大学 | Method for detecting pit leaning behavior of electric power operation and inspection personnel |
| WO2020215492A1 (en) * | 2019-04-23 | 2020-10-29 | 江南大学 | Multi-bernoulli multi-target video detection and tracking method employing yolov3 |
| CN112287729A (en) * | 2019-07-24 | 2021-01-29 | 南京大学 | Drunk driving test system and method based on laser remote sensing and facial behavior and action analysis |
| CN112464895A (en) * | 2020-12-14 | 2021-03-09 | 深圳市优必选科技股份有限公司 | Posture recognition model training method and device, posture recognition method and terminal equipment |
| CN112634233A (en) * | 2020-12-24 | 2021-04-09 | 四川电器集团中低压智能配电有限公司 | Application method in power distribution equipment based on image detection and analysis technology |
| CN112837471A (en) * | 2019-11-22 | 2021-05-25 | 上海弘视通信技术有限公司 | Security monitoring method and device for internet contract room |
| US11094070B2 (en) | 2019-04-23 | 2021-08-17 | Jiangnan University | Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection |
| CN113799118A (en) * | 2021-04-01 | 2021-12-17 | 金陵科技学院 | Coal mine search and rescue robot based on machine vision and motion control |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110025834A1 (en) * | 2009-07-31 | 2011-02-03 | Samsung Electronics Co., Ltd. | Method and apparatus of identifying human body posture |
| CN106611157A (en) * | 2016-11-17 | 2017-05-03 | 中国石油大学(华东) | Multi-people posture recognition method based on optical flow positioning and sliding window detection |
| CN107451568A (en) * | 2017-08-03 | 2017-12-08 | 重庆邮电大学 | Use the attitude detecting method and equipment of depth convolutional neural networks |
| CN107495971A (en) * | 2017-07-27 | 2017-12-22 | 大连和创懒人科技有限公司 | Morbidity's alarm medical system and its detection method based on skeleton identification |
| CN107886089A (en) * | 2017-12-11 | 2018-04-06 | 深圳市唯特视科技有限公司 | A kind of method of the 3 D human body Attitude estimation returned based on skeleton drawing |
-
2018
- 2018-07-02 CN CN201810710454.6A patent/CN109002783A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110025834A1 (en) * | 2009-07-31 | 2011-02-03 | Samsung Electronics Co., Ltd. | Method and apparatus of identifying human body posture |
| CN106611157A (en) * | 2016-11-17 | 2017-05-03 | 中国石油大学(华东) | Multi-people posture recognition method based on optical flow positioning and sliding window detection |
| CN107495971A (en) * | 2017-07-27 | 2017-12-22 | 大连和创懒人科技有限公司 | Morbidity's alarm medical system and its detection method based on skeleton identification |
| CN107451568A (en) * | 2017-08-03 | 2017-12-08 | 重庆邮电大学 | Use the attitude detecting method and equipment of depth convolutional neural networks |
| CN107886089A (en) * | 2017-12-11 | 2018-04-06 | 深圳市唯特视科技有限公司 | A kind of method of the 3 D human body Attitude estimation returned based on skeleton drawing |
Non-Patent Citations (2)
| Title |
|---|
| HAO-SHU FANG 等: "RMPE: Regional Multi-Person Pose Estimation", 《HTTPS://ARXIV.ORG/PDF/1612.00137.PDF》 * |
| QIAN LIU 等: "Fast haze removal from a single image", 《2013 25TH CHINESE CONTROL AND DECISION CONFERENCE (CCDC)》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109815886A (en) * | 2019-01-21 | 2019-05-28 | 南京邮电大学 | A kind of pedestrian and vehicle checking method and system based on improvement YOLOv3 |
| CN109815901A (en) * | 2019-01-24 | 2019-05-28 | 杭州电子科技大学 | A kind of more people's Attitude estimation methods based on YOLOv3 algorithm |
| CN110084166B (en) * | 2019-04-19 | 2020-04-10 | 山东大学 | Intelligent substation smoke and fire identification and monitoring method based on deep learning |
| CN110084166A (en) * | 2019-04-19 | 2019-08-02 | 山东大学 | Substation's smoke and fire intelligent based on deep learning identifies monitoring method |
| US11094070B2 (en) | 2019-04-23 | 2021-08-17 | Jiangnan University | Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection |
| WO2020215492A1 (en) * | 2019-04-23 | 2020-10-29 | 江南大学 | Multi-bernoulli multi-target video detection and tracking method employing yolov3 |
| CN110502965B (en) * | 2019-06-26 | 2022-05-17 | 哈尔滨工业大学 | Construction safety helmet wearing monitoring method based on computer vision human body posture estimation |
| CN110502965A (en) * | 2019-06-26 | 2019-11-26 | 哈尔滨工业大学 | A kind of construction safety helmet wearing monitoring method based on the estimation of computer vision human body attitude |
| CN110605724A (en) * | 2019-07-01 | 2019-12-24 | 青岛联合创智科技有限公司 | Intelligence endowment robot that accompanies |
| CN112287729A (en) * | 2019-07-24 | 2021-01-29 | 南京大学 | Drunk driving test system and method based on laser remote sensing and facial behavior and action analysis |
| CN110852162A (en) * | 2019-09-29 | 2020-02-28 | 深圳云天励飞技术有限公司 | Human body integrity data labeling method and device and terminal equipment |
| CN110852162B (en) * | 2019-09-29 | 2020-10-23 | 深圳云天励飞技术有限公司 | Human body integrity data labeling method and device and terminal equipment |
| CN110811630A (en) * | 2019-10-31 | 2020-02-21 | 瞬联软件科技（北京）有限公司 | Pregnant woman sleeping posture detection method and device |
| CN110811630B (en) * | 2019-10-31 | 2022-07-22 | 瞬联软件科技（北京）有限公司 | Pregnant woman sleeping posture detection method and device |
| CN112837471A (en) * | 2019-11-22 | 2021-05-25 | 上海弘视通信技术有限公司 | Security monitoring method and device for internet contract room |
| CN111091110B (en) * | 2019-12-24 | 2023-11-17 | 山东仁功智能科技有限公司 | Reflection vest wearing recognition method based on artificial intelligence |
| CN111091110A (en) * | 2019-12-24 | 2020-05-01 | 山东仁功智能科技有限公司 | Wearing identification method of reflective vest based on artificial intelligence |
| CN111062356B (en) * | 2019-12-26 | 2024-03-26 | 沈阳理工大学 | Method for automatically identifying abnormal human body actions from monitoring video |
| CN111062356A (en) * | 2019-12-26 | 2020-04-24 | 沈阳理工大学 | Method for automatically identifying human body action abnormity from monitoring video |
| CN111046840B (en) * | 2019-12-26 | 2023-06-23 | 天津理工大学 | Personnel safety monitoring method and system based on artificial intelligence in pollution remediation environment |
| CN111046840A (en) * | 2019-12-26 | 2020-04-21 | 天津理工大学 | Personnel safety monitoring method and system based on artificial intelligence in pollution remediation environment |
| CN111368810B (en) * | 2020-05-26 | 2020-08-25 | 西南交通大学 | Sit-up detection system and method based on human body and skeleton key point identification |
| CN111368810A (en) * | 2020-05-26 | 2020-07-03 | 西南交通大学 | Sit-up detection system and method based on human body and skeleton key point identification |
| CN111428703B (en) * | 2020-06-15 | 2020-09-08 | 西南交通大学 | Method for detecting pit leaning behavior of electric power operation and inspection personnel |
| CN111428703A (en) * | 2020-06-15 | 2020-07-17 | 西南交通大学 | Method for detecting pit leaning behavior of electric power operation and inspection personnel |
| CN112464895A (en) * | 2020-12-14 | 2021-03-09 | 深圳市优必选科技股份有限公司 | Posture recognition model training method and device, posture recognition method and terminal equipment |
| CN112464895B (en) * | 2020-12-14 | 2023-09-01 | 深圳市优必选科技股份有限公司 | Gesture recognition model training method and device, gesture recognition method and terminal equipment |
| CN112634233A (en) * | 2020-12-24 | 2021-04-09 | 四川电器集团中低压智能配电有限公司 | Application method in power distribution equipment based on image detection and analysis technology |
| CN113799118A (en) * | 2021-04-01 | 2021-12-17 | 金陵科技学院 | Coal mine search and rescue robot based on machine vision and motion control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109002783A (en) | Rescue the human testing in environment and gesture recognition method | |
| CN110502965B (en) | Construction safety helmet wearing monitoring method based on computer vision human body posture estimation | |
| CN106845357B (en) | A kind of video human face detection and recognition methods based on multichannel network | |
| CN109819208A (en) | A kind of dense population security monitoring management method based on artificial intelligence dynamic monitoring | |
| CN110425005B (en) | Safety monitoring and early warning method for man-machine interaction behavior of belt transport personnel under mine | |
| CN108629946B (en) | Human body falling detection method based on RGBD sensor | |
| CN109522793A (en) | More people's unusual checkings and recognition methods based on machine vision | |
| CN110188701A (en) | Dress ornament recognition methods, system and terminal based on the prediction of human body key node | |
| CN107832672A (en) | A kind of pedestrian's recognition methods again that more loss functions are designed using attitude information | |
| CN111611874B (en) | Face mask wearing detection method based on ResNet and Canny | |
| CN109670441A (en) | A kind of realization safety cap wearing knows method for distinguishing, system, terminal and computer readable storage medium | |
| CN109446925A (en) | A kind of electric device maintenance algorithm based on convolutional neural networks | |
| CN108898047A (en) | The pedestrian detection method and system of perception are blocked based on piecemeal | |
| CN105760835A (en) | Gait segmentation and gait recognition integrated method based on deep learning | |
| CN104751146B (en) | A kind of indoor human body detection method based on 3D point cloud image | |
| CN112560741A (en) | Safety wearing detection method based on human body key points | |
| JP6773829B2 (en) | Object recognition device, object recognition method, and object recognition program | |
| CN103390151B (en) | Method for detecting human face and device | |
| CN111639602B (en) | Pedestrian shielding and orientation detection method | |
| CN110472473A (en) | The method fallen based on people on Attitude estimation detection staircase | |
| CN112183472A (en) | Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet | |
| CN109886242A (en) | A kind of method and system that pedestrian identifies again | |
| CN106846375A (en) | A kind of flame detecting method for being applied to autonomous firefighting robot | |
| CN112149494A (en) | Multi-person posture recognition method and system | |
| CN115761618A (en) | Key site security monitoring image identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181214 |
|
| RJ01 | Rejection of invention patent application after publication |