CN110533012A - A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system - Google Patents
A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system Download PDFInfo
- Publication number
- CN110533012A CN110533012A CN201911005693.2A CN201911005693A CN110533012A CN 110533012 A CN110533012 A CN 110533012A CN 201911005693 A CN201911005693 A CN 201911005693A CN 110533012 A CN110533012 A CN 110533012A
- Authority
- CN
- China
- Prior art keywords
- image
- light stream
- recognition
- eye
- small server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/04—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons
- G08B21/0407—Alarms for ensuring the safety of persons responsive to non-activity, e.g. of elderly persons based on behaviour analysis
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Psychiatry (AREA)
- Emergency Management (AREA)
- Social Psychology (AREA)
- Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Gerontology & Geriatric Medicine (AREA)
- Psychology (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Image Analysis (AREA)
Abstract
The sleep state monitoring energy conservation that the invention discloses a kind of based on deep learning image recognition is helped the elderly system, this system is made of headend equipment, small server, energy supply control module and rear end equipment, can acquire image and image is uploaded to small server by network；Image processing system in small server analyzes image using human eye state recognition detection technology and limbs recognition detection technology, after judging that old man has fallen asleep, sends control signal to energy supply control module；Blue-teeth communication equipment in energy supply control module receives the control signal of small server and generates corresponding infrared control signal, and the infrared sending device in module then controls television sound volume, brightness and power supply, reaches energy-efficient purpose；The also regular rear end sons and daughters mobile phone of small server sends report, receives the network inquiry request of headend equipment or rear end equipment, achievees the purpose that help the elderly.
Description
Technical field
The present invention relates to image processing and analyzing technical field, especially a kind of sleep shape based on deep learning image recognition
State monitors system.
Background technique
Currently, aging of population has become the extremely severe social concern in one, China, the origin cause of formation has three: family planning
The implementation of policy gradually decreases Chinese neonates, and the fast development of social economy makes the elderly have better endowment condition,
The progress of modern medicine level makes the average life span of the elderly be greatly improved.And simultaneously, sons and daughters can not because being busy with one's work
It often goes home, the solitary phenomenon of Empty nest elderly is universal, if or else old man will use smart phone if, feeling of lonely will be more tight
Weight, emotional affection are increasingly thin.And systematic difference of helping the elderly, making sons and daughters more can work contentedly, and reduce accident.
Old man often looks at TV and just falls asleep, and household electrical appliances standard-sized sheet is until the next morning old man wakes up；Young man chases after play and endures
Night endures and incessantly just sleeps, and household electrical appliances longtime running not only causes the waste of electric energy, it is also possible to lead to the danger such as fire.Many people are
Know how just economize on electricity, but the be deficient in resources awareness of unexpected development and economy consciousness, as television set is standby for a long time, many times namely
One " lazyness " word dislikes pass of coming to go trouble.If only measuring the meaning of resource, construction with the material value of several maos of every degree electricity
Conservation-minded society can only be just empty verbiage, this is not merely the stiver of every family, also examine from national situation, global situation
Consider.
With the continuous social and economic development, the sales volume rapid development of China's consumer electronics product, everybody affords；
Under the main trend in global network epoch, with constantly improve for basic industries facility, the continuous of middle and high end material technology perfects,
Smart phone businessman is in order to attract consumer, it is necessary to make more preferable more excellent mobile phone, constantly promote the competitiveness of oneself, this is just
Mobile phone is caused to update very fast.A large amount of mobile phone state well but abandoneds, only small part are recovered, most of direct
It abandons, is not effectively addressed.In these waste mobile phones and its battery and charger contain a large amount of harmful substance, such as lead,
Zinc, polyvinyl chloride and bromide etc., it has also become a kind of important pollutant causes serious environmental pollution and the wasting of resources.
Summary of the invention
It is an object of the invention to overcome the deficiencies of the prior art and provide a kind of sleeps based on deep learning image recognition
The problem of status monitoring energy conservation is helped the elderly system, and solution must not accompany old man to lead to Electrical Safety for a long time；User's eye shape of the present invention
State recognition detection technology and limbs recognition detection technology analyze image, can quickly judge whether user is in sleep shape
State, and TV can be opened and closed and remotely be controlled, have the advantages that household electrical appliances energy conservation and children is helped to understand old man's animation.
The purpose of the present invention is achieved through the following technical solutions:
A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system, including headend equipment, small-sized
Server, energy supply control module and rear end equipment；Headend equipment is connected by network with small server, and small server passes through
Network is connected with energy supply control module and rear end equipment respectively；Rear end equipment also passes through network and is connected with headend equipment；
The headend equipment is used to acquire the image information in front of TV, and sends the image information of acquisition to small-sized clothes
Business device；The headend equipment is also used to realize long-distance video call with rear end equipment；The headend equipment is also used to inquire small-sized
The data information stored in server；
For receiving and storing image information, and based on the received, image information carries out image recognition to the small server
And image analysis, judge whether the old man in image is in sleep state；The small server is also used to based on the analysis results
Control information is sent to energy supply control module；The small server is also used to periodically end equipment backward and sends report, before receiving
The request of the network inquiry of end equipment or rear end equipment；
The energy supply control module is used to receive the control signal of small server, and is generated accordingly according to control signal
Infrared control signal controls television sound volume, brightness and power supply open/close states by infrared control signal；The energy supply control module
It is also used to detect the open/close states of TV and sends the status information of TV to small server；
The rear end equipment is used to inquire old man's sleep state information of small server feedback, and the rear end equipment is also used
In with headend equipment real-time video call, the rear end equipment is also used to inquire the report or historical information of small server.
Further, the small server includes STM32 interconnection type microcontroller；It is integrated in the small server
Image processing system, image processing system carry out image using human eye state recognition detection method and limbs recognition detection method
Analysis improves accuracy of identification by the innovatory algorithm of optical flow method, the Kalman filtering that angle point is tracked and affine transformation, works as human eye
Identification or the detection of light stream contour feature point and the detection of limbs recognizer be when judging old man for sleep state, small server to
Energy supply control module issues signal.
Further, the headend equipment and the rear end equipment are mobile phone.
Further, energy supply control module include system linear regulated power supply, main control chip, infrared remote sensing control device,
Bluetooth module and serial communication module；System linear regulated power supply is respectively main control chip, infrared remote sensing control device and bluetooth
Module provide electric energy, the signal end of main control chip respectively with infrared remote sensing control device, bluetooth module and serial communication module
Signal is connected；The system linear regulated power supply exports regulated power supply by USB Power Adapter；Infrared remote sensing control device packet
Transmitting unit and receiving unit are included, transmitting unit includes keyboard matrix circuit, code modulated circuits and LED infrared transmitter；It connects
Receiving unit includes photoelectric conversion amplifier, demodulator circuit and decoding circuit.
A kind of sleep state monitoring method of deep learning image recognition, comprising the following steps:
S1, the image information for acquiring human body, upload to small server for acquired image information, by small server
Image information is identified and analyzed；
S2, small server are analyzed and are judged to image information by image processing system, described image processing system
System analyze and determine including by light stream contour feature point and limbs pixel recognition methods to human motion to image information
It is detected, face information is judged by eye recognition detection method, obtain eyes of user opens closed state；
S3, judged to obtain whether user is in sleep state according to human motion judgement and face information, and export human body
The detection data of movement and eye state.
Further, the eye recognition detection method, the eye recognition detection method are closed by detecting opening for eyes
State and calculating frequency of wink carry out sleep state judgement, specifically include:
S201, the feature that eyes of user region is extracted by image information, will input after human face region image uniform sizes
To the convolutional neural networks model for human eye critical point detection, the transverse and longitudinal coordinate value of the central point of left eye and right eye, root are obtained
The rectangular area where eyes is determined according to the wide high level of eye center point coordinate value and 12*6, respectively obtains the area of left eye and right eye
Area image；
S202, human eye critical point detection is carried out using convolutional neural networks model, by facial image statistics at unified size
Gray level image, by Hough transform detect iris edge, Hought convert the fitting to marginal points all in image in turn,
Then optimal edge is looked in parameter space, image midpoint is divided into pacifically three kinds of angle point, edge by Harris, and angle point refers to each
The point of grey scale change severe degree is measured on a direction, angle point amount is bigger, and grey scale change is also more violent；Thus judge blinking for user
Eye frequency；
S203, to iris image carry out Hough circle draw operation, observation be 1.5 inverse ratio can draw Hough circle or 2 it is anti-
Than when can change Hough circle, detection is equal to 0, equal to 1 and greater than 1 when drawn several Houghs circle, so that judgement is to open eyes
State, closed-eye state narrow eye shape state.
Further, further include limbs pixel recognition detection method, the method specifically includes:
Limbs pixel recognition methods be used for be by human synovial and limbs unified Modeling human body the basic element of character, then press appearance
The configuration space of component is divided into several classes, one posture word of every a kind of composition, by associated posture word by state feature
Composition posture sentence is for describing whole body posture；
Compare the posture sentence in front and back two minutes, if there is the equal situation of continuous 15 posture sentences, determines people
It has been fallen asleep that, if not occurring, people does not fall asleep.
Further, limbs pixel recognition detection method further include:
Rough object is irradiated to using laser speckle or penetrates the random diffraction spot formed after frosted glass, to speckle pattern
It is recorded and is demarcated, find out key point, be averaged using the position of key point is dry to the limb of all the same categories, calculated and close
The integral of the dot product of each pixel vector and line vector on key point line；
The threshold range for leveling off to standard is set between multiple key points and pixel, utilizes cloud computing and big number
According to superpower data collection calculate the advantages of, judge the region of collection point, judge that human eye opens closed state.
Further, further include optical flow method human body contour outline feature point detecting method, specifically include:
S901, setting optical flow equation are as follows:
I (x, y, t)=I (x+dx, y+dy, t+dt)；
It enablesThen:
Ixu+Iyv+It=0；
Wherein, I is light stream intensity, and u is movement velocity of the target in the direction x, and v is movement speed of the target in the direction y, Ix
Variation for light stream intensity in the direction x, Iy are to indicate light stream intensity in the variation in the direction y, and It is to indicate light stream intensity in the time
On variation；
Then S902, the light stream for calculating gaussian pyramid top image estimate secondary according to the light stream of top image
The initial value of top layer light stream, then accurate light stream is calculated in secondary top layer images, this process is then repeated, until calculating most bottom
The accurate light stream of tomographic image.Then additional pixel boundary is added in this tomographic image of two frame of front and back, adds additional pixel side
When boundary is to prevent the edge when the light stream of tracking point in image, boundary may be exceeded when calculating the light stream of its neighborhood,
After its true boundary is added to image peripheral in this way, the light stream of the boundary point made is also that can calculate；
Wherein, μ is mean value, and δ is variance.
Define light stream are as follows:
This is, we the matching error of pixel in neighborhood and can be denoted as:
Wherein, A and B is the gray value at coordinate；
S903, it is exactly to utilize least square method to the optical flow computation of every tomographic image, seeks the derivative of matching error sum in neighborhood,
Its optimal solution is derivative when being zero, reaches matching error and minimum, at this time the similarity highest between two frame corresponding points, then use
Taylor's formula expansion can be obtained:
Then the optimal solution of light stream is
S904, human body contour outline area characteristic point light stream campaign so that judge how test object moves, such as:
The motion state of test object can be described according to above-mentioned formula.
The beneficial effects of the present invention are:
The present invention can set up a system by second-hand mobile phone, small server, energy supply control module, low in cost,
System installation is simple, low to technical requirements；Human eye state recognition detection technology and limbs recognition detection technology can be used to image
It is analyzed, can quickly judge whether user is in sleep state, and TV can opened and closed and remotely be controlled, there is household electrical appliances section
The advantages of and children being helped to understand old man's animation.
Detailed description of the invention
Fig. 1 is system structure diagram of the invention；
Fig. 2 is sleep state monitoring method flow chart of the invention；
Fig. 3 opens for human eye of the present invention closes image detection effect picture；
Fig. 4 is limbs pixel integral image detection effect figure of the present invention；
Fig. 5 is the circuit diagram of small server of the present invention；
Fig. 6 is the circuit structure diagram of present system linear stabilized power supply；
Fig. 7 is the circuit diagram of infrared remote sensing control device of the present invention；
Fig. 8 is the circuit diagram of Bluetooth communication modules of the present invention；
In figure, 10- headend equipment, 11- small server, 12- rear end equipment, 13- energy supply control module, 14- TV.
Specific embodiment
Illustrate embodiments of the present invention below by way of specific specific example, those skilled in the art can be by this specification
Other advantages and efficacy of the present invention can be easily understood for disclosed content.The present invention can also pass through in addition different specific realities
The mode of applying is embodied or practiced, the various details in this specification can also based on different viewpoints and application, without departing from
Various modifications or alterations are carried out under spirit of the invention.It should be noted that in the absence of conflict, following embodiment and implementation
Feature in example can be combined with each other.
It should be noted that illustrating the basic structure that only the invention is illustrated in a schematic way provided in following embodiment
Think, only shown in schema then with related component in the present invention rather than component count, shape and size when according to actual implementation
Draw, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel
It is likely more complexity.
Embodiment 1:
The present embodiment discloses a kind of sleep state monitoring system based on deep learning image recognition, please refers to 1 institute of attached drawing
Show, including headend equipment 10, small server 11, energy supply control module 13 and rear end equipment 12；Headend equipment 10 passes through network
Be connected with small server 11, small server 11 by network respectively with 12 phase of energy supply control module 13 and rear end equipment
Even；Rear end equipment 12 is also connected by network with headend equipment 10；
The headend equipment 10 is used to acquire the image information in the front of TV 14, and the image information of acquisition is sent to small
Type server 11；The headend equipment 10 is also used to realize long-distance video call with rear end equipment 12；The headend equipment 10 is also
For inquiring the data information stored in small server 11；
For receiving and storing image information, and based on the received, image information carries out image knowledge to the small server 11
Other and image analysis, judges whether the old man in image is in sleep state；The small server 11 is also used to according to analysis
As a result control information is sent to energy supply control module 13；The small server 11 is also used to periodically end equipment 12 backward and sends report
Table receives the network inquiry request of headend equipment 10 or rear end equipment 12；
The energy supply control module 13 is used to receive the control signal of small server 11, and generates phase according to control signal
The infrared control signal answered controls television sound volume, brightness and power supply open/close states by infrared control signal；The power supply control
Module 13 is also used to detect the open/close states of TV and sends the status information of TV to small server 11；
The rear end equipment 10 is used to inquire old man's sleep state information of the feedback of small server 11, the rear end equipment
12 be also used to 10 real-time video call of headend equipment, the rear end equipment 12 be also used to inquire small server 11 report or
Historical information.
Further, the small server 11 includes STM32 interconnection type microcontroller；Collection in the small server 11
At there is image processing system, image processing system is using human eye state recognition detection method and limbs recognition detection method to image
It is analyzed, accuracy of identification is improved by the innovatory algorithm of optical flow method, the Kalman filtering that angle point is tracked and affine transformation, when
When eye recognition or the detection of light stream contour feature point and the detection of limbs recognizer judge old man for sleep state, small service
Device 11 issues signal to energy supply control module 13.
Further, the headend equipment 10 and the rear end equipment are mobile phone, and second-hand mobile phone can be used and carry out group
Net had not only realized waste product utilization in this way, but also can save the cost.
Further, energy supply control module 13 includes system linear regulated power supply, main control chip, infrared remote sensing control dress
It sets, bluetooth module and serial communication module；System linear regulated power supply is respectively main control chip, infrared remote sensing control device and indigo plant
Tooth module provide electric energy, the signal end of main control chip respectively with infrared remote sensing control device, bluetooth module and serial communication mould
Block signal is connected；The system linear regulated power supply exports regulated power supply by USB Power Adapter；Infrared remote sensing control device
Including transmitting unit and receiving unit, transmitting unit includes keyboard matrix circuit, code modulated circuits and LED infrared transmitter；
Receiving unit includes photoelectric conversion amplifier, demodulator circuit and decoding circuit.
The small server 11 of the present embodiment uses STM32F103RCT6 chip, please refers to shown in attached drawing 5, is able to achieve and connects
Receive and storage image information, at the same it is interior be integrated with image processing system, image processing system uses human eye state recognition detection skill
Art and limbs recognition detection technology analyze image, using based on optical flow method and angle point tracking Kalman filtering with it is affine
The innovatory algorithm of transformation improves accuracy of identification, when eye recognition or the detection of light stream contour feature point and the inspection of limbs recognizers
When survey old man is sleep state, small server issues signal to energy supply control module, while also having periodically end equipment backward
It sends the function of report and receives the function of front and back ends device network inquiry request at any time.
The energy supply control module 13 of the present embodiment receives the control signal of small server 11, generates corresponding infrared control
Signal, infrared sending device control television sound volume, brightness and power supply, turns sound down, dims picture, if testing result during delay
Being continuously sleep state, then server issues cut-off signals, the power supply of the household electrical appliances such as television set is cut off, meanwhile, infrared facility detection
TV switch state, and pass through the status information that blue-teeth communication equipment sends television set to small server.
System linear regulated power supply please refers to shown in attached drawing 6, and power supply inputs 5V, defeated by USB Power Adapter
System power supply 3.3V regulated power supply out removes ripple burr using filter circuit.
It please refers to shown in attached drawing 7, infrared remote control system is made of transmitting unit and receiving unit, dedicated using coding/decoding
IC chip carries out control operation, and transmitting unit includes keyboard matrix circuit, code modulated circuits and the infrared transmission of LED
Device；Receiving unit includes photoelectric conversion amplifier, demodulator circuit and decoding circuit, process are as follows: bluetooth sends instruction, coding electricity
Signal, starting infrared diode and the infrared electric signal of transmitting, television set receive head and receive infrared electric signal, decoding electric signal, CPU
It is instructed, finally realizes function.
Bluetooth HC05 is the bluetooth serial ports module of principal and subordinate's one, please refers to shown in attached drawing 8, briefly, works as bluetooth equipment
After matching successful connection with bluetooth equipment, we can ignore the communication protocol inside bluetooth, directly bluetooth will be regarded serial ports
With.When establishing connection, two equipment are used in conjunction with a channel i.e. the same serial ports, and an equipment is sent data in channel,
Another equipment can data in receiving channel.
Embodiment 2:
The present embodiment on the basis of embodiment 1, provides a kind of sleep state monitoring method of deep learning image recognition,
It please refers to shown in attached drawing 2, specifically includes:
S1, the image information for acquiring human body, upload to small server for acquired image information, by small server
Image information is identified and analyzed；
S2, small server are analyzed and are judged to image information by image processing system, described image processing system
System analyze and determine including by light stream contour feature point and limbs pixel recognition methods to human motion to image information
It is detected, face information is judged by eye recognition detection method, obtain eyes of user opens closed state；
S3, judged to obtain whether user is in sleep state according to human motion judgement and face information, and export human body
The detection data of movement and eye state.
It further include eye recognition detection method in the monitoring method of the present embodiment, the eye recognition detection method is logical
Cross detection eyes open closed state and calculate frequency of wink carry out sleep state judgement, eyes open closed state also with the table of people
Feelings are closely related, and can assist the related works such as Expression Recognition to the state-detection of human eye.Eye recognition detection method can be to eye
The closed state of opening of eyeball is judged, and accuracy rate is higher, to illumination variation, the postures such as scene changes and face's rotation, inclination
Variation have stronger capture, operating rate is fast, can satisfy the requirement of real-time of practical application, and hardware requirement is low, with gram
Take the deficiencies in the prior art.The feature of detection algorithm based on pattern classification mainly extraction eye areas, such as LBP feature,
The features such as Gabor wavelet, and judge that eye areas image is eye opening image by classifiers such as SVM, Adaboost, narrow eye figure
Picture, or close one's eyes.
To iris image carry out Hough circle draw operation, observation be 1.5 inverse ratio can draw Hough circle or 2 inverse ratio when
Time can draw Hough circle, and detection, which is equal to 1 equal to 0 or is greater than 1, judges that several Hough circles of picture close so that judgement is about opening eyes
Eye, still narrows eye；
By the image that Gabor is filtered, it is more advantageous to than original gray level image and carries out image recognition for acquisition
The size of image and the texture feature of eye image, are tested repeatedly and select parameter, and Gabor filter and original image are then used
Carry out convolutional calculation；
By mobile phone camera, image information is collected, Face datection is carried out to input picture, after detecting face, record
Lower current face region, obtains human face region image, if current input image does not detect face, terminates to current defeated
The processing for entering image continues to carry out Face datection to next frame input picture；It will be input to after human face region image uniform sizes
For the convolutional neural networks model of human eye critical point detection, the transverse and longitudinal coordinate value of the central point of left eye and right eye is obtained, according to
The wide high level of eye center point coordinate value and 12*6 determine the rectangular area where eyes, respectively obtain the region of left eye and right eye
Image will be input to the convolutional neural networks mould for being used for eyes and opening closed state classification respectively after left eye and eye image uniform sizes
Type, obtain eyes opens closed state result.Described in using convolutional neural networks model carry out human eye critical point detection, by people
Face image counts the gray level image at unified size.Iris edge is detected by Hough transform, Hought is converted in turn to image
In all marginal points fitting, then look for optimal edge in parameter space.Image midpoint is divided into angle point, edge peace by Harris
Three kinds of ground, angle point refer to the point for measuring grey scale change severe degree in all directions, and angle point amount is bigger, grey scale change also Shaoxing opera
It is strong.Using python acquisition angle point data early period, image procossing is calculated using C++ algorithm, while guaranteeing algorithm arithmetic speed,
Improve robustness.Experiment global procedures are write using Opencv.
The present embodiment further includes limbs pixel recognition detection method, is specifically included:
It is the basic element of character of human body by human synovial and limbs unified Modeling.Posture feature is pressed again by the configuration space of component
Several classes are divided into, associated posture word composition posture sentence is used to describe entirely by one posture word of every a kind of composition
Figure state.An image is obtained per minute.It is the basic element of character of human body by human synovial and limbs unified Modeling.It is special by posture again
The sign such as relative position of the basic element of character, size, angle etc. are arranged, and form posture sentence to describe whole body posture.Place
Posture sentence before and after managing device relatively in two minutes.If there is the equal situation of continuous 15 posture sentences, determine that people has slept
.If not occurring, people does not fall asleep.It uses a kind of pumped FIR laser technology.Different from traditional TOF or structural light measurement
Technology, Light coding, which is used, continuously to be shone and non-pulse, without special sensitive chip, and is only needed common
CMOS sensitive chip allows the cost of scheme to substantially reduce.
Light coding or structured light technique；But unlike traditional method of structured light, his light source is got
What is gone is not the two-dimensional image coding of secondary period property variation, but one there is the body of three-dimensional depth to encode.This light
Source is called laser speckle, is the random diffraction spot when laser irradiation to rough object or penetrate frosted glass after are formed.
These speckles have the randomness of height, and can be with the different changing patterns of distance.Any two i.e. in space
The speckle pattern at place is all different.As long as stamping such structure light in space, entire space is all marked,
One object puts this space into, as long as looking at the speckle pattern above object, so that it may know this object where
.Certainly, the speckle pattern in entire space is all recorded before this, so the mark of primary source need to be carried out in advance
It is fixed.
Key point is found out, is averaged using the position of key point is dry to the limb of all the same categories.Key point is calculated to connect
The integral of the dot product of each pixel vector and line vector on line
P (u)=(1-u) dj1+udj2
The threshold range for leveling off to standard is set between multiple key points and pixel, utilizes cloud computing and big number
According to superpower data collection calculate the advantages of, judge the region of collection point, judge that human eye opens closed state.
The present embodiment further includes the detection of optical flow method human body contour outline characteristic point, is specifically included:
There are two basic assumptions for optical flow method: first is that the brightness constancy of target object is constant, second is that target object movement must
It must be continuous or object movement be small movement.The two conditions are provided to guarantee the establishment of optical flow equation, light stream side
Journey is as follows:
Ixu+Iyv+It=0；
Wherein, I is light stream intensity, and u is movement velocity of the target in the direction x, and v is movement speed of the target in the direction y, Ix
Variation for light stream intensity in the direction x, Iy are to indicate light stream intensity in the variation in the direction y, and It is to indicate light stream intensity in the time
On variation.
Wherein (u, v) is exactly desired optical flow field, but there are two unknown numbers in this equation, so can not simultaneously solve,
It is that light stream is consistent in tracked neighborhood of a point by one extra condition of addition in L-K (Lucas-Kanade) light stream.
In L-K light stream, it is desirable that necessary small movement, but the quick movement of object is very normally, to adopt at this time in practice
What is taken is gaussian pyramid method to solve the problems, such as this, specifically, is carried out first to the size of the image of two frame of front and back
Adjustment guarantees that every layer of obtained size is all integer due to the half that each layer is all preceding layer size, so needing
Exactly image is handled before processing and has been adjusted to suitable size.Then pyramid decomposition is carried out to original image, it is more past
The resolution ratio of pyramid upper layer images is lower, and resolution ratio is higher more down, and original image is in the bottom.Using Gaussian function such as
Shown in lower:
Wherein, μ is mean value, and δ is variance.
It needs first to carry out smothing filtering before being sampled by Gaussian function.Firstly, calculating gaussian pyramid top
Then time initial value of top layer light stream is estimated in the light stream of image according to the light stream of top image, then in secondary top layer images
Accurate light stream is calculated, this process is then repeated, until calculating the accurate light stream of bottom image.Then in two frame of front and back
This tomographic image adds additional pixel boundary, and adding additional pixel boundary is to prevent when the light stream point of tracking is on the side of image
When edge, boundary may be exceeded when calculating the light stream of its neighborhood, after its true boundary is added to image peripheral in this way,
The light stream of the boundary point made is also that can calculate.
Define light stream are as follows:
This is, we the matching error of pixel in neighborhood and can be denoted as:
Wherein, A and B is the gray value at coordinate.
Optical flow computation to every tomographic image is exactly to utilize least square method, seeks the derivative of matching error sum in neighborhood, most
Excellent solution is derivative when being zero, reaches matching error and minimum, at this time the similarity highest between two frame corresponding points.Taylor is used again
Formula can be obtained in formula expansion, as follows:
Then the optimal solution of light stream isIn calculating, each calculated residual error light stream can recursion next iteration
Light stream estimated value.Iterative process will carry out always, until calculate residual error light stream be less than it is given or reached it is maximum repeatedly
Generation number, it is assumed that the final residual error light stream vectors for reaching convergence then this tomographic image for M times are that this layer of all iteration obtain residual error light stream
The sum of vector.It, will necessarily be with losing in the track if characteristic point has exceeded image.Besides if calculated minimal error
Even and if being greater than given threshold value and having traced into and can not say that matched point is all correctly to match between matched two frame.So
Some error values can be rejected with stochastical sampling consistency (RANSAC) algorithm.
In human motion detection, detects the light stream campaign of the characteristic point in human body contour outline area and then how judge test object
Movement.
It can be indicated with following formula:
The motion state of test object can be described according to above-mentioned formula.
It is opened by human eye and closes image detection algorithm identification, please referred to shown in attached drawing 3, advanced row Face datection, then carry out people
Eye recognition detection, finally carries out pupil recognition detection.By observing how much presented white point identifies sleep state on image.It is logical
Limbs pixel integral image and the detection identification of human body optical flow method contour feature point are crossed, please refers to shown in attached drawing 4, passes through pixel
On line algorithm principle is integrated, cloud computing is carried out, image processor detection judgement is mobile by the characteristic point in limbs, shows
The movement of people judges whether to fall asleep.
Embodiment 3:
The present embodiment further includes the method for logging in management, specifically includes on the basis of embodiment 1 and embodiment 2:
User needs to carry out authentication, can just be grasped after login system before operating into main interface
Make.The login interface plan is Gridlayout layout, the design of simple UI is pursued, after operation by automatic spring login system
Interface, correct account and corresponding password are inputted, if can click registration button below screen without account and be infused
Volume.The support that the login function is realized is the SQlite portability database that backstage is possessed.
After inputting correct account and password, user can enter main interface, be different as unit of day in main interface
Option, care situations are classified, user can select daily monitoring to report according to oneself wish, kind of interface letter
Clean main relative layout layout, sets picture buttons for select button, its back is determined in specified xml document
Scape can pass through the customized setting of user preferences.User chooses respective selection to enter next interface, this interface is by each assessment
Mesh item composition, showed one week situation of change come sleep quality by column diagram before this, determined the volume of data such as sleep average value,
And using reference locality average value as comparison other, lower section demonstrate time for falling asleep, sleeping time, wake-up time, snore situation, and
Image analysis is carried out, these types analysis object, analytical standard plans to form waterfall manifold formula with recyclerview, in the hope of shape
It is beautiful and clean in formula.
The present invention can set up a system by second-hand mobile phone, small server, energy supply control module, low in cost,
System installation is simple, low to technical requirements；Human eye state recognition detection technology and limbs recognition detection technology can be used to image
It is analyzed, can quickly judge whether user is in sleep state, and TV can opened and closed and remotely be controlled, there is household electrical appliances section
The advantages of and children being helped to understand old man's animation.
A specific embodiment of the invention above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously
Limitations on the scope of the patent of the present invention therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to guarantor of the invention
Protect range.
Claims (9)
- The system 1. a kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly, which is characterized in that including front end Equipment, small server, energy supply control module and rear end equipment；Headend equipment is connected by network with small server, small-sized Server is connected with energy supply control module and rear end equipment respectively by network；Rear end equipment also passes through network and headend equipment It is connected；The headend equipment is used to acquire the image information in front of TV, and sends the image information of acquisition to small service Device；The headend equipment is also used to realize long-distance video call with rear end equipment；The headend equipment is also used to inquire small-sized clothes The data information stored in business device；For receiving and storing image information, and based on the received, image information carries out image recognition and figure to the small server As analysis, judge whether the old man in image is in sleep state；The small server is also used to based on the analysis results to electricity Source control module sends control information；The small server is also used to periodically end equipment backward and sends report, receives front end and sets Standby or rear end equipment network inquiry request；The energy supply control module is used to receive the control signal of small server, and corresponding infrared according to control signal generation Signal is controlled, television sound volume, brightness and power supply open/close states are controlled by infrared control signal；The energy supply control module is also used In detection TV open/close states and to small server send TV status information；The rear end equipment be used for inquire small server feedback old man's sleep state information, the rear end equipment be also used to Headend equipment real-time video call, the rear end equipment are also used to inquire the report or historical information of small server.
- 2. a kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system according to claim 1, It is characterized in that, the small server includes STM32 interconnection type microcontroller；Image procossing is integrated in the small server System, image processing system analyze image using human eye state recognition detection method and limbs recognition detection method, lead to The innovatory algorithm for crossing optical flow method, the Kalman filtering of angle point tracking and affine transformation improves accuracy of identification, when eye recognition or The detection of light stream contour feature point and the detection of limbs recognizer be when judging old man for sleep state, and small server is to power supply control Molding block issues signal.
- 3. a kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system according to claim 1, It is characterized in that, the headend equipment and the rear end equipment are mobile phone.
- 4. a kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system according to claim 1, It is characterized in that, energy supply control module includes system linear regulated power supply, main control chip, infrared remote sensing control device, bluetooth module And serial communication module；System linear regulated power supply is respectively that main control chip, infrared remote sensing control device and bluetooth module provide Electric energy, the signal end of main control chip are connected with infrared remote sensing control device, bluetooth module and serial communication module signal respectively； The system linear regulated power supply exports regulated power supply by USB Power Adapter；Infrared remote sensing control device includes that transmitting is single Member and receiving unit, transmitting unit include keyboard matrix circuit, code modulated circuits and LED infrared transmitter；Receiving unit packet Include photoelectric conversion amplifier, demodulator circuit and decoding circuit.
- 5. a kind of sleep state monitoring method of the deep learning image recognition based on claim 1, which is characterized in that including with Lower step:S1, the image information for acquiring human body, upload to small server for acquired image information, by small server to figure As information is identified and analyzed；S2, small server are analyzed and are judged to image information by image processing system, described image processing system pair Image information analyze and determine including carrying out by light stream contour feature point and the recognition methods of limbs pixel to human motion Detection, judges face information by eye recognition detection method, obtain eyes of user opens closed state；S3, judged to obtain whether user is in sleep state according to human motion judgement and face information, and export human motion And the detection data of eye state.
- 6. the sleep state monitoring method of deep learning image recognition according to claim 5, which is characterized in that the human eye Recognition detection method, the eye recognition detection method are slept by opening closed state and calculating frequency of wink for detection eyes The judgement of dormancy state, specifically includes:S201, the feature that eyes of user region is extracted by image information, will be input to use after human face region image uniform sizes In the convolutional neural networks model of human eye critical point detection, the transverse and longitudinal coordinate value of the central point of left eye and right eye is obtained, according to eye Eyeball center point coordinate value and the wide high level of 12*6 determine the rectangular area where eyes, respectively obtain the administrative division map of left eye and right eye Picture；S202, human eye critical point detection is carried out using convolutional neural networks model, by facial image statistics at the ash of unified size Image is spent, iris edge is detected by Hough transform, Hought converts the fitting to marginal points all in image in turn, then Optimal edge is looked in parameter space, image midpoint is divided into pacifically three kinds of angle point, edge by Harris, and angle point refers in each side The point of grey scale change severe degree is measured upwards, and angle point amount is bigger, and grey scale change is also more violent；Thus judge the blink frequency of user Rate；S203, Hough circle picture operation is carried out to iris image, observation is that 1.5 inverse ratio can draw the inverse ratio of Hough circle or 2 When can change Hough circle, detection is equal to 0, equal to 1 and greater than 1 when drawn several Houghs circle, thus judgement be eyes-open state, Closed-eye state narrows eye shape state.
- 7. the sleep state monitoring method of deep learning image recognition according to claim 5, which is characterized in that it further includes Limbs pixel recognition detection method, the method specifically include:Limbs pixel recognition methods be used for be by human synovial and limbs unified Modeling human body the basic element of character, then it is special by posture The configuration space of component is divided into several classes by sign, and one posture word of every a kind of composition forms associated posture word Posture sentence is for describing whole body posture；Compare the posture sentence in front and back two minutes, if there is the equal situation of continuous 15 posture sentences, determines that people has slept , if not occurring, people does not fall asleep.
- 8. the sleep state monitoring method of deep learning image recognition according to claim 7, which is characterized in that limbs pixel Point recognition detection method further include:Rough object is irradiated to using laser speckle or penetrates the random diffraction spot formed after frosted glass, and speckle pattern is carried out It records and demarcates, find out key point, be averaged using the position of key point is dry to the limb of all the same categories, calculate key point The integral of the dot product of each pixel vector and line vector on line；The threshold range for leveling off to standard is set between multiple key points and pixel, utilizes cloud computing and big data The advantages of superpower data collection calculates, judges the region of collection point, judges that human eye opens closed state.
- 9. the sleep state monitoring method of deep learning image recognition according to claim 5, which is characterized in that further include light Judicial entity's contour feature point detecting method is flowed, is specifically included:S901, setting optical flow equation are as follows:I (x, y, t)=I (x+dx, y+dy, t+dt)；It enablesThen:Ixu+Iyv+It=0；Wherein, I is light stream intensity, and u is movement velocity of the target in the direction x, and v is target in the movement speed in the direction y, and Ix is light Intensity of flow is in the variation in the direction x, and Iy is to indicate light stream intensity in the variation in the direction y, and It is to indicate light stream intensity in time Variation；S902, the light stream for calculating gaussian pyramid top image, then estimate time top layer according to the light stream of top image The initial value of light stream, then accurate light stream is calculated in secondary top layer images, this process is then repeated, until calculating bottom figure The accurate light stream of picture.Then additional pixel boundary is added in this tomographic image of two frame of front and back, adding additional pixel boundary is When to prevent the edge when the light stream of tracking point in image, boundary may be exceeded when calculating the light stream of its neighborhood, in this way After its true boundary is added to image peripheral, the light stream of the boundary point made is also that can calculate；Wherein, μ is mean value, and δ is variance.Define light stream are as follows:This is, we the matching error of pixel in neighborhood and can be denoted as:Wherein, A and B is the gray value at coordinate；S903, it is exactly to utilize least square method to the optical flow computation of every tomographic image, seeks the derivative of matching error sum in neighborhood, most Excellent solution is derivative when being zero, reaches matching error and minimum, at this time the similarity highest between two frame corresponding points, then use Taylor Formula expansion can be obtained:Then the optimal solution of light stream isS904, human body contour outline area characteristic point light stream campaign so that judge how test object moves, such as:The motion state of test object can be described according to above-mentioned formula.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910463063.3A CN110097039A (en) | 2019-05-30 | 2019-05-30 | A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system |
| CN2019104630633 | 2019-05-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110533012A true CN110533012A (en) | 2019-12-03 |
Family
ID=67449701
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910463063.3A Pending CN110097039A (en) | 2019-05-30 | 2019-05-30 | A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system |
| CN201911005693.2A Withdrawn CN110533012A (en) | 2019-05-30 | 2019-10-22 | A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910463063.3A Pending CN110097039A (en) | 2019-05-30 | 2019-05-30 | A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN110097039A (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111248868A (en) * | 2020-02-20 | 2020-06-09 | 长沙湖湘医疗器械有限公司 | Quick eye movement sleep analysis method, system and equipment |
| CN112596615A (en) * | 2020-12-31 | 2021-04-02 | 广东富美达科技有限公司 | AI intelligent control system based on LED display screen and control method thereof |
| CN114596532B (en) * | 2021-07-23 | 2024-10-22 | 平安科技（深圳）有限公司 | Behavior detection method, device, equipment and storage medium |
| CN113723363B (en) * | 2021-07-28 | 2024-10-18 | 宁波星巡智能科技有限公司 | Infant sleep detection method, device, equipment and medium based on continuous frames |
| CN114419144B (en) * | 2022-01-20 | 2024-06-14 | 珠海市一杯米科技有限公司 | Card positioning method based on external contour shape analysis |
| CN118395371A (en) * | 2024-04-08 | 2024-07-26 | 杭州光特医疗科技有限公司 | Method and system for sleep stage by multistage fusion detection of sleep sound |
-
2019
- 2019-05-30 CN CN201910463063.3A patent/CN110097039A/en active Pending
- 2019-10-22 CN CN201911005693.2A patent/CN110533012A/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097039A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110533012A (en) | A kind of sleep state monitoring energy conservation based on deep learning image recognition is helped the elderly system | |
| Ji et al. | Fatigue state detection based on multi-index fusion and state recognition network | |
| Zhao et al. | Real-time detection of fall from bed using a single depth camera | |
| Soukupova et al. | Eye blink detection using facial landmarks | |
| CN109377616A (en) | A kind of access control system based on two-dimension human face identification | |
| Zheng et al. | Video dynamics detection using deep neural networks | |
| US10769909B1 (en) | Using sensor data to detect events | |
| WO2019051777A1 (en) | Reminding method and reminding system based on intelligent terminal | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| US11302156B1 (en) | User interfaces associated with device applications | |
| CN207663490U (en) | A kind of mixing recognition access control system management system based on neural calculation rod | |
| US10791607B1 (en) | Configuring and controlling light emitters | |
| Planinc et al. | Robust fall detection by combining 3D data and fuzzy logic | |
| CN111914742A (en) | Attendance checking method, system, terminal equipment and medium based on multi-mode biological characteristics | |
| US20230237699A1 (en) | Method and system for itelligently controlling children's usage of screen terminal | |
| CN114255508A (en) | OpenPose-based student posture detection analysis and efficiency evaluation method | |
| US11423762B1 (en) | Providing device power-level notifications | |
| CN115308768A (en) | Intelligent monitoring system under privacy environment | |
| CN107959756B (en) | System and method for automatically turning off electronic equipment during sleeping | |
| CN101196984A (en) | Fast face detecting method | |
| CN112700568A (en) | Identity authentication method, equipment and computer readable storage medium | |
| CN110148234A (en) | Campus brush face picks exchange method, storage medium and system | |
| Hu et al. | Demo abstract: Wireless glasses for non-contact facial expression monitoring | |
| Zheng et al. | Hand-over-face occlusion and distance adaptive heart rate detection based on imaging photoplethysmography and pixel distance in online learning | |
| CN209734011U (en) | Non-contact human body state monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20191203 |
|
| WW01 | Invention patent application withdrawn after publication |