CN110008816B - Method for detecting quilt kicked by baby in real time - Google Patents
Method for detecting quilt kicked by baby in real time Download PDFInfo
- Publication number
- CN110008816B CN110008816B CN201910079528.5A CN201910079528A CN110008816B CN 110008816 B CN110008816 B CN 110008816B CN 201910079528 A CN201910079528 A CN 201910079528A CN 110008816 B CN110008816 B CN 110008816B
- Authority
- CN
- China
- Prior art keywords
- quilt
- layer
- infant
- neural network
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 48
- 230000007958 sleep Effects 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 230000008859 change Effects 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims abstract description 7
- 230000004044 response Effects 0.000 claims description 23
- 238000011176 pooling Methods 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000000474 nursing effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000036578 sleeping time Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for detecting that a baby kicks a quilt in real time, which comprises the steps of constructing a multi-task convolutional neural network, wherein the convolutional neural network comprises a first task of detecting whether the baby kicks the quilt or not and a second task of detecting the position change of the baby before and after the quilt is kicked; acquiring a plurality of original images and quilt position images of an infant under the conditions of normal sleep and quilt kicking, and training a convolutional neural network after presetting a first task output label and a second task output label until the convolutional neural network is converged to obtain a trained detection model; and acquiring an image to be detected and a quilt position image thereof, guiding the image into a detection model for detection, and determining that the infant kicks the quilt if the image integrating the positions of the quilt under the conditions of normal sleep and quilt kicking of the infant is output. By implementing the quilt kicking detection device, the quilt kicking condition of the baby can be detected in real time, the quilt kicking detection device has the advantages of high efficiency, low cost and the like, remote reminding can be realized after the baby kicks the quilt, the baby can be covered with the quilt in time, and the baby can be prevented from catching a cold.
Description
Technical Field
The invention relates to the technical field of computer image processing, in particular to a method for detecting that a baby kicks a quilt in real time.
Background
Generally, infants under two years of age require an average of 12-20 hours of sleep per day, and thus the sleep condition of infants is a major concern for parents. According to research and investigation, 80% of the infant's diseases, colds and other conditions are caused by catching cold during sleep, and are most commonly caused by kicking quilt by the infant. Because the sleeping time of the baby is long, parents cannot monitor the baby at the side of the baby bed all the time, and the sleeping posture of the baby is random, the quilt is likely to be turned over and kicked away in the sleeping process, so that most of the cold reasons come from the quilt. Therefore, the method for detecting the quilt kicked by the baby has wide commercial application prospect in the aspect of baby nursing.
At present, the existing quilt kicking detection method mainly adopts a temperature sensor, a multi-sensor device such as infrared rays and the like and some wearable devices, but the detection methods have high cost, low efficiency and poor real-time performance. In addition, most of existing schemes only detect the kicked quilt and remotely remind through short messages or bluetooth and other modes, but the remote reminding modes cannot immediately process the catching cold condition of the baby, and the baby already catches cold in the time period from remote reminding to the arrival of parents.
Disclosure of Invention
The technical problem to be solved by the embodiment of the invention is to provide a method for detecting the quilt kicked by an infant in real time, which not only can detect the quilt kicked by the infant in real time and has the advantages of high efficiency, low cost and the like, but also can realize remote reminding after the infant kicks the quilt and cover the quilt in time, thereby preventing the infant from catching a cold.
In order to solve the technical problem, an embodiment of the present invention provides a method for detecting that an infant kicks a quilt in real time, where the method includes the following steps:
s1, constructing a multi-task convolutional neural network; the tasks of the convolutional neural network comprise a first task of detecting whether the infant kicks the quilt or not and a second task of detecting the position change before and after the infant kicks the quilt after the infant is detected to kick the quilt;
s2, acquiring a plurality of original images and quilt position images of the infant under the normal sleeping condition, acquiring a plurality of original images and quilt position images of the infant under the quilt kicking condition, further presetting a first task output in the convolutional neural network by taking the type of the images as a label, and presetting a second task output in the convolutional neural network by taking the images integrated with the quilt position under the normal sleeping condition of the infant and the quilt kicking condition of the infant as a label; when the label output by the first task in the convolutional neural network is 1, the corresponding image category is the original image under the condition that the baby kicks the quilt, and when the label output by the first task in the convolutional neural network is 0, the corresponding image category is the original image under the condition that the baby normally sleeps;
s3, training the convolutional neural network by adopting an error back propagation algorithm according to the plurality of original images and the images of the quilt position thereof under the condition that the infant is in normal sleep, the plurality of original images and the images of the quilt position thereof under the condition that the infant kicks the quilt, and the labels output by the first task and the second task in the preset convolutional neural network respectively until the convolutional neural network is converged to obtain a trained detection model;
and S4, acquiring an image to be detected of the infant in the current sleep and a quilt position image thereof, importing the acquired image to be detected of the infant in the current sleep into the obtained trained detection model for detection, and determining that the infant kicks a quilt if the image is output and integrates the images of the quilt position under the condition that the infant is in the normal sleep and under the condition that the infant kicks the quilt.
Wherein the method further comprises:
and generating alarm information and remotely sending the generated alarm information to parents.
Wherein the method further comprises:
and extracting the quilt position under the normal sleeping condition of the infant and the quilt position under the normal sleeping condition of the infant from the output image integrating the quilt positions under the normal sleeping condition of the infant and the quilt kicking condition of the infant, sending the extracted quilt position and the quilt position to the manipulator, and further driving the manipulator to clamp the quilt to move from the extracted quilt position under the normal sleeping condition of the infant to the extracted quilt position under the normal sleeping condition of the infant, so as to cover the quilt for the infant.
In step S1, the convolutional neural network has a specific structure as follows:
1. input layer
2.< =1 convolutional layer 1_1 (3 x3x 64)
3.< =2 nonlinear response Relu layer
4.< =3 convolution layer 1_2 (3 x3x 64)
5.< =4 nonlinear response Relu layer
6.< =5 pooling layer (2 x 2/2)
7.< =6 convolution layer 2_1 (3 x3x 128)
8.< =7 nonlinear response Relu layer
9.< =8 convolutional layer 2 (3 x3x 128)
10.< =9 nonlinear response Relu layer
11.< =10 pooling layer (2 x 2/2)
12.<=11 convolutional layer 3_1 (3 x3x 256)
13.< =12 nonlinear response Relu layer
14.< =13 convolutional layer 3 u 2 (3 x3x 256)
15.< =14 global average pooling layer
16.< =15 fully-connected layers (256 x 100) s
17.< =16 nonlinear response Relu layer
18.< =17 full connection layer (100 x 2)
19.< =14 deconvolution layer D1 (4 x4x 256)
20.< =19 convolutional layer D1_1 (3 x3x 256)
21.< =20 convolutional layer D1_2 (3 x3x 256)
22.<=21 deconvolution layer D2 (4 x4x 128)
23.< =22 convolutional layer D2_1 (3 x3x 128)
24.< =23 convolutional layer D2_2 (3 x3x 128)
25.< =24 convolutional layer D2_3 (3 x3x 2)
Wherein, the number in front of the symbol ". < = is the current layer number, and the number behind the symbol". < = is the input layer number; the inside of brackets behind the convolutional layer and the deconvolution layer are convolutional layer parameters, wherein the product of two multipliers in front of the convolutional layer parameters is the size of a convolutional kernel, and the multiplier behind the convolutional layer parameters is the number of channels; the bracketing layer parameter is arranged in brackets behind the pooling layer, wherein the product of two multipliers in front of the pooling layer parameter is the size of a pooling core, and the multiplier behind the pooling layer parameter is the step size; the parameters of the full connection layer are arranged in brackets behind the full connection layer, wherein the parameters behind the full connection layer are output categories, and whether the baby kicks or not is detected, so that the parameters are classified into two categories; the nonlinear response layer is composed of a nonlinear activation function Relu;
detecting whether the baby kicks the quilt or not is a first task of the convolutional neural network, outputting a result on the 18 th layer, and outputting a result of second classification; the loss function of the first task is a cross-entropy loss function represented by the following formula (1):
Loss 1 ＝-∑y′log(f 1 (x))+(1-y′)log(1-f 1 (x)) (1)
wherein x is the output of layer 18 of the convolutional neural network; y 'is the label of the image, y' is 0,1;
is a sigmoid function;
after the baby kicks the quilt, detecting the position change of the quilt before and after the baby kicks the quilt as a second task of the convolutional neural network, and outputting a result on a 25 th layer; the number of the positions is two, including the position before the quilt is kicked and the position after the quilt is kicked; the loss function of the second task is a multi-class cross-entropy loss function represented by the following formula (2):
Loss 2 ＝-∑y″log(f 2 (x)) (2)
wherein x is the output of layer 25 of the network, and y "is a hot spot image about the kicking quilt position coordinates;
is the softmax function.
The image of the quilt position integrated with the situation that the infant normally sleeps and the situation that the infant kicks the quilt in the step S2 is a heat point diagram related to the position coordinate generated by a Gaussian function of the following formula (3);
wherein x and y are coordinates of each pixel point in the image; a. b is the pixel coordinate position before and after kicking quilt respectively, and delta is the hot spot radius adjustment parameter.
The embodiment of the invention has the following beneficial effects:
the invention not only detects whether the infant kicks the quilt or not, but also can detect the kicked position of the quilt under the condition of kicking the quilt, and sends the position information to the terminal controlling the mechanical arm, so that the mechanical arm can cover the quilt for the infant in time, and simultaneously remotely remind parents in a short message mode, thereby overcoming the defects of the prior art, and solving the problems that the prior art needs various sensor devices, the detection cost is high, the detection efficiency is low, the detection accuracy is influenced by the sleeping posture of the infant, and the like.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is within the scope of the present invention for those skilled in the art to obtain other drawings based on the drawings without inventive labor.
Fig. 1 is a flowchart of a method for detecting that an infant kicks a quilt in real time according to an embodiment of the present invention;
fig. 2 is a comparison diagram of an original image in a case of normal sleep of an infant and an original image in a case of kicking the quilt by the infant in an application scenario of the method for detecting the kicking of the quilt by the infant in real time provided by the embodiment of the present invention; wherein a is an original image of the infant under the condition of normal sleep; b is an original image under the condition that the baby kicks the quilt;
fig. 3 is a heat point diagram formed by coordinates of quilt positions under the condition that an infant kicks the quilt in an application scene of the method for detecting the infant kicking the quilt in real time provided by the embodiment of the invention.
Detailed Description
To make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, in an embodiment of the present invention, a method for detecting that an infant kicks a quilt in real time is provided, the method includes the following steps:
s1, constructing a multi-task convolutional neural network; the tasks of the convolutional neural network comprise a first task of detecting whether the baby kicks the quilt or not and a second task of detecting the position change of the baby before and after the baby kicks the quilt after the baby is detected to kick the quilt;
the specific process is that a multitask convolution neural network is constructed, and the structure of the convolution neural network is as follows:
1. input layer
2.< =1 convolutional layer 1 (3 x3x 64)
3.< =2 nonlinear response Relu layer
4.< =3 convolutional layer 1 (3 x3x 64)
5.< =4 nonlinear response Relu layer
6.< =5 pooling layer (2 x 2/2)
7.< =6 convolution layer 2_1 (3 x3x 128)
8.< =7 nonlinear response Relu layer
9.< =8 convolutional layer 2 (3 x3x 128)
10.< =9 nonlinear response Relu layer
11.< =10 pooling layer (2 x 2/2)
12.<=11 convolutional layer 3_1 (3 x3x 256)
13.< =12 nonlinear response Relu layer
14.< =13 convolution layer 3_2 (3 x3x 256)
15.< =14 global average pooling layer
16.< =15 fully-connected layers (256 x 100) s
17.<=16 nonlinear response Relu layer
18.< =17 full connection layer (100 x 2)
19.< =14 deconvolution layer D1 (4 x4x 256)
20.< =19 convolutional layer D1_1 (3 x3x 256)
21.< =20 convolutional layer D1_2 (3 x3x 256)
22.< =21 deconvolution layer D2 (4 x4x 128)
23.< =22 convolutional layer D2_1 (3 x3x 128)
24.< =23 convolutional layer D2_2 (3 x3x 128)
25.< =24 convolutional layer D2_3 (3 x3x 2)
Wherein, the number before the symbol ". < = is the current layer number, and the number after the symbol". < = is the input layer number; for example, 2.< =1 indicates that the current layer is the second layer and the input is the first layer;
the inside of brackets behind the convolutional layer and the deconvolution layer are convolutional layer parameters, wherein the product of two multipliers in front of the convolutional layer parameters is the size of a convolutional kernel, and the multiplier behind the convolutional layer parameters is the number of channels; for example, 3x3x64, indicating a convolution kernel size of 3x3 and a channel number of 64;
the bracketing layer parameter is arranged in brackets behind the pooling layer, wherein the product of two multipliers in front of the pooling layer parameter is the size of a pooling kernel, and the multiplier behind the pooling layer parameter is the step length; for example, 2x2/2, indicating a pooling kernel size of 2x2, step size of 2;
the parameters of the full connection layer are arranged in brackets behind the full connection layer, wherein the parameters behind the full connection layer are output categories, and whether the baby kicks the quilt or not is detected, so that the classification is a second classification; for example, 2 in 512x2 represents the category of output;
the nonlinear response layer is formed by a nonlinear activation function Relu;
detecting whether the baby kicks the quilt or not is a first task of the convolutional neural network, outputting a result on the 18 th layer, and outputting a result of second classification; the loss function of the first task is a cross-entropy loss function represented by the following formula (1):
Loss 1 ＝-∑y′log(f 1 (x))+(1-y′)log(1-f 1 (x)) (1)
wherein x is the output of layer 18 of the convolutional neural network; y 'is a label of the image, y' belongs to 0;
is a sigmoid function;
detecting the position change before and after the quilt is kicked by the baby to be detected as a second task of the convolutional neural network, and outputting a result on a 25 th layer; the number of the positions is two, including the position before the quilt is kicked and the position after the quilt is kicked; the loss function of the second task is a multi-class cross-entropy loss function represented by the following formula (2):
Loss 2 ＝-∑y″log(f 2 (x)) (2)
wherein x is the output of layer 25 of the network, and y "is a hot spot image about the position coordinates of the kicker;
is the softmax function.
S2, acquiring a plurality of original images and quilt position images of the infant under the normal sleeping condition, acquiring a plurality of original images and quilt position images of the infant under the quilt kicking condition, further presetting a first task output in the convolutional neural network by taking the type of the images as a label, and presetting a second task output in the convolutional neural network by taking the images integrated with the quilt position under the normal sleeping condition of the infant and the quilt kicking condition of the infant as a label; when the label output by the first task in the convolutional neural network is 1, the corresponding image category is the original image under the condition that the baby kicks the quilt, and when the label output by the first task in the convolutional neural network is 0, the corresponding image category is the original image under the condition that the baby normally sleeps;
the specific process is that an original image (shown as b in fig. 2) under the condition that the baby kicks the quilt is collected as a positive sample, and an original image (shown as a in fig. 2) under the condition that the baby is in normal sleep is collected as a negative sample; setting an original image under a normal sleep condition, wherein the label is 0; setting an original image under the condition that a quilt is kicked by an infant, wherein the label is 1; in addition, there is a label of a position in the positive sample, and a gaussian function of the following formula (3) is used to generate a hotspot graph (as shown in fig. 3) about position coordinates, where the coordinate position of a point in the hotspot graph is the detected position:
wherein x and y are coordinates of each pixel point in the image; a. b is the pixel coordinate position before and after kicking quilt respectively, and delta is the hot spot radius adjustment parameter.
S3, training the convolutional neural network by adopting an error back propagation algorithm according to the multiple original images and the images of the quilt position thereof under the condition that the baby normally sleeps, the multiple original images and the images of the quilt position thereof under the condition that the baby kicks the quilt, and the labels respectively output by the first task and the second task in the preset convolutional neural network until the convolutional neural network converges to obtain a trained detection model;
the specific process is that the image collected in the step S2 and the label made are used for training the multitask convolution neural network constructed in the step S1 by adopting an error back propagation algorithm until the convolution neural network model is converged. During training, the positive sample image has a kicked sub-position label only when the convolutional neural network inputs the positive sample image, and the negative sample has no position label. Therefore, only when the positive sample is inputted, the parameters of 19-25 layers of the convolutional neural network in step S1 are trained, otherwise, only the parameters of 1-18 layers of the convolutional neural network are trained.
It should be noted that the back propagation algorithm is mainly used for training a multi-layer model, and the main body of the back propagation algorithm is iterative iteration of two links of excitation propagation and weight updating until a convergence condition is reached. And in the excitation propagation stage, the image passes through the convolution layer of the network to obtain characteristics, an output result is obtained on an output layer of the network, and the difference between the output result and a real result is obtained, so that the error between the output layer and a supervision layer is obtained. In the weight updating stage, the known error is multiplied by the derivative of the current layer response to the previous layer response, so as to obtain the gradient of the weight matrix between two layers, and then the weight matrix is adjusted in a certain proportion along the reverse direction of the gradient. Then, the gradient matrix is regarded as an error of the previous layer to calculate a weight matrix of the previous layer. And the updating of the whole model is finished by analogy.
And S4, acquiring an image to be detected when the infant is sleeping currently and a quilt position image thereof, importing the acquired image to be detected when the infant is sleeping currently into the acquired trained detection model for detection, and determining that the infant kicks the quilt if the image integrating the quilt position under the conditions that the infant is sleeping normally and the infant kicks the quilt is output.
The specific process is that for the detection model trained in step S3, an image (including the position of the quilt) of the infant during sleep is read in real time through the camera. The convolutional neural network outputs two results at the 18 th layer, and the two results respectively correspond to the labels 0 and 1; if the result corresponding to the label 0 is greater than the result corresponding to the label 1, the baby is considered not to kick the quilt, and the result output by the 25 th layer of the convolutional neural network is considered to be invalid and is not output. And if the result corresponding to the label 0 is smaller than the result corresponding to the label 1, the baby is considered to kick the quilt, and when the 18 th layer result is that the baby already kicks the quilt, the 25 th layer of the convolutional neural network outputs the position coordinates of the kick quilt.
At the moment, alarm information is generated and sent to parents remotely, and the parents are notified in a short message, weChat or QQ mode.
In addition, the quilt position under the normal sleeping condition of the infant and the quilt position under the quilt kicking condition of the infant are extracted from the output image integrating the quilt positions under the normal sleeping condition of the infant and the quilt kicking condition of the infant and are sent to the manipulator, the manipulator is further driven to clamp the quilt and move from the extracted quilt position under the quilt kicking condition of the infant to the extracted quilt position under the normal sleeping condition of the infant, the purpose that the quilt is covered on the infant is achieved, namely the terminal of the manipulator is in the standby state all the time until the information of the position coordinates in the step S4 is received, and when the information of the position coordinates is received, the manipulator covers the quilt on the infant according to the received information of the two positions.
The embodiment of the invention has the following beneficial effects:
the invention not only detects whether the infant kicks the quilt or not, but also can detect the kicked position of the quilt under the condition of kicking the quilt, and sends the position information to the terminal controlling the mechanical arm, so that the mechanical arm can cover the quilt for the infant in time, and simultaneously remotely remind parents in a short message mode, thereby overcoming the defects of the prior art, and solving the problems that the prior art needs various sensor devices, the detection cost is high, the detection efficiency is low, the detection accuracy is influenced by the sleeping posture of the infant, and the like.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by relevant hardware instructed by a program, and the program may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (5)
1. A method for detecting in real time that an infant kicks a quilt, the method comprising the steps of:
s1, constructing a multi-task convolutional neural network; the tasks of the convolutional neural network comprise a first task of detecting whether the infant kicks the quilt or not and a second task of detecting the position change before and after the infant kicks the quilt after the infant is detected to kick the quilt;
s2, acquiring a plurality of original images and quilt position images of the infant under the normal sleeping condition, acquiring a plurality of original images and quilt position images of the infant under the quilt kicking condition, further presetting a first task output in the convolutional neural network by taking the type of the images as a label, and presetting a second task output in the convolutional neural network by taking the images integrated with the quilt position under the normal sleeping condition of the infant and the quilt kicking condition of the infant as a label; when the label output by the first task in the convolutional neural network is 1, the corresponding image category is the original image under the condition that the baby kicks the quilt, and when the label output by the first task in the convolutional neural network is 0, the corresponding image category is the original image under the condition that the baby normally sleeps;
s3, training the convolutional neural network by adopting an error back propagation algorithm according to the plurality of original images and the images of the quilt position thereof under the condition that the infant is in normal sleep, the plurality of original images and the images of the quilt position thereof under the condition that the infant kicks the quilt, and the labels output by the first task and the second task in the preset convolutional neural network respectively until the convolutional neural network is converged to obtain a trained detection model;
and S4, acquiring an image to be detected of the infant in the current sleep and a quilt position image thereof, importing the acquired image to be detected of the infant in the current sleep into the obtained trained detection model for detection, and determining that the infant kicks a quilt if the image is output and integrates the images of the quilt position under the condition that the infant is in the normal sleep and under the condition that the infant kicks the quilt.
2. The method for detecting kicking of a quilt by an infant in real time as claimed in claim 1, wherein said method further comprises:
and generating alarm information and remotely sending the generated alarm information to parents.
3. The method for detecting kicking of a quilt by an infant in real time as claimed in claim 1, wherein said method further comprises:
and further driving the manipulator to clamp the quilt to move from the extracted quilt position under the situation that the baby kicks the quilt to the extracted quilt position under the situation that the baby normally sleeps, so as to cover the quilt for the baby.
4. The method for detecting infant quilt kicking in real time as claimed in claim 1, wherein in said step S1, the concrete structure of said convolutional neural network is as follows:
1. input layer
2.< =1 convolutional layer 1 (3 x3x 64)
3.< =2 nonlinear response Relu layer
4.< =3 convolutional layer 1 (3 x3x 64)
5.< =4 nonlinear response Relu layer
6.< =5 pooling layer (2 x 2/2)
7.< =6 convolutional layer 2 (u 1) (3 x3x 128)
8.< =7 nonlinear response Relu layer
9.< =8 convolution layer 2_2 (3 x3x 128)
10.< =9 nonlinear response Relu layer
11.< =10 pooling layer (2 x 2/2)
12.< =11 convolutional layer 3 u 1 (3 x3x 256)
13.< =12 nonlinear response Relu layer
14.< =13 convolutional layer 3 u 2 (3 x3x 256)
15.< =14 global average pooling layer
16.< =15 fully-connected layers (256 x 100) s
17.< =16 nonlinear response Relu layer
18.< =17 full connection layer (100 x 2)
19.< =14 deconvolution layer D1 (4 x4x 256)
20.< =19 convolutional layer D1_1 (3 x3x 256)
21.< =20 convolutional layer D1_2 (3 x3x 256)
22.< =21 deconvolution layer D2 (4 x4x 128)
23.< =22 convolutional layer D2_1 (3 x3x 128)
24.< =23 convolutional layer D2_2 (3 x3x 128)
25.< =24 convolutional layer D2_3 (3 x3x 2)
Wherein, the number before the symbol ". < = is the current layer number, and the number after the symbol". < = is the input layer number; the inside of brackets behind the convolutional layer and the deconvolution layer are convolutional layer parameters, wherein the product of two multipliers in front of the convolutional layer parameters is the size of a convolutional kernel, and the multiplier behind the convolutional layer parameters is the number of channels; the bracketing layer parameter is arranged in brackets behind the pooling layer, wherein the product of two multipliers in front of the pooling layer parameter is the size of a pooling kernel, and the multiplier behind the pooling layer parameter is the step length; the parameters of the full connection layer are arranged in brackets behind the full connection layer, wherein the parameters behind the full connection layer are output categories, and whether the baby kicks or not is detected, so that the parameters are classified into two categories; the nonlinear response layer is composed of a nonlinear activation function Relu;
detecting whether the baby kicks the quilt or not is a first task of the convolutional neural network, outputting a result on the 18 th layer, and outputting a result of second classification; the loss function of the first task is a cross-entropy loss function represented by the following formula (1):
Loss 1 ＝-∑y′log(f 1 (x))+(1-y′)log(1-f 1 (x)) (1)
wherein x is the output of layer 18 of the convolutional neural network; y 'is the label of the image, y' is 0,1;
is a sigmoid function;
detecting the position change before and after the quilt is kicked by the baby to be detected as a second task of the convolutional neural network, and outputting a result on a 25 th layer; the number of the positions is two, including the position before the quilt is kicked and the position after the quilt is kicked; the loss function of the second task is a multi-class cross-entropy loss function represented by the following formula (2):
Loss 2 ＝-∑y″log(f 2 (x)) (2)
wherein x is the output of layer 25 of the network, and y "is a hot spot image about the position coordinates of the kicker;
is the softmax function.
5. The method for detecting infant kicking quilt in real time according to claim 1, wherein the "image integrated with quilt position under the condition of infant normal sleep and infant kicking quilt" in step S2 is a heat point diagram generated by using the gaussian function of the following formula (3) with respect to position coordinates;
wherein x and y are coordinates of each pixel point in the image; a. b is the pixel coordinate position before and after kicking quilt respectively, delta is the hot spot radius adjustment parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910079528.5A CN110008816B (en) | 2019-01-28 | 2019-01-28 | Method for detecting quilt kicked by baby in real time |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910079528.5A CN110008816B (en) | 2019-01-28 | 2019-01-28 | Method for detecting quilt kicked by baby in real time |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008816A CN110008816A (en) | 2019-07-12 |
| CN110008816B true CN110008816B (en) | 2023-01-17 |
Family
ID=67165602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910079528.5A Active CN110008816B (en) | 2019-01-28 | 2019-01-28 | Method for detecting quilt kicked by baby in real time |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008816B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110580466A (en) * | 2019-09-05 | 2019-12-17 | 深圳市赛为智能股份有限公司 | infant quilt kicking behavior recognition method and device, computer equipment and storage medium |
| CN110751063A (en) * | 2019-09-29 | 2020-02-04 | 四川工商学院 | Infant quilt kicking prevention recognition device and method based on deep learning |
| CN113221802A (en) * | 2021-05-24 | 2021-08-06 | 新疆爱华盈通信息技术有限公司 | Quilt kicking identification method and device and electronic equipment |
| CN113465155B (en) * | 2021-06-23 | 2022-07-26 | 深圳市海清视讯科技有限公司 | Method, system, host and storage medium for reducing quilt kicking behavior of infants |
| CN113343891A (en) * | 2021-06-24 | 2021-09-03 | 深圳市起点人工智能科技有限公司 | Detection device and detection method for child kicking quilt |
| CN113384386B (en) * | 2021-08-18 | 2021-11-23 | 宁波星巡智能科技有限公司 | Infant quilt kicking intelligent detection method and device, electronic equipment and medium |
| CN115500639A (en) * | 2022-09-27 | 2022-12-23 | 慕思健康睡眠股份有限公司 | Intelligent bed |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017133009A1 (en) * | 2016-02-04 | 2017-08-10 | 广州新节奏智能科技有限公司 | Method for positioning human joint using depth image of convolutional neural network |
| WO2018035663A1 (en) * | 2016-08-22 | 2018-03-01 | 衣佳鑫 | Infant detection method and system for internet of things quilt cover |
| CN108388890A (en) * | 2018-03-26 | 2018-08-10 | 南京邮电大学 | A kind of neonatal pain degree assessment method and system based on human facial expression recognition |
-
2019
- 2019-01-28 CN CN201910079528.5A patent/CN110008816B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017133009A1 (en) * | 2016-02-04 | 2017-08-10 | 广州新节奏智能科技有限公司 | Method for positioning human joint using depth image of convolutional neural network |
| WO2018035663A1 (en) * | 2016-08-22 | 2018-03-01 | 衣佳鑫 | Infant detection method and system for internet of things quilt cover |
| CN108388890A (en) * | 2018-03-26 | 2018-08-10 | 南京邮电大学 | A kind of neonatal pain degree assessment method and system based on human facial expression recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008816A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008816B (en) | Method for detecting quilt kicked by baby in real time | |
| CN113642361B (en) | Fall behavior detection method and equipment | |
| US9864912B2 (en) | Large margin high-order deep learning with auxiliary tasks for video-based anomaly detection | |
| CN109284733B (en) | Shopping guide negative behavior monitoring method based on yolo and multitask convolutional neural network | |
| CN110291489A (en) | The efficient mankind identify intelligent assistant's computer in calculating | |
| Jayaraman et al. | End-to-end policy learning for active visual categorization | |
| CN101383000A (en) | Information processing apparatus, information processing method, and computer program | |
| Voulodimos et al. | Improving multi-camera activity recognition by employing neural network based readjustment | |
| CN110751063A (en) | Infant quilt kicking prevention recognition device and method based on deep learning | |
| Gao et al. | An end-to-end broad learning system for event-based object classification | |
| CN109766790B (en) | Pedestrian detection method based on self-adaptive characteristic channel | |
| US10621424B2 (en) | Multi-level state detecting system and method | |
| Idrees et al. | Robomem: Giving long term memory to robots | |
| Bo | Human fall detection for smart home caring using yolo networks | |
| Wang et al. | A learning-based prediction model for baby accidents | |
| De Souza Silva et al. | Locar–low-cost autonomous robot for object detection with voice command and MobileNets | |
| Huang et al. | The Development of Artificial Intelligence (AI) Algorithms to avoid potential baby sleep hazards in smart buildings | |
| CN115542759A (en) | Intelligent household control method, system and storage medium based on multi-mode recognition | |
| Soman et al. | A Novel Fall Detection System using Mediapipe | |
| Dichwalkar et al. | Activity recognition and fall detection in elderly people | |
| Rege et al. | Vision-based approach to senior healthcare: Depth-based activity recognition with convolutional neural networks | |
| TW202141344A (en) | Motion recognition system | |
| Qian et al. | Detection and recognition of abnormal behavior based on multi-level residual network | |
| Chen et al. | Overall understanding of indoor scenes by fusing multiframe local rgb-d data based on conditional random fields | |
| US20230386164A1 (en) | Method for training an object recognition model in a computing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190712 Assignee: Wenzhou Kehong Robot Co.,Ltd. Assignor: Wenzhou University Contract record no.: X2024330000007 Denomination of invention: A real-time detection method for infants kicking blankets Granted publication date: 20230117 License type: Common License Record date: 20240124 |