CN104077881A - Infant monitoring method and device based on robot vision - Google Patents
Infant monitoring method and device based on robot vision Download PDFInfo
- Publication number
- CN104077881A CN104077881A CN201410310293.3A CN201410310293A CN104077881A CN 104077881 A CN104077881 A CN 104077881A CN 201410310293 A CN201410310293 A CN 201410310293A CN 104077881 A CN104077881 A CN 104077881A
- Authority
- CN
- China
- Prior art keywords
- infant
- infants
- video image
- baby
- heart rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012544 monitoring process Methods 0.000 title claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 230000003945 visual behavior Effects 0.000 claims abstract description 10
- 230000000007 visual effect Effects 0.000 claims abstract description 10
- 230000017105 transposition Effects 0.000 claims abstract description 9
- 230000007958 sleep Effects 0.000 claims abstract description 8
- 230000005856 abnormality Effects 0.000 claims abstract description 6
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 25
- 238000001228 spectrum Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 16
- 238000012880 independent component analysis Methods 0.000 claims description 14
- 238000012806 monitoring device Methods 0.000 claims description 14
- 230000004927 fusion Effects 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 2
- 238000010191 image analysis Methods 0.000 claims 3
- 230000002159 abnormal effect Effects 0.000 abstract description 4
- 230000003862 health status Effects 0.000 abstract description 3
- 238000012423 maintenance Methods 0.000 abstract description 3
- 210000003128 head Anatomy 0.000 description 12
- 230000035565 breathing frequency Effects 0.000 description 8
- 238000010606 normalization Methods 0.000 description 8
- 230000002087 whitening effect Effects 0.000 description 8
- 238000005259 measurement Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 4
- 210000001015 abdomen Anatomy 0.000 description 2
- 238000013019 agitation Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000000038 chest Anatomy 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000000887 face Anatomy 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036760 body temperature Effects 0.000 description 1
- 230000002612 cardiopulmonary effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000010792 warming Methods 0.000 description 1
Landscapes
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
本发明公开了一种基于机器视觉的婴幼儿监测方法及装置，方法包括：通过眼部视频图像、唇部视频图像和面颊部视频图像，分别检测表征婴幼儿的视觉行为特征和生理参数特征：将上述参数进行融合判断，得出婴幼儿的睡眠状态检测结果。装置包括：嵌入式系统用于提取婴幼儿的嘴巴张合的频度、头部位置及其维持的时间、换位的频度的视觉特征，以及脉率等被测试者的生理参数特征，对这些特征信息进行融合处理和判断报警；当婴幼儿的睡眠状态发现异常时，输出报警信号至报警器，同时输出异常信号给家长或相关监护人员。本发明只采用单一摄像头获取上述信息，获得了包含婴、幼儿的生理参数在内的更多信息，大幅度提高了对婴、幼儿的健康状态监测的可靠性。
The invention discloses a machine vision-based infant monitoring method and device. The method includes: respectively detecting visual behavior characteristics and physiological parameter characteristics representing infants through eye video images, lip video images and cheek video images: The above parameters are fused and judged to obtain the sleep state detection results of infants. The device includes: the embedded system is used to extract the visual features of the frequency of opening and closing the mouth of infants, the position of the head and its maintenance time, the frequency of transposition, and the physiological parameters of the subjects such as pulse rate. These characteristic information are fused and processed and judged and alarmed; when an abnormality is found in the sleeping state of infants and young children, an alarm signal is output to the alarm, and an abnormal signal is output to parents or related guardians at the same time. The present invention only uses a single camera to obtain the above information, obtains more information including physiological parameters of infants and young children, and greatly improves the reliability of monitoring the health status of infants and young children.
Description
技术领域technical field
本发明涉及机器视觉领域，尤其涉及一种基于机器视觉的婴幼儿监测方法及装置。The invention relates to the field of machine vision, in particular to a machine vision-based infant monitoring method and device.
背景技术Background technique
对婴幼儿的监护是许多父母所揪心的事情。然后，现有技术存在诸多的不足：The guardianship of infants and young children is something that many parents worry about. Then, there are many deficiencies in the prior art:
张炎提出的中国发明专利(公开号：CN103700215A)《一种低辐射可穿戴式婴儿监测器》，只能监测温、湿度和声音；J·P·小蒂潘提出的中国发明专利(公开号：CN103220967A)《用于检测婴儿心肺不适的超宽带(UWB)婴儿监测器》，采用微波监测婴儿的状态，不易被家长接受；徐晶提出的中国发明专利(公开号：CN101564302)《基于多源信息融合的婴儿睡眠躁动监测方法及检测系统》，监测温、湿度并由此推测婴儿的躁动，不能监测重要的生命指征；沈云明提出的中国发明专利(公开号：CN202614402U)《基于ZigBee的婴儿培养箱温度监测与报警系统》，仅能监测婴儿培养箱温度；韩猛提出的中国发明专利(公开号：CN103156617A)《胸腹部呼吸运动波形传感器》，利用非晶丝巨应力阻抗效应研制的测量人体胸腹部呼吸运动波形，既需要裹覆在婴幼儿的身体，又仅仅能够监测婴幼儿的呼吸；宁新宝提出的中国发明专利(公开号：CN1377628)《婴儿保温床的智能监护装置》，采用多种传感器采集被监护婴儿的体温和心电等参数，属于接触测量，测量时间一长就容易对婴儿皮肤造成伤害；孙萍提出的中国发明专利(公开号：CN1564213)《婴儿睡眠监控装置》，仅能监测婴儿的声音；丁永生提出的中国发明专利(公开号：CN1961829)《一种婴儿睡眠姿势监测绑带》，依靠婴儿睡眠时穿着的绑带监测婴儿的睡眠姿势，既不可靠，获得的信息太少；崔伶玲提出的中国发明专利(公开号：CN102968753A)《基于RFID技术的新生婴儿看护系统》，采用接触式的传感器和有源RFID标签，属于接触测量，测量时间一长就容易对婴儿皮肤造成伤害，还有长期近距离的无线辐射，不易被家长接受。The Chinese invention patent (publication number: CN103700215A) proposed by Zhang Yan "a low-radiation wearable baby monitor" can only monitor temperature, humidity and sound; the Chinese invention patent (publication number) proposed by J.P. Tippan : CN103220967A) "Ultra-Wideband (UWB) Baby Monitor for Detecting Baby's Cardiopulmonary Discomfort", adopts microwave to monitor the baby's state, which is not easy to be accepted by parents; Xu Jing's Chinese invention patent (publication number: CN101564302) "based on multi-source Infant sleep agitation monitoring method and detection system based on information fusion ", which monitors temperature and humidity and thus infers the baby's agitation, but cannot monitor important vital signs; Shen Yunming's Chinese invention patent (public number: CN202614402U) "Infant based on ZigBee Incubator temperature monitoring and alarm system", which can only monitor the temperature of infant incubator; Han Meng's Chinese invention patent (public number: CN103156617A) "thorax and abdomen respiratory motion waveform sensor", a measurement developed by using the giant stress impedance effect of amorphous wire The waveform of human chest and abdomen breathing movement needs to be wrapped in the body of infants and young children, and can only monitor the breathing of infants and young children; the Chinese invention patent (public number: CN1377628) "Intelligent monitoring device for baby warming bed" proposed by Ning Xinbao, A variety of sensors are used to collect parameters such as body temperature and electrocardiogram of the baby being monitored, which belongs to contact measurement. If the measurement time is long, it will easily cause damage to the baby's skin; the Chinese invention patent (public number: CN1564213) "Baby Sleep Monitoring Device" proposed by Sun Ping ", can only monitor the baby's voice; Ding Yongsheng's Chinese invention patent (public number: CN1961829) "A Baby Sleeping Posture Monitoring Bandage", relying on the baby's sleeping straps to monitor the baby's sleeping position is not reliable. , the information obtained is too little; the Chinese invention patent (public number: CN102968753A) "Newborn Baby Care System Based on RFID Technology" proposed by Cui Lingling uses contact sensors and active RFID tags, which belongs to contact measurement, and the measurement time will be lost after a long time. It is easy to cause damage to the baby's skin, and there is long-term close-range wireless radiation, which is not easily accepted by parents.
总而言之，现有监测婴、幼儿的技术，要么不能监测婴、幼儿的心率、呼吸等重要的生理参数，要么只能有接触的传感器进行测量，或者是采用微波具有辐射的技术来实现。All in all, the existing technologies for monitoring infants and young children either cannot monitor important physiological parameters such as heart rate and respiration of infants and young children, or can only be measured by contact sensors, or use microwave radiation technology to achieve.
发明内容Contents of the invention
本发明提供了一种基于机器视觉的婴幼儿监测方法及装置，本发明获得了包含婴、幼儿的生理参数在内的更多信息，大幅度提高了对婴、幼儿的健康状态监测的可靠性，详见下文描述：The invention provides a method and device for monitoring infants and young children based on machine vision. The invention obtains more information including physiological parameters of infants and young children, and greatly improves the reliability of monitoring the health status of infants and young children , see the description below:
一种基于机器视觉的婴幼儿监测方法，所述方法包括以下步骤：A kind of infant monitoring method based on machine vision, described method comprises the following steps:
通过设置在婴幼儿枕头部位上方的摄像头获取婴幼儿脸部的视频图像；Obtain the video image of the infant's face through the camera arranged above the infant's pillow position;
对视频图像进行清洗，获取清洗后的视频图像；Cleaning the video image to obtain the cleaned video image;
对清洗后的视频图像进行分割，获取眼部视频图像、唇部视频图像和面颊部视频图像；Segment the cleaned video images to obtain eye video images, lip video images and cheek video images;
通过眼部视频图像、唇部视频图像和面颊部视频图像，分别检测表征婴幼儿的视觉行为特征和生理参数特征：Through eye video images, lip video images and cheek video images, the visual behavior characteristics and physiological parameter characteristics of infants and young children are detected respectively:
将上述参数进行融合判断，得出婴幼儿的睡眠状态检测结果。The above parameters are fused and judged to obtain the sleep state detection results of infants and young children.
所述通过眼部视频图像、唇部视频图像和面颊部视频图像，分别检测表征婴幼儿的视觉行为特征和生理参数特征的步骤具体为：The steps of detecting and characterizing visual behavior characteristics and physiological parameter characteristics of infants and young children respectively through eye video images, lip video images and cheek video images are specifically:
1)通过所述唇部视频图像检测婴、幼儿的嘴巴张合的频度、通过所述眼部视频图像检测婴、幼儿的头部位置及其维持的时间、换位频度的视觉特征；1) detect the frequency of mouth opening and closing of infants and young children through the lip video image, detect the head position of infants and young children through the eye video image and the visual characteristics of the time it is maintained and the frequency of transposition;
2)通过对所述面颊部视频图像分析得到婴、幼儿的心率、呼吸次数及其心率变异值、呼吸次数变异值。2) Obtaining the heart rate, respiration rate, heart rate variation value, and respiration rate variation value of infants and young children by analyzing the cheek video images.
当摄像头为彩色摄像头时，所述通过对所述面颊部视频图像分析得到婴、幼儿的心率、呼吸次数及其心率变异值、呼吸次数变异值的步骤具体为：When the camera is a color camera, the step of obtaining the heart rate, the number of breaths and its heart rate variation value and the variation value of the number of breaths of the infant or young child by analyzing the cheek video image is specifically:
(a)分离脸颊部区域的RGB通道，对每个通道的像素点求平均值；(a) Separate the RGB channels of the cheek region, and average the pixels of each channel;
(b)将数据保存成3个通道，每个通道N个数据；(b) Save the data into 3 channels, each with N data;
(c)对每个通道数据依次进行归一化、白化处理以及独立成分分析；(c) Perform normalization, whitening processing and independent component analysis on the data of each channel in turn;
(d)对每个通道独立成分分析后的数据进行傅立叶变化，求出功率谱，得到功率谱最大幅值所对应的频率，即婴、幼儿的心率，另得到功率谱次最大幅值所对应的频率，即婴、幼儿的呼吸次数；(d) Perform Fourier transformation on the data after the independent component analysis of each channel, obtain the power spectrum, and obtain the frequency corresponding to the maximum amplitude of the power spectrum, that is, the heart rate of infants and young children, and obtain the second maximum amplitude of the power spectrum corresponding to The frequency, that is, the number of breaths of infants and young children;
(e)通过单一通道的婴、幼儿的心率和呼吸次数计算婴、幼儿的心率变异值和呼吸次数变异值，进而通过对多个通道的计算获取心率变异值。(e) Calculating the heart rate variation value and the breathing frequency variation value of the infant or toddler through the heart rate and breathing frequency of the infant or young child in a single channel, and then obtaining the heart rate variation value through calculation of multiple channels.
当摄像头为红外摄像头时，所述通过对所述面颊部视频图像分析得到婴、幼儿的心率、呼吸次数及其心率变异值、呼吸次数变异值的步骤具体为：When the camera is an infrared camera, the step of obtaining the heart rate, the number of respirations and its heart rate variation value and the variation value of the number of respirations of the infant or young child by analyzing the cheek video image is specifically:
(a)对脸颊部区域的IR通道的像素点求平均值，IR通道内有N个数据；(a) average the pixels of the IR channel in the cheek region, and there are N data in the IR channel;
(b)对IR通道内数据依次进行归一化、白化处理以及独立成分分析；(b) Perform normalization, whitening processing and independent component analysis on the data in the IR channel in turn;
(c)对每个独立成分分析后的数据进行傅立叶变化，求出功率谱，得到功率谱最大幅值所对应的频率，即婴、幼儿的心率，另得到功率谱次最大幅值所对应的频率，即婴、幼儿的呼吸次数；(c) Perform Fourier transformation on the data after each independent component analysis, obtain the power spectrum, and obtain the frequency corresponding to the maximum amplitude of the power spectrum, that is, the heart rate of infants and young children, and obtain the frequency corresponding to the second maximum amplitude of the power spectrum Frequency, that is, the number of breaths of infants and young children;
(d)通过婴、幼儿的心率和呼吸次数计算婴、幼儿的心率变异值和呼吸次数变异值。(d) Calculating the heart rate variation value and breathing frequency variation value of infants and young children through the heart rate and breathing frequency of infants and young children.
一种基于机器视觉的婴幼儿监测装置，所述婴幼儿监测装置包括：摄像头、编码器、显示器、嵌入式系统、报警器，A kind of infant monitoring device based on machine vision, described infant monitoring device comprises: camera, encoder, display, embedded system, alarm,
所述摄像头用于实时摄录婴幼儿头脸部的影像并输入到所述编码器；The camera is used to record the image of the infant's head and face in real time and input to the encoder;
所述编码器用于将摄像头的影像信息编码成数字图像信号送入所述嵌入式系统；The encoder is used to encode the image information of the camera into a digital image signal and send it to the embedded system;
所述嵌入式系统用于提取婴幼儿的嘴巴张合的频度、头部位置及其维持的时间、换位的频度的视觉特征，以及脉率等被测试者的生理参数特征，对这些特征信息进行融合处理和判断报警；当婴幼儿的睡眠状态发现异常时，输出报警信号至所述报警器，同时输出异常信号给家长或相关监护人员；The embedded system is used to extract the frequency of mouth opening and closing of infants, the visual characteristics of the head position and the time it is maintained, the frequency of transposition, and the physiological parameter characteristics of the testee such as pulse rate. The feature information is fused and judged to be an alarm; when an abnormality is found in the sleep state of the infant, an alarm signal is output to the alarm, and an abnormal signal is output to the parents or related guardians at the same time;
所述报警器用于发出报警声响；The alarm is used to sound an alarm;
所述显示器用于显示婴幼儿的生理参数。The display is used to display the physiological parameters of the infant.
所述摄像头为：The cameras are:
红外摄像头和/或彩色摄像头。Infrared camera and/or color camera.
为了获取到较高精度的视频图像，所述彩色摄像头为：ANC酷睿HD1080P，所述红外摄像头为三星SCO-2080RP。In order to obtain higher-precision video images, the color camera is: ANC Core HD1080P, and the infrared camera is Samsung SCO-2080RP.
进一步地，为了提高处理速度，所述嵌入式系统为：i.MX6处理器、OMAP处理器和TigerSHARC处理器中的一种。Further, in order to increase the processing speed, the embedded system is: one of i.MX6 processor, OMAP processor and TigerSHARC processor.
本发明提供的技术方案的有益效果是：本发明既检测了婴、幼儿的生理参数特征，又检测了婴、幼儿的头脸部的视觉特征，且只采用单一摄像头获取上述信息，既保留了非接触测量所具备的不影响婴、幼儿的行为和增加婴、幼儿的佩戴负担，又获得包含婴、幼儿的生理参数在内的更多信息，大幅度提高了对婴、幼儿的健康状态监测的可靠性。The beneficial effects of the technical solution provided by the present invention are: the present invention not only detects the physiological parameter characteristics of infants and young children, but also detects the visual characteristics of the head and face of infants and young children, and only uses a single camera to obtain the above information, which not only retains The non-contact measurement does not affect the behavior of infants and young children and increases the wearing burden of infants and young children, and obtains more information including physiological parameters of infants and young children, which greatly improves the health status monitoring of infants and young children reliability.
附图说明Description of drawings
图1为本发明提供的一种基于机器视觉的婴幼儿监测方法的流程图；Fig. 1 is the flowchart of a kind of infant monitoring method based on machine vision provided by the present invention;
图2为本发明提供的的基于图像处理提取脉率和呼吸等婴幼儿的生理参数特征的流程图；Fig. 2 is the flow chart that extracts the physiological parameter characteristics of infants and young children such as pulse rate and respiration based on image processing provided by the present invention;
图3为本发明提供的一种基于机器视觉的婴幼儿监测方法的另一流程图；Fig. 3 is another flowchart of a kind of infant monitoring method based on machine vision provided by the present invention;
图4为本发明提供的的基于图像处理提取脉率和呼吸等婴幼儿的生理参数特征的另一流程图；Fig. 4 is another flow chart of extracting the physiological parameter characteristics of infants such as pulse rate and respiration based on image processing provided by the present invention;
图5为本发明提供的一种基于机器视觉的婴幼儿监测装置的结构图。Fig. 5 is a structural diagram of an infant monitoring device based on machine vision provided by the present invention.
附图中，各标号所代表的部件列表如下：In the accompanying drawings, the list of parts represented by each label is as follows:
1：摄像头； 2：编码器；1: Camera; 2: Encoder;
3：显示器； 4：嵌入式系统；3: Display; 4: Embedded system;
5：报警器。5: Alarm.
其中，摄像头1在两个装置实施例中分别为彩色摄像头和红外摄像头。Wherein, the camera 1 is a color camera and an infrared camera in the two device embodiments respectively.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚，下面对本发明实施方式作进一步地详细描述。In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation manners of the present invention will be further described in detail below.
实施例1Example 1
一种基于机器视觉的婴幼儿监测方法，获取视频图像时采用1个彩色摄像头，参见图1和图2，该方法包括以下步骤：A kind of infant monitoring method based on machine vision, adopts 1 color camera when acquiring video image, see Fig. 1 and Fig. 2, this method comprises the following steps:
101：通过设置在婴、幼儿枕头部位上方的彩色摄像头获取婴、幼儿脸部的视频图像；101: Obtain video images of infants and infants' faces through a color camera set above the pillows of infants and infants;
其中，为了提高获取到的视频图像的精度，该步骤中所用的彩色摄像采用ANC酷睿，具体实现时，还可以根据实际应用中的需要进行选择，本发明实施例对此不作限制。Among them, in order to improve the accuracy of the acquired video image, the color camera used in this step adopts ANC Core, which can also be selected according to the needs of practical applications during specific implementation, which is not limited in the embodiment of the present invention.
102：对视频图像进行清洗，获取清洗后的视频图像；102: Clean the video image, and obtain the cleaned video image;
其中，该清洗方法根据采集到的视频图像的不同，可以用图像剪裁筛选、图像灰度化和直方图均衡、图像归一化、视频帧间匹配、图像白化处理、去除奇异图像来优化数据集等处理方法。例如：可以通过直方图均衡和提取图像特征点并归一化等预处理步骤可以去除环境光强变化的影响。该步骤的详细操作为本领域技术人员所公知，本发明实施例对此不做赘述。Among them, the cleaning method can use image clipping and screening, image grayscale and histogram equalization, image normalization, video frame matching, image whitening processing, and removal of singular images to optimize the data set according to the difference of the collected video images. and other processing methods. For example, preprocessing steps such as histogram equalization, extraction of image feature points and normalization can be used to remove the influence of ambient light intensity changes. The detailed operation of this step is well known to those skilled in the art, and will not be repeated in this embodiment of the present invention.
103：对清洗后的视频图像进行分割，获取眼部视频图像、唇部视频图像和面颊部视频图像；103: Segment the cleaned video images to obtain eye video images, lip video images, and cheek video images;
其中，对清洗后的视频图像进行分割的方法为：根据检测到的人脸部的视频图像，按照人脸结构的“三庭五眼”准则对人脸的区域进行划分，以此分割出眼部视频图像、唇部视频图像和面颊部的视频图像。“三庭五眼”准则为本领域技术人员所公知，本发明实施例对此不做赘述。Among them, the method of segmenting the cleaned video image is as follows: according to the detected video image of the human face, the area of the human face is divided according to the "three courts and five eyes" principle of the human face structure, so as to segment the eye area. video images of the upper body, lips, and cheeks. The principle of "three courts and five eyes" is well known to those skilled in the art, and will not be described in detail in the embodiment of the present invention.
104：通过眼部视频图像、唇部视频图像和面颊部视频图像，分别检测表征婴、幼儿的视觉行为特征和生理参数特征：104: Through eye video images, lip video images and cheek video images, respectively detect visual behavior characteristics and physiological parameter characteristics representing infants and young children:
1)通过唇部视频图像检测婴、幼儿的嘴巴张合的频度、通过眼部视频图像检测婴、幼儿的头部位置及其维持的时间、换位频度的视觉特征；1) Detect the frequency of mouth opening and closing of infants and young children through lip video images, and detect the visual characteristics of infants and young children's head position and maintenance time and transposition frequency through eye video images;
例如：如果婴、幼儿的头部位置变换频率过高，根据一些经验和数据标准推断，在某种程度上可以判定为婴、幼儿在打瞌睡。For example: if the head position of the infant or toddler changes too frequently, it can be determined to some extent that the infant or toddler is dozing off according to some experience and data standards.
2)通过对面颊部视频图像分析得到婴、幼儿的心率、呼吸次数及其心率变异值、呼吸次数变异值(示于图2中)。2) By analyzing the video images of the cheeks, the heart rate, the number of respirations and the variation value of the heart rate and the number of respirations of infants and young children are obtained (shown in FIG. 2 ).
该步骤2)具体为：The step 2) is specifically:
(a)分离脸颊部区域的RGB通道，对每个通道的像素点求平均值；(a) Separate the RGB channels of the cheek region, and average the pixels of each channel;
(b)将数据保存成3个通道，每个通道N个数据；(b) Save the data into 3 channels, each with N data;
(c)对每个通道数据依次进行归一化、白化处理以及独立成分分析；(c) Perform normalization, whitening processing and independent component analysis on the data of each channel in turn;
其中，通过白化处理来消除三个通道之间的相关性，防止了通道间的干扰。步骤(c)的处理中采用的归一化、白化处理以及独立成分分析，为本领域技术人员所公知的几种技术，本发明实施例对此不做赘述。Among them, the correlation between the three channels is eliminated through whitening processing, and the interference among the channels is prevented. The normalization, whitening processing and independent component analysis used in the processing of step (c) are several technologies well known to those skilled in the art, which will not be described in detail in the embodiment of the present invention.
(d)对每个通道独立成分分析后的数据进行傅立叶变化，求出功率谱，得到功率谱最大幅值所对应的频率，即婴、幼儿的心率，另得到功率谱次最大幅值所对应的频率，即婴、幼儿的呼吸次数；(d) Perform Fourier transformation on the data after the independent component analysis of each channel, obtain the power spectrum, and obtain the frequency corresponding to the maximum amplitude of the power spectrum, that is, the heart rate of infants and young children, and obtain the second maximum amplitude of the power spectrum corresponding to The frequency, that is, the number of breaths of infants and young children;
(e)通过单一通道的婴、幼儿的心率和呼吸次数计算婴、幼儿的心率变异值和呼吸次数变异值，进而通过对多个通道的计算获取心率变异值；(e) Calculating the heart rate variation value and breathing frequency variation value of infants and young children through the heart rate and breathing frequency of infants and young children in a single channel, and then obtaining the heart rate variation value through the calculation of multiple channels;
其中，该步骤采用了本领域中的Lomb periodogram原理，具体实现时本发明实施例对此不作赘述。Wherein, this step adopts the principle of Lomb periodogram in the field, and this embodiment of the present invention will not repeat it in specific implementation.
105：将上述参数进行融合判断，得出婴、幼儿的状态检测结果。105: Perform fusion judgment on the above parameters to obtain the state detection result of the infant or toddler.
其中，该步骤中所述的融合判断可以为：Wherein, the fusion judgment described in this step can be:
1)通过表征婴、幼儿的视觉行为特征可以直接得出婴、幼儿的检测结果；或，1) By characterizing the visual behavior characteristics of infants and young children, the detection results of infants and young children can be directly obtained; or,
2)通过心率变异值可以直接得出婴、幼儿的检测结果；或，2) The test results of infants and young children can be directly obtained through the heart rate variation value; or,
3)通过表征婴、幼儿的视觉行为特征和心率变异值的加权和来得出婴、幼儿的检测结果。3) The detection results of infants and young children are obtained by characterizing the visual behavior characteristics of infants and young children and the weighted sum of heart rate variation values.
即将表征婴、幼儿的视觉行为特征和心率变异值的加权和预设的阈值进行比较，若大于阈值，则直接判断得出婴、幼儿的检测结果。其中，权重系数和阈值根据实际应用中的需要进行设定，本发明实施例对此不做限制。That is to compare the weighting of visual behavior characteristics and heart rate variation values representing infants and young children with the preset threshold value. If it is greater than the threshold value, the detection result of infants and young children can be directly judged. Wherein, the weight coefficient and the threshold are set according to requirements in practical applications, which are not limited in this embodiment of the present invention.
实施例2Example 2
一种基于机器视觉的婴幼儿监测方法，获取视频图像时采用1个红外摄像头，参见图3和图4，该方法包括以下步骤：A method for monitoring infants and young children based on machine vision, using an infrared camera when acquiring video images, see Figure 3 and Figure 4, the method includes the following steps:
201：通过设置在婴、幼儿枕头部位上方的红外摄像头获取婴、幼儿脸部的视频图像；201: Obtain video images of infants and infants' faces through an infrared camera set above the pillows of infants and infants;
其中，为了提高获取到的视频图像的精度，该步骤中所用的红外摄像头采用三星SCO-2080RP，具体实现时，还可以根据实际应用中的需要进行选择，本发明实施例对此不作限制。Wherein, in order to improve the accuracy of the acquired video image, the infrared camera used in this step adopts Samsung SCO-2080RP, and it can also be selected according to the needs in practical applications during specific implementation, which is not limited in the embodiment of the present invention.
202：对视频图像进行清洗，获取清洗后的视频图像；202: Clean the video image, and obtain the cleaned video image;
203：对清洗后的视频图像进行分割，获取眼部视频图像、唇部视频图像和面颊部视频图像；203: Segment the cleaned video images to obtain eye video images, lip video images and cheek video images;
其中，步骤202和步骤203的操作步骤和实施例1相同，本发明实施例对此不做赘述。Wherein, the operation steps of step 202 and step 203 are the same as those in embodiment 1, which will not be described in detail in this embodiment of the present invention.
204：通过眼部视频图像、唇部视频图像和面颊部视频图像，分别检测表征婴、幼儿的视觉行为特征和生理参数特征：204: Through eye video images, lip video images and cheek video images, respectively detect visual behavior characteristics and physiological parameter characteristics representing infants and young children:
1)通过唇部视频图像检测婴、幼儿的嘴巴张合的频度、通过眼部视频图像检测婴、幼儿的头部位置及其维持的时间、换位频度的视觉特征；1) Detect the frequency of mouth opening and closing of infants and young children through lip video images, and detect the visual characteristics of infants and young children's head position and maintenance time and transposition frequency through eye video images;
例如：如果婴、幼儿的头部位置变换频率过高，根据一些经验和数据标准推断，在某种程度上可以判定为婴、幼儿在打瞌睡。For example: if the head position of the infant or toddler changes too frequently, it can be determined to some extent that the infant or toddler is dozing off according to some experience and data standards.
2)通过对面颊部视频图像分析得到婴、幼儿的心率、呼吸次数及其心率变异值、呼吸次数变异值(示于图4中)。2) Obtain the heart rate, respiration rate and its heart rate variation value and respiration frequency variation value of infants and young children by analyzing the cheek video images (shown in FIG. 4 ).
该步骤2)具体为：The step 2) is specifically:
(a)对脸颊部区域的IR通道的像素点求平均值，IR通道内有N个数据；(a) average the pixels of the IR channel in the cheek region, and there are N data in the IR channel;
(b)对IR通道内数据依次进行归一化、白化处理以及独立成分分析；(b) Perform normalization, whitening processing and independent component analysis on the data in the IR channel in sequence;
其中，步骤(b)的处理中采用的归一化、白化处理以及独立成分分析，为本领域技术人员所公知的几种技术，本发明实施例对此不做赘述。Wherein, the normalization, whitening processing and independent component analysis used in the processing of step (b) are several technologies well known to those skilled in the art, which will not be described in detail in the embodiment of the present invention.
(c)对每个独立成分分析后的数据进行傅立叶变化，求出功率谱，得到功率谱最大幅值所对应的频率，即婴、幼儿的心率，另得到功率谱次最大幅值所对应的频率，即婴、幼儿的呼吸次数；(c) Perform Fourier transformation on the data after each independent component analysis, obtain the power spectrum, and obtain the frequency corresponding to the maximum amplitude of the power spectrum, that is, the heart rate of infants and young children, and obtain the frequency corresponding to the second maximum amplitude of the power spectrum Frequency, that is, the number of breaths of infants and young children;
(d)通过婴、幼儿的心率和呼吸次数计算婴、幼儿的心率变异值和呼吸次数变异值。(d) Calculating the heart rate variation value and breathing frequency variation value of infants and young children through the heart rate and breathing frequency of infants and young children.
其中，该步骤采用了本领域中的Lomb periodogram原理，具体实现时本发明实施例对此不作赘述。Wherein, this step adopts the principle of Lomb periodogram in the field, and this embodiment of the present invention will not repeat it in specific implementation.
205：将上述参数进行融合判断，得出婴、幼儿的状态检测结果。205: Perform fusion judgment on the above parameters to obtain the state detection result of the infant or toddler.
其中，步骤205的操作步骤和实施例1相同，本发明实施例对此不做赘述。Wherein, the operation steps in step 205 are the same as those in Embodiment 1, and are not described in detail in this embodiment of the present invention.
实际应用中，也可以将彩色摄像头和红外摄像头结合使用，处理过程类似实施例1和2，本发明实施例不再赘述。In practical applications, a color camera and an infrared camera can also be used in combination, and the processing process is similar to that of Embodiments 1 and 2, and will not be repeated in this embodiment of the present invention.
实施例3Example 3
一种基于机器视觉的婴幼儿监测装置，和实施例1中的方法所对应，参见图5，该监测装置包括：彩色摄像头1、编码器2、显示器3、嵌入式系统4、报警器5，A kind of infant monitoring device based on machine vision, corresponding to the method in embodiment 1, referring to Fig. 5, this monitoring device comprises: color camera 1, coder 2, display 3, embedded system 4, alarm 5,
处于婴、幼儿枕头部位上方的彩色摄像头1实时摄录婴、幼儿头脸部的影像并输入到编码器2，编码器2将彩色摄像头1的影像信息编码成数字图像信号送入嵌入式系统4，在嵌入式系统4中提取婴、幼儿的嘴巴张合的频度、头部位置及其维持的时间、换位的频度的视觉特征，以及脉率等被测试者的生理参数特征，对这些特征信息进行融合处理和判断报警；当婴、幼儿的睡眠状态发现异常时，报警器5发出报警声响，嵌入式系统4同时输出异常信号给家长或相关监护人员。The color camera 1 above the pillows of infants and young children records the images of the head and face of infants and young children in real time and inputs them to the encoder 2, and the encoder 2 encodes the image information of the color camera 1 into digital image signals and sends them to the embedded system 4 , in the embedded system 4, the visual characteristics of the frequency of opening and closing the mouth of infants and young children, the head position and the time it is maintained, the frequency of transposition, and the physiological parameter characteristics of the testees such as pulse rate are extracted. These feature information are fused and processed and judged and alarmed; when abnormalities are found in the sleep state of infants and young children, the alarm 5 sends out an alarm sound, and the embedded system 4 simultaneously outputs abnormal signals to parents or related guardians.
其中，该彩色摄像头1采用ANC酷睿HD1080P，嵌入式系统采用TI公司生产的OMAP，显示器3可以为LCD显示器等。Wherein, the color camera 1 adopts ANC Core HD1080P, the embedded system adopts OMAP produced by TI, and the display 3 can be an LCD display.
实施例4：Example 4:
一种基于机器视觉的婴幼儿监测装置，和实施例2中的方法所对应，参见图5，该监测装置包括：红外摄像头1、编码器2、显示器3、嵌入式系统4、报警器5，A kind of infant monitoring device based on machine vision, corresponding to the method in embodiment 2, referring to Fig. 5, this monitoring device comprises: infrared camera 1, coder 2, display 3, embedded system 4, alarm 5,
处于婴、幼儿枕头部位上方的红外摄像头1实时摄录婴、幼儿头脸部的影像并输入到编码器2，编码器2将红外摄像头1的影像信息编码成数字图像信号送入嵌入式系统4，在嵌入式系统4中提取婴、幼儿的嘴巴张合的频度、头部位置及其维持的时间、换位的频度的视觉特征，以及脉率等被测试者的生理参数特征，对这些特征信息进行融合处理和判断报警；当婴、幼儿的睡眠状态发现异常时，报警器5发出报警声响，嵌入式系统4同时输出异常信号给家长或相关监护人员。The infrared camera 1 above the pillow of infants and young children records the images of the head and face of infants and young children in real time and inputs them to the encoder 2, and the encoder 2 encodes the image information of the infrared camera 1 into digital image signals and sends them to the embedded system 4 , in the embedded system 4, the visual characteristics of the frequency of opening and closing the mouth of infants and young children, the head position and the time it is maintained, the frequency of transposition, and the physiological parameter characteristics of the testees such as pulse rate are extracted. These feature information are fused and processed and judged and alarmed; when abnormalities are found in the sleep state of infants and young children, the alarm 5 sends out an alarm sound, and the embedded system 4 simultaneously outputs abnormal signals to parents or related guardians.
其中，该红外摄像头1采用三星SCO-2080RP；嵌入式系统采用TI公司生产的OMAP，显示器3可以为LCD显示器等。Wherein, the infrared camera 1 adopts Samsung SCO-2080RP; the embedded system adopts OMAP produced by TI Company, and the display 3 can be an LCD display and the like.
本发明实施例对各器件的型号除做特殊说明的以外，其他器件的型号不做限制，只要能完成上述功能的器件均可。In the embodiments of the present invention, unless otherwise specified, the models of the devices are not limited, as long as they can complete the above functions.
本领域技术人员可以理解附图只是一个优选实施例的示意图，上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410310293.3A CN104077881A (en) | 2014-06-30 | 2014-06-30 | Infant monitoring method and device based on robot vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410310293.3A CN104077881A (en) | 2014-06-30 | 2014-06-30 | Infant monitoring method and device based on robot vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104077881A true CN104077881A (en) | 2014-10-01 |
Family
ID=51599114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410310293.3A Pending CN104077881A (en) | 2014-06-30 | 2014-06-30 | Infant monitoring method and device based on robot vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104077881A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109745028A (en) * | 2017-11-07 | 2019-05-14 | 财团法人资讯工业策进会 | System and method for identifying infant needs |
| CN109840493A (en) * | 2019-01-27 | 2019-06-04 | 武汉星巡智能科技有限公司 | Infantal sleeping condition detection method, device and computer readable storage medium |
| TWI667635B (en) * | 2018-01-09 | 2019-08-01 | 國立交通大學 | Monitoring system and monitoring method for infant |
| CN110279406A (en) * | 2019-05-06 | 2019-09-27 | 苏宁金融服务(上海)有限公司 | A kind of touchless pulse frequency measurement method and device based on camera |
| CN110547782A (en) * | 2019-05-06 | 2019-12-10 | 苏宁金融服务(上海)有限公司 | Pulse rate signal processing method and device for non-contact measurement |
| CN112183201A (en) * | 2020-08-25 | 2021-01-05 | 开放智能机器（上海）有限公司 | System and method for monitoring abnormity of baby during sleep |
| CN114898261A (en) * | 2022-05-07 | 2022-08-12 | 山东大学 | Sleep quality assessment method and system based on fusion of video and physiological data |
| CN115862115A (en) * | 2022-12-23 | 2023-03-28 | 宁波星巡智能科技有限公司 | Infant respiration detection area positioning method, device and equipment based on vision |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937605A (en) * | 2010-09-08 | 2011-01-05 | 无锡中星微电子有限公司 | Sleep monitoring system based on face detection |
| CN103544807A (en) * | 2013-11-07 | 2014-01-29 | 赵俊 | Infant sleeping nursing method and infant sleeping nursing system |
| CN103680059A (en) * | 2013-12-21 | 2014-03-26 | 南通芯迎设计服务有限公司 | Multifunctional wireless infant monitor |
| US20140135598A1 (en) * | 2011-08-05 | 2014-05-15 | Daimler Ag | Method and Device to Monitor at Least One Vehicle Passenger and Method to Control at Least One Assistance Device |
-
2014
- 2014-06-30 CN CN201410310293.3A patent/CN104077881A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937605A (en) * | 2010-09-08 | 2011-01-05 | 无锡中星微电子有限公司 | Sleep monitoring system based on face detection |
| US20140135598A1 (en) * | 2011-08-05 | 2014-05-15 | Daimler Ag | Method and Device to Monitor at Least One Vehicle Passenger and Method to Control at Least One Assistance Device |
| CN103544807A (en) * | 2013-11-07 | 2014-01-29 | 赵俊 | Infant sleeping nursing method and infant sleeping nursing system |
| CN103680059A (en) * | 2013-12-21 | 2014-03-26 | 南通芯迎设计服务有限公司 | Multifunctional wireless infant monitor |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109745028A (en) * | 2017-11-07 | 2019-05-14 | 财团法人资讯工业策进会 | System and method for identifying infant needs |
| TWI667635B (en) * | 2018-01-09 | 2019-08-01 | 國立交通大學 | Monitoring system and monitoring method for infant |
| CN109840493A (en) * | 2019-01-27 | 2019-06-04 | 武汉星巡智能科技有限公司 | Infantal sleeping condition detection method, device and computer readable storage medium |
| CN110279406A (en) * | 2019-05-06 | 2019-09-27 | 苏宁金融服务(上海)有限公司 | A kind of touchless pulse frequency measurement method and device based on camera |
| CN110547782A (en) * | 2019-05-06 | 2019-12-10 | 苏宁金融服务(上海)有限公司 | Pulse rate signal processing method and device for non-contact measurement |
| CN110279406B (en) * | 2019-05-06 | 2022-07-15 | 苏宁金融服务(上海)有限公司 | Non-contact pulse rate measuring method and device based on camera |
| CN112183201A (en) * | 2020-08-25 | 2021-01-05 | 开放智能机器（上海）有限公司 | System and method for monitoring abnormity of baby during sleep |
| CN114898261A (en) * | 2022-05-07 | 2022-08-12 | 山东大学 | Sleep quality assessment method and system based on fusion of video and physiological data |
| CN115862115A (en) * | 2022-12-23 | 2023-03-28 | 宁波星巡智能科技有限公司 | Infant respiration detection area positioning method, device and equipment based on vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104077881A (en) | Infant monitoring method and device based on robot vision | |
| CN108272448B (en) | A non-contact day and night monitoring method for infant physiological parameters | |
| CN104083160A (en) | Sleep state monitoring method and device based on machine vision | |
| TW201918222A (en) | Method and apparatus for high accuracy photoplethysmogram based atrial fibrillation detection using wearable device | |
| US9443289B2 (en) | Compensating for motion induced artifacts in a physiological signal extracted from multiple videos | |
| CN104639799B (en) | Image type heart rate activity detection device and method thereof | |
| US9336594B2 (en) | Cardiac pulse rate estimation from source video data | |
| US9436984B2 (en) | Compensating for motion induced artifacts in a physiological signal extracted from a single video | |
| CN104091408B (en) | Infant sleeping posture intelligent identification method and device based on thermal infrared imaging | |
| CN110021140A (en) | Monitoring system and monitoring method applied to infant | |
| CN114767115A (en) | Fatigue driving visual detection method and system based on facial multi-modal deep learning | |
| CN112806966B (en) | A non-interference sleep apnea early warning system | |
| CN108711452A (en) | The health state analysis method and system of view-based access control model | |
| KR101752873B1 (en) | Method and system for extracting heart information of time domain | |
| WO2016168979A1 (en) | Vital sign analysis method and system | |
| Kyrollos et al. | Noncontact neonatal respiration rate estimation using machine vision | |
| CN112022096A (en) | Sleep state monitoring method and device | |
| CN108771539A (en) | A kind of detection method and its device of the contactless heart rate based on camera shooting | |
| KR101996027B1 (en) | Method and system for extracting Heart Information of Frequency domain by using pupil size variation | |
| Rocque | Fully automated contactless respiration monitoring using a camera | |
| CN113143223A (en) | Edge artificial intelligence infant monitoring method | |
| KR20170004549A (en) | Method and system for extracting Heart Information of Frequency domain | |
| Zhu et al. | Privacy-protected contactless sleep parameters measurement using a defocused camera | |
| CN112924035A (en) | Body temperature and respiration rate extraction method based on thermal imaging sensor and application thereof | |
| CN116098596A (en) | A method and device for monitoring vital signs based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141001 |