CN114494321A - Infant sleep breath real-time monitoring method, device, equipment and storage medium - Google Patents
Infant sleep breath real-time monitoring method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114494321A CN114494321A CN202210135241.1A CN202210135241A CN114494321A CN 114494321 A CN114494321 A CN 114494321A CN 202210135241 A CN202210135241 A CN 202210135241A CN 114494321 A CN114494321 A CN 114494321A
- Authority
- CN
- China
- Prior art keywords
- infant
- image
- pixel
- target
- sleep
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007958 sleep Effects 0.000 title claims abstract description 97
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012544 monitoring process Methods 0.000 title claims abstract description 27
- 230000033001 locomotion Effects 0.000 claims abstract description 129
- 230000029058 respiratory gaseous exchange Effects 0.000 claims abstract description 58
- 230000000241 respiratory effect Effects 0.000 claims abstract description 35
- 238000001514 detection method Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000012216 screening Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 6
- 230000003321 amplification Effects 0.000 claims description 5
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 230000036391 respiratory frequency Effects 0.000 claims description 4
- 238000012806 monitoring device Methods 0.000 claims description 3
- 230000000474 nursing effect Effects 0.000 abstract description 21
- 230000008859 change Effects 0.000 abstract description 12
- 230000002159 abnormal effect Effects 0.000 abstract description 11
- 238000004891 communication Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000035565 breathing frequency Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000036387 respiratory rate Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000005057 finger movement Effects 0.000 description 1
- 230000008518 non respiratory effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/168—Segmentation; Edge detection involving transform domain methods
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/113—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb occurring during breathing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Abstract
The invention belongs to the technical field of video image data processing, solves the technical problems of inaccurate detection results or detection errors in respiratory motion monitoring of an infant when the infant sleeps by using the conventional AI nursing equipment, and provides a method, a device, equipment and a storage medium for monitoring infant sleep respiration in real time. The method comprises the steps of acquiring a sleep video image of an infant; performing data enhancement on image data corresponding to the respiratory motion of the infant in the image to establish a target Laplacian pyramid image representing a local area of the respiratory motion; and outputting real-time breathing state information of the infant in the sleeping process according to the target Laplace pyramid image and the infant sleeping video image. According to the method, the Laplace pyramid image is established after the low-frequency motion of the respiratory motion is subjected to data enhancement, so that the inaccuracy in detection caused by holes caused by uniform change by directly using the Laplace pyramid image can be avoided, the nursing reliability is improved, and the accident caused by the abnormal breathing of the infant is avoided.
Description
Technical Field
The invention relates to the technical field of video data processing, in particular to a method, a device, equipment and a storage medium for monitoring infant sleep breathing in real time.
Background
With the continuous deepening of the urbanization process, young parents can not better pay attention to the growth of the infants because the young parents cannot take care of the infants without a way of personally taking care of the infants, and in order to solve the problem, the young parents choose to adopt AI nursing equipment to carry out auxiliary nursing on the infants.
However, in the prior art, the AI nursing device is only limited to the function of a common camera, and cannot perform action recognition and danger early warning for a specific scene of nursing an infant, for example, respiratory state monitoring of the infant during daytime or night sleep, because respiration belongs to low-frequency motion, the monitoring difficulty is large, and even detection or false detection cannot be performed, so that the AI nursing device has a limited effect on assisting nursing young parents to nurse the infant in the nursing process, and cannot really assist the young parents to nurse the infant, and the experience effect of a user is reduced, and is also not beneficial to popularization of the AI nursing device.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for monitoring infant sleep breathing in real time, so as to solve the technical problem that the detection result is inaccurate or the detection is wrong in the existing AI nursing device for monitoring the breathing movement of an infant during sleep.
The technical scheme adopted by the invention is as follows:
the invention provides a method for monitoring sleep breathing of an infant in real time, which comprises the following steps:
s1: acquiring a sleep video image of an infant;
s2: according to the infant sleep video image, performing data enhancement on image data corresponding to respiratory motion of an infant during sleep, and accordingly establishing a target Laplace pyramid image representing a local area corresponding to the respiratory motion of the infant during sleep;
s3: and outputting real-time breathing state information of the infant in the sleeping process according to the target Laplace pyramid image and the infant sleeping video image.
Preferably, the S1 includes:
s11: acquiring an infrared video image of an infant during sleeping;
s12: carrying out gray level processing on the infrared video image to obtain a gray level image of each frame;
s13: and eliminating redundant motion with the motion amplitude larger than a preset value in each gray level image to obtain the infant sleep video image corresponding to weak motion with the motion amplitude within the preset value.
Preferably, the S2 includes:
s21: establishing Gaussian pyramids corresponding to the frames of images according to the frames of images in the infant sleep video images, wherein the grade of each Gaussian pyramid is preset;
s22: screening corresponding pixel points of the infant sleep video image according to pixel values of the pixel points in each level of the Gaussian pyramid to obtain target pixel points belonging to weak movement;
s23: establishing corresponding Laplacian pyramids of each layer according to the target pixel points;
s24: and obtaining the target Laplacian pyramid image according to the Laplacian pyramid of each layer.
Preferably, the S22 includes:
s221: acquiring pixel value thresholds of pixel points with similar pixel values;
s222: comparing the first pixel value of each first pixel point in each level of the Gaussian pyramid with the second pixel value of each second pixel point of a corresponding frame image in the infant sleep video image to obtain each pixel value difference value;
s223: and screening all pixel points in the infant video image according to the relation between the pixel value difference value and the pixel value threshold value to obtain all the target pixel points.
Preferably, the S23 includes:
s231: sequentially acquiring each target pixel point;
s232: according to each target pixel point, using a formula
Obtaining the Laplacian pyramid of each layer;
wherein, r (i) is the pixel value of the pixel i in the original image, g (n) is the pixel value of the pixel n in the gaussian pyramid image, q (i) is the pixel value of the pixel i in the laplacian pyramid, f is an adjustable parameter, and a is a constant.
Preferably, the S24 includes:
s241: obtaining Laplacian pyramids of all layers;
s242: using formulas
Amplifying and overlapping the Laplacian pyramids of all layers to obtain the target Laplacian pyramid image which moves weakly in the infant sleeping video;
wherein, I (x, t) is the brightness of the pixel at the time t, δ (t) is the displacement distance of the corresponding pixel at the time t compared with the previous time, α is the amplification factor, and f (x) is the pixel value of the pixel x.
Preferably, the S3 includes:
s31: superposing the target Laplace pyramid image and each frame image of the infant sleeping video to obtain a target video for performing data enhancement on image data of weak movement;
s32: obtaining the movement frequency of the weak movement corresponding to the target video according to the video duration of the target video and the movement times of the weak movement;
s33: and comparing the motion frequency with the respiratory frequency of the infant to obtain the real-time respiratory state.
The invention also provides a device for monitoring the sleep breathing of the infant in real time, which comprises:
the video acquisition module: the system is used for acquiring a sleep video image of an infant;
a data processing module: the image data corresponding to the breathing movement of the infant during sleeping is subjected to data enhancement according to the infant sleeping video image, so that a target Laplace pyramid image representing a local area corresponding to the breathing movement of the infant during sleeping is established;
a breath detection module: and the real-time breathing state information is output when the infant sleeps according to the target Laplace pyramid image and the infant sleeping video image.
The present invention also provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory that, when executed by the processor, implement the method of any of the above.
The invention also provides a medium having stored thereon computer program instructions which, when executed by a processor, implement the method of any of the above.
In conclusion, the beneficial effects of the invention are as follows:
according to the method, the device, the equipment and the storage medium for monitoring the infant sleep breath in real time, provided by the invention, the video image of the infant during sleep is collected, and the image data corresponding to the breath motion in each frame image of the infant sleep video is subjected to data enhancement to establish a local Laplace pyramid image, so that the breath motion can be obviously distinguished from other motions; and then overlapping the Laplacian pyramid image corresponding to each frame of image in the infant sleep video and each frame of image in the original infant sleep video, and determining the respiratory state information of the infant in the sleep process by the overlapped frame images.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and for those skilled in the art, without any creative effort, other drawings may be obtained according to the drawings, and these drawings are all within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of a method for monitoring sleep respiration of an infant in real time according to an embodiment 1 of the present invention;
fig. 2 is a schematic flow chart illustrating a process of acquiring a sleep video image of an infant in embodiment 1 of the present invention;
fig. 3 is a schematic flow chart of acquiring a target laplacian pyramid image in embodiment 1 of the present invention;
FIG. 4 is a schematic diagram of an input signal of image data according to embodiment 1 of the present invention;
fig. 5 is a schematic structural diagram of pixel value changes in a weakly moving image area according to embodiment 1 of the present invention;
fig. 6 is a schematic flow chart illustrating a process of acquiring a real-time sleep state of an infant in embodiment 1 of the present invention;
fig. 7 is a schematic structural diagram of a device for monitoring sleep breathing of an infant in real time according to embodiment 2 of the present invention;
fig. 8 is a schematic structural diagram of an electronic device in embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. In the description of the present invention, it is to be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. It is within the scope of the present invention that the embodiments and individual features of the embodiments may be combined with each other without conflict.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for monitoring sleep breathing of an infant in real time according to embodiment 1 of the present invention, where the method includes:
s1: acquiring a sleep video image of an infant;
specifically, the breathing state of the infant during sleeping is monitored in real time by using an infant AI nursing machine, and an infant sleeping video image, specifically an infrared video image during the infant sleeping is obtained in real time, wherein the infrared video image refers to each frame of image in a video stream of the infant during night sleeping and is shot under a 940nm infrared light supplement lamp; in a preferred embodiment, the infant AI nursing machine stores the infant sleep video in segments according to a preset time duration, for example, every 20 seconds or every 30 seconds is used as a video, that is, after the infant sleep video continuously acquired by the infant AI nursing machine reaches 20 seconds or 30 seconds, acquisition of a new video is started, and the acquired video is used for detecting the infant respiratory state.
In one embodiment, referring to fig. 2, the S1 includes:
s11: acquiring an infrared video image of an infant during sleeping;
s12: carrying out gray level processing on the infrared video image to obtain a gray level image of each frame;
s13: and eliminating redundant motion with the motion amplitude larger than a preset value in each gray level image to obtain the infant sleep video image corresponding to weak motion with the motion amplitude within the preset value.
Specifically, for converting the acquired infrared video image of the infant sleeping into a gray image, it needs to be explained that: the infrared video image is a video image collected by a camera and is not a human body thermal infrared image corresponding to a human body heat source; sending the gray level image into a dynamic detection module to eliminate redundant motion in the gray level image, wherein the redundant motion refers to large motion amplitude change and can obviously distinguish image change caused by non-respiratory motion, namely the distance difference between the imaging positions of the same pixel point in two adjacent frames of images is larger than a preset distance, specifically, the redundant motion and the weak motion can be distinguished by setting a motion amplitude preset value, the monitoring precision is different according to different motion amplitude preset values, and an infant sleeping video image corresponding to the weak motion when the infant sleeps is obtained; the dynamic detection module is a Vibe dynamic background modeling algorithm, and the Vibe dynamic background modeling algorithm specifically comprises the following steps: initializing a background model and detecting a foreground; the background model is initialized as: for a pixel point, combining the spatial distribution characteristics of adjacent pixel points with similar pixel values, randomly selecting the pixel value of a neighborhood point of the pixel point as a model sample value of the pixel point; the foreground detection is: the background model stores a sample set for each background point, and then each new pixel value is compared with the sample set to determine whether the background belongs to the background. And calculating the distance between the new pixel value and each sample value in the sample set, wherein if the distance is less than a threshold value, the number of approximate sample points is increased. If the number of the approximate sample points is larger than the threshold value, the new pixel point is considered as a background; the method is used for distinguishing redundant motion and weak motion in the video image, so that redundant motion elimination in the gray image is completed, and the infant sleeping video image with weak motion is obtained, wherein the weak motion comprises but is not limited to: infant breathing exercise, finger movement, hair waving caused by blowing, and the like.
S2: according to the infant sleep video image, performing data enhancement on image data corresponding to respiratory motion of an infant during sleep, and accordingly establishing a target Laplace pyramid image representing a local area corresponding to the respiratory motion of the infant during sleep;
specifically, data enhancement is performed on image data corresponding to respiratory motion in the infant sleep video, specifically, the position offset distance of each pixel point in an image area caused by the respiratory motion is amplified, so that laplacian pyramid images corresponding to the image area of the respiratory motion are established, the laplacian pyramid images are different in hierarchy and have different spatial frequencies and signal-to-noise ratios, the less the hierarchy, the lower the spatial frequency, and the established laplacian pyramid images are preferably 3-6 layers.
In one embodiment, referring to fig. 3, the S2 includes:
s21: establishing a Gaussian pyramid corresponding to each frame of image according to each frame of image in the infant sleep video image, wherein the grade of each Gaussian pyramid is preset;
s22: screening corresponding pixel points of the infant sleep video image according to pixel values of the pixel points in each level of the Gaussian pyramid to obtain target pixel points belonging to weak movement;
specifically, gaussian filtering is performed on an infrared image acquired by infant AI nursing equipment, so as to establish a gaussian pyramid image for each frame of infant sleep video image, wherein the stage number of each gaussian pyramid is preset, any pixel value g (n) in each stage of gaussian pyramid image is traversed, pixel points in the original image close to the pixel value g (n) are determined according to the pixel value g (n), the pixel points close to the pixel value g (n) are taken as target pixel points, and the pixel points can represent details near the pixel value g (n), namely, subtle changes caused by weak motions, and it needs to be explained that: the weak motion in the infrared video not only comprises low-frequency motion corresponding to respiratory motion, but also comprises infant hand micro motion, hair part fluttering and the like, and how to distinguish the weak motion belongs to the respiratory motion can be distinguished through motion frequency.
In one embodiment, the S22 includes:
s221: acquiring pixel value thresholds of pixel points with similar pixel values;
s222: comparing the first pixel values of the first pixel points in each level of the Gaussian pyramid with the second pixel values of the second pixel points of the corresponding frame image in the infant sleep video image to obtain pixel value difference values;
s223: and screening all pixel points in the infant video image according to the relation between the pixel value difference value and the pixel value threshold value to obtain all the target pixel points.
Specifically, the pixel values of the pixels in the gaussian pyramid image are recorded as a first pixel value, and the pixel values of the pixels in the original image are recorded as a second pixel value; the method comprises the steps of comparing each first pixel value with each second pixel value respectively to obtain a pixel value difference value of any one first pixel value and each second pixel value, then comparing each pixel value difference value with a pixel value threshold, marking the second pixel value corresponding to the pixel value difference value smaller than the pixel value threshold as a target pixel value, wherein a pixel point corresponding to the target pixel value is a target pixel point, screening out all target pixel points through the method, and the target pixel points do not directly perform Laplace change to establish a Laplace pyramid image. Please refer to fig. 4, fig. 4 is a structure of an input signal, the input signal is decomposed into a wall edge, a texture and a smooth component, the strong edge is each pixel point representing a salient pixel value of the whole outline of the image, the texture is each pixel point corresponding to each pixel point in the strong edge with a small difference in pixel value, that is, each pixel point representing the detail of the image area corresponding to each pixel point in the strong edge, the smooth component is used for performing low-frequency component enhancement and high-frequency component attenuation on the image to realize the smooth processing of the image, laplacian needs to be derived twice, if laplacian transform is directly performed, the processing result in the color uniform change or the gradual change area is 0, the processed image in these areas becomes a hole or disappears, which affects the accuracy of respiration monitoring.
S23: establishing corresponding Laplacian pyramids of each layer according to the target pixel points;
in one embodiment, the S23 includes:
s231: sequentially acquiring each target pixel point;
s232: according to each target pixel point, using a formula
Obtaining the Laplacian pyramid of each layer;
wherein, r (i) is the pixel value of the pixel i in the original image, g (n) is the pixel value of the pixel n in the gaussian pyramid image, q (i) is the pixel value of the pixel i in the laplacian pyramid image, f is the adjustable parameter, and a is a constant.
Specifically, after each target pixel point is obtained, a gaussian function is used
Adjusting each target pixel point to make the pixel value of each pixel point in the image with weak movement show non-uniform change or non-gradual changeChanging to avoid that holes or gradients disappear due to weak motion in the established Laplace pyramid image, please refer to FIG. 5, wherein f in FIG. 5 is respectively taken as-2, -1, 2 and 4, so that the pixel value transformation of the weak motion area is in non-gradual change or uniform change, thereby avoiding that the pixel value of the area is zero after the Laplace transformation is carried out, so that the area image disappears, a new image is generated after a layer of the Laplace pyramid image is completed, then the new image is converted into a corresponding layer of the Laplace pyramid image, and the operation is repeated, so that the target Laplace pyramid image is finally obtained; by the method, the phenomenon that holes appear in or disappear in the image corresponding to the respiratory motion caused by Laplace change can be avoided, and the integrity of the respiratory motion data is ensured.
S24: and obtaining the target Laplacian pyramid image according to the Laplacian pyramid of each layer.
In one embodiment, the S24 includes:
s241: obtaining Laplacian pyramids of all layers;
s242: using a formula
Amplifying and overlapping the Laplacian pyramids on each layer to obtain the target Laplacian pyramid image which moves weakly in the infant sleeping video;
wherein, I (x, t) is the brightness of the pixel at the time t, δ (t) is the displacement distance of the corresponding pixel at the time t compared with the previous time, α is the amplification factor, and f (x) is the pixel value of the pixel x.
Specifically, for each new generated local laplacian pyramid in each layer, according to the principle that the brightness of the same pixel point is unchanged at the same time, the following formula is provided:
I(x,t)＝f(x+δ(t))
then according to the principle of constant brightness, there are:
I(x,t)＝f(x+(1+α)δ(t))
considering that the sleep breathing motion is low frequency motion, low pass filtering is performed in the low frequency band, so the first order taylor series expansion is used as:
order:
and B (x, t) is an image brightness change signal corresponding to the corresponding breathing signal when the spatial point position in the video channel is x and the time is t.
After amplification and superposition, the following results are obtained:
overlapping the amplified Laplacian pyramid images with the original images to obtain pyramid images at different scales, and reconstructing the pyramid images to obtain the required amplified video; the respiratory movement is amplified, so that the identification speed and accuracy of the respiratory movement are increased, and the detection accuracy is ensured.
S3: and outputting real-time breathing state information of the infant in the sleeping process according to the target Laplace pyramid image and the infant sleeping video image.
In one embodiment, referring to fig. 6, the S3 includes:
s31: superposing the target Laplacian pyramid image with each frame image of the infant sleeping video to obtain a target video for performing data enhancement on image data with weak movement;
s32: obtaining the movement frequency of the weak movement corresponding to the target video according to the video duration of the target video and the movement times of the weak movement;
s33: and comparing the motion frequency with the respiratory frequency of the infant to obtain the real-time respiratory state.
Specifically, the weak motion comprises motion caused by breathing, micro-motion of hands of infants, flapping of hair parts and the like, and the wave band corresponding to the weak motion of the target video is researched to determine the motion frequency corresponding to the weak motion, specifically, the motion frequency is obtained according to the video duration and the motion frequency of the target video; comparing the movement frequency with the breathing frequency of the infant to determine whether the movement is caused by breathing; further, the breathing state can be determined according to the movement frequency, and the breathing state comprises normal breath, urge breath and no breath; the respiratory rate of normal adults is 16-20 times/minute and that of children is 30-40 times/minute. The respiratory rate of the newborn is 40-45 times/minute, and according to the data, the respiratory rate interval of the infant is set to be approximately within the interval range of 30-45 times/minute; according to the breathing frequency, the sleeping video time length of each section of the infant is set to be 20 seconds, namely, the wave band of the 20-second video is researched every time, the corresponding breathing frequency in the 20-second video is 10-15, if the frequency of weak movement is 10-15, the infant is considered to be weak movement caused by breathing, otherwise, the infant is considered to be weak movement caused by other actions, and in order to improve the detection accuracy, the current breathing state can be judged by comprehensively outputting the result after multiple detections are carried out; such as: judging by the output results of three continuous research wave bands, and if the corresponding frequency is not in the range of 10-15 times, determining that the frequency is not the frequency corresponding to the infant breathing; if within this range, the infant is considered to have a respiratory rate.
In one embodiment, the method for calculating the motion frequency of the target video comprises the following steps:
the first step is as follows: and marking the moving pixel points in each frame of image recorded in the Vibe dynamic background detection algorithm as white, and marking the non-moving points as black.
The second step is that: and calculating the frequency of weak movement according to the frequency of the white pixel points in the target video and the video duration of the target video.
Specifically, in the time interval between two weak movements, a relatively long stationary period exists between each pixel point, and at this time, no pixel point moves, and the period is recorded as a non-moving period. Therefore, counting the number of times that the same white point region appears in the video duration, which is denoted as N, the method for calculating the motion frequency of the weak motion is as follows: the movement frequency is 60 × N/T times/min.
In an embodiment, after the S3, the method further includes:
s4: acquiring an infant sleeping video according to preset video duration;
s5: counting the abnormal conditions of the real-time breathing state to obtain abnormal breathing times;
s6: and if the abnormal breathing frequency is greater than a preset value, sending corresponding abnormal breathing information to the terminal.
Specifically, each infant sleep video is collected according to the preset video duration, then call detection is carried out, so that the respiratory information corresponding to each infant sleep video is obtained, if the continuous abnormal breathing frequency exceeds the preset value, the abnormal breathing information is sent to the terminal, the abnormal breathing information comprises abnormal categories and processed target videos, a user can conveniently check the abnormal breathing condition and know what reason causes abnormal breathing, the infant sleep state is pertinently adjusted, and the auxiliary nursing effect of the infant AI nursing device is achieved.
By adopting the real-time monitoring method for the infant sleep breathing, the video image of the infant during sleep is collected, and the image data corresponding to the breathing motion in each frame image of the infant sleep video is subjected to data enhancement to establish a local Laplacian pyramid image, so that the breathing motion can be obviously distinguished from other motions; and then overlapping the Laplacian pyramid image corresponding to each frame of image in the infant sleep video and each frame of image in the original infant sleep video, and determining the respiratory state information of the infant in the sleep process by the overlapped frame images.
Example 2
The invention also provides a device for monitoring infant sleep breathing in real time, please refer to fig. 7, which includes:
the video acquisition module: the system is used for acquiring a sleep video image of an infant;
a data processing module: the image data corresponding to the breathing movement of the infant during sleeping is subjected to data enhancement according to the infant sleeping video image, so that a target Laplace pyramid image representing a local area corresponding to the breathing movement of the infant during sleeping is established;
a breath detection module: and the real-time breathing state information is output when the infant sleeps according to the target Laplace pyramid image and the infant sleeping video image.
By adopting the infant sleep respiration real-time monitoring device, video images of an infant during sleep are collected, the video images of the infant during sleep are collected, and image data corresponding to respiratory motion in each frame of image of the infant sleep video are subjected to data enhancement to establish a local Laplace pyramid image, so that the respiratory motion can be obviously distinguished from other motions; and then overlapping the Laplacian pyramid image corresponding to each frame of image in the infant sleep video and each frame of image in the original infant sleep video, and determining the respiratory state information of the infant in the sleep process by the overlapped frame images.
In one embodiment, the video capture module comprises:
a video acquisition unit: acquiring an infrared video image of an infant during sleeping;
a gradation processing unit: carrying out gray level processing on the infrared video image to obtain a gray level image of each frame;
a background removal unit: and eliminating redundant motion with the motion amplitude larger than a preset value in each gray level image to obtain the infant sleep video image corresponding to weak motion with the motion amplitude within the preset value.
In one embodiment, the data processing module comprises:
an image format unit: establishing a Gaussian pyramid corresponding to each frame of image according to each frame of image in the infant sleep video image, wherein the grade of each Gaussian pyramid is preset;
a target screening unit: screening corresponding pixel points of the infant sleep video image according to pixel values of the pixel points in each level of the Gaussian pyramid to obtain target pixel points belonging to weak movement;
an object reconstruction unit: establishing corresponding Laplacian pyramids of each layer according to the target pixel points;
an image conversion unit: and obtaining the target Laplacian pyramid image according to the Laplacian pyramid of each layer.
In one embodiment, the target screening unit includes:
pixel threshold unit: acquiring pixel value thresholds of pixel points with similar pixel values;
pixel difference value unit: comparing the first pixel values of the first pixel points in each level of the Gaussian pyramid with the second pixel values of the second pixel points of the corresponding frame image in the infant sleep video image to obtain pixel value difference values;
a target pixel unit: and screening all pixel points in the infant video according to the relation between the pixel value difference value and the pixel value threshold value to obtain all the target pixel points.
In one embodiment, the target reconstruction unit includes:
a target acquisition unit: sequentially acquiring each target pixel point;
a pixel calculation unit: according to each target pixel point, using a formula
Obtaining the Laplacian pyramid of each layer;
wherein, r (i) is a pixel value of a pixel point i in the original image, g (n) is a pixel value of a pixel point n in the gaussian pyramid image, q (i) is a pixel value of a pixel point i in the laplacian pyramid, f is an adjustable parameter, and a is a constant.
In one embodiment, the image conversion unit includes:
an image acquisition unit: obtaining Laplacian pyramids of all layers;
an image superimposing unit: using formulas
Amplifying and overlapping the Laplacian pyramids on each layer to obtain the target Laplacian pyramid image which moves weakly in the infant sleeping video;
wherein, I (x, t) is the brightness of the pixel at the time t, δ (t) is the displacement distance of the corresponding pixel at the time t compared with the previous time, α is the amplification factor, and f (x) is the pixel value of the pixel x.
In one embodiment, the breath detection module comprises:
a data enhancement unit: superposing the target Laplace pyramid image and each frame image of the infant sleeping video to obtain a target video for performing data enhancement on weak movement;
a motion frequency unit: obtaining the movement frequency of the weak movement corresponding to the target video according to the video duration of the target video and the movement times of the weak movement;
a respiratory state unit: and comparing the motion frequency with the respiratory frequency of the infant to obtain the real-time respiratory state.
By adopting the infant sleep respiration real-time monitoring device, video images of an infant during sleep are collected, the video images of the infant during sleep are collected, and image data corresponding to respiratory motion in each frame of image of the infant sleep video are subjected to data enhancement to establish a local Laplace pyramid image, so that the respiratory motion can be obviously distinguished from other motions; and then overlapping the Laplacian pyramid image corresponding to each frame of image in the infant sleep video and each frame of image in the original infant sleep video, and determining the respiratory state information of the infant in the sleep process by the overlapped frame images.
Example 3
The present invention provides an electronic device and medium, as shown in fig. 8, comprising at least one processor, at least one memory, and computer program instructions stored in the memory.
Specifically, the processor may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present invention, and the electronic device includes at least one of the following: the wearing equipment that camera, mobile device that has the camera, have the camera.
The memory may include mass storage for data or instructions. By way of example, and not limitation, memory may include a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or Universal Serial Bus (USB) Drive or a combination of two or more of these. The memory may include removable or non-removable (or fixed) media, where appropriate. The memory may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory is non-volatile solid-state memory. In a particular embodiment, the memory includes Read Only Memory (ROM). Where appropriate, the ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory or a combination of two or more of these.
The processor reads and executes the computer program instructions stored in the memory to realize the method for automatically capturing the baby highlight video highlights in any one of the above embodiment modes.
In one example, the electronic device may also include a communication interface and a bus. The processor, the memory and the communication interface are connected through a bus and complete mutual communication.
The communication interface is mainly used for realizing communication among modules, devices, units and/or equipment in the embodiment of the invention.
A bus comprises hardware, software, or both that couple components of an electronic device to one another. By way of example, and not limitation, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industrial Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus or a combination of two or more of these. A bus may include one or more buses, where appropriate. Although specific buses have been described and shown in the embodiments of the invention, any suitable buses or interconnects are contemplated by the invention.
In summary, embodiments of the present invention provide a method, an apparatus, a device, and a medium for monitoring infant sleep breathing in real time, where a preset snapshot rule is set in an AI nursing device, and a snapshot image is displayed on an App display interface according to a first display rule, and a user may add mark information to the snapshot image and display the snapshot image with the mark information on the App display interface according to a second display rule; then, analyzing the operation information of the image points displayed on the App display interface, and determining the display interest degree of the user on the target images; and then extracting a plurality of characteristic information of the target image according to the display interest to optimize a preset snapshot rule, wherein the plurality of characteristic information are associated characteristic vectors used when the AI nursing device carries out snapshot rule training, and the plurality of characteristic information include but are not limited to bone key points, facial shielding information, facial imaging area and the like, so that the target snapshot rule is obtained, the target snapshot rule is used as a snapshot condition for instantly taking a snapshot of the wonderful baby when the AI nursing device carries out nursing on the baby in the next time period, the snapshot rule is adjusted by combining feedback information of the user on the snapshot image, the individual requirements of the user can be met, even if the aesthetic style of the user changes, the favorite image of the user can be snapshot according to the feedback information of the user, the stickiness of the user can be kept in real time, and the user experience effect is improved.
It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for monitoring infant sleep breathing in real time, the method comprising:
s1: acquiring a sleep video image of an infant;
s2: according to the infant sleep video image, performing data enhancement on image data corresponding to respiratory motion of an infant during sleep, and accordingly establishing a target Laplace pyramid image representing a local area corresponding to the respiratory motion of the infant during sleep;
s3: and outputting real-time breathing state information of the infant in the sleeping process according to the target Laplace pyramid image and the infant sleeping video image.
2. The method for monitoring sleep breathing of an infant in real time as claimed in claim 1, wherein the S1 includes:
s11: acquiring an infrared video image of an infant during sleeping;
s12: carrying out gray level processing on the infrared video image to obtain a gray level image of each frame;
s13: and eliminating redundant motion with the motion amplitude larger than a preset value in each gray level image to obtain the infant sleep video image corresponding to weak motion with the motion amplitude within the preset value.
3. The method for monitoring sleep breathing of an infant in real time as claimed in claim 2, wherein the S2 includes:
s21: establishing a Gaussian pyramid corresponding to each frame of image according to each frame of image in the infant sleep video image, wherein the grade of each Gaussian pyramid is preset;
s22: screening corresponding pixel points of the infant sleep video image according to pixel values of the pixel points in each level of the Gaussian pyramid to obtain target pixel points belonging to weak movement;
s23: establishing corresponding Laplacian pyramids of each layer according to the target pixel points;
s24: and obtaining the target Laplacian pyramid image according to the Laplacian pyramid of each layer.
4. The method for monitoring sleep breathing of an infant in real time as claimed in claim 3, wherein the S22 includes:
s221: acquiring pixel value thresholds of pixel points with similar pixel values;
s222: comparing the first pixel values of the first pixel points in each level of the Gaussian pyramid with the second pixel values of the second pixel points of the corresponding frame image in the infant sleep video image to obtain pixel value difference values;
s223: and screening all pixel points in the infant video image according to the relation between the pixel value difference value and the pixel value threshold value to obtain all the target pixel points.
5. The method for monitoring sleep breathing of an infant in real time as claimed in claim 3, wherein the S23 includes:
s231: sequentially acquiring each target pixel point;
s232: according to each target pixel point, using a formula
Obtaining the Laplacian pyramid of each layer;
wherein, r (i) is the pixel value of the pixel i in the original image, g (n) is the pixel value of the pixel n in the gaussian pyramid image, q (i) is the pixel value of the pixel i in the laplacian pyramid, f is an adjustable parameter, and a is a constant.
6. The method for monitoring sleep breathing of an infant in real time as claimed in claim 3, wherein the S24 includes:
s241: obtaining Laplacian pyramids of all layers;
s242: using formulas
Amplifying and overlapping the Laplacian pyramids of all layers to obtain the target Laplacian pyramid image which moves weakly in the infant sleep video image;
wherein, I (x, t) is the brightness of the pixel at the time t, δ (t) is the displacement distance of the corresponding pixel at the time t compared with the previous time, α is the amplification factor, and f (x) is the pixel value of the pixel x.
7. The method for monitoring sleep breathing of an infant in real time according to any one of claims 3 to 6, wherein the S3 includes:
s31: superposing the target Laplace pyramid image and each frame image of the infant sleep video image to obtain a target video for performing data enhancement on image data with weak movement;
s32: obtaining the movement frequency of the weak movement corresponding to the target video according to the video duration of the target video and the movement times of the weak movement;
s33: and comparing the motion frequency with the respiratory frequency of the infant to obtain the real-time respiratory state information.
8. The utility model provides an infant sleep breathing real-time monitoring device which characterized in that includes:
the video acquisition module: the system is used for acquiring a sleep video image of an infant;
a data processing module: the image data corresponding to the breathing movement of the infant during sleeping is subjected to data enhancement according to the infant sleeping video image, so that a target Laplace pyramid image representing a local area corresponding to the breathing movement of the infant during sleeping is established;
a breath detection module: and the real-time breathing state information is output when the infant sleeps according to the target Laplace pyramid image and the infant sleeping video image.
9. An electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory that, when executed by the processor, implement the method of any of claims 1-7.
10. A medium having stored thereon computer program instructions, which, when executed by a processor, implement the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135241.1A CN114494321A (en) | 2022-02-14 | 2022-02-14 | Infant sleep breath real-time monitoring method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210135241.1A CN114494321A (en) | 2022-02-14 | 2022-02-14 | Infant sleep breath real-time monitoring method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494321A true CN114494321A (en) | 2022-05-13 |
Family
ID=81480059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210135241.1A Pending CN114494321A (en) | 2022-02-14 | 2022-02-14 | Infant sleep breath real-time monitoring method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494321A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116386278A (en) * | 2023-03-20 | 2023-07-04 | 武汉星巡智能科技有限公司 | Intelligent recognition reminding method, device and equipment based on infant sleeping posture |
-
2022
- 2022-02-14 CN CN202210135241.1A patent/CN114494321A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116386278A (en) * | 2023-03-20 | 2023-07-04 | 武汉星巡智能科技有限公司 | Intelligent recognition reminding method, device and equipment based on infant sleeping posture |
| CN116386278B (en) * | 2023-03-20 | 2023-11-10 | 武汉星巡智能科技有限公司 | Intelligent recognition reminding method, device and equipment based on infant sleeping posture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Al-Naji et al. | Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle | |
| CN108701357B (en) | Device, system and method for skin detection | |
| CN105266787B (en) | A kind of contactless heart rate detection method and system | |
| US8805019B2 (en) | Processing images of at least one living being | |
| CN113194359B (en) | Method, device, equipment and medium for automatically grabbing baby wonderful video highlights | |
| Park et al. | Remote pulse rate measurement from near-infrared videos | |
| CN114494321A (en) | Infant sleep breath real-time monitoring method, device, equipment and storage medium | |
| CN113516095A (en) | Infant sleep monitoring method, device, equipment and medium | |
| KR101741904B1 (en) | Image-processing-based heartrate measuring method and, newborn baby image providing system | |
| Dosso et al. | Neonatal face tracking for non-contact continuous patient monitoring | |
| CN113038272B (en) | Method, device and equipment for automatically editing baby video and storage medium | |
| Tan et al. | Lightweight video-based respiration rate detection algorithm: An application case on intensive care | |
| Lin et al. | Step count and pulse rate detection based on the contactless image measurement method | |
| Andreotti et al. | Improved heart rate detection for camera-based photoplethysmography by means of Kalman filtering | |
| CN115862115B (en) | Infant respiration detection area positioning method, device and equipment based on vision | |
| CN112244796A (en) | Method for intelligently detecting human body physiological indexes and nursing equipment | |
| CN113693573B (en) | Video-based non-contact multi-physiological-parameter monitoring system and method | |
| Tran et al. | Intelligent non-invasive vital signs estimation from image analysis | |
| CN116386278B (en) | Intelligent recognition reminding method, device and equipment based on infant sleeping posture | |
| CN112907454B (en) | Method, device, computer equipment and storage medium for acquiring image | |
| CN113378762A (en) | Sitting posture intelligent monitoring method, device, equipment and storage medium | |
| Ibrahim et al. | Analysis of minimum face video duration and the effect of video compression to image-based non-contact heart rate monitoring system | |
| CN113974933A (en) | Infant quilt kicking detection method and device, electronic equipment and medium | |
| KR102381204B1 (en) | Apparatus and method for monitoring breathing using thermal image | |
| CN113256648B (en) | Self-adaptive multi-scale respiration monitoring method based on camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |