WO2016151966A1

WO2016151966A1 - Infant monitoring device, infant monitoring method, and infant monitoring program

Info

Publication number: WO2016151966A1
Application number: PCT/JP2015/085599
Authority: WO
Inventors: 靖和田中; 安川　徹
Original assignee: ノーリツプレシジョン株式会社
Priority date: 2015-03-23
Filing date: 2015-12-21
Publication date: 2016-09-29
Also published as: JPWO2016151966A1

Abstract

Provided is a system that can appropriately keep watch over an infant. The infant monitoring device according to one aspect of the present invention is provided with: an image acquisition unit that acquires photographed images that capture an infant sleeping and that include depth data that indicates the depth of each pixel in the photographed images; a vital information acquisition unit that acquires vital information that includes the pulse rate and the breathing state of the infant; a sleeping state analysis unit that extracts a person region that is the region in which the infant appears in the photographed images, references the depth of the pixels included in the extracted person region, and analyzes the body movements and the sleeping position of the infant; a warning-sign determination unit that determines, on the basis of the pulse rate, the breathing state, the body movements, and the sleeping position, whether the infant shows a warning sign of sudden infant death syndrome; and a notification unit that, when the determination results have determined that the infant shows a warning sign of sudden infant death syndrome, performs warning-sign detection notification that is for making it known that a warning sign of sudden infant death syndrome is present.

Description

Infant monitoring apparatus, infant monitoring method, and infant monitoring program

The present invention relates to an infant monitoring apparatus, an infant monitoring method, and an infant monitoring program.

In recent years, countermeasures against sudden infant death syndrome (sudden infant death syndrome, hereinafter also referred to as “SIDS”) have been activated, and various systems for detecting signs of SIDS in infants have been developed. ing.

For example, Patent Document 1 proposes a motion detector that detects the respiratory motion of an infant in order to detect signs of SIDS in the infant. According to this motion detector, it is possible to detect that the infant is in an apneic state, thereby preventing the infant from suddenly dying while sleeping.

Also, for example, Patent Document 2 proposes a system for watching an infant who is sleeping by photographing the infant with a camera and analyzing image data obtained thereby. In this system, the obtained image data is analyzed to detect a high-risk movement of the infant related to SIDS. Thereby, the sign of SIDS of the infant can be detected.

JP 05-176898 A Special table 2007-534032 gazette

However, in the motion detector exemplified in Patent Document 1, since the respiratory state of the infant is analyzed based on the force applied to the surface of the piezoelectric crystal converter, the motion detector is not sufficiently in contact with the infant. Was unable to accurately measure the infant's respiratory status. Therefore, such a motion detector may not be able to properly detect the signs of SIDS in infants. In addition, with such a motion detector, the sleeping posture of the infant cannot be specified, and it has been difficult to detect prone sleep, which is a risk factor for SIDS.

In the system exemplified in Patent Document 2, a two-dimensional image is used to detect the movement of the infant. Therefore, depending on the viewing direction (viewpoint) of the camera with respect to the infant, the movement of the infant may not be sufficiently reflected in the two-dimensional image. In this case, the movement of the infant cannot be detected. Therefore, even with such a system, there is a possibility that a sign of SIDS cannot be properly detected.

In one aspect, the present invention has been made in consideration of such points, and an object thereof is to provide a system that can appropriately detect a sign of SIDS appearing in an infant.

The present invention adopts the following configuration in order to solve the above-described problems.

That is, the infant monitoring apparatus according to one aspect of the present invention is an image acquisition that captures a captured image that is a captured image of a sleeping infant and that includes depth data indicating the depth of each pixel in the captured image. A vital information acquisition unit that acquires vital information including a pulse rate and a breathing state of the infant, and a person area in which the infant appears in the acquired captured image, and each of the extracted person areas includes With reference to the depth of the pixel, a sleeping state analyzing unit that analyzes the body movement and sleeping posture of the infant, the pulse rate and the respiratory state indicated by the acquired vital information, and the analyzed body movement and the Based on the sleeping posture, a sign determination unit that determines whether or not the infant has a sign of sudden infant death syndrome, and as a result of the determination, there is a sign of sudden infant death syndrome. If it is determined that the serial infants includes a notification unit that performs signs detection notification for notifying that the signs made to the sudden infant death syndrome in the infant, the.

According to the above configuration, the captured image acquired for analyzing the body movement and sleeping posture of the infant includes depth data indicating the depth of each pixel. The depth of each pixel indicates the depth from the photographing apparatus to the subject. More specifically, the depth of the subject is acquired with respect to the surface of the subject. That is, if the depth data is used, the position of the subject surface in the real space can be specified. Therefore, by using this depth data, it is possible to analyze the physical movement of the infant in the real space (three-dimensional space) regardless of the visual field direction (viewpoint) of the photographing apparatus with respect to the infant.

Moreover, according to the said structure, not only the infant's physical movement (body movement and sleeping posture) but also whether the infant has signs of becoming SIDS based on vital information including the pulse rate and breathing state. judge. Therefore, it can be determined from a multifaceted viewpoint whether or not the infant has a sign of SIDS.

Therefore, for these reasons, according to this configuration, it is possible to provide a system that can appropriately detect the signs of SIDS appearing in infants. The sleeping posture is a posture when the infant is sleeping, such as a supine posture (a supine position), a lateral posture (a lateral position), a prone posture (a prone position). The information indicating the respiratory state included in the vital information may be appropriately selected according to the embodiment, and is, for example, the respiratory rate, the oxygen saturation (SpO ₂ ) of arterial blood, or the like. The respiration rate indicates the number of respirations per unit time. The pulse rate indicates the number of pulses per unit time. The time for measuring the number of breaths and the number of pulses may be appropriately set according to the embodiment.

As another form of the infant monitoring apparatus according to the above aspect, the infant has a sign of sudden infant death syndrome when the sign determination unit detects that the infant's sleeping posture is prone May be determined. One of the risk factors for SIDS is lying down. When the infant monitoring apparatus according to the configuration detects that the sleeping posture of the infant is prone, the infant monitoring apparatus determines that the infant has a sign of sudden infant death syndrome. Therefore, according to the said structure, the prone sleep which is one of the risk factors of SIDS can be detected, and it can prevent appropriately the infant from becoming SIDS.

Further, as another form of the infant monitoring apparatus according to the above aspect, the sign determination unit, the pulse rate indicated by the acquired vital information is higher than a normal pulse rate of the infant by a predetermined value or more, The respiratory state indicated by the acquired vital information is shallower than a normal respiratory state of the infant by a predetermined amount or more, and the analyzed amount of body movement is more than the normal amount of body movement of the infant If it is detected that there is less than a predetermined value, it may be determined that the infant has a sign of sudden infant death syndrome.

In a study of SIDS, it is pointed out that if an infant is overheated while sleeping, the infant may develop SIDS. In a sleeping infant's body, as the sleep goes deeper, muscle tone decreases and the sympathetic response decreases. As a result, a decrease in heart rate and dilation of peripheral blood vessels occur, increasing the amount of heat released from the infant's body.

At this time, if the infant is maintained at an appropriate temperature, the body temperature decreases due to heat radiation, so the infant who felt cold enhances the response of the sympathetic nervous system, contracts the peripheral blood vessels, and releases heat. Decrease. At the same time, the infant's heart rate increases, muscle tension increases, and sleep decreases. Thereafter, if no stimulating stimulus is applied to the infant, the infant's body temperature rises and sleep becomes deeper again. Sleeping infants adjust body temperature while repeating such expansion and contraction of peripheral blood vessels.

On the other hand, when the infant is overheated, the infant's body temperature does not decrease even if the amount of heat released from the infant's body increases. Therefore, the infant's sleep remains deep, the infant's response to the sympathetic nervous system continues to decrease, the muscle tone also decreases, and the state where the response to the stimulus is reduced continues. Then, the infant's breathing becomes shallow (or becomes apnea), and hypoxia progresses, eventually the infant can become SIDS.

That is, if an infant's breathing becomes excessively shallow, the pulse rate increases excessively, and the amount of body movement decreases excessively, the infant develops hypoxia and develops SIDS there is a possibility. On the other hand, the infant monitoring apparatus according to the configuration is such that the acquired pulse rate is higher than the normal pulse rate by a predetermined value or more, the acquired respiratory state is shallower than the normal respiratory state by a predetermined amount, and When it is detected that the analyzed body movement amount is smaller than the normal body movement amount by a predetermined value or more, it is determined that the infant has a sign of sudden infant death syndrome. Therefore, according to the said structure, it can detect that the sleeping infant falls into the above hypoxic states, and can prevent appropriately the said infant becoming SIDS.

As another form of the infant monitoring apparatus according to the above aspect, the vital information acquisition unit is configured to be able to measure a pulse rate, and acquires the infant's pulse rate from a measuring device attached to the infant. Also good. According to the said structure, the vital information utilized in determining the sign of SIDS can be acquired appropriately.

Further, as another form of the infant monitoring apparatus according to the above aspect, the measuring device may be configured to be able to further measure the oxygen saturation of arterial blood, and the vital information acquisition unit is information indicating the respiratory state The oxygen saturation of the infant's arterial blood may be acquired from the measuring device. The sign determination unit compares the acquired oxygen saturation of the arterial blood with a reference value of the arterial oxygen saturation at the normal time of the infant, and the acquired oxygen saturation of the arterial blood is determined by the infant. When it is determined that the oxygen saturation level of arterial blood in the normal time is lower than the reference value, it is detected that the respiratory state indicated by the acquired vital information is shallower than a predetermined level than the normal respiratory state of the infant May be. According to the said structure, the vital information utilized in determining the sign of SIDS can be acquired appropriately.

As another form of the infant monitoring apparatus according to the above aspect, the measuring device may be configured to be able to further measure a respiratory rate, and the vital information acquisition unit may use the infant as the information indicating the respiratory state. May be obtained from the measuring device. Then, the sign determination unit compares the acquired respiration rate with the normal breathing rate of the infant and determines that the acquired respiration rate is lower than a predetermined value than the normal respiration rate of the infant In this case, it may be detected that the respiratory state indicated by the acquired vital information is shallower than a predetermined level than the normal respiratory state of the infant. According to the said structure, the vital information utilized in determining the sign of SIDS can be acquired appropriately.

As another form of the infant monitoring apparatus according to the above aspect, the vital information acquisition unit refers to the depth of each pixel included in the person area in the captured image, and the number of changes in the infant's rib cage By measuring the respiratory rate of the infant as information indicating the respiratory state. Then, the sign determination unit compares the acquired respiration rate with the normal breathing rate of the infant and determines that the acquired respiration rate is lower than a predetermined value than the normal respiration rate of the infant In this case, it may be detected that the respiratory state indicated by the acquired vital information is shallower than a predetermined level than the normal respiratory state of the infant. According to the said structure, the information which shows the infant's respiratory condition can be acquired noninvasively. Therefore, it is possible to reduce the number of devices attached to the infant and to reduce the possibility of an accident such as the wiring of such devices being entangled in the infant's neck.

In addition, as another form of the infant monitoring apparatus according to each of the above forms, an information processing system that realizes each of the above configurations, an information processing method, or a program may be used. It may be a storage medium that can be read by a computer, other devices, machines, or the like in which such a program is recorded. Here, the computer-readable recording medium is a medium that stores information such as programs by electrical, magnetic, optical, mechanical, or chemical action. The information processing system may be realized by one or a plurality of information processing devices.

For example, in the infant monitoring method according to one aspect of the present invention, a computer captures a captured image obtained by capturing a sleeping infant and includes depth data indicating the depth of each pixel in the captured image. A step of acquiring vital information including a pulse rate and a breathing state of the infant, a step of extracting a person region in which the infant is captured in the acquired captured image, and each of the steps included in the extracted person region The step of analyzing the body movement and sleeping posture of the infant with reference to the pixel depth, the pulse rate and the breathing state indicated by the acquired vital information, and the analyzed body movement and sleeping posture. And determining whether or not the infant has a sign of sudden infant death syndrome, and the result of the determination is sudden infant death syndrome. If the weather is determined to be in the infant is an information processing method indications made to the sudden infant death syndrome executes, and performing signs detection notification for notifying that it is in the infant.

Further, for example, an infant monitoring program according to one aspect of the present invention is a captured image obtained by capturing a sleeping infant on a computer, and includes a captured image including depth data indicating the depth of each pixel in the captured image. , Acquiring vital information including the infant's pulse rate and breathing state, extracting a person area in which the infant appears in the acquired captured image, and including the extracted person area Analyzing the body movement and sleeping posture of the infant with reference to the depth of each pixel, the pulse rate and the respiratory state indicated by the acquired vital information, and the analyzed body movement and the sleeping state Determining whether the infant has a sign of sudden infant death syndrome based on the posture; and, as a result of the determination, sudden infant death A step of performing a sign detection notification for notifying that the infant has a sign of sudden infant death syndrome when it is determined that there is a sign of a symptom group in the infant. It is.

According to the present invention, it is possible to provide a system capable of appropriately detecting a sign of SIDS.

FIG. 1 schematically illustrates a scene where the present invention is applied. FIG. 2 illustrates a hardware configuration of the infant monitoring apparatus according to the embodiment. FIG. 3 illustrates the relationship between the depth acquired by the camera according to the embodiment and the subject. FIG. 4 illustrates a functional configuration of the infant monitoring apparatus according to the embodiment. FIG. 5 exemplifies a processing procedure related to watching an infant by the infant monitoring apparatus according to the embodiment. FIG. 6 illustrates a captured image acquired by the camera according to the embodiment. FIG. 7 illustrates the coordinate relationship in the captured image according to the embodiment. FIG. 8 illustrates the positional relationship between an arbitrary point (pixel) of the captured image and the camera in the real space according to the embodiment.

Hereinafter, an embodiment according to one aspect of the present invention (hereinafter also referred to as “this embodiment”) will be described with reference to the drawings. However, this embodiment described below is only an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. That is, in implementing the present invention, a specific configuration according to the embodiment may be adopted as appropriate. Although data appearing in the present embodiment is described in a natural language, more specifically, it is specified by a pseudo language, a command, a parameter, a machine language, or the like that can be recognized by a computer.

§1 Application scene First, the scene where the present invention is applied will be described with reference to FIG. FIG. 1 shows an example of a scene where the infant monitoring apparatus 1 according to the present embodiment is used. The infant monitoring apparatus 1 according to the present embodiment photographs an infant with the camera 2 and analyzes the photographed image 3 obtained thereby to monitor the infant's state in the photographed image 3 and to watch over the infant. An information processing apparatus to perform. Therefore, the infant monitoring apparatus 1 according to the present embodiment can be widely used in a scene where an infant to be watched over is watched over.

Specifically, first, the infant monitoring apparatus 1 according to the present embodiment acquires a photographed image 3 obtained by photographing a sleeping infant from the camera 2. In the scene illustrated in FIG. 1, the infant (person to be watched over) is sleeping on the bed within the shooting range of the camera 2. The camera 2 is installed to photograph such an infant. However, the infant need not always go to bed, and may be temporarily awakened.

The camera 2 is configured to be able to acquire the depth corresponding to each pixel in the captured image 3. In the present embodiment, the camera 2 includes a depth sensor (a depth sensor 21 described later) that measures the depth of the subject so that the depth of each pixel can be acquired. The infant monitoring apparatus 1 according to the present embodiment is connected to such a camera 2 and acquires a photographed image 3 obtained by photographing an infant to be monitored.

The acquired captured image 3 includes depth data indicating the depth obtained for each pixel, as illustrated in FIG. The captured image 3 only needs to include data indicating the depth of the subject within the imaging range, and the data format can be appropriately selected according to the embodiment. For example, the captured image 3 may be data (for example, a depth map) in which the depth of the subject within the imaging range is two-dimensionally distributed. For example, the captured image 3 may include an RGB image together with the depth data. Furthermore, for example, the captured image 3 may be configured with a moving image or one or a plurality of still images as long as the physical motion of the infant can be analyzed.

In addition, the infant monitoring apparatus 1 according to the present embodiment acquires vital information including the infant's pulse rate and respiratory state. Pulse rate and respiratory status may be measured in any way. For example, at least one of the pulse rate and the respiratory state may be measured by analyzing the captured image 3. In addition, a measurement device capable of measuring at least one of the pulse rate and the respiratory state may be used for measuring at least one of the pulse rate and the respiratory state. In the scene of FIG. 1, the pulse rate and the respiratory state are measured by the measuring device 4 attached to the infant. The infant monitoring apparatus 1 according to the present embodiment acquires vital information from the measuring device 4.

Next, the infant monitoring apparatus 1 extracts a person area in which the infant appears in the acquired photographed image 3. As described above, the captured image 3 includes depth data indicating the depth of each pixel. Therefore, the infant monitoring apparatus 1 can specify the position of the subject in the captured image 3 in the real space by using this depth data. More specifically, the depth of the subject is acquired with respect to the surface of the subject. That is, the infant monitoring apparatus 1 can identify the position of the subject surface in the real space by referring to the depth of each pixel indicated by the depth data.

Therefore, the infant monitoring apparatus 1 refers to the depth of each pixel included in the extracted person area and analyzes the body movement and sleeping posture of the infant shown in the photographed image 3. Furthermore, the infant monitoring apparatus 1 determines whether or not the infant has a sign of SIDS based on the pulse rate and respiratory state indicated by the acquired vital information and the analyzed body movement and sleeping posture.

If the infant monitoring apparatus 1 determines that there is a sign of SIDS in the infant as a result of the determination, the infant monitoring apparatus 1 performs a sign detection notification for notifying that the infant has a sign of SIDS. In other words, when a sign of SIDS appears in the infant imaged in the captured image 3, the infant monitoring apparatus 1 issues a warning for notifying the SIDS sign. Thereby, the user of the infant monitoring apparatus 1 according to the present embodiment can know that the sign of SIDS has appeared in the infant existing in the imaging range of the camera 2, and can appropriately watch the infant.

As described above, according to the present embodiment, the physical movement (body movement and sleeping posture) of the infant is analyzed based on the captured image 3 including the depth data indicating the depth of each pixel. As described above, since the depth of each pixel is acquired with respect to the subject surface, the position of the subject surface in the real space can be specified by using the depth data. Therefore, by using this depth data, it is possible to analyze the physical movement of the infant in the real space (three-dimensional space) regardless of the viewing direction (viewpoint) of the camera 2 with respect to the infant.

In addition, the infant monitoring apparatus 1 according to the present embodiment provides an infant with signs of becoming SIDS based on vital information including not only the physical movement (body movement and sleeping posture) of the infant but also the pulse rate and respiratory state. It is determined whether or not there is. Therefore, according to the present embodiment, it is possible to determine from the multifaceted viewpoint whether or not the infant has a sign of SIDS.

Therefore, according to the present embodiment, a sign of SIDS can be appropriately detected, and a sleeping infant can be prevented from developing SIDS.

The sleeping posture is a posture when the infant is sleeping, such as a supine posture (a supine position), a lateral posture (a lateral position), a prone posture (a prone position). The information indicating the respiratory state included in the vital information may be appropriately selected according to the embodiment, and is, for example, the respiratory rate, the oxygen saturation (SpO ₂ ) of arterial blood, or the like. The respiration rate indicates the number of respirations per unit time. The pulse rate indicates the number of pulses per unit time. The time for measuring the number of breaths and the number of pulses may be appropriately set according to the embodiment.

Also, the location of the infant monitoring device 1 can be determined as appropriate according to the embodiment as long as the captured image 3 can be acquired from the camera 2 and vital information can be acquired from the measuring device 4. For example, the infant monitoring apparatus 1 may be disposed so as to be close to the camera 2 as illustrated in FIG. In addition, the infant monitoring apparatus 1 may be connected to the camera 2 and the measuring device 4 via a wireless and / or wired network, and may be arranged at a place completely different from the camera 2 and the measuring device 4.

§2 Configuration example <Hardware configuration>
Next, the hardware configuration of the infant monitoring apparatus 1 will be described with reference to FIG. FIG. 2 illustrates a hardware configuration of the infant monitoring apparatus 1 according to the present embodiment. As illustrated in FIG. 2, the infant monitoring apparatus 1 stores a control unit 11 including a CPU, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, a program 5 executed by the control unit 11, and the like. Unit 12, a touch panel display 13 for displaying and inputting images, a speaker 14 for outputting sound, an external interface 15 for connecting to an external device, a communication interface 16 for communicating via a network, and This is a computer to which a drive 17 for reading a program stored in the storage medium 6 is electrically connected. In FIG. 2, the communication interface and the external interface are described as “communication I / F” and “external I / F”, respectively.

It should be noted that regarding the specific hardware configuration of the infant monitoring apparatus 1, the components can be omitted, replaced, and added as appropriate according to the embodiment. For example, the control unit 11 may include a plurality of processors. In addition, for example, the touch panel display 13 may be replaced with an input device and a display device that are separately connected independently. For example, the speaker 14 may be omitted. Further, for example, the speaker 14 may be connected to the infant monitoring apparatus 1 as an external apparatus instead of as an internal apparatus of the infant monitoring apparatus 1. The infant monitoring apparatus 1 may incorporate a camera 2.

In the present embodiment, the camera 2 and the measuring device 4 are connected to the infant monitoring apparatus 1 via the external interface 15. Among these, the camera 2 is installed around the infant to be monitored and photographs the infant. The installation location of the camera 2 may be appropriately selected according to the embodiment. The camera 2 may be arrange | positioned so that the said infant can be image | photographed on the fence etc. of the bed where an infant sleeps, for example.

The camera 2 includes a depth sensor 21 for measuring the depth of the subject in order to capture the captured image 3 including depth data. The type and measurement method of the depth sensor 21 may be appropriately selected according to the embodiment. For example, the depth sensor 21 may be a sensor of TOF (TimeＦOf Flight) method or the like.

However, the configuration of the camera 2 is not limited to such an example as long as the depth can be acquired, and can be appropriately selected according to the embodiment. For example, the camera 2 may be a stereo camera so that the depth of the subject within the shooting range can be specified. Since the stereo camera shoots the subject within the shooting range from a plurality of different directions, the depth of the subject can be recorded. Further, the camera 2 may be replaced with the depth sensor 21 as long as the depth of the subject within the shooting range can be specified.

In addition, in order to photograph a sleeping infant, it is assumed that the place where the infant is photographed is basically dark. Therefore, the depth sensor 21 may be an infrared depth sensor that measures the depth based on infrared irradiation so that the depth can be acquired without being affected by the brightness of the shooting location. Examples of relatively inexpensive imaging apparatuses including such an infrared depth sensor include Kinect from Microsoft, Xtion from ASUS, and CARMINE from PrimeSense.

Here, the depth measured by the depth sensor 21 according to the present embodiment will be described in detail with reference to FIG. FIG. 3 shows an example of a distance that can be handled as the depth according to the present embodiment. The depth represents the depth of the subject. As exemplified in FIG. 3, the depth of the subject may be expressed by, for example, a straight line distance A between the camera 2 and the object, or a perpendicular distance B from the horizontal axis with respect to the subject of the camera 2. It may be expressed as

That is, the depth according to the present embodiment may be the distance A or the distance B. In the present embodiment, the distance B is treated as the depth. However, the distance A and the distance B can be converted into each other based on, for example, the three-square theorem. Therefore, the following description using the distance B can be applied to the distance A as it is. The infant monitoring apparatus 1 according to the present embodiment can analyze the state of the infant by using such a depth.

The measuring device 4 is configured to be able to measure at least one of a pulse rate and a respiratory state, and is attached to a body part such as a foot sole of an infant. The pulse rate and the respiratory state may be measured with the same device, or may be measured with separate devices. In addition, the index indicating the respiratory state may be appropriately selected according to the embodiment as long as the respiratory state of the infant can be specified. The index indicating the respiratory state is, for example, the respiratory rate, arterial oxygen saturation (SpO ₂ ), or the like. The respiratory state may be represented by a plurality of indices. For example, both respiratory rate and arterial oxygen saturation may be used to indicate respiratory status.

It should be noted that a known measuring device such as a pulse oximeter, a respiration sensor, or a capnometer may be used as such a measuring device 4. For example, as a measuring instrument capable of measuring the pulse rate and the oxygen saturation of arterial blood, Nellcor Oxysensor III manufactured by Covidien may be mentioned. Further, for example, a pulse oxy capnometer manufactured by Covidien Co., Ltd. can be used as a measuring instrument capable of measuring the pulse rate and the respiratory rate. In one form of this embodiment described below, a known pulse oximeter capable of measuring the pulse rate and arterial blood oxygen saturation is used as the measuring device 4.

In the present embodiment, the storage unit 12 stores the program 5. The program 5 is a program for causing the infant monitoring apparatus 1 to execute each process related to the infant watching described later, and corresponds to the “infant monitoring program” of the present invention. The program 5 may be recorded on the storage medium 6.

The storage medium 6 stores information such as a program by an electrical, magnetic, optical, mechanical, or chemical action so that information such as a program recorded by a computer or other device or machine can be read. It is a medium to do. The storage medium 6 corresponds to the “storage medium” of the present invention. 2 illustrates a disk-type storage medium such as a CD (Compact Disk) or a DVD (Digital Versatile Disk) as an example of the storage medium 6. However, the type of the storage medium 6 is not limited to the disk type and may be other than the disk type. Examples of the storage medium other than the disk type include a semiconductor memory such as a flash memory.

Note that such an infant monitoring apparatus 1 may be, for example, an apparatus designed exclusively for the provided service, or a general-purpose apparatus such as a PC (Personal Computer) or a tablet terminal. Furthermore, the infant monitoring apparatus 1 may be implemented by one or a plurality of computers.

<Functional configuration example>
Next, the functional configuration of the infant monitoring apparatus 1 will be described with reference to FIG. FIG. 4 illustrates a functional configuration of the infant monitoring apparatus 1 according to the present embodiment. In the present embodiment, the control unit 11 of the infant monitoring apparatus 1 develops the program 5 stored in the storage unit 12 in the RAM. And the control part 11 interprets and runs the program 5 expand | deployed by RAM by CPU, and controls each component. Thereby, the infant monitoring apparatus 1 functions as a computer including the image acquisition unit 51, the vital information acquisition unit 52, the sleeping state analysis unit 53, the sign determination unit 54, and the notification unit 55.

The image acquisition unit 51 acquires the captured image 3 captured by the camera 2. The vital information acquisition unit 52 acquires vital information 40 including the pulse rate and the respiratory state measured by the measuring device 4. The acquired captured image 3 includes depth data indicating the depth of each pixel. As described above, according to the depth data, the position of the subject in the captured image 3 in the real space, more specifically, the position of the subject surface in the real space can be specified.

Therefore, the sleeping state analyzing unit 53 extracts a person region in which the infant is photographed in the acquired photographed image 3. In addition, the sleeping state analysis unit 53 refers to the depth of each pixel included in the extracted person region, and analyzes the body movement and sleeping posture of the infant captured in the captured image 3.

Further, the sign determination unit 54 determines whether or not there is a sign of SIDS in the infant imaged in the captured image 3 based on the pulse rate and respiratory state included in the acquired vital information 40 and the analyzed body movement and sleeping posture. Determine whether. Then, as a result of the determination, when it is determined that there is a sign of SIDS in the infant, the notification unit 55 performs a sign detection notification for notifying that the infant has the sign of SIDS.

In the present embodiment, an example is described in which all of these functions are realized by a general-purpose CPU. However, some or all of these functions may be realized by one or more dedicated processors. Further, regarding the functional configuration of the infant monitoring apparatus 1, functions may be omitted, replaced, and added as appropriate according to the embodiment. Each function will be described in detail in an operation example described later.

§3 Operation Example Next, an operation example of the infant monitoring apparatus 1 will be described with reference to FIG. FIG. 5 exemplifies a processing procedure relating to infant watching by the infant monitoring apparatus 1. It should be noted that the processing procedure relating to infant watching described below corresponds to the “infant monitoring method” of the present invention. However, the processing procedure related to watching over the infant described below is merely an example, and each processing may be changed as much as possible. Further, in the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

(Step S101)
In step S <b> 101, the control unit 11 functions as the image acquisition unit 51 and acquires the captured image 3 captured by the camera 2. Then, after acquiring the captured image 3, the control unit 11 advances the processing to the next step S102.

In the present embodiment, the camera 2 includes a depth sensor 21. Therefore, the captured image 3 acquired in step S101 includes depth data indicating the depth of each pixel measured by the depth sensor 21. For example, the control unit 11 acquires the captured image 3 illustrated in FIG. 6 as the captured image 3 including the depth data.

FIG. 6 shows an example of the captured image 3 including depth data. The captured image 3 illustrated in FIG. 6 is an image in which the gray value of each pixel is determined according to the depth of each pixel. A black pixel is closer to the camera 2. On the other hand, a white pixel is farther from the camera 2. Based on the depth data, the control unit 11 can specify the position of each pixel in the real space. That is, the control unit 11 can specify the position in the three-dimensional space (real space) of the subject captured in each pixel from the coordinates (two-dimensional information) and the depth of each pixel in the captured image 3. . Hereinafter, a calculation example in which the control unit 11 specifies the position of each pixel in the real space will be described with reference to FIGS. 7 and 8.

FIG. 7 schematically illustrates the coordinate relationship in the captured image 3. FIG. 8 schematically illustrates a positional relationship between an arbitrary pixel (point s) of the captured image 3 and the camera 2 in the real space. 7 corresponds to a direction perpendicular to the paper surface of FIG. That is, the length of the captured image 3 shown in FIG. 8 corresponds to the length in the vertical direction (H pixels) illustrated in FIG. Further, the length in the horizontal direction (W pixels) illustrated in FIG. 7 corresponds to the length in the direction perpendicular to the paper surface of the captured image 3 that does not appear in FIG.

Here, as illustrated in FIG. 7, the coordinates of an arbitrary pixel (point s) of the captured image 3 are (x _s , y _s ), the horizontal field angle of the camera 2 is V _x , and the vertical direction Let the angle of view be V _y . The number of pixels in the horizontal direction of the captured image 3 is W, the number of pixels in the vertical direction is H, and the coordinates of the center point (pixel) of the captured image 3 are (0, 0).

The control unit 11 can acquire information indicating the angle of view (V _x , V _y ) of the camera 2 from the camera 2. Further, the control unit 11 may acquire information indicating the angle of view (V _x , V _y ) of the camera 2 based on a user input or may be acquired as a preset setting value. Further, the control unit 11 can acquire the coordinates (x _s , y _s ) of the point s and the number of pixels (W × H) of the captured image 3 from the captured image 3. Furthermore, the control unit 11 can acquire the depth Ds of the point s by referring to the depth data included in the captured image 3.

The control unit 11 can specify the position of each pixel (point s) in the real space by using these pieces of information. For example, the control unit 11 performs vector S (S _x , S _y , S _z) from the camera 2 to the point s in the camera coordinate system illustrated in FIG. , 1) can be calculated. Thereby, the position of the point s in the two-dimensional coordinate system in the captured image 3 and the position of the point s in the camera coordinate system can be mutually converted.

However, the vector S is a vector of a three-dimensional coordinate system centered on the camera 2. As illustrated in FIG. 8, the camera 2 may be inclined with respect to a horizontal plane (ground). That is, the camera coordinate system may be tilted from the world coordinate system of a three-dimensional space with respect to the horizontal plane (ground). Therefore, the control unit 11 applies the projective transformation using the roll angle, pitch angle (α in FIG. 8), and yaw angle of the camera 2 to the vector S, so that the vector S of the camera coordinate system is converted to the world coordinate system. And the position of the point s in the world coordinate system may be calculated. Each of the camera coordinates and the world coordinates is a coordinate system representing a real space. In this way, the control unit 11 can specify the position of the subject in the captured image 3 in the real space by using the depth data.

The acquired captured image 3 may be composed of a moving image or may be composed of one or a plurality of still images. Further, the control unit 11 may acquire the captured image 3 in synchronization with the video signal of the camera 2 in order to monitor the infant. And the control part 11 may perform immediately with respect to the picked-up image 3 which acquired the process of each step mentioned later. The infant monitoring apparatus 1 can perform real-time image processing by continuously executing such an operation continuously, and can monitor the infant existing in the photographing range of the camera 2 in real time.

(Step S102)
Returning to FIG. 5, in the next step S <b> 102, the control unit 11 functions as the vital information acquisition unit 52 and acquires vital information 40 including the infant's pulse rate and respiratory state. And the control part 11 advances a process to the following step S103, after acquiring the vital information 40. FIG.

In this embodiment, the measuring device 4 is attached to an infant. Therefore, the control unit 11 acquires vital information 40 from the measurement device 4. Moreover, in one form of this embodiment, the well-known pulse oximeter which can measure a pulse rate and the oxygen saturation of arterial blood is utilized as the measuring apparatus 4 with which an infant is equipped. Therefore, the control unit 11 acquires vital information 40 including the oxygen saturation level of arterial blood as information indicating the respiratory state.

The vital information 40 may include information on other types of vital signs in addition to the pulse rate and the respiratory state. Further, this step S102 may be executed before step S101, or may be executed at an arbitrary timing before step S104 described later. The timing for executing step S102 may be appropriately set according to the embodiment.

(Step S103)
In the next step S103, the control unit 11 functions as the sleeping state analysis unit 53, and extracts a person region in which an infant is photographed as illustrated in FIG. 6 from the captured image 3 acquired in step S101. Then, the control unit 11 extracts a person area from the captured image 3 and then proceeds to the next step S104.

Note that there are various known methods for extracting a person region, and the method may be appropriately selected according to the embodiment. For example, the control unit 11 may extract a person region in the captured image 3 by performing image analysis such as pattern detection and graphic element detection based on the shape of the infant. In this case, the control unit 11 may extract the person region by detecting the three-dimensional shape (contour) of the infant using the depth data. Moreover, the control part 11 may extract a person area | region by carrying out pattern detection etc. of the infant's two-dimensional shape, without using depth data.

Also, for example, infants are moving in real space. Therefore, the person area varies in the captured image 3. Such a fluctuating region can be extracted by the background subtraction method. Therefore, the control unit 11 may extract the fluctuating area as a person area based on the background difference method.

More specifically, first, the control unit 11 acquires a background image used for the background subtraction method. This background image may be acquired by an arbitrary method, and is set as appropriate according to the embodiment. For example, the control unit 11 may acquire a photographed image before an infant exists in the photographing range of the camera 2, in other words, a photographed image without an infant as a background image. And the control part 11 calculates the difference of the picked-up image 3 acquired at the time of the said step S101, and a background image, and extracts the foreground area | region of the said picked-up image 3. FIG. The foreground region is a region where a change has occurred from the background image, and is a region where a fluctuating object (moving object) is captured.

Therefore, when the extracted foreground area has an area equal to or larger than the threshold, the control unit 11 may recognize the foreground area as a person area. Alternatively, the control unit 11 may extract a person area from the foreground area by pattern detection or the like. The process for extracting the foreground area is merely a process for calculating the difference between the captured image 3 and the background image. Therefore, according to the process, the control unit 11 (infant monitoring device 1) can narrow the range in which the person area is detected without using advanced image processing. Therefore, according to the processing, the processing load in step S103 can be reduced.

Note that there are various types of background subtraction methods, and the background subtraction method applicable to the present embodiment is not limited to the above example. Other types of background subtraction methods include, for example, a method of separating the background and the foreground using three different images, and a method of separating the background and the foreground by applying a statistical model. . With these methods, the control unit 11 may extract a person region.

(Step S104)
In the next step S104, the control unit 11 functions as the sleeping state analysis unit 53, refers to the depth of each pixel included in the person area extracted in step S103, and the body movement and sleeping of the infant in the captured image 3 Analyze posture. And the control part 11 advances a process to following step S105, after analyzing the body movement and sleeping posture of an infant.

The analysis of the infant's body movement and sleeping posture may be performed by an arbitrary method. For example, the body motion analysis may be performed by the following method. The control unit 11 continuously specifies the position of each pixel included in the person area in the real space using the depth of each pixel included in the person area. Thereby, the control unit 11 can plot the position of each pixel included in the person area on the real space coordinates.

Here, as described above, since the depth of each pixel is acquired with respect to the subject surface, each pixel included in the person region corresponds to the contour of the infant in real space. Therefore, data obtained by plotting the position of each pixel included in the person area on the real space coordinates (hereinafter also referred to as measurement data) indicates a change in the contour of the infant, that is, the body movement of the infant. Therefore, the control unit 11 can measure the body movement of the infant by plotting the position of each pixel included in the person region on the real space coordinates.

Then, the control unit 11 can analyze the body movement of the infant by determining whether or not a predetermined pattern appears in the measurement data by pattern matching or the like, for example. Further, the control unit 11 can analyze the body movement of the infant by calculating the amount of body movement of the infant based on the measurement data. Note that the target range for observing body movements may be the entire person region or a part of the person region. Further, the amount of body movement may be expressed by an average value of displacements of pixels included in the target range, or may be expressed by a total sum of displacements of pixels included in the target range. A method for analyzing body movement may be appropriately selected according to the embodiment.

Also, for example, the sleeping posture analysis may be performed by the following method. The control unit 11 specifies the position and orientation of each body part such as an infant's face, shoulder, and chest that appear in the person area by pattern matching or the like. When Microsoft's Kinect is adopted as the camera 2, the position and orientation of each body part can be specified by using, for example, the FaceFTracking function.

And the control part 11 determines the sleeping posture of the infant according to the direction of the body with respect to the bed. Specifically, it is assumed that the infant is sleeping on the bed. For this reason, the control unit 11 determines that the sleeping posture of the infant is a supine posture (a supine position) when the body parts such as the identified face, shoulder, and chest are facing vertically upward.

In addition, when the body parts such as the identified face, shoulder, and chest are facing in the horizontal direction (horizontal direction), the control unit 11 determines that the sleeping posture of the infant is the horizontal posture (side-down position). . In addition, the control unit 11 determines that the sleeping posture of the infant is a prone posture (prone position) when the body parts such as the identified face, shoulder, and chest are vertically downward.

In this way, the sleeping posture of an infant can be analyzed. Note that the method of analyzing the sleeping posture is not limited to such an example, and may be appropriately selected according to the embodiment. Also, the types of sleeping postures to be analyzed need not be limited to the above three types. For example, the control unit 11 may omit the determination of the horizontal posture and specify the sleeping posture of the infant as either the supine posture or the prone posture. For example, the control unit 11 may set postures other than the above three types as analysis targets.

(Step S105)
In the next step S105, the control unit 11 functions as the sign determination unit 54, and based on the pulse rate and the respiratory state indicated by the vital information 40 acquired in step S102, and the body movement and sleeping posture analyzed in step S104. Then, it is determined whether or not an SIDS sign appears in the infant imaged in the photographed image 3.

And as a result of the determination, if it is determined that a sign of SIDS appears in the infant, the control unit 11 proceeds to the next step S106. On the other hand, when no sign of SIDS appears in the infant, the control unit 11 omits the process of the next step S106 and ends the process according to this operation example.

Each condition of the pulse rate, the respiratory state, the body movement, and the sleeping posture for detecting the signs of SIDS may be appropriately set according to the embodiment. In the present embodiment, the control unit 11 detects two types of states, a lying-down state and a hypoxic state, as signs of SIDS. Hereinafter, an example of conditions for detecting each state will be described.

(A) Prone sleeping state First, the lying prone state will be described. As a result of SIDS research, it is known that infants sleep on their face as one of the risk factors for SIDS. Therefore, in the present embodiment, the control unit 11 detects the infant lying on the basis of the sleeping posture of the infant analyzed in step S104.

For example, when the control unit 11 detects in step S104 that the sleeping position of the infant shown in the captured image 3 is a prone position (prone position), the control unit 11 determines that the infant has a sign of SIDS. On the other hand, if the control unit 11 detects in step S104 that the sleeping posture of the infant shown in the captured image 3 is a supine posture (a supine position) or a lateral posture (a laterally lying position), the infant is informed of the SIDS. It is determined that there is no sign. This makes it possible to detect prone sleep, which is one of the risk factors for SIDS, and appropriately prevent infants from becoming SIDS.

(B) Low oxygen state Next, the low oxygen state will be described. As mentioned above, as a result of SIDS research, if an infant is overheated during sleep, hypoxia will progress in the sleeping infant's body, and this infant may eventually become SIDS. It has been pointed out that there is. Therefore, in the present embodiment, the control unit 11 detects this hypoxia state based on the pulse rate and the respiratory state indicated by the vital information 40 acquired in step S102 and the body movement of the infant analyzed in step S104. To do.

Specifically, if a sleeping infant falls into hypoxia, breathing becomes excessively shallow (or apnea) and the pulse rate is excessive to compensate for the decreased blood oxygen level. It is assumed that the amount of body movement increases excessively. Therefore, the control unit 11 determines whether or not the pulse rate, the respiratory state, and the body motion acquired in Step S102 and Step S104 satisfy the following conditions.

First, the pulse rate will be explained. The control unit 11 compares the pulse rate indicated by the vital information 40 acquired in step S102 with the normal pulse rate of the infant. When the control unit 11 determines that the pulse rate acquired in step S102 is higher than the normal pulse rate by a predetermined value or more, the pulse rate indicated by the acquired vital information 40 is the normal pulse rate. It is detected that it is higher than a predetermined value.

In addition, the pulse rate at the normal time may be appropriately set according to the embodiment. For example, when the infant is awake, the infant's pulse rate measured with a measuring device or tactile sense may be set as the normal pulse rate. The method for measuring the pulse rate at the normal time can be appropriately selected according to the embodiment.

Further, the predetermined value for determining that the pulse rate has increased excessively may be appropriately set according to the embodiment. The predetermined value may be given by a normal pulse rate magnification. For example, assume that the normal pulse rate is set to 120 times per minute and the magnification is set to 1.2 times. In this case, when the control unit 11 determines that the pulse rate acquired in step S102 is 144 times or more, the pulse rate indicated by the acquired vital information 40 is equal to or greater than the normal pulse rate. Detect high.

In addition, the control unit 11 determines that the pulse rate indicated by the acquired vital information 40 is the normal pulse rate when the acquired pulse rate is lower than the normal pulse rate by a predetermined value or more for a predetermined time. It may be detected that it is higher than a predetermined value. The value of the predetermined time serving as a reference for the detection may be appropriately set according to the embodiment. Accordingly, it is possible to consider the time during which the state in which the pulse rate has excessively increased continues for SIDS sign detection.

Next, the respiratory state will be described. In one form of this embodiment, as above-mentioned, the control part 11 acquires the oxygen saturation of arterial blood as information which shows a respiratory state in step S102. Therefore, the control unit 11 determines the infant's respiratory state based on the oxygen saturation of arterial blood. Specifically, the control unit 11 compares the arterial oxygen saturation acquired in step S102 with the reference value of the arterial oxygen saturation at normal time of the infant. When the control unit 11 determines that the oxygen saturation level of the arterial blood acquired in step S102 is lower than the reference value of the arterial blood oxygen saturation level at the normal time, the respiratory state indicated by the acquired vital information 40 Is detected to be shallower than the normal breathing state.

It should be noted that the reference value of the oxygen saturation level of arterial blood in normal times may be set as appropriate according to the embodiment. For example, it is generally known that the arterial oxygen saturation in normal times of infants is approximately 95% or more. Therefore, the reference value may be set as appropriate with a value of 94% or less. For example, when the reference value is set to 94%, the control unit 11 determines that the oxygen saturation of the arterial blood acquired in step S102 is 94% or less, and the respiration indicated by the acquired vital information 40 It is detected that the state is shallower than the normal breathing state.

In addition, when the state in which the oxygen saturation of the acquired arterial blood is lower than the reference value continues for a predetermined time, the control unit 11 determines that the respiratory state indicated by the acquired vital information 40 is higher than the normal respiratory state. The shallowness may be detected. The value of the predetermined time serving as a reference for the detection may be appropriately set according to the embodiment. Accordingly, it is possible to consider the time during which the state of the child is excessively shallow in the SIDS sign detection.

Next, body movement will be described. The control unit 11 compares the amount of body movement analyzed in step S104 with the amount of body movement at normal time of the infant. When the control unit 11 determines that the amount of body motion analyzed in step S104 is smaller than the normal amount of body motion by a predetermined value or more, the amount of body motion analyzed in step S104 is normal. It is detected that the amount of body movement is less than a predetermined value.

It should be noted that the amount of normal body movement may be appropriately set according to the embodiment. For example, the baby is photographed by the camera 2 when the baby is sleeping without abnormality. And the control part 11 may set the amount of body movement obtained by analyzing the acquired picked-up image by the method similar to said step S104 as a body movement amount in normal time. The method for measuring the amount of body movement at normal time can be appropriately selected according to the embodiment.

Further, the predetermined value for determining that the amount of body movement has excessively decreased may be set as appropriate according to the embodiment. The control unit 11 determines that the amount of body motion analyzed in step S104 is normal when the analyzed amount of body motion is lower than the normal amount of body motion by a predetermined value or more for a predetermined time. It may be detected that the amount of movement is less than a predetermined value. The value of the predetermined time serving as a reference for the detection may be appropriately set according to the embodiment. Accordingly, it is possible to consider the time during which the amount of body movement is excessively decreased in detecting the sign of SIDS.

Then, the control unit 11 has a pulse rate acquired in step S102 that is higher than the normal pulse rate by a predetermined value or more, the respiratory state acquired in step S102 is shallower than the normal respiratory state by a predetermined amount, and If it is detected that the amount of body movement analyzed in step S104 is smaller than the amount of body movement at the normal time by a predetermined value or more, it is determined that the infant has a sign of SIDS. On the other hand, when the control unit 11 detects that any one of the three conditions is not satisfied, the control unit 11 determines that the infant has no sign of SIDS. As a result, it is possible to accurately detect that the sleeping infant falls into the low oxygen state as described above, and appropriately prevent the infant from becoming SIDS.

(C) Summary As described above, the control unit 11 can detect a sign of SIDS that appears in an infant. If the controller 11 detects either the prone state or the hypoxic state in step S105, the control unit 11 proceeds to the next step S106. On the other hand, if the control unit 11 does not detect the state of lying down or hypoxia in step S105, the control unit 11 omits the next step S106 and ends the process according to the operation example. .

It should be noted that the infant state detected as a sign of SIDS is not limited to the above-mentioned lying-down state and hypoxic state, and may be appropriately selected according to the embodiment. For example, when the sleeping posture of an infant changes from a supine posture (a supine position) to a lateral posture (a lateral position), it is assumed that there is a high probability that the sleeping posture of the infant becomes a prone posture (a prone position). Therefore, when the control unit 11 detects in step S104 that the sleeping posture of the infant shown in the photographed image 3 has changed from the supine posture (the supine position) to the horizontal posture (the laterally lying position), the control unit 11 applies the SIDS to the infant. It may be determined that there are signs. Further, for example, any of the above-mentioned lying-down state and hypoxic state may be excluded from the detection target. Furthermore, the state of the infant to be detected as a sign of SIDS may be selected by the user or may be set in advance.

Further, in this step S105, when neither the prone sleep state nor the hypoxic state is detected, the control unit 11 may recognize that the infant is in a normal state. Then, the control unit 11 may notify the user of the infant monitoring apparatus 1 that the infant is in a normal state. For example, the control unit 11 may perform the notification by displaying that the infant is in a normal state on the touch panel display 13.

(Step S106)
In the next step S106, the control unit 11 functions as the notification unit 55, and when it is determined that there is a sign of SIDS in the infant as a result of the determination in step S105, the sign of the SIDS is given to the infant. A sign detection notification is made to notify that there is something. Thereby, the processing according to this operation example is completed. The means by which the control unit 11 performs the sign detection notification can be appropriately selected according to the embodiment.

For example, when the infant monitoring apparatus 1 is used in a facility such as a hospital, the infant monitoring apparatus 1 can be connected to equipment such as a nurse call system via the external interface 15. In this case, the control unit 11 may perform sign detection notification in cooperation with the equipment such as the nurse call system. That is, the control unit 11 may control the nurse call system via the external interface 15. And the control part 11 may perform the call by the said nurse call system as a sign detection notification. Accordingly, it is possible to appropriately notify a nurse or the like who watches the infant that the risk of becoming SIDS is approaching the target infant.

Further, for example, the control unit 11 may display a screen on the touch panel display 13 for notifying that a sign of SIDS that has appeared in an infant has been detected as a sign detection notification. Further, for example, the control unit 11 may perform the sign detection notification by outputting a predetermined sound from the speaker 14 connected to the infant monitoring apparatus 1. By installing the touch panel display 13 and the speaker 14 in the bedroom or the like of the infant's parents, it is possible to appropriately notify the parents that the risk of becoming SIDS is approaching the target infant.

Further, for example, the control unit 11 may perform such a sign detection notification using an e-mail, a short message service, a push notification, or the like. When performing such sign detection notification, the e-mail address, telephone number, and the like of the user terminal that is the notification destination may be registered in advance in the storage unit 12. Then, the control unit 11 may perform sign detection notification using the pre-registered e-mail address, telephone number, and the like.

(Action / Effect)
As described above, the infant monitoring apparatus 1 according to the present embodiment analyzes the physical movement (body movement and sleeping posture) of the infant based on the captured image 3 including the depth data indicating the depth of each pixel. As described above, since the depth of each pixel is acquired with respect to the subject surface, the position of the subject surface in the real space can be specified by using the depth data.

For example, when the camera is installed vertically above the infant and the acquired captured image is a two-dimensional image, it is difficult to specify the vertical movement of the infant from the two-dimensional image. On the other hand, in the present embodiment, since the captured image 3 includes depth data indicating the depth of each pixel, the control unit 11 can specify the vertical movement of the infant. Therefore, according to the present embodiment, by using the depth data, it is possible to analyze the physical movement of the infant in the real space (three-dimensional space) regardless of the viewing direction (viewpoint) of the camera 2 with respect to the infant. it can.

In addition, the infant monitoring apparatus 1 according to the present embodiment has an indication that an indication of SIDS is based on vital information 40 including not only the physical movement (body movement and sleeping posture) of the infant but also the pulse rate and respiratory state. It is determined whether or not. Therefore, according to the present embodiment, it is possible to determine from the multifaceted viewpoint whether or not the infant has an indication of SIDS, such as the hypoxic state.

§4 Modifications Embodiments of the present invention have been described in detail above, but the above description is merely an illustration of the present invention in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention.

(1) Information indicating respiratory state For example, in the above-described embodiment, the control unit 11 acquires the oxygen saturation of arterial blood as information indicating the respiratory state in step S102. However, the information indicating the breathing state is not limited to such an example, and may be appropriately selected according to the embodiment. For example, the information indicating the respiratory state may be an infant's respiratory rate.

In this case, the method for measuring the respiratory rate may be appropriately selected according to the embodiment. For example, when the measuring device 4 is configured to be able to measure the infant's respiratory rate, the control unit 11 acquires the infant's respiratory rate from the measuring device 4 as information indicating the respiratory state in step S102. May be. Thereby, vital information used for judging the sign of SIDS can be acquired appropriately.

For example, the control unit 11 may acquire the respiratory rate of the infant by performing image analysis on the captured image 3 acquired in step S101 instead of from the measuring device 4. Specifically, the control unit 11 functions as the vital information acquisition unit 52 and refers to the depth of each pixel included in the person region of the captured image 3 to measure the number of changes in the infant's rib cage.

The position of the rib cage may be arbitrarily set within the person area. For example, the position of the thorax may be set in the person area by designation by the user of the infant monitoring apparatus 1. Further, for example, the control unit 11 may specify the position of the thorax in the person region by pattern matching or the like. And the control part 11 can measure the fluctuation | variation of the said infant's rib cage by plotting the depth of each pixel contained in the range which the rib cage in a person area shows.

Here, since the change in the thorax is assumed to be caused by respiration, the change in the measured thorax is expressed by a wave in which peaks and troughs are alternately repeated. Therefore, the control unit 11 counts the number of peaks or valleys appearing within a predetermined time in the measured changes in the rib cage, thereby measuring the number of chest ribs of the infant and measuring the chest fluctuation measured during the predetermined time. The number of times can be acquired as the respiratory rate of the infant.

As described above, when the respiratory rate is measured by analyzing the captured image 3, information indicating the respiratory state of the infant can be acquired non-invasively. Therefore, in this case, the number of devices to be attached to the infant can be reduced, and the possibility of an accident such as the wiring of such devices being entangled in the infant's neck can be reduced.

Moreover, when acquiring a respiratory rate as information which shows a respiratory state, the control part 11 is based on the respiratory rate in the said step S105, and the respiratory state shown by the acquired vital information 40 is from the normal respiratory state of the infant. It is possible to detect whether it is shallower than a predetermined depth. Specifically, the control unit 11 compares the acquired respiration rate with the normal respiration rate. When the control unit 11 determines that the acquired respiration rate is lower than the normal respiration rate by a predetermined value or more, the respiration state indicated by the acquired vital information 40 is lower than the normal respiration state of the infant. Detect that it is shallower than a predetermined depth.

Note that the normal respiratory rate may be set as appropriate according to the embodiment. For example, when the infant is awake, the respiratory rate of the infant measured with a measuring device or visually may be set as the normal respiratory rate. The method for measuring the respiration rate at the normal time can be appropriately selected according to the embodiment.

Further, the predetermined value for determining that the breathing state has become shallower than a predetermined value may be set as appropriate according to the embodiment. Further, the predetermined value may be given as a rate of decrease in the respiration rate at the normal time. For example, it is assumed that the normal respiration rate is set to 30 times per minute and the decrease rate for detecting that the respiration state is shallow is set to 30%. In this case, when the control unit 11 determines that the acquired respiratory rate is 21 times or less per minute, the respiratory state indicated by the acquired vital information 40 is greater than or equal to the normal respiratory state of the infant. Detect shallowness.

In addition, when the acquired respiration rate is lower than the normal respiration rate by a predetermined value or more for a predetermined time, the control unit 11 determines that the respiration state indicated by the acquired vital information 40 is the normal respiration of the infant. You may detect that it is shallower than a predetermined state. The value of the predetermined time serving as a reference for the detection may be appropriately set according to the embodiment. Accordingly, it is possible to consider the time during which the shallow breathing state or the apnea state is continued in the detection of the SIDS sign.

(2) Pulse rate acquisition method For example, in the above-described embodiment, the pulse rate is measured by the measuring device 4 attached to the infant. However, the method for measuring the pulse rate is not limited to such an example, and can be appropriately selected according to the embodiment. For example, the control unit 11 may acquire the infant's pulse rate by analyzing the captured image 3 acquired in step S101.

Measurement of the pulse rate by image analysis may be performed by a known method. For example, the color of the skin such as the face surface changes due to blood flow. Specifically, when the blood flow volume increases due to blood pulsation, the amount of hemoglobin increases in the portion where the blood flows. On the other hand, when the blood flow decreases, the amount of hemoglobin decreases at the part where the blood flows. Here, hemoglobin has a characteristic of absorbing light such as infrared light and green light. Therefore, when the blood flow volume increases, light such as infrared light and green light is easily absorbed, and the light becomes thin in the acquired captured image 3. On the other hand, when the blood flow is reduced, light such as infrared light and green light is not easily absorbed, and the light becomes dark in the captured image 3 to be acquired. The control unit 11 can measure the pulse rate by measuring the number of times such light shading is repeated in the acquired captured image 3.

1 ... Infant monitoring device,
2 ... Camera, 21 ... Depth sensor,
3 ... taken image, 4 ... measuring instrument,
5 ... Program, 6 ... Storage medium,
11 ... Control unit, 12 ... Storage unit, 13 ... Touch panel display,
14 ... Speaker, 15 ... External interface, 16 ... Communication interface,
17 ... drive,
51 ... Image acquisition unit, 52 ... Vital information acquisition unit, 53 ... Sleeping state analysis unit,
54 ... sign determination unit, 55 ... notification unit

Claims

An image acquisition unit that captures a captured image of a sleeping infant, and includes a captured image including depth data indicating the depth of each pixel in the captured image;
A vital information acquisition unit that acquires vital information including the infant's pulse rate and respiratory state;
A sleeping state analysis unit that extracts a person region in which the infant is photographed in the acquired captured image and analyzes a body movement and a sleeping posture of the infant with reference to the depth of each pixel included in the extracted person region; ,
An indication for determining whether or not the infant has a sign of sudden infant death syndrome based on the pulse rate and the respiratory state indicated by the acquired vital information, and the analyzed body movement and sleeping posture. A determination unit;
As a result of the determination, when it is determined that the infant has a sign of sudden infant death syndrome, a notification unit that performs a sign detection notification for notifying the infant that there is a sign of sudden infant death syndrome. When,
Comprising
Infant monitoring device.
The sign determination unit determines that the infant has a sign of sudden infant death syndrome when detecting that the infant's sleeping posture is prone,
The infant monitoring apparatus according to claim 1.
The symptom determination unit is configured such that the pulse rate indicated by the acquired vital information is higher than a normal pulse rate of the infant by a predetermined value or more, and the respiratory state indicated by the acquired vital information is that of the infant. Sudden infant death syndrome is detected when it is detected that the amount of body motion analyzed is shallower than a predetermined amount than the normal breathing state and the amount of the analyzed body motion is smaller than the normal amount of body motion of the infant by a predetermined value or more. Determining that the infant has the indication
The infant monitoring apparatus according to claim 1 or 2.
The vital information acquisition unit is configured to be able to measure a pulse rate, and acquires the infant's pulse rate from a measuring device attached to the infant.
The infant monitoring apparatus according to any one of claims 1 to 3.
The measuring device is configured to further measure the oxygen saturation of arterial blood,
The vital information acquisition unit acquires oxygen saturation of arterial blood of the infant from the measuring device as information indicating the respiratory state,
The sign determination unit compares the acquired oxygen saturation of the arterial blood with a reference value of the arterial oxygen saturation at the normal time of the infant, and the acquired oxygen saturation of the arterial blood is the normal time of the infant Detecting that the respiratory state indicated by the acquired vital information is shallower than a normal respiratory state of the infant when it is determined that it is lower than a reference value of oxygen saturation of arterial blood in
The infant monitoring apparatus according to claim 4.
The measuring device is configured to further measure the respiratory rate,
The vital information acquisition unit acquires the infant respiratory rate from the measuring device as information indicating the respiratory state,
The sign determination unit compares the acquired respiratory rate with the normal infant respiratory rate, and determines that the acquired respiratory rate is lower than the normal infant respiratory rate by a predetermined value or more. , Detecting that the respiratory state indicated by the acquired vital information is shallower than a normal respiratory state of the infant by a predetermined amount or more,
The infant monitoring apparatus according to claim 4 or 5.
The vital information acquisition unit refers to the depth of each pixel included in the person region in the photographed image, and measures the number of changes in the infant's rib cage as information indicating the respiratory state, as the infant Get the respiratory rate of
The sign determination unit compares the acquired respiratory rate with the normal infant respiratory rate, and determines that the acquired respiratory rate is lower than the normal infant respiratory rate by a predetermined value or more. , Detecting that the respiratory state indicated by the acquired vital information is shallower than a normal respiratory state of the infant by a predetermined amount or more,
The infant monitoring apparatus according to claim 4 or 5.
Computer
A captured image obtained by photographing a sleeping infant, including a captured image including depth data indicating the depth of each pixel in the captured image;
Obtaining vital information including the infant's pulse rate and respiratory status;
Extracting a person region in which the infant appears in the acquired captured image;
Analyzing the body movement and sleeping posture of the infant with reference to the depth of each pixel included in the extracted person region;
A step of determining whether or not the infant has a sign of sudden infant death syndrome based on the pulse rate and the respiratory state indicated by the acquired vital information and the analyzed body movement and the sleeping posture. When,
As a result of the determination, if it is determined that there is a sign of sudden infant death syndrome in the infant, performing a sign detection notification for notifying that the infant has the sign of sudden infant death syndrome; ,
Perform infant monitoring method.
On the computer,
A captured image obtained by photographing a sleeping infant, including a captured image including depth data indicating the depth of each pixel in the captured image;
Obtaining vital information including the infant's pulse rate and respiratory status;
Extracting a person region in which the infant appears in the acquired captured image;
Analyzing the body movement and sleeping posture of the infant with reference to the depth of each pixel included in the extracted person region;
A step of determining whether or not the infant has a sign of sudden infant death syndrome based on the pulse rate and the respiratory state indicated by the acquired vital information and the analyzed body movement and the sleeping posture. When,
As a result of the determination, if it is determined that there is a sign of sudden infant death syndrome in the infant, performing a sign detection notification for notifying that the infant has the sign of sudden infant death syndrome; ,
Infant monitoring program for running.