JP4703049B2 - Monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a monitoring device and a monitoring method for monitoring an object to be monitored, and more particularly to a monitoring device and a monitoring method for monitoring changes such as a sleeping person's breathing.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as a monitoring device for monitoring a sleeper's respiration change, a device for monitoring a sleeper's respiration change based on a time transition of a pressure distribution detected by a load sensor or a pressure sensor has been proposed.
[0003]
[Problems to be solved by the invention]
However, according to the conventional apparatus as described above, the monitoring device has a very small signal to be measured, and therefore, a high-performance signal amplifier or some kind of signal processing is required to acquire and detect a stable signal. It was a complicated and large-scale system.
[0004]
Therefore, an object of the present invention is to provide a monitoring device that not only reliably detects the state of a sleeping person but also is small and simple.
[0005]
[Means for Solving the Problems]
To achieve the above objective,First aspectThe monitoring device according to the invention according to the invention is obtained by, for example, an imaging unit 11 that continuously images a monitoring object 2 illuminated with a constant light amount, as shown in FIGS. 1, 2, and 3. Difference image generation means 14 for generating a difference image of two frames at a predetermined time interval; calculation means 32 for obtaining a change output relating to a change between the two frames from the difference image; and monitoring from a time change of the change output. Detecting means 33 for detecting the movement of the object 2..
[0006]
If comprised in this way, since the imaging means 11, the difference image generation means 14, the calculating means 32, and the detection means 33 are provided, the difference image of the image of 2 frames of the predetermined time interval obtained by the imaging means 11 is obtained. By generating and detecting the movement of the monitoring object 2 from the temporal change of the change output relating to the change between the two frames from the difference image, not only the state of the sleeper 2 is reliably detected, but also small and simple. A certain monitoring device 1 can be provided. Illumination with a certain amount of light is not limited to illumination with an artificial light source, but may be illumination with natural light. The movement of the monitoring object 2 is typically a periodic movement.
[0007]
AlsoThe second aspect is,First aspectMonitoring device 1 described inInThe illumination pattern projecting means 12 for illuminating the monitoring object 2 with a predetermined pattern illumination light 13a is provided, and the pattern illumination light 13a is parallel to the direction connecting the illumination pattern projection means 12 and the imaging means 11. It may be characterized by having an illumination distribution of
[0008]
AlsoThe third aspect is,First aspectOrSecond aspectMonitoring device 1 described inInThe imaging means 10 has an imaging device in which a plurality of pixels are arranged.
[0009]
AlsoThe fourth aspect is,First aspectThruThird aspectEither1 aspectMonitoring device 1 described inInThe difference image generation means 14 is characterized by performing a binarization process on the difference image.
[0010]
AlsoThe fifth aspect is  ,First aspectThruFourth aspectEither1 aspectMonitoring device 1 described inInThe calculating means 32 may include accumulating means 32a for accumulating each pixel value of the difference image.
[0011]
AlsoThe sixth aspect is,First aspectThruFourth aspectEither1 aspectMonitoring device 1 described inInThe calculating means 32 calculates a pattern movement amount of each moving part of the pattern on the difference image as the change output. Further, the calculation means 32 may calculate the cumulative pattern movement amount.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol or a similar code | symbol is attached | subjected to the mutually same or equivalent member, and the overlapping description is abbreviate | omitted.
[0013]
FIG. 1 is a schematic perspective view of a monitoring device 1 according to an embodiment of the present invention. A person 2 (hereinafter referred to as a sleeping person 2) as a monitoring object lies on a bed 6 in the figure. Further, a bedding 3 is further hung on the sleeping person 2 and covers a part of the sleeping person 2 and a part of the bed 6.
[0014]
On the other hand, an imaging unit 11 as an imaging unit for continuously imaging the sleeping person 2 illuminated with a constant light amount is installed immediately above the abdomen around the sleeping person 2. The constant light amount may be substantially constant without significant change during the exposure of the imaging unit 11. More specifically, it is sufficient that there is no large change, for example, a change in light quantity of 10% or more during a time interval for obtaining two frames for generating a differential image described later. That is, on the difference image, bright spots that have not moved are almost removed together with the background (the smaller the change in the amount of light, the higher the degree of removal), and there should be no change in the amount of light so that the moved bright spot can be extracted. Further, the illumination may not be performed outside the exposure time.
[0015]
Further, a light projecting unit 12 as an illumination pattern light projecting unit for illuminating the sleeper 2 with a predetermined pattern illumination light is installed approximately above the foot or the head of the sleeper 2 (in the case of the foot upper part in the figure). The light is illuminated around the bedding 3 on the abdomen of the sleeping person 2. The illuminated range may be set to a range that covers the positions that the abdomen, chest, back, and shoulders of the sleeping person 2 can take during sleeping. Similarly, the range of the imaging region by the imaging unit 11 may be set to a range that covers the positions that the abdomen, chest, back, and shoulders of the sleeping person 2 can take while sleeping. In the present embodiment, the imaging unit 11 is installed immediately above the periphery of the abdomen of the sleeping person 2, and the light projecting unit 12 is installed approximately above the feet or the head of the sleeping person 2, but is not limited thereto. These may be positions where it is easy to detect the periodic movement of the sleeping person 2. Or it is good also as a position which is easy to install, for example, you may install avoiding the sleeping person 2 upper direction. Here, the periodic movement is typically the breathing of the sleeping person 2.
[0016]
Furthermore, the imaging unit 11 and the light projecting unit 12 are installed so that the axis connecting the imaging unit 11 and the light projecting unit 12 and the center line of the bed 6 are approximately parallel. The installation location of the imaging unit 11 and the light projecting unit 12 is preferably installed on the ceiling, for example, but if there is a wall in the vicinity, it may be a wall or may be attached to a stand. That is, the installation location may be appropriately determined according to the purpose and specifications of the monitoring device. By installing in this way, it is possible to sensitively detect small periodic movements of the sleeper 2 such as respiration. The imaging unit 11 has an imaging element in which a plurality of pixels are arranged, and is typically a CCD camera.
[0017]
The light projecting unit 12 is typically a light projecting unit using a diffraction element having a periodic structure. The diffractive element is typically a fiber grating (FG) element, but may be any element as long as it is spatially discrete and projects light, such as a diffraction grating, It may be a lens array, an image of a light source array or an aperture array, a collimated light emitted from the light source array or the aperture array, or the like. The pattern illumination light has an illumination distribution in a direction parallel to the direction connecting the light projecting unit 12 and the imaging unit 11. The illumination distribution means that the illumination light has a distribution of intensity change. That is, the pattern illumination light may be a line illumination light as shown in FIG. In the line illumination light, the intensity of the illumination light is constant in the direction orthogonal to the parallel direction, but there is a distribution of intensity change in the intensity in the parallel direction. The light projecting unit 12 illuminates the sleeping person 2 with a grating pattern 13a in which bright spots are arranged in a square lattice pattern. The FG element 13 will be described later with reference to FIG.
[0018]
An example of the configuration of the monitoring device 1 will be described with reference to the block diagram of FIG. The monitoring device 1 includes an imaging device 10 that includes an imaging unit 11 and a light projecting unit 12, an arithmetic device 30, and a notification device 40. The arithmetic device 30 is typically a personal computer or a microcomputer. The arithmetic device 30 includes a control unit 31 and controls the monitoring device 1. An interface 35 is connected to the control unit 31, and the imaging device 10 and the notification device 40 are connected to the control unit 31 via the interface 35 and controlled.
[0019]
Here, the imaging device 10 will be described with reference to the conceptual perspective view of FIG. Here, it will be described that the sleeper 2 is easy to understand as the object 20 having a rectangular parallelepiped shape, and the surface illuminated by the FG element is the plane 21. Further, for easy understanding, the change (movement) of the object 20 will be described when the object 20 exists and when it does not exist.
[0020]
In the drawing, an object 20 is placed on a plane 21. The orthogonal coordinate system XYZ is taken so that the XY axis is placed in the plane 21, and the object 20 is placed in the first quadrant of the XY coordinate system.
[0021]
On the other hand, the imaging unit 11 is arranged above the plane 21 on the Z axis in the drawing. The imaging lens 11a of the imaging unit 11 is arranged so that its optical axis coincides with the Z axis. An imaging plane 15 ′ (image plane) of the imaging element 15 on which the imaging lens 11 a forms an image of the plane 21 or the object 20 is a plane orthogonal to the Z axis. An xy orthogonal coordinate system is set in the imaging plane 15 ′ so that the Z axis passes through the origin of the xy coordinate system.
[0022]
The FG element 13 is disposed at an equal distance from the plane 21 with the imaging lens 11a and at a distance d from the imaging lens 11a in the negative direction of the Y axis. As will be described later with reference to FIG. 3, the laser beam L1 is continuously incident on the FG element 13 in the Z-axis direction, and a pattern in which dots are arranged in a square lattice pattern is continuously planar 21. Is irradiated. That is, the object 20 and the plane 21 are illuminated with the pattern illumination light. The image processing unit 14 is electrically connected to the image capturing unit 11 as a differential image generating unit that generates a differential image of two frames of images with a predetermined time interval obtained by the image capturing unit 11. The image processing unit 14 generates a difference image by taking a difference for each pixel of the image sensor 15. The predetermined time interval is an interval sufficient to monitor the fine periodic movement of the object 20, that is, the breathing of the sleeping person 2, and is, for example, about 2 to 3 frames / second. In the figure, the image sensor 15 and the image processing unit 14 are illustrated as separate bodies, but may be configured as a single unit.
[0023]
In addition, when a substantially single wavelength light source such as a laser is used for the light projecting unit 12 as in the present embodiment, the imaging unit 11 reduces light of wavelengths other than the peripheral portion of the approximately single wavelength. A light filter 11b may be provided. The filter 11b is typically an optical filter and may be disposed on the optical axis of the imaging lens 11a. If it does in this way, since the intensity of the light irradiated from the light projection part 12 will rise relatively among the light received by the image pick-up element 15, the imaging part 11 can reduce the influence by disturbance light.
[0024]
The FG element will be described with reference to FIG. The FG element is a structure in which about 100 optical fibers having a diameter of several tens of microns and a length of about 10 mm are arranged in a sheet, and the two fibers are overlapped so that they are orthogonal to each other. In the FG element 13, the above-described sheet is arranged in parallel to the plane 21 (perpendicular to the Z axis). Laser light L1 is incident on the FG element 13 in the Z-axis direction. Then, the laser beam L1 is condensed at the focal point of each optical fiber, and then becomes a diffraction beam array. A pattern 13a, which is a bright spot matrix, is projected in a square lattice pattern on the plane 21 that is a projection surface.
[0025]
In the above description, the imaging apparatus 10 is described using the FG element 13 as a diffraction element, but a two-dimensional microlens array may be used as the diffraction element.
[0026]
The two-dimensional microlens array 16 (hereinafter simply referred to as the lens array 16) will be described with reference to the schematic diagram of FIG. (A) is a perspective view, (b) is the fragmentary sectional view which showed the cross section in the left half of the front view. For example, the lens array 16 has a plurality of lenses 17 formed on one side of a plate with a period of several tens of microns. Here, as the lens array 16, a surface 18 (hereinafter, simply referred to as a lens surface 18) on which the plurality of lenses 17 of the lens array 16 are formed is used. The high reflection treatment is typically an aluminum coating 18a. In this way, since the lens 17 has the same configuration as the concave mirror, the lens array 16 can be easily shortened in focus.
[0027]
Further, the lens array 16 may be provided with a plane mirror immediately after the lens surface 18 instead of applying an aluminum coating to the lens surface 18. In this way, since the light enters the lens 17 twice, the focal length of the lens array 16 can be easily reduced.
[0028]
In this case, as shown in the schematic diagram of FIG. 6, the lens array 16 includes a beam splitter 19 that reflects or transmits light according to an incident angle of incident light. Laser light L1 is incident on the lens array 16 in the direction perpendicular to the Z axis. Then, the laser beam L 1 is reflected by the beam splitter 19 and enters the lens array 16. The incident laser light L1 is reflected by the lens surface 18 of each lens 17, collected at the focal point of the lens 17, and then transmitted through the beam splitter 19, spreads as a spherical wave, interferes, A pattern 16a, which is a bright spot matrix, is projected in a square lattice pattern on the plane 21, which is the projection surface.
[0029]
As shown in FIG. 7, the lens array 16 may be provided with two identical lens arrays 16 with the lens surface 18 in proximity. In this case, the lens surface 18 is used without the aluminum coating 18a. Laser light L1 is incident in the Z-axis direction on a lens array 16 'in which the two lens arrays 16 are close to each other. Then, the laser beam L1 is condensed at the combined focal point of each of the two lenses 17, and then spreads as a spherical wave, interferes, and shines in a square lattice pattern on the plane 21 which is the projection surface. A pattern 16a ′ is projected.
[0030]
In the lens array 16, it is desirable that the gap between adjacent lenses 17 is as small as possible. Therefore, if the lenses 17 can be arranged in an orthogonal lattice shape, the aperture shape should be rectangular rather than circular.
[0031]
The imaging apparatus 10 can further use a low-power laser by using the lens array 16 with high wavefront accuracy as described above for the light projecting unit 12.
Hereinafter, the case where the FG element 13 is used will be described.
[0032]
Returning to FIG. 3, the imaging device 10 will be further described. In the pattern 13 a projected onto the plane 21 by the FG element 13, the portion where the object 20 exists is blocked by the object 20 and does not reach the plane 21. Here, if the object 20 does not exist, the bright spot to be projected onto the point 21 a on the plane 21 is projected onto the point 20 a on the object 20. Since the bright spot is moved from the point 21a to the point 20a and the imaging lens 11a and the FG element 13 are separated by a distance d, the point 21a ′ (x, y) is formed on the imaging plane 15 ′. Is imaged at a point 20a ′ (x, y + δ). That is, the bright point moves by a distance δ in the y-axis direction between the time point when the object 20 does not exist and the time point when the object 20 exists.
[0033]
The image processing unit 14 can generate a differential image with two bright points moved by generating a differential image of two frames received (imaged) on the imaging surface 15 ′ at these two time points. Further, this difference image is obtained by taking the difference between the images of the two frames, so that the bright spots that did not move are removed together with the background, and the moved bright spots are extracted. The two frames here may be the latest imaged frame and a frame at a past time point from this frame. For example, the two frames are shifted by one frame.
[0034]
Thus, since the imaging device 10 generates a difference image, the influence of disturbance light can be eliminated. For example, even if shadows caused by objects other than the sleeping person 2 are applied to the sleeping person 2 due to sunlight or the illumination intensity due to ambient light varies in the portion of the sleeping person 2, such shadows or Since the influence of variation can be eliminated, it is easy to perform the binarization process described later. The difference image can be obtained by A / D converting the image signal and storing it in the frame memory, and calculating the corresponding image data. In particular, this feature is remarkable when a moving object detection element is used as the imaging element 15. The moving object detection element has a function of storing a pixel value of a frame at each pixel of the image sensor 15, for example, and outputting a value by thresholding the difference from the pixel value of the latest frame shifted by one frame or the difference. With this element, it is possible to generate a difference image without being affected by noise in the signal transmission process.
[0035]
Further, the image processing unit 14 performs binarization processing on the difference image. In the binarization process, for example, when the above-described moving body detection element is used, the output value of each of the two frames including the bias is peak-held at each pixel of the image of two frames, and a comparator (not shown) is used. For example, threshold processing is performed such that 1 is output when the absolute value of the difference between pixels is equal to or greater than the threshold, and 0 is output when the absolute value is less than the threshold. If it does in this way, a difference calculation can be performed using SN of an image sensor fully. For this reason, detection with a wider dynamic range than normal image processing (for example, quantizing pixel data into 8 bits) can be performed. Thereby, the imaging device 10 can easily detect a slight change in the pattern without using a high-power laser or performing a short exposure (imaging) by synchronizing the shutter with the light emission. As a result, the laser light L1 may be a low-power laser, and it is not necessary to synchronize the light emission and exposure of the laser and can be continuously irradiated.
[0036]
Further, the laser light L1 is pulsed light that is intermittently irradiated with light, and the imaging unit 11 may include a shutter (not shown) and may be configured to intermittently image on the imaging plane 15 ′. . In this case, the light projecting unit 12 may use a pulse light source in which the light source emits light for a predetermined light emission time, or may be configured to repeat light shielding and irradiation in a pulse shape with a shutter. The shutter is preferably an electronic shutter, but may be a mechanical shutter.
[0037]
With reference to the chart of FIG. 8, the relationship between intermittent irradiation of the light L1, that is, illumination of the plane 21, and intermittent imaging will be described. In both (a), (b), and (c), the horizontal axis is the time t-axis. The vertical axis of (a) indicates the illumination intensity of the patterned illumination light that has passed through the FG element 13, the vertical axis of (b) indicates the illumination intensity due to disturbance light, and the vertical axis of (c) indicates the combined illumination intensity due to both.
[0038]
The illumination light shown in (a) is pulsed light that lasts for a pulse duration t1. On the other hand, as shown in (c), the imaging of the imaging unit 11 is performed by opening the shutter for the exposure time t2. In the figure, the exposure time t2 is shown to be slightly longer than the pulse duration t1. The exposure time may be the same exposure time t2 'as shown, which is shorter than the pulse duration t1. For example, the exposure time t2 'may be 0.01 ms and the pulse duration t1 may be 0.05 ms. However, preferably they are almost the same.
[0039]
The duration (pulse width) t1 of the pulsed light is in the range of about 1 / 100,000 to 0.1 seconds, but is preferably 0.03 seconds or less which is the video rate.
[0040]
The timing of illumination with pulsed light and the timing of imaging are synchronized. That is, in FIG. 8, the pulsed light irradiation start time (starting point of time t1) and the exposure starting time (starting point of time t2) are made the same. As described above, the synchronization is preferably performed so that the irradiation start and the exposure start occur at the same time, but may be slightly shifted. During this time t2 or t2 ', an image is formed on the imaging surface 15'.
[0041]
Thereby, since the imaging device 10 can relatively weaken the intensity of disturbance light, the influence of the disturbance light can be suppressed, and since the pulse light is used, the absolute amount of the light amount can be suppressed low. Therefore, even when the imaging target is a person (sleeping person 2), adverse effects on the eyes can be minimized. In other words, since the exposure energy of disturbance light relative to the exposure energy of illumination light is relatively reduced, the influence of disturbance light can be reduced by performing bright illumination. In addition, a large illuminating means with problems of heat generation and large energy consumption is not necessary. In addition, the above threshold processing is facilitated, and at the same time, it is very easy to evaluate the moving amount of the bright spot that has moved.
[0042]
Returning to FIG. 2, the monitoring device 1 will be further described. A storage unit 34 is connected to the control unit 31 and can store data such as a difference image input from the imaging device 10 and calculated information. The arithmetic device 30 is configured to store the difference image input from the imaging device 10 via the interface 35 in the storage unit 34 in time series. Further, in the storage unit 34, a breathing pattern storage unit 36 that stores the normal breathing pattern and the abnormal breathing pattern of the sleeping person 2 is provided. The breathing pattern will be described later with reference to FIG.
[0043]
Further, an input device 37 for inputting information for operating the monitoring device 1 and an output device 38 for outputting a result processed by the monitoring device 1 are connected to the control unit 31. The input device 37 is, for example, a touch panel, a keyboard, or a mouse, and the output device 38 is, for example, a display, a printer, or an alarm device. Although the input device 37 and the output device 38 are illustrated as externally attached to the arithmetic device 30 in this figure, they may be built in.
[0044]
The control unit 31 includes a calculation unit 32 as a calculation unit that obtains a change output related to a change between two frames from the difference image input from the imaging device 10. Furthermore, the calculating part 32 is provided with the accumulation part 32a as an accumulation means which accumulates each pixel value of a difference image. The change output is typically a cumulative value of the pixel value. In other words, the change output represents the shape change in the monitoring target area obtained from the difference image. This is, for example, extracting movements of the sleeper 2 such as breathing. Thereby, the extracted movement of the sleeping person 2 forms a waveform pattern.
[0045]
The computing unit 32 obtains a change output by accumulating each pixel value of the difference image by the accumulating unit 32a. In this way, when the sum of the pixel output values of the difference image is simply taken, the sum is the smallest at the transition timing of expiration and inspiration, and the sum is largest during the expiration / inhalation. Conceivable. Since the difference image is input as a binarized image, as described above, if the output of the pixel whose absolute value of the difference is equal to or greater than the threshold is 1, the cumulative value is obtained by counting the number. I want. A change output can be obtained by arranging the accumulated values in time series.
FIG. 9 shows a waveform pattern of change output obtained from the accumulated value of each pixel value.
[0046]
Moreover, you may comprise the calculating part 32 so that the pattern movement amount of each moving part of the pattern on a difference image may be calculated as a change output. At this time, the pattern movement amount is calculated by distinguishing between positive and negative according to the movement direction of the pattern. Accordingly, the respiration rate can be detected by measuring the number of “zero crosses” (intersections where the sign is inverted) appearing between the inspiration pattern and the expiration pattern by detecting the respiration rate by the detection unit 33 described later. The pattern movement amount may be calculated by calculating the pattern movement amount in the axial direction connecting the light projecting unit 12 and the imaging unit 11 on the difference image.
[0047]
Here, the pattern movement amount will be described with reference to the schematic diagram of FIG. The x'y 'axis in the figure corresponds to the xy axis in FIG. That is, the y′-axis direction on the difference image is parallel to the axial direction connecting the light projecting unit 12 and the imaging unit 11. As shown in the figure, the input differential image is an image in which bright spots that have not moved are removed together with the background, and the bright spots that have moved are extracted. Here, it is assumed that the moved bright spot A has moved by a distance δ in the y′-axis direction.
[0048]
The monitoring device 1 can detect the movement of the sleeping person 2 by the detection unit 33 described later by following the change of δ. Further, if the pitch of the pattern 13a, that is, the pitch of the bright spots is made fine enough that the correspondence of the bright spots is not unknown, the movement of the sleeping person 2 can be detected in detail.
[0049]
At this time, the calculation unit 32 may calculate the distance δ as the pattern movement amount. This is because when the imaging apparatus 10 is installed as in the present embodiment, the vertical movement caused by the sleep of the sleeper 2 appears in the movement of the pattern in the y′-axis direction. Also, by doing so, the calculation unit 32 does not have to calculate the pattern movement amounts in both the x′-axis direction and the y′-axis direction, so that the calculation amount can be greatly reduced.
[0050]
In addition, if the pattern movement amount is picked up in the order of no light emission, light emission, and light emission by the imaging device 10, and each difference image is generated, the position of the original bright spot can be taken in the first difference image, and the next difference image can be obtained. Only moved bright spots can be taken. Therefore, it is possible to know where the bright spot has moved from where by using the original bright spot position. That is, the moving direction of the bright spot (pattern) is known.
[0051]
When calculating the pattern movement amount, the calculation unit 32 obtains a change output by accumulating the pattern movement amount of each movement part calculated above. A change output can be obtained by arranging the movement amount accumulated values in time series. In addition, the calculation unit 32 can discriminate between expiration and inhalation by detecting respiration by the detection unit 33 to be described later by accumulating the positive and negative.
FIG. 11 shows a waveform pattern of change output of the movement amount accumulated value.
[0052]
Further, the calculation unit 32 may obtain a change output by accumulating the absolute value of the pattern movement amount of each moving part calculated above. In this way, by making the movement amount cumulative value an absolute value, it becomes impossible to discriminate expiration / inspiration by detection of respiration by the detection unit 33 described later, but the plus / minus cancels and the sensitivity to movement decreases. Can be prevented.
[0053]
In addition, the calculation unit 32 may obtain a change output by accumulating the pattern movement amount having the larger one of the positive and negative movement portions among the pattern movement amounts of the respective movement portions calculated above. In this way, while knowing the sign of the pattern movement amount, it can be prevented that the plus and minus cancel each other and the sensitivity to movement is reduced.
[0054]
In addition, the calculation unit 32 subtracts the accumulation of the smaller pattern movement amount from the accumulation of the pattern movement amount of the larger one of the positive and negative movement portions with the sign among the pattern movement amounts of the respective movement portions calculated above. The change output may be obtained by calculating the calculated value. In this way, the positive / negative and plus / minus of the pattern movement amount are not canceled out, and the sensitivity to movement can be increased.
[0055]
In addition, the calculation unit 32 compares the phases of the pattern movement amounts of the respective moving parts calculated as described above, and groups those that are close in phase by this comparison, thereby obtaining the sum of the pattern movement amounts of each group. . And you may make it subtract the sum total of each group between the groups near an antiphase. In this way, each of the pattern movement amounts that are close in phase is grouped and the sum is obtained, so that, for example, the sleep of the sleeping person 2 can be extracted and amplified as a group. Furthermore, between the groups close to the opposite phase, the sum of each group is subtracted, and the movement of the sleeping person 2 is detected based on the value obtained by the subtraction. Even if there is a part that falls, by calculating the difference, it is possible to amplify the respiration pattern amplitude and to detect respiration reliably.
[0056]
Further, the calculation unit 32 may be configured to time-integrate the pattern movement amount for each pixel. In this way, the value obtained by time integration becomes a value similar to the absolute height. If it does in this way, the value similar to the absolute value of the height of the sleeping person 2 can be obtained as a change output. In this case, the calculation unit 32 can obtain a change output by time-integrating the pattern movement amount of each moving part calculated above for each pixel. Further, the calculation unit 32 may obtain a change output by accumulating the time integration value calculated for each pixel in this way. A change output can be obtained by arranging the movement amount accumulated time integral values in time series.
FIG. 12 shows a waveform pattern of change output of the movement amount time integrated value.
[0057]
As described above, the calculation unit 32 can obtain a waveform pattern of a change output regardless of the configuration described above. In addition, the configuration of the calculation unit 32 described above may be appropriately selected depending on the installation state, use state, purpose, and the like of the device. Moreover, the calculating part 32 should just be able to obtain the waveform pattern of the change output in which the motion of the sleeper 2 was reflected, and is not restricted to the structure demonstrated above.
[0058]
Further, the control unit 21 is provided with a detection unit 33 as detection means for detecting the movement of the sleeping person 2 from the time change of the change output. The detection unit 33 is configured to detect the respiration and the respiration rate of the sleeper 2 when the change output has a periodic fluctuation. The detection unit 33 is configured to consider that there is a motion other than respiration when the change output is not a periodic fluctuation. Furthermore, the detection unit 33 may be configured to consider movement other than breathing when the absolute value of the change output exceeds a certain value.
[0059]
Further, when the change output is the above-described cumulative value (see FIG. 9), the detection unit 33 detects the respiration rate by taking the two cycles as one breath when the change output has a periodic fluctuation. When the change output is the aforementioned movement amount accumulated value or movement amount accumulated time integral value (see FIGS. 11 and 12), the detection unit 33 determines that the change output has a periodic variation. The respiratory rate is detected with one cycle as one breath.
[0060]
As described above, the detection unit 33 detects the movement of the sleeping person 2 such as breathing. The respiratory pattern detected by the detection unit 33 is, for example, when the period of the respiratory pattern is disturbed in a short period of time or when the period of the respiratory pattern changes abruptly, for example, spontaneous pneumothorax, bronchial asthma It can be presumed that the disease is a pulmonary disease such as congestive heart failure, or a cerebrovascular disease such as cerebral hemorrhage. Moreover, when the disappearance of the breathing pattern continues, it can be estimated that the sleep of the sleeper 2 has stopped. When the sleeper 2 moves frequently instead of the breathing pattern in a short time, it can be inferred that the sleeper 2 suffers for some reason and is rampant.
[0061]
Furthermore, the detection unit 33 is configured to determine the state of the sleeping person 2 from the detected movement of the sleeping person 2. Thereby, the detection part 33 can determine that the sleeper 2 is in a dangerous state in the above cases.
[0062]
In addition, the movement of the sleeper 2 other than breathing can be easily detected because the waveform pattern fluctuates much more than when only the breathing is detected. In this case, the detection unit 33 can also detect from the change output whether the sleeping person 2 is moving on the spot such as turning over or moving up such as getting up from the bed. Further, even when the sleeper 2 makes periodic and small movements such as convulsions, an abnormality can be determined from the waveform pattern. Furthermore, it is possible to determine that the sleeper 2 is in a state of spasm by storing the waveform pattern in the state of spasm in the storage unit 34.
[0063]
The detection unit 33 may determine that the area where the sleeper 2 is present on the bed 6 is in a state of being extremely close to one of the ends of the bed 6. Thereby, the monitoring apparatus 1 can determine the situation where the sleeper 2 is in a dangerous position where the bed 2 falls from the bed 6.
[0064]
Here, an example of normal and abnormal breathing patterns will be described with reference to FIG. The normal breathing pattern stored in the breathing pattern storage unit 36 in the storage unit 34 is a periodic pattern as shown in FIG. Abnormal breathing patterns stored in the breathing pattern storage unit 36 are physiologically internal such as, for example, Chain-Stokes breathing, central hyperventilation, ataxic breathing, and Kasmaul breathing. It is a breathing pattern that is thought to occur when there is a disorder.
[0065]
FIG. 13B shows a respiratory pattern of Cheyne-Stokes breathing, FIG. 13C shows a respiratory pattern of central hyperventilation, and FIG. 13D shows a respiratory pattern of ataxic breathing.
Furthermore, FIG. 14 shows a disease name or a disease location when the above-described abnormal breathing pattern occurs.
[0066]
The detection unit 33 determines which breathing pattern the sleeping person 2 belongs to using the fact that the breathing frequency, the number of appearances, and the depth of each breathing pattern are different, and determines the state of the sleeping person 2. judge.
[0067]
Furthermore, when the detection unit 33 determines that the respiration of the sleeping person 2 belongs to a respiration pattern that is considered to occur in a hot water where a disorder occurs in the body physiologically, the awakening of the sleeping person 2 is abnormal. It is determined that it is in a dangerous state. The state of the sleeper 2 determined as described above is output from the output device 38 by the control unit 31. The output contents include the determined respiratory rate and frequency of movement of the sleeper 2, names of abnormal breathing patterns, disease names considered to cause the breathing, diseased organs, diseased locations, and the like.
[0068]
Here, the report device 40 for reporting the determination result of the state of the sleeping person 2 by the detection unit 33 to the outside will be described. The reporting device 40 reports the determination result of the state of the sleeping person 2 by the detection unit 33 to the outside. In the notification device 40, the state of the sleeping person 2 determined by the detection unit 33, information on sound acquired from a microphone (not shown) provided in the bed 6, and the like, a nurse station, a living room, a dining room, another bedroom, etc. A wireless communication device using wireless communication for delivery to another location, and a wired communication device using a power line or a dedicated line. Information sent by these communication devices can be used to send a message informing an emergency, make an alarm sound, or emit a light signature. These information, messages, and the like can also be sent to a person or apparatus at a remote location through a fixed telephone line, a data communication line, a CATV line, a mobile telephone line, or the like.
[0069]
Moreover, the monitoring apparatus 1 may be provided with a pressure-sensitive switch in the bedding laid under the sleeping person as an auxiliary means for accurately detecting whether or not the sleeping person 2 is present. By turning this switch on / off, the presence / absence of a sleeping person can be easily determined.
[0070]
In the above, the monitoring device 1 has been described with respect to detecting the movement of the sleeping person 2 on the bed 6, but is not limited to this and is limited to a monitoring area such as a toilet or bathroom. In some cases, it works particularly effectively. For example, as shown in FIG. 15, when the monitoring device 1 is installed on the ceiling of a bathroom, it is possible to reliably detect the breathing and movement of a person taking a bath. Further, although the case where the light projecting unit 12 is provided has been described in the present embodiment, the same configuration can be implemented without the light projecting unit 12.
[0071]
According to the present embodiment as described above, the monitoring device 1 can reliably detect the sleeping person's breathing by simple image processing without being affected by the posture or disturbance light of the sleeping person 2. it can. Thereby, the monitoring apparatus 1 can implement | achieve a quick emergency response when an elderly person or a sick person falls into a critical situation.
[0072]
【The invention's effect】
As described above, according to the present invention, an imaging unit that continuously images a monitoring object illuminated with a constant light amount, and a difference image between two frames of images obtained by the imaging unit at a predetermined time interval are generated. A difference image generating means for performing, a calculating means for obtaining a change output relating to the change between the two frames from the difference image, and a detection means for detecting the movement of the monitoring object from the time change of the change output. It is possible to provide a monitoring device that not only reliably detects a person's condition but also is small and simple.
[Brief description of the drawings]
FIG. 1 is a schematic perspective view showing an outline of a monitoring apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration example of a monitoring device used in the embodiment of the present invention.
FIG. 3 is a conceptual perspective view illustrating a configuration example of an imaging apparatus used in an embodiment of the present invention.
FIG. 4 is a conceptual perspective view for explaining an FG element used in an embodiment of the present invention.
5A is a schematic perspective view and FIG. 5B is a schematic partial cross-sectional view for explaining a two-dimensional microlens array used in an embodiment of the present invention.
6 is a conceptual perspective view for explaining a case where high reflection processing is applied to the two-dimensional microlens array in the case of FIG.
7 is a conceptual perspective view illustrating a case where two two-dimensional microlens arrays in the case of FIG. 5 are used.
FIG. 8 is a chart for explaining the relationship between intermittent irradiation of illumination light and intermittent exposure.
FIG. 9 is a schematic diagram showing a waveform pattern of change output obtained from a cumulative value of each pixel value of a difference image.
FIG. 10 is a schematic diagram illustrating a case where a pattern movement amount is calculated from a difference image.
FIG. 11 is a schematic diagram showing a waveform pattern of a change output obtained from a movement amount accumulated value of a pattern movement amount.
FIG. 12 is a schematic diagram showing a waveform pattern of a change output obtained from a movement amount accumulated time integral value of a pattern movement amount;
FIG. 13 is a schematic diagram showing waveform patterns of normal and abnormal breathing.
FIG. 14 is a diagram showing a table of disease names or disease locations corresponding to abnormal respiratory waveform patterns.
FIG. 15 is a schematic perspective view showing an outline when the monitoring apparatus according to the embodiment of the present invention is installed in a bathroom.
FIG. 16 is a schematic diagram illustrating an example of an illumination pattern.
[Explanation of symbols]
1 Monitoring device
2 Sleeper
3 Bedding
6 beds
10 Imaging device
11 Imaging unit
11b filter
12 Projector
13 FG element
13a Grating pattern
14 Image processing unit
15 Image sensor
16 Two-dimensional microlens array
30 arithmetic unit
31 Control unit
32 Calculation unit
33 detector
34 Memory
35 interface
36 Breathing pattern storage
37 Input device
38 Output device
40 Reporting device

Claims (5)

Imaging means for continuously imaging a monitoring object illuminated with a constant amount of light;
Differential image generation means for generating a differential image of two frames of images at predetermined time intervals obtained by the imaging means;
Computing means for obtaining a change output relating to a change between the two frames from the difference image;
Detecting means for detecting a movement of the monitoring object due to respiration from a time change of the change output ;
Illumination pattern light projecting means for illuminating the monitored object with a predetermined pattern illumination light;
The patterned illumination light has an illumination distribution in a direction parallel to a direction connecting the illumination pattern light projecting unit and the imaging unit ,
The computing means includes accumulating means for accumulating each pixel value of the difference image ,
The accumulating means arranges accumulated accumulative values in time series,
Monitoring device. The detection means detects the respiratory rate from two cycles as one breath from the periodic variation of the change output,
The monitoring apparatus according to claim 1. A breathing pattern storage unit that stores a normal breathing pattern and an abnormal breathing pattern of the monitoring target;
The detection means determines whether or not the breathing pattern of the monitoring target belongs to the abnormal breathing pattern from the breathing frequency, the number of appearances, and the depth of the breathing pattern indicated by the detected breathing movement.
The monitoring apparatus according to claim 1 or 2. An output device for outputting the state of the monitored object;
The output device outputs any one of a disease name, a diseased organ, and a diseased part considered to be the cause of abnormal breathing, together with the respiratory rate of the monitored object, the frequency of movement, and the name of an abnormal breathing pattern.
The monitoring device according to claim 3. The imaging means may have a sequence is an imaging device of a plurality of pixels,
The imaging element is a moving object detection element.
The monitoring apparatus according to any one of claims 1 to 4 .

Images