JP2013078433A - Monitoring device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a monitoring device allowing accurate and reproducible detection of movement of a person that is a monitoring target by automatically detecting an area to be monitored with a bed as a reference.SOLUTION: A range imaging sensor 10 generates a range image wherein a pixel value is a range value to an object. A visual field area of the range image sensor 10 includes the entirety of the bed that is a monitoring target. A bed recognition unit 21 uses the range image outputted by the range image sensor 10 to extract a position of the bed. Within the range image outputted by the range image sensor 10, a person recognition unit 22 detects areas occupied by the person inside and outside a range of the bed recognized by the bed recognition unit 21. A movement decision unit 23 distinguishes the movement of the person to the bed by a combination between the area of the bed detected by the bed recognition unit 21 and the area of the person detected by the person recognition unit 22.

Description

  The present invention relates to a monitoring device for monitoring a person on a bed and a program for realizing a function used for the monitoring device by a computer.

  As this type of monitoring device, a technique has been proposed in which a distance-related value corresponding to the distance to the monitoring object is obtained, and the distance-related value is compared with a threshold value to detect a sleeper's movement and breathing on the bed. (For example, see Patent Document 1). Patent Document 1 describes a technique for detecting bed removal by determining that the monitoring target has moved out of the monitoring target area when a change in the shape of the sleeping person is not detected within a predetermined time.

  In Patent Document 1, a plurality of target points including a part of a sleeping person on a bed are set, and a distance from the sleeping person information to the target point is measured by a distance sensor. A plurality of target points are arranged in a monitoring area set so as to include a part of the sleeping person when the sleeping person is on the bed. In the technique described in Patent Literature 1, breathing and getting up are detected in addition to bedtime of a sleeping person by using a time change in distance-related values obtained from these target points and the position of the target point.

JP 2003-290154 A

  By the way, in the technique for monitoring the state of a sleeping person on a bed, a distance sensor is necessary for performing three-dimensional measurement, and various principles of this type of distance sensor are known. However, as a distance sensor that detects a sleeping person's state on the bed that changes with time, a distance sensor based on a stereo image method and a distance sensor based on a time-of-flight method are mainly employed.

  A distance sensor using the stereo imaging method requires two or more imaging devices, and the processing for images is relatively complicated, so that it is difficult to supply to the market at a low price. On the other hand, the distance sensor using the time-of-flight method has a problem that an error occurs when the energy (for example, light) radiated into the target space is reflected by multiple surrounding objects. For example, when a curtain around the bed is placed around the bed, such as a multi-bed room in a hospital, elderly facility, nursing facility, etc., the flight time of the reflected wave may change due to opening and closing of the curtain around the bed There is sex.

  The present invention reproduces three-dimensional information by reducing the influence of multiple reflections caused by the surrounding environment of the bed when measuring three-dimensional information using the time-of-flight method to monitor a subject on the bed. An object of the present invention is to provide a monitoring device for detecting with good quality and a program for realizing the function of the monitoring device by a computer.

  In order to achieve the above object, the monitoring device according to the present invention generates a distance image in which a pixel value is a distance value to an object using a time difference between light projection and reception, and includes a bed to be monitored in a visual field region. A distance image sensor placed in the bed, a bed recognition unit that recognizes the position of the bed using the distance image, a behavior monitoring processing unit that monitors a specific behavior of a person related to the bed, and a distance image sensor generated in advance The variation rate of the distance value for the pixel at the same position of the background image obtained by the distance image sensor and the distance image generated by the range image sensor is calculated, and the representative value of the variation rate obtained from the pixel whose variation rate is within the reference range is used. A distance fluctuation correction unit that gives a distance image whose distance value is corrected to the behavior monitoring processing unit, and the behavior monitoring processing unit uses the distance image whose distance value is corrected by the distance fluctuation correction unit to perform a specific action of a person Characterized in that it monitors.

  In this monitoring apparatus, it is preferable that the bed includes an observation board disposed above the footboard, and the distance fluctuation correction unit calculates the fluctuation rate using pixels in a region corresponding to the observation board.

  In this monitoring apparatus, it is preferable that the distance variation correction unit calculates the variation rate using the pixels within the bed range extracted by the bed recognition unit.

  In this monitoring apparatus, the bed includes a plurality of markers arranged at the center position in the left-right direction on the footboard and at other positions on the footboard, and the bed recognition unit uses the position of the marker as a reference. Is preferably calculated.

  In this monitoring device, the bed includes three or more markers arranged at the center position in the left-right direction on the footboard and both side edges of the bed, and the bed recognition unit determines the position of the bed using the marker position as a reference. It is preferable to calculate.

  In this monitoring apparatus, the behavior monitoring processing unit automatically sets a threshold value related to the height defined above the bed in order to monitor a specific behavior using an index indicating a physique given to a person using the bed. It is preferable to do.

  The program according to the present invention generates a distance image in which a pixel value is a distance value to an object using a time difference between light transmission and reception, and a distance image arranged so that a bed to be monitored is included in a visual field region A behavior monitoring processing unit that monitors a specific behavior of a person related to the bed, and a background distance generated in advance by the distance image sensor, including a bed recognition unit that extracts a bed position using a distance image obtained from the sensor Calculate the rate of change of the distance value for the pixel at the same position of the image and the range image generated by the range image sensor, and use the representative value of the rate of change obtained from the pixel whose rate of change is within the reference range. This is to make the corrected distance image function as a distance variation correction unit that gives the behavior monitoring processing unit.

  According to the configuration of the present invention, when measuring three-dimensional information using the time-of-flight method to monitor the subject on the bed, the influence of multiple reflections caused by the surrounding environment of the bed is reduced. There is an advantage that the dimensional information can be detected with good reproducibility. In other words, even in an environment where multiple reflections occur due to the principle of time-of-flight method, three-dimensional information is accurately measured, and as a result, human recognition and human behavior related to the bed are accurately performed. There is an effect that becomes possible.

It is a block diagram which shows the whole structure of this invention. (A) is a perspective view which shows the example of use same as the above, (b) is a perspective view which shows the example of a structure same as the above. It is a block diagram which shows the distance image sensor used for the same as the above. It is principle explanatory drawing of the coordinate transformation in the same as the above. It is principle explanatory drawing of the coordinate transformation in the same as the above. It is a perspective view which shows the other usage example same as the above. It is a block diagram which shows the distance fluctuation correction part used for the same as the above. It is a perspective view which shows the example using the observation board same as the above. It is operation | movement explanatory drawing corresponding to FIG. 8 in the same as the above. It is operation | movement explanatory drawing of the distance fluctuation correction part used for the same as the above. It is a perspective view which shows the example of 1 arrangement | positioning of a marker in the same as the above. It is principle explanatory drawing which shows an example of the process which uses a marker in the same as the above. It is a perspective view which shows the other example of arrangement | positioning of a marker in the same as the above. It is principle explanatory drawing which shows the other example of the process using a marker in the same as the above. It is operation | movement explanatory drawing when a visual field area | region is not appropriate in the same as the above. (A) is explanatory drawing of the division area | region in the same as the above, (b) is explanatory drawing of the bed surface in the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is a perspective view which shows the example which an object overlaps with a bed in the same as the above. It is operation | movement explanatory drawing which shows the example of a setting of a mask area | region in the same as the above. The example which an object overlaps with a bed in the same as the above is shown, (a) is a plan view, (b) is a side view. It is a figure which shows the example of a setting of the detection surface in the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is a figure which shows the relationship between a physique and a threshold value in the same as the above. It is operation | movement explanatory drawing same as the above.

  In the embodiments described below, a person's behavior on a bed is monitored in a place equipped with a curtain around the bed (so-called cubicle curtain) such as a bed placed in a multi-bed room such as a hospital, an elderly facility, or a care facility. An example of using a monitoring device to do this will be shown. However, the same technology can be used in a normal room with windows and doors.

  As shown in FIG. 2, the monitoring device of the present embodiment includes a distance image sensor 10 that outputs a distance image including the entire bed 30 in the field of view. A distance image is an image with a pixel value as a distance value, and there are two types of technologies for generating a distance image: a passive type that only receives light from a target space and an active type that receives light projected into the target space. Are known. The distance image sensor described below is an active type that employs the principle of time-of-flight (Time Of Flight) that measures the time from when light is projected into a target space and when reflected light from an object existing in the target space is received. Use. Hereinafter, the time-of-flight method is abbreviated as “TOF method”.

  The present embodiment may be configured in any manner as long as it is a distance image sensor using the TOF method. However, in the present embodiment, the intensity modulation light obtained by modulating the light intensity with a modulation signal having a constant period is projected onto the target space, and the time until the reflected light from the object existing in the target space is received is expressed as the intensity. A configuration for detecting the phase difference between light projection and reception of modulated light is adopted. This distance image sensor has a function of converting a phase difference between light projection and reception of intensity-modulated light into a distance to an object.

  The light projected into the target space is preferably light that is reflected by the surface of the object without passing through many objects and is not perceived by humans. For this reason, it is desirable to use near infrared rays as the light to be projected. However, visible light can be used when it is desirable to be perceived by a person, such as when adjusting the imaging region.

  The waveform of the intensity-modulated light is assumed to be a sine wave, but a triangular wave, a sawtooth wave, a square wave, or the like can be used. When a sine wave, a triangular wave, or a sawtooth wave is used, the period of the intensity modulated light is set to a constant period. In addition, when using a square wave, in addition to making the period of intensity modulated light constant, the ratio of the on period (light emitting source light emitting period) to the off period (light emitting source non-light emitting period) is randomized. It is also possible to adopt changing technology. That is, the on period and the off period are irregularly changed so that the probability of the on period occurring is 50% in a sufficiently long time with respect to the on period and the off period, and accumulated in the sufficiently long time. The amount of received light may be used.

  In addition, when the intensity-modulated light has a constant period, for example, it is preferable to reduce the influence of shot noise by modulating the light to be projected with a modulation signal of 20 MHz and accumulating the received light amount of about 10,000 periods. Even when the on period and the off period are changed randomly, for example, the unit period is a period corresponding to one cycle of 20 MHz, and the on period and the off period are changed within a range of several times the unit period. It is preferable to accumulate the amount of light received for a period of about 10,000 times the unit period. With this operation, the amount of received light after accumulation can be handled in the same manner as when the amount of received light is accumulated using intensity-modulated light having a fixed period.

  The intensity-modulated light reflected by the object is received by the imaging device. The imaging device includes an imaging element for capturing a grayscale image in which a plurality of pixels are two-dimensionally arranged, and a light receiving optical system that limits a range in which light enters the light receiving surface of the imaging element. As the image pickup device, an image pickup device having a well-known configuration for picking up a gray image provided as a CCD image sensor or a CMOS image sensor can be used, but an image pickup device designed exclusively for having a structure suitable for a distance image sensor. It is desirable to use

  Hereinafter, the following configuration will be described as an example of the distance image sensor, but this configuration is not intended to limit the present invention, and the modulation waveform of the intensity-modulated light, the configuration of the imaging device, the control of the imaging device, etc. It is possible to replace various configurations known in the TOF type distance image sensor.

  As shown in FIG. 3, the distance image sensor 10 used in the following description includes a light projecting device 11 that projects intensity-modulated light onto a target space, and an imaging device 12 that receives light from the target space. The light projecting device 11 is a light emitting element 13 such as a light emitting diode or a laser diode that can obtain a light output proportional to the instantaneous value of the input, and light projecting optics that projects the light emitted from the light emitting element 13 into the range of the target space. A system 14. The light projecting device 11 includes an appropriate number of light emitting elements 13 in order to ensure light output. The imaging device 12 includes the above-described imaging element 15 and a light receiving optical system 16 that determines the field of view of the imaging element 15.

  The intensity-modulated light emitted from the light emitting element 13 is projected into the target space through the light projecting optical system 14. The imaging element 15 receives light from the target space through the light receiving optical system 16. The light projecting optical system 14 and the light receiving optical system 16 are arranged close to each other with the light projecting and receiving directions parallel to each other. Here, it is assumed that the distance between the light projecting optical system 14 and the light receiving optical system 16 can be substantially ignored with respect to the visual field region.

  The distance image sensor 10 includes a modulation signal generation unit 17 that generates a modulation signal to be applied to the light emitting element 13 in order to emit intensity modulated light from the light emitting element 13. In addition, the distance image sensor 10 includes a timing control unit 18 that generates a light reception timing signal that defines a light reception timing at the image sensor 15 from a modulation signal in order to control a timing at which the image sensor 15 receives light from the target space. The charge corresponding to the amount of received light obtained by the image sensor 15 is read from the image sensor 15 and input to the arithmetic processing unit 19. The arithmetic processing unit 19 obtains the distance to the object existing in the target space from the relationship between the light reception timing and the light reception amount.

  The modulation signal generator 17 generates a modulation signal whose output voltage changes with a sine waveform having a constant frequency (for example, 20 MHz). The light emitting element 13 is driven by this modulation signal, and intensity modulated light whose light output changes in a sine wave shape is emitted from the light emitting element 13.

  The imaging element 15 used in the present embodiment generates an electric charge according to the light reception intensity only during a period synchronized with the light reception timing signal by using an electronic shutter technique. Further, the generated charges are transferred to a light-shielded accumulation area, accumulated in an accumulation period corresponding to a plurality of periods (for example, 10000 periods) of a modulation signal in the accumulation area, and then received and output to the outside of the image sensor 15. As taken out.

  The timing control unit 18 generates a light reception timing signal synchronized with the modulation signal. Here, the timing control unit 18 generates four types of light reception timing signals having a light reception period of a certain time width for every four phases having different modulation signals. In addition, one type of light reception timing signal selected from the four types of light reception timing signals is supplied to the image sensor 15 for each of the above-described accumulation periods.

  That is, by supplying one type of light reception timing signal to the image sensor 15 in one accumulation period, charges in the light reception period corresponding to a specific phase period of the modulation signal are generated in each pixel of the image sensor 15. The accumulated charge is taken out from the image sensor 15 as a light receiving output. When each light reception timing signal that differs for each accumulation period is given to the image sensor 15 and the operation of taking out the electric charge generated by the image sensor 15 as a light reception output is repeated, light reception corresponding to four types of light reception timing signals in four accumulation periods. An output is obtained from the image sensor 15.

Now, four types of light reception timing signals are set to phases different by 90 degrees in one cycle of the modulation signal, and the light reception outputs (charge amounts) output from the image sensor 15 corresponding to the respective light reception timing signals are respectively A0. , A1, A2, A3. At this time, using the relationship of the trigonometric function, the phase difference ψ [rad] between when the intensity-modulated light is projected and received can be expressed in the form of the following equation.
ψ = tan ⁻¹ {(A0−A2) / (A1−A3)}
Since the frequency of the modulation signal is constant, the phase difference ψ can be converted into a time difference from light projection to light reception, and since the speed of light is known, the distance to the object can be obtained if the time difference is obtained.

  That is, the distance to the object can be obtained from the four types of light reception outputs (charge amounts) A0 to A3. Note that the light receiving period is appropriately set so that an appropriate amount of received light can be obtained in each pixel. For example, a light receiving period corresponding to a quarter cycle of the modulation signal is used. Further, the time widths of the respective light receiving periods need to be equal to each other.

  The arithmetic processing unit 19 obtains the phase difference ψ based on the light reception outputs (charge amounts) A0 to A3, converts the distance image by converting the distance, and extracts only the reflected light component of the intensity modulated light. Processing to generate a reflection intensity image that is a grayscale image is also performed.

  When generating a reflection intensity image, the modulation signal generation unit 17 operates to provide a light projection period in which the modulation signal is input to the light projection device 11 and a non-light projection period in which the modulation signal to the light projection device 11 is paused. To do. That is, intensity-modulated light is projected from the light projecting device 11 to the target space during the light projecting period, and light projection from the light projecting device 11 to the target space is suspended during the non-light projecting period. Accordingly, the amount of light received by the imaging device 12 during the light projection period includes the reflected light component and the ambient light component of the intensity-modulated light, and the amount of light received by the imaging device 12 during the non-light projection period includes only the environment light component. .

  Using the above-described relationship, the reflected light component of the intensity-modulated light can be extracted from the amount of light received during the light projection period and the amount of light received during the non-light projection period. That is, the arithmetic processing unit 19 calculates the reflected light component of the intensity-modulated light by appropriately using the light reception outputs A0 to A3, and generates a reflection intensity image having the calculated reflected light component as a pixel value. Since the reflection intensity image generated in this way has a pixel value corresponding to the reflection intensity of intensity-modulated light, it includes information on the reflectance of the object and the distance to the object. The reflected light component is obtained using the received light amount of any of the four types of received light outputs A0 to A3, the average received light amount of the four types of received light outputs A0 to A3, and the like.

  Whether the arithmetic processing unit 19 generates a distance image or a reflection intensity image is appropriately selected as necessary. Since the pixel positions of the distance image and the reflection intensity image are the same, the distance image and the reflection intensity image are used so that two types of the distance to the object and the reflectance of the object at the same position in the target space are used. Can be obtained.

  The arithmetic processing unit 19 is configured by using a digital signal processing device selected from a microcomputer, a DSP, an FPGA, and the like, and the above-described processing is realized by executing a program in the digital signal processing device. Further, not only the arithmetic processing unit 19 but also the configuration excluding the light emitting element 13 and the imaging element 15 can be realized by using the digital signal processing apparatus described above. Although only one digital signal processing device can be used, it is preferable to distribute the processing to different digital signal processing devices at least for the function used for the distance image sensor 10 and the function described below. It is also possible to use more digital signal processing devices.

  In the above-described operation example, four types of light reception timing signals are used. However, the phase difference ψ can be obtained using three types of light reception timing signals. It is possible to obtain the phase difference ψ even with the light reception timing signal.

  Furthermore, in the above-described operation, charges corresponding to one type of light reception timing signal are accumulated for one pixel, so that four types of light reception outputs (charge amounts) A0 to A3 are extracted four times from the image sensor 15. An accumulation period is required. On the other hand, if charges corresponding to two types of light reception timing signals are accumulated for one pixel, it is possible to read light reception outputs corresponding to the two types of light reception timing signals from the image sensor 15 at a time. Similarly, if the charge corresponding to the four types of light reception timing signals can be accumulated for one pixel, the light reception outputs corresponding to the four types of light reception timing signals can be read out once.

  Since the above-described distance image sensor 10 uses an imaging element in which a plurality of pixels are two-dimensionally arranged as a light receiving element for receiving light from the target space, a distance value is obtained as a pixel value of each pixel. As a result, a distance image is generated. In other words, the light receiving surface of the image sensor becomes a virtual projection surface that projects the visual field region of the distance image sensor 10.

  Furthermore, since the distance image sensor 10 projects light from the light emitting element 13 into the target space and images the field area of the imaging element 15 as the target space, the shape of the target space is from the distance image sensor 10 with the distance image sensor 10 as a vertex. The more you leave, the wider it will be. For example, when the light projecting optical system 14 and the light receiving optical system 16 are formed isotropically around the optical axis, the shape of the target space is a pyramid shape with the distance image sensor 10 as a vertex.

  Therefore, the positions of the pixels arranged on the virtual projection plane described above correspond to the direction in which the target space is viewed from the distance image sensor 10, and the pixel value of each pixel indicates the distance to the object existing in the direction. Will represent. In other words, the distance image generated by the distance image sensor 10 (and the image generating means) represents the position of the object in the polar coordinate system. Such a distance image is called a polar coordinate system distance image.

  The polar coordinate system distance image described above is highly convenient when information on the distance from the distance image sensor 10 is necessary, but the correspondence with each position in the real space that is the target space is difficult to understand and exists in the real space. It is inconvenient to specify a region based on the object to be used. Therefore, the arithmetic processing unit 19 performs coordinate conversion for generating an image having each coordinate value of the orthogonal coordinate system from the distance image of the polar coordinate system. Hereinafter, the image after the coordinate conversion is referred to as a coordinate axis-specific image.

  A procedure for generating the coordinate axis-specific image from the polar coordinate system distance image will be described with reference to FIGS. To generate an image for each coordinate axis from a polar coordinate system distance image, first, a polar coordinate system distance image set in the imaging device is converted into a three-dimensional image in a camera coordinate system, which is an orthogonal coordinate system set in the imaging device. Furthermore, this three-dimensional image is converted into a three-dimensional image in a global coordinate system which is an orthogonal coordinate system defined in the target space. When a three-dimensional image of the global coordinate system is obtained, it can be decomposed into images for each coordinate axis.

  In order to generate a three-dimensional image in the camera coordinate system from the distance image in the polar coordinate system, the following calculation is performed. Here, as shown in FIG. 4, the horizontal direction and the vertical direction on the light receiving surface of the image sensor 15 are defined as the u direction and the v direction. The image sensor 15 and the light receiving optical system 16 are arranged so that the optical axis 16A of the light receiving optical system 16 passes through the pixel at the center position of the light receiving surface of the image sensor 15. Further, the light receiving surface of the image sensor 15 is positioned at the focal point of the light receiving optical system 16. In this positional relationship, the coordinates (unit: pixel) of the pixel at the center position of the light receiving surface of the image sensor 15 are (uc, vc), and the pitch between the u direction and the v direction of the pixel of the image sensor 15 (unit: mm). ) Is (su, sv). Further, the focal length of the light receiving optical system 16 is f [mm], and light receiving optics is used for an object that exists in a direction corresponding to the position (u, v) of the coordinates (unit: pixel) of each pixel on the light receiving surface of the image sensor 15. The distance from the center of the system 16 to the object is d [mm]. These values become known values by associating the distance d to the object with the position of each pixel in the distance image sensor 10.

The coordinate values (X1, Y1, Z1) in the camera coordinate system for the object are obtained as follows. The unit of each component of the coordinate values (X1, Y1, Z1) is mm. The origin of the coordinate system set on the light receiving surface of the image sensor 15 is the position of one corner of the light receiving surface of the image sensor 15 that is rectangular, and the origin of the orthogonal coordinate system is the center of the light receiving optical system 16.
X1 = u1 · d / R
Y1 = v1 · d / R
Z1 = f · d / R
However,
u1 = su (u-uc)
v1 = sv (v−vc)
R = (u1 ² + v1 ² + f ² ) ^1/2
In order to simplify the explanation, the influence of the optical distortion of the light receiving optical system 16 is ignored. In order to correct the optical distortion of the light receiving optical system 16, a distortion correction formula for correcting the distance R from the optical center is used.

  As described above, after the distance image in the polar coordinate system is converted into a three-dimensional image in the camera coordinate system, the coordinate conversion to the global coordinate system is performed. The distance image sensor 10 is arranged so that the vertical direction (that is, the v direction) on the light receiving surface of the image sensor 15 is parallel to the wall surface 51 and the floor surface 52 of the room.

  Now, as shown in FIG. 5A, the depression angle of the optical axis 16A of the light receiving optical system 16 in the distance image sensor 10 is θ. In FIG. 5A, the viewing angle of the distance image sensor 10 is represented by φ. For the conversion from the camera coordinate system to the global coordinate system, as shown in FIG. 5B, the coordinate value (X1, Y1, Z1) in the camera coordinate system corresponds to the depression angle θ around the Y axis. Rotate. As a result, a global coordinate system three-dimensional image having coordinate axes orthogonal to the indoor wall surface 51 and floor surface 52 is generated as shown in FIG.

Hereinafter, the coordinate values in the global coordinate system are assumed to be (X, Y, Z). In the illustrated example, a direction away from the wall 51 in a direction perpendicular to the wall 51 is defined as a positive direction in the X direction, and a downward direction perpendicular to the floor 52 is defined as a positive direction in the Z direction. A right-hand system is used as the coordinate system. The rotation of the coordinate system with respect to the depression angle θ is performed by the following calculation.
X = X1 · cos (90 ° −θ) + Z1 · sin (90 ° −θ)
Y = Y1
Z = −X1 · sin (90 ° −θ) + Z1 · cos (90 ° −θ)
The image by coordinate axis is not the image itself mapped to the global coordinate system, but an image in which the X value, the Y value, and the Z value are individually associated with the coordinate position (u, v) of each pixel of the distance image. is there. In other words, each coordinate position (u, v) is an image in which X (u, v), Y (u, v), and Z (u, v) are associated with each other. On the other hand, three coordinate axis-specific images are generated. In order to obtain an image for each coordinate axis from a polar coordinate system distance image, the above calculation may be performed. However, if a table for converting a polar coordinate system distance image into a coordinate value (X, Y, Z) is prepared, Processing load can be reduced.

  Hereinafter, among the images by coordinate axes, an image having an X value as a pixel value is referred to as an X image, an image having a Y value as a pixel value is referred to as a Y image, and an image having a Z value as a pixel value is referred to as a Z image. Accordingly, three types of tables for performing coordinate conversion are required: an X conversion table for conversion to an X image, a Y conversion table for conversion to a Y image, and a Z conversion table for conversion to a Z image. The technique using the coordinate axis-specific image will be described later.

  The X image, the Y image, and the Z image described above are stored in the memory of the digital signal processing device and represent a three-dimensional virtual space corresponding to the target space that is a real space. Therefore, setting the condition in the virtual space is equivalent to setting the condition in the target space.

  The X image, the Y image, and the Z image generated by the distance image sensor 10 are given to the behavior monitoring processing unit 20. The behavior monitoring processing unit 20 classifies the behavior related to the bed 30 with respect to the subject 40 to be monitored. Moreover, in order to classify | categorize the action of the subject 40, it is necessary to extract the bed 30 irrespective of the presence or absence of the subject 40 on the bed 30. Accordingly, the distance image sensor 10 is disposed at a position where the bed 30 is looked down from above so that the subject 40 on the bed 30 is included in the visual field. In order to satisfy this condition, it is possible to arrange the distance image sensor 10 on the ceiling. However, it takes time to install the distance image sensor 10 on the ceiling in an existing room.

  Therefore, as shown in FIG. 2B, a configuration in which the distance image sensor 10 is attached to the upper end portion of the stand 31 that stands on the floor is adopted, or one end portion is attached to the wall 51 as shown in FIG. A configuration in which the distance image sensor 10 is attached to the other end of the arm 32 is employed. In the configuration shown in FIG. 2, a stand 31 having a support plate 35 attached on a base plate 34 having casters 33 is provided, and the distance image sensor 10 is attached to the upper end portion of the support plate 35. In addition, an action monitoring processing unit 20 is provided in an intermediate portion of the support plate 35.

  The behavior monitoring processing unit 20 notifies the contents of monitoring indoors or outdoors through the notification unit 24 (see FIGS. 1 and 3). In order to notify the contents of monitoring outside, communication is performed using a local area network (LAN) or a nurse call system communication network. For example, when a stand 31 is installed in a hospital room in a hospital, a reception device that receives notification from the notification unit 24 is provided in the nurse center, so that the behavior of the subject classified in the behavior monitoring processing unit 20 is stored in the nurse center. It is possible to notify the provided receiving device. In the following, it is referred to as “notify the nurse center” or “notify the nurse center” in the sense of notifying the receiving device provided in the nurse center.

  As described above, if the distance image sensor 10 and the behavior monitoring processing unit 20 are provided on the stand 31 provided with the casters 33, the stand 31 is movable, so that it can be moved to a required place. That is, there is no need to bring the stand 31 into a room where it is not necessary to monitor the behavior of the subject 40. Even when the behavior of the subject 40 is monitored, the stand 31 can be disposed between the headboard 36 and the wall surface 51 of the bed 30 as shown in FIG. Never become.

  As is clear from the above configuration, the distance image sensor 10 is disposed at a position higher than the bed 30. Actually, the distance image sensor 10 is disposed at such a height that the entire bed 30 is included in the target space that is the visual field region. By disposing at such a height position, a person can be prevented from inadvertently touching the distance image sensor 10, and the distance image sensor 10 can be disposed so as not to disturb the person. For example, the stand 31 does not interfere with work when a subject (patient) is treated in a hospital. Moreover, since the distance image sensor 10 detects the distance to the object existing in the target space without contact, the distance image sensor 10 does not disturb the behavior of the subject and does not disturb the sleeping comfort of the bed 30.

  As shown in FIG. 2A, the distance image sensor 10 is disposed with the optical axis 16A (see FIG. 5) obliquely downward so that the bed 30 can be seen from above. In this state, the tilt angle (the depression angle θ) of the optical axis 16A is set so that the entire top surface of the bed 30 is included in the visual field region of the distance image sensor 10. Here, since the optical axis 16A is obliquely downward, the distance to the upper surface of the bed 30 is relatively small on the headboard 36 side of the bed 30, and the upper surface of the bed 30 is on the footboard 37 side of the bed 30. The distance to becomes relatively large.

  As described above, the present embodiment assumes a case in which there is a curtain around the bed (hereinafter simply referred to as “curtain”) surrounding the bed 30, and the distance image sensor 10 uses a TOF type. Therefore, when the curtain is closed, there is a possibility that an error occurs in the distance to the object due to multiple reflection of the projected intensity-modulated light. That is, in a closed space where multiple reflection occurs, there is a possibility that the distance value obtained becomes larger than when used in an open space. This is because the influence of the multiple reflected intensity modulated light reflected in the surroundings on the linear intensity modulated light up to the object is greatly superimposed, and in effect demodulated as the intensity modulated light up to the object becomes longer It is. For this reason, the distance value obtained by the distance image sensor 10 may vary depending on the opening / closing of the curtain. If the distance value included in the distance image generated by the distance image sensor 10 fluctuates with the opening and closing of the curtain, an error occurs in information to be detected using the distance image, and as a result, information to be recognized is not detected. This will lead to false detections that are detected undetected or missed information.

  As shown in FIG. 1, the present embodiment includes a distance variation correction unit 60 that monitors the output of the distance image sensor 10, and the distance variation correction unit 60 monitors a variation in distance corresponding to the opening and closing of the curtain. In addition, a configuration that prevents undetected and erroneous detection is adopted. The distance fluctuation correction unit 60 is configured by using a digital signal processing device (a microcomputer, a DSP, an FPGA, or the like), similarly to the arithmetic processing unit 19 of the distance image sensor 10. That is, by executing the program in the digital signal processing device, the function of the distance variation correction unit 60 described below is realized. Note that the digital signal processing devices constituting the arithmetic processing unit 19 and the distance variation correcting unit 60 may be provided separately or shared.

  As shown in FIG. 7, the distance variation correction unit 60 includes a background image storage unit 61 that stores a distance image output from the distance image sensor 10 as a background distance image at the start of use of the system. As will be described later, the person to be monitored is bedded on the bed 30 at the start of use of the system.

  The distance variation correction unit 60 includes a current image acquisition unit 62 that sequentially captures the distance images output from the distance image sensor 10. The background distance image stored in the background image storage unit 61 and the distance image acquired by the current image acquisition unit 62 are compared by the variation rate calculation unit 63 provided in the distance variation correction unit 60, and newly added to the background distance image. The variation rate is calculated from the acquired distance image. The calculation of the fluctuation rate will be described later.

  The fluctuation rate calculation unit 63 compares the obtained fluctuation rate with the reference range, and when the fluctuation rate is within the reference range, the fluctuation is not caused by the movement of the subject, but by opening and closing the curtain (multiple reflection) Judge that the change has occurred. The variation rate calculation unit 63 obtains a representative value of the variation rate in consideration of the variation of the variation rate caused by the influence of multiple reflection, and the distance correction unit 64 provided in the distance variation correction unit 60 uses the representative value of the variation rate. To hand over. The distance correction unit 64 corrects the distance value of the distance image according to the representative value of the variation rate, and passes the corrected distance image to the behavior monitoring processing unit 20 through the output unit 65.

  By the way, the mechanism relating to the variation of the distance value of the distance image due to multiple reflection is complicated, and it is difficult to completely simulate an event caused by multiple reflection. That is, it is difficult to completely correct the error of the distance image due to multiple reflection. However, in the use environment as described above, the greater the distance from the distance image sensor 10, the greater the variation in distance due to multiple reflection, and the smaller the distance from the distance image sensor 10, the smaller the variation in distance due to multiple reflection. . For this reason, it has been experimentally obtained that the value becomes approximately constant when each fluctuation amount is normalized by the distance. In the present embodiment, using this knowledge, the fluctuation rate calculation unit 63 obtains a fluctuation rate for each pixel of the distance image and evaluates whether there is an influence of multiple reflection, and further, the fluctuation rate due to the influence of multiple reflection. The distance correction unit 64 corrects the distance image using the representative value of the variation rate and the distance value.

  Here, the fluctuation rate calculation unit 63 calculates the fluctuation rate k corresponding to each pixel of the distance image as k = (pixel value of the background distance image) / (pixel value of the acquired distance image). If no change occurs in the field of view of the distance image sensor 10, the rate of change k is substantially constant, and the rate of change k changes depending on the degree of the influence of multiple reflections and the displacement of an object in the field of view. That is, if correction is performed with (variation rate k) × (pixel value of the distance image acquired from the distance image sensor 10) as (pixel value of the corrected distance image), the variation of the distance value due to multiple reflections is corrected. Thus, it is possible to correct the pixel value without causing a large error.

  The variation rate calculation unit 63 evaluates whether or not there is an influence of multiple reflection by comparing the variation rate k obtained from the distance image with the reference range. In addition, the fluctuation rate calculation unit 63 obtains a representative value of the fluctuation rate k for a pixel in which the fluctuation rate k has changed only due to the influence of multiple reflection. The representative value of the variation rate k is the average value of the variation rate k for each pixel in the entire distance image, the maximum value in the distance image of the variation rate k for each pixel, and the variation rate belonging to the specified ratio from the maximum value in the distance image. It is selected from an average value of k, an average value in an area set in the distance image, and the like.

  By the way, the variation rate is obtained as a ratio of pixel values for each pixel of the distance image obtained from the distance image sensor 10 and the background distance image. In the above-described example, the variation rate is obtained for the entire distance image. Therefore, it is not guaranteed that the distance value included in the background distance image is a fixed value. That is, if the background distance image includes a region where the distance value fluctuates in a short time, that is, a distance variation region caused by a physical movement of the object, the distance value of the region is in the process of changing. Since it is only an instantaneous distance value, it is not appropriate as a reference for obtaining the fluctuation rate. Therefore, in order to accurately evaluate the variation rate, it is preferable to use a known area in the distance image that does not change the distance value as a reference.

  As the area where the distance value does not change, for example, an area corresponding to the foot board 37 of the bed 30 is used. It is considered that the distance from the distance image sensor 10 to the foot board 37 does not change unless the bed 30 is moved. However, if the target person 40 is present on the bed 30, it may be difficult to include the entire footboard 37 in the field of view of the distance image sensor 10. Therefore, as shown in FIG. 8, an observation plate 45 for determining a reference for the distance value is arranged on the foot board 37.

  The observation board 45 is disposed so as to extend upward from the upper end of the foot board 37. In order to extract the observation board 45 from the distance image, the distance image is given to the behavior monitoring processing unit 20, and the bed recognition unit 21 performs a process described later, thereby detecting the range of the bed 30. When the bed recognition unit 21 detects the range of the bed 30 from the distance image, the bed recognition unit 21 calculates a region corresponding to the footboard 37, and further uses a region corresponding to the footboard 37 to region S {i corresponding to the observation board 45. , J}.

  As shown in FIG. 9A, when the distance image with the curtain opened is stored as the background distance image, the curtain 46 is closed as shown in FIG. 9B. Assume that a distance image is obtained. Here, for the background distance image and the distance image acquired from the distance image sensor 10, the distance value for each pixel included in the region S {i, j} corresponding to the observation plate 45 is represented by bg (i, j), Let fg (i, j). Therefore, the variation rate k (i, j) of each pixel is expressed as k (i, j) = bg (i, j) / fg (i, j).

  In the region S {i, j} corresponding to the observation plate 45, it is expected that the distance does not vary regardless of the opening / closing of the curtain 46, that is, the variation rate k (i, j) is approximately 1. . However, in practice, the variation rate k (i, j) varies depending on the presence or absence of multiple reflections or the movement of the subject 40.

  When the target person 40 on the bed 30 moves, a part of the observation board 45 is shielded by the target person 40 within the field of view of the distance image sensor 10, so that the variation rate k (i) in the region S {i, j}. , J) occur where there is a relatively large fluctuation. On the other hand, depending on the ambient environment of the bed 30 and the specifications of the distance image sensor 10, the limit value of the variation rate k (i, j) is approximately 1.0 ± 0 due to the influence of multiple reflections associated with the opening and closing of the curtain 46. The experimental result of .1 was obtained.

  Based on this result, if the fluctuation rate k (i, j) is 0.9 to 1.1, the fluctuation rate calculation unit 63 determines that the distance fluctuation is due to multiple reflection, and the fluctuation rate k (i, j). Is less than 0.9 or greater than 1.1, it is configured to determine that the subject 40 has moved. If the distance is in the range of about 2 m, a variation rate k (i, j) of 0.1 (10%) corresponds to 20 cm. Considering the movement of the subject 40 and the size of the subject 40, it is considered that the distance hardly changes to 20 cm or less when the subject 40 moves. Therefore, the change rate calculation unit 63 compares the change rate k (i, j) with the reference range (0.9 to 1.1), thereby changing due to the influence of multiple reflections or changing due to the movement of the subject 40. Can be determined.

  As described above, the distance correction unit 64 corrects the distance value by multiplying the distance value of the distance image acquired from the distance image sensor 10 by the fluctuation rate k (i, j) calculated by the fluctuation rate calculation unit 63. The distance value is output from the output unit 65.

  An example of the operation of the fluctuation rate calculation unit 63 and the distance correction unit 64 is shown in FIG. In the operation example shown in FIG. 10, the average value of the fluctuation rates excluding the existence range of the subject 40 is used as the representative value of the fluctuation rate.

  The fluctuation rate calculation unit 63 includes a region S {i, j corresponding to the observation plate 45 from the background distance image stored in the background image storage unit 61 and the distance image acquired by the current image acquisition unit 62 from the distance image sensor 10. } Is extracted (S11).

  Thereafter, for each pixel {i, j} included in the region S {i, j}, a variation rate k (i, j) is calculated (S13), and the variation rate k (i, j) is calculated as the reference range (0. 9-1.1) (S14). When the variation rate k (i, j) is within the reference range (S14: Yes), the total variation rate k (i, j) and the number of corresponding pixels are obtained (S15). When the variation rate k (i, j) exceeds the reference range (S14: No), the corresponding pixel {i, j} is ignored.

  The processes in steps S13 to S15 described above are performed for all the pixels {i, j} in the region S {i, j} (S12, S16). Further, by dividing the sum of the variation rates k (i, j) obtained in step S15 by the number of pixels, for the pixel {i, j} where the variation rate k (i, j) is within the reference range, An average value of the fluctuation rates k (i, j) is obtained (S17). In the operation example shown in FIG. 10, this average value is used as a representative value of the fluctuation rate k (i, j). The representative value of the obtained variation rate k (i, j) is the representative value of the variation rate k (i, j) obtained by excluding pixels whose distance value has changed due to the movement of the subject 40.

  Next, the distance correction unit 64 multiplies the distance value of all the pixels of the distance image acquired by the current image acquisition unit 62 by the representative value of the variation rate k (i, j) calculated by the variation rate calculation unit 63. Then, the distance value is corrected (S19). The correction of the distance value is performed for all pixels of the distance image (S18, S20). The distance correction unit 64 outputs a distance image having the corrected distance value through the output unit 65 (S21). In this way, all the distance images are corrected by multiplying the distance value by the representative value of the variation rate k (i, j), thereby correcting all the distance images in the visual field of the distance image sensor 10. The influence of multiple reflection from the pixel can be reduced.

  As described above, since the process in the behavior monitoring processing unit 20 is performed using the distance image having the corrected distance value, the influence of the multiple reflection caused by opening and closing of the curtain 46 is suppressed, and the determination in the behavior monitoring processing unit 20 is performed. Can be performed with high accuracy, and the operation of the subject 40 is prevented from being mistaken for multiple reflection. Note that the portion where the observation plate 45 is provided is not limited to the footboard 37, and the observation plate 45 can be arranged at an appropriate portion as long as it is within the field of view of the distance image sensor 10.

  By the way, when the observation board 45 is attached to the foot board 37, the region S {i, j} for obtaining the variation rate k (i, j) is not obstructed by the movement of the subject 40 in the bed, and the variation rate k. The representative value of (i, j) can be obtained with high accuracy. However, it takes time and effort to attach the observation board 45 to the footboard 37. The observation board 45 can be integrated with the foot board 37, but in that case, a specially shaped foot board 37 is required.

  In the following, a technique for evaluating the influence of multiple reflections using the variation rate k (i, j) without providing the observation plate 45 will be described. Whether or not there is a movement of the subject 40 for each pixel {i, j} of the focused area S {i, j} when determining the presence or absence of the influence of multiple reflections based on the variation rate k (i, j). Therefore, a slight change in the distance value in the region S {i, j} is allowed. Therefore, instead of the foot board 37 of the bed 30, the region S {i, j} for obtaining the variation rate k (i, j) may be set as the region inside the bed 30. For example, in the case of the upper surface of the comforter, an area where the variation of the distance value is small over a relatively wide range occurs.

  When the region S {i, j} is set on the bed 30, not only the change of the distance value due to the influence of the multiple reflection but also the change of the distance value due to the movement of the subject 40 becomes large. However, since the rate of change k (i, j) is evaluated for each pixel {i, j} in the region S {i, j}, the pixel whose distance value has changed due to the movement of the subject 40 is removed and changed. The rate k (i, j) can be evaluated, and the representative value of the variation rate k (i, j) can be obtained with relatively high accuracy. For example, in the case of a distance range of about 2 m, it is considered that the change of the distance value due to the movement of the subject 40 is 40 cm (20%) or more with respect to the change of the distance value of about 20 cm (10%) due to multiple reflection. If a threshold value is appropriately set, both can be discriminated.

  As described above, the variation rate k (i, j) is evaluated using an appropriate region on the upper surface of the comforter, the entire surface of the bed 30, and the like, so that the change in the distance value due to the influence of the multiple reflection without using the observation plate 45. Can be detected and the distance value can be corrected.

  Hereinafter, the behavior monitoring processing unit 20 will be described in more detail. In order to classify the behavior of the target person 40, it is necessary to extract the bed 30 from the distance image output by the distance image sensor 10, as described above. Accordingly, the behavior monitoring processing unit 20 includes a bed recognition unit 21 that recognizes the bed 30 using information included in the distance image. Further, since the bed 30 is recognized by the bed recognition unit 21, it is possible to recognize the presence or absence of a person in the target space by using the relative relationship between the information included in the distance image and the bed 30.

  Further, the behavior monitoring processing unit 20 is provided with a person recognition unit 22 that recognizes the presence or absence of a person from the relationship between the bed 30 recognized by the bed recognition unit 21 and other objects. In addition, the behavior monitoring processing unit 20 classifies the behavior of the target person by combining the recognition by the bed recognition unit 21 and the recognition by the person recognition unit 22, and performs behavior notification when necessary. 23.

  First, the bed recognition unit 21 detects the outer periphery of the bed 30 when the bed 30 is viewed in plan to recognize the position of the bed 30. Since the general bed 30 has a quadrangular shape in plan view, the bed recognition unit 21 recognizes the position of the bed 30 by extracting four sides of the bed 30.

  In the present embodiment, a target is provided at a known position of the bed 30 and the four sides of the bed 30 are easily extracted by using the position of the target as a reference. The target only needs to be provided on any one of the four sides of the bed 30. However, since the distance image sensor 10 is disposed on the headboard 36 side of the bed 30, the target is within the field of view of the distance image sensor 10. The target is preferably provided on the footboard 37 side so that it can easily enter.

  In the example shown in FIG. 11, three markers P0, P1, and P2 are attached to the footboard 37 as a target. The marker P1 is attached to the center position in the left-right direction at the upper end of the footboard 37, and the markers P0 and P2 are attached to the footboard 37 away from the left and right of the marker P1. That is, the markers P0 and P2 are arranged symmetrically with the marker P1 interposed therebetween. It is preferable that the distance between the marker P1 and the markers P0 and P2 is as large as possible. Further, the markers P0, P1, and P2 have the same height from the upper surface of the bed 30. The markers P0, P1, and P2 are preferably highly reflective, for example, are provided with retroreflectivity. The markers P0, P1, and P2 are preferably coated with an adhesive on one side so that the markers can be attached to the footboard 37. Furthermore, the markers P0, P1, and P2 may be pasted on the bed 30 to be monitored in advance, and it is not necessary to repeat pasting and peeling.

  In order to extract the positions of the markers P0, P1, and P2 arranged on one side of the bed 30 in the bed recognition unit 21, first, a reflection intensity image is generated. In the reflection intensity image, the brightness of the parts of the markers P0, P1, and P2 is higher than the surroundings. Therefore, when a binary image is generated from the reflection intensity image by applying an appropriate threshold value to the brightness, only the markers P0, P1, and P2 are used. Are extracted into the binary image. Note that the threshold value for generating the binary image from the reflection intensity image may be either a fixed threshold value or a floating threshold value. When the reflectance of the markers P0, P1, and P2 is sufficiently large compared to the surroundings, a fixed threshold value may be used. When the difference between the reflectance of the surrounding objects and the reflectances of the markers P0, P1, and P2 is small A floating threshold may be used. In order to prevent erroneous detection of the markers P0, P1, and P2, it is preferable to limit in advance the region for extracting the markers P0, P1, and P2 at known positions from the reflection intensity image.

  When the position of the pixel corresponding to the markers P0, P1, and P2 is obtained using the reflection intensity image, the three-dimensional position corresponding to the pixel is obtained by using the distance image. That is, the three-dimensional positions of the markers P0, P1, and P2 in the camera coordinate system are obtained. Therefore, when the X image and the Y image are used, the X value and the Y value corresponding to the markers P0, P1, and P2 are obtained, and the markers correspond to the markers P0, P1, and P2 in the global coordinate system as shown in FIG. The coordinate positions of the points WP0, WP1, and WP2 are obtained. Since the markers P0, P1, and P2 have sizes, the coordinate position of the representative point (for example, the center of gravity) in the area corresponding to the markers P0, P1, and P2 is set as the coordinate position of the points WP0, WP1, and WP2. In the global coordinate system, the vertical downward direction is the positive direction of the Z direction, and the bed recognition unit 21 only needs to detect the position of the peripheral edge of the bed 30 on the floor surface, so the Z value is not obtained.

  When the coordinate positions of the points WP0, WP1, WP2 corresponding to the markers P0, P1, P2 in the plane parallel to the XY plane are obtained as described above, the bed recognizing unit 21 determines the three points WP0, WP1. , An approximate straight line L0 passing through WP2. The approximate straight line L0 is obtained using, for example, the least square method. The obtained approximate straight line L0 can be regarded as a straight line passing through the foot board 37 in the XY plane. That is, it can be said that the approximate straight line L0 defines one edge of the bed 30. Further, since the point WP1 is located at the center of the footboard 37, the coordinate position of the perpendicular foot drawn from the point WP1 to the approximate straight line L0 is the coordinate of the center of the footboard 37 obtained from the markers P0 to P2. It can be regarded as a position. Hereinafter, the point corresponding to the leg of the perpendicular drawn from the point WP1 to the approximate straight line L0 is referred to as WP4.

  In a plane parallel to the XY plane, a circle whose diameter is the left-right width D of the bed 30 with the point WP4 as the center is set, and when the intersections C0 and C1 between the circle and the approximate straight line L0 are obtained, the intersections C0 and C1 are This corresponds to the intersection between the left and right edges of the bed 30 and the footboard 37. Here, a known dimension is used for the left-right width D of the bed 30. Accordingly, when the straight lines L2 and L3 passing through the intersections C0 and C1 and orthogonal to the approximate straight line L0 are obtained, the straight lines L2 and L3 can be regarded as straight lines passing through the left and right edges of the bed 30. Here, the straight lines L2 and L3 are extended in a direction in which the coordinate value of the X coordinate becomes smaller than the approximate straight line L0.

  Finally, a circle whose radius is the length W of the bed 30 with the intersection C0 as the center is set, an intersection C2 between the circle and the straight line L2 is obtained, and the length W of the bed 30 with the radius as the center is the intersection C1. And a point of intersection C3 between the circle and the straight line L3 is obtained. If a straight line L1 passing through the intersections C2 and C3 is set, the straight line L1 can be regarded as a straight line passing through the head board 36 of the bed 30. Here, a known dimension is used for the length W of the bed 30.

  The bed recognition unit 21 obtains four straight lines L0, L1, L2, and L3 corresponding to the four edges surrounding the bed 30 by the above-described procedure. At this time, four intersections C0, C1, C2, and C3 corresponding to the four corners of the bed 30 are also obtained. Furthermore, the bed recognition unit 21 obtains the inclination of the bed 30 with respect to the coordinate axes in the XY plane in the global coordinate system. The inclination of the bed 30 is obtained as an angle Φ formed by a straight line (for example, a straight line L2) with respect to one coordinate axis (for example, the X axis) of global coordinates. Note that the angle Φ is obtained by an angle formed by the X axis or the Y axis of the global coordinates and any of the four straight lines L0, L1, L2, and L3. As described above, even when the bed 30 is disposed to be inclined with respect to the coordinate axis of the global coordinate system, the bed recognition unit 21 detects the position of the bed 30 with high accuracy by the above-described processing.

  After obtaining the four sides of the bed 30, an area inside the rectangle surrounded by the four sides is set as an area within the range of the bed 30. That is, a rectangular area surrounded by the edges in the long direction and the short direction (left-right direction) is set within the range of the bed 30.

  In the operation example described above, three markers P0, P1, and P2 are used. However, if the marker P1 is arranged at the center of the left and right of the footboard 37, the number of remaining markers is not particularly limited. There may be one or more markers. For example, by providing four or more markers, the error between the straight line passing through the footboard 37 and the approximate straight line L0 may be reduced.

  In the example described above, the three markers P1, P1, and P2 are arranged on the footboard 37. However, as shown in FIG. 13, the footboard 37 of the bed 30 and the left and right side edges of the bed 30 (side rails). ) And one marker P10, P11, and P12 may be arranged. About the foot board 37, the marker P11 is arrange | positioned in the center of the left-right direction similarly to the example mentioned above. As long as the markers P10 and P12 are the left and right edges of the bed 30, the position in the XY plane and the height in the Z direction are not limited.

  Each marker P10, P11, P12 is preferably highly reflective and is affixed to the bed 30 as in the above example. The bed recognition unit 21 obtains the positions of the markers P10, P11, and P12 in the image from the reflection intensity image, and further obtains the three-dimensional positions of the markers P10, P11, and P12 from the distance image, as in the above example. Here, in order to prevent erroneous detection of the markers P10, P11, and P12, the area for extracting the markers P10, P11, and P12 at known positions from the reflection intensity image is limited in advance to the range of the footboard 37 and the side rail. It is preferable to keep it. Furthermore, the bed recognizing unit 21 obtains the coordinate positions of the points WP10, WP11, and WP12 in the global coordinate system of the markers P10, P11, and P12 using the X image and the Y image, as shown in FIG.

  In order to recognize the four sides of the bed 30, the bed recognizing unit 21 first obtains a line segment connecting the point WP10 and the point WP12, and obtains the midpoint WPm of this line segment. Based on the premise that the bed 30 has a rectangular shape, the midpoint WPm is located on the center line Lm that bisects the left-right width of the bed 30. Accordingly, a straight line passing through the points WP11 and Pm becomes the center line Lm of the bed 30. A straight line L10 that is orthogonal to the center line Lm and passes through the point WP11 is set. The straight line L10 corresponds to a straight line passing through the foot board 37 of the bed 30.

  After obtaining the center line Lm, the bed recognition unit 21 obtains a distance D1 between the point WP10 and the center line Lm and a distance D2 between the point WP12 and the center line Lm. Since the sum of both distances D1 and D2 corresponds to the left-right width D of the bed 30, when a circle having a diameter D centered on the point WP11 is set and intersections C10 and C11 between this circle and the straight line L10 are obtained, the intersection C10 , C11 correspond to the intersection of the left and right edges of the bed 30 and the footboard 37. When the straight lines L12 and L13 passing through the intersections C10 and C11 and parallel to the center line Lm are obtained, the straight lines L12 and L13 can be regarded as straight lines passing through the left and right edges of the bed 30. Here, the straight lines L12 and L13 are extended in a direction in which the coordinate value of the X coordinate becomes smaller than the straight line L10. In determining the straight lines L12 and L13, a straight line passing through the intersection C10 and the point WP10 may be L12, and a straight line passing through the intersection C12 and the point WP12 may be L13.

  The procedure for obtaining the straight line L11 corresponding to the head board 36 of the bed 30 is the same as the above-described example. That is, a circle having a radius of the length W of the bed 30 with the intersection C10 as the center is set, an intersection C12 of the circle and the straight line L12 is obtained, and the length W of the bed 30 with the radius of the intersection 30 is set as the center. A circle to be used is set, and an intersection C13 between the circle and the straight line L13 is obtained. The straight line L11 passing through the intersections C12 and C13 can be regarded as a straight line passing through the head board 36 of the bed 30. A known dimension is used for the length W of the bed 30.

  Thus, even if the markers P10, P11, and P12 are arranged on the three sides of the bed 30, straight lines L10, L11, L12, and L13 corresponding to the four sides of the bed 30 are obtained. When this procedure is adopted, if the number of markers is increased, the detection accuracy of the center line Lm is improved.

  When adopting a process of detecting the position of the bed 30 by providing the markers P10 and P12 on the side rail, it is preferable to attach the markers P10 and P12 to the side rail in advance. The bed recognition unit 21 preferably issues a warning notification through the notification unit 24 when the markers P10 and P12 are not detected and the number of markers is insufficient. When this operation is adopted, when the side rail is not installed, a warning is notified by the fact that the markers P10 and P12 are not detected, so that forgetting to install the side rail is prevented. In particular, when the user of the bed 30 needs to be careful about getting out of bed, it is common to attach about three fences to the bed 30 in order to prevent the bed 30 from falling. Therefore, not only the position of the bed 30 can be specified but also the notification of forgetting to install the side rail can improve convenience.

  If the four sides of the bed 30 are detected by any of the procedures described above, even if there are members or structures having the same height as the bed 30 around the bed 30, the bed is not affected by them. It is possible to detect the range of 30 with high accuracy and reproducibility.

By the way, even if each edge of the bed 30 is included in the visual field region of the distance image sensor 10, the action of the subject 40 cannot be determined unless there is a margin in the peripheral part of the visual field region. Therefore, the following determination is performed in order to ensure that a region other than the bed 30 is included around the bed 30 in the visual field region of the distance image sensor 10.
Xmax−Xs ≧ Mgx1
Xi-Xmin ≧ Mgx2
Ymax−Ys ≧ Mgy1
Yi-Ymin ≧ Mgy2
Here, Xmax is the maximum value of the X value in the X direction of the visual field area, Xmin is the minimum value of the X value in the X direction of the visual field area, Ymax is the maximum value of the Y value in the Y direction of the visual field area, and Ymin is the visual field area. This is the minimum Y value in the Y direction. Further, margin dimensions to be set around the bed 30 are Mgx1, Mgx2, Mgy1, and Mgy2. In this case, the margin dimensions Mgx1, Mgx2, Mgy1, and Mgy2 have different values depending on the arrangement of the bed 30 in the room and the arrangement of the distance image sensor 10 with respect to the bed 30, so that it is preferable that the margin dimensions Mgx1, Mgx2, Mgy1, and Mgy2 can be set by setting means (not shown). In the present embodiment, since the stand 31 provided with the distance image sensor 10 is disposed between the headboard of the bed 30 and the wall surface (see FIG. 1), the margin dimension Mg2 on the headboard side is not necessarily used. It does not have to be.

  When even one of the above-described determination conditions is not satisfied, it is determined that the field-of-view range Fv is deviated from the bed 30 as illustrated in FIG. 15, and a notification unit indicates that the position setting of the distance image sensor 10 is inappropriate. It is desirable to notify by 24. In the example shown in FIG. 15, one end in the longitudinal direction of the bed 30 is not included in the visual field region Fv, so Xmax = Xs, and one edge in the short direction of the bed 30 is included in the visual field region Fv. Therefore, Ymin = Yi. Therefore, the condition of Xmax−Xs ≧ Mgx1 and the condition of Yi−Ymin ≧ Mgy2 are not satisfied. By making such a determination, it is possible to monitor the behavior of the subject 40 entering and exiting the bed 30.

  When the bed recognition unit 21 recognizes the position of the bed 30, it next detects the position of the upper surface of the bed 30. Hereinafter, the upper surface of the bed 30 is referred to as a bed surface. In detecting the height of the bed surface, it is assumed that the subject 40 is sleeping on the bed 30 (this state is referred to as “bed bed”). In the state where the subject person 40 is lying on the bed, the subject person 40 and a bedding such as a comforter are placed on the bed 30. In the global coordinate system described above, the downward direction of the real space is set to the positive direction of the Z axis. However, in the case of explaining the height of the real space, the upward direction is set to the direction in which the value increases. That is, the direction in which the Z value increases in the Z direction is treated as the direction in which the height decreases in the height direction.

  The bed recognition unit 21 divides the range of the bed 30 into a predetermined number (for example, 10) of divided regions 38 in the longitudinal direction as shown in FIG. The number of the divided regions 38 depends on the size of the bed 30 in the longitudinal direction, but the number is determined so that one divided region 38 has a width of about 10 to 20 cm. Next, a representative value related to the height of each divided region 38 is obtained. As the representative value, either an average value or a mode value may be used.

  When using the average value, the position (u, v) belonging to each divided region 38 is obtained from the Z image, and the average value of the Z values for each divided region 38 is obtained. On the other hand, in the case of using the mode value, the position (u, v) belonging to each divided region 38 is obtained for the Z image, the frequency distribution of the Z value for each divided region 38 is obtained, and the highest frequency distribution is obtained from the obtained frequency distribution. Find the mode value. Although depending on the location of the divided region 38, in this frequency distribution, the frequency of the bed surface or the frequency of the upper surface of the bedding increases, and then the frequency corresponding to the subject 40 increases. This is based on the knowledge that the area occupied by the bed surface or the bedding on the bed 30 is larger than the area occupied by the subject 40.

  When the representative value regarding the height is obtained from each divided region 38 as described above, the representative value Vt is distributed as shown by the solid line in FIG. Further, an offset value considering the thickness of the person is subtracted from the representative value Vt (added in the case of a Z value) and used as the height of the bed surface. Although there are individual differences in the thickness of a person, for example, a value such as 20 cm is used, and half of the value is used as an offset value. Since there is no special condition for the offset value, the offset value is appropriately set so that the value obtained by subtracting the offset value from the representative value becomes the height of the bed surface in many divided regions 38. The broken line in FIG. 16B indicates an example of a value obtained by subtracting the offset value from the representative value Vt. Thus, the height of the bed surface obtained for each divided region 38 becomes the reference height for each divided region 38.

  In the subsequent processing, a set of bed surfaces for each divided region 38 is treated as a bed surface in the entire bed 30. Further, since the divided region 38 divided in the longitudinal direction is used instead of the entire bed 30, the bed surface obtained for each portion of the bed 30 is used even when the bed 30 is a reclining type and the bed surface is not flat. This makes it possible to separate from the subject 40.

  By the way, in the above-described example, the height of the bed surface is obtained on the assumption that the subject 40 on the bed 30 is in the bedside state. However, as shown in FIG. When the user stands up, the bed surface height (shown as Vh in FIG. 17) determined as described above is not appropriate. Therefore, when the upper limit value is set for each divided region 38 and the value obtained as the bed surface height Vh exceeds the upper limit value, the notification unit 24 notifies that the height of the bed surface Vh cannot be adopted. To do.

  Here, since the measurement of the bed surface in the bed 30 is performed immediately after instructing the start of use of the system, it is possible to take appropriate measures by notifying from the notification unit 24. The instruction to start using the system means an operation of turning on an operation start switch (not shown). Appropriate response means that the instruction for starting the use of the system is given again after the subject 40 is placed in the bedside position at the correct position.

  The bed 30 described above assumes the case where the height of the bed surface is fixed, but in order to facilitate the work of nursing and nursing care, the height of the bed surface can be changed. 30 is also provided. In the liftable bed 30, if the user forgets to lower the bed surface after nursing or care work, the bed surface becomes higher than usual. Therefore, when the subject 40 leaves the bed 30, his feet are on the floor. It becomes difficult to reach, and the impact increases when the subject 40 falls from the bed 30. Therefore, the bed surface must be lowered after work.

  When the bed 30 is a lifting type, as shown in FIG. 18, it is considered that the lower limit of the bed surface exceeds a predetermined height threshold Thh during nursing or nursing work, so that the height threshold Thh is appropriately set. By setting, it is possible to detect that the bed surface is rising. That is, when the bed surface height obtained for all the divided regions 38 exceeds the height threshold value Thh, it is determined that the bed surface remains elevated and the notification unit 24 notifies the user.

  Here, since monitoring of the subject person 40 is unnecessary during the care or nursing work, the system operation is temporarily stopped, and the system operation is resumed after the work is completed. Therefore, when an instruction to resume the operation of the system is given after the work is completed, an abnormality in the bed surface height is notified, and forgetting to lower the bed surface can be prevented.

  Further, as shown in FIG. 19, when the table 39 is placed on the bed 30, the table 39 may be forgotten. On the other hand, if the height threshold value Thh is set so that the height of the table 39 can be detected in the divided region 38 corresponding to the table 39, the notification unit 24 can notify that the table 39 has been left behind. become.

  In the case of detecting both forgetting to lower the bed 30 and forgetting to leave the table 39, the height threshold value Thh is appropriately determined, and when the height threshold value Thh is exceeded in a predetermined number or more of the divided regions 38, the notification unit You may make it alert | report from 24. Since the determination using the height threshold Thh is performed immediately after the instruction to start the use of the system, as in the case of the measurement of the bed surface, if there is a problem, the notification unit 24 immediately notifies. Therefore, appropriate measures such as lowering the bed 30 and clearing the table 39 are possible. Thereafter, an instruction to start using the system may be given again.

  When the bed 30 has a reclining function and the upper body of the subject 40 can be raised as shown in FIG. 20, the determination based on the height threshold Thh is divided into a plurality of lower body sides (five in the illustrated example). This may be done only in the region 38. Since the position of the bed 30 in the distance image is known, for example, the determination based on the height threshold Thh may be performed only for the divided region 38 having an X value larger than (Xs + Xi) / 2. As described above, since the range in which the determination based on the height threshold Thh is performed can be formulated and automatically determined, manual adjustment work is unnecessary.

  By the way, various devices may be arranged around the bed 30. For example, as illustrated in FIG. 21, an object 41 that is not the target person 40 may be arranged above the bed 30 so as to overlap. is there. Examples of this type of object 41 include an illumination stand, a monitor device, and various medical devices.

  When such an object 41 overlaps the bed 30, it is necessary to exclude it from the area where the target person 40 is detected. Therefore, when the nurse or caregiver starts up the apparatus and causes the bed recognition unit 21 to recognize the position of the bed 30, it is preferable to perform processing for excluding the area of the object 41 from the area of the bed 30. That is, the area where the object 41 overlaps the bed 30 in the visual field of the distance image sensor 10 may be stored in a state where it is excluded as the mask area 42 as shown in FIG. The mask area 42 is handled as an area outside the range of the bed 30 when the person recognition unit 22 recognizes the target person 40, and can prevent the object 41 from being misidentified as the target person 40.

  The mask area 42 is set immediately after the bed recognition unit 21 extracts the position of the bed 30. That is, the bed recognizing unit 21 performs a process of determining the presence or absence of the mask region 42 immediately after recognizing the position of the bed 30 by a nurse or caregiver operating the apparatus. As shown in FIG. 23, the bed recognizing unit 21 is within the range of the bed 30 and above the specified exclusion threshold with respect to the reference height (bed surface height) obtained for each divided region 38. When the object 41 exists, the existence range of the object 41 is set as a mask area 42. The exclusion threshold is appropriately set according to the actual situation. The mask area 42 set in this way is stored in the bed recognition unit 21 to be treated as an area that is excluded when the subject person 40 is recognized.

  In the above-described operation, the height of the bed surface is used as the reference height for each divided region 38. However, when the bed recognition unit 21 recognizes the position of the bed 30, the target person 40 exists in the bed 30. It is preferable to use a bed detection surface Pd (see FIG. 24) described later at a reference height. The bed detection surface Pd is a reference height in a state where the subject 40 is lying on the bed 30. As described above, when the bed detection surface Pd is used as the reference height, it is possible to prevent the area where the subject 40 exists from being mistaken as the mask area 42. However, when the bed detection surface Pd is defined at a certain height from the bed surface, using the bed detection surface Pd as a reference height is equivalent to using the bed surface as a reference height. .

  It should be noted that the size of the mask area 42 is evaluated, and if there is a mask area 42 having a size exceeding a specified threshold, it is determined that a table or the like is placed. In this case, the reliability at the time of recognizing the target person 40 cannot be maintained, and there is a possibility that the target person 40 may come into contact with the object 41. It is preferable to notify from the notification unit 24.

  Below, the technique which recognizes the subject 40 by the person recognition part 22 is demonstrated. As described above, since the bed recognition unit 21 detects the bed surface, the person recognition unit 22 sets a plurality of detection surfaces using the bed surface as a reference. Here, as the state on the bed 30, three states are assumed: a bedbed state, a rising state where the upper body is raised, and a standing state where the body is raised. In order to detect each state, as shown in FIG. 24, the height of each divided region 38 defined as the bed surface Pb is used as a reference, and the bed detection surface Pd, the rising detection surface Pr, and the standing detection surface Ps are defined. Set.

  The bed detection surface Pd, the rising detection surface Pr, and the standing detection surface Ps are set by displacing the bed surface Pb by a predetermined amount in the Z direction. That is, since the bed surface Pb has a Z value determined for each divided region 38, the bed detection surface Pd, the rising detection surface Pr, and the standing-up detection surface Ps are also determined to be spaced apart by a predetermined distance upward for each divided region 38. It is done. Further, the distance from the bed surface Pb increases in the order of the bed detection surface Pd, the rising detection surface Pr, and the standing detection surface Ps. For example, the bed detection surface Pd is set 10 cm above the bed surface Pb, the rising detection surface Pr is set 30 cm above the bed surface Pb, and the standing detection surface Ps is set 60 cm above the bed surface Pb. The bed detection surface Pd, the rising detection surface Pr, and the standing detection surface Ps are used as thresholds for detecting the position of a person on the bed 30.

  In the person recognition unit 22, in the area of the bed 30, if there is an object above the bed detection surface Pd and no object exists above the rising detection surface Pr, the target person 40 is in the bed position. Is determined. Further, if there is an object above the rising detection surface Pr and no object above the standing detection surface Ps, it is determined that the subject 40 is in a rising state where the upper body is raised. Since the rising detection surface Pr is set above the bed surface, if an appropriate rising detection surface Pr is set, there is almost no erroneous detection that determines turning over or lifting of a futon as rising. Further, when an object is present above the standing detection surface Ps, it is determined that the target person 40 is standing on the bed 30.

  As described above, the person recognition unit 22 detects the state of the target person 40 using the detection surface set with the bed surface Pb as a reference. This process is advantageous in that, for example, the process is simple compared to a case where a difference image from the image in the state where the subject 40 is not present in the bed 30 is used, and that it does not take time and effort to capture a background image. In addition, when a difference image is used, since an object such as a futon is also detected as a difference, it is difficult to distinguish it from the target person 40, but if the state of the target person 40 is determined using the detection surface, this There is no problem.

  In the above-described example, in order to distinguish and detect the three states, a three-step detection surface is set on the bed surface Pb. However, it is not essential to set the detection surface in three steps. It is possible to set in two steps, or to set four or more detection surfaces. For example, when the state of the start of use of the system is set to the bed state, the bed state may not necessarily be detected. That is, it is only necessary to detect a state that is not a bed, that is, a rising state or a standing state, and in this case, a two-step detection surface may be set.

  In the above-described operation, only the height position of the object with respect to the detection surface is used, but the size of the object may also be determined by using the area cut out by the detection surface. For example, in the state of the bed, the area cut out by the bed detection surface Pd in the Z image is larger than the area cut out by the rising detection surface Pr and the standing detection surface Ps. In addition, in the rising state, the area cut out on the bed detection surface Pd in the Z image is smaller and the area cut out on the rising detection surface Pr is increased in the Z image. By using such an area relationship, the state (posture) of the target person 40 can be grasped more accurately.

  For example, if the ratio of the area cut by the rising detection surface Pr within the range of the bed 30 to the area cut by the bed detection surface Pd is equal to or greater than a predetermined threshold value, the rising state may be determined. If this threshold value is set appropriately, even if the arm 40 crosses the rising detection surface Pr due to the subject 40 extending his / her arm, the ratio is smaller than the threshold value, so that it is detected as a rising state. Can be prevented.

  For example, the white areas in FIGS. 25 (a) and 26 (a) indicate areas where the subject 40 exists on the bed detection surface Pd, and the white areas in FIGS. 25 (b) and 26 (b). These areas indicate areas where the subject 40 exists on the rising detection surface Pr. In FIGS. 25 and 26, the ratio E2 / E1 is obtained by setting E1 as the area of the white area in FIG. 25A and E2 as the area of the white area in FIG. 25B, and this ratio E2 / E1. Can be distinguished from the rising state (FIG. 25) and the other states (FIG. 26).

  Similarly, within the range of the bed 30, if the ratio of the area cut by the standing detection surface Ps to the area cut by the bed detection surface Pd is equal to or greater than a predetermined threshold, the subject 40 stands on the bed 30. Judged to be standing. Even if there is a portion that exceeds the standing detection surface Ps in the state where the subject 40 is standing up, the area crossing the standing detection surface Ps is considered to be small compared to the state where the subject 40 is standing up. The state can be distinguished.

  For example, the white areas in FIGS. 27 (a) and 28 (a) indicate areas where the subject 40 exists on the bed detection surface Pd, and the white areas in FIGS. 27 (c) and 28 (c). This area indicates an area where the subject 40 exists on the standing-up detection surface Ps. In the illustrated example, there is no white area in FIG. 27C, and the target person 40 does not exist. Moreover, the white area | region of FIG.27 (b) and FIG.28 (b) has shown the area | region where the subject 40 exists in the rising detection surface Pr. 27 and 28, the ratio E3 / E1 is obtained by assuming that the area of the white area in FIG. 27A is E1, and the area of the white area in FIG. When E3 / E1 is compared with an appropriate threshold, it is possible to distinguish the rising state (FIG. 27) and the standing state (FIG. 28).

  In the above-described processing, when the person recognition unit 22 recognizes the target person 40, the case where the rising detection surface Pr and the standing detection surface Ps are fixed values is exemplified. However, in actual use, since there are individual differences in the physique of the subject 40, it is preferable that the rising detection surface Pr and the standing detection surface Ps are variable.

  For example, the height when the subject 40 with a small physique (height) rises is different from the height when the subject 40 with a large physique rises. Therefore, if the rising detection surface Pr is set to match the subject 40 with a small physique, the subject 40 with a large physique is erroneously detected as a rising state due to a change in the height of the torso when turning over. there's a possibility that. Conversely, if the rising detection surface Pr is set so as to match the subject 40 with a large physique, there is a possibility that even if the subject 40 with a small physique gets up, it will not be detected.

  In order to prevent such erroneous detection and detection failure, the person recognition unit 22 of the present embodiment has a function of adjusting the rising detection surface Pr and the standing detection surface Ps. That is, when the use of the system is started, the person recognition unit 22 prompts the input of at least the height among the indices indicating the physique of the subject 40, and the rising detection surface Pr is determined based on the inputted physique (height). The standing detection surface Ps is automatically set. Since height correlates with waist circumference and sitting height, height is used as an indicator of physique. However, when improving the accuracy of evaluation related to the physique, at least one of the waist circumference and the sitting height may be used as an index representing the physique together with the height.

  The index representing the physique need not be a strict value, and the height may be in increments of 10 cm, for example. Therefore, the height given to the person recognition unit 22 may be a value such as 130 cm, 140 cm, and so on. As described above, the person recognition unit 22 evaluates the size of the target person 40 and the state of the target person 40 by evaluating the area where the target person 40 rises and crosses the detection surface Pr and the standing detection surface Ps. Therefore, by adjusting the threshold value for evaluating the obtained area according to the physique, it is possible to accurately evaluate information obtained from the area.

  That is, by setting a threshold according to the physique of the subject 40, it is possible to prevent an erroneous state from being detected for the subject 40 having a different physique. For example, in order not to falsely detect a state in which the subject 40 swings his arm up toward the ceiling in the bedded state as a rising state, the threshold for the cross-sectional area of the arm 40 of the subject 40 having a small physique and the subject 40 having a large physique It is preferable to individually set a threshold for the cross-sectional area of each arm. By setting the threshold for the cross-sectional area of the arm according to the physique, the detection accuracy of the state is improved, the false alarm and the false alarm are eliminated, and the rising or standing state of the subject 40 is accurately detected. .

  As described above, by evaluating the area where the subject 40 crosses the rising detection surface Pr and the standing detection surface Ps, the state of the subject 40 is determined only by whether or not the rising detection surface Pr and the standing detection surface Ps are crossed. It is possible to increase the accuracy of the determination result than when doing so. That is, since the state of the object (target person 40) is determined using two types of methods, the state of the object can be accurately detected.

  Note that the person recognition unit 22 uses, for example, the relationship indicated by the symbol A in FIG. 29A for height and sitting height, and for example, for the relationship between height and waist or the area described above, for example, FIG. ) Is represented by the symbol B. The relationship represented by the symbols A and B uses typical values that do not consider individual differences. In FIG. 29, the physique is represented by a height, and the physique is divided by dividing the height into a plurality of divisions (in the illustrated example, four divisions). That is, in FIG. 29A, as indicated by reference numerals Th11 to Th14, the height of the rising detection surface Pr is defined for each section. In addition, in FIG. 29B, as indicated by reference numerals Th21 to Th24, a threshold for the height or area of the standing detection surface Ps is defined for each section. Therefore, by giving a height as a physique to the human recognition unit 22 and using a relationship as shown in FIGS. 29 (a) and 29 (b), the rising detection surface Pr, the standing detection surface Ps according to the physique, A threshold for the area is automatically determined.

  In addition, since the threshold for the area to be cut off by the rising detection surface Pr and the standing detection surface Ps is set according to the physique, the rising or rising is not required without using the area ratio E2 / E1 described with reference to FIGS. Can be accurately detected.

  By the way, not the state (posture) of a person within the range of the bed 30 but the state of a person outside the range of the bed 30 can be detected using the height of the floor as a reference. That is, as shown in FIG. 30, when the fall threshold value Thf is determined based on the height from the floor 52 and a pixel outside the range of the bed 30 in the Z image has a Z value larger than the fall threshold value Thf, It can be determined that at least a part of the person 40 is out of the range of the bed 30. The fall threshold value Thf is set to a value higher than the floor 52 but lower than the bed surface Pb. For example, the floor 52 may be set to 10 cm.

  In the above-described processing, it is determined only by the fall threshold Thf that the person is out of the range of the bed 30, but the target person 40 is out of the range of the bed 30 by using the difference image outside the range of the bed 30. May be detected. For this purpose, a Z image obtained from a distance image taken at the start of use of the system is stored as a background image, and a difference image obtained by subtracting the Z image obtained thereafter from the background image is binarized.

  At the start of use of the system, the subject 40 is in a bedded state and therefore does not exist outside the range of the bed 30. Therefore, no object exists outside the range of the bed 30 in the background image. On the other hand, in the image obtained by subtracting the Z image in a state where the subject 40 is out of the range of the bed 30 from the background image, an object is present outside the range of the bed 30. Therefore, in the image obtained by binarizing the difference image, the area (number of pixels) of the region changed in the region outside the range of the bed 30 is compared with a predetermined threshold value. It can be determined that it has gone out of the range. Since the value outside the range of the bed 30 in the difference image substantially corresponds to the height from the floor 52, the fall threshold value Thf may be used as the binarization threshold value.

  In addition, when instructing the start of use of the system, it is considered that the nurse or caregiver instructing the start of use exists in an area near the headboard. However, this area is outside the area for detecting the target person 40, and when the difference image is binarized, it is removed together with the floor 52, so that no problem occurs even if it is included in the background image.

  As described above, the position of the bed 30 is recognized by the bed recognition unit 21, and the posture of the subject 40 with respect to the bed 30 is detected by the person recognition unit 22. That is, since the bed 30 and the subject person 40 are separated, the behavior determination unit 23 classifies the action of the subject person 40.

  As an operation to be classified, there is a state in which the subject 40 is sitting on the end of the bed 30 (hereinafter referred to as “sitting”). “Sitting” is detected as a state in which the target person 40 is present straddling the areas inside and outside the range of the bed 30 on any of the four sides of the bed 30. The bed recognizing unit 21 can separate the areas inside and outside the range of the bed 30, and the person recognizing unit 23 can detect the presence of a person in the areas inside and outside the range of the bed 30. it can.

  Using this, when the person's existence area is continuous or close to the inside of the bed 30 and outside the range, it is determined that the same person “crosses” the end of the bed 30. . Then, this case is determined as “sitting”. Here, “proximity” means that the distance between the contour lines of the person's existence area in and out of the range of the bed 30 is obtained, and the obtained distance is within the specified range. The distance between the contour lines may be an average value obtained on a plurality of straight lines in a direction orthogonal to the side of the bed 30 that is focused on.

  Further, the ratio of the area of the person who occupies outside the range of the bed 30 to the whole area of the person who occupies the area within and outside the range of the bed 30 is obtained. An appropriate reference range may be set for this ratio, and if the ratio is within the reference range, it may be determined as “sitting”. In this case, it is possible to adjust the reference range with respect to the ratio of the area so as to increase the determination accuracy of “sitting” according to the body shape of the subject 40 and the like.

  Furthermore, it is also possible to determine “sitting” by combining both determinations described above. That is, if the area ratio is determined in the crossed state, the possibility that the person detected within and outside the range of the bed 30 is the target person 40 increases, and the reliability of the determination is improved. .

  By the way, the bed 30 may be provided with a fall prevention fence. In particular, it is necessary to provide a fence for the subject 40 who is likely to fall from the bed 30. Even in the case where such a fence is provided, an entrance without a fence is provided in at least one place of the bed 30 in order to enter and exit the bed 30.

  When the fence is provided in this way, the position of the entrance / exit in the Z image is registered in advance by an input device (not shown). Since the attachment position of the fence with respect to the bed 30 is often selected from a plurality of places (for example, six places), the input device may be configured to select the attachment position of the fence using a dip switch or the like.

  When the above-mentioned “intersection” is detected in a region other than the entrance / exit registered by the input device, it is determined that the vehicle is over the fence (hereinafter referred to as “over”).

  In the determination of “override”, the area occupied within and outside the range of the bed 30 can be used as in the above-described “stool”. That is, the ratio of the area of the person who occupies outside the range of the bed 30 to the whole area of the person who occupies the area outside and within the range of the bed 30 at a place other than the entrance is obtained. An appropriate reference range is set for this ratio, and if the ratio exceeds the reference range, it is determined that “override”. In this case, if the reference range is set appropriately, it is possible to distinguish between the state where the arm is thrown out of the fence and the “override”.

  In the action determination unit 23, it is also possible to detect an action (hereinafter referred to as “beginning of bed leaving”) when the target person 40 stands up from the bed 30 and leaves the bed 30 as an initial action of “getting out of bed”. is there. In this case, the area of the person occupying the area within and outside the range of the bed 30 is used, and the total area of the person occupying the area within and outside the range of the bed 30 is calculated in the same manner as “stool” and “override” The total area of the person. The ratio of the area of the person outside the range of the bed 30 to the total area of the person is obtained, and when this value is close to 1 (that is, the person is not detected within the range of the bed 30 or is small even if detected) ), It is determined that “the bed is early”. In other words, a reference range close to 1 is set, and when the ratio exceeds the reference range, it is determined as “beginning of bed leaving”.

  By detecting “the early stage of getting out of bed”, it is possible to detect the initial stage of getting out without tracking the subject 40. When the technique for tracking the subject person 40 is used, the tracking may be interrupted for some reason. However, the technique described above makes it possible to detect the “initial stage of bed leaving” regardless of the passage of time.

  By the way, when the target person 40 falls from the bed 30 or when the target person 40 falls from the bed 30 in an attempt to get out of the bed, it is necessary to notify immediately. Hereinafter, these states are collectively referred to as “falling”. The behavior determination unit 23 measures the time during which the state continues when detecting “falling”. That is, when an object having a specified height (for example, 30 cm) or less is detected from the floor 52 only outside the range of the bed 30, the object is determined to be a person. The action determination unit 23 determines “falling” when the state in which the person is detected continues until the specified determination time is reached. The determination time may be set to about 10 seconds, for example.

  The above-mentioned “sitting”, “passing over”, “being out of bed”, and “falling” are determined by paying attention to a specific state regardless of the passage of time. It can be further increased. That is, the behavior determination unit 23 tracks the position of the representative point (for example, the center of gravity) of the area occupied by the target person 40 detected by the person recognition unit 22.

  Here, the person recognition unit 22 uses the detection surface to determine whether the subject 40 is lying on the bed 30, rising up, or standing up within the range of the bed 30, and the presence of the bed 30 outside the range is used as the detection surface. Judgment is made using the corresponding fall threshold value Thf. Therefore, it is possible to obtain the center of gravity within the range of the bed 30 and outside the range of the target person 40 obtained by cutting off each detection surface. However, the area occupied by the target person 40 may be detected three-dimensionally to obtain the center of gravity, and the mode value of the X value and the mode value of the Y value are within and outside the range of the bed 30. It can also be used for the X value and Y value of the representative point to be tracked.

  When a person's position is determined by integrating the person's area within and outside the range of the bed 30, if the movement distance of the representative point within the specified time is compared with an appropriate threshold, It can be determined whether or not the position movement is continuously occurring. Accordingly, any position movement without continuity can be eliminated as noise. By tracking the position of the representative point of the target person 40 in this way, it is possible to eliminate noise that is not the movement of the target person 40, such as movement of a part of the body of the target person 40 or movement of the bedding.

  Furthermore, since the representative point of the subject 40 starts from within the range of the bed 30, the representative point moves from the state existing within the range of the bed 30 to the end of the bed 30, and further the range of the bed 30. It can be determined that the user is getting out of bed when following a movement route of going out. By adding this condition, it is possible to determine that the movement route starting from outside the range of the bed 30 is based on a person who is not the target person 40 (for example, a nurse, a caregiver, a visitor, etc.). That is, when a movement route starting from outside the range of the bed 30 is being tracked, the behavior determination unit 23 does not determine that the person is getting out of bed.

  However, in the operation in which the subject person 40 returns to the bed 30 after getting out of bed, it is necessary to detect that the subject person 40 has returned to the bed 30. Therefore, after the representative point reaches within the range of the bed 30 on the movement route starting from outside the range of the bed 30, the time during which the state existing within the range of the bed 30 or the “sitting” state continues is within the specified time. It is determined that it has returned to the bed 30 when it has reached. After this determination, the starting point for tracking the movement route of the target person 40 is updated within the range of the bed 30. That is, when the subject person 40 returns to the bed 30, it is possible to automatically return to the initial state.

  By the way, the technology for automatically determining whether or not to be detected as described above may be unnecessary depending on the environment in which the apparatus is used and the target person 40. In preparation for such a case, it is desirable to be able to select whether or not to automatically determine whether or not to detect. In addition, when an object that is not a detection target is tracked, the notification is not automatically made and the notification is not performed at all. If the release is not performed, the other apparatus may be notified of getting out of bed.

  The notification unit 24 performs notification according to the operation of the target person 40 detected by the behavior monitoring processing unit 20. Here, the notification unit 24 has a function of setting how to notify which operation of the subject person 40 according to the subject person 40. For example, if the target person 40 is a patient or an elderly person, it may be necessary to notify a nurse or caregiver even if the subject 40 is in a standing state or sitting.

  In the configuration of the present embodiment, it is possible to discriminate various operations of the subject person 40 simply by imaging the subject person 40 using the distance image sensor 10. In addition, it is possible to arbitrarily set which operation is notified. Which operation is to be notified may be selected by providing the notification unit 24 with a dip switch or the like.

  The notification unit 24 can be arranged in the room provided with the bed 30, but it is desirable to notify the nurse center. For example, if the notification unit 24 is provided with a function for generating a contact output that is branched off from the nurse call button, even a patient who does not or cannot press the nurse call button can be notified of getting out of bed.

  The notification unit 24 may be operated in association with an operation device such as a nurse call button. That is, the notification unit 24 may accept an input from an operating device operated by a person in addition to the determination result of the behavior determination unit 23. In this case, the presence / absence of notification of the determination result by the behavior determination unit 23 is switched depending on the combination of the determination result by the behavior determination unit 23 and the input by the operation device.

  For example, the subject 40 who is required to be in a stationary state such as absolute rest is logically ORed when the behavior determining unit 23 detects that the subject 40 has moved and when the nurse call button is pressed. The operation that the notification unit 24 notifies the nurse center may be selected. In addition, for the target person 40 who is allowed to move, an operation for setting the temporary notification state without notifying the nurse center when the action determination unit 23 detects that the target person 40 has moved may be selected. . In this case, when the nurse call button is pressed within a predetermined time from the temporary notification state, the notification unit 24 releases the temporary notification state and returns to the original state, and the nurse call button is pressed within the predetermined time from the temporary notification state. If not, an operation to notify the nurse center is performed.

  As described above, in the notification unit 24, not only the determination result based on the distance image but also the input from the operation device such as the nurse call button can be used to perform desired notification according to the operation of the target person 40. become. Here, the operation device is not limited to the nurse call button, and may be a touch panel of a bedside terminal or the like as will be described later.

  In addition, if a communication function by a nurse call communication network or a local area network (LAN) laid in the building is provided in the notification unit 24 and the determination result of the behavior monitoring processing unit 20 is notified to the nurse center, the operation of the target person 40 can be performed. The contents can be remotely monitored. In this case, all the various operation states of the target person 40 may be output as state signals and masked so that only a desired operation is notified by a notification receiving device (such as a nurse call parent device). In addition, by superimposing the operation status information of the target person 40 on a private PHS exchange linked to the nurse call master unit or a wireless transmitter for information terminals, nurses and caregivers can use the PHS or pager to reach the nurse center. Even if he / she is not present, it is possible to grasp the behavior of the target person 40 and take a quick response.

  In the above-described embodiment, it is assumed that a bed-surrounding curtain exists around the bed 30, but the above-described function of the behavior monitoring processing unit 20 can be used regardless of the presence or absence of the bed-surrounding curtain. It is. That is, it is possible to adopt a technique for determining the position of the bed 30 regardless of the presence / absence of the curtain around the bed, and the determination condition (high detection surface height) according to the physique irrespective of the presence / absence of the curtain around the bed. It is possible to automatically set a threshold for the sheath area.

DESCRIPTION OF SYMBOLS 10 Distance image sensor 20 Action monitoring process part 21 Bed recognition part 30 Bed 37 Foot board 40 Target person (object)
41 Object 45 Observation plate 60 Distance fluctuation correction unit P0, P1, P2 marker P10, P11, P12 marker

Claims (7)

A distance image sensor arranged to generate a distance image in which a pixel value is a distance value to an object using a time difference between light projection and reception and to include a bed to be monitored in a visual field region;
A behavior monitoring processing unit that monitors a specific behavior of a person related to the bed, comprising a bed recognition unit that recognizes the position of the bed using the distance image;
The variation rate of the distance value regarding the pixel at the same position of the background distance image generated in advance by the distance image sensor and the distance image generated by the distance image sensor is calculated, and the variation rate is calculated from the pixels within the reference range. A monitoring apparatus, comprising: a distance fluctuation correction unit that provides a distance image obtained by correcting a distance value using a representative value of the obtained fluctuation rate to the behavior monitoring processing unit. The bed comprises an observation board arranged above the footboard,
The monitoring apparatus according to claim 1, wherein the distance variation correction unit calculates the variation rate using pixels in a region corresponding to the observation plate. The monitoring apparatus according to claim 1, wherein the distance variation correction unit calculates the variation rate using a pixel within the range of the bed extracted by the bed recognition unit. The bed includes a plurality of markers arranged at a center position in the left-right direction on the footboard and other positions on the footboard,
The monitoring apparatus according to any one of claims 1 to 3, wherein the bed recognition unit calculates the position of the bed using the position of the marker as a reference. The bed includes a center position in the left-right direction on the footboard and three or more markers arranged on both side edges of the bed,
The monitoring apparatus according to any one of claims 1 to 3, wherein the bed recognition unit calculates the position of the bed using the position of the marker as a reference. The behavior monitoring processing unit automatically sets a threshold value related to a height defined above the bed in order to monitor the specific behavior using an index indicating a physique given to a person using the bed. The monitoring device according to any one of claims 1 to 5, wherein Computer
Using the distance image obtained from a distance image sensor that generates a distance image whose pixel value is a distance value to an object using a time difference between light projection and light reception and includes a bed to be monitored in a visual field region An action monitoring processing unit for monitoring a specific action of a person related to the bed, comprising a bed recognition unit for extracting the position of the bed;
The variation rate of the distance value regarding the pixel at the same position of the background distance image generated in advance by the distance image sensor and the distance image generated by the distance image sensor is calculated, and the variation rate is calculated from the pixels within the reference range. A program for causing a distance image obtained by correcting a distance value using a representative value of the obtained variation rate to function as a distance variation correction unit that provides the behavior monitoring processing unit.

Images