JP2009033564A - Display device and display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device and a display program for enabling correct transmission of the state when a monitoring object person has fallen and the state after the monitoring object person has fallen to a monitoring person. <P>SOLUTION: A monitoring person terminal 200 includes a display control unit 230 for displaying imaged data within a display region provided to a display 240 and a communication unit 210 for obtaining message information indicating that fall of the monitoring object person has been detected. The display control unit 230 displays an image when a monitoring object person has fallen within the display region and also displays an image after fall within the display region. A first frame interval that is the interval of frame extracted as the image when the fall occurs from a plurality of the frames forming the imaged data is shorter than a second frame interval that is the interval of frame extracted as the image after the fall from the plurality of frames forming the imaged data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a display device and a display program that provide a captured image including a monitoring target person to the monitoring person.

  2. Description of the Related Art Conventionally, there are known life monitoring systems that watch over the actions of monitoring subjects such as elderly people to be watched over, and fall detection systems that detect falls of the monitoring subjects.

  In these systems, the person to be monitored is imaged by a camera or the like, and then the captured image captured by the camera or the like is transmitted to a display device (such as a TV monitor or PC monitor) on the supervisor side who watches over the elderly or the like. Is done. The display device on the monitor side has a display area in which the captured image is displayed, and the monitor grasps the operation (falling) of the monitoring target person by the captured image.

  As an image display method applied to these systems, an image before the time when some abnormality occurs (past image) and an image at the current time (current image, that is, a real-time image) are simultaneously displayed in the display area. A method has been proposed (for example, Patent Document 1).

As a result, the supervisor can compare the past image with the current image, and can grasp the situation in which an abnormality has occurred.
Japanese Unexamined Patent Application Publication No. 2004-266592 (for example, claim 1, [0025], [0054] to [0057], etc.)

  Here, as in the background art described above, there may be a case where it is not possible to accurately grasp the state where some abnormality has occurred by simply displaying the past image and the current image simultaneously in the display area.

  For example, consider a fall detection system that detects the fall of the person being monitored. According to the background art described above, the monitoring person can grasp the situation of the monitoring target person at the time of the fall to some extent from the past image. On the other hand, the monitoring person can grasp the monitoring target person's situation at the current time to some extent from the current image.

  However, since the display device is currently only displaying an image, when the time elapses from the time of falling, the situation of the monitoring target person within a predetermined time after the falling (for example, how the monitoring target person woke up, It is not possible to grasp from the current image (how much time interval there is no movement, etc.) while the monitoring subject falls down.

  Therefore, the present invention has been made to solve the above-described problems, and enables the situation of the monitoring target person to fall and the situation after the monitoring person to fall can be appropriately transmitted to the monitoring person. An object is to provide a display device and a display program.

  In one aspect of the present invention, the display device (monitorer-side terminal 200) that acquires a captured image including a monitoring target person and provides the captured image to the monitoring person detects a fall of the monitoring target person. An acquisition unit (communication unit 210) that acquires notification information indicating that the image is falling, and an image at the time of the fall including at least the captured image before the detection timing that is a timing at which the monitoring subject is detected to fall. And a display control unit (display control unit 230) that displays in the display area an image after the fall including at least the captured image after the detection timing, and a plurality of frames constituting the captured image A fall-time image extraction unit (display control unit 230) that extracts a fall-time image, and extracts the post-fall image from a plurality of frames constituting the captured image. An after-falling image extraction unit (display control unit 230), and a first frame interval that is a frame interval of the fall-down image extracted by the over-falling image extraction unit is extracted by the after-falling image extraction unit. It is shorter than the second frame interval which is the frame interval of the image after the fall.

  In this feature, the display control unit displays the image after the fall in addition to the image at the fall. Therefore, even if time has elapsed since the time of the fall, the monitor can grasp the situation after the fall of the monitoring target person from the image after the fall.

  Here, in general, it is assumed that the movement of the monitoring target person at the time of the fall is fast and the movement of the monitoring target person after the fall is slow.

  In this feature, since the first frame interval of the image at the time of the fall is shorter than the second frame interval of the image after the fall, the situation at the time of the fall of the monitoring target person and the situation after the fall of the monitoring target person are appropriately set to the supervisor. Can communicate.

  In the above-described feature of the present invention, it is preferable that the fall-time image is the captured image from a first time before the detection timing to a second time after the detection timing.

  In this feature, the range of the image at the time of the fall is from the first time before the detection timing of the fall of the monitoring subject to the second time after the detection timing of the fall of the monitoring subject. Therefore, the supervisor can fully grasp the situation at the time of the fall of the monitoring subject by the image at the time of the fall.

  In the above-described feature of the present invention, the post-fall image is the captured image from a third time after the detection timing to a recovery timing that is a timing when recovery from the monitoring subject's fall is detected. Is preferred.

  In this feature, the range of the after-fall image is from the third time after the detection timing of the monitoring subject's fall to the recovery timing at which recovery from the fall of the monitoring subject is detected. Therefore, the supervisor can fully grasp the situation after the fall of the monitoring subject by the image after the fall.

  In the above-described feature of the present invention, the display area includes a first display area in which the image at the time of the fall is displayed, and a second display area in which the image after the fall is displayed. The single frame constituting the image at the time of the fall is configured to be displayed, or the second display area is configured to display the single frame constituting the image after the fall, and In the first display area, the display control unit switches and displays a single frame constituting the fall-time image in chronological order, or a single frame constituting the post-fall image in the second display area. Are preferably displayed in a time-series order.

  In this feature, a single frame composing the fall-time image is displayed in the first display area by switching in time series, or a single frame composing the image after the fall is switched to the second display area in time-series order Displayed. Therefore, the display area that can be assigned to one frame can be increased, and the monitor can easily see the image at the time of the fall or the image after the fall.

  In the above-described feature of the present invention, the display area includes a first display area in which the image at the time of the fall is displayed, and a second display area in which the image after the fall is displayed. It is configured to display a plurality of frames constituting the image at the time of the fall, or the second display area is configured to display a plurality of frames constituting the image after the fall, The display control unit simultaneously displays a plurality of frames constituting the fall-time image in the first display area in chronological order, or displays a plurality of frames constituting the post-fall image in the second display area. It is preferable to display them in chronological order at the same time.

  In this feature, a plurality of frames constituting the fall-time image are simultaneously displayed in the first display area in chronological order, or a plurality of frames constituting the image after the fall are simultaneously displayed in the second display area in chronological order. Is done. Therefore, the supervisor can simultaneously compare the situation when the monitoring subject falls and the situation after the supervisor falls.

  In the above-described feature of the present invention, it is preferable that the display control unit displays an optical flow of frames constituting the fall-time image in the display area.

  In this feature, the optical flow is displayed in the display area for the fall-down image. Accordingly, the monitor can easily grasp the operation speed of the monitoring target person. The optical flow is more useful when applied to a fall-down image that is assumed to have a large movement of the monitoring subject.

  In the above-described feature of the present invention, the display control unit is configured to extract a motion region extracted from the target frame based on difference information between the target frame constituting the post-falling image and a reference frame constituting the post-falling image. Is preferably displayed in the display area.

  In this feature, the motion area extracted from the target frame is displayed in the display area for the after-falling image. Therefore, the monitoring person can easily grasp whether or not the monitoring target person moves after the fall. It should be noted that the motion region extracted from the target frame is more useful when applied to an after-falling image in which the motion of the monitoring target person is assumed to be small.

  In the above-described feature of the present invention, the display control unit displays the captured image at the current time in the display area in addition to the image at the time of the fall and the image after the fall, and according to the acquisition of the notification information The alarm information for notifying the fall of the person to be monitored is displayed in the display area, and depending on the optical flow of the frames constituting the image at the time of the fall or the difference information between the frames constituting the image after the fall It is preferable to change the display method of the alarm information.

  In this feature, a captured image (real-time image) at the current time is displayed in the display area. Therefore, the supervisor can grasp the situation of the person to be monitored in more detail.

  In this feature, the alarm information display method is changed according to the optical flow or the difference information between frames. Therefore, the supervisor can easily grasp the urgency level.

  One feature of the present invention is that a display program for acquiring a captured image including a monitoring target person and providing the captured image to the monitoring person acquires notification information indicating that the monitoring target person has fallen. Displaying a fall-time image including at least the captured image before the detection timing, which is a timing when the monitoring subject's fall is detected, in a display area, and at least the captured image after the detection timing. A step B for displaying the image after the fall within the display area, a step C for extracting the image at the time of the fall from the plurality of frames constituting the captured image, and the after the fall from the plurality of frames constituting the captured image Step D for extracting an image is executed by a computer, and the frame interval of the fallen image extracted in step C is calculated. The first frame interval, shorter than the second frame interval is an interval of the frame of the fall after the image extracted in the step D.

  ADVANTAGE OF THE INVENTION According to this invention, the display apparatus and display program which enable it to transmit appropriately the situation at the time of the fall of a monitoring subject and the situation after the fall of a monitoring subject to a supervisor can be provided.

  Hereinafter, a fall detection system according to an embodiment of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

  However, it should be noted that the drawings are schematic and ratios of dimensions and the like are different from actual ones. Therefore, specific dimensions and the like should be determined in consideration of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings.

[First Embodiment]
(Configuration of fall detection system)
Hereinafter, the configuration of the fall detection system according to the first embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a fall detection system according to the first embodiment.

  As shown in FIG. 1, the fall detection system includes one or a plurality of monitoring subject side systems 100 and one or a plurality of monitoring side terminals 200. The monitoring subject side system 100 and the supervisor side terminal 200 are connected by a wired line or a wireless line.

  The monitoring subject side system 100 is provided in a room of a monitoring subject such as an elderly person to be watched, and acquires a captured image including the monitoring subject. Specifically, the monitoring subject side system 100 includes an imaging unit 110, an input data processing unit 120, a determination criterion storage unit 130, a determination unit 140, an output data processing unit 150, and a communication unit 160. .

  The imaging unit 110 acquires a captured image including the monitoring target person by imaging the monitoring target person. Note that the imaging unit 110 may be configured by a single camera or a plurality of cameras.

  The input data processing unit 120 acquires a captured image from the imaging unit 110. Subsequently, the input data processing unit 120 performs processing necessary for detecting the fall of the monitoring subject on the captured image.

  For example, as illustrated in FIG. 2, the input data processing unit 120 extracts the monitoring target person included in the captured image, and the width (Lx), height (Ly), and depth (Lz) of the rectangular parallelepiped surrounding the monitoring target person. ) To get. Subsequently, the input data processing unit 120 acquires the ratio between the width and the height (Lx / Ly) and the ratio between the depth and the height (Lz / Ly).

  The determination criterion storage unit 130 stores the posture of the monitoring subject (standing position, supine position, sitting position, etc.), the motion model of the monitoring target person (forward tilting, backward tilting, sitting, lying down, etc.). is doing. Further, the determination criterion storage unit 130 stores a determination criterion for determining the posture of the monitoring target person.

  For example, the determination criterion storage unit 130 stores an Lx / Ly threshold (0.4) and an Lz / Ly threshold (0.4) for determining standing. The determination reference storage unit 130 stores an Lx / Ly threshold (1.5) and an Lz / Ly threshold (1.5) for determining the supine position. The determination criterion storage unit 130 stores an Lx / Ly threshold (0.7) and an Lz / Ly threshold (0.7) for determining an operation detection start timing and an operation detection end timing. When the criterion storage unit 130 is configured by the width (Lx), the height (Ly), and the depth (Lz) of the rectangular parallelepiped surrounding the monitoring subject from the motion detection start timing to the motion detection end timing. A motion model (such as a hidden Markov model) is stored that associates series data (hereinafter referred to as time series motion data) with the motion of the person being monitored.

  Based on the information acquired from the input data processing unit 120 and the determination criteria stored in the determination criterion storage unit 130, the determination unit 140 performs the actions of the person being monitored (forward tilting, backward tilting (sirimochi)), and sitting. , Lie down, etc.).

  For example, the determination unit 140 determines the threshold value read from the determination criterion storage unit 130, the width / height ratio (Lx / Ly) and the depth / height ratio (Lz / Ly) acquired from the input data processing unit 120. ) To determine the posture of the person being monitored. Specifically, the posture of the monitoring subject is determined as follows.

(1) Standing position: Lx / Ly <0.4 or Lz / Ly <0.4
(2) Supine ... Lx / Ly> 1.5 or Lz / Ly> 1.5
(3) Sitting position: 0.4 ≦ Lx / Ly ≦ 1.5 and 0.4 ≦ Lz / Ly ≦ 1.5
The determination unit 140 determines to start the motion detection of the monitoring subject when the state of Lx / Ly ≧ 0.7 or Lz / Ly ≧ 0.7 continues for 5 frames or 1 sec. On the other hand, when the state where Lx / Ly <0.7 and Lz / Ly <0.7 continues for 7 frames or 1.4 seconds, the determination unit 140 determines to end the monitoring target person's motion detection. Alternatively, the determination unit 140 determines to end the monitoring of the monitoring target person when 10 to 15 sec (Z time described later) has elapsed since the start of the monitoring target person's movement detection.

  Hereinafter, the fall detection method described above is referred to as a fall detection method using a rectangular parallelepiped.

  The determination unit 140 determines the operation of the monitoring target person based on time-series operation data acquired between the operation detection start timing and the operation detection end timing. Specifically, the determination unit 140 refers to the hidden Markov model, and when the motion model that outputs the highest likelihood with respect to time-series motion data is a fall model, the monitoring target person falls. judge.

  As described above, the time-series motion data includes the width (Lx), the height (Ly), and the depth (Ly) of the rectangular parallelepiped surrounding the person to be monitored between the motion detection start timing and the motion detection end timing. Lz) is time-series data.

  The output data processing unit 150 acquires a captured image from the imaging unit 110 and acquires a determination result from the determination unit 140. Subsequently, the output data processing unit 150 generates information to be transmitted to the supervisor side terminal 200.

  Specifically, the output data processing unit 150 transmits the captured image acquired from the imaging unit 110 to the communication unit 160. When the determination result by the determination unit 140 indicates that the monitoring target person has fallen, the output data processing unit 150 generates notification information indicating that the monitoring target person has fallen.

  Here, the notification information includes the time when the monitoring subject's motion detection is started as the time when the monitoring subject's fall was detected. The notification information may include a time when the monitoring target person's motion detection is completed as a time when the monitoring target person's recovery from falling is detected.

  The communication unit 160 transmits information (such as a captured image and notification information) generated by the output data processing unit 150 to the supervisor side terminal 200.

  Here, it should be noted that various methods related to the fall detection method using a rectangular parallelepiped can be applied as the fall detection method of the monitoring subject (for example, Japanese Patent Application No. 2006-336063). Gazette, Japanese Patent Application No. 2006-342654). In addition to the fall detection method using a rectangular parallelepiped, other existing methods can be applied as the fall detection method for the person being monitored.

  In the following, the description will be continued on the assumption that the fall of the monitoring subject has been detected by any of the methods described above and that the monitoring subject has actually fallen.

  In addition, the fall of the monitoring subject may be detected at any time point from when the monitoring subject falls from the standing position (or sitting position) to the lying position. Therefore, the fall of the monitoring subject can be detected by the fall detection method using a rectangular parallelepiped. In addition, the fall of the monitoring subject may be detected at the time when the movement of the monitoring subject is the largest during the period from the standing position to the lying position. Moreover, the fall of the monitoring subject may be detected at the time when the monitoring subject becomes prone. Furthermore, the fall of the monitoring subject may be detected when at least a part of the monitoring subject touches the floor or the like.

  The supervisor-side terminal 200 acquires a captured image including the monitoring target person from the monitoring target person-side system 100, and provides the captured image to a monitoring person who watches an elderly person or the like. Specifically, the supervisor-side terminal 200 includes a communication unit 210, an image storage unit 220, a display control unit 230, and a display 240.

  The communication unit 210 receives a captured image including the monitoring target person from the monitoring target person side system 100. The communication unit 210 receives notification information indicating that the monitoring target person has fallen from the monitoring target person side system 100.

  The image storage unit 220 stores captured images received from the monitoring subject side system 100 in units of frames.

  The display control unit 230 displays a captured image including the monitoring subject in a display area provided on the display 240. Specifically, the display control unit 230 displays the following three types of images after extracting the following three types of images from a plurality of frames constituting the captured image.

(1) Image at the time of falling: an image including at least a captured image before the timing (hereinafter referred to as detection timing) when the monitoring subject's falling is detected. An image including at least an image (3) Current image (real-time image) ... a captured image at the current time Specifically, an image at the time of a fall and an image after the fall will be described with reference to FIG. As shown in FIG. 3, “when the monitoring subject falls” is a period from X hours (first time) before the detection timing to Y hours (second time) after the detection timing. “After the fall of the monitoring subject” is a period from the Y time (third time) of the detection timing to the timing when the recovery from the fall of the monitoring subject is detected (hereinafter, recovery timing).

  Here, the X time is determined such that, for example, a state where the standing position of the monitoring subject starts to collapse due to a stumbling or a fall is included in “when the monitoring subject falls”.

  The Y time is determined so that the state where the whole body of the monitoring subject is lying on the floor is included in “when the monitoring subject falls”.

  The Z time is the time from the detection timing until the operation detection time times out. In addition, the Z time is determined so that sufficient time can be obtained to start the operation determination of the monitoring subject and notify the monitoring person of the state after the monitoring subject falls.

  Note that the timing at which the monitoring target person's motion detection is started is a detection timing, and the timing at which the monitoring target person's motion detection is completed is a recovery timing.

  As shown in FIG. 3, the overturned image is a captured image from X time (first time) before the detection timing to Y time (second time) after the detection timing. The display control unit 230 displays the overturned image at the first frame interval. The first frame interval is an interval between frames extracted as a fall-down image from a plurality of frames constituting a captured image.

  On the other hand, the after-fall image is a captured image from the time Y after detection time (third time) to the recovery timing. The display control unit 230 displays the after-falling image at the second frame interval. Note that the second frame interval is an interval between frames extracted as a post-falling image from a plurality of frames constituting the captured image.

  Here, it is assumed that the image at the time of falling includes an action in which the monitoring subject falls according to the direction of gravity. That is, since it is assumed that the movement of the monitoring subject is large in unit time, it is preferable that the frame interval (first frame interval) of the image at the time of the fall is short.

  On the other hand, it is assumed that the image after the fall includes an operation in which the monitoring subject stands up against the direction of gravity or an operation in which the monitoring subject is stationary. In addition, it is assumed that the operation is slow because the monitoring target is an elderly person, and that the monitoring target is struck by the body and damaged, and the monitoring target is performing slowly. That is, since it is assumed that the movement of the monitoring subject in the unit time is small, it is preferable that the frame interval (second frame interval) of the image after the fall is long.

  Accordingly, the frame interval (first frame interval) of the image at the time of the fall is set shorter than the frame interval (second frame interval) of the image after the fall.

  Here, the display control unit 230 specifies the detection timing as follows. For example, the display control unit 230 specifies the time at which the monitoring subject's fall was detected, that is, the operation detection start time included in the notification information as the detection timing.

  On the other hand, the display control unit 230 specifies the recovery timing as follows. For example, the display control unit 230 specifies the time when the recovery from the fall of the monitoring subject is detected, that is, the end time of the operation detection included in the notification information as the recovery timing.

  Note that if the notification information does not include the end time of the motion detection of the monitoring target person (the time when recovery from the fall is detected), the display control unit 230 detects the detection time Z time (> Y time). A later timing may be specified as the recovery timing.

  Further, in response to receiving the notification information, the display control unit 230 displays alarm information (for example, a character string of “falling”) indicating that the monitoring subject has fallen in the display area.

  Note that when there are a plurality of monitoring target persons, the alarm information may include information for identifying the monitoring target persons (for example, the name of the monitoring target person, the room number of the monitoring target person, etc.). The alarm information may include the date and time when the monitoring subject's fall was detected.

  Here, the above-described X time is preferably in the range of 0.5 to 2 seconds, for example. The Y time described above is preferably in the range of 0.5 to 2 seconds, for example. The Z time described above is preferably in the range of 10 to 15 seconds, for example.

  Further, in the case where the frame interval of the current image (real-time image) is 0.033 sec (30 frames / sec), the frame interval (first frame interval) of the image at the time of falling is, for example, 0.033-0. It is preferably within a range of 5 seconds. On the other hand, the frame interval (second frame interval) of the after-falling image is preferably in the range of 0.5 to 2 seconds, for example.

  The display 240 has a display area where captured images and the like are displayed. Specifically, as shown in FIG. 4, the display area displays a fall-time image display area in which a fall-time image is displayed, an after-fall image display area in which an after-fall image is displayed, and a current image (real-time image). A real-time image display area, and an alarm information display area where alarm information is displayed.

  Here, the fall-time image display area is configured to display a plurality of frames. The display control unit 230 simultaneously displays a plurality of frames constituting the fall-time image in chronological order in the fall-time image display area.

  The after-falling image display area is configured to display a plurality of frames. The display control unit 230 simultaneously displays a plurality of frames constituting the post-fall image in chronological order in the post-fall image display area.

  The real-time image display area is configured to display a single frame. The display control unit 230 switches and displays a single frame constituting the current image in chronological order in the real-time image display area.

  Note that the supervisor can obtain the following information from the fall-time image displayed in the fall-time image display area.

(1) Things that the monitoring subject stumbles and the place where the trip or collapse occurred (2) Time (speed) from when the monitoring subject starts to stumble or fall into the decubitus
(3) The part of the body that was struck against an object, etc., placed in the living room after the person to be monitored stumbled or started to fall down and became supine (4) When the person to be monitored became supine The body part struck against the floor, etc. (5) The posture when the person to be monitored is placed in the prone position. Also, the observer uses the post-fall image displayed in the post-fall image display area to provide the following information: It is possible to obtain

(1) Presence or absence of movement of monitored person (2) Whether monitored person is going to stand up (3) Whether monitored person is bleeding (operation of fall detection system)
Hereinafter, the operation of the fall detection system according to the first embodiment will be described with reference to the drawings. FIG. 5 is a sequence diagram showing the operation of the fall detection system according to the first embodiment.

  As shown in FIG. 5, in step 10, the monitoring subject side system 100 acquires a captured image including the monitoring subject.

  In step 20, the monitoring subject side system 100 transmits a captured image including the monitoring subject person to the supervisor side terminal 200. Note that the monitoring subject-side system 100 continuously transmits captured images to the monitoring-side terminal 200.

  In step 30, the supervisor-side terminal 200 stores the captured image including the monitoring subject in the image storage unit 220. Subsequently, the supervisor-side terminal 200 displays a current image (real-time image) that is a captured image at the current time in the real-time image display area.

  In step 40, the monitoring subject side system 100 starts detecting the monitoring subject's motion. For example, the monitoring target person side system 100 determines that the monitoring target person side system 100 determines that the monitoring target person side system 100 is in a state where Lx / Ly ≧ 0.7 or Lz / Ly ≧ 0.7 continues for five frames or one second. It is determined that motion detection is started.

  In step 50, the monitoring subject person side system 100 complete | finishes the monitoring subject's operation | movement detection. For example, the monitoring target person side system 100 determines to end the monitoring target person's motion detection when the state of Lx / Ly <0.7 and Lz / Ly <0.7 continues for 7 frames or 1.4 seconds. To do.

  In step 60, the monitoring target person side system 100 performs the monitoring target person's operation based on the time-series operation data acquired from the start of the monitoring target person's motion detection to the end of the monitoring target person's motion detection. Determine. Specifically, the monitoring target person side system 100 refers to the hidden Markov model, and when the motion model that outputs the highest likelihood with respect to time-series motion data is the fall model, Determined to have fallen.

  In the following, the description will be continued assuming that it is determined in step 60 that the monitoring subject has fallen.

  In Step 70, the monitoring target person side system 100 transmits notification information indicating that the monitoring target person has fallen to the monitoring person side terminal 200. Here, the notification information includes the time when the monitoring target person's motion detection started as the time when the monitoring target person's fall was detected, and the time when the monitoring target person's motion detection ended was recovered from the fall of the monitoring target person It is assumed to be included as the time when is detected.

  In step 80, the supervisor-side terminal 200 displays alarm information indicating that the monitoring subject's fall has been detected, and uses the captured image stored in the image storage unit 220 and the fall-time image and the post-fall image. Start displaying.

  Here, the supervisor-side terminal 200 displays the fall-time image from X time (first time) before the detection timing to Y time (second time) after the detection timing. Further, the supervisor-side terminal 200 displays the after-falling image from the time Y after the detection timing (third time) to the recovery timing.

(Operation of display device)
Hereinafter, the operation of the display device according to the first embodiment will be described with reference to the drawings. FIG. 6 is a flowchart showing the operation of the supervisor side terminal 200 (display device) according to the first embodiment.

  As shown in FIG. 6, in step 110, the supervisor-side terminal 200 receives a captured image including the monitoring subject from the monitoring subject-side system 100.

  In step 120, the supervisor side terminal 200 stores the captured image including the monitoring subject in the image storage unit 220.

  In step 130, the supervisor-side terminal 200 starts displaying the current image (real-time image) in the real-time image display area.

  In step 140, the supervisor-side terminal 200 determines whether or not notification information indicating that the monitoring subject has fallen has been received. When the monitoring terminal 200 receives the notification information, the monitoring terminal 200 proceeds to the process of step 150.

  In step 150, the supervisor-side terminal 200 displays alarm information indicating that the monitoring subject has fallen in the alarm information display area.

  In step 160, the supervisor-side terminal 200 displays the fall-time image in the fall-time image display area using the captured image stored in the image storage unit 220. The range of the image at the time of falling is from X hours (first time) before the detection timing to Y hours (second time) after the detection timing.

  In step 170, the supervisor-side terminal 200 starts displaying the after-falling image in the after-falling image display area. The range of the image after the fall is from the time Y after the third detection time to the recovery timing.

(Function and effect)
In the first embodiment, the display control unit 230 displays the image after the fall in addition to the image at the fall. Therefore, even when a predetermined time has elapsed from the time of the fall, the monitor can grasp the situation after the fall of the monitoring target person from the image after the fall.

  In addition, since the first frame interval of the image at the time of the fall is shorter than the second frame interval of the image after the fall, the situation when the monitoring subject falls and the situation after the fall of the monitoring subject are appropriately transmitted to the supervisor. be able to.

  In the first embodiment, the range of the image at the time of falling is from X hours (first time) before the detection timing to Y hours (second time) after the detection timing. Therefore, the supervisor can fully grasp the situation at the time of the fall of the monitoring subject by the image at the time of the fall.

  In the first embodiment, the range of the after-falling image is from Y time (third time) after the detection timing to the recovery timing. Therefore, the supervisor can fully grasp the situation after the fall of the monitoring subject by the image after the fall.

  In the first embodiment, a plurality of frames constituting the fall-time image are simultaneously displayed in the fall-time image display area, and a plurality of frames constituting the after-fall image are simultaneously displayed in the after-fall image display area. Therefore, the supervisor can simultaneously compare the situation when the monitoring subject falls and the situation after the supervisor falls.

  In the first embodiment, the current image (real-time image) at the current time is displayed in the real-time image display area. Therefore, the supervisor can grasp the situation of the person to be monitored in more detail.

[Second Embodiment]
The second embodiment will be described below with reference to the drawings. In the following, differences between the first embodiment and the second embodiment described above will be mainly described.

  Specifically, in the first embodiment described above, the image display area at the time of fall and the image display area after the fall are configured to display a plurality of frames. In contrast, in the second embodiment, the image display area at the time of the fall and the image display area after the fall are configured to display a single frame.

(Configuration of display area)
The configuration of the display area according to the second embodiment will be described below with reference to the drawings. FIG. 7 is a diagram illustrating a configuration of a display area provided in the display 240 according to the second embodiment.

  As shown in FIG. 7, the fall-time image display area is configured to display a single frame. The display control unit 230 switches and displays a single frame constituting the fall-time image in chronological order in the fall-time image display area (frame advance).

  The after-falling image display area is configured to display a single frame. The display control unit 230 switches and displays a single frame constituting the after-falling image in time-series order in the after-falling image display area (frame advance).

  It should be noted that the frame interval (first frame interval) of the image at the time of the fall is set shorter than the frame interval (second frame interval) of the image after the fall, as in the first embodiment. .

(Function and effect)
In the second embodiment, a single frame constituting the fall-time image is switched and displayed in the time-sequential image display area, and a single frame constituting the post-fall image is displayed in the time-sequential order in the post-fall image display area. Switched and displayed. Therefore, the display area that can be assigned to one frame can be increased as compared with the first embodiment, and the monitor can easily view the fall-time image and the after-fall image.

[Third Embodiment]
Hereinafter, a third embodiment will be described with reference to the drawings. In the following, differences between the first embodiment and the third embodiment described above will be mainly described.

  Specifically, although not particularly mentioned in the first embodiment described above, in the third embodiment, the optical flow of the frames constituting the fall-time image is displayed in the fall-time image display area. On the other hand, based on the difference information between the target frame constituting the post-fall image and the reference frame constituting the post-fall image, the motion area extracted from the target frame is displayed in the post-fall image display area.

(Configuration of display device)
The configuration of the display device according to the third embodiment will be described below with reference to the drawings. FIG. 8 is a block diagram showing a configuration of the supervisor-side terminal 200 (display device) according to the third embodiment.

  As shown in FIG. 8, the supervisor-side terminal 200 includes an optical flow calculation unit 250 and an inter-frame difference calculation unit 260 in addition to the configuration shown in FIG.

  The optical flow calculation unit 250 calculates the optical flow of the frames constituting the fall-down image. Examples of the optical flow calculation method include a block matching method and a gradient method.

  The inter-frame difference calculation unit 260 calculates difference information between frames constituting the after-falling image. For example, the inter-frame difference calculation unit 260 acquires the difference between the target frame and the reference frame in units of pixels, and binarizes the difference acquired in units of pixels. Subsequently, the inter-frame difference calculation unit 260 performs the contraction / expansion process on the binarized difference, and calculates difference information (difference area) between the target frame and the reference frame as a motion area.

  As shown in FIG. 9, the display control unit 230 displays the optical flow calculated by the optical flow calculation unit 250 in the image display area at the time of falling. It is preferable that the display control unit 230 displays the optical flow together with the overturned image.

  The display control unit 230 may display only an optical flow whose flow length exceeds a predetermined threshold, or may display only an optical flow having the maximum flow length.

  On the other hand, the display control unit 230 extracts the difference area calculated by the inter-frame difference calculation unit 260 from the target frame constituting the after-falling image. Subsequently, as illustrated in FIG. 9, the display control unit 230 displays a motion area, which is a difference area extracted from the target frame, as a post-fall image in the post-fall image display area.

  Note that the display control unit 230 may display only a motion region whose area exceeds a predetermined threshold, or may display only a motion region whose area is maximum.

(Function and effect)
In the third embodiment, an optical flow is displayed in the fall-time image display area for an fall-time image that is assumed to have a large movement of the monitoring subject in a unit time. Therefore, the monitoring person can easily grasp the falling speed of the monitoring target person.

  In the third embodiment, for a post-fall image that is assumed to have a small movement of the monitoring subject in a unit time, a motion area extracted from the target frame based on difference information between frames is displayed in the post-fall image display area. The Therefore, the monitoring person can easily grasp whether or not the monitoring target person cannot move after the fall.

[Fourth Embodiment]
Hereinafter, a first embodiment will be described with reference to the drawings. In the following, differences between the first embodiment and the fourth embodiment described above will be mainly described.

  Specifically, although not particularly mentioned in the first embodiment described above, in the fourth embodiment, the display control unit 230 causes the optical flow of the frames constituting the image at the time of the fall or between the frames constituting the image after the fall. The alarm information display method is changed according to the difference information (difference area).

  Note that the supervisor-side terminal 200 includes an optical flow calculation unit 250 and an inter-frame difference calculation unit 260 as in the third embodiment.

  Specifically, when the optical flow having the maximum flow length is smaller than a predetermined threshold, the display control unit 230 displays alarm information in a color with a low urgency (for example, blue). On the other hand, when the optical flow having the maximum flow length is larger than the predetermined threshold, the display control unit 230 displays alarm information in a color with a high degree of urgency (for example, red).

  The urgency level is a degree indicating how urgent the action of the monitoring target person is necessary for the monitoring person.

  Here, in the case where the optical flow is large, it is considered that the fall speed of the monitoring subject is large. Therefore, it is considered that the degree of urgency is high when the optical flow is large.

  The display control unit 230 displays the alarm information in a color with a low degree of urgency (for example, blue) when the difference area having the maximum area is larger than a predetermined threshold. On the other hand, the display control unit 230 displays alarm information in a color with a high degree of urgency (for example, red) when the difference area having the maximum area is smaller than a predetermined threshold.

  Here, in a case where the difference area is small, it is considered that there is a high possibility that the monitoring target person cannot move after the fall. Therefore, it is considered that the degree of urgency is high in the case where the difference area is small.

  The display control unit 230 may change the alarm information display method in consideration of both the flow length of the optical flow and the area of the difference area. The alarm information display method is not only the color of the alarm information, but also the size of the characters in the alarm information, the presence or absence of blinking of the alarm information, the blinking interval of the alarm information, the presence or absence of sound corresponding to the alarm information, the zoom of the captured image It may be changed depending on the presence or absence.

(Function and effect)
In the fourth embodiment, the alarm information display method is changed according to optical flow or difference information between frames. Therefore, the supervisor can easily grasp the urgency level.

[Other Embodiments]
Although the present invention has been described with reference to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

  For example, in the above-described embodiment, the range of the image at the time of falling is from X hours before the detection timing to Y hours after the detection timing, but is not limited thereto. The range of the image at the time of falling may be from X hours before the detection timing to the detection timing.

  Similarly, the range of the after-falling image is from the time Y after the detection timing to the recovery timing, but is not limited to this. The range of the image after the fall may be from the detection timing to the recovery timing.

  In the embodiment described above, both the second time and the third time are the Y time, and the range of the image at the time of the fall does not overlap with the range of the image after the fall, but is not limited to this. The range of the image at the time of the fall may overlap with the range of the image after the fall.

  In the embodiment described above, the monitoring target person's fall detection is performed by the monitoring target person side system 100, but is not limited to this. The detection of the fall of the monitoring subject may be performed by the supervisor side terminal 200. In such a case, the monitoring subject side system 100 may simply transmit the captured image to the supervisor side terminal 200, and the supervisor side terminal 200 includes the input data processing unit 120 to the output data processing unit 150. is doing.

  In the above-described embodiment, the optical flow corresponding to the image at the time of falling is displayed in the image display area at the time of falling, but the present invention is not limited to this. An optical flow corresponding to the after-falling image may be displayed in the after-falling image display area.

  Similarly, the difference information between frames corresponding to the after-falling image is displayed in the after-falling image display area, but is not limited thereto. The difference information between the frames corresponding to the fall-time image may be displayed in the fall-time image display area.

  In the above-described embodiment, the current image is displayed in the real-time image display area, but the present invention is not limited to this. From the viewpoint of privacy protection, the current image may not be displayed. Also, a password may be required to view the current image.

  The first embodiment and the second embodiment described above may be combined. Specifically, the fall-time image display area may be configured to display a single frame, and the after-fall image display area may be configured to display a plurality of frames. Conversely, the image display area at the time of the fall may be configured to display a plurality of frames, and the image display area after the fall may be configured to display a single frame.

  Although not particularly mentioned in the above-described embodiment, the operation of the supervisor-side terminal 200 (display device) may be provided as a program (display program) that realizes the flow shown in FIG.

It is a figure which shows the structure of the fall detection system which concerns on 1st Embodiment. It is a figure for demonstrating the attitude | position and motion determination of the monitoring subject which concern on 1st Embodiment. It is a figure for demonstrating the image at the time of a fall and the image after a fall which concerns on 1st Embodiment. It is a figure which shows the structure of the display area provided in the display 240 which concerns on 1st Embodiment. It is a sequence diagram which shows operation | movement of the fall detection system which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the supervisor side terminal 200 (display apparatus) which concerns on 1st Embodiment. It is a figure which shows the structure of the display area provided in the display 240 which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the supervisor side terminal 200 (display apparatus) which concerns on 3rd Embodiment. It is a figure which shows the structure of the display area provided in the display 240 which concerns on 3rd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Monitored person side system, 110 ... Imaging part, 120 ... Input data processing part, 130 ... Determination criteria storage part, 140 ... Determination part, 150 ... Output data processing part , 160 ... communication unit, 200 ... supervisor side terminal, 210 ... communication unit, 220 ... image storage unit, 230 ... display control unit, 240 ... display, 250 ... Optical flow calculation unit, 260 ... inter-frame difference calculation unit

Claims (9)

A display device that obtains a captured image including a monitoring target person and provides the captured image to the monitoring person,
An acquisition unit that acquires notification information indicating that the monitoring subject's fall has been detected;
A fall-time image including at least the captured image before the detection timing, which is a timing at which the monitoring subject's fall is detected, is displayed in a display area, and an after-fall image including at least the captured image after the detection timing is displayed. A display controller for displaying in the display area;
A fall-time image extraction unit that extracts the fall-time image from a plurality of frames constituting the captured image;
An after-falling image extraction unit that extracts the after-falling image from a plurality of frames constituting the captured image;
The first frame interval that is the frame interval of the image at the time of the fall extracted by the image extraction unit at the time of fall is the second frame interval that is the interval of the frames of the image after the fall extracted by the image extraction unit after the fall A display device characterized by being shorter than that.   2. The display device according to claim 1, wherein the overturning image is the captured image from a first time before the detection timing to a second time after the detection timing.   The after-fall image is the captured image from a third time after the detection timing to a recovery timing that is a timing at which recovery from the fall of the monitoring subject is detected. The display device according to claim 2. The display area includes a first display area in which the image at the time of the fall is displayed, and a second display area in which the image after the fall is displayed,
The first display area is configured to display a single frame constituting the image at the time of the fall, or the second display area is configured to display a single frame constituting the image after the fall. Is composed of
In the first display area, the display control unit switches and displays a single frame constituting the fall-time image in chronological order, or a single frame constituting the post-fall image in the second display area. The display device according to any one of claims 1 to 3, wherein the frames are switched and displayed in chronological order. The display area includes a first display area in which the image at the time of the fall is displayed, and a second display area in which the image after the fall is displayed,
The first display area is configured to display a plurality of frames constituting the fall-time image, or the second display area is configured to display a plurality of frames constituting the post-fall image. Is composed of
The display control unit simultaneously displays a plurality of frames constituting the fall-time image in time series in the first display area, or a plurality of frames constituting the post-fall image in the second display area. The display device according to claim 1, wherein the display devices are simultaneously displayed in chronological order.   The display device according to claim 1, wherein the display control unit displays an optical flow of a frame constituting the fall-time image in the display area.   The display control unit displays a motion area extracted from the target frame in the display area based on difference information between a target frame constituting the post-fall image and a reference frame constituting the post-fall image. The display device according to claim 1. The display control unit
In addition to the image at the time of the fall and the image after the fall, the captured image at the current time is displayed in the display area,
In response to the acquisition of the notification information, alarm information for notifying the fall of the monitoring subject is displayed in the display area,
The display method of the alarm information is changed according to an optical flow of a frame constituting the image at the time of the fall or difference information between frames constituting the image after the fall. 8. The display device according to any one of 7. A display program for acquiring a captured image including a monitoring target person and providing the captured image to the monitor, the computer,
Step A for obtaining notification information indicating that the monitoring subject's fall has been detected;
A fall-time image including at least the captured image before the detection timing, which is a timing at which the monitoring subject's fall is detected, is displayed in a display area, and an after-fall image including at least the captured image after the detection timing is displayed. Displaying in the display area B;
Extracting the fall-time image from a plurality of frames constituting the captured image; and
Performing the step D of extracting the post-falling image from a plurality of frames constituting the captured image;
The first frame interval that is the frame interval of the image at the time of the fall extracted in the step C is shorter than the second frame interval that is the interval of the frame of the image after the fall extracted in the step D. Display program.

Images